Re: [jdt-dev] JDT-Core: Report on Gerrit code review

[Enrico Fregnan <fregnan@xxxxxxxxxx>]

Hi Stephan,

First of all, thank you very much for finding the time to send us an email. 

Yes, it is definitely a way easier and practical way to report your extensive feedback. If I am not asking too much, it would be great if you could quickly fill in the survey questions too. That would make much easier for us to process the data for our publication.

> My first guess from looking at your report was: "that looks like a proposal for a new management tool”…

Sort of. The idea of this tool would be to analyse what happens in the code review process looking at which kinds of changes usually occurs and classifying them in the categories mentioned in the report.
I am also envisioning it more as a management tool (that’s the reason behind the two questions “I find this information useful as a developer” or “I find this information useful as a project manager”. However, I was interested in hearing the opinion on it of people with different backgrounds.

This tool analyses the review process retrospectively and should give some insights about were the effort is spent. For example, if many changes are documentation changes maybe having a check style in tool in place or a strict stylistic convention could be useful (of course, this is just an example, not necessarily this is the case in JDT).

> Things would look differently with my industrial hat on. In that context I would be ambivalent about this kind of report:
> On the one hand, perhaps a closer look at the review process would help management to better understand development.

Yes, this was the idea behind this tool and the goal that I was hoping to achieve. 

> On the other hand, I would be afraid that management would take a report and then "decide" that, e.g., changes in visual representation and/or structure need to be reduced in favor of functional changes. If that would happen, then it would make the life of developers harder, rather then better, with questionable outcome.
> If you ask a manager, and he answers, "cool, now I better understand my developers", this would be a good outcome. If he answers "cool, now I can better control my developers" I would be worried.

Good point. Hopefully the kind of insight here is general enough to avoid the second option in favour of the first. I on purpose did not add any information about the single developer to avoid a possible misuse of the tool to “rate” a reviewer. 

> Another question could be: would I as a developer draw benefit from a report of *just my own review activity*? Would it tell me s.t. I'd like to improve but didn't realize? I'd tend to say I already know all that, but I *might* be wrong :)

This could be possible. As I mentioned before, I did not include data about the developers to avoid a possible misuse, but of course the situation could be different if you and only you get a report about your own activity.

Again, thanks a lot for your feedback. It was extremely appreciated.

Feel free to ask any further questions or keep the discussion going if you want. I would love to hear more of your opinions on what we are trying to do with this tool.

Best,
  Enrico


> On 18 Aug 2019, at 14:26, Stephan Herrmann <stephan.herrmann@xxxxxxxxx> wrote:
> 
> Hi Enrico,
> 
> I feel I can give you a more substantial answer in free prose rather than making check marks in a questionnaire. While it's just my personal view, sharing this via the mailing list might also help to find out if the team agrees in these respects.
> 
> Background: My professional live spans years of research, "industrial" software development & open source development (in my spare time).
> The perspective of people doing JDT development as their Job may slightly differ from my perspective, but I don't think the difference is huge.
> 
> My first guess from looking at your report was: "that looks like a proposal for a new management tool". To which my immediate reaction is: "In JDT we hardly need any management tool (on top of bugzilla & gerrit), because we have very little management". We have team leads who are primarily developers themselves, who just additionally coordinate a bit more and occasionally need to give the essential +2 or -2 in case of doubt. On top of that Dani and the PMC do IMHO a good job in not getting in our way, not making our job harder than just to find solutions to known problems (this is meant in an entirely positive sense). Perhaps Dani would be the only person using a management tool, but I feel he already knows everything he needs to now about the development, as he is involved directly in many bug discussions already.
> 
> I don't want to imply, reports are generally not needed, but we developers rely more on technical reports that indicate when some aspect of quality is going south, when technical debt is being created etc.
> 
> Things would look differently with my industrial hat on. In that context I would be ambivalent about this kind of report:
> On the one hand, perhaps a closer look at the review process would help management to better understand development.
> On the other hand, I would be afraid that management would take a report and then "decide" that, e.g., changes in visual representation and/or structure need to be reduced in favor of functional changes. If that would happen, then it would make the life of developers harder, rather then better, with questionable outcome.
> If you ask a manager, and he answers, "cool, now I better understand my developers", this would be a good outcome. If he answers "cool, now I can better control my developers" I would be worried.
> 
> I think one of the hardest parts during reviewing is the human aspect of it: in many cases in JDT a review is the first contact between a contributor and a reviewer (committer), and we still have to establish a mode of operation between us.
> 
> Another question could be: would I as a developer draw benefit from a report of *just my own review activity*? Would it tell me s.t. I'd like to improve but didn't realize? I'd tend to say I already know all that, but I *might* be wrong :)
> 
> best,
> Stephan
> 
> On 17.08.19 18:13, Enrico Fregnan wrote:
>> Dear JDT-Core developers,
>> I am Enrico, a researcher at the University of Zurich and I am studying how to evaluate the code review process to understand how it could be improved.
>> I created a tool that analyzes the changes that happen during a review on Gerrit, to measure the effects of code review on code and understand which kinds changes reviewers pay attention to.
>> For example, are you aware that many of the changes happened during a code review in the last year in your project involve documentation issues?
>> You will find this and more data about your project at the following link:
>> https://www.surveygizmo.com/s3/5169941/Code-review-changes-report-JDT-Core
>> The link contains the report and some questions we would like to ask you about it. The report is designed in a way that you don't have to answer the questions (if you just want to look at the report), but it would be great if we could know what you think about our tool/report. Answering will take you no more than 5 minutes.
>> If you have any question about the report or our research, please don't hesitate to contact me.
>> Thank you very much for your time.
>> Cheers,
>> Enrico
>> _______________________________________________
>> jdt-dev mailing list
>> jdt-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://www.eclipse.org/mailman/listinfo/jdt-dev
>