[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
[technology-pmc] RE: EclipseCon program selection
|
Hi Richard, dear Technology PMC,
Re-reading my email I have come to the conclusion that it is in some
parts unfair and over generalizing. I am sorry about that.
I am aware that the program committee members are putting a lot of
effort and good will into creating an exiting EclipseCon, and I have
really enjoyed all EclipseCons to date - no doubt that 2007 will be
another success.
There are still a couple of items in my post that I think remain valid
points, and I comment below:
> -----Original Message-----
> From: Richard Gronback [mailto:richard.gronback@xxxxxxxxxxx]
> Sent: Thursday, December 21, 2006 1:20 PM
> To: Krause, Jochen; technology-pmc@xxxxxxxxxxx
> Subject: Re: EclipseCon program selection
>
> Hi Jochen,
>
> Comments inserted throughout...
>
> Best Regards,
> Rich
>
>
> On 12/20/06 4:29 PM, "Krause, Jochen" <jkrause@xxxxxxxxxxxxx> wrote:
>
> > Dear Richard, dear Technology PMC,
> >
> > In the spirit of openness I am sending you this letter to share my
> > experience about the way the EclipseCon program is created
> and why I
> > think this is not consistent with the needs of our community:
> >
> > Two statements upfront: No doubt that I am disappointed.
> However, this
> > is not me whining that my talk proposal has not been accepted - the
> > same has happened to many others and everybody has to
> accept it. This
> > is about how the program for EclipseCon is selected, and if this
> > process is suiting the needs of our community.
>
> OK, great. (I voted for the RAP talk, btw ;)
>
> > From my experience with program committees it is the job of such an
> > committee to provide a program that is interesting to the target
> > audience.
>
> Agreed, but I'm not sure we have reached a general consensus
> on who the audience is, or who we're trying to attract. This
> is probably an important thing to work out and publicize
> before the next EclipseCon.
>
> > At Eclipse we don't have a profit motivation for running EclipseCon
> > (although we don't want to loose money on it)
>
> Right, and I think most conferences run this way.
It seems you have more luck than I do with the conferences you attend
;-) Many of those that I attend are for profit and they need to appeal
to as many people as possible. This is from my point of view not
necessary for EclipseCon, and thus the program committee has greater
flexibility creating the program.
>
> > , but we want to
> > foster the community around Eclipse. That's why the program should
> > cover a good mix of established technologies and immature but
> > innovative technologies and not only mainstream interest.
>
> Agreed, and that's what I thought we were aiming to do.
>
> > EclipseCon is the prime
> > event in the Eclipse community every year. Especially for
> new projects
> > EclipseCon is the primary platform to get in touch with the
> community,
> > and thus is a pretty important cornerstone.
>
> Yes, meeting members of the development and user community is
> a key aspect of the conference (to me, anyway).
>
> > Missing EclipseCon means missing one year ...
>
> Well, there are now several Eclipse-related
> conferences/gatherings. And although I agree EclipseCon is
> the most important, I'd be careful not to overestimate its
> value to the success of a project. Also, a year isn't really
> that long (they seem to get shorter all the time ;).
I agree that a projects success has only little to do with a long talk
at EclipseCon. However, if you look at the requirements (or
recommendations) we impose on projects with respect to adopters,
committers and end users (dev process) we want to make sure that
projects get a chance to meet like minded people in person and not
magically expect that just great technology will create a big following.
And EclipseCon is the gathering of the "in crowd", the other events
(beside Eclipse Summit Europe) are more end user oriented. The Eclipse
community has grown quite significantly, and I think EclipseCon is
essential in bringing the diversity of 70 projects into one place. From
my point of view the long talks are the only format that are suited to
the needs of new projects (maybe the demos are too, but have by design
little background info). At Eclipse Summit Europe we had a "New and
Noteworthy" track with 20-25 minute presentations - this was a little
short too, but it was the chance to deliver more than just an idea.
>
> > That is why I am quite dissatisfied with the way the Technology PMC
> > has chosen to select the content for the Technology track:
> >
> > From my point of view the following criteria should be taken into
> > account when selecting content for EclipseCon:
> >
> > - technology (how big is the impact of the technology on Eclipse)
> > - presenter (meritocracy should play a role here - is the
> presenter
> > an active member of the community)
> > - competency == expected quality (is the presenter close enough to
> > the project / the technology to provide insight)
> > - interest (expected / stated by the target audience)
>
> Is this an ordered list? How do you judge/estimate "impact"?
This is not an ordered list. I think it is o.k. if the pc members do the
judgement from their experience and background.
> I agree these are important general criteria, and I think
> we've kept these and others in mind during the selection process.
>
The link to the technology pmc meeting notes has been added as an
explanation for why submissions have been accepted / declined. And I
still think that the criteria for selection that has been offered there
is inadequate. Following the link to previous pmc minutes reveals that
the "interest by target audience" criteria has been applied. But I stay
in disagreement that the "audience draw" is the most important criterion
(as laid out above).
> > The criteria that has been applied was
> > (http://www.eclipse.org/technology/pmc-minutes.php?key=2006.12.14),
> > the order has been changed below to provide comments:
> >
> >> * Community comments: We were disappointed at the lack of community
> > comments on the submissions.
> >> 7/25 had one comment, the rest none. We accepted 5 of those 7 and
> > declined 2.
> >> * PC votes: There were few PC votes either from within our
> sub-PC or
> > overall.
> >> * Community votes: As per Bjorn's blog post, we discounted
> community
> > votes that did not include comments.
> >
> > It is quite obvious that neither the community nor the program
> > committee was very active in providing feedback and votes.
> That makes
> > it a poor decision criteria. With respect to the program
> committee I
> > find only two possible explanations: All submitted content was so
> > mediocre that no votes have been given, or the program
> committee did
> > not get to evaluate the submissions in the necessary depth. This
> > becomes even more relevant when taking into account that seemingly
> > only two out of five program committee (== pmc) members
> have taken the decision.
>
> Votes and comments are marginally effective, in general, if
> you ask me.
> When I consider the value of a vote, I look at who cast it.
> If I see a series of votes from one organization, I generally
> consider it a single vote. Whether it includes a comment or
> not is also not too important, to me (I disagree with Bjorn a
> bit here). Comments are fine if they provide some valuable
> or constructive feedback. Just saying "I think this talk
> sounds interesting and will be worthwhile for attendees..."
> is implicit in the vote, imo. So, why add the comment? If a
> comment calls out something like, "Perhaps you could add
> this, or combine with this other talk, ..." is much more valuable.
>
> What I'm saying is that it takes some time to cast votes, and
> even longer to add worthwhile comments. Given that most of
> us have little time, I'd be happy to see just PC member votes
> cast, even without comments. Community votes are important,
> as are their comments, but they are only a part of the
> selection equation; that is, if we are indeed looking to
> provide a balanced program.
>
I fully agree with you.
> >> * Duplicate presentations: None of our accepted presenters
> have other
> > submissions.
> >
> > The number of presentations should definitively not play a
> role in the
> > selection process, meritocracy would be a better guideline.
>
> I'm not sure I follow this one, on both counts.
>
Maybe I got this wrong, but I understand that it is about how many
submissions a presenter has submitted. Anyway, I don't see a problem if
someone who is highly regarded in the community (based on meritocracy)
is doing more then one talk.
> >> * Discussion with the authors: When we had discussions with the
> > authors, the accepted talk
> >> authors responded quickly. A few of the declined talk
> authors never
> > responded to questions
> >> from the program committee which led us to assume a lack of
> >> interest
> > on the part of the submitter.
> >
> > Assuming lack of interest from late responses is disputable, but
> > making the speed and not the quality of the feedback a selection
> > criteria does not seem right.
>
> I think it's a valid assumption. How do you assess the
> quality of feedback that never arrives? We have deadlines to
> consider on both sides, so it's in the submitter's interest
> to provide timely responses.
>
I was talking about the first part of the sentence: "the accepted talk
authors responded quickly"
If someone is not replying it is reasonable to assume a lack of
interest.
> >> * Eclipse projects: Talks about Eclipse projects were
> given a slight
> > priority.
> >
> > I think that EclipseCon is a very important platform for
> projects, and
> > this should be taken into account (given the fact that we
> have only 7
> > slots for 25 projects). Maybe we want to share topics of general
> > interest like "Eclipse on Swing" or "Prototyping, Automating,
> > Exploring
> > - Interactively Scripting Eclipse" that are not technology projects
> > with tracks like "Fundamentals", "Rich Client" or others.
>
> Not sure I get your point.
The technology track is not only the place that 25 projects are
competing for, but also scripting and other interesting stuff. This
reduces the available slots for the projects - and we are incubating
many projects in technology that should get a chance to present.
>
> >> * Premature results: Topics that appeared premature were
> given a much
> > lower priority.
> >
> > The "premature results" seems to be an important criteria
> ("much lower
> > priority"). With the given IP process at Eclipse that makes
> delivering
> > of new technologies sometimes very difficult (at RAP we are still
> > waiting for approval for 4 classes since more than 6 month) I would
> > assume that the program committee consults with the submitters
> > (projects) on this topic. I am not aware that this has
> happened with
> > our submission.
>
> Let me see if I got this one: if a project is still working
> through the IP process, then it doesn't yet have a download
> available (the PMC would know this, right?).
I am not sure that the technology pmc is aware of the state of the IP
status of all technology projects. This seems almost like an
unreasonable effort to expect them keep up with that.
> I assume that
> the "premature results" criterion is aimed at avoiding the
> situation where someone goes to EclipseCon, hears about some
> new project, then can't actually do anything because it's not
> "real" yet.
> Seems sensible to me. Perhaps a short talk is more
> appropriate for this "coming attraction" type of project, or
> even a Demo if it's close to becoming real?
>
Yes, I think it is a reasonable criterion too. But it should take the
special situations of new projects into accout.
Anyway, the EMO has put a great proposal forward to make the IP process
better suitable to projects in incubation. So I expect this problem to
go away soon.
> > It was one among the stated goals of the EclipseCon program to make
> > the decision making progress transparent and to involve the
> community.
> > To me, this does not seem to have worked in this case. If
> others feel
> > that I am not completely mistaken I would welcome a
> discussion on how
> > to improve the process for future EclipseCons.
>
> Thanks for providing the feedback, and for offering to help
> improve the process. Clearly, we'll never make everyone
> happy, but we can definitely improve as we go. I'm not sure
> how we can make it more transparent, aside from having a
> posted set of general criteria and goals beforehand (in time
> to solicit feedback and make sure we're mostly in agreement
> before the submissions arrive). With that, I still find
> EclipseCon to be the most open and transparent of any
> conference, w.r.t. the selection process.
>
EclipseCon is the most open and transparent conference with regard to
the selection process that I know of, too. This transparency enabled be
to disagree. Maybe I have missed the discussion on the general goals and
selection criteria, otherwise I think it would be good to have it (maybe
at EclipseCon ...).
Best Regards,
Jochen
> > Jochen Krause
> > RAP project lead
>
> --
> Richard C. Gronback
> Borland Software Corporation
> richard.gronback@xxxxxxxxxxx
> +1 860 227 9215
>
>
