Re: [cross-project-issues-dev] Retiring the UDC and removing it from the

[ERIC CLONINGER <dcp874@xxxxxxxxxxxx>]

Whew! Finally back from the October conference run....

My use of the word "feature" is not analogous to the Eclipse
definition. I would think we would want to give implementers the
freedom to inject information into the stream at their discretion. As
a product manager, I would want to track menu commands that my plugins
contribute, activations of my perspective(s), dialog(s) and view(s).

We've done this in our isolated example enough to see if it's viable.
If we'd like to have a separate discussion and/or a demo, we're happy
to accommodate.

-E

On Mon, Sep 26, 2011 at 1:51 PM, Wayne Beaton <wayne@xxxxxxxxxxx> wrote:
> How do you define "feature"? Commands? Menu selections? View
> activations? Or "feature" in the p2 sense?
>
> Wayne
>
> On 09/23/2011 12:11 PM, ERIC CLONINGER wrote:
>> I spoke at I/O to the Sr. Manager for Google Analytics (GA) and he
>> said that using it for this purpose is very much a valid use case.
>> Because GA was originally intended for web sites, some mechanism needs
>> to be placed to turn application identifiers into things that look
>> like web requests. Fortunately, with the Java naming standards we
>> already have, it's a trivial matter.  Then there is the matter of
>> cost. The terms for GA differ somewhat depending on what geo is used,
>> but somewhere north of 5m hits/month there is a cost associated with
>> it.
>>
>> Most of us tend to be online during the day, but there certainly are
>> users who aren't. A decision of how to incorporate those users' data
>> into the mix may be useful. The safeguards we've put in place with UDC
>> to handle batch uploading and obfuscation of identifying details are
>> necessary. I think Google has done this already as I would think it
>> would be an obvious privacy concern.
>>
>> I would like to be able to track
>> * load/unload of plugins
>> * use of features
>> * use of "this feature" followed by "that feature" (e.g. workflow improvements)
>> * average time through a workflow
>>
>> These are just examples. There are lots more that we could devise. The
>> most important thing to me (as an adopter/packager) is to be able to
>> access not only the core statistics, but those that are specific to my
>> product. Whether that requires my own GA account or using the
>> Foundation, is a detail to resolve.
>>
>> -E
>>
>> On Wed, Sep 21, 2011 at 1:42 PM, Wayne Beaton <wayne@xxxxxxxxxxx> wrote:
>>> I'm interested. I know just enough about Google Analytics to know that I
>>> don't know enough about GA. What sort of information would/could we capture?
>>>
>>> Wayne
>>>
>>> On 09/21/2011 12:28 PM, ERIC CLONINGER wrote:
>>>> Wayne,
>>>>
>>>> I briefly proposed to Ian a few months back that we look to redesign
>>>> UDC to use Google Analytics in place of the UDC backend. In fact,
>>>> we're experimenting with this in MOTODEV Studio for an upcoming
>>>> release.
>>>>
>>>> Would it make sense to have this discussion in a larger group? As a
>>>> product manager of a product based on Eclipse, I would like to know
>>>> what features my users find the most compelling. We tried with UDC,
>>>> but I think GA might give the type of statistics that I want to see.
>>>> To be honest, if there's enough interest in this, we could donate the
>>>> effort.
>>>>
>>>> -E
>>>>
>>>> On Wed, Sep 21, 2011 at 10:43 AM, Wayne Beaton <wayne@xxxxxxxxxxx> wrote:
>>>>> The plug _has_ been pulled. That's not to say that we can't plug it back
>>>>> in...
>>>>>
>>>>> Unfortunately, the privacy issues do not disappear. It seems obvious,
>>>>> for example, that you might want to capture error messages. How big a
>>>>> deal could that be? Well... it turns out that error messages are
>>>>> probably the biggest source of potentially private information in
>>>>> everything that we currently collect. With the UDC framework in place,
>>>>> it would be relatively easy for somebody (either maliciously or
>>>>> innocently) to create a plug-in that provides all sorts of questionable
>>>>> data.
>>>>>
>>>>> It's the questionable data that gives me the willies. We get a lot of
>>>>> that already.
>>>>>
>>>>> I'm not against resurrecting the UDC, but we need to sort out how to
>>>>> make this workable. Maintaining privacy is only one issue.
>>>>>
>>>>> Managing the data requires EF resources. Frankly, it takes quite a lot
>>>>> of time and effort to stay on top of the data being brought in. We have
>>>>> to balance that effort against the actual value of the data. Thus far,
>>>>> we have not been able to get any kind of value out of the data. Other
>>>>> than, perhaps, good will with a small handful of university researchers
>>>>> (which is real value IMHO, just not enough to warrant the ongoing expense).
>>>>>
>>>>> How valuable is this data to you? Is it worth directing EF resources off
>>>>> of other projects to make it happen?
>>>>>
>>>>> Before we can talk about resurrecting this, we need a reasonable
>>>>> strategy for managing the data moving forward.
>>>>>
>>>>> FWIW, I love the "everything and nothing in particular" (tm) line.
>>>>>
>>>>> Wayne
>>>>>
>>>>> On 09/20/2011 11:25 PM, Pascal Rapicault wrote:
>>>>>> Before we actually definitely pull the plug on this, I'd like to throw a last minute twist on this and see if the fundamental problems can't be addressed and if what is I suggest would be of interest to anyone.
>>>>>> As I understood it, the two fundamental pbs are: too much data, data confidentiality.
>>>>>>
>>>>>> IMO all the problems stem from gathering everything about a particular instance (e.g. all the plugins), which in turns generates the confidentiality problem (for example, we can't give out the complete data because it contains the list of all commercial plugins installed by the user), which makes any request to access the data a no-go without signing of your blood and thus render the initiative kinda moot.
>>>>>>
>>>>>> What if instead of gathering "everything and nothing in particular" (tm), the UDC became an opt-in mechanism where plug-in authors from eclipse.org could decide to monitor a specific aspect of their plug-ins, thus allowing them to make educated decisions about a particular functionality. For example, for p2, I would like to know how many ppl actually enable the automatic check for updates preference? To achieve this, I would explicitly tell the UDC to gather the value for the known preference key, and later on I could just do a few sql queries against the system to see the results (since the data would have been curated by explicit inclusion, there would not be any confidentiality issue, or at least significantly reduced).
>>>>>> With this sort of opt-in mechanism, the data collected is specific and well-known, thus alleviating the confidentiality issue (for example we would no longer gather the complete list of plugins) and also the size aspect, and consequently making the UDC more consumable by committers since it would allow them to have some sort of eclipse integrated polling system.
>>>>>>
>>>>>> Does it make sense? As a committer would you use that?
>>>>>>
>>>>>> PaScaL
>>>>>>
>>>>>> Ps: Sorry for the late reaction on this topic but it literally came to my mind as I was coming on thinking of the e4 skinning (and don't ask the question I would like to see answered :p).
>>>>>>
>>>>>> On 2011-09-20, at 1:04 PM, Wayne Beaton wrote:
>>>>>>
>>>>>>> Greetings folks.
>>>>>>>
>>>>>>> Putting on my Eclipse Foundation staff member hat... those of us at the
>>>>>>> Eclipse Foundation who have been managing the collected data have
>>>>>>> decided that it's time to pull the plug on the UDC server [1]. That is,
>>>>>>> we have decided to stop collecting UDC data. In this regard, we will be
>>>>>>> flipping a bit on the UDC server at the end of September which is cause
>>>>>>> it stop retaining uploaded data. Clients in the field will continue to
>>>>>>> function as though everything is hunky-dory on the server (i.e. they
>>>>>>> should continue to clean up after themselves).
>>>>>>>
>>>>>>> Changing hats... on behalf of the Eclipse Packaging Project (EPP)--the
>>>>>>> project responsible for the UDC--given that the Eclipse Foundation will
>>>>>>> no longer be collecting usage data, we have decided to pull the UDC from
>>>>>>> the simultaneous release, starting with Juno. By extension, the UDC will
>>>>>>> be removed from all Juno packages.
>>>>>>>
>>>>>>> The UDC will be included in the Indigo SR-1 and SR-2 releases. I am
>>>>>>> willing to discuss removing the UDC from SR-2 packages if there is
>>>>>>> agreement. The reason for keeping it in the simultaneous release is that
>>>>>>> I am aware of several organizations currently using the UDC as the basis
>>>>>>> for their own collection of usage data.
>>>>>>>
>>>>>>> The long term status of the project is currently in limbo. There has
>>>>>>> been almost no new development work done on the UDC in a very long time.
>>>>>>> It may be time to archive the code. If anybody is interested in stepping
>>>>>>> up to maintain the code, please express that interest on the epp-dev
>>>>>>> mailing list.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Wayne
>>>>>>>
>>>>>>> [1] http://waynebeaton.wordpress.com/2011/09/16/bye-bye-mon-udc/
>>>>>>>
>>>>>>> --
>>>>>>> Wayne Beaton
>>>>>>> The Eclipse Foundation
>>>>>>> Twitter: @waynebeaton
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> cross-project-issues-dev mailing list
>>>>>>> cross-project-issues-dev@xxxxxxxxxxx
>>>>>>> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>>>>>> _______________________________________________
>>>>>> cross-project-issues-dev mailing list
>>>>>> cross-project-issues-dev@xxxxxxxxxxx
>>>>>> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>>>>> --
>>>>> Wayne Beaton
>>>>> The Eclipse Foundation
>>>>> Twitter: @waynebeaton
>>>>>
>>>>> _______________________________________________
>>>>> cross-project-issues-dev mailing list
>>>>> cross-project-issues-dev@xxxxxxxxxxx
>>>>> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>>>>>
>>>>
>>> --
>>> Wayne Beaton
>>> The Eclipse Foundation
>>> Twitter: @waynebeaton
>>>
>>> _______________________________________________
>>> cross-project-issues-dev mailing list
>>> cross-project-issues-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>>>
>>
>>
>
> --
> Wayne Beaton
> The Eclipse Foundation
> Twitter: @waynebeaton
>
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>



-- 
Eric Cloninger (ericc@xxxxxxxxxxxx)
Product Line Manager, MOTODEV Tools
Eclipse Sequoyah Project Lead