Re: [rt-pmc] Feature version range policy

[Jeff McAffer <jeff@xxxxxxxxxxxxxxxxx>]

>> 1) Feature semantics.  If you have a feature that says Includes X and then the p2 metadata indicates an inexact range, the p2 metadata does match the feature. This will be the source of numerous and subtle problems. 
> 
> What kind of subtle problems are you talking about?

Largely the issues discussed elsewhere in this thread.  But there is a separate topic around variations between what the feature.xml says and what the corresponding IUs say.  If the feature says include X v1.0.0.3 (no range because hey, that's what an *include* is) and the metadata turns out to say require X [1.0.0.3, 2.0.0) then the ultimate provisioning behaviour will not match the authored semantics.  It is the moral equivalent of the Java compiler compiling 1+2 to result in 4.  The code sure says 1+3 but 4 keeps getting printed on my screen.

>> I am not saying that Feature semantics are perfect but we can't just go around changing implementation in incompatible ways.  It would be like having a concrete class not conform to its interface/API. You/others can seek to develop new semantics etc but simply ignoring those that are in widespread use is not a recipe for success.
> 
> Nobody intents to change things in incompatible ways or break existing
> customers. That's why we are having this discussions early in the Indigo
> release cycle. :)

Great.  So just to be clear then, publishing features such that includes are converted to an inexact range IS a breaking change wrt established feature semantics.

>> 2) Determinism.  Using version ranges introduces non-determinism.  For example, say there is an "ECF" feature that currently *includes* the ecf.foo and ecf.bar bundles. After being built those inclusions would be bound to exact versions of ecf.foo and ecf.bar (say 1.2.3.xxx for both). Further, the ECF feature itself would have an exact version, say 1.3.6.yyy. Given that, a consumer could provision their system (runtime or target) with ECF 1.3.6.yyy and know exactly what they will be getting regardless of whatever else is available.  Deterministic.
>> 
>> Say now that we relax the version range in the p2 metadata to effectively have a *requirement* on ecf.foo and ecf.bar.  The resultant metadata would say that the ECF.feature.group iu requires the bundles with a range of say [1.0.0, 2.0.0). The feature IU itself would still have a precise version number say 1.3.6.zzz.  Now the consumer goes to provision ECF 1.3.6.zzz. The versions of ecf.foo and ecf.bar that get provisioned will depend on the particular versions that are available at the time of the operation. Add a newer repo and you will get newer versions.  If a repo happens to be offline at that precise moment you could get an older version.  You might even get a mix of new and old.  Non-deterministic.
> 
> The algorithm that chooses the IUs is still deterministic. Given a
> defined set of repositories, it will always provision the same set of
> IUs (newest matching set).

Indeed the algorithm is deterministic but the two people running the same algorithm will get different results unless every detail of their inputs are the same.  Unfortunately, that is nearly impossible to guarantee since network reachability, caching and other factors come to play.  So, for all intents and purposes the provisioning operation is non-deterministic in the fact of ranges.

> However, while the classical (strict version ranges) approach just fails
> in case of non-available repositories, non-exact version ranges provide
> some sort of best effort approach in a way that it tries to come up with
> a solution that is possible with the current set of available repositories.
> Furthermore feature version ranges emphasize the importance of bundle
> version ranges. Currently feature includes "downgrade" bundle version
> ranges to a second class citizen. A newer matching bundle will not be
> installed even though it is available.

Best effort certainly can be interesting/useful. However, if you were installing some Helios SR1 EPP package you likely do want it to fail if some of the SR1 bits cannot be found. Conversely, if there happens to be a reference to last night's build repo available, it is unlikely you want that content installed. 

There is a difference between runtime dependency expression and provisioning time dependencies.  As a bundle producer you spec what your bundle needs at a functional level.  Integration and provisioning is all about what sets of bundles can be composed at an operational level.  Features and p2 work at this level.  The runtime dependencies are used to inform provisioning but to make an operational system refinement and additional information is required.

>> Furthermore, two people doing the exact same operation may get different results due to repo visibility or other constraints in our respective systems.  Similarly for one person doing the same operation twice.  Finally, in a repo with multiple versions, consumers will never be able to provision anything but the latest so folks would not be able to install Indigo once Indigo SR1 comes out.
> 
> Having a feature with relaxed version ranges does not stop anybody from
> providing other features that define strict version ranges (which allow
> consumers to install Indigo plain).

Sure.  And vice verse. If ECF would like to produce both that would likely be fine (as long as the effects of the relaxed version range one are clearly documented).  However, from an community point of view, I think it would be against the release train spirit and community consumption patterns to supply only the relaxed version.  For example, if this had happened in the last release, no one would be able to install Helios SR0 packages right now and consumers of products based on Helios SR0 would be getting random updates happening.

>> To be sure, some of the semantics just described *can* be useful.  Simply getting the latest is very cool and if you are in a scenario with managed repos you can control what gets installed by controlling repo contents. In my understanding however, this is not the general case.
> 
> I guess a good part of the discussion is to define the "general case"
> and find out if today there is still only one use case for RT. For a
> commercial RCP product it is probably true that relaxed version ranges
> make things much more complicated. In other settings though, one
> definitely wants to exploit the power of OSGi WRT to (bundle) version
> ranges and not restrict oneself to static features at deploy-time.
> 
> Maybe projects should simply choose which approach works best for them
> and their consumers.

Projects should definitely be in tune with their consumers. It would be very interesting to hear details of the ECF consumer scenarios that are driving this discussion.

>> Summary, evolving the way we define and manage and tool groups of bundles etc. is likely something that needs to be addressed.  This is a topic that will be affected by and will impact many areas in Eclipse from build to p2 and PDE to the simultaneous release and our consumers.  As such, the topic should be brought up in the Architecture Council or other widely scoped forums.
> 
> Are you going to start this discussion at the architecture council? Or
> what is the proper way to bring this up?

It seems like the ECF team is in the best position to put forward the scenarios and proposals for this change.  Likely the best bet is to open a bug against the architecture council with all the details.

Jeff