Re: [virgo-dev] p2 & Virgo

["Todorova, Katya" <katya.todorova@xxxxxxx>]

Hi guys, 
 
As a follow-up from the EclipseCon discussions about p2 & Virgo here's an initial plan and proposals for execution:

Requirements:
    Should provide isolation between the two regions - the app should not resolve against the hidden kernel stuff, that should stay the same with p2
    Should provide a way to transport the user regions between Virgo kernel instances
    Should keep hot deploy on a p2-enabled Virgo 
    Should be possible to update the runtime when Virgo is offline
    Should keep the "quasi" functionality

Scope: 
    Virgo fully managed by p2 - every flow which affects the runtime (in terms of bundles or/and dependencies)  is done by p2

Out of scope:
    Virgo repositories format - should they be p2 repositories? If the format is not required to be changed in order to cover the requirements listed above it will be left out of scope. So is the concept of  "remote" repositories.

Time frame:
    Should be productive for Virgo release in June

Benefits: 
    Performance optimizations: 
                -  Pre-calculations could be performed at build time instead on first startup (similar to Eclipse behavior
                -  Additional calculations only when there's a change, not on each startup for example
    Faster resolution (done only on top of metadata).  At least it could help in the negative scenarios  - fast failure when trying to install an artifact with missing dependencies
    System is always in a determined state (explicitly described)

Technical notes:

1. Virgo deployment entry points - AFAIK these are Virgo deployer and pickup folder:
Virgo deployer 
There was an idea of replacement the relevant stages of deployer with p2-aware stages and to switch between them using configuration.
Some questions here: 
Is the configuration supposed to be changed at runtime in order  to drive the deployment mode? In a sense  you deploy a bundle via p2 then change the configuration and the next artifacts you deploy are unknown to p2?
Or the configuration would  let you pick a Virgo distribution with or without p2? 
What's the benefit of switching between p2 and non p2 deployment? From my perspective it has a significant drawback in terms of consistency and performance so I'm curious to know the motivation? 

We have to pay special attention to  the supportability features of the deployer quasi steps if this is moved or delegated to p2.

A question about plans  - what do you need ordering for? Is it something that people rely on and should stay as it is or Virgo can stop supporting that from a particular version on? It would be tricky to implement solution that offers ordering since we step on OSGi.

Pickup
Is pickup functionality designed to be used productive? Or it's applicable only in development use cases?
It could easily be replaced with "dropins" functionality without any glue code if there's no productive usage.
Are there other supported ways to deploy something in Virgo? Equinox console for example? 

2. Standard solution (p2 provisioning of subsystems)
The solution targets covering more general subsystems use cases rather than just Virgo specifics. Since subsystem RFC is not final and probably won't be in the specified time frame we will step on the currently available concepts and Virgo implementations of digraph, region manager, etc. Glyn is aware of how the things are going on in the spec space so I guess Virgo implementation will be aligned with the specification rather than introducing a custom alternative. 
Given that, as soon as the spec is ready Virgo provisioning could be the first working example how subsystems could be provisioned with p2.

3. Design changes 
p2
In order to keep the "quasi" functionality with minimum duplication of concerns p2 should support the option for "equinox resolution" validation on install. AFAIK there's such feature in p2 so this is not expected to be a significant effort (don't know whether it's implemented, the important here is that it fits to p2 vision) If that's not implemented maybe "quasi" steps could be contributed from Virgo to p2.

Representation of Virgo artifacts in p2 - configuration files for both regions, plans, etc. Special attention to minimizing duplication of metadata (equinox state files, digraph, bundles.info, p2 profiles, Virgo custom configuration)

Isolation between regions in p2 terms -the question whether that representation would affect p2 internal files and how is being discussed in p2-dev ML but it's still open and it's probably the hardest problem.

Virgo
If the standard solution takes place there would be just configuration changes at runtime: System startup would be delegated to p2. Virgo custom configuration files could be replaced by p2 configuration files if they are not supposed to be used by the end user. (For example  properties for launching kernel could be replaced by bundles.info while userregion.properties could stay as it is and has an appropriate representation in p2 bundles.info and profile files)

The only major change is expected to be the way Virgo distributions are built.

Here it's important to clarify the user perspective and whether they  find dealing with "low-level" properties file useful.

Risks
It's not clear to me how release cycles of both projects come together. What could be consumed from Virgo - only released stuff or a development build as well? Is there other time restrictions that come out of that?  

Isolation between regions could be harder than expected and there could be a potential conflict between Virgo and p2 requirements. Solving that could be tricky in the specified time frame so there should be a fallback option:
Virgo could implement p2 representations of its custom artifacts and build a solution based on currently available p2 means.  
The drawbacks are: 
    we sacrifice the "standard" aspect 
    the p2 Virgo wouldn't be installed by p2 but would be distributed as zip (current state)
    cannot cover "transport the user regions" requirement

Execution plan
Next two weeks Bobby and I will be extremely focused on that topic. 
Starting point - the prototype we have. It has two significant defects - modifying existing IUs which is unacceptable from p2 POV and requiring manual build steps.

1. Address the defects - not sure that on top of the assumptions I had I could bring that to success but it won't take much time to prove whether this could be done right.
2. Quasi support
3. Research and initial implementation of the requirement:"Provide a way to transport the user regions between virgo kernel instances" - it would result in an idea/requirements how to represent the layout of subsystems both in terms of p2 internal representation and on the file system.

That's what I have in mind for now. Sharing use cases, comments, ideas, concerns are welcome. 

P.S. I'm subscribed for virgo-dev ML, no need to copy me. So is Pascal.

Thank you,
Katya

> -----Original Message-----
> From: Glyn Normington [mailto:gnormington@xxxxxxxxxx] 
> Sent: Saturday, March 26, 2011 2:03 AM
> To: Dmitry Sklyut
> Cc: Virgo Project; Todorova, Katya; Pascal Rapicault
> Subject: Re: [virgo-dev] p2 & Virgo
> 
> Hi Dmitry
> On 23 Mar 2011, at 08:50, Dmitry Sklyut wrote:
> 
> > p2 kind off replaces
> > Quasi phases
> 
> One thing we need to be careful about is that the p2 resolver 
> does not take into account uses constraints. So it is 
> possible for p2 to provision a collection of bundles that 
> will not resolve. Of course this is unlikely in normal 
> scenarios, but could happen e.g. as soon as multiple versions 
> of a given bundle are installed in the user region.
> 
> Glyn