[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [virgo-dev] p2 & Virgo
|
Hi all,
I'll try to give some answers to the respectable amount of questions that came up :)
> Is the configuration supposed to be changed at runtime in order to drive the deployment mode?
> In a sense you deploy a bundle via p2 then change the configuration and the next artifacts you deploy are unknown to p2?
I would expect the deployment modes cannot be switched during runtime as that may cause severe inconsistencies. Here it is interesting to explore does it make sense to even have separate distributions for each deployment mechanism as a first step - IMO it does, as it safeguards us from unintended confusion from the users' POV.
---------------
> Is pickup functionality designed to be used productive? Or it's applicable only in development use cases?
AFAIK it is suggested for development purposes only. The productive side is covered by the repositories concept. So it would be nice, as was already expressed by a developer in this mailing list to have p2's dropins with similar hot-deploy functionality and point it to pickup.
----------------
> What could be consumed from Virgo - only released stuff or a development build as well?
Virgo's release consume only other released artifacts so if we can't align that with the p2 release I think it's OK to have milestone builds of Virgo with p2, until the integration matures and possibly have a separate release then.
----------------
> What does "Should be possible to update the runtime when Virgo is offline" mean? Is it just the kernel, just the user region or both?
Both.
----------------
> What does "transport of the user region mean"? Do you want to take a region from one instance and copy it over to another fwk,
> or do you just mean installing the content of the user region from a p2 repository? Could you please detail more?
The use case is this - we want to be able to take an offline Virgo, provision the app on it, start it. When it's no longer needed stop it, and revert it to that clean initial state. Whether that means to delete a user region folder and copy a new unprovisioned one or uninstall the app depends on which approach gives best performance and/or is easier to implement.
----------------
> What does "hot-deploy" mean? Add remove bundles to a region? Add remove bundles to the kernel? How does that relate to the "pickup"?
Yes, add/remove deployment artifacts to a region - the user region to be precise. I think I already covered the "pickup" part above.
----------------
> In this iteration, are we limiting the p2 support to only one user region or more?
For now I think it would be safer to start with one user region, having in mind the possibility of many when designing.
----------------
I do agree that it is highest priority to see how p2 and the regions presentation fit together. That is our focus at the moment.
When we have something more detailed we'll update the thread to sync with you guys, meanwhile we are taking all comments into account.
And last on the Quasi comments. It seems that the conditions are just right to suggest the notion of a Quasi step switch, set by default to ON. That seems to fit nicely to both sides - having it on will guarantee you uniform resolution, but if you want to sacrifice some safety for speed in your use case then why limit you :)
I'll keep in touch.
Best Regards
Bobby
-----Original Message-----
From: virgo-dev-bounces@xxxxxxxxxxx [mailto:virgo-dev-bounces@xxxxxxxxxxx] On Behalf Of Glyn Normington
Sent: Monday, April 04, 2011 6:44 AM
To: Virgo Project
Cc: Thomas Watson
Subject: Re: [virgo-dev] p2 & Virgo
I'm in transit and then at an OSGi F2F for 3 days, so I'll just make two comments for now.
Firstly, you can deploy various kinds of artifacts using the Virgo deployer: bundles, configurations, factory configurations, plans, PARS, and user defined artifacts (although I don't know of anyone doing that in the wild). Let's not get too focussed on the bundle case.
Secondly, our users have hit deployment failures due to uses constraints so the "quasi" function needs to be retained or there will be a serious regression.
Regards,
Glyn
On 3 Apr 2011, at 15:07, Pascal Rapicault wrote:
> Here are some notes / questions / feedback based on this last mail:
>
> - What does "Should be possible to update the runtime when Virgo is offline" mean? Is it just the kernel, just the user region or both?
>
> - What does "transport of the user region mean"? Do you want to take a region from one instance and copy it over to another fwk, or do you just mean installing the content of the user region from a p2 repository? Could you please detail more?
>
> - What does "hot-deploy" mean? Add remove bundles to a region? Add remove bundles to the kernel? How does that relate to the "pickup"?
>
> - In this iteration, are we limiting the p2 support to only one user region or more?
>
> On the changes required to p2:
> I'm not quite sure where the "configuration of the digraph" fit in all that, but I think this is an area that should not be underestimated. Tom and I had a conversation Friday afternoon on the topic. Our current thinking is that to provide an "atomic" way of updating the region (where atomic means that the bundles are directly installed in the appropriate region, and not moved from the 'primordial' region to the actual regions), it may be necessary to have the digraph bundle take over some of the responsibilities of the "simple.configurator" (maybe completely supplant it) to read the bundles.info like file that would deal with the bundles and the region wiring. This also means that a "digraph.configurator.manipulator" would have to be provided since there would likely have to be a new file format at least to capture the wiring info. I think that here you have quite some flexibility since you are controlling the digraph, and what will configure it.
>
> Quasi step: Though I can understand why you would want to keep the quasi step, I'm questioning its necessity in a first iteration. IIRC Virgo needs the quasi step in order to figure out what needs to be installed (figure out missing deps, etc), and since it is using the fwk resolver, it gets uniform resolution. We know that p2 resolver is not a 100% OSGi resolver (this is by design) and that some slight discrepancies can occur. However there is no clear evidence that the applications that will be deployed will actually run into the pitfalls that everybody mentions (mostly based around uses-clause). After all p2 has been provisioning quite a few apps so far :)
> The other thing with the quasi step is that I don't know if it can be executed on a non-running framework and how that works with the hooks. For example, when p2 is managing a non-running server, a quasi step would have to load the fwk state from the non-running server (or at least all the bundle.info and digraph wiring infos files).
> Could you tell more about the "supportability" features from the quasi step? Is that relating to explanation in case of failures?
>
> The representation of regions in p2 is something that can be made as complex as desired. For example one could decide to go down a path where all regions are resolved together in one shot, taking into account the wiring, etc. Though this could provide the best solution, it could require a major overhaul of the dependency resolver and some changes in the director. Another possibility is to assume that the regions will be provisioned one after the other (thus the question about how many regions).
>
> Release cycles and execution plan:
> The release cycles of p2 and Virgo don't align. Equinox and p2 have been notoriously conservative and stick with the platform schedule (http://www.eclipse.org/projects/project-plan.php?projectid=eclipse). This means that by April 29th we are feature frozen and that we will not be able to accommodate large changes. That said, I don't see any problem in running special builds from another branch or something like that.
>
> On the execution plan, I just want to emphasize the importance on figuring out how the digraph data (and other virgo config files) gets "configured" sooner than later as this will likely shape the necessary metadata
>
> HTH
>
> PaScaL
>
> On 2011-04-03, at 9:04 AM, Todorova, Katya wrote:
>
>> Hi guys,
>>
>> As a follow-up from the EclipseCon discussions about p2 & Virgo here's an initial plan and proposals for execution:
>>
>> Requirements:
>> Should provide isolation between the two regions - the app should not resolve against the hidden kernel stuff, that should stay the same with p2
>> Should provide a way to transport the user regions between Virgo kernel instances
>> Should keep hot deploy on a p2-enabled Virgo
>> Should be possible to update the runtime when Virgo is offline
>> Should keep the "quasi" functionality
>>
>> Scope:
>> Virgo fully managed by p2 - every flow which affects the runtime (in terms of bundles or/and dependencies) is done by p2
>>
>> Out of scope:
>> Virgo repositories format - should they be p2 repositories? If the format is not required to be changed in order to cover the requirements listed above it will be left out of scope. So is the concept of "remote" repositories.
>>
>> Time frame:
>> Should be productive for Virgo release in June
>>
>> Benefits:
>> Performance optimizations:
>> - Pre-calculations could be performed at build time instead on first startup (similar to Eclipse behavior
>> - Additional calculations only when there's a change, not on each startup for example
>> Faster resolution (done only on top of metadata). At least it could help in the negative scenarios - fast failure when trying to install an artifact with missing dependencies
>> System is always in a determined state (explicitly described)
>>
>> Technical notes:
>>
>> 1. Virgo deployment entry points - AFAIK these are Virgo deployer and pickup folder:
>> Virgo deployer
>> There was an idea of replacement the relevant stages of deployer with p2-aware stages and to switch between them using configuration.
>> Some questions here:
>> Is the configuration supposed to be changed at runtime in order to drive the deployment mode? In a sense you deploy a bundle via p2 then change the configuration and the next artifacts you deploy are unknown to p2?
>> Or the configuration would let you pick a Virgo distribution with or without p2?
>> What's the benefit of switching between p2 and non p2 deployment? From my perspective it has a significant drawback in terms of consistency and performance so I'm curious to know the motivation?
>>
>> We have to pay special attention to the supportability features of the deployer quasi steps if this is moved or delegated to p2.
>>
>> A question about plans - what do you need ordering for? Is it something that people rely on and should stay as it is or Virgo can stop supporting that from a particular version on? It would be tricky to implement solution that offers ordering since we step on OSGi.
>>
>> Pickup
>> Is pickup functionality designed to be used productive? Or it's applicable only in development use cases?
>> It could easily be replaced with "dropins" functionality without any glue code if there's no productive usage.
>> Are there other supported ways to deploy something in Virgo? Equinox console for example?
>>
>> 2. Standard solution (p2 provisioning of subsystems)
>> The solution targets covering more general subsystems use cases rather than just Virgo specifics. Since subsystem RFC is not final and probably won't be in the specified time frame we will step on the currently available concepts and Virgo implementations of digraph, region manager, etc. Glyn is aware of how the things are going on in the spec space so I guess Virgo implementation will be aligned with the specification rather than introducing a custom alternative.
>> Given that, as soon as the spec is ready Virgo provisioning could be the first working example how subsystems could be provisioned with p2.
>>
>> 3. Design changes
>> p2
>> In order to keep the "quasi" functionality with minimum duplication of concerns p2 should support the option for "equinox resolution" validation on install. AFAIK there's such feature in p2 so this is not expected to be a significant effort (don't know whether it's implemented, the important here is that it fits to p2 vision) If that's not implemented maybe "quasi" steps could be contributed from Virgo to p2.
>>
>> Representation of Virgo artifacts in p2 - configuration files for both regions, plans, etc. Special attention to minimizing duplication of metadata (equinox state files, digraph, bundles.info, p2 profiles, Virgo custom configuration)
>>
>> Isolation between regions in p2 terms -the question whether that representation would affect p2 internal files and how is being discussed in p2-dev ML but it's still open and it's probably the hardest problem.
>>
>> Virgo
>> If the standard solution takes place there would be just configuration changes at runtime: System startup would be delegated to p2. Virgo custom configuration files could be replaced by p2 configuration files if they are not supposed to be used by the end user. (For example properties for launching kernel could be replaced by bundles.info while userregion.properties could stay as it is and has an appropriate representation in p2 bundles.info and profile files)
>>
>> The only major change is expected to be the way Virgo distributions are built.
>>
>> Here it's important to clarify the user perspective and whether they find dealing with "low-level" properties file useful.
>>
>> Risks
>> It's not clear to me how release cycles of both projects come together. What could be consumed from Virgo - only released stuff or a development build as well? Is there other time restrictions that come out of that?
>>
>> Isolation between regions could be harder than expected and there could be a potential conflict between Virgo and p2 requirements. Solving that could be tricky in the specified time frame so there should be a fallback option:
>> Virgo could implement p2 representations of its custom artifacts and build a solution based on currently available p2 means.
>> The drawbacks are:
>> we sacrifice the "standard" aspect
>> the p2 Virgo wouldn't be installed by p2 but would be distributed as zip (current state)
>> cannot cover "transport the user regions" requirement
>>
>> Execution plan
>> Next two weeks Bobby and I will be extremely focused on that topic.
>> Starting point - the prototype we have. It has two significant defects - modifying existing IUs which is unacceptable from p2 POV and requiring manual build steps.
>>
>> 1. Address the defects - not sure that on top of the assumptions I had I could bring that to success but it won't take much time to prove whether this could be done right.
>> 2. Quasi support
>> 3. Research and initial implementation of the requirement:"Provide a way to transport the user regions between virgo kernel instances" - it would result in an idea/requirements how to represent the layout of subsystems both in terms of p2 internal representation and on the file system.
>>
>> That's what I have in mind for now. Sharing use cases, comments, ideas, concerns are welcome.
>>
>> P.S. I'm subscribed for virgo-dev ML, no need to copy me. So is Pascal.
>>
>> Thank you,
>> Katya
>>
>>> -----Original Message-----
>>> From: Glyn Normington [mailto:gnormington@xxxxxxxxxx]
>>> Sent: Saturday, March 26, 2011 2:03 AM
>>> To: Dmitry Sklyut
>>> Cc: Virgo Project; Todorova, Katya; Pascal Rapicault
>>> Subject: Re: [virgo-dev] p2 & Virgo
>>>
>>> Hi Dmitry
>>> On 23 Mar 2011, at 08:50, Dmitry Sklyut wrote:
>>>
>>>> p2 kind off replaces
>>>> Quasi phases
>>>
>>> One thing we need to be careful about is that the p2 resolver
>>> does not take into account uses constraints. So it is
>>> possible for p2 to provision a collection of bundles that
>>> will not resolve. Of course this is unlikely in normal
>>> scenarios, but could happen e.g. as soon as multiple versions
>>> of a given bundle are installed in the user region.
>>>
>>> Glyn
>> _______________________________________________
>> virgo-dev mailing list
>> virgo-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/virgo-dev
>
> _______________________________________________
> virgo-dev mailing list
> virgo-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/virgo-dev
_______________________________________________
virgo-dev mailing list
virgo-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/virgo-dev
