Hi, Ernesto,
See replies in-line.
On 22 September, 2016 at 11:44:38, Ernesto Posse (eposse@xxxxxxxxxxxxx) wrote: Thanks Christian. Some follow up question inline below. Hi, Ernesto,
See some replies in-line, below.
On 21 September, 2016 at 17:23:00, Ernesto Posse (eposse@xxxxxxxxxxxxx) wrote: Hi. I finally had time to read the description of the build infrastructure at https://wiki.eclipse.org/Papyrus-RT/Developer_Guide/Builds, and I have a few questions:
First an RCP/Product newbie question here: if I wanted to package Papyrus-RT as a zip file to distribute, how would I do it? Are the "rcp" and/or "product" projects supposed to provide this? If not, do we have plans to do this?
Yes, the Papyrus-RT-<Stream>-Product build should package the RCP archive when the time comes to define its make-up.
Ok, so it is currently not doing that?
Looking in the console log, it appears that this build job is building RCP packages: [INFO] Building zip: /jobs/genie.papyrus-rt/Papyrus-RT-Master-Product/workspace/source/releng/org.eclipse.papyrusrt.product/target/products/org.eclipse.papyrusrt.product-win32.win32.x86_64.zip
[INFO] Building zip: /jobs/genie.papyrus-rt/Papyrus-RT-Master-Product/workspace/source/releng/org.eclipse.papyrusrt.product/target/products/org.eclipse.papyrusrt.product-linux.gtk.x86_64.zip
[INFO] Building tar: /jobs/genie.papyrus-rt/Papyrus-RT-Master-Product/workspace/source/releng/org.eclipse.papyrusrt.product/target/products/org.eclipse.papyrusrt.product-macosx.cocoa.x86_64.tar.gz
but I wasn’t aware of that until I did more digging because these archives don’t appear in the build’s artifacts, which is where I would expect to find them.
What is the difference then between the oeprt.product and the oeprt.rcp plugins?
I assume that the *.rcp project defines the content of the RCP archives referenced above, but I don't know the purpose of these plug-ins. I would have just had the *.product project build the p2 repository and the RCPs, but probably there’s a good reason for the distinction of these two projects that I just don’t know.
Also, I'm preparing the tutorial for MODELS, which is supposed to be a hands-on tutorial, but using the Oomph setup(s) seems impractical because it takes a certain amount of time, and it crashed whenever a dependency repo is unavailable, so we thought that the best thing would be to package the whole thing in a zip and pass around some USB keys so that people could just unzip it and get the full thing ready to go. Is this something that the RCP will do? How hard is it to make it? I figure that if the packaging is not ready yet, I could just do a couple of installations for Linux, Windows and OS X and zip those, but if the build process can do that, it might be preferable. Under "Integration Jobs" it says:
* Papyrus-RT-<Stream>-Product-Releases doesn publish and archive
Is that a "doesn't" or a "does"?
That should be “doesn’t”. I’ll fix the wiki. The point is that we don’t want to automatically publish a release candidate build to the Eclipse download server until it has been thoroughly tested. For integration and nightly builds, the bar is much lower.
But is should package it so that publishing and archiving would be easier, right?
My suggestion for these the *-Milestone and *-Release jobs was that they would just promote the output of the last nightly or integration build as the milestone or release candidate. So, they would not actually have anything to “build” or to “package” because they would just be re-labelling and/or shuffling around artifacts that were already built before.
Having either of these builds actually build everything from scratch would mean extra testing effort where we probably would have had binaries available already that had undergone some testing.
Under "Target Platforms", should the -Product path be updated to the updates/nighly/<Stream>?
It could, but that would be a trip to download.eclipse.org just to be redirected back to the location listed in the wiki.
Ah, so it doesn't really matter, eh?
Nope. Maybe one prefers a more abstract URL when there’s nothing more practical to recommend one or the other? I don't know.
Under "Triggering", it seems to me that codegen is a bit of a special case, because changes to plugins/xtumlrt/... should trigger a codegen build as well, or is this supposed to be handled by triggering the codegen build from a (not yet existing) xtumlrt build?
At this point, the xtumlrt component is lumped together with codegen.
Well, it is lumped together in the Papyrus-RT-CodegenRTS-BuildFrom-gerrit-Master job, but not in the new Papyrus-RT-Gerrit-Codegen job which only looks at the codegen folders.
Oh, that’s a problem, then. A bug in the build system. The Papyrus-RT-Master-Codegen (not gerrit) job looks okay to me.
To get this new architecture deployed long enough ahead of our release window, it was agreed to continue with a fairly coarse-grained componentization for now.
So for now at least shouldn't we add the xtumlrt folders to the triggers of the new Papyrus-RT-Gerrit-Codegen job? (as well as the runtime folders)?
Yup. But the maven profile that determines what the codegen Gerrit job builds also needs to be fixed in the top POM, because it may be triggered on changes in xtumlrt but it won’t build them otherwise!
When there’s a practical motive for splitting out an xtumlrt component, then yes, its build would trigger a downstream codegen build. I’ll add a section in the wiki to explain the currently reduced componentization.
In fact, shouldn't there be triggering dependencies for the Gerrit jobs according to the components? E.g., shouldn't a gerrit build of "Core" trigger a build of "Tooling", "Codegen", etc?
No, Gerrit jobs can’t really trigger one another. If the Core build is triggered for some commit A and then this cascades a few levels to other components, then in the mean-time another commit B could trigger a new Core build and the build from commit A will no longer be current. Besides, we don’t want to incur the cost of archiving the outputs of these builds. However, each Gerrit build does run all of the tests for its dependents, to verify that the changes don’t break those components. I suppose we may find that this isn’t sufficient, if our test coverage isn’t good enough. This also helps to enforce component API boundaries: there should be no need to rebuild Tooling for a change in Core because that change in Core must maintain the APIs that Tooling depends on. For the extraordinary cases where we do have to break API, then we’ll have nightly builds that are broken until all components catch up, which shouldn’t take long because the developer will have Gerrits for all components prepared. If this doesn’t pan out as we hope, then of course we’ll have to re-think the strategy.
I want to make sure I understand, so let's make it a bit more concrete. Suppose that I have a bug that requires modifying some construct in the language, say, a change in the way exclusions are to be specified. This entails changes to the xtumlrt meta-model, possibly the xtext projects and the xtumlrt utilities and APIs and to the code generator. If we have an xtumlrt Gerrit job separate from the codegen Gerrit job, then can I still make all the required updates in one commit rather than splitting the changes in one commit for each component? If everything is done in one commit, both jobs would be triggered (since they listen to their respective folders), but would the codegen job get the xtumlrt updates? Just to be sure, the maven build builds on the full commit checked out from git which will contain all the changes and not just the ones in the folders that are being listened, is that right?
This would have to be done in two commits: one for xtumlrt and one for codegen. So, it could flow like this: - Make a Gerrit patch for the xtumlrt changes first.
- Gerrit builds xtumlrt with your changes. Submit the Gerrit.
- Now the main build for the xtumlrt component builds it and triggers a codegen build.
- If there are incompatible API changes in the xtumlrt API, the codegen build will break.
- Now make a Gerrit patch for the corresponding codegen changes.
- Gerrit builds codegen with your changes. Submit the Gerrit.
- Now the main build for the codegen component builds it and succeeds.
If you tried to do this with one commit, then your Gerrit change set would require both the xtumlrt and codegen components to build successfully, but they can’t because the codegen build would not see the xtumlrt changes (it pulls the latest nightly build of xtumlrt). So it does have to be staged in two commits.
Of course, if you can evolve the APIs in a compatible way (e.g., with deprecations if necessary), then it’s easier: you can do it with one commit. And there won’t be an interval in which the codegen build is broken after merging xtumlrt changes.
But, come to think of it, since in this scenario at step 2 the xtumlrt Gerrit build actually runs the codegen JUnit tests also, they will probably fail and so reject the change. This would need the committer to use his judgment to override the Gerrit vote: just remove Hudson CI as a reviewer and +1 it yourself. If it turns out that committers cannot remove Hudson CI from a review, then we would have to ask the project lead or webmaster to fix that.
|
