|
Thanks, Céline.
The options that you outline are clear. I guess we’re really fighting the Hudson architecture on this one, eh? Our scenario strikes me as so perfectly common that I’m really surprised it isn’t better accounted for by the available tools. Is there something
we’ve overlooked? Or maybe what we’re trying to do with component builds isn’t so common, and that for good reason?
I honestly don’t know which of these options I would recommend over the others. All three have characteristics that appeal to me.
In any case, I think that perhaps we should put any plans to further decompose the builds on ice until we get a conclusion to this question.
cW
On 11 October, 2016 at 11:30:15, Céline JANSSENS (celine.janssens@xxxxxxxxxxx) wrote:
Hi Christian,
No it won’t trigger the Product if only Tooling is impacted.
This is the regression of the join method.
If I keep the cascading from Tooling and from Codegen to Product, in addition with the Join trigger, Product will be triggered 3 times ;s
So there are 3 solutions with their inconvenience:
1)
We keep it as previously => Product triggered twice if Core is impacted, once if Tooling or Codegen, and all combinations if changes impact several
components.
2)
We keep the join and trigger the Core on every modification => 3 Components are built even if not necessary but only once
3)
We cascading Core > Tooling > Codegen > Product => If only Tooling is impacted, this configuration builds Codegen even if not necessary and all
combinations if changes impact several components.
So if in most of the case we modify only one component by Change or every 15 minutes, the 3d solution is the more efficient.
Otherwise, if there are several components impacted most of time, the second solution is the one we should put in place.
I don’t know a solution meeting 100% the requirements.
Regards
Céline
This looks very cool, thanks!
One question: I see that all of the configuration is now on the Core job, with the new join trigger. Will the Product build still be run if only,
for example, the Tooling build is triggered by a commit that makes only tooling changes? Or will Product now only build after both Codegen and Tooling builds are triggered by a Core build (from a commit that changes core projects)?
On 11 October, 2016 at 10:48:03, Céline JANSSENS (celine.janssens@xxxxxxxxxxx) wrote:
Hi All,
The Join trigger plugin being in place, I configured the different build to avoid to trigger Papyrus-RT-Master-Product
twice.
Papyrus-RT-Master-Product is now triggered when both Tooling and Codegen are complete
More information about the Join trigger plugin installed on the HIPP available
here [1]
Let me know if you meet any issues.
Regards
Céline
[1] :
https://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin
Hi team,
Yes, I am (finally!) back from vacation.
I installed the “join” plugin, as this seems to be the most suitable plugin for our needs. I also installed a utility
to manage job configs (a kind of version control management for job configurations)
Hudson will be restarted when current jobs are finished, so these plugins will be activated.
Regards,
Rémi
-------------------------------------------------------
Rémi SCHNEKENBURGER
+33 (0)1 69 08 48 48
CEA Saclay Nano-INNOV
Institut CARNOT CEA LIST
www.eclipse.org/papyrus
Hello Chrsitian,
That is Remi that has such access rights, for the HIPP.
He is back from his vacations, so do not hesitate to see with him ;)
Your solution a) seems a good point. I didn’t know that it exists. Well done.
Regards
Céline
Yes, it’s a problem that we have the downstream Product build running once for each completion of its multiple upstreams.
There are plug-ins that can help this. Hudson Plugin Central has at least two that address this situation:
·
join: triggers a job after a group of other jobs finish. It’s the simplest implementation, and I think would be suitable
for our case (I have seen reports that it doesn’t work well when parallel job chains to be joined are more than 1 job long)
·
build-pipeline-plugin: a visual configuration of complex pipelines of jobs
I don’t have admin privilege on our HIPP to see which, if any, of these plug-ins might already be available for us to use. Nor do I know the
process for requesting to install new plug-ins, if that’s a thing that we can do for our own HIPP should we decide to do so.
And, yes, a change in core should not trigger both the Core and Tooling builds but Core only, because it will kick off Tooling as a downstream
anyways. That much is easy to address; I can take care of that.
Christian
On 4 October, 2016 at 12:04:34, Peter Cigéhn (peter.cigehn@xxxxxxxxx) wrote:
I am not sure I understand the proposed change. Currently the product build is triggered by the component builds, core, tooling and so on. I assumed
that the reason for the product build to be triggered by the component builds was due to that the product build used the output from the component builds in some way.
But now you are saying that the product build can be triggered "on its own", which I interpret as if it does not really depend on the component builds
have been run at all. Is that correct? Then what is the purpose of the individual component builds? I am bit confused here...
And when you say each 30 minutes, I assume that you mean that the product build should check if there has been an SCM change, i.e. a commit being
merged, each 30 minutes. But why should that be check only be made each 30 minutes? That I assume could be checked each 15 minutes as for the component builds. Otherwise you can theoretically have to wait for 30 minutes before the build is triggered (in case
the merge is committed right after the last SCM poll) and then the build that takes 17-20 minutes, i.e. up to 50 minutes.
Sure, that is much better than the current situation where you need to wait up to 15 minutes for the SCM poll on the component builds, then build core and tooling in parallell (I assume a few minutes), then build product for up to 24 minutes, in parallel with
the second tooling build,, and then the second product build (triggered by the second tooling build) for 24 minutes more, which adds up to more than an hour. Still not sure if I can use the first product build or if I need to wait for the second one.
I thought that the triggering of builds could be made a bit more intelligent, i.e. if it is concluded that a commit impacts both core and tooling,
then skip triggering tooling and only trigger core, since you know that core in its turn will trigger the build of tooling. That would avoid the double builds of tooling and product.
On 4 October 2016 at 17:45, Céline JANSSENS <celine.janssens@xxxxxxxxxxx> wrote:
Hi Peter,
As far as I remember, The Core build triggers Tooling and Codegen jobs. Codegen and Tooling jobs both trigger the Product
job.
So you are right, every time a modification is done on the core, then tooling and codegen jobs are triggered. And both
trigger the Product job.
It has been whished that the Product should be up to date at least every 15 minutes. That is why, Core, Tooling and Codegen
Build check for modification every 15 minutes.
Product job is based on Core, Tooling and Codegen sub-repos as it builds the “releng” Target platform.
A solution would be not to trigger the product directly from the Components builds, but on a regular basis, 30 minutes
seem reasonable.
In addition, the RCPTT is being extract from the Product Build to decrease the time of the Product Build to 17-20 minutes.
Any remarks on this suggestion ?
Regards
Céline
De :
papyrus-rt-dev-bounces@xxxxxxxxxxx [mailto:papyrus-rt-dev-bounces@xxxxxxxxxxx]
De la part de Peter Cigéhn
Envoyé : mardi 4 octobre 2016 17:19
À : papyrus-rt developer discussions <papyrus-rt-dev@xxxxxxxxxxx>
Objet : [papyrus-rt-dev] Multiple Papyrus-RT-Master-Product builds for each merged commit
I have a question regarding the new build system. I find a bit confusing knowing when it is safe to start upgrading and testing a merged commit. Almost all the latest
commits that I have been merged and which I wanted to test, and for which I then I have checked the builds when they are finished, have been triggering the final Papyrus-RT-Master-Product to be built (at least) twice.
I guess that it is related to that a commit contains changes, e.g. for both core and tooling, which triggers both these builds (in parallell). When these are finished,
they trigger the final product build. And then when the core is finished that triggers yet another tooling build, which in its turn triggers a product build again. In more or less all cases I have seen, the second product build has gotten stuck in the queue
also since you cannot build two product builds in parallell.
Is it safe to test already after the first product build have finished, or do I always have to wait for the second one to finish? More or less always having to wait
for two product builds feels pretty time consuming (since each takes up to half an hour to complete).
Is the setup of the build system really correct? Why is both core and tooling triggered at the first step? If you only trigger building core, which then triggers tooling,
which finally triggers product, then you don't have tooling and product building twice.
Any comments? Is this just how it is supposed to work? And that I (to be on the safe side) always needs to wait for the second (or whatever number the final will be)
build before I can upgrade safely?
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
|
