Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [papyrus-rt-dev] Concurrent Hudson builds

Thanks for the responses.

The problem of triggering the Product build on Tooling-only changes is, of course, an orthogonal one.  The Hudson Join Plug-in that we are using to avoid redundant builds of the Product when components higher up the dependency lattice (Core, Profile) are built has this unfortunate limitation.  To address it, we would have to look for an update of this plug-in that fixes the problem, look for a better plug-in, create our own plug-in (!), or else look for another build dependency strategy.

For the disk usage problem, to avoid going back to a shared repository (I prefer not to, not just because of the concerns raised by Camille) I did prepare an implementation of regular private maven repository clean-up:

https://git.eclipse.org/r/#/c/99537/

Review and feed-back are welcome.  It seems to work (I tested it in a feature branch build job and, of course, in the Releng Gerrit job) but perhaps there is a better way to achieve the same result, that I have not seen.

Thanks,

Christian

On Jun 19, 2017, 04:24 -0400, Camille Letavernier <cletavernier@xxxxxxxxxxxxxxxxx>, wrote:
Hi,

You need to be careful with shared repositories, as sometimes, Tycho will resolve artifacts in the repo (If they can't be found in the current reactor). This can lead to false positive or false negative builds, especially when dependencies are loosely specified and you use the same shared repo for, say, Neon and Oxygen (It's less likely to be an issue in a pure-Oxygen repo).

(Note that Tycho behavior may have changed since I observed this. Especially, we didn't use target platforms at this time - maybe it's a little bit more strict, now that we do, as we should have less "lazy" resolution - hopefully)

Camille

On Mon, Jun 19, 2017 at 10:17 AM, Peter Cigéhn <peter.cigehn@xxxxxxxxx> wrote:
Hi,

I really don't have any opinion (or insight) regarding we need concurrent builds or not. But something needs to be done with the build structure. The major flaw that whenever a change only impacts tooling, then the final product build is never triggered. So you always have to manually trigger the product build (or wait for something else to trigger the product build). So, whatever is done with build structure, I really think that this issue needs to be resolved as well.

/Peter Cigéhn

On 17 June 2017 at 13:21, Christian Damus <give.a.damus@xxxxxxxxx> wrote:
Hi, Team,

In looking at the Disk Usage Report in our Hudson instance, it occurred to me that the report only counts the size of artifact and workspace storage for every build job. The shared Maven repository is not included in the report, nor anything else in the genie user's home directory.

We can easily maintain clean and evergreen workspaces if every build starts by wiping the workspace (a standard Hudson build option) instead of just doing a git clean. We can also try to get clever by deleting the builds' private Maven repositories on some interval so that not every build has to download all of the Tycho tooling and dependencies.

But the private Maven repositories are only necessary because we have concurrent builds on three executors. If we had only a single executor, then all of the builds could use the shared Maven repository for maximal space efficiency and that could easily be purged from time to time by a scheduled build job. They would also save some time by not having to re-download every dependency.

Do we really need concurrent builds?

Cheers,

Christian

_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev



_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev




--
Camille Letavernier

Senior Software Engineer
EclipseSource Paris

Email: cletavernier@xxxxxxxxxxxxxxxxx
Web: http://eclipsesource.com/paris
Phone: +33 1 85 41 09 21
Fax: +49 89 21 555 30 - 19

Palaiseau-Entreprises
7 rue de la Croix Martre
91873 Palaiseau

General Manager: Rémi Schnekenburger
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev

Back to the top