| Re: [papyrus-rt-dev] Fwd: [cross-project-issues-dev] Disk usage report for HIPP/JIPPs |
+1On Thu, Jun 15, 2017 at 4:31 PM Christian Damus <give.a.damus@xxxxxxxxx> wrote:Correct. When the jobs running now finish, I plan to delete the shared repo /home/hudson/genie.papyrus-rt/ m2/repository. Not, of course, settings.xml and toolchains.xml.
Unless there are objections? I can’t imagine what purpose it would serve if the jobs are no longer using it.
Cheers,
Christian______________________________
On Jun 15, 2017, 16:29 -0400, Ernesto Posse <eposse@xxxxxxxxxxxxx>, wrote:
Wow! That's something. When I looked this morning it seemed to me that at least Gerrit-Codegen was already using a private repo. If all use it, then this .m2 won't get repopulated, right?
On Thu, Jun 15, 2017 at 4:12 PM Christian Damus <give.a.damus@xxxxxxxxx> wrote:FYI, the disk usage of /home/hudson/genie.papyrus-rt/.m2/ is about 93 GB. We should be able to delete the entirety of that now that all of the build jobs use a private repo. There is an unbelievable number of Papyrus and other bundle JARs in there.
Cheers,Christian______________________________
On Jun 15, 2017, 16:03 -0400, Christian Damus <give.a.damus@xxxxxxxxx>, wrote:
Indeed, several of our builds are configured to use the shared maven repository, including the Master-Product build. This build, in particular, has been subject to frequent apparently random failures. Perhaps this is the cause?In any case, I’m ensuring that all builds use a private repository. Then, I need to figure out how to destroy the Papyrus-RT genie’s (shared) maven repository. And then we can look into how best to periodically (hopefully automatically) trim each build’s private maven repository, following up Ernesto’s comments.
Cheers,Christian
On Jun 15, 2017, 03:14 -0400, Camille Letavernier <cletavernier@eclipsesource.com >, wrote:
Hi,Maven repos are not thread safe, so whenever 2 builds may run in parallel (and download new artifacts, e.g. Papyrus Nightly), a shared repo is not an option. On top of that, Eclipse plug-ins use a qualifier, whereas Maven works with "-SNAPSHOT". Since qualifiers change at every build, they keep stacking in the repository (until the repository is deleted). That's not an issue for plain-Maven, because Snapshots have the same name, so they just overwrite each other.Then you have to multiply this by the number of builds, and there's a tradeoff between build time and workspace size. In general, I like to preserve repos for stable builds, and wipe them for Gerrit (Which is much more likely to introduce incorrect bundle versions in the repo, thus potentially causing all later builds to fail). But at least every once in a while, Maven repos should be deleted.HTH,
Camille______________________________
On Wed, Jun 14, 2017 at 10:23 PM, Ernesto Posse <eposse@xxxxxxxxxxxxx> wrote:
We used to run some codegen tests that actually installed, but we haven't done that in ages. This is controlled by the "TEST_MODEL" variable in the Hudson build job. If it's empty, the test is not run. See releng/scripts/build-after.sh. --Ernesto
On Wed, Jun 14, 2017 at 4:05 PM Christian Damus <give.a.damus@xxxxxxxxx> wrote:These builds use private maven repositories. That means that maven artifacts accumulate in every job and are not shared by the jobs. The best reason I can think of for this would be if our builds do an install step, which I suppose might not be good for parallel builds. But they wouldn’t need to install, right? but then there’s the question of where a common repository is kept and how to maintain that.
Cheers,
Christian______________________________
On Jun 14, 2017, 15:59 -0400, Ernesto Posse <eposse@xxxxxxxxxxxxx>, wrote:
Right, it would make sense to keep the workspace for the Gerrit-* jobs for post-mortems. But do we need to keep them for the Master-* jobs too? These are also big. Master-Codegen is 20GB. Not sure why it's so much more than Gerrit-Codegen, though.--Ernesto
On Wed, Jun 14, 2017 at 3:53 PM Christian Damus <give.a.damus@xxxxxxxxx> wrote:Interesting. I thought the jobs were all configure to wipe the workspace on start of a build. At the very least that would prevent creeping size of the workspace.Thanks for the tip; I’ll look into it.I’d rather not wipe out the workspace after a troubled build because we do often need to look into the workspace to diagnose problems (esp. finding Eclipse platform logs from test execution). Maybe it can be wiped out only after a successful (green-ball) build?
Cheers,
Christian______________________________
On Jun 14, 2017, 15:50 -0400, Ernesto Posse <eposse@xxxxxxxxxxxxx>, wrote:
The workspace of each job is wiped out after the job finishes right? If not, we could have a "leak". I noticed that Gerrit-Codegen's workspace is about 250 MB, but Gerrit-Tooling is 20GB! Gerrit-Core is about 6G, and Gerrit-Profile is now over 4GB (although interestingly, it seems be be growing linearly).I agree that redirecting from download to HIPP is not nice. I think we "inherited" this approach from Papyrus itself.--Ernesto
On Wed, Jun 14, 2017 at 3:43 PM Christian Damus <give.a.damus@xxxxxxxxx> wrote:Hi, Ernesto,See bug 518265 for tracking this issue.Every one of our jobs keeps 30 builds, but only the artifacts of the last 3. I doubt we even need 3 builds of artifacts.The logs of the other 27 builds add up, to. I don’t think we need more than 5 logs to compare changes in builds, if even that.But even accounting for all this, I’m not sure how we are using more than 180 GB of space. The number of 10-12 MB builds that Hudson reports don’t add up to that. Is there some good way to track down where this space is actually being consumed on the HIPP? Do we have real access to the HIPP filesystem?Also, we might want to consider having the nightly Product job actually upload its artifacts to the download server (replacing the previous nightly) instead of keeping any artifacts, itself. The download server is designed to scale. This will also make the entire world’s access to our nightlies much more efficient; the current strategy of re-directing from download to the HIPP really should stop.
Cheers,ChristianWhy do we use so much space? Our Hudson jobs are configured to discard old builds, keeping a maximum of 30. Should we reduce this? What else could it be?
---------- Forwarded message ---------
From: Denis Roy <denis.roy@eclipse-foundation.org >
Date: Wed, Jun 14, 2017 at 3:26 PM
Subject: Re: [cross-project-issues-dev] Disk usage report for HIPP/JIPPs
To: <cross-project-issues-dev@eclipse.org >
Folks,
I fail to understand why these projects need >80G of workspace. For
Packaging, I can perhaps understand since the packages are huge. But for
others?
Can those projects PLEASE make sure HIPPs are not used to store
indefinite builds and artifacts?
Thank you for helping keep the house clean and tidy.
Denis
On 06/13/2017 01:30 PM, genie@xxxxxxxxxxx wrote:
> Compiled 2017-06-13-13:30:01
>
> Projects exceeding 80G of disk space on HIPP/JIPPs:
> ===================================================
> 486G /jobs/genie.packaging
> 245G /jobs/genie.capella
> 220G /jobs/genie.papyrus
> 206G /jobs/genie.platform
> 203G /jobs/genie.ice
> 194G /jobs/genie.papyrus-rt
> 159G /jobs/genie.app4mc
> 147G /jobs/genie.tracecompass
> 125G /jobs/genie.kitalpha
> 102G /jobs/genie.osee
> 100G /jobs/genie.sirius
> 100G /jobs/genie.ptp
> 84G /jobs/genie.ecp
> 83G /jobs/genie.webtools
> 82G /jobs/genie.emfcompare
> ===================================================
>
> If you find your project's name in the above list, please check
> your Hudson/Jenkins configuration and try to lower the disk usage.
>
> You can find more information about CI best practices and disk usage
> here: https://wiki.eclipse.org/CI_best_practices#Disk_usage .
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@eclipse.org
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/cross- project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross- project-issues-dev --Ernesto Posse
Zeligsoft_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx______________________________
To change your delivery options, retrieve your password, or unsubscribe from this list, visit_________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev --______________________________Ernesto Posse
Zeligsoft_________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev _________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev --______________________________Ernesto Posse
Zeligsoft_________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev _________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev --Ernesto Posse
Zeligsoft
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev
--
Camille Letavernier
Senior Software Engineer
EclipseSource Paris
Email: cletavernier@xxxxxxxxxxxxxxxxx
Web: http://eclipsesource.com/paris
Phone: +33 1 85 41 09 21
Fax: +49 89 21 555 30 - 19
Palaiseau-Entreprises
7 rue de la Croix Martre
91873 Palaiseau
General Manager: Rémi Schnekenburger_________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev _________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev --______________________________Ernesto Posse
Zeligsoft_________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev _________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev
--Ernesto PosseZeligsoft
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt- dev