| Re: [papyrus-rt-dev] Fwd: [cross-project-issues-dev] Disk usage report for HIPP/JIPPs |
Right, it would make sense to keep the workspace for the Gerrit-* jobs for post-mortems. But do we need to keep them for the Master-* jobs too? These are also big. Master-Codegen is 20GB. Not sure why it's so much more than Gerrit-Codegen, though.--Ernesto
On Wed, Jun 14, 2017 at 3:53 PM Christian Damus <give.a.damus@xxxxxxxxx> wrote:Interesting. I thought the jobs were all configure to wipe the workspace on start of a build. At the very least that would prevent creeping size of the workspace.Thanks for the tip; I’ll look into it.I’d rather not wipe out the workspace after a troubled build because we do often need to look into the workspace to diagnose problems (esp. finding Eclipse platform logs from test execution). Maybe it can be wiped out only after a successful (green-ball) build?
Cheers,
Christian_______________________________________________
On Jun 14, 2017, 15:50 -0400, Ernesto Posse <eposse@xxxxxxxxxxxxx>, wrote:
The workspace of each job is wiped out after the job finishes right? If not, we could have a "leak". I noticed that Gerrit-Codegen's workspace is about 250 MB, but Gerrit-Tooling is 20GB! Gerrit-Core is about 6G, and Gerrit-Profile is now over 4GB (although interestingly, it seems be be growing linearly).I agree that redirecting from download to HIPP is not nice. I think we "inherited" this approach from Papyrus itself.--Ernesto
On Wed, Jun 14, 2017 at 3:43 PM Christian Damus <give.a.damus@xxxxxxxxx> wrote:Hi, Ernesto,See bug 518265 for tracking this issue.Every one of our jobs keeps 30 builds, but only the artifacts of the last 3. I doubt we even need 3 builds of artifacts.The logs of the other 27 builds add up, to. I don’t think we need more than 5 logs to compare changes in builds, if even that.But even accounting for all this, I’m not sure how we are using more than 180 GB of space. The number of 10-12 MB builds that Hudson reports don’t add up to that. Is there some good way to track down where this space is actually being consumed on the HIPP? Do we have real access to the HIPP filesystem?Also, we might want to consider having the nightly Product job actually upload its artifacts to the download server (replacing the previous nightly) instead of keeping any artifacts, itself. The download server is designed to scale. This will also make the entire world’s access to our nightlies much more efficient; the current strategy of re-directing from download to the HIPP really should stop.
Cheers,ChristianWhy do we use so much space? Our Hudson jobs are configured to discard old builds, keeping a maximum of 30. Should we reduce this? What else could it be?
---------- Forwarded message ---------
From: Denis Roy <denis.roy@xxxxxxxxxxxxxxxxxxxxxx>
Date: Wed, Jun 14, 2017 at 3:26 PM
Subject: Re: [cross-project-issues-dev] Disk usage report for HIPP/JIPPs
To: <cross-project-issues-dev@xxxxxxxxxxx>
Folks,
I fail to understand why these projects need >80G of workspace. For
Packaging, I can perhaps understand since the packages are huge. But for
others?
Can those projects PLEASE make sure HIPPs are not used to store
indefinite builds and artifacts?
Thank you for helping keep the house clean and tidy.
Denis
On 06/13/2017 01:30 PM, genie@xxxxxxxxxxx wrote:
> Compiled 2017-06-13-13:30:01
>
> Projects exceeding 80G of disk space on HIPP/JIPPs:
> ===================================================
> 486G /jobs/genie.packaging
> 245G /jobs/genie.capella
> 220G /jobs/genie.papyrus
> 206G /jobs/genie.platform
> 203G /jobs/genie.ice
> 194G /jobs/genie.papyrus-rt
> 159G /jobs/genie.app4mc
> 147G /jobs/genie.tracecompass
> 125G /jobs/genie.kitalpha
> 102G /jobs/genie.osee
> 100G /jobs/genie.sirius
> 100G /jobs/genie.ptp
> 84G /jobs/genie.ecp
> 83G /jobs/genie.webtools
> 82G /jobs/genie.emfcompare
> ===================================================
>
> If you find your project's name in the above list, please check
> your Hudson/Jenkins configuration and try to lower the disk usage.
>
> You can find more information about CI best practices and disk usage
> here: https://wiki.eclipse.org/CI_best_practices#Disk_usage.
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev--Ernesto Posse
Zeligsoft_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx_______________________________________________
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev--_______________________________________________Ernesto Posse
Zeligsoft
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev--_______________________________________________Ernesto Posse
Zeligsoft
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev