[epsilon-dev] FW: [eclipse-mirrors] Eclipse mirror size~1.4 TB

[Sina Madani <sinadoom@xxxxxxxxxxxxxx>]

Hi everyone,

 

I was wondering if any of you know where Epsilon’s JARs are stored in download.eclipse.org (so we can investigate – see attached email) and how we would go about cleaning up / archiving old releases? I would imagine there only needs to be a single interim release at a time and releases older than 1.5.x should be archived. 9.5 GB seems odd, perhaps every single build (as triggered from Jenkins) is being kept around?

 

Thanks,

Sina

 

From: Denis Roy
Sent: 15 April 2019 15:41
To: eclipse.org-committers@xxxxxxxxxxx
Subject: [eclipse.org-committers] Fwd: [eclipse-mirrors] Eclipse mirror size~1.4 TB

 

Folks,

The disk footprint of mirrored content on download.eclipse.org is up to 1.4TB. We need to do a better job (much better) of keeping it small.  I've filed https://bugs.eclipse.org/bugs/show_bug.cgi?id=546434 to track this.

 

download.eclipse.org should only be used to store current release builds. Archived builds should be moved to archive.eclipse.org, and nightly and integration builds should be deleted once stale.

See the docs for more info:

https://wiki.eclipse.org/IT_Infrastructure_Doc#Downloads

Below is a table of projects that seem to be consuming more disk space than I'd expect.

_ALL PROJECTS SHOULD CLEAN THEIR AREA, NOT JUST THOSE LISTED BELOW_

14G    4diac  

6.6G   acceleo 

15G    app4mc 

57G    birt   

11G    dirigible     

20G    e4     e4/sdk/drops 2011 -> move to archive.eclipse.org

72G    eclipsescada   

12G    ecp    

9.7G   efxclipse     

9.5G   epsilon 

16G    gemoc  

16G    ice    

25G    jdtls  

9.8M   keyple 

13G    kura   

6.5G   mat    

4.8G   mmt    

160G   modeling      

37G    emf    

36G    gmp    

72G    mdt    

8.0G   tmf    

14G    n4js   

41G    orion  

98G    rcptt  

76G    eclipselink   

38G    sirius 

18G    staging 

398G   epp    

71G    cdt    

110G   orbit  

41G    ptp    

36G    tracecompass   

40G    webtools      Lots of old stuff that can be moved

-------- Forwarded Message --------

Subject:	[eclipse-mirrors] Eclipse mirror size ~1.4 TB
Date:	Sat, 13 Apr 2019 19:29:29 +0100
From:	Gökdeniz Karadağ <gokdenizk@xxxxxxxxx>
Reply-To:	Eclipse download mirror administrators discussion and announcements <eclipse-mirrors@xxxxxxxxxxx>
To:	eclipse-mirrors@xxxxxxxxxxx

 

Hi Eclipse Mirror team,

 

Currently eclipse mirror is ~1.4 TB, surpassing both debian and ubuntu in size.

 

Is it possible to create a more compact rsync module containing most requested files?

 

For example OpenSuse has "hotstuff" modules for mirrors who want to provide value while conserving space and downstream sync bandwidth. (i.e. I don't like to download 375 versions of a file -over the life time of the mirror- for that single person over there using that project, they can go to the main mirror for that file)

Here's their doc on this: https://en.opensuse.org/openSUSE:Mirror_infrastructure#rsync_modules

--

Gokdeniz Karadag