[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [eclipse-mirrors] Size of a full Eclipse mirror?
|
Hello every body,
A couple of weeks ago we hit that problem too...
After discussing with eclipse-webmaster, it was suggest to do a clean first
and let it run even when there are errors :
<snip>
# path to your RSYNC binary
# RSYNC=/usr/bin/rsync
# Adapted the RSYNC definition based on the advise from the administrators
# mailing list
# RSYNC="/usr/bin/rsync --delete-excluded --exclude=drops/I*
--exclude=drops/N* "i
# MODIFIED based on recommendation of eclipse server admin
RSYNC="/usr/bin/rsync --delete-excluded --exclude=drops/N*
--exclude=drops/I* --exclude=nightly/* --exclude=drops/\d+.\d+.\d+/N*
--delete --ignore-errors "
</snip>
What was detected :
_ rsync failed during clean up --> clean up was not completed successfully
(continued to grow)
_ Right before the latest release there were many many builds produced => a
lot of changes on the master server generating waves of updates (making the
process less stable).
_ deletion & updates generated jobs that were extremely long so putting the
"--ignore-errors" allowed to have each run to be as best as possible
I dunno why I put 3 points while there is in fact 2 major problems ...
I hope this helps ...
\T,
On Fri, 02 Jul 2010 16:05:14 +0200, Gábriel Ákos
<akos.gabriel@xxxxxxxxxx> wrote:
> On Fri, 2010-07-02 at 12:52 +0200, Markus Knauer wrote:
>> If you find /technology/epp/downloads/release/helios/Mxxx or RCxxx
>> something went wrong with the sync. Weeks ago I did a clean up and
>> moved everything that can be moved to the Eclipse archive server.
>> Today I moved even more to the archive server, so from *tomorrow* on
>> you should only see
>>
>> /technology/epp/downloads/release/galileo/SR2 (9 GBytes) and
>> /technology/epp/downloads/release/helios/R (10 GBytes)
>>
>> If there are still other directories
>> below /technology/epp/downloads/release/ we need to investigate *why*
>> those old directories are not removed.
>
> I am using UK mirrorservice as base.
> Now I've removed the old releases by hand (47GB freed), we'll see what
> happens (if they come back or not).
>
> Best,
> Akos
>
>
>>
>> Thanks and regards, Markus
>>
>>
>> On 2 July 2010 12:39, Gábriel Ákos <akos.gabriel@xxxxxxxxxx> wrote:
>> Hi,
>>
>> Thanks for the info, I have exactly the same problem but I
>> definitely
>> don't have that amount of space, so I will give up on that.
>>
>> I tried to leave "technology" out of mirrors, that worked,
>> limited the
>> space needed somewhere around 200GB, but suddenly I discovered
>> that this
>> way I got only the classic eclipse distribution mirrored.
>> As I switched on the "technology" mirror it started
>> downloading all
>> other distributions (all versions of them :( ).
>>
>> Look at /technology/epp/downloads/release/ ...
>> There is helios/M5-M7 RC1-4 and R.
>> I suppose you'd need only R like me.
>>
>> For me it looks like the site was restructured and the mirror
>> script was
>> not adapted, this is the problem. It'd be nice if Dennis would
>> have a
>> statement how the problem will be solved. Either the structure
>> has to be
>> reverted or the script should be adapted.
>>
>> Now I'm thinking about adapting the mirror script. Temporarily
>> it would
>> serve the purpose but my problem is what if the site is
>> restructured
>> again? That's why I'd wait for Dennis' reply.
>>
>> Best,
>> Akos
>>
>>
>>
>>
>> On Fri, 2010-07-02 at 12:58 +0300, Mikko Jaaskelainen wrote:
>> > Hi,
>> >
>> > We're setting up a full Eclipse mirror. We have been
>> rsyncing for quite some
>> > time and so far we have downloaded 432 GB of data and it
>> seems that there is
>> > no end - we are worried that we will run out of disk space
>> before the initial
>> > rsync completes.
>> >
>> > How much data does a full Eclipse mirror currently have? I
>> read somewhere
>> > that it should be around 350 GB, but we're way beyond that.
>> We have used
>> > eclipse.org-rsync.sh script manually for the download as
>> documented.
>> >
>> > Any advice or suggestions?
>> >
>> > Thanks in advance.
>> >
>> > Regards,
>> >
>> > Mikko Jaaskelainen
>> > Nordnet
>> > _______________________________________________
>> > eclipse-mirrors mailing list
>> > eclipse-mirrors@xxxxxxxxxxx
>> > https://dev.eclipse.org/mailman/listinfo/eclipse-mirrors
>> >
>>
>>
>>
>> --
>> Üdvözlettel,
>> Gábriel Ákos
>>
>>
>> _______________________________________________
>> eclipse-mirrors mailing list
>> eclipse-mirrors@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/eclipse-mirrors
>>
>>
>>
>> _______________________________________________
>> eclipse-mirrors mailing list
>> eclipse-mirrors@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/eclipse-mirrors
>
>
>
--
--
T. : +32 (0)2 742 05 94
M. : +32 (0)497 44 68 12
@ : tsmets@xxxxxxxxxxxx
Do you skype too ... I am tsmets
Get Skype and call me for free ?
-----------------------------------------
This e-mail was not scanned for VIRUS
I am on Mac
-----------------------------------------
