User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0
Jonah,
It's okay that it's relatively soon after. I'll regenerate the
catalog tomorrow. Release day is stressful. I really appreciate
the emails you sent out earlier today in advance of the release
hour. That really helped us, with Denis' help, avert any problem
visible to the public.
Thanks,
Ed
On 18.03.2020 15:08, Jonah Graham
wrote:
Thanks Ed,
My checklist says to create technology/epp/packages/2020-06
immediately after release, so I am in the midst of doing that
now. Would you prefer it is created earlier for the next
release? If so, when? I guess at the same time that
releases/2020-06 is created? @Fred, when is that?
I've been able to install the C/C++ IDE with the released
installer and the current repositories. Mirrors are in
place for both the train and EPP repos.
I am looking at this now to see if I can
understand. Denis/Webmaster if you can help
that would be great as I don't understand how
different clients are getting different
responses.
$curl -I http://download.eclipse.org/technology/epp/packages/2020-03/compositeContent.jar
% Total % Received % Xferd Average
Speed Time Time Time Current
Dload
Upload Total Spent Left Speed
0 429 0 0 0 0
0 0 --:--:-- --:--:-- --:--:--
0HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Mar 2020 13:06:13 GMT
Content-Type: application/x-java-archive
Content-Length: 429
Connection: keep-alive
Last-Modified: Fri, 06 Mar 2020 14:22:22
GMT
ETag: "1ad-5a03061c70fb5"
X-NodeID: download2
X-Proxy-Cache: STALE
Accept-Ranges: bytes
When I do this it bypasses the cache:
$curl -I --header "X-Cache-Bypass: true"
http://download.eclipse.org/technology/epp/packages/2020-03/compositeContent.jar
% Total % Received % Xferd Average
Speed Time Time Time Current
Dload
Upload Total Spent Left Speed
0 0 0 0 0 0
0 0 --:--:-- --:--:-- --:--:--
0HTTP/1.1 404 Not Found
Server: nginx
Date: Wed, 18 Mar 2020 13:09:15 GMT
Content-Type: text/html
Connection: keep-alive
Vary: Accept-Encoding
X-NodeID: download2
But a subsequent call still returns it as
existing so it does not flush the cache:
$curl -I http://download.eclipse.org/technology/epp/packages/2020-03/compositeContent.jar
% Total % Received % Xferd Average
Speed Time Time Time Current
Dload
Upload Total Spent Left Speed
0 429 0 0 0 0
0 0 --:--:-- --:--:-- --:--:--
0HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Mar 2020 13:09:21 GMT
Content-Type: application/x-java-archive
Content-Length: 429
Connection: keep-alive
Last-Modified: Fri, 06 Mar 2020 14:22:22
GMT
ETag: "1ad-5a03061c70fb5"
X-NodeID: download2
X-Proxy-Cache: STALE
Accept-Ranges: bytes
Both compositeContent.jar and
compositeArtifacts.jar need to yield 404
for a HEAD request...
Note how it says "X-Proxy-Cache: STALE"
in each case.
Several calls like this:
$curl -I http://download.eclipse.org/technology/epp/packages/2020-03/content.jar
% Total % Received % Xferd Average
Speed Time Time Time Current
Dload
Upload Total Spent Left Speed
0 77058 0 0 0 0
0 0 --:--:-- --:--:-- --:--:--
0HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Mar 2020 13:15:15 GMT
Content-Type: application/x-java-archive
Content-Length: 77058
Connection: keep-alive
Last-Modified: Tue, 17 Mar 2020 11:19:04
GMT
ETag: "12d02-5a10b1a79ee91"
X-NodeID: download2
X-Proxy-Cache: HIT
Accept-Ranges: bytes
Show "X-Proxy-Cache: HIT".
I think both those files were deleted two
hours ago.
Regards,
Ed
On 18.03.2020 13:22, Jonah Graham
wrote:
I believe everything is now
in place for the release except that
I and CI user cannot touch the
directory so that mirrors are working
for p2. See Bug 561215
For
completeness... Ed's assessment
of file timestamps vs. directory
is spot on, since p2 typically
uses the directory as the
mirrorsURL parameter.
Denis
On 2020-03-17 1:20 a.m., Ed
Merks wrote:
Markus,
I believe that the server's
general approach via
download.php is to consider the
time stamp of the local file
versus its timestamp on the
mirror. If they match, the
mirror is a good one and will be
returned in the list. For the
download folder, I expect the
timestamp is not so important
because only individual files
will be downloaded from it.
E.g., for this link, the
timestamp of the file is
important, but not the
timestamps of any of the parent
folders:
And here the timestamp of the
2020-03 folder itself is
definitely important.
That all being said, an install
will only need to download 4
artifacts from the EPP repo,
while needing to download 100s
from the train repo, so if there
were no mirrors for the EPP
repo, it would not cause such a
huge problem as it does when
there are no mirrors for the
train repo. That repo already
exists and has many mirrors:
You are right. I used to
be very careful with the
modification times of files
and directories, especially
the download directories are
important. I got used to
using my TIMESTAMP files
that can be found here and
there on the download
server, usually in commands
like
touch -r . TIMESTAMP
touch -r TIMESTAMP .
to "store" the
modification times of the
current working directory,
and to use it afterwards to
copy the very same
modification time back to
it.
Of course it's not
clear over which time
period that happened,
but there's always
quite a bit of
activity the day of
and the days after...
The mirror failures
right after the most
2019-12 release
certainly brought that
download server to its
knees enough to take
note that there was an
ongoing problem. And
that's in December,
close to Christmas
time, when you
wouldn't expect quite
so much either.
That's all to say
that I very much
appreciate Jonah being
conservative on this
front.
Note too that I see
that the mirrors are
back now!
Now that I'm thinking
about it though, we
should be careful what
happens to the
timestamp of the EPP
folder and the train
folder when the
content/artifacts
change just minutes
before the release.
If the folder's
timestamp changes, I
believe all the
mirrors will be
disabled again for a
while...
Regards,
Ed
On 16.03.2020
16:42, Markus Knauer
wrote:
Full
agreement... 24h
is typically
enough.
Sometimes, if
it was really
urgent in the
past, I
double-checked
with Denis and the
webmasters in
order to make them
aware that there
may be a download
peak. On the other
hand, the peaks
are not that high
(especially for a
March release)
than they used to
be.
And yes, I
would definitely
vote for the RC2a
version (without
aeri).
I believe
24 hours is
normally more
than enough,
though the
problem in https://bugs.eclipse.org/bugs/show_bug.cgi?id=561125
is still a
problem so
sooner is
probably
better than
later. I
feel that it's
safe to assume
that RC2a
versus RC2
will be good
because the
difference is
really very
trivial and
I've confirmed
that the Java,
JEE and RCP
packages
installed from
this basis via
the installer
start up
fine...
Thanks
Ed for
confirming. I
will do the
downloads this
evening
(Ottawa time)
Note that
if you're
going to
transform the
EPP repo into
a simple
repository,
you should get
the binary/features/plugins folders
copied up into
the 2020-03
folder in
preparation as
well.
I
will place all
the binary/features/plugins
files in place
too, leaving
just the
artifacts and
content files
to be updated
Wed morning.
I will ensure that the composites are
retuning 404
in time for
the release
too.
Thanks,
Jonah
Thanks for
all the heavy
lifting!!
Regards,
Ed
On
16.03.2020
15:42, Jonah
Graham wrote:
Hi
folks,
When
do the files
need to be in
place for
release day? I
assume today
is better than
tomorrow.
In
that case we
need to have
RC2a vs RC2
decision ASAP.
If I don't
hear anything
further I will
be making the
RC2a files the
R files.