Re: [epp-dev] EPP 2020-03 M3

[Frederic Gurr <frederic.gurr@xxxxxxxxxxxxxxxxxxxxxx>]

Hey,

Sorry for the late reply. I was in a meeting.

I'd recommend to keep the responsibilities for updating
* http://download.eclipse.org/releases/2020-03 and
* http://download.eclipse.org/technology/epp/packages/2020-03
separate as before but fix the synchronization.

I use https://ci.eclipse.org/simrel/job/simrel.releng.makeVisible to
make the repo "visible" half an hour before the actual release. This
script used to call
https://ci.eclipse.org/packaging/job/epp-makeVisible/, but this call
across Jenkins masters has been failing for a long time. We should get
it fixed.

Regards,

Fred

On 28.02.20 16:05, Jonah Graham wrote:
> Hello folks,
> 
> The https://download.eclipse.org/technology/epp/packages/2020-03/ has
> the composite files ready for M3 - they just need copying over the
> existing jars.
> 
> Fred, are you able to do that so it is synchronized? Or do I do it, and
> if so, when?
> 
> Thanks,
> Jonah
> 
> ~~~
> Jonah Graham
> Kichwa Coders
> www.kichwacoders.com <http://www.kichwacoders.com>
> 
> 
> On Fri, 28 Feb 2020 at 02:49, Jonah Graham <jonah@xxxxxxxxxxxxxxxx
> <mailto:jonah@xxxxxxxxxxxxxxxx>> wrote:
> 
>     Hi Ed,
> 
>     > How are the changes to
>     http://download.eclipse.org/technology/epp/packages/2020-03 and
>     http://download.eclipse.org/releases/2020-03 coordinated?
> 
>     Thanks for asking the question. As you know this step was initially
>     missed on M2 release day.
> 
>     I don't know how the coordination was done in the past. I assume
>     that Fred @ EF does the /releases/2020-03/ one and that Markus had
>     historically done the EPP one. It seems what is needed here is for
>     one of us to do both. 
> 
>     I assume I don't have access to /releases/2020-03/  and that Fred
>     does have access to EPP's p2. Therefore, if,
>     like /releases/2020-03/ directory I provide a ready to
>     go compositeContentM3.jar/compositeArtifactsM3.jar in the EPP's p2
>     location then Fred can rename both at the same time? 
> 
>     *Fred, does that work for you?*
> 
> 
>     > A note of caution as well.  [...]
> 
>     At the moment I only have "For the final release, this composite p2
>     repository is being transformed into a flat p2 repository." in the
>     process doc. I will read through the mentioned comments and make
>     sure I expand on that.
> 
>     Thanks,
>     Jonah
> 
>     ~~~
>     Jonah Graham
>     Kichwa Coders
>     www.kichwacoders.com <http://www.kichwacoders.com>
> 
> 
>     On Fri, 28 Feb 2020 at 02:35, Ed Merks <ed.merks@xxxxxxxxx
>     <mailto:ed.merks@xxxxxxxxx>> wrote:
> 
>         Jonah,
> 
>         How is the updates of the p2 repos coordinated?  
> 
>         Currently
>         /home/data/httpd/download.eclipse.org/releases/2020-03/compositeContent.jar
>         <http://download.eclipse.org/releases/2020-03/compositeContent.jar>
>         is like this:
> 
>         <?xml version='1.0' encoding='UTF-8'?>
>         <?compositeMetadataRepository version='1.0.0'?>
>         <repository name='Eclipse Repository'
>         type='org.eclipse.equinox.internal.p2.artifact.repository.CompositeMetadataRepository'
>         version='1.0.0'>
>           <properties size='3'>
>             <property name='p2.timestamp' value='1313779613118'/>
>             <property name='p2.compressed' value='true'/>
>             <property name='p2.atomic.composite.loading' value='true'/>
>           </properties>
>           <children size='4'>
>             <child location='../../technology/epp/packages/2020-03/'/>
>             <child location='202002071000' />
>             <child location='202001171000' />
>           </children>
>         </repository>
> 
>         Note the wrong size in "<children size='4'>", so this is clearly
>         being manually maintained...
> 
>         It appears that
>         /home/data/httpd/download.eclipse.org/releases/2020-03/compositeContentM3.jar
>         <http://download.eclipse.org/releases/2020-03/compositeContentM3.jar>
>         is ready as a replacement and its content looks correct.
> 
>         Of course that replacement's timing must be coordinated with the
>         state change in
>         /home/data/httpd/download.eclipse.org/technology/epp/packages/2020-03/
>         <http://download.eclipse.org/technology/epp/packages/2020-03/>. 
>         I see M3 as a child so that looks mostly ready.
> 
>         I can (and do) test the generation of the product catalogs as if
>         the repos were already updated with an option like this for the
>         generator:
> 
>           -staging 2020-03 
>         http://download.eclipse.org/technology/epp/packages/2020-03/M3
>         http://download.eclipse.org/releases/2020-03/202002281000
> 
>         I can then inspect the changes to the generated catalogs (and
>         can even test that they install).  In this case the only
>         significant semantic change is in the JEE product and that looks
>         correct because I can see that did indeed change for M3:
> 
>          
>         https://git.eclipse.org/c/epp/org.eclipse.epp.packages.git/commit/?id=e5905ffa4dd1ac3f66a31fa4d2c455dea5d22f4e
> 
>         So the question is: 
> 
>           *How are the changes to
>         **http://download.eclipse.org/technology/epp/packages/2020-03
>         and **http://download.eclipse.org/releases/2020-03 coordinated?*
> 
>         In the ideal world, the timing of the generated catalogs is also
>         coordinated.   This is less essential when there are no changes
>         to product definitions because the catalog uses the composites
>         (for the currently release cycle versions) so will see new
>         content as soon as there is new content in the composites.  At a
>         later point in time, I remove this option for the generator:
> 
>           -useComposite 2020-03
> 
>         Then the generated catalog will directly refer to the child of
>         http://download.eclipse.org/releases/2020-03/<release-repo>,
>         avoiding that users hammer the down.eclipse.org
>         <http://down.eclipse.org> server with far fewer p2 requests for
>         repository metadata.
> 
>         *A note of caution as well.  * On the release day, the
>         http://download.eclipse.org/technology/epp/packages/2020-03
>         repository transitions from being a composite to being a simple
>         repository.  This has been a source of significant grief over
>         the years.  See this comment for the most recent incident during
>         the most recent release:
> 
>           https://bugs.eclipse.org/bugs/show_bug.cgi?id=548497#c29
> 
>         Of course we all hope to avoid such an incident again, but it's
>         particularly problematic that such incidents occur precisely
>         when it's most essential that there not be a problem, i.e., on
>         the release day when users see the problems.
> 
>         Thanks so much for stepping up to manage all this and for
>         transparently documenting the process!
> 
>         Regards,
>         Ed
> 
> 
>         On 27.02.2020 20:02, Jonah Graham wrote:
>>         Hi everyone,
>>
>>         Our next milestone build is available for testing: EPP 2020-03 M3
>>
>>         /I have been following the steps
>>         on https://hackmd.io/@jonahgraham/eclipse-epp-release-process - you
>>         can see the checkmarks as to what is done./
>>
>>         /Can someone on Mac please verify the Notarization
>>         <https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution>
>>         for Mac is working as expected.  I have added the notarization
>>         step into the build but I want to make sure it produced the
>>         correct results./
>>
>>         https://ci.eclipse.org/packaging/job/simrel.epp-tycho-build/896/artifact/org.eclipse.epp.packages/archive/
>>         or
>>         https://download.eclipse.org/technology/epp/downloads/release/2020-03/M3/
>>
>>         with the p2 repositories at
>>
>>         http://download.eclipse.org/staging/2020-03 and
>>         http://download.eclipse.org/technology/epp/packages/2020-03/M3
>>
>>         Please test and send your +1 to this mailing list.
>>
>>         Thank you for testing!
>>
>>         Regards,
>>         Jonah
>>         ~~~
>>         Jonah Graham
>>         Kichwa Coders
>>         www.kichwacoders.com <http://www.kichwacoders.com>
>>
>>         _______________________________________________
>>         epp-dev mailing list
>>         epp-dev@xxxxxxxxxxx <mailto:epp-dev@xxxxxxxxxxx>
>>         To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>>         https://www.eclipse.org/mailman/listinfo/epp-dev
>         _______________________________________________
>         epp-dev mailing list
>         epp-dev@xxxxxxxxxxx <mailto:epp-dev@xxxxxxxxxxx>
>         To change your delivery options, retrieve your password, or
>         unsubscribe from this list, visit
>         https://www.eclipse.org/mailman/listinfo/epp-dev
> 

-- 
Frederic Gurr
Release Engineer | Eclipse Foundation Europe GmbH

Annastr. 46, D-64673 Zwingenberg
Handelsregister: Darmstadt HRB 92821
Managing Directors: Ralph Mueller, Mike Milinkovich, Gaël Blondelle