Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartaee-tck-dev] TCK "Promotion"

On Aug 5, 2020, at 7:08 PM, Scott Marlow <smarlow@xxxxxxxxxx> wrote:



On Wed, Aug 5, 2020 at 8:41 PM David Blevins <dblevins@xxxxxxxxxxxxx> wrote:
Hey All,

I've been thinking to lend my bash skills to the TCK promotion concept for the TCKs produced from this project specifically.

The goal is to make it easier for a project to promote (keep) a TCK build after they see the tests have passed.  The two potential "gotchas" we have are:

 - TCK that worked and could be passed could potentially be overwritten before a team can say, "hey can you promote that one"
 - Eliminating any need to "run one more time to be sure"

On the last note, we spent an additional $30k on TCK infrastructure this year and we're also moving our release timeline out, so we've exceeded both budget and deadlines making any "just a bit more time and another run" less attractive.

Currently we drop builds of the TCK into a stage directory.  These binaries have a stable file name that makes CI jobs easy.  It also makes promotion hard as all files have the same name and we only keep one copy.

What if we had another area where we published the TCKs and this area kept say the last 3 or 5 builds of the TCKs.  The SHA would be added to the filename by the script.

We are also already printing the SHA to the $tckname-tckinfo.txt file but if the Specifications teams are only copying the TCK zip, sure, encoding the SHA into the name sounds helpful.

I've seen many projects are computing the SHA of the zip they download and outputting that into the TCK results.  So we're well positioned there too.

IMO, in the first pass of this change, just adding the SHA to the filename would be enough.  Especially since we have already staged the Standalone TCKs via https://ci.eclipse.org/jakartaee-tck/job/eftl-standalonetck-build-run-900 but will stage TCKs again on an as needed basis.  

Right.  I think the key thing here is to leave the file name they expect now as to not break the existing CI jobs, but as you say just upload it again with the expanded name.


Perhaps we could also include a timestamp to make it obvious which tck zip is the latest, something like:
mv $NAME ${NAME%.*}_$(date +%F-%H:%M).zip

I tried ^ locally with jakarta-annotations-tck-2.0.0.zip:

export NAME=jakarta-annotations-tck-2.0.0.zip
ls *.zip
jakarta-annotations-tck-2.0.0_2020-08-05-21:56.zip

What do you think about me moving that script into our source control?  Our script in the job could be basically  2 or 3 lines of "clone this repo, execute this script"

I guess the real question is where.  Maybe somewhere in jakartaee-tck-tools?  Not sure if that is a dead repo with a good name.


-David



Attachment: smime.p7s
Description: S/MIME cryptographic signature


Back to the top