Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartaee-tck-dev] TCK "Promotion"
I cross linked to this thread from https://www.eclipse.org/lists/jakarta.ee-spec/threads.html#00707 and am now adding a cross link from here to there. :-) 

More inline below.

On 8/5/20 10:52 PM, David Blevins wrote:
On Aug 5, 2020, at 7:08 PM, Scott Marlow <smarlow@xxxxxxxxxx <mailto:smarlow@xxxxxxxxxx>> wrote:
On Wed, Aug 5, 2020 at 8:41 PM David Blevins <dblevins@xxxxxxxxxxxxx <mailto:dblevins@xxxxxxxxxxxxx>> wrote: 

    Hey All,

    I've been thinking to lend my bash skills to the TCK promotion
    concept for the TCKs produced from this project specifically.

    The goal is to make it easier for a project to promote (keep) a
    TCK build after they see the tests have passed.  The two potential
    "gotchas" we have are:

     - TCK that worked and could be passed could potentially be
    overwritten before a team can say, "hey can you promote that one"
     - Eliminating any need to "run one more time to be sure"

    On the last note, we spent an additional $30k on TCK
    infrastructure this year and we're also moving our release
    timeline out, so we've exceeded both budget and deadlines making
    any "just a bit more time and another run" less attractive.

    Currently we drop builds of the TCK into a stage directory.  These
    binaries have a stable file name that makes CI jobs easy.  It also
    makes promotion hard as all files have the same name and we only
    keep one copy.

    What if we had another area where we published the TCKs and this
    area kept say the last 3 or 5 builds of the TCKs.  The SHA would
    be added to the filename by the script.
We are also already printing the SHA to the $tckname-tckinfo.txt file but if the Specifications teams are only copying the TCK zip, sure, encoding the SHA into the name sounds helpful.
I've seen many projects are computing the SHA of the zip they download and outputting that into the TCK results.  So we're well positioned there too.
IMO, in the first pass of this change, just adding the SHA to the filename would be enough.  Especially since we have already staged the Standalone TCKs via https://ci.eclipse.org/jakartaee-tck/job/eftl-standalonetck-build-run-900 but will stage TCKs again on an as needed basis.
Right.  I think the key thing here is to leave the file name they expect now as to not break the existing CI jobs, but as you say just upload it again with the expanded name. 

+1
Your changes could probably go directly inline to https://ci.eclipse.org/jakartaee-tck/job/eftl-standalonetck-build-run-900/config
Perhaps we could also include a timestamp to make it obvious which tck zip is the latest, something like: 
mv $NAME ${NAME%.*}_$(date +%F-%H:%M).zip

I tried ^ locally with jakarta-annotations-tck-2.0.0.zip:

export NAME=jakarta-annotations-tck-2.0.0.zip
ls *.zip
jakarta-annotations-tck-2.0.0_2020-08-05-21:56.zip
What do you think about me moving that script into our source control?  Our script in the job could be basically  2 or 3 lines of "clone this repo, execute this script"
+1 that would be awesome! Then we could easily switch the format as well across multiple Jenkins jobs that stage TCKs. :-)
I guess the real question is where.  Maybe somewhere in jakartaee-tck-tools?  Not sure if that is a dead repo with a good name.
https://github.com/eclipse-ee4j/jakartaee-tck-tools sounds fine, adding the script to it will liven it up, especially if the non-jakarta-platform-tck projects also use it for staging! If in doing this, you come up with a different repo, that is also fine.
Thanks David for starting this thread and volunteering to help improve the staged names so SPEC API teams can be make sense of what each staged TCK file represents in terms of content (SHA) and timestamp of when the script was invoked. As well as know that a new staged TCK will not replace the a TCK that has already been staged! 

Scott




-David




_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev

Back to the top