Ed,
I believe we have addressed all of the issues you raised and have thus far attempted several times to update the TCK certification results accordingly, but every time we do so another update pops up that one
of the other vendors wants made to the TCK, and we need to start over. After an update that was requested earlier this week, we got agreement on the Jakarta EE Platform call that that would be the last one and we would push forward with what we have after
making that change. We are very close to getting the TCK results published from after that. It should be noted that just today another update request did come in which is being discussed. However, per the prior agreement, I assume it will be deferred and
possibly covered with a challenge if need be.
From:
cu-dev <cu-dev-bounces@xxxxxxxxxxx> on behalf of Ed Bratt via cu-dev <cu-dev@xxxxxxxxxxx>
Date: Friday, May 17, 2024 at 4:59 PM
To: Kyle Aure <kylejaure@xxxxxxxxx>, cu developer discussions <cu-dev@xxxxxxxxxxx>
Cc: Ed Bratt <ed.bratt@xxxxxxxxxx>
Subject: Re: [cu-dev] [External] : Re: Mentor Review Feedback, Concurrency 3.1
Checking back in -- Can someone (Kyle maybe?) provide a status update with respect to the issues I've raised and/or when we might project starting the release ballot.
Thank you, -- Ed On 5/2/2024 7: 29 AM, Kyle Aure wrote: Hey Ed, Thanks for
This Message Is From an External Sender
This message came from outside your organization.
Checking back in -- Can someone (Kyle maybe?) provide a status update with respect to the issues I've raised and/or when we might project starting the release ballot.
Thank you,
-- Ed
On 5/2/2024 7:29 AM, Kyle Aure wrote:
Thanks for the clarification.
Taking a closer look at the two TCK Distribution zips I figured out what was causing the checksums to be different.
The HTML version of the TCK guide had a footer with a generated date.
Hi Kyle
There will only be one TCK tracked by the Spec. committee. Whatever that file is, should be the reference archive (docs + binaries + ancillary materials). If that reference archive contains artifacts that are used to run the TCKs, those subset archives (JARs)
must have the same SHA sum of the files that are in the reference file tracked by the committee. You are confirming that it true. However the TCK ZIP, from which it is extracted isn't identical to the one listed in the _index.md. (e.g. concurrency-tck-3.1.0.zip
has a SHA sum that is different from jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip), Therefore, I have no way of knowing how these relate and our process has no way to track this other tck ZIP file. So, even though the embedded JARs are the same, the
archive that contains it isn't going to match anything. Had concurrency-tck-3.1.0.zip simply been a rename of jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip the SHA sums would have matched and we'd have been fine. In fact, since the TCK binary from
within the larger archive is the same, the test results are valid. However, the TCK is defined as the binaries, ancillary files, and all their included documentation. Hence the larger, reference container archive has to ultimately be the one that we track.
(I'm sorry if this is confusing.)
In short, I think, jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip should be identical to concurrency-tck-3.1.0.zip. If there is another reason for these to differ, let me know and we can try to figure out how to resolve this. Ultimately, this file
is the normative TCK and what should be referenced in all reports.
Once the TCK has the correct license, I'm sure this can all be squared away. I regret we didn't do a better job informing everyone of the new TCK and Spec. license tiles.
Regarding the number of tests -- All I want is the Spec committer team to confirm the number of tests. If that is done with these update, I'm satisfied.
Thank you,
-- Ed
On 5/1/2024 3:42 PM, Kyle Aure via cu-dev wrote:
Thanks for sending this along.
Here are responses to your concerns and some followup questions:
-
Currently the TCK can be obtained from 3 different locations:
-
Embedded JAR - SHA 256: 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
-
Embedded JAR - SHA 256: 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
-
So the question we (Open Liberty) has is which SHA should we be reporting?
-
We reported the SHA for the zip downloaded from eclipse, but it seems we should have reported the SHA sum for the TCK jar itself.
-
I have updated the certification template for concurrency to reflect this: https://github.com/jakartaee/concurrency/pull/485
-
Pull request opened: https://github.com/jakartaee/concurrency/pull/484
-
Our documentation listed the number of tests ran (268) and tests skipped (27)
-
Whereas, the maven-surefire-plugin lists the number of tests total (295) and tests
skipped (27)
-
So the number of skipped tests is double counted and our documentation did not account for that.
Spec landing page (_index.md):
Specification license text needs to be updated everywhere it appears:
FYI - Seeing as how I need to update the license we will need to re-build and re-stage the final release meaning we will need to re-run the TCK and post results.
Hi there,
First off, I'm very grateful that you have delivered all the material needed for release review of this specification version.
Dmitry and I are going to be reviewing the materials you have put together for release review. As we have in the past, we will be using a couple of longer checklists to ensure that all the materials are ready to go and there aren't any SNAFUs during the
ballot. I have pasted the checklist into the PR and I'll be following up if we find any issues.
Here is a short-list of issues I'd like to get your feedback on. My PR review also contains these details.
TCK
-
Please revise the TCK license to EFTL v1.1. This refers explicitly to Eclipse Foundation AISBL
-
License included in the TCK zip -- /LICENSE
-
License in the TCK reference guide. -- since this just references by link, the only thing incorrect is that it says 'v 1.0' -- you might consider just dropping the version (though I wouldn't expect this to change again but who knows.)
-
Note, I recommend this be addressed prior to the addressing the following point
SHA Sums for the TCK -- this seems to be a challenge for all of the specifications and I hope that we can simplify this in the future. The TCK that is to be referenced for release must be the exact TCK that will be posted with the final artifacts. The only
SHA Sum we track is for the full distribution TCK (includes the tests, the documentation, and any ancillary artifacts). When TCKs provide subset JAR files (e.g. a binary TCK JAR), that must have the same SHA as the same JAR in the distribution. If this does
not hold true, we have no way of accurately tracking that the vendor actually used the TCK that is referenced from the Specification Summary Page. I have noted the following SHA-256 Sums (note they all differ):
-
SHA Sum of contained file (jakarta.enterprise.concurrent-tck-3.1.0.jar) -- 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
-
TCK SHA sum in PR/Alternate (jakarta.enterprise.concurrent-tck-3.1.0.jar) -- 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
The TCK artifacts in the PR seem consistent. However, the TCK used by OpenLiberty doesn't seem to match. Could you please investigate this with your contact from OpenLiberty and correct the record and/or the test target? While the Spec. committee would prefer
to only track the main distribution TCK (in this case tck-dist-3.1.0), we will accept the sub-component SHA, so long as it matches the SHA in the distribution TCK.
It seems there is something different in the Staged TCK. Remember, even if you just rebuild the TCK, the SHA sums will differ.
-
Please confirm the test count for OpenLiberty is as expected. The result lists skipped tests and the count total differs from the 'expected output' of the TCK User Guide (OpenLiberty reports 295 while the UG suggests 268. Both have the same number of skipped
tests -- in an ideal world, the initial CCR and the UG wouldn't have skipped tests but that's not a requirement).
Spec landing page (_index.md):
-
Please revise the landing page to reflect that OpenLiberty 24.0.0.6-beta is the initial CI. (the text suggests there might be another CI and I don't see another 3.1 CCR in the
concurrency spec. issue list.)
-
Please confirm that you are happy with the summary/change text content. To my read, it still has a bit of 'we could do this, or these bugs might be fixed). I'd recommend, for example, you pick a few issues that you think highlight the work accomplished. If
you have a release tag, milestone or other change tracking document, you may refer to that as well (some document that lists all the changes).
Specification license text need to be updated everywhere it appears (in the Specifications and in JavaDocs) to reference
Specification License 1.1 (this has explicit reference to Eclipse Foundation AISBL). Please revise each of the following:
-
Specification PDF -- license text
-
Specification HTML -- license text
-
JavaDocs -- URL to license in Spec. git repository. You should update the license in the javadoc folder )y and leave the link in the JavaDocs alone or, you could revise the link in the Javadocs to point at the primary specification location (here).
Thank you!
-- Ed Bratt
_______________________________________________
cu-dev mailing list
cu-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cu-dev
_______________________________________________
cu-dev mailing list
cu-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cu-dev
|
