Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartaee-platform-dev] TCK tests in the same repo as API and Spec
Lance and Bill,

The current approach I've taken is actually quite flexible, and is mainly inspired by the way MicroProfile does its TCKs. It can easily be pulled into any other project using Maven and the tests are simply executed with JUnit. Once the JSONB TCKs are moved, I was planning to update the jakartaee-tck repo to pull in the JSONB TCKs as a maven dependency and still have them run and hook into the same entry points as before, so from a "CTS team" perspective nothing would change.

As Dmitry said, we can hold off on the modifications in the jakartaee-tck repo, but long term we really need to get the TCKs split out. As one of the spec developers I see this as an essential quality-of-life improvement so it was on the top of my list of TODOs for EE9.


On Fri, Jan 31, 2020 at 1:35 PM Lance Andersen <lance.andersen@xxxxxxxxxx> wrote:
I also agree with Bill.  There has to be thought  given to CTS structure first IMHO so that there is consistency in the test structure to make it easier to pull in the standalone tests  and  where the platform specific tests are maintained for the standalone technologies.  

This will take some time to flush this out.  During the Java EE days, we did find it easier to work out of one master workspace, but part of that was due to the fact there was one team primarily responsible  for the test development for all of the Java EE technologies.

On Jan 31, 2020, at 2:01 PM, dmitry.kornilov@xxxxxxxxxx wrote:

Bill, you are right. On the other hand the process of splitting has to be started somehow. It will never start with an assumption that some developer with TCK knowledge will take an initiative and start working on it. There are not too many developers like this and no one wants to take a risk and responsibility. 
 
On the other hand I don’t think that it’s a right time to do it now as part of Jakarta EE 9 release. I already proposed that we will not move JSONB TCK tests from CTS yet. We will do it after Jakarta EE 9 is released. We will keep them in sync until that time. Users may use TCK tests in JSONB repo as more convenient way of checking compatibility. But it cannot be used officially for compliance testing for now.
 
-- Dmitry
 
From: jakartaee-platform-dev-bounces@xxxxxxxxxxx <jakartaee-platform-dev-bounces@xxxxxxxxxxx> On Behalf Of Bill Shannon
Sent: Friday, January 31, 2020 7:06 PM
To: jakartaee-platform developer discussions <jakartaee-platform-dev@xxxxxxxxxxx>; Andy Guibert <andy.guibert@xxxxxxxxx>
Subject: Re: [jakartaee-platform-dev] TCK tests in the same repo as API and Spec
 

This has been discussed numerous times over the last 3 years.  No one disagrees with the goal of splitting up the TCK repo.  Whether the individual spec TCKs are in the same repo as the API classes or in a different repo is just a detail.  The challenge is in splitting up the existing TCK repo such that the resulting TCK for each spec is functionally identical to the existing standalone TCKs for the specs, and that the platform TCK is somehow created by combining all the individual TCKs to produce a new platform TCK that is functionally identical to the existing platform TCK.

Needless to say, this is not a small job.

No one with deep knowledge of the existing TCKs has come forward with a detailed plan for how to achieve the above.  Without that, we're all just wishing and hoping.

And I strongly encourage you to not just start hacking on the TCK to create what you want for your spec.  We need to solve the larger problem and a bunch of uncoordinated hacks to individual TCKs will not get us there.

Thanks.

P.S.  You should be able to find some of the previous history of this discussion in the jakartaee-tck-dev mailing list.

Andy Guibert wrote on 1/30/20 10:45 AM:
Hi all, 
 
Currently all of the Jakarta EE TCK tests are housed in one big repo and they use a custom test framework from the Java EE days:
 
 It is more convenient to have the TCK tests in the same repo as the API and spec docs because as the technologies change over time all 3 parts (tck/api/spec) can be updated in the same PR. Additionally, implementations can then consume the TCK tests as Maven artifacts and run/verify that they pass the TCK. This is what MicroProfile has done and it works very well.
 
As an example, I've started this off with the JSON-B TCK test here:
 
Just wanted to share this with the wider dev community and encourage other specs to follow suit as time permits.
 
Cheers, Andy
-- IBM


_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
 
_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev


Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen@xxxxxxxxxx



_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev

Back to the top