Hi Jan, I think you've raised some good questions -- I've picked out 2 basic questions from your note which I will try and answer.
1) How/where do we test multi-spec interactions?
Lets say we have an interaction between CDI + JSONB that needs to be tested. I propose we pick one of the specs of the involved technologies to house the test. This way we don't have any TCK tests in "no mans land". If there are optional integrations (e.g. do X IF the runtime is CDI-capable) we can control this in a number of ways such as: A) do a Class.forName() of some CDI API to see if CDI is available in the test or B) control test inclusion with sysprops/env vars like `mvn test -Dcdi=false` to disable CDI-capable tests.
2) What standard test framework do we settle on? We don't want every spec doing their own thing because it will make running all TCK tests very difficult.
Fully agree. I propose we do the following:
- Create a new entry point in the jarkartaee-tck repo which will be a maven-based project that can test the new "externalized" TCK tests. Then we can incrementally migrate TCK tests on a per-spec level to the new externalized form.
- Externalized TCK tests MUST use JUnit + Arquillian, even if the technology is standalone (like JSONB). This way, application servers can still certify that the "standalone" technologies still work when deployed in applications on the servers. We can create a "no-op" Arquillian container that standalone implementations can use (e.g. Yasson for JSONB) for when components want to verify TCK tests in standalone mode. Although MicroProfile does not have a deployment spec, its TCK tests still use Arquillian to create JEE deployments such as .war or .ear files.
- Create a new umbrella entry point so we can run all TCK tests in one shot: the old ones based on the JavaTest harness, and the new/externalized ones based on JUnit+Arquillian
- Andy