| Re: [jakartabatch-dev] Ideas on expanding TCK to include tests for both with and without JobOperator bean in test app variations ? |
Hi,the JUnit 5 layer is the TCK "core" and that the Arquillian extension is just a way to bridge from JUnit 5 into a remote/managed Arquillian container.I supported and implemented this idea with the following assumptions (or lack of knowledge of):
- At that time, the classpath was flat for all the tests, both in the SE TCK variant or the Jakarta CTS variant, which made it simple to just wrap the SE tests and run them in a Jakarta EE container using Arquillian. Arquillian was just used as a deployer and executor of the test suite - the whole JUnit 5 test suite was deployed and executed in a Jakarta EE container
- I assumed that if different deployments are needed for some tests, then additional work is needed
- I didn't realize that the Jakarta TCK runs tests under different contexts (vehicles), I only focused on running the tests in a container using any (undefined) context. By default, Arquillian uses the Servlet context to deploy and also run the tests
With the latest changes (support injection of the JobOperator), the point 1 changes and we need change the classpath for some tests. This also impacts the point 2 - the deployment is no longer the same for all tests.Your solution, Scott, is not a bad solution to address this quickly. My intention was to build a TCK framework to support this nicely and more flexibly, using some TCK API and annotations, that could be picked up by any execution mechanism, not just Arquillian. I hope I will be able to develop this later but I have no time to do it for Jakarta EE 10.For point 3 (various execution contexts/vehicles), I don't know a full solution yet. I'm trying to figure out what's the best way to support this using either the plain JUNit5 approach or JUnit5 + Arquillian approach. I've studied the Jakarta EE TCK deeper today and it seems that it's doing a lot behind the scenes. Arquillian doesn't provide all of it. Arquillian only provides a deployment and execution mechanism for tests. Arquillian executes the whole test suite inside the tested container, so it's not easy to control what happens outside of the container, everything is controlled by Arquillian (e.g. how tests are triggered).The Jakarta EE TCK controls everything, it only delegates some things to the porting package, like deployment. Execution of tests with Jakarta EE TCK is fully controlled by the TCK:
- The TCK creates the deployment package (using an Ant task)
- calls the porting package to deploy the deployable
- The TCK then triggers the tests using each required vehicle. E.g. for the servlet vehicle, it will trigger the test using an HTTP request to a servlet, for the EJB vehicle it will invoke a remote EJB to execute the test
The step 3 is what Arquillian can do too but it only uses one predefined mechanism. Arquillian can't execute a test method multiple times using different mechanisms. On the other hand, JUnit5 extension can execute a test multiple times, but I haven't found a way to instruct Arquillian to use a different mechanism for each execution. We might need to extend the Arquillian JUnit5 extension to modify it and pass some context from JUnit5. I know, I know, extending an extension sounds too complicated even to me :)Ondro_______________________________________________pi 7. 1. 2022 o 16:55 Scott Kurz <skurz@xxxxxxxxxx> napísal(a):_______________________________________________A couple responses in follow-up to this:
1. For the SE path through the TCK, I put together a PR that added a second Maven module to construct the classpath with (vs. without) the app (TCK)-provided JobOperator bean. Still a bit awkward since the execution instructions now require a second failsafe execution in the Maven build, but less ugly than my idea to add a second artifact to one module via classifier.
It's here: https://github.com/eclipse-ee4j/batch-tck/pull/33 if anyone wants to take a look.
2. I think I need to backtrack. I'm not sure the EE/platform description I mentioned:
> the JUnit 5 layer is the TCK "core" and that the Arquillian extension is just a way to bridge from JUnit 5 into a remote/managed Arquillian container.
is the best way forwards. That's because to reach equivalence with the existing Platform TCK, we need to do the platform testing out of BOTH a servlet and an EJB context. So we either would have to invent a mechanism to add the servlet/EJB paths in the JUnit 5 core layer (i.e. create something like the "vehicles" in the Platform TCK) or rely on Arquillian to provide the "bridge" to execute within each servlet/EJB context.
I like the idea of relying on Arquillian and don't expect this will be a barrier to anyone certifying against the Platform. Remember too an implementation can choose to simply certify against our standalone Batch TCK in "SE mode" rather than the full Platform "EE mode".
More of this discussion on Slack with Romain & Ondro: https://eclipsefoundationhq.slack.com/archives/C02K9FX2ETA/p1641227943000100?thread_ts=1639785463.000500&cid=C02K9FX2ETA
------------------------------------------------------
Scott Kurz
WebSphere / Open Liberty Batch and Developer Experience
skurz@xxxxxxxxxx
--------------------------------------------------------"Scott Kurz" ---01/03/2022 09:36:26 AM---Hi all.. hope everyone had a good holiday and New Year's. As we aim to wrap up Batch 2.1 this month,
From: "Scott Kurz" <skurz@xxxxxxxxxx>
To: jakartabatch-dev@xxxxxxxxxxx
Date: 01/03/2022 09:36 AM
Subject: [EXTERNAL] [jakartabatch-dev] Ideas on expanding TCK to include tests for both with and without JobOperator bean in test app variations ?
Sent by: "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>
Hi all.. hope everyone had a good holiday and New Year's.
As we aim to wrap up Batch 2.1 this month, let me solicit ideas for one piece I'm not clear how best to implement.
From the JobOperator bean discussion (https://www.eclipse.org/lists/jakartabatch-dev/msg00269.html) we decided to add at least two TCK tests: one with and one without a TCK-provided JobOperator bean (the one "with" to simulate a user app-provided JobOperator bean).
A question then: what mechanics do we use to provide the variation?
I'm not sure.
One thought is that if we had settled on using Arquillian at the "core" of the TCK, I think we would add this variation along with the custom ShrinkWrap deployments we're building, building the deployments differently for one test class vs. another.
But in the TCK we're technically saying that the JUnit 5 layer is the TCK "core" and that the Arquillian extension is just a way to bridge from JUnit 5 into a remote/managed Arquillian container.
And it's possible to execute the TCK without using Arquillian at all, (e.g. how the com.ibm.jbatch.tck.exec executes the TCK against jbatch).
So...another idea I could easily imagine would be using two Maven failsafe executions, with different configurations, one with vs. without the JobOperator bean.
That packaging / configuration feels a bit off to me, however. Instead of providing one package and requiring the users to run multiple variations, carving it up and excluding a piece... maybe we should just provide two packages?
So I'm thinking of adding a second 'maven-jar-plugin' execution with a 'classifier' to produce a second JAR package without the JobOperator bean. (Or we could use a second module if we'd rather have one artifact per module.)
I'll proceed with that unless someone has a better idea, and will be working off of https://github.com/eclipse-ee4j/batch-tck/issues/32.
Any thoughts?
Thanks,
------------------------------------------------------
Scott Kurz
WebSphere / Open Liberty Batch and Developer Experience
skurz@xxxxxxxxxx
--------------------------------------------------------
_______________________________________________
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev