Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartabatch-dev] Batch TCK discussion continued - Junit 5, Arquillian, etc.
Hi Scott M.,

Thanks for reaching out. I plan to join the discussion on the TCK ML soon and present what we're doing in Batch TCK. But I'll answer your question about how much effort it is to support Arquillian even here.

First, I'd like to clarify that our approach is not to write tests that depend on Arquillian but to have pure JUnit 5 tests. We found a simple solution to turn such tests into Arquillian tests on the fly, without changing a line of code and without depending on Arquillian. With that approach, it's theoretically possible to deploy and run the same tests using the TCK porting package although I haven't looked at that in detail.

JUnit5 allows running the tests in Arquillian without changing any code in those tests. To do that, it's enough to add the Arquillian JUnit5 extension to the project and enable it using a service loader file. On top of that, Arquillian requires a "deployment", which is usually specified by a method annotated by the@Deployment annotation. But it's also possible to specify a deployment using an Arquillian plugin which can also be enabled for all tests using service loader. And the plugin can either generate the same deployment for all tests (this is what we do in Batch TCK) or different deployments for different tests.

I created a sample runner project for GlassFish, that takes an artifact with plain JUnit5 Batch TCK tests, adds all this and runs the tests using GlassFish Arquillian connector on a deployment generated dependencies in the pom.xml. Here it is: https://github.com/OndroMih/Payara-Batch-TCK-Runner/. It only contains configuration files and a single class that generates the deployment and is pretty small (42 lines of code including imports): https://github.com/OndroMih/Payara-Batch-TCK-Runner/blob/poc-junit5-over-arquillian/jbatch-arquillian-extension/src/main/java/jbatch/arquillian/extension/MavenTestDependenciesDeploymentPackager.java

Turning TestNG tests into JUnit5 tests required a lot more code changes, but a lot of those changes were automated by a simple script, although I had to go through each test manually to fix some imports and review the changes. Converting from JTH to JUnit 5 would be more complicated because it doesn't use annotations like Before..., After... but I'm sure it can be automated too.

Ondro

st 9. 6. 2021 o 16:54 Scott Marlow <smarlow@xxxxxxxxxx> napísal(a):

FYI https://www.eclipse.org/lists/jakartaee-tck-dev/msg01268.html is looking for feedback on updating the Platform TCK for EE 10/10.1.  If we were to switch the Platform TCK to use Junit5 and/or TestNG, how much effort would it be to create a pull request that updates https://github.com/eclipse-ee4j/jakartaee-tck/tree/master/src/com/ibm/jbatch/tck to contain a Maven pom.xml that can build/run the Batch tests with Arquillian?

I'm trying to get a sense for how much effort it might be to convert most of the Platform TCK tests (around 2,557,861 lines of Java code) to use Arquillian + Junit5/TestNG.

Thanks,
Scott

On 6/9/21 9:05 AM, Ondro Mihályi wrote:
I've updated https://github.com/eclipse-ee4j/batch-tck/issues/22 with a list of all the things to do. Please add anything I missed.

I'd rather add all general TCK functionality like signature tests into the tck module and only add configuration for JBatch into the tck.exec module. So that the tck.exec module can be used as an example how to run the TCK. Later we can add a similar module for GlassFish to demonstrate how to run the TCK using Arquillian. It would help to keep things separated so that it's clear for spec implementers what they need to set up to run the TCK.

st 9. 6. 2021 o 14:34 Scott Kurz <skurz@xxxxxxxxxx> napísal(a):

Ah.. they run from Maven in the tck.dist.exec module .. I was looking in the tck.exec module... so we'd just have to move that around.

Sure.. whatever PR combination makes sense, and I did open issue     https://github.com/eclipse-ee4j/batch-tck/issues/22  for Ant->Maven refactor.

Thanks for working on this, this will be a nice step forwards !
------------------------------------------------------
Scott Kurz
WebSphere / Open Liberty Batch and Developer Experience
skurz@xxxxxxxxxx
--------------------------------------------------------

Inactive hide details for "Ondro Mihályi" ---06/08/2021 04:52:01 PM---By the way, signature tests now run also from maven, this"Ondro Mihályi" ---06/08/2021 04:52:01 PM---By the way, signature tests now run also from maven, this was already merged earlier: https://urldef

From: "Ondro Mihályi" <ondrej.mihalyi@xxxxxxxxx>
To: jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
Date: 06/08/2021 04:52 PM
Subject: [EXTERNAL] Re: [jakartabatch-dev] Batch TCK discussion continued - Junit 5, Arquillian, etc.
Sent by: "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>




By the way, signature tests now run also from maven, this was already merged earlier: https://github.com/eclipse-ee4j/batch-tck/pull/19. So there's no blocker for switching from Ant to Maven. And everything will be much easier to do and configure with maven, a lot of the documentation can be simplified. I'll start working on it.

ut 8. 6. 2021 o 22:43 Ondro Mihályi <ondrej.mihalyi@xxxxxxxxx> napísal(a):
    I'm not sure about doing all this in a single PR, it's then harder to review, isn't it? But I agree that we need to do all or nothing to avoid having things out of synch.

    What about raising an issue for all we mentioned and anything else we spot later, and mark those issues with a common label, e.g. "tck-refactor"? We can also create a project to track this, where we would order the issues in the order we want to implement them and raise PRs. Then we can raise a chain of PRs which depend on each other and each PR would show a difference between 2 particular steps.

    Do you think it's worth this? Or it's better to include all those changes in a single PR? Then I would create the PR from a branch in the original repository so that we can all make changes before anything is merged to master.



    ut 8. 6. 2021 o 21:40 Scott Kurz <skurz@xxxxxxxxxx> napísal(a):
      One important detail I glossed over is that only the com.ibm.jbatch.tck.dist.exec module now runs a signature test whereas  com.ibm.jbatch.tck.exec only now runs the Test NG suite.    

      I think one of your PRs might have added this (but not 21, at the moment).

      Whichever PR(s) we use, I think we should probably commit to updating the doc       https://github.com/eclipse-ee4j/batch-tck/blob/master/jakarta.batch.official.tck/src/main/asciidoc/batch-tck-reference-guide.adoc   at the same time as moving from Ant->Maven.
      Otherwise we risk leaving things out of synch, and this would be the easiest time to update the doc while the details are fresh.

      Thanks,
      ------------------------------------------------------
      Scott Kurz
      WebSphere / Open Liberty Batch and Developer Experience
      skurz@xxxxxxxxxx
      --------------------------------------------------------

      Inactive hide details for "Ondro Mihályi" ---06/08/2021 11:09:05 AM---I agree with dropping Ant and replacing it with Maven. I"Ondro Mihályi" ---06/08/2021 11:09:05 AM---I agree with dropping Ant and replacing it with Maven. I already though how we would rewrite the Ant

      From:       "Ondro Mihályi" <ondrej.mihalyi@xxxxxxxxx>
      To:       jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
      Date:       06/08/2021 11:09 AM
      Subject:       [EXTERNAL] Re: [jakartabatch-dev] Batch TCK discussion continued - Junit 5, Arquillian, etc.
      Sent by:       "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>



      I agree with dropping Ant and replacing it with Maven. I already though how we would rewrite the Ant scripts to use JUNit 5 and it seemed awkward and unnecessary unless we had some good reasons to support Ant.

      With my PR, it's already possible to use       com.ibm.jbatch.tck.exec as a dependency in a new maven project created for an implementation to run against the TCK. The dependency can be grabbed directly from Maven Central. So we only need to update the docs to instruct how to use it to run the TCK using a maven project and then drop the com.ibm.jbatch.tck.exec module as you suggest, Scott.

      I can do it, either in the PR I raised or I'll raise a separate PR if we merge it by then.

      Ondro

      po 7. 6. 2021 o 17:37 Scott Kurz <      skurz@xxxxxxxxxx> napísal(a):
      Thanks Ondro,    I'll take a look.

      Personally, I don't see a problem with abandoning Test NG at this point.  At one point, when the TCKs were less open, it was nice to be able to use the single XML file as suite definition (include/exclude), etc.     But I'm confident the Jakarta processes we have in place can help us track includes/excludes well enough without it (and we can handle the process of incorporating a TCK challenge, change, etc. with or without it).
      -------------------------------------------------------------------------------

      On a separate but related note, I'd like to also propose abandoning the Ant-based TCK and declaring the "official" Batch TCK execution runs against a Maven artifact (or however exactly you'd phrase it).

      The legacy here:  we originally built a TCK zip with some Ant scripting and only added the Maven "runner" exec later... so we were kind of stuck with the pre-Maven zip.  
      But I think we can abandon that, which would let us simplify and remove the       com.ibm.jbatch.tck.dist.exec module which runs an "official", Ant execution of the TCK against jbatch, and let us focus on just the   com.ibm.jbatch.tck.exec module which does a Maven failsafe execution of the TCK against jbatch.

      The only downside is the TCK guide documentation would need to be updated.      Hopefully this wouldn't be a big deal.. maybe we can say less and speak in more general terms ("set a property") whereas with this custom, one-off Ant doc we had to describe every single piece of it.  

      Well, I captured this issue:          https://github.com/eclipse-ee4j/batch-tck/issues/22 and added to the milestone.

      ------------------------------------------------------
      Scott Kurz
      WebSphere / Open Liberty Batch and Developer Experience
      skurz@xxxxxxxxxx
      --------------------------------------------------------

      Inactive hide details for "Ondro Mihályi" ---06/07/2021 09:44:05 AM---Hi all, Here's my new and improved PR that rebased onto m"Ondro Mihályi" ---06/07/2021 09:44:05 AM---Hi all, Here's my new and improved PR that rebased onto master:

      From:       "Ondro Mihályi" <ondrej.mihalyi@xxxxxxxxx>
      To:       jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
      Date:       06/07/2021 09:44 AM
      Subject:       [EXTERNAL] Re: [jakartabatch-dev] Batch TCK discussion continued - Junit 5, Arquillian, etc.
      Sent by:       "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>



      Hi all,

      Here's my new and improved PR that rebased onto master:       https://github.com/eclipse-ee4j/batch-tck/pull/21

      It also links to an example project that uses Arquillian to run the TCK against GlassFish, including the EE test.

      This solution also solves the problem of packaging. Each implementation can create a project similar to the example GlassFish project, modify the Arquillian plugin which it contains and build any deployment package supported by Arquillian, not only WAR. And implementations that run on a flat classpath can just run without Arquillian. It's also possible to use any other technology instead of Arquillian, e.g. reuse an existing Jakarta EE porting package used in the EE TCK.

      As I wrote in the PR, a similar solution can be achieved with Test NG but it requires a bit of a different approach to overcome the limitations of Test NG. I personally prefer JUnit 5 because it's newer, more modern and has a very powerful design. But Test NG has some features that are just coming to JUnit 5 or may not come at all. We can have a separate discussion about that.

      Ondro

      št 3. 6. 2021 o 16:05 Scott Kurz <      skurz@xxxxxxxxxx> napísal(a):
      Ondro,

      Yes, your solution sounds ideal given that we want to be able to use this one standalone TCK to certify both within and outside of the Jakarta EE Platform.  

      Using a single WAR as packaging should be fine in EE (no need to have any finer application scopes).   The platform TCK also at one time exercised the same set of tests in an EJB JAR package;  I'd assume it still does unless there have been changes.

      And no need or desire to standardize any other package for non-EE, this hasn't been an issue in the past so just putting it all on the classpath should be fine there.

      So thanks again, for rebasing, (and if the reformatting is too painful we can try to standardize that better.)
      ------------------------------------------------------
      Scott Kurz
      WebSphere / Open Liberty Batch and Developer Experience
      skurz@xxxxxxxxxx
      --------------------------------------------------------

      Inactive hide details for "Ondro Mihályi" ---06/02/2021 01:35:27 PM---Hi, Yes Scott, I will rebase my PR onto master. I just wa"Ondro Mihályi" ---06/02/2021 01:35:27 PM---Hi, Yes Scott, I will rebase my PR onto master. I just waited until people here

      From:       "Ondro Mihályi" <ondrej.mihalyi@xxxxxxxxx>
      To:       jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
      Date:       06/02/2021 01:35 PM
      Subject:       [EXTERNAL] Re: [jakartabatch-dev] Batch TCK discussion continued - Junit 5, Arquillian, etc.
      Sent by:       "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>



      Hi,

      Yes Scott, I will rebase my PR onto master. I just waited until people here are interested in such a solution.

      To clarify my proposal again and the options we have with TestNG / JUnit 5 / Arquillian
    • We could continue with Test NG and just add Arquillian, but it's a cumbersome and JUnit 5 offers more flexibility and a modern toolkit
    • We could migrate to JUnit 5 and then just add Arquillian as Arjan suggested, but that would require some standard packaging of the test application like WAR. However, JBatch doesn't require implementations to support WAR so it adds extra burden for them to add an Arquillian extension to repackage the test application. We could also create plain JAR packages but then implementations that require WAR would need to repackage the app
    • The solution I propose in [1] allows running the tests with JUnit 5 on the current classpath without Arquillian. To run the the tests inside a container like Jakarta EE server, it provides a stockplugin for Arquillian to package all the classes and resources on the classpath into a WAR. Implementors can add the JUnit 5 Arquillian extension and use our plugin for Arquillian to package the test apps as WARs. But, in theory, they can also use any other mechanism to package the apps and run the test with it, so it's probably even possible to reuse the existing Jakarta EE TCK porting package to deploy the tests on an application server.
      Of course, it would be nice to have a standard way to package a test application but even Arquillian doesn't abstract well enough here. It uses Shrinkwrap to create a specific type of archive (WAR or JAR), which might not be suitable for some runtimes.

      For Batch we don't even need to have any packaging because it's OK to have everything on the classpath for all tests. It's OK to run the tests with plain JUnit 5 and it's also OK to assemble all classes and resources in a single test app (e.g. WAR) used by all the tests. I demonstrated how to automate the latter for Arquillian with a simple plugin for Arquillian.

      I can see that for other specs, it's needed to package only a subset of the classpath for tests. Then it would be nice to have some standard and abstract way to set up the contents of the package, e.g. by some annotations or a simple API. Arquillian or any other test framework could then adopt such an API and use it to package the test application as needed (e.g. as WAR).

      But first, I'll follow up with rebasing my PR against the master branch so that we can discuss about it in the PR. When we have a good solution for Batch, we can raise a proposal on the TCK project level and propose it as a standard approach that can be easily adopted by other standalone TCKs.

      Ondro

      [1]       https://github.com/OndroMih/batch-tck/pull/3 


      st 2. 6. 2021 o 14:47 Scott Kurz <      skurz@xxxxxxxxxx> napísal(a):
      So steering back to the Batch TCK, it seems we should move forward with Ondro's:        https://github.com/OndroMih/batch-tck/pull/3 as a next step, converting to JUnit 5.
      From there a further step would be to add Arq adapter runners, etc.

      Ondro graciously offered to merge on the latest master.   (I know we've reformatted since but without formalizing via checkstyle.)    

      (Also I see we don't have a SNAPSHOT version set.. feel free anyone to improve project shortcomings like these).

      But if we're good with this as a high level plan, Ondro, can you please do that when you get a chance and reply here with the updated PR for anyone who wants to do a code review?

      Thanks,
      ------------------------------------------------------
      Scott Kurz
      WebSphere / Open Liberty Batch and Developer Experience
      skurz@xxxxxxxxxx
      --------------------------------------------------------

      Inactive hide details for arjan tijms ---05/29/2021 07:23:18 AM---Hi, On Fri, May 28, 2021 at 11:08 PM Romain Manni-Bucau <rmanarjan tijms ---05/29/2021 07:23:18 AM---Hi, On Fri, May 28, 2021 at 11:08 PM Romain Manni-Bucau <rmannibucau@xxxxxxxxx>

      From:       arjan tijms <arjan.tijms@xxxxxxxxx>
      To:       jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
      Date:       05/29/2021 07:23 AM
      Subject:       [EXTERNAL] Re: [jakartabatch-dev] Batch TCK discussion continued - Junit 5, Arquillian, etc.
      Sent by:       "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>



      Hi, On Fri, May 28, 2021 at 11:08 PM Romain Manni-Bucau <      rmannibucau@xxxxxxxxx> wrote: This is what arquillian is and it is adopted. Another api wouldnt get as much traction - tomcat wouldnt be under radar for ex - until it is done by
      Hi,

      On Fri, May 28, 2021 at 11:08 PM Romain Manni-Bucau <      rmannibucau@xxxxxxxxx> wrote:
      This is what arquillian is and it is adopted. Another api wouldnt get as much traction - tomcat wouldnt be under radar for ex - until it is done by the api creator.
      So personally i wouldnt invest in it or ewpect vendors to follow and redo arquillian already work so we would end up with another arquillian bridge i think.
      Arquillian was bringing a simpler way to write tck tests, a standard arquillian would bring a vendor neutral way, less value IMHO.

      Maybe, anyway, I just wanted to put the idea out there. Perhaps, if Red Hat would be open to it, donating Arquillian core (without the forest of extensions) to EE4J and/or the TCK project, might be worth considering.

      Kind regards,
      Arjan
             _______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev




      _______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev_______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev




      _______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev_______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev




      _______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev_______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev




      _______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev_______________________________________________
      jakartabatch-dev mailing list
      jakartabatch-dev@xxxxxxxxxxx
      To unsubscribe from this list, visit       https://www.eclipse.org/mailman/listinfo/jakartabatch-dev




_______________________________________________
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev

_______________________________________________
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev

Back to the top