Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartabatch-dev] Do you plan to move your TCK tests out of the Platform TCK project for Jakarta EE 10+?
 
I'm sorry I've had a hard time responding.   For one, the question of whether we want to switch to a different "test harness" hasn't been pinned down.  On top of that, I'm not too familiar with the details of either JTHarness or Arquillian.
 
Let me make an attempt though of starting with what I do know about Batch and both the standalone / Platform TCKs, and running through the issues your proposal doc raies.
 
Platform TCK benefits
 
First, there's the set of features that the Standalone Batch TCK completely lacks.  Without solving these issues in detail, I'm not sure what we've really accomplished with this proposal
 
1. Packaging & Deployment of Tests to Web/EJB Container
 
The platform TCK wraps the tests in WAR/EJB "vehicles" (the term, IIRC).   Though the tests don't particularly exercise any subtleties or finer scoping of EE packaging (they can do their validation jobs just as well run in one single WAR vs. a more finer-grained packaging), we do have compatible impls, e.g. Open Liberty that you can't just embed into the test JVM (i.e. can't just add them to the classpath).   Packaging and deploying the tests into WAR/EJB format is an important component to being able to run the tests.
 
2. Single setup with other TCKs
 
This isn't an issue really from the Batch POV, but rather from the Jakarta Platform implementer and certifier roles.    This person doesn't want to have to go through 20 different custom setup steps and provide the same type of config values in 20 different formats.   This is not something I've ever personally done;  I'm sure you are way more familiar than me, but my impression of the effort is that this is a key part of the value the aggregate Platform TCK brings to the table.
 
Platform TCK negatives
 
There are a set of contributors that could potentially grasp the project at the standalone Specification level and make a meaningful contribution that could be discouraged by having to understand complications introduced by the full platform TCK (and we want to make it easier).
So these are some negatives from the batch POV.

1. Custom technology for: a) authoring  b) execution (not just `mvn verify`) creates a learning curve for new devs
 
2. Duplication of source - Because we wish to provide the "SE Profile" and more common `mvn verify` experience, we maintain the standalone TCK in addition to forking the artifacts into the Platform TCK.  (We run a 'sed' type of find/replace which creates its own code maintenance / debugging challenges)
 
Other / Misc.
 
Then there's a handful of other items the proposal touches on.  Not that these are all trivial but I don't see any as fundamentally gating the discussion.   Here's a quick take where we stand on some:
 
1. "Profile" (subsets of tests)
The Standalone TCK is essentially a subset of the Platform TCK with the divergence being the tests that run in EE only, e.g. the tests that use global transactions.  It is easy to use TestNG to do this.  I'd imagine any common way of making profiles would be fine for Batch.    I think the key question is whether it is approporiate for there to be a platform vs. non-platform subset/profile of tests maintained by the spec project ?
 
2. Reporting
Again, I'm not one who has ever run the full platform TCK, but my impression is we should allow some flexibility for individual spec components of the aggregate.  This could mean that someone running the tests has to know their way around the logs / results structures of individual specs, and also that there is some custom logic needed to aggregate it all together.   On the other hand, maybe if getting to the point of being able to do a single, common deployment/setup means that we are close enough that we could define a single reporting mechanism...well, that might make sense and I'd be OK with that too.   
 
3. properties files?
While we have some batch TCK-specific properties, they are tightly coupled to the fine details of the batch tests (e.g. they allow you to tweak wait times to balance test execution time vs. failures because you didn't wait long enough, etc.).
 
4. sigtest
We haven't talked much about signature tests.  I'll note we are currently duplicating these too.  I'm sure these would be easier to collapse than runtime tests.
 
5. test assertionids/Javadoc
Our effort here has been of mixed quality.  In some areas we are OK, in some kind of weak.  As long as we can grandfather in our starting point, it would be easy enough to change to any particular format (through whatever annotation,etc. mechanism).   
 
6. maven artifacts
We already release our Standalone Batch TCK tests but I don't think it helps anyone much without solving some of the common setup/harness etc. problems.
 
Conclusion
 
So, Scott M., hope that moves the discussion along though that might not have been exactly the answer you were looking for.  I also realize I might not have kept up with other ML conversations.    If there's another forum you'd hope I'd bring these points up in, feel free to let me know.
 
Thanks,
------------------------------------------------------
Scott Kurz
WebSphere / Open Liberty Batch and Developer Experience
skurz@xxxxxxxxxx
--------------------------------------------------------
 
 
----- Original message -----
From: Scott Marlow <smarlow@xxxxxxxxxx>
Sent by: "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>
To: jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
Cc:
Subject: [EXTERNAL] Re: [jakartabatch-dev] Do you plan to move your TCK tests out of the Platform TCK project for Jakarta EE 10+?
Date: Fri, Jun 25, 2021 3:31 PM
 

Will this work be easier since the Jakarta Batch team already manually syncs the Platform TCK with https://github.com/eclipse-ee4j/batch-tck?? 

Is there anything in https://github.com/eclipse-ee4j/jakartaee-tck/tree/master/src/com/ibm/jbatch/tck that isn't already in https://github.com/eclipse-ee4j/batch-tck??

We still will want to run the jbatch tests from the Platform TCK but that could be done via Maven artifacts that the batch-tck repo can generate.

Thoughts?

On 6/22/21 3:37 PM, Scott Marlow wrote:
[1][2] is a proposal for how to produce Standalone TCKs by extracting TCK tests from the Platform TCK.  The process for extracting Standalone TCK tests is described via the GUIDELINE FOR EXTRACTING TCK section [3].

Do you plan to move your TCK tests out of the Platform TCK project and into your Specification project?  If yes, you will publish your Standalone TCK artifacts to Maven so that the Platform TCK can consume your tests.  Some Standalone Specification TCKs will also be consumed by the Core Profile TCK (and possibly other EE profiles).

From the [1] feedback received so far, both TestNG + Junit5 are popular test frameworks.  Maven is the likely choice to switch to using in the EE Platform TCKs.

The Platform TCK community needs to take a deep look at the Platform TCK internals and consider what the Jakarta EE 10+ Platform TCK architecture will look like.  But I think first, we should do some incremental hacking to bring in some Maven based testing as suggested by jakartaee-tck/issues/51 [4].

Scott

[1] https://docs.google.com/document/d/1yDXTUUULUrFrUi1DV_9OcBKIiZLVxrZkA38MMvYdP-U/edit#
[2] https://github.com/eclipse-ee4j/jakartaee-platform/issues/333
[3] https://docs.google.com/document/d/1yDXTUUULUrFrUi1DV_9OcBKIiZLVxrZkA38MMvYdP-U/edit#heading=h.gt76s6wqkrva
[4] https://github.com/eclipse-ee4j/jakartaee-tck/issues/51
_______________________________________________
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev
 


Back to the top