Before giving some code pointers i'd like to mention I agree TCK are very low priority IMHO (users don't like that part anyway in general because it does not bring any feature and it should be global for Jakarta probably).
This does not prevent us to put tags on them (@Tag, it is like categories for previous junit) to mark them as being EE, standalone, CDI, etc and then let the launcher module have multiple executions or so.
But high level this enables to write test once, run them in all possible environments without having to do anything not standard in tests themself.
100% sensible. I think most contributors will understand if we
just want to focus on Jakarta EE 9 right now and only do other
stuff if it is not a distraction. If needed, I think we can easily
redirect people to other specifications that also probably need
Jakarta EE 9 execution help right now. So the real question is,
can we productively get Nishant and other such contributors
engaged with the Jakarta EE 9 effort or something less intrusive
than the TCK technology choice?
By the way, I appreciate the candor, focus and transparency. I am
sure others do too.
Reza Rahman
Jakarta EE Ambassador, Author, Speaker, Blogger
Please note views expressed here are my own as an individual
community member and do not reflect the views of my employer.
On 3/19/2020 8:20 PM, Kevin Sutter
wrote:
Since you
asked,
Reza, I'll pipe in... :-) Of course, I've been talking with
Scott offline about this discussion and the work required for
Batch in
general. Although I would like to figure out ways of getting
more
people involved with Batch, I would caution against spending too
many cycles
on this TCK question when there is still so much to do for the
Jakarta
EE 9 deliverable in general. Specific for Batch, we still need
the
Specification completed. As Scott's most recent reply outlined,
we
still need the TCK to be updated and ready to be plugged into
the Platform
TCK. And, we need a Compatible Implementation available
somewhere
for testing purposes. In my mind, all of these items are more
important
than attempting to figure out whether a standalone TCK should be
using
Arquillian or JUnit5 or some other framework. Once we start to
get
a handle on the completion of the Spec, API, TCK, and CI, then
we can go
back to this question of how to make a standalone TCK workable.
That's my
view.
But, of course, I do have a larger goal in mind as the Jakarta
EE
9 co-release lead... :-)
---------------------------------------------------
Kevin Sutter
STSM, MicroProfile and Jakarta EE architect @ IBM
e-mail: sutter@xxxxxxxxxx Twitter: @kwsutter
phone: tl-553-3620 (office), 507-253-3620 (office)
LinkedIn: https://www.linkedin.com/in/kevinwsutter
To be honest I would like to hear
what
some of the others folks here think so far. I count 33
subscribers to this
list and we have hardly even had 10 opinions. Surely other
people have
some thoughts on how Jakarta EE project TCKs should look like?
Seems to
me like a pretty important topic?
Reza Rahman
Jakarta EE Ambassador, Author, Speaker, Blogger
Please note views expressed here are my own as an individual
community
member and do not reflect the views of my employer.
On 3/19/2020 7:36 PM, Scott Kurz
wrote: You can see me taking some time to
get
my head around this thread. I'm realize one area in which I
misspoke and
possibly misled was using terms like "SE mode".
I also should emphasize this is a post-Jakarta EE 9 discussion !
I'm trying
to set us up to work on new function, but for EE 9 we will go
with the
status quo described below.
---
OK, so a better way to think of what we have right now, in
Jakarta EE 8,
is that we allow certification in one of two ways:
1. Platform TCK - The batch tests within the Platform TCK (https://github.com/eclipse-ee4j/jakartaee-tck/):
2. Standalone TCK - Built from this batch-tck project,
(available
at https://projects.eclipse.org/projects/ee4j.batch/downloads)
Problems with the two-TCK status quo:
The batch-tck copies of the test are the true
"upstream".
This is the repo that holds the issues and PRs that we primarily
discuss
on the mailing list.
I'm not sure if everyone realizes this, (since this was
previously done
by IBM and handed directly to Oracle/CTS), we actually do a
find/replace
operation to transform these tests into the Platform TCK format
(this was
previously Oracle's "CTS" ). The "transform" complicates
debug; one has to understand the transform to, say, work
backwards to a
batch-tck change. We also need to limit to the common
denominator
of what's available in Platform TCK (that's why we wrote our own
assert
util instead of just using JUnit assert, say).
In theory some of this could be mitigated or improved by
engaging with
the Platform TCK and making it more user-friendly (for our
needs). There's
a non-trivial learning curve there, this project has its own
conventions
and features and you'd possibly have to convince other
components to accept
change they don't see the need for.
So I'm starting from the perspective of wanting to get rid of
the Platform
TCK, and to make our Standalone TCK usable enough that app
servers, e.g.
Glassfish, Open Liberty, can easily certify against our
standalone TCK.
Problems with the current standalone TCK:
The current "plug points" built into our standalone TCK only
include the JobOperator impl itself (via the BatchRuntime's
use of ServiceLoader) and a "waiter" to allow strategies
other than polling, the default.
There is no plug point for deployment, or even for "setup". A
batch implementation like Open Liberty (which I work on) is not
"embeddable"
in the integration test JVM. To bridge the execution of the test
jobs and
execute them in a separate app server JVM would require some
custom code,
unique to each server.
Ideas:
1. Arquillian (my original idea)
Again, I haven't really worked with Arquillian but the problems
with our
standalone TCK seem to be exactly what Arquillian can solve, and
this seems
to be the approach existing MicroProfile TCKs and a few EE TCKs
are using.
We could use existing adapters/extensions, and stick to proven
paths (IIUC,
MicroProfile TCKs use TestNG rather than JUnit to integrate with
Arquillian
better).
One other point that's been raised: I'm not sure how we recreate
the "vehicles"
of Platform TCK... e.g. run test within a servlet context vs.
running within
an EJB context. (Though the batch spec says basically nothing
about one
EE context vs. another, so doesn't seem too valuable, but
perhaps could
be one day possibly?)
2. JUnit 5 (Romain)
I can appreciate some of Romain's points and criticisms of
Arquillian,
but after reading that a couple times, I'm still not sure what
he's suggesting
looks like. Romain, would you be interested in prototyping or
show a proof
of concept for a couple tests so that we could understand and
evaluate?
3. Custom
We could do something altogether different. What if our
"deliverable"
includes a WAR, within the test zip, packaging the same tests in
a different
way. The instructions for the WAR say: "go create these two
datasources
pointing to these two DB tables and set up your job repository
however
you want. "
I think there's a case to be made that this is a valid TCK. The
problem
is, someone certifying against the platform doesn't want 20
completely
different TCKs, with completely different setup steps. If two
types are
OK, I guess a case could be made for three, but this really sets
us up
to be an outlier from the overall platform perspective..a
problem child
:)
---
Since Romain and Cheng both spoke up against Arquillian, I'm not
going
to suggest anyone prototype that approach..until we give a
chance for other
suggestions. But I do think we need some kind of POC of another
approach
to take the next step and move the conversation forward.
Thanks,
------------------------------------------------------
Scott Kurz
WebSphere Batch and Developer Experience skurz@xxxxxxxxxx
--------------------------------------------------------
Le mer. 18 mars 2020 à 21:28, reza_rahman <reza_rahman@xxxxxxxxx>
a écrit : My suggestion is that we engage the
Arquillian
team here. While Arquillian does have weaknesses, I think
adoption into
Jakarta EE TCKs should be important enough such that the
development team
should at least commit to working with us to solve issues
together. I imagine
you can easily reach that team but if you need help just let me
know.
Hmm, they were not planning junit5 support some months ago "by
design",
not sure it changes and quarkus to quote only this one tends to
validate
that for future.
We must keep the opening and stay friendly to people not used to
ee IMHO.
No tool is a golden hammer. I don't think we need to adopt
Arquillian for
every test necessary. However, in cases where it is clearly
helpful creating
yet another one off solution with likely it's own problems does
not make
much sense to me - for this project or a given Java EE
enterprise application.
Why would you create a new one without Arquillian ;)?
All servers are getting junit5 integrations just to be up to
date + adding
extensions dynamically is quite easy so no need of anything new
for JBatch.
Just plain junit5.
It will also enable to open more tck rather than keeping using a
specific
heavy stack imho.
Maybe next step is to identify tests requiring "inline"
packaging
(it is the only arquillian feature needing/providing an api, all
others
are not needed *in tck* and specs).
Guess none are in jbatch stack - or none cant be rewritten to
not need
it.
Reza Rahman
Jakarta EE Ambassador, Author, Blogger, Speaker
Please note views expressed here are my own as an individual
community
member and do not reflect the views of my employer.
Sent via the Samsung Galaxy S7, an AT&T 4G LTE smartphone
-------- Original message --------
From: Scott Kurz <skurz@xxxxxxxxxx>
Date: 3/18/20 7:58 AM (GMT-05:00)
To: jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
Subject: Re: [jakartabatch-dev] TCK directions / form factor ,
e.g. Arquillian
- opinions wanted Romain,
Forgive me if I repeat myself, it's not to drown out your
argument, but
to say it differently to try to understand.
...
So backing up, in suggesting Arquillian, I have in mind a
developer new
to the Jakarta Batch. Maybe they ask a question about some
behavior on
the mailing list, we discuss, and then say, "we should add a TCK
test
to clarify, would you like to volunteer to write that test?".
At this point, it's too much of a hurdle to expect that they
learn the
details of the Jakarta EE platform TCK in order to verify their
test. Arquillian,
even with its flaws, is something that's out there that they can
find examples
of, and, IIUC, there are existing "adapters" (I'm not using
correct
terminology) for app server products and also SE+CDI "mode",
Now, if the flaws are great enough, and you're raising some
concerns, this
is a moot point because Arquillian just doesn't "work"
correctly.
But trying to understand where the trouble lies... people use
Arquillian...
I would say the Batch TCK is relatively simple, deploy once,
run, assert/validate.
Sure we have some timing issues and we poll a lot but I see that
would
apply in any solution.
Yes, to the person with the role of certifying the entire app
server against
the entire platform TCK, we have potentially added more work. By
pulling
Batch out of the Jakarta EE platform TCK, we've created one more
"special"
TCK ("standalone" if that term isn't too confusing/overloaded).
But I'm OK with that because: 1) This is a more specialized
role.. not
too many people will do this, maybe one per product.. and this
person already
may have to deal with some quirks of each API within the
platform TCK 2)
We're not branching out in 20 different directions, just two:
Arquillian
vs. Jakarta TCK...hopefully there's some commonality / reuse
among the
set of Arquillian TCKs.
---
That all said, I don't think I really understand what you're
proposing
though.
If so, my objection is that we have to make source updates
twice, plus
the ultimate validation of these updates requires working with
the complicated
Jakarta EE platform TCK.
Thanks,
------------------------------------------------------
Scott Kurz
WebSphere Batch and Developer Experience skurz@xxxxxxxxxx
--------------------------------------------------------
Romain Manni-Bucau ---03/18/2020 02:45:30 AM---Hmm, think I see
reality
differently Reza, let me expand why: 1. Arquillian was de facto
standard
fo
Hmm, think I see reality differently Reza, let me expand why:
1. Arquillian was de facto standard for EE testing something
like 5-7 years
ago but it kind of shut down. Today it is poorly maintained and
the junit
5 big bang helped a lot to make it abandonned by users. All
vendors also
slowly move to it.
2. EE tck never fully migrated to arquillian except bval, cdi
and a few
others. Most of the dozens of thousands tests are still "ee
custom".
The strength of arquillian is no more its ecosystem - which is
often inconsistent
and not well compatible - or test facing api but its container
abstraction
only - and this is usable standalone. We could envision to use
it directly
but it brings very few for JBatch and most important point for
me is to
stay smooth for ee consumers. If they want to consume tck they
will mainly
integrate jakarta tck so better to not force to do an arquillian
solution
which is often broken in embedded/standalone mode (currznt way
to deploy,
even EE).
JSON-P/B tck - for ex - had the advantage to have a process per
test and
just require to have the impl in the classpath so execution was
representative
even in standalone.
Now back to JBatch, what do we need?
CDI? Be able to have a CDI container with some beans and not
some other?
SeContainer is about that and is trivially backable by
arquillian container
impl if *some vendors* want. Same applies for the "auto war
deployment",
vendors can do it with arquillian just implementing a custom
deployer filtering
the classpath from their client libs.
So at the end I think JBatch should stay standalone and CDI
centric, even
for its tests.
Le mer. 18 mars 2020 à 05:00, Reza Rahman <reza_rahman@xxxxxxxxx>
a écrit : I really prefer we not do anything
one
off and go the Arquillian approach. The biggest benefit to
maintaining
uniformity is that contributors across Jakarta EE technologies
will not
need to re-learn the same thing.
The reality also is that Arquillian remains the de-facto testing
solution
for Jakarta EE. It is unfortunately still far from perfect but
workable.
I would like to believe we could adopt Arquillian across Jakarta
EE and
help smooth out the wrinkles in Arquillian itself in the
process.
Reza Rahman
Jakarta EE Ambassador, Author, Speaker, Blogger
Please note views expressed here are my own as an individual
community
member and do not reflect the views of my employer.
On 3/17/2020 2:31 PM, Scott Kurz wrote: So one suggestion in the https://github.com/eclipse-ee4j/batch-api/wiki/Ways-To-Contributewas the idea that we could/should
convert the TCK to Arquillian (like the
JSON-B TCK did).
Let me just back up and walk through my thought process, since I
know there
was no discussion about that.
And I'd like to solicit opinions about what to do
-----
The problems I see with 1.):
- there's a learning curve with the jakartaee-tck as it's its
own, custom,
kind of complex thing
- how do we keep our "SE" form factor? Not all impls can/will
leverage this but some may find it quicker and advantageous to
execute
tests in this "mode". For batch 1.0 we actually did an Ant
transform
(find/replace), which seems kind of primitive.
As far as 2., well, it wouldn't be too hard to have a second
project take
all the test classes, packaging them in a WAR, and putting some
sort of
JUnit and/or Test NG suite, "wrapping" them.
That would preserve the existing SE mode, and enable an EE mode
that avoids
the jakartaee-tck complexities.
But from the overall platform perspective, there's value in
keeping things
simpler and having a smaller number of different directions
among the individual
component projects. It makes it that much easier to certify
against the
whole platform.
So if we take for granted that the JSON-B, and beanValidation
TCKs have
proven the concept of and provided a pattern for a relatively
simple Arquillian-based
TCK, and that it faciliates SE, non-app server execution mode
(which I
admit I haven't looked into in detail, not being too familiar w/
Arquillian),
then, rather than going off in a new direction (that we don't
have yet),
I'd like to try to follow these TCKs, and make a second group or
pattern
of Jakarta TCKs using Arquillian.