[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [jakartaee-platform-dev] TCK tests in the same repo as API and Spec
|
The original, very rough, idea was to convert the build of the CTS
to use Maven (no small task), then use maven to pull in these test
artifacts from other TCK projects. There's a ton of details to be
worked out there, but it seems like a workable approach.
Kevin Sutter wrote on 2/4/20 6:15 AM:
Dmitry,
The one
problem
with this approach is how do we keep the two test buckets in
sync? Is
there any way that we could have a single source repository for
the testcases
themselves? And, they get cloned into the respective test
bucket
is attempting to test with them? I fully understand that the
test
harnesses and frameworks are different, but does that preclude
the use
of common testcase source? Or, maybe there's some other way to
ensure
that these do not diverge?
---------------------------------------------------
Kevin Sutter
STSM, MicroProfile and Jakarta EE architect @ IBM
e-mail: sutter@xxxxxxxxxx Twitter: @kwsutter
phone: tl-553-3620 (office), 507-253-3620 (office)
LinkedIn: https://www.linkedin.com/in/kevinwsutter
From:
Dmitry
Kornilov <dmitry.kornilov@xxxxxxxxxx>
To:
jakartaee-platform
developer discussions <jakartaee-platform-dev@xxxxxxxxxxx>
Cc:
Kevin
Sutter <sutter@xxxxxxxxxx>
Date:
02/04/2020
04:52
Subject:
[EXTERNAL]
Re: [jakartaee-platform-dev] TCK tests in the same repo as API
and Spec
We should consider job Andy’s done
as
a PoC. It should be merged to JSONB repository and Yasson team
should adopt
it without giving up TCK testing using CTS. TCK team should
review it and
recommend it (or not recommend) it to other project as a
template for Jakarta
EE 10 work. I don’t want a situation that splitting is
artificially blocked
because no-one from TCK team wants to do it.
Andy’s effort doesn’t affect Jakarta
EE 9 release. As I was saying before, JSONB TCKs in CTS
repository will
be used for certification. TCKs in JSONB repository can be used
for preliminary
compatibility testing. It’s much easier to use, it works much
faster and
it’s easier to create Jenkins jobs with these tests.
-- Dmitry
On 4 Feb 2020, at 04:54, Bill
Shannon
<bill.shannon@xxxxxxxxxx>
wrote:
I think it's fine for people to do
experiments,
ideally in their own personal forks so there's no confusion as
to its relationship
with the official version.
What I don't want is for (e.g.) Andy to get it to the point that
it's good
enough for him, he decides he's done and merges it in to the
official repos
for his projects, and the rest is left for the rest of us to
figure out.
I'm fine for not everything to be done before some things are
merged, but
I want to see a plan that the jakartaee-tck project
committers agree
to for how we're going to get to the desired end point.
Among other things, we need to know that the plan will get us to
a working
platform tck in time for a corresponding platform release.
And it would be best if the plan didn't have us running both the
"old"
and "new" TCKs in parallel for releases for some non-trivial
time.
This is a big project, so we're going to have to make some
compromises,
but that doesn't mean giving up on any sense of planning. An
agile
approach is good, but we need confidence that we'll be able to
see the
endpoint.
P.S. I'm also fine with adding lots more committers to the
jakartaee-tck
project if that helps us make progress more quickly in the short
term.
There should be few impediments to spec projects updating their
TCK
tests within the existing framework. They can start with PRs
against
the jakartaee-tck repo and become committers as needed.
Kevin Sutter wrote on 2/3/20 5:40
AM:
Jan,
You bring up some excellent points. Having a well thought out
plan
before jumping in with both feet would be nice. But, on the
other
hand, we don't want to stifle the excited effort that Andy is
demonstrating.
It's a tough line. Maybe we need to allow one or two teams
to experiment with the processes and see what it takes. And,
then
we can take those experiences and develop a plan that works
across all
the projects.
---------------------------------------------------
Kevin Sutter
STSM, MicroProfile and Jakarta EE architect @ IBM
e-mail: sutter@xxxxxxxxxx Twitter:
@kwsutter
phone: tl-553-3620 (office), 507-253-3620 (office)
LinkedIn: https://www.linkedin.com/in/kevinwsutter
From: Jan
Supol <jan.supol@xxxxxxxxxx>
To: jakartaee-platform-dev@xxxxxxxxxxx
Date: 01/31/2020
15:02
Subject: [EXTERNAL]
Re: [jakartaee-platform-dev] TCK tests in the same repo as API
and Spec
Sent by: jakartaee-platform-dev-bounces@xxxxxxxxxxx
The discussion on the various email lists is quite extensive,
and I
admit I do not follow every thread that is discussed. But I am
waiting
for the plan from any of the committee we have since the
transition to
Eclipse Foundation. And I am not aware about any clear
statement about
what the final goal is. For start:
- Is the splitting really what is recommended?
- Should the Platform Jakarta EE TCK still be available in the
future
after the split?
- Should the interoperability between Specs tests (the tests
that
require multiple Specifications to cooperate as it is
described in one
or the other specification, the tests that used to be part of
the full
CTS only) still be part of the combined (?) Jakarta EE TCK?
- Is there a recommended framework to be used in the future
with the
TCKs (such as the existing JavaTest + perhaps Arquillian
similarly to
Microprofile + ?)?
Those should be goals presented by the Jakarta representatives
before
the TCK repo is started to be ripped apart. Not on email, but
clearly on
the main Jakarta informational page (whatever that is now).
-- Jan
On 31.01.2020 20:34, Lance Andersen wrote:
> I also agree with Bill. There has to be thought given
to CTS
> structure first IMHO so that there is consistency in the
test
> structure to make it easier to pull in the standalone
tests and
> where the platform specific tests are maintained for the
standalone
> technologies.
>
> This will take some time to flush this out. During the
Java
EE days,
> we did find it easier to work out of one master
workspace, but part
of
> that was due to the fact there was one team primarily
responsible
for
> the test development for all of the Java EE technologies.
>
>> On Jan 31, 2020, at 2:01 PM, dmitry.kornilov@xxxxxxxxxx
>> <mailto:dmitry.kornilov@xxxxxxxxxx>
wrote:
>>
>> Bill, you are right. On the other hand the process of
splitting
has
>> to be started somehow. It will never start with an
assumption
that
>> some developer with TCK knowledge will take an
initiative and
start
>> working on it. There are not too many developers like
this and
no one
>> wants to take a risk and responsibility.
>> On the other hand I don’t think that it’s a right
time to do
it now
>> as part of Jakarta EE 9 release. I already proposed
thatwe will
not
>> move JSONB TCK tests from CTS yet. We will do it
after Jakarta
EE 9
>> is released. We will keep them in sync until that
time. Users
may use
>> TCK tests in JSONB repo as more convenient way of
checking
>> compatibility. But it cannot be used officially for
compliance
>> testing for now.
>> -- Dmitry
>> *From:*jakartaee-platform-dev-bounces@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev-bounces@xxxxxxxxxxx><jakartaee-platform-dev-bounces@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev-bounces@xxxxxxxxxxx>>*On
Behalf
>> Of*Bill Shannon
>> *Sent:*Friday, January 31, 2020 7:06 PM
>> *To:*jakartaee-platform developer discussions
>> <jakartaee-platform-dev@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev@xxxxxxxxxxx>>;
Andy Guibert
>> <andy.guibert@xxxxxxxxx<mailto:andy.guibert@xxxxxxxxx>>
>> *Subject:*Re: [jakartaee-platform-dev] TCK tests in
the same repo
as
>> API and Spec
>>
>> This has been discussed numerous times over the last
3 years.
No one
>> disagrees with the goal of splitting up the TCK repo.
Whether
the
>> individual spec TCKs are in the same repo as the API
classes or
in a
>> different repo is just a detail. The challenge is in
splitting
up
>> the existing TCK repo such that the resulting TCK for
each spec
is
>> functionally identical to the existing standalone
TCKs for the
specs,
>> and that the platform TCK is somehow created by
combining all
the
>> individual TCKs to produce a new platform TCK that is
functionally
>> identical to the existing platform TCK.
>>
>> Needless to say, this is not a small job.
>>
>> No one with deep knowledge of the existing TCKs has
come forward
with
>> a detailed plan for how to achieve the above.
Without that,
we're
>> all just wishing and hoping.
>>
>> And I strongly encourage you to*not*just start
hacking on the
TCK to
>> create what you want for your spec. We need to solve
the
larger
>> problem and a bunch of uncoordinated hacks to
individual TCKs
will
>> not get us there.
>>
>> Thanks.
>>
>> P.S. You should be able to find some of the previous
history
of this
>> discussion in the jakartaee-tck-dev mailing list.
>>
>> Andy Guibert wrote on 1/30/20 10:45 AM:
>>> Hi all,
>>> Currently all of the Jakarta EE TCK tests are
housed in one
big repo
>>> and they use a custom test framework from the
Java EE days:
>>> https://github.com/eclipse-ee4j/jakartaee-tck
>>> It is more convenient to have the TCK tests in
the same
repo as the
>>> API and spec docs because as the technologies
change over
time all 3
>>> parts (tck/api/spec) can be updated in the same
PR. Additionally,
>>> implementations can then consume the TCK tests as
Maven artifacts
>>> and run/verify that they pass the TCK. This is
what MicroProfile
has
>>> done and it works very well.
>>> As an example, I've started this off with the
JSON-B TCK test
here:
>>> jsonb-api pr: https://github.com/eclipse-ee4j/jsonb-api/pull/221
>>> yasson (impl) pr: https://github.com/eclipse-ee4j/yasson/pull/379
>>> Just wanted to share this with the wider dev
community and
encourage
>>> other specs to follow suit as time permits.
>>> Cheers, Andy
>>> -- IBM
>>>
>>>
>>> _______________________________________________
>>> jakartaee-platform-dev mailing list
>>> jakartaee-platform-dev@xxxxxxxxxxx <mailto:jakartaee-platform-dev@xxxxxxxxxxx>
>>> To change your delivery options, retrieve your
password, or
unsubscribe from this list, visit
>>> https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
>> _______________________________________________
>> jakartaee-platform-dev mailing list
>> jakartaee-platform-dev@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev@xxxxxxxxxxx>
>> To change your delivery options, retrieve your
password, or
>> unsubscribe from this list, visit
>> https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif><http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance
> Andersen| Principal Member of Technical Staff |
+1.781.442.2037
> Oracle Java Engineering
> 1 Network Drive
> Burlington, MA 01803
> Lance.Andersen@xxxxxxxxxx<mailto:Lance.Andersen@xxxxxxxxxx>
>
>
>
>
> _______________________________________________
> jakartaee-platform-dev mailing list
> jakartaee-platform-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password,
or unsubscribe
from this list, visit
> https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe
from this list, visit
https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe
from this list, visit
https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe
from this list, visit
https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev