Hi everyone,
RCPTT needs a
product to run the integration tests.
Furthermore, RCPTT does not aim to replace Junit Tests. This is a way to test a product “functionally” speaking.
So Yes, the RCPTT is triggered after the Product build.
So if we decide to build the Product once a day or once a week. The RCPTT will be launched accordingly, once a day, or
once a week.
I have recently created a Gerrit Job for RCPTT, but it is triggered in case of changes on RCPTT tests. And tests are done
on the last built Product.
Running RCPTT on each modification is imho meaningless, for 3 reasons:
1)
RCPTT are integration tests not unit tests so it should take the whole application into account
2)
Junit tests are already launched on the Gerrit of each component to prevent basic regression.
3)
RCPTT would make the Gerrit jobs very much longer.
Knowing this, we should define a frequency for building the product and thus the RCPTT.
So this is the compromise to make:
1)
Higher is the frequency, bigger is the disk and Hudson usage, less backup we could store.
2)
Lower is the frequency, later the regression can be detected, higher would be the Hudson availability.
Regards
Céline
De : papyrus-rt-dev-bounces@xxxxxxxxxxx [mailto:papyrus-rt-dev-bounces@xxxxxxxxxxx]
De la part de charles+zeligsoft.com
Envoyé : lundi 17 octobre 2016 14:34
À : papyrus-rt developer discussions <papyrus-rt-dev@xxxxxxxxxxx>
Objet : Re: [papyrus-rt-dev] [Suggestion for 0.9.0] Split p2 and Product builds
I’m a bit confused, so please help me understand the problem.
In addition to Peter’s concerns, below, about the testing aspect, this raises some other questions:
Are we currently running tests using RCPTT recordings or test suites?
If so, what is used to run these tests? The existing Tester Setup (as per wiki)? The existing User Setup (as per wiki)? A developer runtime environment?
Is there an assumption that RCPTT tests would generate different results depending on how the Papyrus-RT instance is created/deployed? In particular, for the same user type, RCP vs. Oomph-base installations?
This would have some major implications to both our testing and support approaches!
I think this sounds as a good idea in general. I have one concern/question though, related to the QA aspect with RCPTT, and when and thus how fast we can detect regressions by the
RCPTT test suites.
I am not sure if this change of breaking out the RCP build will impact the RCPTT testing. Is the RCPTT test suites executed against the latest RCP build, which then will be executed
more seldom? Or is is some other build that is used for executing the RCPTT test suites?
I have already before raised a question whether the RCPTT test suites should be executed also for Gerrit changes, to detect that the Gerrit change causes regressions in the RCPTT
test suites (or requires updates of the scripts, e.g. when we do changes in the UI). As I have understood it, currently we do not detect regression by the RCPTT test suites until the Gerrit change actually have been merged (which could be considered to late).
Or do we expect the RCPTT test suites to only detect regressions once a week (based on the new RCP build run once a week)? What will the process be to make needed changes? Will we
track that by new Bugzillas?
On 14 October 2016 at 19:07, Christian Damus <give.a.damus@xxxxxxxxx> wrote:
I agree with all of Céline’s arguments and Charles's.
On 14 October, 2016 at 12:06:37, charles+zeligsoft.com (charles@xxxxxxxxxxxxx)
wrote:
It think that’s a good approach.
The RCP is really only necessary on an external facing site for releases and milestones, so we don’t need to build them that often, keeping in mind that there would be little need at the beginning of a release (milestone)
cycle, as we can do our testing using the tester installation, and more at the end when we need to validate the RCP, along with the Oomph-based installations.
Charles Rivet
Senior Product Manager, Papyrus-RT product lead
Hi everyone,
During this release 0.8.0, I had a better overview of the user needs in terms of deliverables.
As you could see the Papyrus-RT-Master-Product job, takes long time and is very big (800 MB each build).
So I suggest to split it into 2 builds, the p2 which is still build every 15min and the RCP itself (which is the haviest part) to build and sign it every week for example.
Of course it could be run on demand if needed like it did so far.
If you have any remarks on this... let me know ;)
regards
Céline
[cid:image001.jpg@01D22642.65398090]
<winmail.dat>_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
