Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [papyrus-rt-dev] Runtime tests [was: [Bug 469429] [runtime] [bmaher/runtime/cygwin] branch - application crashes after de-registering SAP port]

Comments inline…of course...


On 2016.11.02, at 11:06 , Ernesto Posse <eposse@xxxxxxxxxxxxx> wrote:

Comments inline.

On Wed, Nov 2, 2016 at 10:40 AM charles+zeligsoft.com <charles@xxxxxxxxxxxxx> wrote:
BTW, you referred to this test as “Barry’s model.” would that not then make him a “modeler” ;-)

Slip of the tongue. The tests mimic some models but they are not generated from models.

That was a tongue-in-cheek comment, as indicated winking smiley… ;-) <- winking smiley

 

The only downside to having target language self-coded “models" is that there is no validation of those “pseudo” models to conform to UML-RT as implemented in Papyrus-RT, past their ability to compile and run against the RTS. They would also not be portable if (when) we add other implementation languages.

Not all tests are meant to conform to UML-RT. For example, we have several unit tests which test only specific aspects of the code generator. It doesn't even make sense to talk about these as "conforming" to a language. 

Agreed. But I am a proponent of reuse and of modeling, so I want to promote both whenever I can. In the end, it is up to the development team to pick the right approach. 

I will still try to trigger discussions… (:-o) <- amazement smiley ;-)


The point is that there are tests (such as unit tests) that are not only reasonable, but even necessary and which we cannot call "UML-RT conformant", because they only tests certain aspects of either the code generator or the runtime, and we wouldn't get rid of unit tests, so it is reasonable for the project to keep tests that are not "UML-RT conformant". Although not proper "unit tests", I would classify Barry's tests in the same category. Perhaps it would be desirable if Barry turned these into proper unit tests integrated into the build, but I'm not sure this is either easy or desirable, and in any case, I think Barry's opinion on this would be important.

Agreed.

Although if Barry is the only one who can work on this, we may have a resource issue…

In any case, Simon should be involved when it comes to asking Barry to do work on anything.

 

If we decide continue to have non-modeler developers create these tests, but they may very well be limited to checking for regressions during development and could not be used to express any other quality metric against the whole of the Papyrus-RT product.

Just like unit tests: we need not just tests for the product as a whole, but also tests for parts of it, and this includes parts that the user doesn't see or experience directly. That's why Barry's tests and other unit tests are in separate folders, and not under the "models" hierarchy in the repo.

Agreed. It is always good to fully understand the expectation and the use of the results from these tests, especially when they really look like models… ;-)

And the content of the “Models” hierarchy is another topic entirely, with its mix of sample and test models…but that is a separate discussion… ;-)

 

I think that this is also is a good case for using the textual capability being built to express the runtime tests!


To the extent that you want to test full models, rather than individual aspects of the component in question, yes.

Granted, but these “hand-coded” tests look a lot like code generated from models. And even models can be created to only address a small, targeted,  subset of the runtime functionality. The question then becomes: when should tests that are hand-coded vs. code generated from small specialized models be (textual or graphical) used? Or if at all?

By the way, two other aspects related to this:
  • Tests generated from models could introduce issues from the code generator that you might (would?) not want as part of the runtime unit tests.
  • Toolsmiths who provide extensions or modifications to the runtime (independently of or in parallel with code generator creation/modification) would benefit from a set of “compliance” unit tests that are not model-based. 


 

Regards,

Charles Rivet
Senior Product Manager

On 2016.11.02, at 10:21 , Ernesto Posse <eposse@xxxxxxxxxxxxx> wrote:

These tests are in the git repo under plugins/umlrt/runtime, rather than in the actual model tests, and they are not packaged or deployed. They are intended exclusively for testing the runtime "directly", this is, they are not intended to exercise the generated code, but the runtime itself. Those have been written by hand by Barry for him to do his own testing of the runtime, so they do not necessarily correspond to generated code. They do not have a model. They are just plain C++.

It might be that it is preferable that tests have models, but not all developers are modellers. If a developer works best with code, we might not be improving the process, by forcing all tests to be models. I think it makes sense to allow for some flexibility towards developers who have their own non-model-based tests and to allow those developers to push tests into the repo that are not model-based.

--
Ernesto Posse
Zeligsoft






On Wed, Nov 2, 2016 at 8:07 AM charles+zeligsoft.com <charles@xxxxxxxxxxxxx> wrote:
We need to review the tests found in "plugins/umlrt/runtime/tests” to determine which aren’t runnable or maintainable and, therefore, need to be removed or updated.

Although it was totally justified to do this at the time, my take on this is that we really should be creating tests by modelling them and not creating them by hand.


Regards,

Charles Rivet
Senior Product Manager, Papyrus-RT product lead

Begin forwarded message:

Subject: [Bug 469429] [runtime] [bmaher/runtime/cygwin] branch - application crashes after de-registering SAP port
Date: November 1, 2016 at 15:07:03 EDT

https://bugs.eclipse.org/bugs/show_bug.cgi?id=469429
Product/Component: Papyrus-rt / runtime

Ernesto Posse <eposse@xxxxxxxxx> changed:

          What    |Removed                     |Added
----------------------------------------------------------------------------
                CC|                            |eposse@xxxxxxxxx

--- Comment #2 from Ernesto Posse <eposse@xxxxxxxxx> ---
I can't reproduce.

I tested SAPs on Windows 7 with Cygwin, more precisely the Services model in
the git repo and it works as expected.

I am unable to build Barry's custom (hand-written) tests under
plugins/umlrt/runtime/tests on Windows, and there are no instructions on how to
do it, hence, I assume that Barry's model may be outdated, and I would suggest
that he looks into it.

--
You are receiving this mail because:
You are on the CC list for the bug.
You are watching the assignee of the bug.

_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev

_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev
_______________________________________________
papyrus-rt-dev mailing list
papyrus-rt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/papyrus-rt-dev


Back to the top