Re: [wtp-releng] More on JUnit's policy and principles

[David M Williams <david_williams@xxxxxxxxxx>]

Sure, notes all around when declared.
But so far, our JUnit's are not mature enough, or comprehensive enough,
to indicate "regression free" to the general community. 
That's why I think they currently only
serve a role to committers, to let them know of problems. 

And, again, the "yank" is
only a last resort if the bug can not be fixed ... to fix is always the
first choice! 
And ... it's sad, but some of the failing
JUnit's have been failing for quite a few days before anyone even 
started to investigate them. If there
is a project process problem to solve, it is that one. 

John (thanks for volunteering :) ...
can I ask you to monitor the day to day JUnits and make sure teams are
addressing failures? 
I used to do that, but am finding I
should not be on that critical path. 
Long term, I'll investigate better notification
to specific component leads if their component has 
failures .. but, that's not going to
happen over the next few weeks. 

Thanks,

John Lanuti/Raleigh/IBM@IBMUS
Sent by: wtp-releng-bounces@xxxxxxxxxxx

01/26/2007 09:20 AM

Please respond to
Webtools releng discussion list <wtp-releng@xxxxxxxxxxx>

To	Webtools releng discussion list <wtp-releng@xxxxxxxxxxx>
cc
Subject	Re: [wtp-releng] More on JUnit's policy and principles

I'm a bit skeptical about the blanket yanking of failing JUnits which are
actually functioning properly and detecting real problems.

David - "And, it's my belief it's actually a bad practice to leave
'known failing tests' in the suite, since it teaches everyone to ignore
nonzero numbers, since they are often nonzero. "

Well, one could certainly argue doing the opposite is just as bad.  Everyone
will ignore zero numbers because they are always zero because WTP always
removes any failing test. 

So, neither solution is really ideal in my mind.  Certainly, I think
if a JUnit is pulled, then there should be a build note explaining which
JUnit and why it was pulled.  Or on the contrary, if we leave a failing
JUnit in, and still want to declare, we should add a build note describing
why the failing JUnit is still failing.  I see the platform with failing
JUnits quite often.  I'm not sure there is precedence for this test
removing. 

I guess I could go either way, but I'd like to see us use the build notes,
so an adopter/committer can look there for "truthful" information
about the build's health. 

John Lanuti
IBM Web Tools Platform Technical Lead, IBM Rational
IBM Software Lab - Research Triangle Park, NC
jlanuti@xxxxxxxxxx
t/l 441-7861

Lawrence Mandel <lmandel@xxxxxxxxxx>
Sent by: wtp-releng-bounces@xxxxxxxxxxx

01/25/2007 07:20 PM

Please respond to
Webtools releng discussion list <wtp-releng@xxxxxxxxxxx>

To	Webtools releng discussion list <wtp-releng@xxxxxxxxxxx>
cc
Subject	Re: [wtp-releng] More on JUnit's policy and principles

Hi David, 

I do not object to declaring an I-build this week without this problem
fixed. I agree that we do not need to wait for the schema to be restored
to declare the RC. There is a simple work around using the XML catalog
that can be used where necessary. 

Unfortunately I was not on today's status call and did not get the whole
story. Thank you for bringing me up to speed. I'd like to retract my statement
about disapproving the removal of this test. With the bug open so we won't
forget about this issue I'm happy to proceed as you suggest with commenting
out the test until we have a fix. I certainly wouldn't want to be a part
of facilitating the bad practice of ignoring failing JUnits. :)

Thanks again, 

Lawrence Mandel

Software Developer
IBM Rational Software
Phone: 905 - 413 - 3814   Fax: 905 - 413 - 4920
lmandel@xxxxxxxxxx

David M Williams <david_williams@xxxxxxxxxx>
Sent by: wtp-releng-bounces@xxxxxxxxxxx

01/25/2007 06:32 PM

Please respond to
Webtools releng discussion list <wtp-releng@xxxxxxxxxxx>

To	Webtools releng discussion list <wtp-releng@xxxxxxxxxxx>
cc
Subject	[wtp-releng] More on JUnit's policy and principles

>  I've got to voice my disapproval of commenting out this failing
test. 

Lawrence, perhaps you mean you object to us declaring a  build this
week if this problem is not fixed? If so, please let us know. 

But otherwise, the failing JUnit's are first and foremost for committers
to know there's a problem. Once known, a bug is open, 
folks begin to work on it with a priority measured right along with every
other bug, but if it can not be fixed right away, there is no further 
advantage to displaying the failing JUnit's. And, it's my belief it's actually
a bad practice to leave "known 
failing tests" in the suite, since it teaches everyone to ignore nonzero
numbers, since they are often nonzero.   

I requested this action the status call, so you might not have known the
whole story behind the brief summary, 
if you were not on the status call. 

I definitely agree with you this problem is serious and needs to be fixed.
My own opinion is we do not need to wait until the "ibm.com"
site is 
fixed for us to declare this RC, but if you and other committers think
we should wait for ibm.com to fix their site, then I'd want to know that,
and 
the details of why. 

Thanks _______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng
_______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng
_______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng