[eclipse-dev] Top 10 (slowest) Unit Test Suites ... plus proposal to run

[David M Williams <david_williams@xxxxxxxxxx>]

As briefly discussed on status call this
week, here are the most time consuming unit test suites.  It is not
necessarily bad to be on this list ... just drawing attention to it, in
case something looks suspicious.  It could mean your component is
doing a really good job of testing your code! 

More details in bug 377718
https://bugs.eclipse.org/bugs/show_bug.cgi?id=377718


  1.  org.eclipse.jdt.ui.tests.refactoring
             54.8 (m)
  2.  org.eclipse.jdt.ui.tests    
                     40.7
(m)
  3.  org.eclipse.ui.tests    
                     
   37.4 (m)
  4.  org.eclipse.core.tests.resources  
               33.4 (m)
  5.  org.eclipse.equinox.p2.tests  
                   30.9
(m)
  6.  org.eclipse.team.tests.cvs.core  
                30.3 (m)
  7.  org.eclipse.pde.build.tests  
                    29.7
(m)
  8.  org.eclipse.osgi.tests    
                     
 24.2 (m)
  9.  org.eclipse.jdt.core.tests.model  
               23.3 (m)
 10.  org.eclipse.jdt.core.tests.compiler
              22.6 (m)


These are "overall times" ... including
time to "setup" and "teardown" tests (each of those
methods is called for each unit tests; not once per suite, as I'm sure
most of you know). These overall times are useful, since usually the tests
themselves are very fast (just a few minutes, or so, at most) so much time
is spent in "setup" and "teardown", presumably.  

As far as I know, these are all required and reasonable
for what is being tested ... but, how would I know!?  In other words,
if any of this lists surprises a component's committers they may want to
take a look if there's something that could be done better. 

Plus, more significantly, I propose that for nightly
builds, we run (only) a subset of tests ... a set of the quick tests, and
run the full set (still) for the more official I- and M-builds. The set
of "quickTests" should take only about 25% of the current time,
so I think is a better use of our shared resources. To put in perspective,
we could do an extra N-build with that same time to do the long running
tests!

If concerns, objections, or suggestions, please comment
in bug
377718.

Thanks,