RE: [wtp-releng] Perf tests on M5 and later

[David M Williams <david_williams@xxxxxxxxxx>]

I can only make some quick observations,
now. Not as concrete as my last hint. 

First, you're not running the tests
in Bulgarian are you? Just kidding (well, half kidding, ... it should work
fine in either, but surprised ICU comes into play, so there may be some
odd bug that's causing too much work to go on). 

Second, you might be hitting some new,
real bugs in the platform. First, the out-of-memory might be that something
about M5 just requires more memory than it used to? Are you running with
a generous memory limit? Which JRE? (If Sun, it's max-perm space may need
increasing). From a quick read of the stack trace, it appears the stack
overflow might really be some deeply nested recursion, (as opposed to an
infinite loop). If that's the case, doubling the stack size might help?
Say, to -X1024k. If more memory _is_ being used,
it'd be nice to find out why ... might be something in the platform. If
we can narrow it down a bit, it'd be worth a bug report ... though, seems
the server team should investigate :) 

Third, and perhaps most concrete ....
that  message "Do
you want to exit the workbench > Yes/No"
is not supposed to pop up if you use the Eclipse test framework, _and_
specify the test-ui type of test. So, not sure what that particular test
uses, but you might check ... that'd at least stop the tests from blocking
all the rest, even if it fails to run. 

HTH

From:	"Raev, Kaloyan" <kaloyan.raev@xxxxxxx>
To:	"Webtools releng discussion list" <wtp-releng@xxxxxxxxxxx>
Date:	04/10/2008 07:15 AM
Subject:	RE: [wtp-releng] Perf tests on M5 and later

---

Thanks, David, 
 
This helped! The releng.wtpbuilder
is in the workspace where I run the pert test script, so changes in this
project are easy. 
Now the xsd-runtime is extracted
in we fall in the next problem :-)
 
Now the perf test procedure is
blocked on the org.eclipse.jst.server.tomcat.tests.performance.tomcat50.AllTests
test. An "Internal Error > Do you want to exit the workbench >
Yes/No" message box pops up in the tested IDE. This effectively blocks
the test to finish and therefore the whole procedure blocks. 
 
This happens in both M5 and M6,
but the reason (found in the logs) are different. 
In M5 we have an OutOfMemoryError
in com.ibm.icu (see stack trace below), while in M6 we have a StackOverflowError
in org.eclipse.core (see stack trace below). 
 
Any hints on these issues are
warmly welcome and higly appreciated. 
 
Greetings,
Kaloyan
 
M5: OutOfMemoryError
=========================
 
!ENTRY org.eclipse.core.jobs 4
2 2008-04-09 04:58:13.883
!MESSAGE An internal error occurred during: "Process resource updates".
!STACK 0
java.lang.OutOfMemoryError: Java heap space
 at java.lang.String.<init>(String.java:174)
 at com.ibm.icu.impl.ICUResourceBundle.getStringValue(ICUResourceBundle.java:1054)
 at com.ibm.icu.impl.ICUResourceBundleImpl$ResourceString.<init>(ICUResourceBundleImpl.java:122)
 at com.ibm.icu.impl.ICUResourceBundle.createBundleObject(ICUResourceBundle.java:948)
 at com.ibm.icu.impl.ICUResourceBundleImpl$ResourceTable.handleGet(ICUResourceBundleImpl.java:204)
 at com.ibm.icu.impl.ICUResourceBundle.getBundle(ICUResourceBundle.java:887)
 at com.ibm.icu.impl.ICUResourceBundle.createBundle(ICUResourceBundle.java:848)
 at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUResourceBundle.java:751)
 at com.ibm.icu.impl.ICUResourceBundle.getBundleInstance(ICUResourceBundle.java:719)
 at com.ibm.icu.util.UResourceBundle.getRootType(UResourceBundle.java:465)
 at com.ibm.icu.util.UResourceBundle.instantiateBundle(UResourceBundle.java:513)
 at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:137)
 at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:217)
 at com.ibm.icu.text.RuleBasedCollator.<init>(RuleBasedCollator.java:1818)
 at com.ibm.icu.text.CollatorServiceShim$1$CollatorFactory.handleCreate(CollatorServiceShim.java:112)
 at com.ibm.icu.impl.ICULocaleService$LocaleKeyFactory.create(ICULocaleService.java:378)
 at com.ibm.icu.impl.ICUService.getKey(ICUService.java:460)
 at com.ibm.icu.impl.ICUService.getKey(ICUService.java:380)
 at com.ibm.icu.impl.ICULocaleService.get(ICULocaleService.java:72)
 at com.ibm.icu.impl.ICULocaleService.get(ICULocaleService.java:56)
 at com.ibm.icu.text.CollatorServiceShim.getInstance(CollatorServiceShim.java:32)
 at com.ibm.icu.text.Collator.getInstance(Collator.java:478)
 at com.ibm.icu.text.Collator.getInstance(Collator.java:312)
 at org.eclipse.ui.internal.views.markers.MarkerEntry.getCollationKey(MarkerEntry.java:150)
 at org.eclipse.ui.internal.views.markers.MarkerDescriptionField.getDescriptionKey(MarkerDescriptionField.java:144)
 at org.eclipse.ui.internal.views.markers.MarkerDescriptionField.compare(MarkerDescriptionField.java:124)
 at org.eclipse.ui.internal.views.markers.MarkerProblemSeverityAndMessageField.compare(MarkerProblemSeverityAndMessageField.java:39)
 at org.eclipse.ui.internal.views.markers.MarkerComparator.compare(MarkerComparator.java:99)
 at java.util.Arrays.mergeSort(Arrays.java:1284)
 at java.util.Arrays.mergeSort(Arrays.java:1295)
 at java.util.Arrays.mergeSort(Arrays.java:1295)
 at java.util.Arrays.mergeSort(Arrays.java:1295)
 
=========================
 
M6. StackOverflowError
=========================
 
!ENTRY org.eclipse.core.jobs 4
2 2008-04-10 13:11:03.296
!MESSAGE An internal error occurred during: "Compacting memory".
!STACK 0
java.lang.StackOverflowError
 at java.util.HashMap.get(HashMap.java:343)
 at org.eclipse.core.internal.utils.StringPool.add(StringPool.java:47)
 at org.eclipse.core.internal.resources.MarkerAttributeMap.shareStrings(MarkerAttributeMap.java:274)
 at org.eclipse.core.internal.resources.MarkerInfo.shareStrings(MarkerInfo.java:194)
 at org.eclipse.core.internal.resources.MarkerSet.shareStrings(MarkerSet.java:241)
 at org.eclipse.core.internal.resources.ResourceInfo.shareStrings(ResourceInfo.java:454)
 at org.eclipse.core.internal.dtree.DataTreeNode.storeStrings(DataTreeNode.java:342)
 at org.eclipse.core.internal.dtree.AbstractDataTreeNode.storeStrings(AbstractDataTreeNode.java:536)
 at org.eclipse.core.internal.dtree.AbstractDataTreeNode.storeStrings(AbstractDataTreeNode.java:536)
 at org.eclipse.core.internal.dtree.AbstractDataTreeNode.storeStrings(AbstractDataTreeNode.java:536)
 at org.eclipse.core.internal.dtree.DataTreeNode.storeStrings(DataTreeNode.java:338)
 at org.eclipse.core.internal.dtree.DeltaDataTree.storeStrings(DeltaDataTree.java:956)
 at org.eclipse.core.internal.dtree.DeltaDataTree.storeStrings(DeltaDataTree.java:958)
 at org.eclipse.core.internal.dtree.DeltaDataTree.storeStrings(DeltaDataTree.java:958)
 at org.eclipse.core.internal.dtree.DeltaDataTree.storeStrings(DeltaDataTree.java:958)
 at org.eclipse.core.internal.dtree.DeltaDataTree.storeStrings(DeltaDataTree.java:958)
........................................
 
=========================

---

From: wtp-releng-bounces@xxxxxxxxxxx
[mailto:wtp-releng-bounces@xxxxxxxxxxx]
On Behalf Of David M Williams
Sent: Tuesday, April 08, 2008 7:52 AM
To: Webtools releng discussion list
Subject: RE: [wtp-releng] Perf tests on M5 and later


Yes, I can give some hints. 
Assuming you use the "wtp.perf" distribution in releng.wtpbuilder
... 
I see it using the dependancies in the releng map files. 
When we switched to the split-up emf zips, the emf runtime was left with
an ID of emf 
and a new one added for the xsd info set with an ID of emfxsd ... so, the
following needs to be 
added to the "setup" target in the build.xml. I've released this
change to head ... but not sure if/how/where you can version it to make
it part of your M5 run. 

      <antcall
target="getAndInstall">
          <param
              name="groupId"
              value="emfxsd"
/>
          <param
              name="clean"
              value="true"
/>
      </antcall>

Thanks,

From:	"Raev, Kaloyan" <kaloyan.raev@xxxxxxx>
To:	"Webtools releng discussion list" <wtp-releng@xxxxxxxxxxx>
Date:	04/07/2008 11:47 AM
Subject:	RE: [wtp-releng] Perf tests on M5 and later

---

One step ahead...

The described performance tests plugins cannot be started, because they
depend (indirectly) on the org.eclipse.xsd plugin, which is not
available in the target installation. It turns out that the perf test
procudure extracts only the emf-runtime.zip, but not the
xsd-runtime.zip. 

As far as I remember the old emf-xsd-sdo.zip has been split to
emf-runtime.zip, xsd-runtime.zip, etc. starting M5. But the perf test
script does not reflect this change correctly. 

David, can you give some hints on this?

Greetings,
Kaloyan


-----Original Message-----
From: wtp-releng-bounces@xxxxxxxxxxx
[mailto:wtp-releng-bounces@xxxxxxxxxxx]
On Behalf Of Raev, Kaloyan
Sent: Wednesday, April 02, 2008 7:38 PM
To: Webtools releng discussion list
Subject: [wtp-releng] Perf tests on M5 and later

Hello, this mail is a follow-up from the last status call:
http://wiki.eclipse.org/WTP_2008-03-27

There I mentioned that the perf test procedure experience problems (new
one) with M5 and later builds. Perf results can be seen here:
http://download.eclipse.org/webtools/downloads/drops/R3.0/S-3.0M5-200802
18021547/perfresults/graph/performance.php

There are some test groups that failed executing. Reasons:
                Could not find
plugin "org.eclipse.wst.xsd.tests.performance"
                Could not find
plugin "org.eclipse.wst.wsdl.tests.performance"
                Could not find
plugin "org.eclipse.jst.ws.tests.performance"
                Could not find
plugin
"org.eclipse.jst.j2ee.core.tests.performance"

It is strange that the above plugins are actually in the plug-in
directory of the installation and they should be found with no problem.

Another issue is that there are some tests that prints lots of messages
to System.out: total of ~1,500,000 lines or ~120 MB. The tests are:
                
org.eclipse.wst.common.tests.performance.internal.CleanBuildTestCase 
                
org.eclipse.jst.server.tomcat.tests.performance.tomcat50.AllTests
                
org.eclipse.wst.html.ui.tests.performance.scalability.Validate1KBFileTes
tCase
I guess this is the reason that the perf test procedure takes three
times more than normal. The pattern of all of the dumps is: "<timestamp>
<some_id> TestValidator3 <something>". 

Here is the console log (~120 MB) for reference:
http://download.eclipse.org/webtools/downloads/drops/R3.0/S-3.0M5-200802
18021547/perfresults/consolelogs/wtpperflog.txt

I hope someone have an idea why the above issues happen and give me a
hand on finding solution. 

Greetings,
Kaloyan Raev
Eclipse WTP Committer
<http://www.eclipse.org/webtools/people/person.php?name=raev>
Senior Developer
NW C JS TOOLS JEE (BG)
SAP Labs Bulgaria
T +359/2/9157-416
mailto:kaloyan.raev@xxxxxxx
www.sap.com

P Save a tree - please do not print this email unless you really need
to! 

_______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng
_______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng
_______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng