| RE: [wtp-releng] Performance Tests |
Hello again, Running the performance tests generally goes OK. I am trying now to execute them for all declared builds on the WTP download page: http://download.eclipse.org/webtools/downloads/ For older builds (before Dec 2007) the test procedure completes successfully. Unfortunately, for the latest one there is some strange problem. It seems to occur when the performance data is stored in the Derby database. As result, the performance graphs are not generated. An example of such results can be seen here: http://download.eclipse.org/webtools/downloads/drops/R2.0/M-2.0.2-20071213160559/perfresults/graph/performance.php The logs of the PerfBuilder are full with exceptions like the one below. Does anybody have an idea what causes this? The only difference in the execution is the build I pick up for target. Greetings, Kaloyan !ENTRY org.eclipse.test.performance 4 1 2008-01-14 14:21:44.718 !MESSAGE Internal Error !STACK 0 ERROR 42Y07: Schema 'GUEST' does not exist at org.apache.derby.iapi.error.StandardException.newException(StandardException.java) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(DataDictionaryImpl.java) at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(QueryTreeNode.java) at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(QueryTreeNode.java) at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(FromBaseTable.java) at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(FromBaseTable.java) at org.apache.derby.impl.sql.compile.FromList.bindTables(FromList.java) at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(SelectNode.java) at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(DMLStatementNode.java) at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(DMLStatementNode.java) at org.apache.derby.impl.sql.compile.ReadCursorNode.bind(ReadCursorNode.java) at org.apache.derby.impl.sql.compile.CursorNode.bind(CursorNode.java) at org.apache.derby.impl.sql.GenericStatement.prepMinion(GenericStatement.java) at org.apache.derby.impl.sql.GenericStatement.prepare(GenericStatement.java) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(GenericLanguageConnectionContext.java) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(EmbedPreparedStatement.java) at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(EmbedPreparedStatement20.java) at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(EmbedPreparedStatement30.java) at org.apache.derby.jdbc.Driver30.newEmbedPreparedStatement(Driver30.java) at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(EmbedConnection.java) at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(EmbedConnection.java) at org.eclipse.test.internal.performance.db.SQL.getVariations(SQL.java:321) at org.eclipse.test.internal.performance.db.DB.internalStore(DB.java:322) at org.eclipse.test.internal.performance.db.DB.store(DB.java:227) at org.eclipse.test.internal.performance.InternalPerformanceMeter.commit(InternalPerformanceMeter.java:92) at org.eclipse.test.performance.PerformanceTestCase.commitMeasurements(PerformanceTestCase.java:151) at org.eclipse.wst.xsd.tests.performance.OpenEditorOAGISXSDTestcase.testReadWSDL(OpenEditorOAGISXSDTestcase.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at junit.framework.TestCase.runTest(TestCase.java:164) at junit.framework.TestCase.runBare(TestCase.java:130) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:120) at junit.framework.TestSuite.runTest(TestSuite.java:230) at junit.framework.TestSuite.run(TestSuite.java:225) at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354) at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206) at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195) at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35) at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:130) at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3727) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3364) at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2381) at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2345) at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2211) at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:473) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:288) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:468) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:106) at org.eclipse.test.UITestApplication.runApplication(UITestApplication.java:138) at org.eclipse.test.UITestApplication.run(UITestApplication.java:60) at org.eclipse.test.UITestApplication.start(UITestApplication.java:210) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:362) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:175) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:561) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:501) at org.eclipse.equinox.launcher.Main.run(Main.java:1239) at org.eclipse.equinox.launcher.Main.main(Main.java:1215) at org.eclipse.core.launcher.Main.main(Main.java:30) -----Original Message----- From: wtp-releng-bounces@xxxxxxxxxxx [mailto:wtp-releng-bounces@xxxxxxxxxxx] On Behalf Of Raev, Kaloyan Sent: Friday, January 11, 2008 5:23 PM To: wtp-releng@xxxxxxxxxxx Subject: FW: [wtp-releng] Performance Tests Hi, I forward the below to the releng mailing list. It would be nice if we can resolve bug 213985 for the next I-builds. The resolution should not be something sophisticated. Greetings, Kaloyan -----Original Message----- From: Raev, Kaloyan Sent: Friday, December 28, 2007 5:44 PM To: 'Carl Anderson'; 'ictewksb@xxxxxxxxxx'; 'jsholl@xxxxxxxxxx' Cc: Chuck Bridgham; 'David M Williams' Subject: RE: [wtp-releng] Performance Tests Hello, I had some time to look deeper into this problem. It turned out that there was some refactoring of the j2ee junit tests that happened about a week after WTP 2.0 was shipped. See bug 190496: https://bugs.eclipse.org/bugs/show_bug.cgi?id=190496 The problem is that after that refactoring the corresponding wrapping performance test case was not updated properly and it now calls some non-existing test methods. More details on this can be seen in bug 213985: https://bugs.eclipse.org/bugs/show_bug.cgi?id=213985 This needs to be solved for both 2.0.2 and 3.0 M5 streams. Unfortunately, I do not see any chance to solve the problem for earlier releases and we will have the errors in the perf test results. I have opened one more bug that suggest an improvement in the logging of the PerformanceTestCaseWrapper that will help investigating problems like this one: https://bugs.eclipse.org/bugs/show_bug.cgi?id=213986 Greetings, Kaloyan -----Original Message----- From: Carl Anderson [mailto:ccc@xxxxxxxxxx] Sent: Thursday, December 06, 2007 4:07 AM To: Raev, Kaloyan Cc: Chuck Bridgham Subject: Fw: [wtp-releng] Performance Tests Kaloyan, I reviewed the code itself (especially the history) and talked this over with John Lanuti- these tests have not changed since WTP 2.0 shipped, and these tests passed successfully while John Lanuti was running them. The next thing to do would be to look into the logs themselves and see if there is a reason why the wrapped test cases are failing. (Perhaps try running the EJBProjectCreationOperationTest by itself to make sure it runs.) FWIW, - Carl Anderson WTP programmer ----- Forwarded by Carl Anderson/Raleigh/IBM on 12/05/2007 09:02 PM ----- Carl Anderson/Raleigh/ IBM@IBMUS To Sent by: Webtools releng discussion list wtp-releng-bounce <wtp-releng@xxxxxxxxxxx> s@xxxxxxxxxxx cc Subject 12/03/2007 11:58 Re: [wtp-releng] Performance Tests AM Please respond to Webtools releng discussion list <wtp-releng@eclip se.org> Kaloyan, The j2ee core perf tests used to pass successfully. I guess that the contact for the j2ee performance tests (which would be me) will have to investigate this. FWIW, - Carl Anderson WTP programmer "Raev, Kaloyan" <kaloyan.raev@sap .com> To Sent by: <wtp-releng@xxxxxxxxxxx> wtp-releng-bounce cc s@xxxxxxxxxxx Subject [wtp-releng] Performance Tests 12/03/2007 10:00 AM Please respond to Webtools releng discussion list <wtp-releng@eclip se.org> Hello, I am happy to announce that I have the performance tests setup and run successfully. I have executed the perf test on two builds already: http://download.eclipse.org/webtools/downloads/drops/R2.0/R-2.0.1-2007092604 2742/ http://download.eclipse.org/webtools/downloads/drops/R2.0/M-2.0.2-2007100405 3715/ Since all builds older than 2.0.1 are already moved to the archive site, I cannot use the PerfBuilder tool to execute the perf tests for them. This is why I use the official 2.0.1 release for a "reference base". The first 2.0.2 I-build (M-2.0.2-20071004053715) perf test results are referred to the results of that 2.0.1 release. I am going to build perf test results for all declared builds on the WTP download site in the next days. One execution last for around 12 hours. But, before this I want to ask what is the for referring perf results. - 2.0.2 I-builds - Do I refer the results of the 2.0.2 I-build to the 2.0.1 build? Or, do I refer the results of the 2.0.2 I-build to the results of the previous 2.0.2 I-build? - 3.0 M-builds - I will refer 3.0 M1 to 2.0.1. But how about 3.0 M2? Referred to 3.0 M1, or again to 2.0.1? And one more thing. There are still 5 failures in the j2ee core perf test: http://download.eclipse.org/webtools/downloads/drops/R2.0/M-2.0.2-2007100405 3715/perfresults/html/org.eclipse.jst.j2ee.core.tests.performance_.html The reason is one and the same: illegal data set: contains neither AVERAGE nor AFTER values. Is this a bug? Or there is still something wrong in my setup? Greetings, Kaloyan Raev Eclipse WTP Committer Senior Developer NW C JS TOOLS JEE (BG) SAP Labs Bulgaria T +359/2/9157-416 mailto:kaloyan.raev@xxxxxxx. www.sap.com (See attached file: smime.p7s) _______________________________________________ wtp-releng mailing list wtp-releng@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/wtp-releng (See attached file: smime.p7s) _______________________________________________ wtp-releng mailing list wtp-releng@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/wtp-releng
Attachment:
smime.p7s
Description: S/MIME cryptographic signature