| [cdt-patch] Automated JUnit test rev 2 |
Here are the patch files for the 2nd pass at the Automated JUnit test addressing some of the points brought up by Peter. Notes: 1) The AutomatedTest.properties file is optional, if it is not present, the test will run on everything in the resources.cFiles and resources.cppFiles directories. People can drop files in these directories to be included in the auto test. 2) The Log file is only generated if (a) the AutomatedTest.properties file exists and (b) the option ouputFile = <file> is present in the property file. 3) The C/CPP nature of each file is determined in the following manner: if the extension is .cc, .cpp, .hpp, .hxx, .hh, then it is a cpp file. If the extension is .c, its a c file, otherwise for .h it has the default nature. The default nature for the cFiles folder is c, the cppFiles folder is cpp. (Note that files with extensions other than those outlined here are not considered) 4) The properties file specifies files/folders to test using the "source = ..." property, which is a comma seperated list of files/directories and default natures. EX: source = Z:\test\cDir1, c, \ Z:\test\cppDir1, cpp, \ Z:\test\file1.h, cpp, \ Z:\test\file2.h, c -Andrew Files: patch_04.17.03(cdt.core).txt : changes to allow getting the line number that caused the parse to fail. patch_04.17.03(cdt.ui.tests).txt : contains the AutomatedTest.java files.zip : initial contents of cdt.ui.tests\parser\org.eclipse.cdt.core.parser.resources.cFiles and org.eclipse.cdt.core.parser.resources.cppFiles directories -----Original Message----- From: Peter Graves [mailto:pgraves@xxxxxxx] Sent: Thursday, April 17, 2003 8:03 AM To: cdt-patch@xxxxxxxxxxx Subject: Re: [cdt-patch] Automated JUnit test I have a couple questions/concerns about this test. First, why are you not including the test files (.c/.cpp) in the cvs tree? It seems like for test repeatability they should all be included in revision control, and available in the same place as the test cases. This is the way most of the current CDT tests work, and how the core JDT seems to do it. In general they will create a .resource directory that contains any source/projects that are needed for the tests. That would also make it possible for other people to easily drop in new problematic source files that would then just be part of the normal testing run, which should become an automatic thing at some point. Second, why the separate failure log file? Why not generate a full error message String, and then fail the test with that? Once again, once we have regular test runs, it will be much easier to deal with various failures if we only need to deal with a single results file. And the third and final question, why does the test have a global C or C++ flag? I am guessing for your testing you just have a directory of C files, and a directory of C++ files that you switch between, but if all the source does end up in cvs, it would be nice if you could do a run over both the C and the C++ parsing stuff all at the same time (one again, especially for the automated tests run..). I am guessing it would be fairly easy to distinguish c files from c++ files by the file names. Header files may cause a problem, but since we would control the files we test against, we could leave c headers as .h and make all the c++ headers .hpp/.hxx or something like that. Thanks, -Peter Niefer, Andrew wrote: >Attached are the patches for the automated test framework. Note, that >like >John's line number test, this automated test requires the PDE JUnit >plugin >and should be run as a "JUnit Plugin Test". > >The automated framework will take a directory and for each c/cpp/h file >in >it and in its subdirectories, the framework will add a JUnit test to its >testSuite which will parse that file. > >The JUnit test will fail if the parse fails or throws an exception. > >Filename and line number for each failing test is output to a specified >file. > >There is a properties file (AutomatedTest.properties) in which you >specify >the following options, which should be customized before running the >test: >cppNature = true, means the files will be cpp files, any other value >means c >files. >outputFile = Z:\CDT\report.txt, the file to output the results >testDir = Z:\CDT\test, the directory to look for source files in. > >-Andrew > > > > >------------------------------------------------------------------------ > >Index: ChangeLog >=================================================================== >RCS file: /home/tools/org.eclipse.cdt.ui.tests/ChangeLog,v >retrieving revision 1.30 >diff -u -r1.30 ChangeLog >--- ChangeLog 16 Apr 2003 13:52:08 -0000 1.30 >+++ ChangeLog 16 Apr 2003 19:38:35 -0000 >@@ -1,3 +1,6 @@ >+2003-04-16 >+ Added AutomatedTest >+ > 2003-04-15 John Camelon > Added ScannerTestCase::testBug36434(). > Added ScannerTestCase::testMultipleLines(). >Index: parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties >=================================================================== >RCS file: parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties >diff -N parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties >--- /dev/null 1 Jan 1970 00:00:00 -0000 >+++ parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties 16 Apr 2003 19:38:36 -0000 >@@ -0,0 +1,3 @@ >+cppNature = true >+outputFile = Z:\\CDT\\report.txt >+testDir = Z:\\CDT\\test >Index: parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java >=================================================================== >RCS file: parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java >diff -N parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java >--- /dev/null 1 Jan 1970 00:00:00 -0000 >+++ parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java 16 Apr 2003 19:38:36 -0000 >@@ -0,0 +1,205 @@ >+/************************************************************************* ****** >+ * Copyright (c) 2001 IBM Corporation and others. >+ * All rights reserved. This program and the accompanying materials >+ * are made available under the terms of the Common Public License v0.5 >+ * which accompanies this distribution, and is available at >+ * http://www.eclipse.org/legal/cpl-v05.html >+ * >+ * Contributors: >+ * IBM Corp. - Rational Software - initial implementation >+ **************************************************************************** **/ >+ >+package org.eclipse.cdt.core.parser.tests; >+ >+import java.io.File; >+import java.io.FileInputStream; >+import java.io.FileOutputStream; >+import java.io.FilenameFilter; >+import java.util.LinkedList; >+import java.util.Properties; >+ >+import org.eclipse.core.runtime.Path; >+ >+import org.eclipse.cdt.internal.core.parser.IParserCallback; >+import org.eclipse.cdt.internal.core.parser.NullParserCallback; >+import org.eclipse.cdt.internal.core.parser.Parser; >+ >+import junit.framework.AssertionFailedError; >+import junit.framework.Test; >+import junit.framework.TestCase; >+import junit.framework.TestSuite; >+ >+ >+ >+/** >+ * @author aniefer >+ * >+ * To change the template for this generated type comment go to >+ * Window>Preferences>Java>Code Generation>Code and Comments >+ */ >+public class AutomatedTest extends TestCase { >+ public AutomatedTest(String name){ >+ super(name); >+ } >+ >+ public void doFile() throws Throwable { >+ assertNotNull( fileList ); >+ >+ File file = null; >+ Parser parser = null; >+ >+ try{ >+ file = (File)fileList.removeFirst(); >+ FileInputStream stream = new FileInputStream( file ); >+ >+ parser = new Parser( stream, nullCallback, true); >+ parser.setCppNature( cppNature ); >+ >+ assertTrue( parser.parse() ); >+ } >+ catch( Throwable e ) >+ { >+ if( e instanceof AssertionFailedError ) >+ { >+ String output = file.getCanonicalPath() + ": Parse failed on line "; >+ output += parser.getLastLineNumber() + "\n"; >+ >+ if( report != null ){ >+ report.write( output.getBytes() ); >+ } >+ >+ fail( output ); >+ } else { >+ if( report != null ){ >+ StackTraceElement frames[] = e.getStackTrace(); >+ String output = file.getCanonicalPath() + ": " + e.getClass().toString(); >+ output += " on line " + parser.getLastLineNumber() + "\n"; >+ output += "\t" + "at " + frames[0].getClassName() + "." + frames[0].getMethodName() + "\n"; >+ output += "\t" + "at " + frames[1].getClassName() + "." + frames[1].getMethodName() + "\n"; >+ report.write( output.getBytes() ); >+ } >+ throw e; >+ } >+ } >+ } >+ >+ public void reportFailed(){ >+ fail( "Unable to open " + outputFile + "for output of results." ); >+ } >+ >+ public void propertiesFailed(){ >+ fail( "Unable to load properties file." ); >+ } >+ >+ public static Test suite() >+ { >+ TestSuite suite = new TestSuite(); >+ >+ try{ >+ loadProperties(); >+ } catch( Exception e ){ >+ suite.addTest( new AutomatedTest( "propertiesFailed") ); >+ } >+ >+ if( outputFile != null && !outputFile.equals("") ){ >+ try{ >+ >+ File output = new File( outputFile ); >+ >+ if( output.exists() ){ >+ output.delete(); >+ } >+ >+ output.createNewFile(); >+ >+ report = new FileOutputStream( output ); >+ >+ } catch( Exception e ) { >+ suite.addTest( new AutomatedTest( "reportFailed" ) ); >+ } >+ } >+ >+ File file = new File( testDir ); >+ if( file.exists() ) >+ fillSuite( suite, file ); >+ >+ index = 0; >+ >+ return suite; >+ } >+ private static void fillSuite( TestSuite suite, File path ){ >+ File files[] = null; >+ if( path.isFile() ){ >+ files = new File[ 1 ]; >+ files[0] = path; >+ } >+ else >+ files = path.listFiles(); >+ >+ File file = null; >+ int i = 0; >+ try{ >+ file = files[ i++ ]; >+ while( file != null ) >+ { >+ if( file.isDirectory() ) >+ fillSuite( suite, file ); >+ else if( file.isFile() && nameFilter.accept( file.getParentFile(), file.getName() ) ){ >+ fileList.add( file ); >+ suite.addTest( new AutomatedTest( "doFile" ) ); >+ } >+ >+ file = files[ i++ ]; >+ } >+ } catch( ArrayIndexOutOfBoundsException e ){ >+ //done >+ } >+ } >+ >+ protected void tearDown () throws Exception { >+ if( fileList != null && fileList.size() == 0 ){ >+ report.flush(); >+ report.close(); >+ } >+ } >+ >+ static private void loadProperties() throws Exception{ >+ String fileName = org.eclipse.core.runtime.Platform.getPlugin("org.eclipse.cdt.ui.tests").find (new Path("/")).getFile(); >+ fileName += "/parser/org/eclipse/cdt/core/parser/resources/AutomatedTest.properties"; >+ FileInputStream propertiesIn = new FileInputStream(fileName); >+ >+ properties.load( propertiesIn ); >+ >+ String val = null; >+ >+ cppNature = ( properties.getProperty( "cppNature", "true" ).equals("true") ); >+ outputFile = properties.getProperty( "outputFile", "" ); >+ testDir = properties.getProperty( "testDir", "" ); >+ } >+ >+ private static LinkedList fileList = new LinkedList(); >+ private static int index = 0; >+ private static FilenameFilter nameFilter = new Filter(); >+ private static FileOutputStream report = null; >+ private static IParserCallback nullCallback = new NullParserCallback(); >+ private static Properties properties = new Properties(); >+ >+ private static boolean cppNature = true; >+ private static String outputFile = null; >+ private static String testDir = null; >+ >+ static private class Filter implements FilenameFilter >+ { >+ public boolean accept(File dir, String name) { >+ if( name.endsWith(".cpp") || >+ name.endsWith(".c") || >+ name.endsWith(".cc") || >+ name.endsWith(".h") ) >+ { >+ return true; >+ } >+ else >+ return false; >+ } >+ } >+} > > >------------------------------------------------------------------------ > >Index: parser/org/eclipse/cdt/internal/core/parser/Parser.java >=================================================================== >RCS file: /home/tools/org.eclipse.cdt.core/parser/org/eclipse/cdt/internal/core/parser /Parser.java,v >retrieving revision 1.31 >diff -u -r1.31 Parser.java >--- parser/org/eclipse/cdt/internal/core/parser/Parser.java 16 Apr 2003 12:30:46 -0000 1.31 >+++ parser/org/eclipse/cdt/internal/core/parser/Parser.java 16 Apr 2003 19:19:59 -0000 >@@ -106,13 +106,17 @@ > // Mark as failure and try to reach a recovery point > failParse(); > >- if (lastBacktrack != null && lastBacktrack == LA(1)) { >- // we haven't progressed from the last backtrack >- // try and find tne next definition >- consumeToNextSemicolon(); >- } else { >- // start again from here >- lastBacktrack = LA(1); >+ try { >+ if (lastBacktrack != null && lastBacktrack == LA(1)) { >+ // we haven't progressed from the last backtrack >+ // try and find tne next definition >+ consumeToNextSemicolon(); >+ } else { >+ // start again from here >+ lastBacktrack = LA(1); >+ } >+ } catch (EndOfFile e){ >+ break; > } > } > catch( Exception e ) >@@ -2269,5 +2273,12 @@ > public int getLineNumberForOffset(int offset) > { > return scanner.getLineNumberForOffset(offset); >+ } >+ >+ public int getLastLineNumber(){ >+ if( lastToken != null ){ >+ return scanner.getLineNumberForOffset( lastToken.offset ); >+ } >+ return -1; > } > } > > _______________________________________________ cdt-patch mailing list cdt-patch@xxxxxxxxxxx http://dev.eclipse.org/mailman/listinfo/cdt-patch
Index: ChangeLog =================================================================== RCS file: /home/tools/org.eclipse.cdt.ui.tests/ChangeLog,v retrieving revision 1.31 diff -u -r1.31 ChangeLog --- ChangeLog 17 Apr 2003 13:41:45 -0000 1.31 +++ ChangeLog 17 Apr 2003 17:14:43 -0000 @@ -1,3 +1,8 @@ +2003-04-17 Andrew Niefer + Added AutomatedTest + Added resources.cFiles + Added resources.cppFiles + 2003-04-16 John Camelon Added DOMTests::testBug36532(). Added DOMTests::testBug36432(). Index: parser/org/eclipse/cdt/core/parser/resources/.cvsignore =================================================================== RCS file: parser/org/eclipse/cdt/core/parser/resources/.cvsignore diff -N parser/org/eclipse/cdt/core/parser/resources/.cvsignore --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ parser/org/eclipse/cdt/core/parser/resources/.cvsignore 17 Apr 2003 17:14:43 -0000 @@ -0,0 +1 @@ +AutomatedTest.properties Index: parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java =================================================================== RCS file: parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java diff -N parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ parser/org/eclipse/cdt/core/parser/tests/AutomatedTest.java 17 Apr 2003 17:14:43 -0000 @@ -0,0 +1,267 @@ +/******************************************************************************* + * Copyright (c) 2001 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Common Public License v0.5 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/cpl-v05.html + * + * Contributors: + * IBM Corp. - Rational Software - initial implementation + ******************************************************************************/ + +package org.eclipse.cdt.core.parser.tests; + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.io.FileOutputStream; +import java.io.FilenameFilter; +import java.util.HashMap; +import java.util.Iterator; +import java.util.LinkedList; +import java.util.NoSuchElementException; +import java.util.Properties; +import java.util.Set; +import java.util.StringTokenizer; + +import org.eclipse.core.runtime.Path; + +import org.eclipse.cdt.internal.core.parser.IParserCallback; +import org.eclipse.cdt.internal.core.parser.NullParserCallback; +import org.eclipse.cdt.internal.core.parser.Parser; + +import junit.framework.AssertionFailedError; +import junit.framework.Test; +import junit.framework.TestCase; +import junit.framework.TestSuite; + + + +/** + * @author aniefer + * + * To change the template for this generated type comment go to + * Window>Preferences>Java>Code Generation>Code and Comments + */ +public class AutomatedTest extends TestCase { + public AutomatedTest(String name){ + super(name); + } + + public void doFile() throws Throwable { + assertNotNull( fileList ); + + File file = null; + Parser parser = null; + + try{ + file = (File)fileList.removeFirst(); + FileInputStream stream = new FileInputStream( file ); + + String filePath = file.getCanonicalPath(); + String nature = (String)natures.get( filePath ); + + boolean cppNature = nature.equalsIgnoreCase("cpp"); + + parser = new Parser( stream, nullCallback, true); + parser.setCppNature( cppNature ); + + assertTrue( parser.parse() ); + } + catch( Throwable e ) + { + String output = null; + if( e instanceof AssertionFailedError ){ + output = file.getCanonicalPath() + ": Parse failed on line "; + output += parser.getLastLineNumber() + "\n"; + } else { + StackTraceElement frames[] = e.getStackTrace(); + output = file.getCanonicalPath() + ": " + e.getClass().toString(); + output += " on line " + parser.getLastLineNumber() + "\n"; + output += "\t" + "at " + frames[0].getClassName() + "." + frames[0].getMethodName() + "\n"; + output += "\t" + "at " + frames[1].getClassName() + "." + frames[1].getMethodName() + "\n"; + } + if( report != null ){ + report.write( output.getBytes() ); + } + + fail( output ); + } + } + + public void reportFailed(){ + fail( "Unable to open " + outputFile + "for output of results." ); + } + + public void propertiesFailed(){ + fail( "Unable to load properties file." ); + } + + protected void runTest() throws Throwable { + String name = getName(); + + if( name.equals("propertiesFailed") ) + propertiesFailed(); + else if ( name.equals("reportFailed") ) + reportFailed(); + else + doFile(); + } + + public static Test suite() + { + TestSuite suite = new TestSuite(); + + try{ + loadProperties(); + } catch( Exception e ){ + suite.addTest( new AutomatedTest( "propertiesFailed") ); + } + + if( outputFile != null && !outputFile.equals("") ){ + try{ + + File output = new File( outputFile ); + + if( output.exists() ){ + output.delete(); + } + + output.createNewFile(); + + report = new FileOutputStream( output ); + + } catch( Exception e ) { + suite.addTest( new AutomatedTest( "reportFailed" ) ); + } + } + + Set keys = testSources.keySet(); + Iterator iter = keys.iterator(); + int size = keys.size(); + String item = null; + for( int i = size; i > 0; i-- ) + { + item = (String) iter.next(); + File file = new File( item ); + if( file.exists() ){ + defaultNature = (String) testSources.get( item ); + fillSuite( suite, file ); + } + } + + return suite; + } + private static void fillSuite( TestSuite suite, File path ){ + File files[] = null; + if( path.isFile() ){ + files = new File[ 1 ]; + files[0] = path; + } + else + files = path.listFiles(); + + File file = null; + String filePath = null; + int i = 0; + try{ + file = files[ i++ ]; + while( file != null ) + { + if( file.isDirectory() ) + fillSuite( suite, file ); + else if( file.isFile() && nameFilter.accept( file.getParentFile(), file.getName() ) ){ + try{ + filePath = file.getCanonicalPath(); + } catch ( Exception e ){ + continue; + } + + if( filePath.endsWith(".cpp") || filePath.endsWith(".hpp") || + filePath.endsWith(".hxx") || filePath.endsWith(".hh") ) + { + natures.put( filePath, "cpp" ); + } else if( filePath.endsWith(".c") ){ + natures.put( filePath, "c" ); + } else { + natures.put( filePath, defaultNature ); + } + + fileList.add( file ); + suite.addTest( new AutomatedTest( file.getName() ) ); + } + file = files[ i++ ]; + } + } catch( ArrayIndexOutOfBoundsException e ){ + //done + } + } + + protected void tearDown () throws Exception { + if( fileList != null && fileList.size() == 0 && report != null ){ + report.flush(); + report.close(); + } + } + + static private void loadProperties() throws Exception{ + String resourcePath = org.eclipse.core.runtime.Platform.getPlugin("org.eclipse.cdt.ui.tests").find(new Path("/")).getFile(); + resourcePath += "/parser/org/eclipse/cdt/core/parser/resources"; + + try{ + FileInputStream propertiesIn = new FileInputStream( resourcePath + "/AutomatedTest.properties"); + properties.load( propertiesIn ); + + outputFile = properties.getProperty( "outputFile", "" ); + String sourceInfo = properties.getProperty( "source", "" ); + if( sourceInfo.equals("") ) + throw new FileNotFoundException(); + else{ + StringTokenizer tokenizer = new StringTokenizer( sourceInfo, "," ); + String str = null, val = null; + try{ + while( tokenizer.hasMoreTokens() ){ + str = tokenizer.nextToken().trim(); + val = tokenizer.nextToken().trim(); + + testSources.put( str, val ); + } + } catch ( NoSuchElementException e ){ + //only way to get here is to have a missing val, assume cpp for that str + testSources.put( str, "cpp" ); + } + + } + } catch ( FileNotFoundException e ){ + testSources.put( resourcePath + "/cppFiles", "cpp" ); + testSources.put( resourcePath + "/cFiles", "c" ); + } + } + + private static LinkedList fileList = new LinkedList(); + private static FilenameFilter nameFilter = new Filter(); + private static FileOutputStream report = null; + private static IParserCallback nullCallback = new NullParserCallback(); + private static Properties properties = new Properties(); + private static String defaultNature; + private static String outputFile = null; + private static HashMap testSources = new HashMap(); + private static HashMap natures = new HashMap(); + + static private class Filter implements FilenameFilter + { + public boolean accept(File dir, String name) { + if( name.endsWith(".cpp") || + name.endsWith(".c") || + name.endsWith(".cc") || + name.endsWith(".h") || + name.endsWith(".hh") || + name.endsWith(".hxx")) + { + return true; + } + else + return false; + } + } +}
Index: parser/org/eclipse/cdt/internal/core/parser/Parser.java
===================================================================
RCS file: /home/tools/org.eclipse.cdt.core/parser/org/eclipse/cdt/internal/core/parser/Parser.java,v
retrieving revision 1.32
diff -u -r1.32 Parser.java
--- parser/org/eclipse/cdt/internal/core/parser/Parser.java 17 Apr 2003 13:41:43 -0000 1.32
+++ parser/org/eclipse/cdt/internal/core/parser/Parser.java 17 Apr 2003 16:59:02 -0000
@@ -106,13 +106,17 @@
// Mark as failure and try to reach a recovery point
failParse();
- if (lastBacktrack != null && lastBacktrack == LA(1)) {
- // we haven't progressed from the last backtrack
- // try and find tne next definition
- consumeToNextSemicolon();
- } else {
- // start again from here
- lastBacktrack = LA(1);
+ try {
+ if (lastBacktrack != null && lastBacktrack == LA(1)) {
+ // we haven't progressed from the last backtrack
+ // try and find tne next definition
+ consumeToNextSemicolon();
+ } else {
+ // start again from here
+ lastBacktrack = LA(1);
+ }
+ } catch (EndOfFile e){
+ break;
}
}
catch( Exception e )
@@ -2279,5 +2283,12 @@
public int getLineNumberForOffset(int offset)
{
return scanner.getLineNumberForOffset(offset);
+ }
+
+ public int getLastLineNumber(){
+ if( lastToken != null ){
+ return scanner.getLineNumberForOffset( lastToken.offset );
+ }
+ return -1;
}
}
Attachment:
files.zip
Description: Binary data