| Re: [cdt-dev] Help with Indexer Bug |
Adding a comment that Brandon Philips sent to me directly:
Could CDT store in the database the preprocessor symbols actually referenced by each header it parses?
Then CDT can check if those specific values have changed to decide if re-parsing is necessary.
Maybe in the realm of "not a simple change" though. :)
(Sorry to jump in randomly.)
-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Andrew Eidsness
Sent: Wednesday, October 02, 2013 12:34 PM
To: cdt-dev@xxxxxxxxxxx
Subject: Re: [cdt-dev] Help with Indexer Bug
On 13-10-02 12:01 PM, Joseph Henry wrote:
> What was the fairly larger project that you tested?
>
> I ran a test of my own. The first one is I added a ctx.trackSignificantMacros(); statement in the detectIncludeGuard function in CPreprocessor even if an include guard was found. This fixes all of my issues. This is the output of the log:
>
> !MESSAGE Indexed 'Index1' (10 sources, 242 headers) in 3.45 sec:
> 31,198 declarations; 65,703 references; 0 unresolved inclusions; 0
> syntax errors; 0 unresolved names (0.00%)
>
> Now this is how the output without my change:
>
> !MESSAGE Indexed 'Index1' (10 sources, 221 headers) in 3.18 sec:
> 26,224 declarations; 56,348 references; 0 unresolved inclusions; 101
> syntax errors; 323 unresolved names (0.39%)
>
> I do not see a significant performance increase, yet I do see that my project resolves perfectly.
The sample projects are just from internal development. I'm not at all familiar with the structure of the projects, so I can't make any guesses at why one was so much worse. However, it also happened to be the largest of my three samples.
I don't have the full results from the test anymore, but here is the summary of the indexer (ran just now) from the project that was 7x slower:
C/C++ Indexer: Project 'seven_times_slower' (504 sources, 5844 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=true, unusedHeaders=useCPP, skipReferences=false, skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
Database: 119873536 bytes
Timings: 1342211 total, 319891 parser, 9345 resolution, 51686 index update.
Errors: 0 internal, 0 include, 0 scanner, 0 syntax errors.
Names: 399371 declarations, 1735121 references, 350(0.02%) unresolved.
Cache[64MB]: 4872643790 hits, 7259(0.00%) misses.
Indexer: completed PDOMRebuildTask[1342246ms]
I wouldn't call this project large, but it was bigger the other toy projects that I was using for testing.
Also, I would caution against the change you described. I was not able to make that variation work without introducing new problems in other projects.
Finally, a technique that I found useful was to put print statements in the constructors for PDOMFile, PDOMMacro, and PDOMInclude. I used the record id as a unique identifier. In PDOMInclude I printed both the containing and the target files. Printing the lists of significant and internallyModified macros generated ALOT of output, so I used filtering and the focused on only one or two at a time.
While looking for these results I found notes on a related problem that I had meant to raise as a bug. I've done that now (418533). From what I remember, the description of that bug was one of the things that wound up introducing new bugs when I made the change you described.
-Andrew
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev