Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-dev] Help with Indexer Bug 
I would personally like to see a preference to make it possible to disable this optimization. It could be left on by default but at least it will be possible for small and medium-sized projects to get more a more accurate index. Would that be acceptable?

Marc-Andre

________________________________________
From: cdt-dev-bounces@xxxxxxxxxxx [cdt-dev-bounces@xxxxxxxxxxx] on behalf of Schorn, Markus [Markus.Schorn@xxxxxxxxxxxxx]
Sent: Thursday, 03 October 2013 3:54 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

We have tried to provide a complete solution, which worked for smaller projects but did not work for projects with a few thousands of files. You can study what we have tried and you will need an idea beyond that to come up with a scalable solution:  https://bugs.eclipse.org/bugs/show_bug.cgi?id=197989

To make indexing work for your code, I think your best chance is to exempt files that need multiple variants from CDTs pragma once treatment.
This can for instance be done by one of the following means:
* Change your source code (intentionally disturb CDT's pragma once detection)
* Provide some preference mechanism in CDT, where you can explicitly force multiple versions for some files
* Identify a pattern to automatically detect files for which we should force multiple versions (although they have
    pragma once semantics)

Markus.

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Joseph Henry
Sent: Wed, 02. 10. 2013 18:47
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

Adding a comment that Brandon Philips sent to me directly:

Could CDT store in the database the preprocessor symbols actually referenced by each header it parses?  Then CDT can check if those specific values have changed to decide if re-parsing is necessary.  Maybe in the realm of "not a simple change" though. :)

(Sorry to jump in randomly.)

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Andrew Eidsness
Sent: Wednesday, October 02, 2013 12:34 PM
To: cdt-dev@xxxxxxxxxxx
Subject: Re: [cdt-dev] Help with Indexer Bug

On 13-10-02 12:01 PM, Joseph Henry wrote:
> What was the fairly larger project that you tested?
>
> I ran a test of my own. The first one is I added a ctx.trackSignificantMacros(); statement in the detectIncludeGuard function in CPreprocessor even if an include guard was found. This fixes all of my issues. This is the output of the log:
>
> !MESSAGE Indexed 'Index1' (10 sources, 242 headers) in 3.45 sec:
> 31,198 declarations; 65,703 references; 0 unresolved inclusions; 0
> syntax errors; 0 unresolved names (0.00%)
>
> Now this is how the output without my change:
>
> !MESSAGE Indexed 'Index1' (10 sources, 221 headers) in 3.18 sec:
> 26,224 declarations; 56,348 references; 0 unresolved inclusions; 101
> syntax errors; 323 unresolved names (0.39%)
>
> I do not see a significant performance increase, yet I do see that my project resolves perfectly.

The sample projects are just from internal development.  I'm not at all familiar with the structure of the projects, so I can't make any guesses at why one was so much worse.  However, it also happened to be the largest of my three samples.

I don't have the full results from the test anymore, but here is the summary of the indexer (ran just now) from the project that was 7x slower:

C/C++ Indexer: Project 'seven_times_slower' (504 sources, 5844 headers)
    Options: indexer='PDOMFastIndexer', parseAllFiles=true, unusedHeaders=useCPP, skipReferences=false, skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
    Database: 119873536 bytes
    Timings: 1342211 total, 319891 parser, 9345 resolution, 51686 index update.
    Errors: 0 internal, 0 include, 0 scanner, 0 syntax errors.
    Names: 399371 declarations, 1735121 references, 350(0.02%) unresolved.
    Cache[64MB]: 4872643790 hits, 7259(0.00%) misses.
Indexer: completed PDOMRebuildTask[1342246ms]

I wouldn't call this project large, but it was bigger the other toy projects that I was using for testing.

Also, I would caution against the change you described.  I was not able to make that variation work without introducing new problems in other projects.

Finally, a technique that I found useful was to put print statements in the constructors for PDOMFile, PDOMMacro, and PDOMInclude.  I used the record id as a unique identifier.  In PDOMInclude I printed both the containing and the target files.  Printing the lists of significant and internallyModified macros generated ALOT of output, so I used filtering and the focused on only one or two at a time.

While looking for these results I found notes on a related problem that I had meant to raise as a bug.  I've done that now (418533).  From what I remember, the description of that bug was one of the things that wound up introducing new bugs when I made the change you described.

-Andrew
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

Back to the top