Re: [cdt-dev] Proofing the CDT w/ large projects

[Bogdan Gheorghe <gheorghe@xxxxxxxxxx>]

cdt-dev-admin@xxxxxxxxxxx wrote on 02/02/2004 04:49:08
AM:

> 
> 
> 
> Folks,
> 
>   Just thought you would be interested to hear a bit about the
environment
> that one of our customers is working with using CDT 1.2:
> 
> - Project tree contains 5.68 GB of content when fully built
> - Distributed over 38,601 files in 3,783 directories
> - Development file breakdown:
>   27   *.cxx
>   464  *.c
>   6039 *.cpp  
>   
>   3    *.hxx
>   2453 *.hpp   
>   9061 *.h
> 
>   2    *.so
>   35   *.dll
>   45   *.lib
>   75   *.exe
>   994  *.a
>   6206 *.so
> 
> The time it takes to load this project into the IDE initially (on
a 3Ghz
> machine) is ~3m, but subsequent IDE start-ups are in the 7s timeframe.
 
> Compared to previous versions of CDT, this is pretty awesome!
> 
> The big bottleneck right now seems to be the indexer.  Initially
it takes
> >5m to index the 15,000+ source files and then >2m to pull out
all of the
> classes and types from a search (I'm testing via the Open Type action
> Chris Wiebe implemented).  (Note: It is also killer for the code
completion 
> perfomance it seems since the user doesn't get any feedback about
what is
> happening).
> 
> Some things I've noticed and am looking into:
> - There are a number of scanner and parser errors (ranging from failures
>   to outright exceptions).  This is likely due to an improperly
configured
>   source base (no #defines, #includes set), but shouldn't affect
the 
>   re-indexing operation.
>

Provided that the parser can get through all of the
files without getting nuked then, yes, it 
shouldn't affect re-indexing...

> - There are a couple of "expected exceptions" which then
return null, for
>   example in FileListBlock.getFile(int fileNum) we hit the exception
>   condition very often.  Checking the range and returning
null improves
>   the performance
> 
> - There are a couple of mystery exceptions I still haven't figured
out.
>   For example WordEntry.mapRefs(int [] mappings) is often throwing
an
>   ArrayIndexOutOfBounds exception ... this is trouble I believe
and 
>   leads to:
> 

Both of these situations should never happen - I'll
try to reproduce on large project.

> - The fact that in the IndexManager.getIndex(IPath, boolean, boolean)
>   we (in this particular project) are never retrieving an index
which
>   has a known state (it is always UNKNOWN_STATE) which means
we go and
>   rebuild the index for everything.

Hmm..this sounds like the indexer is not completing
all of its jobs.

Can you turn on tracing (under Run options, tracing
tab, click on org.eclipse.cdt.core->set "debug/indexmanger"
to true. Load up your project and see if the indexer makes it through all
of the source files or if it dies somewhere.

> 
> I think that with Chris Wiebe's "Open Type"
contributed plugin and a 
> moderately large project you should be able to see this kind of churn.
> I'm looking into it, but thought that the indexer guy (Hi Bogdan!)
might
> have some thoughts about what is going on.
> 
> Also ... it takes a fair amount of time to add in 15,000+ entries
using
> the IndexAllProject.execute() -> IndexManager.addSource() method,
several
> seconds at least in my environment.  
> 
> Just passing along some observations.  Unfortunately I can't
pass along the
> source base, but I will certainly try and pull out relevant information
as
> requested/required and dig into this while I can.
> 
> Thanks,
>  Thomas
> _______________________________________________
> cdt-dev mailing list
> cdt-dev@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/cdt-dev