Re: [cdt-dev] LLVM Project Blog: A path forward for an LLVM toolchain on

[Nathan Ridge <zeratul976@xxxxxxxxxxx>]

> Clang's parsing is indeed significantly slower than the CDT one, on  
> order of 5-6 seconds for a typical source file of few hundred lines.  

Are you referring to CDT or clang? I find that clang (or any other 
compiler) can often take 30-45 seconds to compile a source file, 
especially if the code in it is template-heavy.

> Indexing would be particularly sensitive to parsing slowness since it  
> would bring us back to the days of full indexer. One possible solution  
> to this problem is parsing of many source files in parallel. 

If we assume that the amount of time taken to parse/compile a file 
or translation unit is proportional to the number of lines of code
in it, then the difference betwen indexing by running the compiler
on every translation unit (the hypothetical clang way) versus 
indexing by parsing each file (the current CDT way) is roughly the 
ratio of the number of lines in the average translation unit (T) to 
the number of lines in the average source file (F).

I'd say typical numbers are T being in the tens of thousands of
lines, and F being hundreds of lines, so we are looking at a
factor of ~100 difference. You'd need a lot of parallelism to
overcome that.

Think of a large C++ project you worked on recently. How long 
did it take to do a full (clean), parallel build of it? (That's 
about how long clang-based indexing would take. Not quite as 
much since clang wouldn't be doing the optimization and code 
generation phases, but on that order of magnitude). How long 
did it take for CDT to index it the first time? In my experience, 
indexing is much faster - and that's with a non-parallel indexer
written in Java, compared to a highly runtime- and memory-
optimized C++ compiler being run in parallel.

More importantly, even if parallelism brings down the amount
of time required to index all translation units in a project,
it doesn't bring down the amount of time required to index a
single translation unit. That means that the amount of time
between you writing a line of code, and the IDE underlining
errors in it (or semantically coloring it, or allowing you
to perform an 'Open Declaration' or a refactoring on it, or
whatever else) will be the amount of time it takes to
recompile the translation unit - a lot more time than it
does now.

Regards,
Nate