Re: [cdt-dev] weird scanner discovery behaviour

[James Blackburn <jamesblackburn@xxxxxxxxx>]

Hi Chris,

I do something *very* similar for my scanner discovery alternative -- dwarf settings parser.  There are some gotchas though which make this not suitable as a general fix to push into CVS:

 - The language setting entries are used by managed build for storing it's paths and preproc symbols.  If you use this method on managed projects you'll be changing the set of includes / macros that are passed to the compiler, won't you?
- There's no way to distinguish between paths you added and paths contributed from elsewhere, be it user UI or ManagedBuild. I notice in your code you only every add entries.  This means that, if you change compiler, or change an include directory on a file, you'll end up retaining the older entries
- Your code uses ICConfigurationDescription#getLanguageSettingForFile(resource, false).  The false means that it returns a the parent resource configuration's language settings if the file specified by path doesn't have any of its own. Isn't it the case that, if there aren't any per file settings to start with, the settings are only ever set at the project level / source folder level?  Should your not be creating resource configs where they don't exist?
- This isn't scalable. All the settings are stored in the giant XML-file-of-doom. Even medium sized projects here struggle with a generous 768m heap. The XML processing just kills performance :(  There are tricks to get around this, but it's not pretty.

I suspect it's deliberate that you can't set the read-only attribute on the setting entries. Read-only was used for scanner contributed settings that don't form part of the arguments passed to the managed build tools, and that aren't persisted in the .cproject.

There are so many APIs for contributing paths: PathEntries, ScannerDiscovery, ManagedBuild, externalSettings providers. They all plug in in non-obvious ways to the model. With the new project model, someone with a project in the unified Makefile/Managed world, will have all their custom settings and externalSettings stored in the .cproject.  These will contribute to the set of langEntries passed to the Toolchain.  Paths contributed via scanner discovery contribute to cdt.core but aren't persisted by the model, and aren't passed to the toolchain.

I think this is why we really need a unified path store in cdt.core which handles all issues of persistence and loading, and allows ISVs to add & remove paths under their own unique key. Users can then control which path contributors they want enabled.

Cheers,
James

2009/11/25 Chris Recoskie <recoskie@xxxxxxxxxx>

Basically, I do not rely on the scanner discovery infrastructure at all to update the project description, nor do I try to get the infrastructure to automagically try to update the cache by reloading the project description. Rather, in my job that already serializes the scanner info to the .sc file (via the DiscoveredPathManager), I added some code to directly take the newly discovered scanner info, and directly write it to the ICLanguageSettingEntries myself, then set the project description to get the new data to take.

The only weird behaviour I have remaining is that when I create the settings, I specify that the discovered info is built-in and read only, which is what the project level settings were already populated as. This is supposed to ensure that they do not show up unless you specify in the UI to show the built-in info, and you cannot delete or modify them. Despite setting those flags when creating the settings from the user's build output, when the data is read later from the project description, the flags are blank, so the discovered info shows up in the UI as user-defined info, and is deletable/modifiable. I would still like to fix that, but I can live with that for now. It is far better to have data the user can accidentally delete (and will get recreated when they build again), as opposed to no data at all.

Note that one thing that also seems to be missing from DiscoveredPathManager is the ability to handle changes to include files (i.e. explictitly included via command line options) and macro files, as opposed to the include paths and macro definitions, which it does handle. Hence, my code below doesn't handle include/macro files, because this part of discovery system is not tracking that data anyway. Since overall the scanner info system is supposed to handle that type of data, it would be nice at some point when this part of scanner discovery is overhauled to make sure that those features are supported. The build output parser already supports it, but there's no way to do anything with the data given the current infrastructure. It is rather weird that the lowest software layer supports it, as does the top-most layer, but the middle layer doesn't.

Here is a snippet of some of the code. When I get more time in a week or two I will have a look at doing this for the GNU scanner discovery if required. I suspect it has the same issues but I have not tested for sure that that is the case.