[cdt-dev] CMake daemon for user tools

[Stephen Kelly <steveire@xxxxxxxxx>]

Hello,

I've been working on adding a daemon mode for cmake to provide
information to user tools - such as IDEs - about the buildsystem.

Following the discussion about providing metadata for IDEs to consume
I proposed creating a long-running process which would provide a protocol
to access information about the buildsystem, and about the content of the
cmake files themselves:

 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004

The original post in that thread lists design goals.

This design is independent of any generator, and compared to the solution of
writing json to a file, this solution doesn't require generating
anything that is
not asked for.  This also side-steps the problem of 'stale' files remaining
in the build directory, and the problem of needing to trigger re-generation
of the file, or watching it for changes.  This solution also allows
parameters (eg the config) to be specified in requests, side-stepping a
difficulty we encountered before to represent things like that.  This
solution also does not require actually generating the buildsystem
files. Only the configure and compute steps of cmake are executed.

I am scheduled to give a talk at FOSDEM about the feature and how user
tools can interact with it:

 https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code

with a preview in Berlin:

 http://www.meetup.com/berlincplusplus/events/227896427/

I have now also pushed a branch to my github clone with the start of the
server mode:

 https://github.com/steveire/CMake/commits/cmake-daemon

Currently the branch only makes targets and backtraces etc available via
the protocol.  I have also created a plan for extending the protocol to
make code completion and variable debugging possible, and analysed the
depedencies of the tasks:

 http://www.steveire.com/cmake-daemon-tasks.png

However, I can't complete those tasks myself: I don't have
relevant experience building IDEs to know how best to design the
protocol, what IDE tools really need, how to design a fail-safe parser
etc.  Additionally, I think a design which incorporates design ideas
and implementation from more than one person will be better in the end.

So, this needs to be a collaborative effort if it is to go anywhere, with
more people writing commits in the cmake repo.

The above (copied below) task list would be enough to get a read-only
browser of a cmake project quite quickly.

Following that, an effort would be needed to handle the difference
between the dirty state of an editor and the on-disk file content. That's
a solvable problem and together with filesystem notifications provided
by libuv, it will lead to features like code completion.

Also necessary eventually would be to make the cmake parser more
fail-safe, so that information is still available if the user
had not yet closed a command argument list for example, and make the
processing more fail-safe in the case where they have written
add_subdirectory(foo), but have not yet created foo/CMakeLists.txt.

I would be looking to people who work on IDEs for guidance on how to do
things like that, as some of them are likely already for handling C++
code in the IDEs.

So, this needs to be a collaborative effort. I can provide guidance to
anyone who wishes to participate in changing the source code of CMake
to make it possible.

Is anyone interested enough in this potential cmake feature to
join the development effort?

Thanks,

Steve.


* Provide basic daemon mode with event loop and communication protocol
-- Written, but not reviewed
* Provide protocol to query the buildsystem structure
-- Written, but not complete or reviewed
* Extend cmListFileParser to record location of start and end of commands
* Rewrite cmIfCommand to record which branches are executed
* Store the location of start and end of user-defined functions
* Provide protocol to query for unexecuted lines
* Define a new snapshot type to occur at an arbitrary point
* Create snapshots after branching to other files (after an include() or
find_package() or add_subdirectory etc)
* Create snapshots after points with external effects (file(WRITE),
configure_file(), execute_process etc)
* Find most recent snapshot occuring before a specified line
* Implement feature to start at a specific snapshot and execute commands
up to a particular line
* Make server find correct snapshot to start at and execute required
commands up to specified line
* Provide protocol to dump variable defintions at a particular line
* Extract implementation of IssueMessage from cmMakefile and
cmLocalGenerator into cmake
* Extract computed target properties from cmTarget::GetProperty into
cmComputedTargetProperties
* Implement cmGeneratorTarget::GetProperties in terms of
cmComputedTargetProperties
* Move implementation of cmTarget::{Get,Set,Append}Property to cmState
* Extract all IssueMessage parts of cmake into a cmMessenger
* Extract the cmSystemTools::Message part of cmMessenger into a virtual
method
* Publish messages through server protocol
* Implement snapshot-versioning of target properties
* Dump target property content
* Implement snapshot versioning of policies
* Implement snapshot-versioning of directory properties
* Implement snapshot-versioning of global properties
* Dump directory, global properties and policies
* Extend protocol to provide dirty buffers from editors, instead of only
executing on-disk commands
* Provide code completion protocol