| Re: [cdt-dev] CDT as a Standalone API |
On 31.10.2014 16:20, Krishna Narasimhan wrote:
This is great.
Is it open source?
Not the translation. That's a commercial product.
Or at least the part where you extended CDT?
Lets see:
> extended multiple AST classes to have more rewriting support (we
> extended AST rewriting in general).
It would be great if this could be put into CDT. The AST classes are sometimes somewhat messy (inconsistent/incomplete). E.g.:
- IASTFunctionCallExpression.setArguments(IASTInitializerClause[])
- IASTStandardFunctionDeclarator.addParameterDeclaration(IASTParameterDeclaration)
The first one is not very nice, but at least it allows you to add, remove and insert arguments. The second has a nicer interface, but removing and inserting parameters is not possible. You have to replace the declarator in its parent which might be a function definition, a standard declaration or another declarator :-/ And of course you loose its file location information when replacing the function declarator with a new one.
There are more cases like this. We added a rewriter class that takes care of all that. But if the interfaces were unified and extended, it would be much easier to do AST rewriting in CDT.
> - reimplemented comment handling.
> - reimplemented binding handling.
> - reimplemented type computation.
> - reimplemented constant _expression_ evaluation.
> - reimplemented pretty printing.
This functionality is already in CDT. It's just not always completely correct, so we had to reimplement it. What's in CDT now should be ok for most use cases. Replacing them with our implementation would break too much of the API.
> - added CFG computation/analysis features.
I assume this can be done with codan. But I didn't find too much documentation and judging by what we already had to reimplement, it seemed easier to just write that ourselves from scratch. Especially since it also has to handle Java AST (e.g. exceptions).
> - added macro analysis/transformation features.
> - added support for keeping user formatting (e.g. empty lines).
Probably only needed by us and certainly not compatible with the CDT API.
I now quickly looked through our CDT patches. We also rewrote the assembler handling. This would also be great to get into CDT.
So, except the AST rewriting and assembler support, I don't think adding our extensions to CDT makes sense (or is possible). But I'm more than happy to help out with getting the other two in.
Thanks!
Marco
On Fri, Oct 31, 2014 at 4:15 PM, Marco Trudel <marco.trudel@xxxxxxxxxxxx<mailto:dmitrynpetrov@gmail.__com<mailto:marco.trudel@mtsystems.ch>> wrote:
On 30.10.2014 01:59, Alena Laskavaia wrote:
Along same lines as Dmitry mentioned, it is theoretically
possible to
use headless CDT for code transformations. You do have to use
workspace
concept,
We use CDT for extensive code analysis and transformation. Without
creating a workspace, just by parsing and processing C files. I
doing this is quite common, but from this thread it seems it's not.
This is surprising since we found CDT very useful for that. And it
is (IMHO) certainly more maintainable than writing C or C++ for Clang.
Since what we're doing might be interesting to the list, let me give
a quick overview and then CDT related technical details:
We implemented an automatic C source code to Java source code
translator:
http://www.mtsystems.ch (contains an online demo and translated
programs)
Such a translation allows to:
- Easily upgrade legacy C projects to Java.
- Automatically create interfaces of C libraries.
- Offer services for Java (e.g. code analysis/proofs) also for C.
The translation is completely automatic, supports the entire C
language and creates functionally equivalent Java code - ready to be
executed. Features:
- Pointers are optimized away or translated to readable Java
classes, function pointers are translated to method references.
- Full support for unsigned types.
- Full support for goto statements.
- Full support for native libraries; the translated Java project
seamlessly interfaces with libraries (GMP, Ncurses, X11, ...) the C
project used.
- Macros are translated to Java methods or constants.
- Comments are preserved and reformatted to Javadoc.
- The translated code adheres to the Java design and naming conventions
- ...
We have evaluated our translation software on a number of C
programs, including wget, less, xeyes, and micro httpd. We keep
extending and optimizing it and will soon also put the translation
of vim online.
As great and helpful CDT was for this project, there were some
things we had to extend, replace or add. We:
- extended multiple AST classes to have more rewriting support (we
extended AST rewriting in general).
- reimplemented comment handling.
- reimplemented binding handling.
- reimplemented type computation.
- reimplemented constant _expression_ evaluation.
- reimplemented pretty printing.
- added CFG computation/analysis features.
- added macro analysis/transformation features.
- added support for keeping user formatting (e.g. empty lines).
And probably some more I forgot now.
Cheers!
Marco
but I don't think it is big deal, because nobody has to know
how you doing it internally. I.e. you can import code in temp
workspace
as project, configure scanner discovery, run indexer, do code
transformations, rewrite code, exit and delete workspace afterwards.
Location of the code does not have to be physically inside
workspace, it
is eclipse place to store writeable stuff. And if you do all of
this and
it works, would be nice if you contribute this framework/example
back :)
On Wed, Oct 29, 2014 at 12:02 PM, Dmitry Petrov
<dmitrynpetrov@xxxxxxxxx <mailto:dmitrynpetrov@gmail.com>
<mailto:dmitrynpetrov@gmail.com>>> wrote:
You can implement a stand-alone application based on CDT.
org.eclipse.cdt.codan.__internal.core.CodanApplication
might be a good
example if you want to do something with source code.
Let me share some of my experience as a person who uses
both Clang
and CDT for tool development.
Unfortunately, neither Clang nor CDT solves C/C++
source-to-source
transformation issues related to the preprocessor quite
well. That's
a very difficult area, though.
ClangTooling doesn't add anything special to Clang ASTs model.
Clang doesn't represent preprocessor statements in AST.
There are
some mechanisms to match comments to declarations, but
nothing more.
You'll have to hack things on top of preprocessor callbacks
and AST
traversal yourself.
On Wed, Oct 29, 2014 at 6:49 PM, Krishna Narasimhan
<krishna.nm86@xxxxxxxxx <mailto:krishna.nm86@xxxxxxxxx>
<mailto:krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>__>> wrote:
I tried clang. Not the ClangTooling. The core Clang
seemed to
have issues like not preserving comments after rewrites
etc. Do
the Tooling handle Preprocessor statements too?
I thought clang was primarily an optimmization
framework and not
a source code transformation tool.
On Wed, Oct 29, 2014 at 4:40 PM, Nathan Ridge
<zeratul976@xxxxxxxxxxx <mailto:zeratul976@xxxxxxxxxxx>
<mailto:zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>__>> wrote:
> Basically I would like to perform write test
codes that work on ASTs
> not obtained from any project from the IDE.
Basically Parsed from
> Strings or other text files. Perform
modifications to that AST and be
> able to use the rewritten AST , all this possibly
without a code from
> the IDE
If you're writing a standalone tool for creating,
analyzing,
and rewriting
ASTs, you might want to consider the clang
infrastructure [1].
Regards,
Nate
[1] http://clang.llvm.org/docs/__Tooling.html
<http://clang.llvm.org/docs/Tooling.html>
_________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
To change your delivery options, retrieve your
password, or
unsubscribe from this list, visit
https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>
--
------------------------------__-----------------------
I dare do all that may become a man; Who dares do more,
is none
- Macbeth, twelfh night!
Regards
Krishna
_________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>
_________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>
_________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>
_________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>
--
-----------------------------------------------------
I dare do all that may become a man; Who dares do more, is none -
Macbeth, twelfh night!
Regards
Krishna
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev