Textual Modeling Framework (TMF) proposal

Introduction

The Eclipse Textual Modeling Framework (TMF) project is a proposed open source project under the Eclipse Modeling Project.

This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process document) and is written to declare its intent and scope. This proposal is written to solicit additional participation and input from the Eclipse community. You are invited to comment on and/or join the project. Please send all feedback to the http://www.eclipse.org/newsportal/thread.php?group=eclipse.modeling.tmf newsgroup.

Background

EMF is an essential piece of many Eclipse-based modeling projects. It does a very good job of providing abstract syntax for models. However, there was little support at the level of concrete syntax, i.e. concrete notations to describe models, at least until GMF entered the stage.

There are several ways of displaying and editing models. The two primary alternatives are graphical (or visual) concrete syntaxes and textual notations. EMP already has a solution to efficiently develop editors for graphical notations, namely GMF. However, in many situations, textual representations are much better suited. Therefore, the proposed TMF project will provide a means to efficiently create editors for custom textual notations.

While graphical notations are often useful to describe structural concepts (e.g. class diagrams), textual notations are often a better fit for describing behavior or algorithms (e.g. expressions). Another advantage that textual representations have over graphical notations is that expert users often find them faster and easier to modify, and that there is a rich set of existing tooling for dealing with text files (diff, merge, copy & paste, search & replace ...).

Related Issues

XMI defines a standardized way to textually serialize arbitrary models. XMI however was not specified with human readability in mind, and it is very verbose and not a good basis for manual editing.

Because XMI is not ment to be used by humans the OMG issued the Human Usable Text Notation spec that describes the serialization of MOF models using a more consice syntax. This is just another serialization format and can not be used to design textual DSLs, because there is not much you can do about the concrete syntax (it is implied).

The IBM alphaWorks site hosts the Emfatic project (http://www.alphaworks.ibm.com/tech/emfatic), a textual editor for EMF models with a very natural syntax, which resembles well-known programming language syntaxes and hence is really easy to understand. However, the Emfatic editor has a fixed metamodel, ECore, and doesn't solve the needs of users with any other meta-model.

Existing Approaches

TCS (http://www.eclipse.org/gmt/tcs) is a GMT component and a language to define textual notations for metamodels. TCS-based tools also include the Textual Generic Editor (TGE) - a configurable Eclipse textual editor parameterized with information obtained from TCS.

Xtext is part of openArchitectureware and as such part of Eclipse GMT (http://www.eclipse.org/gmt/oaw). It is a framework and generator that provides a specialized Eclipse editor and an EMF metamodel from a simple EBNF-style grammar. It's focus is on very short turn-arounds and it provides powerful abstractions for development of textual DSLs.

Concrete syntax development deemed essential part of the Eclipse Modeling Project (see charter). Textual editors tailored to specific needs seem to be important part of any DSL toolkit and providing tools to create such editors may help adoption of Eclipse modeling technologies.

Description

Based on a metamodel and a syntax specification, the TMF tooling provides the following functionalities:

• A parser that reads the textual representation of the model and instantiates the corresponding EMF model.
• A feature-rich Eclipse Editor (based on the Eclipse text editor infrastructure) that is aware of the concrete syntax specified. The editor supports syntax highlighting, code completion, navigation (hyperlinks), hovers, folding, outline and other features known from Eclipse text editors.
• The editor as well as the parser must be able to plug in various constraint engines (such as OCL or oAW's Check language) in order to provide model validation.

Figure 1 gives an overview of TMF architecture. TMF is intended to be layered. The TMF Common framework provides a generic and powerful EBNF notation. This is just a thin layer over existing parser generator languages (such as Antlr 3). This provides the possibility to derive as much tooling (e.g. content assist) as possible from a specified grammar and decouples from the underlying parser generator technology.

The AST construction is implied for the most common cases but can be modified using actions in order reconstruct the AST. The metamodel is not derived from the grammar and can be of arbitrary types.

TMF Common makes use of existing Eclipse technologies where possible. For instance, the tooling will be based on Eclipse Platform and the new IMP framework. The default AST technology will be EMF.

In order to provide a more convient API for our target audience (DSL developer), TMF has two common types of DSL abstractions on top of the TMF common framework:

• The Xtext language is focused on structural DSLs and short turn-arounds. The metamodel is derived from the grammar.
• The TCS language can binds a concrete syntax to an existing metamodel. It provides model2text transformation based on the grammar as well as an interpretative editor.

Additional Features

Another feature that falls into the scope of this project is defining and generating support for so called 'in-place' editors, i.e. not full-fledged editors for complete models, but rather flexible single-line or multi-line input fields for model fragments that are commonly used in diagrams (think e.g. the UML class attribute, with all the visibility, type, multiplicity information stacked into a compact string) or views (think file rename in the Project Explorer). TMF could therefore provide in-place editing to GMF.

Organization

We propose this project should be undertaken within the Eclipse Modeling Project. The goal of the project is to work along with GMF to complement concrete syntax tooling around EMF. The intention of the project is to utilize M2M and M2T technologies from the Modeling project. Other dependencies are the Eclipse Platform (especially, Text Editor and JFace frameworks) and EMF.

Initial Committers

• Sven Efftinge - itemis AG (proposed co-lead)
• Bernd Kolb - independent consultant
• Frédéric Jouault - INRIA (proposed co-lead)

Interested Parties

• Borland, Inc. (Artem.Tikhomirov@borland.com)
• openArchitectureWare Team
• INRIA TCS Team
• itemis AG
• Gentleware AG
• Elmar Jürgens, Technical University Munich

Contribution

Few projects with the capabilities similar to what this proposal describes are being developed, and we intend to review any proposed contribution during the Validation Phase. Contributions that are aligned with the project goals may be refactored and used as initial code contributions. In particular, the following contributions are deemed worthy of attention:

• TCS
• Xtext

The two contributions have a different approach. The next paragraph will elaborate on the two and explain why both of this contributions are valid and should go into the project as own components.

TCS

TCS is starting with a meta-model. This meta-model has to be annotated by the user with syntax information. From this information TCS generates an ANTLR-based grammar. The provided Eclipse-Editor works interpretative. Furthermore TCS offers model to text generation based on the specified syntax. People who have existing meta models might want to use this approach.

Xtext

As opposed to TCS, with Xtext's grammar language one describes the abstract and concrete syntax at once. Therefore you cannot directly map a concrete syntax to a given meta model but have to do that in a separate step using model-to-model transformation (M2M). Xtext generates an Ecore model (the meta model or abstract syntax), an ANTLR 3 parser and a DSL-specific Eclipse-Editor. The generated code runs on a generic framework based on Eclipse technology. One can declaratively define constaints on the DSL which will be checked in the parser and in the editor. Xtext provides a low learning curve as well as fast turn-arounds. Users which are aware of grammars will use this approach.

Integration with IMP

A group at the T.J. Watson Research Center of IBM is working on SAFARI, a platform to provide rich Eclipse-based tooling for arbitrary textual languages (http://www.oopsla.org/2006/program/demonstrations/d19:_safari:_a_platform_for_generating_language-specific_ide_support.html). SAFARI provides much infrastructure that would be very useful in developing TMF. SAFARI has been proposed as initial contribution for a new Eclipse project called Eclipse IDE Meta-tooling Platform ("The Eclipse IMP") (http://www.eclipse.org/proposals/imp/). TMF will be based on IMP in further versions and support modeling related concepts and abstractions over IMP.

User/developer community

TCS as well as Xtext have proven their usefulness in industrial environments. Therefore both approaches already have a lively user community.

It is also anticipated that the initial user community will be formed by existing GMF users, those already interested in DSL tooling and various concrete syntaxes.