Chapter 5. Developing a DSL Textual Syntax

You’ve seen how it’s possible to create a graphical concrete syntax using Graphical Modeling Framework (GMF), but a graphical notation might not always be appropriate or sufficient for a domain-specific language (DSL). In software development, textual syntaxes are the most popular and well-supported means by which to work with programming languages, both general purpose and domain specific. Tools to generate parsers, abstract syntax trees, and semantic analysis are quite mature and plentiful. Frameworks designed to support editing, refactoring, versioning, comparison, and merging of textual languages are also common, with Eclipse representing perhaps the most popular on the planet today.

As discussed already, expressing an abstract syntax using metamodels is superior in many ways to using traditional methods of grammar definition. Fortunately, it’s possible to combine abstract syntax definition, grammar specification, and the generation of textual editors that leverage a common underlying framework. Given the importance of textual syntaxes, their support in a DSL Toolkit is essential. An attractive goal is to provide support for simultaneous editing of textual and graphical elements for any language.

In the context of the Modeling project, two components within the Textual Modeling Framework (TMF) project support the development of textual syntaxes for DSLs. Furthermore, as the concept of “language workbenches” becomes increasingly popular, support continues to grow for the generation of high-quality textual editors for a provided language grammar. Within Eclipse, the IDE Meta-Tooling Platform (IMP) project aims to provide such support. In the future, the components of TMF are expected to use IMP as a complementary component for developing textual editors for languages based on an underlying Ecore-based abstract syntax.

At the time of this writing, the TMF project is just underway, and the IMP project is updating its support to allow for the generation of editors for the Eclipse 3.3 platform. Unfortunately, much work remains to be done before a comprehensive chapter on developing a DSL textual syntax can be written that will not be outdated as soon as it is published. Therefore, this chapter introduces each of the two TMF components at a high level, leaving details to the sample projects and a subsequent version of this book. In the meantime, you can visit the TMF home page and wiki to get the latest information on the development progress of these important components.

5.1 Xtext

Xtext is a component of TMF that supports the development of a DSL grammar using an Extended Backus-Naur Form (EBNF)-like language, which can use this to generate an Ecore-based metamodel, Eclipse-based text editor, and corresponding ANTLR-based parser. Unfortunately, the default nature of Xtext is to begin with a grammar and produce an Ecore model, rather than begin with an Ecore model and derive the grammar. Nevertheless, it allows for transformation to and from an Ecore-based model, thereby providing for interoperability with EMF-based technologies such as QVT Operational Mapping Language (QVTO) and Xpand. Additionally, Xtext allows a grammar specification to reference an existing metamodel by using an import mechanism, which gets us closer to the ideal case.

The Checks language is used in the context of the generated metamodel to validate the language semantics. The editor can provide validation to give the user feedback in the form of error and warning messages, complete with the expected Eclipse decorators and markers. Alternatively, validation can be provided during workflow of the model instances in the context of model-to-model or model-to-text transformation. Furthermore, Xtext registers an EMF ResourceFactory for the generated DSL file extension, allowing it to read in (but not write) an instance of the model from the textual syntax.

Although Xtext is capable of generating a functional Eclipse textual editor, complete with syntax highlighting, code assist, outline view, and so on, it does not yet use the IMP project. Hopefully, Xtext and its TMF counterpart, Textual Concrete Syntax (TCS), will provide interoperability with IMP in the future, thereby supporting the proper separation of concerns and reducing duplicated effort in the area of textual editor generation.

At the time of this writing, Xtext is available only from the GMT downloads and includes dependencies to underlying technologies that are incompatible with the versions used to develop the samples in this book. Therefore, this book does not include Xtext examples; you can visit the Xtext component Web site for examples and additional information.

5.2 TCS

The TCS component of TMF provides an alternative to Xtext in defining a textual concrete syntax for a DSL. TCS is itself a DSL that facilitates specifying a textual syntax by linking syntactic information to a metamodel. In doing so, it allows an Ecore-based model to be used with a TCS model to generate a grammar. Sufficient richness of definition allows TCS to generate both model-to-text and text-to-model capabilities.

TCS can produce an annotated grammar using an ATL transformation to be used by the ANTLR parser generator. This provides the injector component for text-to-model translation. An extractor is derived from the language metamodel and TCS model and is used to provide model-to-text translation. The extractor is based on an interpreter that may be used for any language, although it is possible to generate an extractor per DSL.

As with Xtext, TCS does not yet use the IMP project for production of its text editor. Instead, TCS uses the Textual Generic Editor (TGE), which builds upon TCS to provide the editor, outline view, and text-to-model traceability. TCS does provide a “zoo” of languages that have been defined and are available from the project Web site.

At the time of this writing, no downloads are available for TCS, although code is present in the CVS repository. Therefore, this book does not provide detailed examples or sample projects at this time. When TCS and Xtext mature and provide integration with IMP, the book’s sample applications likely will include a textual syntax using one of these components.

5.3 Summary

The ability to define and generate support for textual concrete syntaxes is essential for the future of DSL tooling. With Xtext and TCS providing the beginnings of this last major element in our Eclipse DSL Toolkit, look to the TMF project website and future editions of this book for detailed information on their usage.

Next, we’ll explore model transformation, beginning with model-to-model transformation using QVT Operational Mapping Language.