Xtext is an open source Eclipse framework for implementing Domain Specific Languages together with their integration in the Eclipse IDE. Xtext allows you to implement languages quickly by covering all aspects of a complete language infrastructure, starting from the parser, code generator, or interpreter, up to a full Eclipse IDE integration, with all the typical IDE features such as editor with syntax highlighting, code completion, error markers, automatic build infrastructure, and so on.
This book will incrementally guide you through the very basics of DSL implementation with Xtext and Xtend, such as grammar definition, validation, and code generation. The book will then cover advanced concepts such as unit testing, type checking, and scoping. Xtext comes with good and smart default implementations for all these aspects. However, every single aspect can be customized by the programmer.
Although Java can be used for customizing the implementation of a DSL, Xtext fosters the use of Xtend, a Java-like programming language completely interoperable with the Java type system which features a more compact and easier to use syntax and advanced features such as type inference, extension methods, multi-line template strings and lambda expressions. For this reason, we will use Xtend throughout the book.
Most of the chapters have a tutorial nature and will describe the main concepts of Xtext through uncomplicated examples. The book also uses test driven development extensively. The last chapters will describe more advanced topics such as Continuous Integration and Xbase, a reusable Java-like expression language that ships with Xtext which can be used in your DSLs.
This book aims at being complementary to the official documentation, trying to give you enough information to start being productive in implementing a DSL with Xtext. This book will try to teach you some methodologies and best practices when using Xtext, filling some bits of information that are not present in the official documentation.
The chapters are meant to be read in order, since they typically refer to concepts that were introduced in the previous chapters.
All the examples shown in the book are available online, see the section Downloading the example code. We strongly suggest that you first try to develop the examples while reading the chapters and then compare their implementations with the ones provided by the author.
After a small introduction to the features that a DSL implementation should cover, including integration in an IDE, the book will introduce Xtend since it will be used in all the examples. The book proceeds by explaining the main concepts of Xtext. For example, validation, code generation, and customizations of runtime and UI aspects. The book will then show how to test a DSL implemented in Xtext with JUnit in order to follow a Test Driven Development strategy that will help you to quickly implement cleaner and more maintainable code. The test-driven approach is used in the rest of the book when presenting advanced concepts such as type checking and Scoping. The book also shows how to build a DSL with Continuous Integration mechanisms and how to produce a release of your DSL so that others can install it in Eclipse. At the end of the book Xbase is introduced. Finally, the book describes some advanced topics and mechanisms that can be used when implementing an Xtext DSL.
Chapter 1, Implementing a DSL, gives a brief introduction to Domain Specific Languages (DSL) and sketches the main tasks for implementing a DSL and its integration in an IDE. The chapter also shows how to install Xtext and gives a first idea of what you can do with Xtext.
Chapter 2, Creating Your First Xtext Language, shows a first example of a DSL implemented with Xtext and gives an introduction to some features of the Xtext grammar language. The chapter describes the typical development workflow of programming with Xtext and provides a small introduction to EMF (Eclipse Modeling Framework), a framework on which Xtext is based.
Chapter 3, Working with the Xtend Programming Language, describes the main features of the Xtend programming language, a Java-like language interoperable with the Java type system. We will use Xtend in all the other chapters, to implement every aspect of an Xtext DSL.
Chapter 4, Validation, describes validation, in particular, the Xtext mechanism to implement validation, that is, the validator. This chapter is about implementing additional constraint checks that cannot be done at parsing time. It also shows how to implement quickfixes corresponding to the errors generated by the validator.
Chapter 5, Code Generation, shows how to write a code generator for an Xtext DSL using the Xtend programming language. The chapter also shows how a DSL implementation can be exported as a Java standalone command-line compiler.
Chapter 6, Customizing Xtext Components, describes the main mechanism for customizing Xtext components, Google Guice, a Dependency Injection framework. In particular, the chapter shows how to customize both the runtime and the UI aspects of an Xtext DSL.
Chapter 7, Testing, describes how to test a DSL implementation using JUnit and the additional utility classes provided by Xtext. The chapter shows the typical techniques for testing both the runtime and the UI aspects of a DSL implemented in Xtext.
Chapter 8, An Expression Language, covers the implementation of a DSL for expressions, including arithmetic, boolean, and string expressions. The chapter shows how to deal with recursive rules and with typical problems when writing Xtext grammars. The implementation will be described incrementally and in a test-driven way. The chapter also shows how to implement a type system for checking that expressions are correct with respect to types and how to implement an interpreter for these expressions. Some hints for optimizing the performance of a DSL implementation are also presented.
Chapter 9, Type Checking, covers the implementation of a small object-oriented DSL, which can be seen as a smaller version of Java that we call SmallJava. This chapter shows how to implement some type checking techniques that deal with object-oriented features, such as inheritance and subtyping. The chapter also describes some good practices in Xtext DSL implementations.
Chapter 10, Scoping, covers the main mechanism behind visibility and cross-reference resolution in Xtext. Since scoping and typing are often strictly connected and inter-dependent especially for object-oriented languages, the chapter is based on the SmallJava DSL introduced in the previous chapter. The chapter describes both local and global scoping and how to customize them.
Chapter 11, Continuous Integration, describes how you can release your DSL implementation by creating an Eclipse update site so that others can install it in Eclipse. The chapter also describes how to build and test your DSL implementation on a continuous integration server, using Maven/Tycho or Gradle. We will also show how to have a web application with a web editor for your DSL, and how your DSL implementation can be easily ported to IntelliJ.
Chapter 12, Xbase, describes Xbase a reusable expression language interoperable with Java. By using Xbase in your DSL, you will inherit all the Xbase mechanisms for performing type checking according to the Java type system and the automatic Java code generation.
Chapter 13, Advanced Topics, describes a few advanced topics concerning an Xtext DSL implementation, and some advanced techniques. In the first part of the chapter we will show how to manually maintain the Ecore model for the AST of an Xtext DSL. We will show how to create an Xtext DSL starting from an existing Ecore model, how to switch to a manually maintained Ecore model, starting from the one generated by Xtext, and how to use Xcore to maintain the Ecore model for the AST. In the last section we will show how to extend an Xbase DSL with new expressions, customizing the Xbase type system and the Xbase compiler in order to handle the new Xbase expressions.
Chapter 14, Conclusions, concludes the book by providing a few references to some additional material available online.