In this chapter, we will implement a DSL for expressions, including arithmetic, boolean, and string expressions. We will do that incrementally and in a test-driven way. Since expressions are by their own nature recursive, writing a grammar for this DSL requires some additional efforts, and this allows us to discover additional features of Xtext grammars.
You will also learn how to implement a type system for a DSL to check that expressions are correct with respect to types, for example, you cannot add an integer and a boolean. We will implement the type system so that it fits the Xtext framework and integrates correctly with the corresponding IDE tooling.
Finally, we will implement an interpreter for these expressions. We will use this interpreter to write a simple code generator that creates a text file with the evaluation of all the expressions of the input file, and also to show the evaluation of an expression in the editor.
This chapter will cover the following topics:
- Additional details on Xtext grammars
- Dealing with left recursion in an Xtext grammar
- Writing a small type system for expressions
- Writing an interpreter
- Some hints on fine tuning a DSL implementation
In the DSL we develop in this chapter, which we call Expressions DSL, we want to accept input programs consisting of the following statements: variable declarations with an initialization expression and evaluations of expressions. Variable declarations have the shape var name = exp and evaluation statements have the shape eval exp. Expressions can refer to variables and can perform arithmetic operations, compare expressions, use logical connectors (and and or), and concatenate strings. We will use + both for representing arithmetic addition and for string concatenation. When used with strings, the + will also have to automatically convert integers and Booleans occurring in such expressions into strings.
Here is an example of a program that we want to write with this DSL:
var i = 0 var j = (i > 0 && 1 < (i+1)) var k = 1 eval j || true eval "a" + (2 * (3 + 5)) // string concatenation eval (12 / (3 * 2))
For example, "a" + (2 * (3 + 5)) should evaluate to the string "a16".
First of all, we will use the Xtext project wizard to create the projects for our DSL (following the same procedure explained in Chapter 2, Creating Your First Xtext Language).
Start Eclipse and perform the following steps:
- Navigate to File | New | Project..., and in the dialog, navigate to the Xtext category and click on Xtext Project.
- In the next dialog, provide the following values:
- Project name:
org.example.expressions - Name:
org.example.expressions.Expressions - Extensions:
expressions
- Project name:
- Press Finish.
- The wizard will create several projects, and it will open the
Expressions.xtextfile, which is the grammar definition.
Before writing the Xtext grammar for the Expressions DSL, it is important to spend some time to understand how the rules in an Xtext grammar and the corresponding EMF model generated for the AST are related.
From the previous chapters, we know that Xtext, while parsing the input program, will create a Java object corresponding to the used grammar rule. Let's go back to our Entities DSL example and consider the rule:
Entity:
'entity' name = ID ('extends' superType=[Entity])? '{'
attributes += Attribute*
'}' ;When the parser uses this rule, it will create an instance of the Entity class (that class has been generated by Xtext during the MWE2 workflow). However, the actual creation of such an instance will be deferred to the first assignment to a feature of the rule; in this example, no object will be created when the input only contains entity; the object will be created as soon as a name is specified, for example, when the input contains entity A. This happens because such an ID is assigned to the feature name in the rule. This is reflected in the outline view, as shown in the following two screenshots:
This also means that the created Entity object is not "complete" at this stage, that is, when only a part of the rule has been applied. That is why when writing parts of the DSL implementation, for example, the validator, the UI customizations, and so on, you must always take into consideration that the model you are working on may have some features set to null.
The actual creation of the object of the AST can be made explicit by the programmer using the notation {type name} inside the rule; for example:
Entity:
'entity' {Entity} name = ID ('extends' superType=[Entity])? '{'
attributes += Attribute*
'}' ;If you change the rule as shown, then an Entity object will be created as soon as the entity keyword has been parsed, even if there has not been a feature assignment.
In the examples we have seen so far, the type of the object corresponds to the rule name; however, the type to instantiate can be specified using returns in the rule definition:
A returns B:
... rule definition ...
;In this example, the parser will create an instance of B. A is simply the name of the rule, and there will be no generated A class. Indeed, the shape of the rule definitions we have used so far is just a shortcut for:
A returns A:
... rule definition ...
;That is, if no returns is specified, the type corresponds to the name of the rule.
Moreover, the returns statement and the explicit {type name} notation can be combined:
A returns B:
... {C} ... rule definition ...
;In this example, the parser will create an instance of C (and the class C is generated as a subclass of B). However, the object returned by the rule will have type B. Also in this case, there will be no generated A class.
The Xtext editor highlights rule's name and rule's type differently—the types are in italic font.
When writing the grammar for the Expressions DSL, we will use these features.