Never tell people how to do things. Tell them what to do and they will surprise you with their ingenuity.
—George S. Patton
U.S. general (1885–1945)
When speaking of creating languages, you need to know the roles of the people involved. What expertise do they need? What are their tasks? What tools are they using? In this chapter, you will learn the difference between using a software language to design software and designing a software language.
Software languages entail two different processes: (1) the design of a language and (2) the use of that language. The first process centers on building a specification of the language and the tooling needed to use the language, so we might call that process language centered. The second process focuses on building software, which is, of course, done using any software language that is for some reason appropriate to the task. In other words, the second process is not focused on the language at all and is more appropriately called application centered. In this book, we focus on the first: the language-centered process.
Likewise, we can think of two completely different roles, or actors: the language engineer and the language user. The language user is responsible for creating software that suits the end user. The language engineer is responsible for supporting the language user in this task. This support comes in the form of languages and tools that help in the use of these languages. The languages created by the language engineer are to be used in the process of application creation by the language user, or application developer. In fact, one could call the language engineer the servant of the language user. The language engineer should make the life of the language user as easy as possible by developing a language or languages that make the job of creating software easier. Part of this task is to create the tooling that will support the language user efficiently and effectively.
Language users far outnumber language engineers. Any language should have many more users than creators. Even so, there is yet another role to be considered, a role that still fewer people will play: the one who creates the tools and methods for the language engineer. Let’s call this role the meta–language engineer. Often this role will be played by an expert language engineer, because to a large extent it resembles the role of language engineer. The meta–language engineer must develop a language to write language specifications in and provide support to use it. This role is a servant to the other language engineers.
A meta–language engineer works in a very dedicated domain, one that he or she knows very well, being part of the targeted user community. The language engineer, by contrast, often has to deal with a much broader spectrum of domains that are not very well known to him or her. Also, being a meta–language engineer is a very specialized job. They can afford to put much time and effort into creating their languages and tool sets. Ordinary language engineers often do not have that luxury. A language engineer who does not quickly produce that language and support tools for, say, specifying the rules that control traffic lights may lose out to someone else.
A language developed by the meta–language engineer is called a formalism; that developed by the language engineer is, in fact, simply called a language. But note that, in fact, a formalism and a language are the same, although they have a different purpose.
For the purpose of this book, a language user’s expertise and tasks are relevant insofar as they concern the language(s) used. From that point of view, the task of a language user is to create a (set of) mogram(s) written in a certain language in order to develop a software application. However, because the language engineer needs to create the tooling that will support the language user, we will take a closer look at the tool set of the language user.
The set of tools that support the language user will, for most readers, be obvious (Figure 2-1). They are the elements of a good integrated development environment (IDE) and can be categorized as follows.
• Editors: tools that enable the language user to create a mogram from scratch or to change an existing mogram.
• Transformers: tools that create a new mogram from an existing one. Well-known examples are compilers, model transformers as known in model-driven development, reverse-engineering tools, documentation generators, and code generators.
• Executors: tools that execute a mogram. Well-known examples are debuggers, simulators, virtual machines, automated testers, and model executors.
• Analyzers: tools that analyze a mogram and produce some judgment about it. Well-known examples are cost estimators—for instance, based on source line count—and model verifiers, which, for instance, proof freedom of deadlocks.
In addition, the language user will use tools that support the process side of application development: for instance, version control and bug reporting. Usually, these tools are generally applicable and not language specific.
For the language user, the line between these tools is blurred. It is not without meaning that the acronym IDE stands for Integrated Development Environment. For example, when a language user wants to execute a Java program in the Eclipse IDE, Eclipse makes sure that any changes in the editor are saved, that the compiler is run and bytecode is generated, and that this bytecode is executed by a virtual machine. For the language user, what appears to be one task involves a number of tools.
Just as the line between different tools is unclear, most language users think about language and tool as intertwined. From the point of view of the language user, it is not the language’s syntax rules that forbid the language user from writing an error; it is the compiler providing an error message. Some people state, erroneously, that a program is a language with a tool. Yet when creating languages, it is very important to be aware of the difference between the two.
One task of a language engineer is to develop languages that make the job of creating software easier. Another task is to create the tooling for the newly created language that will support the language user efficiently and effectively. What tools are available to a language engineer?
Many people working in the IT industry are familiar with the application-centered process and its supporting tools, the programming language IDEs, and the modeling tools. But not many people are familiar with the language-design process and its tool environment, which is often called language workbench [Fowler 2005]. Note that a language workbench is used by both the language engineer and the language user.
A language workbench should support the language engineer in creating
• A language specification that is consistent, coherent, and comprehensible
• Tools that support the language user
A language workbench is somewhat similar to an IDE, but you should clearly distinguish the elements that support the language engineer and the ones that support the language user.
To support the language engineer in creating a language specification, a language workbench must include at least one editor for the formalism in which the language specification is written. (Later on, we will see that a language engineer can use several formalisms for one language specification.) It would be helpful to include some process-support tools and some analyzers for language specifications, although I am not aware of any standard ways to analyze a language specification. The language workbench need not include an executor for the formalism, because you cannot execute a language specification. For instance, you cannot execute the specification (metamodel) of how the language user is going to create rules for traffic light control systems, although it would be a good idea to test the tools that are created for this purpose. There is at the moment a slight need for a transformer of language specifications from one language workbench to another, because the language-engineering support provided by the various language workbenches differs greatly. One often wants to enjoy the possibilities of a number of workbenches together.
To support the language user, the language workbench needs to operate as an ordinary IDE. For this purpose, every language should have at least one editor and one transformer or one executor. Since not all languages need to be executable, an executor is not always used. In that case, the language workbench should at least contain a transformer, in order to make the mograms produced in the language available to other tools. For instance, the rules that specify the controls of a set of traffic lights need not be executable but need to be translated into the hardware description language, such as VHDL or Verilog.
Because some software languages—for instance, some specialized DSLs—will be used by only a limited number of people, the effort of creating a language and its tool support should be kept to a minimum. Thus, the tools that support the language user should not be hand-crafted, but instead they should be generated by the language workbench. For a language engineer, therefore, a very important part of a language workbench is the set of tool generators available. These tools are transformers, which change a language specification into a (set of) tool(s).
Note that there must be several generators, because the language user needs several types of tools. The language engineer should create at least one editor and one transformer or executor. For instance, for most DSLs, an editor and a code generator are created. Because generators are such important instruments for a language engineer, we take a closer look at them in the next section.
The tool generators the language engineer uses are a special form of transformer. Each tool generator takes as input a language specification and produces the source code for the tool that the language user is to use. Tool generators are by no means new. Some interesting earlier generators of language user tools are Centaur [Borras et al. 1988] and the Synthesizer Generator [Reps and Teitelbaum 1988].
Unfortunately, research into tool generators has not yet fully developed. For instance, there are no generators for incremental parsers. (There are incremental parser generators [Heering et al. 1989], but these produce nonincremental parsers.) Some current frameworks that include generators for language user tools are
• Eclipse Modeling Framework (EMF) together with the Graphical Modeling Framework (GMF) and openArchitectureWare (oAW)
• Generic Modeling Environment (GME)
• Intentional Domain Workbench
• Jetbrains Meta Programming System (MPS)
• MetaEdit+
• Microsoft DSL Tools
• Together DSL Toolkit (a combination of EMF, GMF, and oAW)
In the context of language engineering, we have to distinguish the following roles:
• The language user, the person who simply uses a software language to create an application
• The language engineer, the person who creates software languages
• The meta–language engineer, the person who creates tools for the language engineers
This book is aimed at language engineers. Associated with the different roles is a difference in the tool sets of a language user and a language engineer. The tool set for a language user is called an Integrated Development Environment (IDE) and it contains
• Editors: tools that enable the language user to create a mogram from scratch or to change an existing mogram
• Transformers: tools that create a new mogram from an existing one
• Executors: tools that execute a mogram
• Analyzers: tools that analyze a mogram and produce some judgment about it
• Process support tools: tools that support the process side of application development
The tool set used by a language engineer is called a language workbench. It is similar to an IDE but contains generators that are able to generate (parts of) the tool set for the language user for a given language.