I always have doubts when I explain the advantages of multiple mograms to create one application. Do I really need to explain this? When you think of a program, it is all so obvious, but in the modeling world, these ideas have not yet been embraced. So let’s start by looking at the characteristics of using multiple source code files to build one application, all of which are taken for granted but are nevertheless very important for the process of creating software.

The source code for one application is divided into separate files because (1) a single source file would be too large to handle and (2) doing so enables the following features:

I don’t think that any programmer would want to work with one single source file containing all the code for one application. The days are long gone when programmers were creating a single main program in a single file, but this is what we continue to do in the modeling world. One huge (main) model file contains all we need to specify about the application. In practice, this file often becomes too large to handle. It takes minutes to load and when you want to generate code from it (read: compile it), it takes a large part of an hour. No wonder most programmers prefer programs over models.

If you want your DSLs to be used, your IDE should offer your language users at least the same features their programming environment offers them. The users will reject anything less from what they are currently used to.

The arguments for working with multiple files apply when all mograms are written in a single language. Therefore, the remainder of this chapter should be of interest to you, whether you will be creating a large set of coordinated software languages or only one simple language.

Keep in mind that the number of applications written in a single language is extremely small. Most commonly, one application is created using multiple mograms in multiple languages. Language engineers will need to address this. The rest of this chapter explains how the preceding advantages can be realized in such a complex situation. Using this knowledge in the simpler situation of a single language is up to you.

Combining mograms from different languages means that one mogram can hold references to things defined in another mogram; for example, one class file can hold references to other classes and operations. Here, two different types of references are of importance: the soft reference, which is also known as referencing by name, and the hard reference, which is also known as referencing by address.

Hard referencing (by address) means that you know exactly where the element that you are referencing is positioned in the other mogram. You use this position to indicate which element in the second mogram your mogram is referring to. This makes the two mograms closely connected. When the mogram to which your mogram holds a reference changes, your mogram needs to be changed as well. An example of hard referencing is an IP address, such as 101.92.85.100.

Soft referencing (by name) depends on identifiers. It means that the first mogram holds an identifier that unambiguously identifies an element in another mogram. This identifier can be a name, but it can also have another form: for instance, a path name in which the (file) name of the mogram containing the element occurs. An example of a soft reference is a Web site address, such as www.mycompany.com.

When you are as bad at remembering names as I am, you often encounter the problems of hard referencing. For instance, I know that I recently spoke to an interesting person whose office was in the second room on the left on the third floor. Now that I want to renew this contact, it appears that higher management decided to make some office changes. The room in which I was expecting to find this interesting person is now occupied by someone else, someone who does not know where the previous resident is located. The only thing I can hope for is to remember some distinguishing mark so that I will be able to ask the secretary something like, “Do you know where the tall guy with the large mustache is?” On the other hand, the problem with soft referencing is that every time you want to contact this interesting person, you have to somehow determine where he is located. You always need to visit the secretary first.

We know from past experience that in software, the close connection of hard referencing is unwanted. General knowledge in software engineering says that parts of a software system should be loosely coupled. So to avoid strongly coupled mograms, our new software languages should cater to soft referencing.

Soft referencing has two sides—or even or three, as discussed in Section 11.4.1. First, you must be able to identify elements in the mogram to be referenced. Let’s call this mogram the passive one. Second, a way to specify outgoing references in the referencing mogram must be provided. Let’s call this second mogram the active one.

To cater to the passive side, the language of the passive mogram should provide a form of identification for the elements defined in a mogram. The language developer can use several identification schemes: for instance, model name followed by the type of the element in the abstract syntax model followed by the name of the element. Instead of using the name of an element, the language engineer may decide to use a specially added attribute, representing a unique identifier within a mogram, to each concept (class) in the abstract syntax model.

To cater to the active side, the language engineer must design a way to reference an element in another mogram. Certain elements in the active mogram are references to elements in the passive mogram. Still, the active mogram needs to be an instance of its abstract syntax model. So the question is: What is a reference in terms of the abstract syntax model?

Because every concept is represented in the abstract syntax model as a class, references must also be represented as one or more classes. You can choose one class to represent all references. In that case, it is practical to have a number of attributes in this class that hold metadata: that is, data on the reference, such as the language of the passive mogram, the version number, and so on. This simplifies resolving the reference. You can also decide to use several classes: for instance, one for every passive language that a language user is allowed to access or even one for every type of element (read: abstract syntax class) you want to reference.

All reference classes must be aware of the identification scheme of the passive language. A reference class usually holds attributes that correspond to the elements of the identification scheme. For instance, the class OperationRef in Figure 11-1 implements a reference to an operation of a class in a UML model. Each instance of this class holds as metadata the version of the UML language. The actual reference is divided into four elements: the model-id, the diagram-id, the class-name, and the operation-name. The diagram-id is present because the UML model can have multiple diagrams, each of which could hold the definition of the requested class. Note that the two classes in Figure 11-1 are not part of the abstract syntax model of UML but instead are part of the abstract syntax model of another (active) language. The UML is the passive language in this case.

Figure 11-1. An example of a reference class

The language engineer has full control over the identification scheme used in the passive language. Thus, the language engineer can also decide to hide some elements from being referenced by any active mogram. For instance, in most object-oriented programming languages, you can write a class that contains references to other classes, operations (methods), and attributes (fields). But you cannot reference a parameter of an operation, a statement, or the initial value expression of an attribute. These (and more) are hidden.

Note that this kind of hiding is different from that indicated by the use of public and private markers. According to the language specification, there is a way to reference a private attribute or operation; thus, language users are able to make a reference, but the compiler will tell you that it is illegal. In other words, the private marker operates in the same manner as a constraint: When your mogram contains an illegal reference to a private element, the mogram is still an instance of the abstract syntax model, but a constraint is violated, and so it is not a valid instance.

By hiding elements, the language engineer makes it impossible for a language user to refer to them. For example, in an object-oriented programming language, there is no way to reference the first assignment in an operation. No language user is able to do that. In our terminology, you can say that the mogram is not an instance of the abstract syntax model, not even an invalid instance. The notion of information hiding by the language engineer can be brought to an extreme when you define a language interface.

So far, we have looked at language interfaces from the passive side. We have determined that a language interface can hide elements from active mograms. This type of language interface, called the offered language interface, defines what is available to other languages, without knowing the other languages that might make use of this.

We can also look at language interfaces from the active side: What would an active mogram want to reference from another language? In other words, on what information from passive mograms does the active mogram depend? The language of the active mogram may define that its mograms depend on certain information in mograms of a certain language. The active mogram’s language specification should in that case define a required language interface, which defines what is needed from the other language or languages.

OCL is an example of a language that needs another language. Expressions written in OCL require information from a UML model. Figure 11-3 shows part of the OCL abstract syntax model [OMG-OCL 2005]. The gray classes belong to OCL; the white classes, to UML. When we were working on this model, we had no idea of separating the two languages by a language interface. But already, our intuition was to provide a small and clean way to link them. Thus, the number of classes from the UML used in the OCL model is relatively small, and all associations between OCL classes and UML classes are one-way: from OCL to UML. Because of this neat interface to UML, OCL is easily combined with other languages. All that needs to be done is to provide the right classes to substitute for the UML classes.

Figure 11-3. Part of the OCL abstract syntax model, which interfaces with UML

Note that not every mogram needs to use the elements in a required interface. For instance, I could write perfectly legal OCL expressions without making any references to properties or operations. In this example, however, the expressions would be very simple indeed, such as “23 – 11 = 12” or “true or false implies true.”

The most obvious task of a multilanguage IDE is to resolve and check references. You could implement a reference resolver per language, but it is probably wiser to implement a single resolver that is able to resolve references to all the languages supported by the IDE. Having similar identification schemes in all languages makes this easier.

I know of two examples of multilanguage IDEs that offer this type of support. One is xText, which is part of Open Architecture Ware [openArchitectureWare 2008]; the other was created using Microsoft DSL tools by my longtime colleague and writing partner, Jos Warmer. Present in this environment are four DSLs corresponding to the layers of the application architecture, which is a common architecture for Web-based, service-oriented administrative applications:

1. The Web Scenario DSL for the presentation layer

2. The Business Class DSL for the business classes

3. The Services DSL for the service interface and business processes

4. The Data Contract DSL for the data contract

On Save, each mogram will write to file (an easy-to-access XML format) the information on its elements that are part of its language’s offered interface. At the moment this export file becomes available in the environment, a special broker component reads it. When a reference needs to be resolved, the editor or other component that needs it requests the information from the broker. For instance, when an editor needs a list of elements of a certain type to be used for code completion, it asks the broker for a list of names. The broker then selects this list from all the elements known based on the required language, required type of element, and possibly other things, such as the first characters of the name of the element. In fact, the broker is a rather simple database that stores and retrieves the information in all the export files of all mograms.

Checking references, of course, also involves the broker. In this case, the broker is asked whether a certain element still exists. If the broker cannot find this element in its database, the reference is faulty, and an error message is presented to the language user. You can find more information in Warmer and Kleppe [2006].

How should the IDE support changes? If a language user changes a name of an element in a passive mogram and this element is being referred to (by name) in a number of active mograms, what should happen? What if the element is completely removed?

There are a number of ways to do this. The choice depends on the time and budget for the development of the IDE and on what you think your users will and will not accept.

Having no support at all does work but is cumbersome for the language users. The problem with automatic support is that the language users do not know where automatic changes take place and may encounter unexpected results. This might lead to the need to retest the whole system, because the changes were not controlled.

The best option seems to be refactoring support. Note that in this case, renaming a model element works exactly as renaming a class in C# or Java code. The user either changes the name, which results in dangling references, or requests an explicit refactoring and is offered the possibility of reviewing the resulting changes and applying them. The reference broker holds all the information needed to implement refactoring support.

In both automatic and refactoring support, the following problem may occur. Especially in large projects, there will be many dozens of mograms, each of which can be edited simultaneously by different users. To allow for automatic change propagation or refactoring, the user performing the change needs to have change control over all affected mograms. The use of the merge feature of a version control system solves this problem. However, most version-control systems work only with textual mograms and cannot handle graphical mograms very well.

The use of multiple languages also has consequences for code generation. Similar to compilation in programming language IDEs, we want to provide code generation per mogram whenever possible. In the scenario of multiple mograms per application, both a mogram and the code generated from that mogram are partial descriptions of the application. For the process of code generation, it is important to determine when the parts need to be brought together. In the extreme, there are two possibilities.

1. Link all mograms together to form the complete description of the application. Transform the complete description into code.

2. Transform every single mogram into code. Link the generated code.

Again, the programming language world offers a good example. In C#, the notion of partial classes exists. A partial class is part of a class description saved in a separate file. Thus, one single class description is scattered over a number of files. When this class needs to be compiled, every partial class must be known, as in option 1. On the other hand, the compilation of a number of different classes is completely independent of each other, as in option 2.

As this example shows, the two options are extremes. In most cases, a mixture of both is used. The language designer should focus on option 2 and use option 1 only if the information in two or more mograms needs to be transformed into a single source code file. For example, when the class and state diagrams belonging to one UML model are saved in separate files, you need to combine a class with the state chart defined for it before you can generate code. In my experience, such situations are rare. Most often, you will generate multiple source code files from one mogram. Remember that when you take option 2, the mapper pattern (Section 10.3) can be of help, especially for references to other mograms.