Chapter 54. Embedment Helper

An object that minimizes code in a templating system by providing all needed functions to that templating mechanism.

Many systems allow you to extend the capability of a simple representation by embedding general-purpose code into that representation to do things that otherwise would not be possible. Examples include embedding code into web page templates, putting code actions into grammar files, and putting callouts into code generation templates. This mechanism of general-purpose Foreign Code adds a lot of power to the representation it’s embedded into, without complicating the basic representation itself. However, a common problem when you do this is that the Foreign Code can end up being quite involved and obscure the representation that it’s embedded into.

Embedment Helper moves all the complex code to a helper class, leaving only simple method calls in the host representation. This allows the host representation to be dominant and retain its clarity.

54.1 How It Works

The basic idea behind Embedment Helper is similar to a refactoring. Create the Embedment Helper, make the Embedment Helper visible to the host representation, and take all the code from the host representation and move it to the Embedment Helper, leaving just a method call behind.

There’s only one potentially tricky technical aspect to this, which is getting an object into the visible scope when processing the host representation. Most systems give you some mechanism to do this—they need to in order to call libraries—but it’s sometimes a bit messy.

Once you have the Embedment Helper visible, any code that’s more than a simple method call should move into the Embedment Helper, so the only code left in the host representation is simple calls.

This leaves another complication in this technique, which isn’t anything particularly technical: How to ensure that it’s clear what the code in the Embedment Helper is doing? The key to this, as with any abstraction, is careful naming of the methods, so they clearly state the intention of the called code without revealing its implementation. This is the same basic skill as method and function naming in any context—a central skill of a good programmer.

Embedment Helper is often used with Templated Generation, and when you find this combination, a common question is whether the Embedment Helper should generate output. I often hear this as an absolute point: Helpers must never generate output. I don’t agree with this absoluteness. Certainly, there is a problem with generating output in the helper—any such output isn’t visible from the template. Since the whole point of Templated Generation is that you see the output with holes, such hiding of generated material is, without doubt, a problem.

However, I think that this problem has to be weighed against the complexity of retaining the output in the template and the more complicated constructs of Foreign Code you may need if you want to avoid generating output in it. This is a balance that you have to consider in each case, and although I would say it’s good to avoid generating from the Embedment Helper, I’m not inclined to agree that it is always better than the alternative.

54.2 When to Use It

I’m very suspicious of patterns that someone claims should always be used, but Embedment Helper is one of those things I would always suggest doing, except in really trivial cases. I’ve looked at a fair bit of code using Foreign Code in my time, and I see a huge difference if Embedment Helper is present. Without it, it’s hard to see the host representation, so much so that it rather defeats the purpose of using an alternative representation at all. For instance, a grammar file with lots of Foreign Code in actions makes it very hard to see the basic flow of the grammar.

While preserving the clarity of the host representation is the critical reason for using Embedment Helper, there’s another benefit in terms of tooling. This is most evident if you use a sophisticated IDE. In this case, any embedded code can’t be edited with the IDE’s tooling, but if you move it to a Embedment Helper, you’re back in your full editing environment. Even simple text editors benefit a bit by simple things such as code coloring, which usually won’t work properly for embedded code.

Still, there is one situation where you don’t need an Embedment Helper: where you are using classes that act as a natural home for providing this kind of information. An example of this is if you are doing Templated Generation with a Semantic Model. In this case, much of the behavior that you would have in an Embedment Helper can reasonably be part of the Semantic Model itself—provided this doesn’t make the Semantic Model too complex.

54.3 Secret Panel States (Java and ANTLR)

Perhaps the easiest way to explain how Embedment Helper works is to show what it looks like when you don’t use it. For this, I’ll take an ANTLR grammar file, pretty much the same one I used in the example for Embedded Translation. I won’t show the entire grammar file, but here are a few of the rules:

Along with the code in the code actions, I also need to set up Symbol Tables and any general functions that can avoid duplicated code, such as obtainState. I do this in the members section of the grammar file.

With such inlined code, grammar files can have more lines of Java than of the grammar DSL. For comparison, here’s what it looks like with an Embedment Helper:

transition [Token sourceState]
 : trigger = ID '=>' target = ID {helper.addTransition($sourceState, $trigger, $target);};

The difference is moving the code to the helper. To do this, the first step is to put a helper object into the generated parser. ANTLR allows me to do this by declaring a field in the members section.

This will put a field in the generated parser class. I set its visibility to package so that I can manipulate it with another class. I could make it private and provide getters and setters, but I don’t think that’s worthwhile in this case.

In the overall flow of running this program, I have a loader class that orchestrates the parse. It holds the state machine result, and I create it with a reader.

The run method executes the parse and populates the machine field.

The ANTLR parse is initiated by the line parser.machine. You’ll see that I set the helper in the line before that. In this case, the loader class also acts as the helper. The loader is really quite simple, so it seems better to add the helper behavior to the loader than to make them separate classes.

I then have methods on the helper to handle the various calls. I won’t show them all; here’s the one for adding an event:

To keep the amount of code in the grammar file to the minimum, I pass in the token and let the helper extract the text payload.

One thing I often fret about when using a Parser Generator is whether I should use event-oriented or command-oriented naming for my Embedment Helper. In this case, I’ve used command-oriented names: addEvent and addState. Event-oriented names would be something like eventRecognized and stateNameRecognized. The argument for event-oriented names is that it doesn’t imply any action on the helper, leaving it up to the helper to decide what to do. This is particularly handy if you use different helpers with the same parser that do different things in reaction to the parse. The problem with event-oriented names is that you can’t tell what’s going on by just reading the grammar. In a case where I’m only using the grammar for one activity, I’d rather be able to read the grammar and see from the naming what’s happening at each step.

In this example, I used a separate object as a Embedment Helper. Another approach I can use with ANTLR is to use a superclass. The superClass option in ANTLR allows me to set any class as the superclass of the generated parser. I can then use the superclass as the Embedment Helper, putting all the necessary data and functions in there. The benefit of this is that I can say addEvent rather than helper.addEvent.

54.4 Should a Helper Generate HTML? (Java and Velocity)

A common rule I hear is that a Embedment Helper should not generate any of the output. I don’t consider this to be a useful rule, but I felt that an example would be a good way to explore the tradeoffs a bit more. The example isn’t really connected to DSLs, as it involves creating HTML, but the principles are the same and it saves me having to dream up another contrived example.

Suppose we have a collection of person objects, and we want to print out their names in an unordered list. Each person may have an email address or an URL. If they have an URL, we want a link around the name pointing to the URL; if an email address, we want a mailto link; but there must be no link if there’s neither. Using Velocity as my templating engine, here’s the code to show that:

The problem with this is that I now have a bunch of logic in my template file. This logic can obscure the template layout itself, which is exactly what a Embedment Helper can help with. Here’s an alternative layout using the Embedment Helper to generate the output:

By moving the logic to the helper, I make it easier to follow the template at the cost of some of the HTML not being visible in the template.

But before contemplating the tradeoff fully, I should point out that often in these arguments there is an important middle ground to explore. This is where some of the logic can go into the Embedment Helper without having it generate output.

My point here is that putting some output generation in the Embedment Helper is a reasonable choice. The more complicated the logic and the more complicated the overall template, the more I gain by moving output generation to the Embedment Helper where I can factor it better. The biggest objection to this occurs when you have separate people working on the template (such as an HTML designer) and the code. This leads to a coordination cost for some changes. For instance, suppose the HTML designer wants to add a style class to the link output; if the Embedment Helper generates that link, then the designer has to coordinate with a programmer to make that change. Of course, this is only a problem if you have different people working on the different files; when generating code for a DSL, this is usually not the case.