C H A P T E R 12
Dynamic Languages on JVM
So far, we've been discussing the DLR and its applications. In this chapter, we will look at the framework in the Java world that's equivalent to the DLR. We will see how that Java framework facilitates running dynamic language code on the Java virtual machine, how to host dynamic languages in a Java program, and what kind of language interoperability the framework supports. When appropriate, we'll compare the Java framework and the DLR. If you have a need to write Java programs that use dynamic languages, this chapter will help you quickly get up to speed by leveraging what you've learned so far.
Quick Comparisons
You'll find that support for dynamic languages in the .NET and Java worlds corresponds to each other quite nicely. Before we compare each feature area in more detail, it's helpful to have an overall view of how the .NET and Java platforms stack up against each in their support for dynamic languages. To that end, Table 12-1 shows some quick comparisons. The first column shows dynamic language support on the .NET platform, the second shows the corresponding support on the Java platform.
Table 12-1. Support for Dynamic Languages on .NET and Java
| .NET | Java |
| IronPython is an implementation of the Python language that runs on .NET CLR. | Jython is an implementation of the Python language that runs on the JVM. |
| IronRuby is an implementation of the Ruby language that runs on .NET CLR. | JRuby is an implementation of the Ruby language that runs on the JVM. |
| DLR Hosting API | Java Specification Request (JSR) 223 Scripting for the Java Platform is the equivalent API of the DLR Hosting API in the Java world. The JSR 223 API allows hosting dynamic languages in a Java program. |
| DLR binders, dynamic objects | On the Java side, the closest thing to DLR binders and dynamic objects is JSR 292: Supporting Dynamically Typed Languages on the Java Platform. |
In Table 12-1 you can see that the equivalent of the DLR Hosting API on the Java side is the JSR 223 API. Table 12-2 compares the two APIs.
Table 12-2. Comparison of DLR Hosting API and JSR 223.
| DLR Hosting API | JSR 223 Scripting for the Java Platform |
| ScriptRuntime class | ScriptEngineManager class |
| ScriptEngine class | ScriptEngine class |
| App.config. DLR Hosting API uses the App.config file to discover script engines that are available in a deployment environment. | META-INF/services/javax.script.ScriptEngineFactory. JSR 223 uses the javax.script.ScriptEngineFactory file to discover script engines that are available in a deployment environment. |
| ScriptScope class | ScriptContext class |
| CompiledCode class | CompiledScript class |
| ScriptSource class | N/A. JSR 223 does not define a class that's equivalent to DLR Host API's ScriptSource class. |
| Silverlight. DLR Hosting API makes it possible to run dynamic language code not only in a standalone application but also in Silverlight applications. | Servlet container. Using JSR 223, one can host dynamic language code in not only standalone Java applications but also in a Servlet container. |
This chapter is organized into two parts. The first part will look at some well-known dynamic language implementations that run on the JVM and also how to host those languages in Java programs using JSR 223. The second part will develop a simple programming language that deals with logic operations. We will see how to provide a JSR 223 implementation for the simple language so that it can be hosted in Java programs. You will find that almost all of the discussions in this chapter correspond nicely to our earlier discussions on DLR.
Python and Ruby on JVM
Many dynamic language implementations use the JVM as their runtime. That means those implementations will turn their code into Java bytecode at run time and have the JVM execute the bytecode. This is analogous to what happens on the .NET side. Like the JVM executing Java bytecode, the .NET CLR executes Intermediate Language (IL) instructions. Language implementations such as IronPython and IronRuby run on the CLR by eventually translating their code into IL code and having the CLR execute the IL code.
In this part of the chapter, we will look at Jython and JRuby, two dynamic language implementations that run on the JVM. You'll see how those language implementations provide interoperability with Java code, as well as how to use the Java equivalent of the DLR Hosting API to host Python and Ruby code in Java programs. You'll find the code for this in the JavaHostingExamples project. In order to run the code, you need the following software components installed:
- JDK 6 or above. The version I used for developing the code examples is JDK 1.6.0 update 14.
- Eclipse IDE. The version I used is Eclipse 3.6.
- Jython. I downloaded version 2.5.1 and installed it in C:jython2.5.1. After the installation, you need to copy C:jython2.5.1jython.jar to C:ProDLRsrcExamplesChapter12JavaScriptinglibJython
- JRuby. I downloaded version 1.5.1 and installed it in C:jruby-1.5.1. After the installation, you need to copy C:jruby-1.5.1libjruby.jar to C:ProDLRsrcExamplesChapter12JavaScriptinglibJRuby1.5.1.
- Normally you would need to download the script engines that implement the JSR 223 API for the languages we want to host in a Java program. However, in this case, we don't need to bother because the files jruby.jar and jython.jar already contain the script engines. If you use a dynamic language implementation that doesn't include a JSR 223 script engine in its binaries, you can go to
https://scripting.dev.java.net/and see if a JSR 223 script engine that implements the JSR 223 API is available for the language.
Hosting Python Code in Java Programs
Jython is to JVM as IronPython is to CLR. IronPython is an implementation of the Python language that runs on the CLR. Similarly, Jython is an implementation of the Python language that runs on the JVM. One key benefit of targeting the CLR or JVM as the runtime is the ability to leverage the enormous assets of readily available .NET or Java libraries. IronPython gives Python developers access to .NET libraries while Jython gives them access to Java libraries. Earlier we saw examples of Python code that uses classes written in C#. Now we'll look at some examples of Python code that run on JVM and uses classes written in Java.
Listing 12-1 shows an example that uses the JSR 223 API to host Python code in a Java program. If you recall in Chapter 6 how we used the DLR Hosting API to host Python code in a C# program, you'll understand the code in Listing 12-1 right away. With the DLR Hosting API, we create an instance of ScriptRuntime and use it to get an instance of ScriptEngine for executing Python code. Here in Listing 12-1 we create an instance of javax.script.ScriptEngineManager and use it to get an instance of javax.script.ScriptEngine. To get an instance of javax.script.ScriptEngine that's capable of running Python code, the code in Listing 12-1 calls the getEngineByName method on the ScriptEngineManager instance and passes the string “jython” as the argument. In Chapter 6, to get an instance of Microsoft.Scripting.Hosting.ScriptEngine that's capable of running Python code, we call the GetEngine method on a ScriptRuntime instance and pass the string “python” as the argument.
To show the interoperability between Jython and the Java language, the example in Listing 12-1 creates an instance of a Java class called Product whose code is shown in Listing 12-2. The Product class has two fields: name and price. For each field, the Product class provides getter and setter methods for getting and setting the field's value. Once the Product instance is created, the example in Listing 12-1 calls the put method on the ScriptEngine instance to put the Product instance into the script engine's variable bindings. In this case, the Product instance is bound to the name handClapper. This step is equivalent to calling the SetVariable method on a Microsoft.Scripting.Hosting.ScriptEngine instance. With the DLR Hosting API, we call the SetVariable method on a Microsoft.Scripting.Hosting.ScriptEngine instance to bind an object to a name. Once an object is bound to a name, the Python code can retrieve the object by that name and that's what the Python code in Listing 12-1 does. The code is the string “print handClapper.getName()” passed to the eval method of ScriptEngine. This step is equivalent to calling the Execute method on a Microsoft.Scripting.Hosting.ScriptEngine instance.
Listing 12-1. A Java Program That Uses the JSR 223 API to Host Python Code
private static void printProductNameInPython()
throws ScriptException, NoSuchMethodException, IOException {
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jython");
Product handClapper = new Product("Hand Clapper", 6);
engine.put("handClapper", handClapper);
engine.eval("print handClapper.getName()");
}
Listing 12-2. A Java Class Whose Instance Will be Passed to Python Code
public class Product {
private String name;
private int price;
public Product(String name, int price) {
this.name = name;
this.price = price;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getPrice() {
return price;
}
public void setPrice(int price) {
this.price = price;
}
}
As you can see, each line of code in Listing 12-1 has a counterpart in the DLR Hosting API. The concept of passing Java objects to Python code is also the same as that in the DLR Hosting API. The example in Listing 12-1 prints the name of a product in Python code. The next example in Listing 12-3 also prints the name of a product in Python code. The difference here is that the Python code is used as an invocable function in the Java program. The code in Listing 12-3 is largely the same as the code in Listing 12-1 except for lines 6 and 7. In Listing 12-3 at line 6, instead of evaluating Python code as a string, the example code reads the Python code from the file printProductName.py in the src/main/resources folder. This is analogous to calling the ExecuteFile of DLR Hosting API's ScriptEngine class to execute Python code in a file. If you open the file printProductName.py, you'll see that it defines a Python function like this:
def printProductName(x):
print x.getName()
In line 7 of Listing 12-3, the code casts the type of the engine variable to Invocable and calls the invokeFunction method on the type-casted variable. The purpose of this line of code is to invoke the printProductName function defined in printProductName.py. As you can see, the syntax here for invoking a Python function in Java is not as seamless and intuitive as the syntax we get with the DLR Hosting API. Using the DLR Hosting API, we refer to a Python function as a .NET delegate and we directly invoke the .NET delegate to execute it. Here in Listing 12-3, we have to invoke the Python function by its name, instead of calling it directly as if it were a Java method.
Listing 12-3. A Java program That Uses the JSR 223 API to Invoke a Python Function
1) private static void printProductNameByInvokingPythonFunction()
2) throws ScriptException, NoSuchMethodException, IOException {
3) ScriptEngineManager manager = new ScriptEngineManager();
4) ScriptEngine engine = manager.getEngineByName("jython");
5) Product handClapper = new Product("Hand Clapper", 6);
6) engine.eval(new FileReader(new File("src/main/resources/printProductName.py")));
7) ((Invocable) engine).invokeFunction("printProductName", handClapper);
8) }
Hosting Ruby Code in Java Programs
The previous examples show how to host Python code in Java programs. Let's see how to do the same thing for Ruby code. The example code in Listing 12-4 uses some Ruby code to calculate the total price of two products. The Ruby code is placed in the calculateTotal.rb file in the src/main/resources folder, and it looks like this:
def calculateTotal(x, y)
return x.getPrice() + y.getPrice()
end
The Ruby code defines a function called calculateTotal that takes two input parameters. The calculateTotal function calls the getPrice method on the two input parameters and returns the sum of the two prices. The code in Listing 12-4 is very similar to the code in Listing 12-3. In Listing 12-4, we get a script engine that knows how to evaluate Ruby code by calling the getEngineByExtension method of ScriptEngineManager. We create two instances of the Java Product class and pass them to the Ruby calculateTotal function. In order to call the Ruby calculateTotal function, we need to cast the type of the engine variable to Invocable and then we invoke the Ruby function by its name.
Listing 12-4. A Java Program That Uses the JSR 223 API to Host Ruby Code
private static void calculateProductTotalInRuby() throws ScriptException, NoSuchMethodException, IOException {
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByExtension("rb");
Product handClapper = new Product("Hand Clapper", 6);
Product stretchString = new Product("Stretch String", 8);
engine.eval(new FileReader(new File("src/main/resources/calculateTotal.rb")));
Long total = (Long) ((Invocable) engine).invokeFunction("calculateTotal", handClapper, stretchString);
System.out.println("Total is: " + total);
}
As we saw in the examples, the JSR 223 API is the equivalent API of the DLR Hosting API. Like the DLR Hosting API, the JSR 223 API defines a uniform API for hosting different dynamic languages. Before JSR 223, Scripting for the Java Platform, (and its predecessor, the Bean Scripting Framework, or BSF), many languages were already communicating with Java. Some languages would take textual code as input from a Java program and return the evaluation result back. Others would keep references to objects in a Java program, invoke methods on those objects, or create new instances of a Java class. Because each language would communicate with Java in its own way, developers would have to learn the script engine's proprietary programming interface every time they wanted to use a script engine in their Java programs.
To solve this problem, JSR 223 defines a contract that all script engines conforming to the specification must honor. The contract consists of a set of Java interfaces and classes, as well as a mechanism for packaging and deploying a script engine. The DLR Hosting API is also designed to solve the same problem that JSR 223 addresses. When you work with script engines conforming to JSR 223 or the DLR Hosting API, you'll always program to the same set of interfaces defined by the specification. The details specific to the script engine are well encapsulated, and you'll never need to concern yourself with them.
JSR 223 and the DLR Hosting API help not only consumers, but also producers of script engines. If you have designed and implemented a programming language, you can reach out to a broader audience and make your software friendlier to use by wrapping it with a layer that implements the JSR 223 interfaces or one that supports the DLR Hosting API.
Overview of the BoolScript Example
In the rest of this chapter, we'll play the role of a script-engine producer. The examples so far show how to use JSR 223 script engines that others have implemented. In the rest of this chapter, you will learn how to implement a JSR 223 script engine for a custom language of your own. Before we look at the JSR 223 interfaces and an implementation of them, I'd like to point out that though the name of the JSR contains the word scripting, that's not to say you're limited on the languages that can be integrated with Java the JSR 223 way. You can take any language you fancy and wrap it with a layer that conforms to the contract laid out in JSR 223. The language can be object-oriented, functional, or in any other programming paradigm. The rest of this chapter will implement a simple language and then wrap it with a layer that supports the JSR 223 API. Because the language is simple, we can stay focused on the topic of JSR 223 without the details of a complex language overwhelming us.
Don't worry whether you have prior experience constructing a programming language of your own. This article is not about programming languages; it's about JSR 223's contract between programming languages and Java.
Figure 12-1 shows all the parties in our example and how they relate to each other. The example defines a simple language that I affectionately call BoolScript. I will refer to the program that compiles and executes BoolScript code as the BoolScript engine. Besides compiling and executing BoolScript code, to qualify as a JSR 223 script engine, the BoolScript engine also implements the contract defined in the JSR 223 specification. As depicted in Figure 12-1, all the BoolScript engine's code is packaged into a single jar file called boolscript.jar.
Figure 12-1. Overview of the BoolScript example
JSR 223 is a specification in and of itself. Beginning with version 6.0, the Java Standard Edition (J2SE) includes a JSR 223 framework that implements the JSR 223 specification, and that's the middle box in Figure 12-1. In addition to BoolScript.jar and J2SE's JSR 223 framework, the third component, shown on the right, is a Java program that uses the BoolScript engine. The program hosts the BoolScript engine, and its code is in BoolScriptHostApp.java. Notice that a host Java program always interacts with a script engine indirectly via a JSR 223 framework.
To run the example, you need Java SE 6.0 or above. You'll also need the ANTLR runtime component because I used ANTLR to develop the lexer and parser of BoolScript. You can download the Java runtime of ANTLR at www.antlr.org/download.html. The file I downloaded is antlr-runtime-3.2.jar. Put the jar file in the C:ProDLRsrcExamplesChapter12JavaScriptinglibAntlr3.2 folder.
Once you have the software components installed, you're all set to run this part of the chapter's code example, which is divided into the following two Eclipse projects:
- BoolScript contains the source code of the BoolScript engine.
- BoolScriptDemo contains the source code of the host Java program.
The BoolScript project is already compiled and packaged into a jar file called boolscript.jar, which goes in C:ProDLRsrcExamplesChapter12JavaScriptinglibBoolScript and is referenced by the BoolScriptDemo project.
To run the example, you need only compile the host Java program in BoolScriptHostApp.java and run the generated Java .class file. You need to include the two files antlr-runtime-3.2.jar and boolscript.jar in the Java classpath when running the host Java program. After running the example, you'll see output like this:
Bool Script Engine
Mozilla Rhino
answer of boolean expression is: false
answer of boolean expression is: false
answer of boolean expression is: true
answer of boolean expression is: false
answer of boolean expression is: true
BoolScript Language
Before we delve into the details of JSR 223, let's quickly go over the BoolScript language. BoolScript is so simple that all you can do with it is evaluate Boolean expressions. Here's what code written in BoolScript looks like:
As you can see, BoolScript supports two operators: & (logic AND) and | (logic OR). Besides operators, it supports three operands: True, False, and variables whose values might be either True or False. That's it for BoolScript.
You'll find the ANTLR grammar file that defines the syntax of BoolScript in C:ProDLRsrcExamplesChapter12JavaScriptingBoolScriptsrcmain esourcesBoolScript.g
Listing 12-5 shows the simplified version of BoolScript's syntax grammar, which essentially defines that a BoolScript program is zero or more expressions. Each expression is terminated by a semicolon, and a program is terminated by the EOF (end of file) token. The grammar goes on and defines an expression as a term followed by either “& or “|”, followed by another term. A term is defined as a variable identifier (the IDENT token), an expression enclosed by left and right parentheses, and the keyword “True” or the keyword “False”. During development time, the grammar file BoolScript.g is used to generate the lexer and parser Java code that knows how to parse BoolScript code at runtime. The generated lexer and parser Java code is in the BoolScriptLexer.java and BoolScriptParser.java files of the BoolScript Eclipse project.
Listing 12-5. Simplified Version of the BoolScript Syntax Grammar
program : (expression ';')* EOF ;
expression : term ('&' term | '|' term)*;
term : IDENT
| '(' expression ')'
| 'True'
| 'False';
fragment LETTER : ('a'..'z' | 'A'..'Z') ;
fragment DIGIT : '0'..'9';
IDENT : LETTER (LETTER | DIGIT)*;
Script Engine Factory
To see what a JSR 223 framework does between a host Java program and a script engine, let's assume you want to use a script engine in your Java program. First, you'll need to create an instance of the script engine. Second, you'll need to pass textual code to the engine and have the engine evaluate it. Alternatively, you might want the engine to compile the code and save the compiled code for later execution. Let's walk through these steps, bearing in mind that whatever we do, we want to use the script engine only through the JSR 223 framework.
To check whether a script engine is available for a certain language in your deployment environment, you first create an instance of javax.script.ScriptEngineManager and then use it to query the existence of a script engine. You can query the existence of a script engine by its name, its mime types, or file extensions. If we store BoolScript code in *.bool files, the file extension in our case would be bool. The code below queries the existence of the BoolScript engine by file extension:
ScriptEngineManager engineMgr = new ScriptEngineManager();
ScriptEngine bsEngine = engineMgr.getEngineByExtension("bool");
The BoolScript project specifies the file extensions of BoolScript source files in BoolScriptEngineFactory. The class implements the methods getExtensions, getMimeTypes, and getNames of the javax.script.ScriptEngineFactory interface. And it is in those methods that I declared the name, mime types, and file extensions of the BoolScript language. Listing 12-6 shows the code of the BoolScriptEngineFactory class. As you can see, the name of the BoolScript engine is declared to be “Bool Script Engine” and the version is 1.0.0. The name of the BoolScript language is declared to be “Bool Script Language”. The mime type of BoolScript language source code is declared to be “code/bool”.
Listing 12-6. BoolScriptEngineFactory Class
public class BoolScriptEngineFactory implements ScriptEngineFactory {
public String getEngineName() {
return "Bool Script Engine";
}
public String getEngineVersion() {
return "1.0.0";
}
public String getLanguageName() {
return "Bool Script Language";
}
public String getLanguageVersion() {
return "1.0.0";
}
public ScriptEngine getScriptEngine() {
return new BoolScriptEngine();
}
@Override
public List<String> getNames() {
ArrayList<String> extList = new ArrayList<String>();
extList.add("bool script");
return extList;
}
@Override
public List<String> getExtensions() {
ArrayList<String> extList = new ArrayList<String>();
extList.add("bool");
return extList;
}
@Override
public List<String> getMimeTypes() {
ArrayList<String> extList = new ArrayList<String>();
extList.add("code/bool");
return extList;
}
//Other methods omitted.
}
Script Engine Discovery Mechanism
Both the DLR Hosting API and JSR 223 allow us to create a script engine in a language-specific way or in a language- agnostic way. For example, with the DLR Hosting API, we can create a Python script engine in a language-specific way like this:
ScriptEngine engine = Python.CreateEngine();
We can achieve the same thing in a language-agnostic way like this:
ScriptRuntime scriptRuntime = ScriptRuntime.CreateFromConfiguration();
ScriptEngine scriptEngine = scriptRuntime.GetEngine("python");
Chapter 6 already covers those two different ways of creating a Python script engine in the DLR Hosting API, so I'll save the explanations here and go straight to showing you the language-specific and -agnostic ways of creating a script engine in JSR 223. We can create a BoolScript script engine in a language-specific way by invoking the BoolScriptEngine constructor like this:
ScriptEngine bsEngine = new BoolScriptEngine();
We can achieve the same thing in a language-agnostic way like this:
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByExtension("bool");
You might wonder, why bother using ScriptEngineManager to create an instance of BoolScriptEngine when we can create it ourselves simply by invoking the constructor of BoolScriptEngine. Well, you can certainly do that. In fact, I did that a few times for the purpose of quick testing when I developed the example code. Creating a script engine directly might be okay for testing a script engine, but for a real usage scenario, it violates the principle that a host Java program should always interact with a script engine indirectly via a JSR 223 framework. It defeats JSR 223's purpose of information hiding. JSR 223 achieves information hiding by using the Factory Method design pattern to decouple script engine creation from a host Java program. Another problem with directly instantiating a script engine's instance is that it bypasses any initializations that ScriptEngineManager might perform on a newly created script engine instance. Are there initializations like that? Read on.
Given the string bool, how does ScriptEngineManager find BoolScriptEngine and create an instance of it? The answer is something called the script engine discovery mechanism in JSR 223. This is the mechanism by which ScriptEngineManager finds the BoolScriptEngine class. In the following discussion of this mechanism, you'll see what initializations ScriptEngineManager does to a script engine and why.
According to the script engine discovery mechanism, a script engine provider needs to package all the classes that implement a script engine plus one extra file in a jar file. The extra file must have the name javax.script.ScriptEngineFactory. The jar file must have the folder META-INF/services, and the file javax.script.ScriptEngineFactory must reside in that folder. If you look at boolscript.jar's contents, you'll see this file and folder structure.
The content of the file META-INF/services/javax.script.ScriptEngineFactory must contain the full names of the classes that implement ScriptEngineFactory in the script engine. In our example, we have only one such class, and the file META-INF/services/javax.script.ScriptEngineFactory looks like this:
net.sf.model4lang.boolscript.engine.BoolScriptEngineFactory
After a script engine provider packages his or her script engine in a jar file and releases it, users install the script engine by putting the jar file in the Java classpath. Figure 12-2 shows the events that take place when a host Java program asks the JSR 223 framework for a script engine.
Figure 12-2. How JSR 223 discovers a script engine
When asked to find a particular script engine by name, mime types, or file extensions, a ScriptEngineManager will go over the list of ScriptEngineFactory classes (i.e., classes that implement the ScriptEngineFactory interface) that it finds in the classpath. If it finds a match, it will create an instance of the engine factory and use the engine factory to create an instance of the script engine. A script engine factory creates a script engine in its getScriptEngine method. It is the script engine provider's responsibility to implement the method. If you look at BoolScriptEngineFactory, you'll see that our implementation for getScriptEngine looks like this:
public ScriptEngine getScriptEngine()
{
return new BoolScriptEngine();
}
The method is very simple. It just creates an instance of our script engine and returns it to ScriptEngineManager (or whoever the caller is). What's interesting is after ScriptEngineManager receives the script engine instance, and before it returns the engine instance back to the client Java program, it initializes the engine instance by calling the engine's setBindings method. This brings us to one of the core concepts of JSR 223: variable bindings. After I explain the concepts and constructs of bindings, scope, and context, you will know what the setBindings call does to a script engine.
Bindings, Scope, and Context
Recall that the BoolScript language allows you to write code like this:
(True & x) | y
But it doesn't have any language construct for you to assign values to the variables x and y. I could have designed the language to accept code like this:
x = True
y = False
(True & x) | y
But I purposely left out the assignment operator = and require that BoolScript code must execute in a context where the values of the variables are defined. This means that when a host Java program passes textual code to the BoolScript engine for evaluation, it also needs to pass a context to the script engine, or at least tell the script engine which context to use. The idea of using a context to pass objects between a host Java program and the hosted script code is the same as the ScriptScope class in the DLR Hosting API. The ScriptContext class defined in JSR 223 is equivalent to the ScriptScope class in the DLR Hosting API.
You can think of a script context as a bag that contains data you want to pass back and forth between a host Java program and a script engine. The construct that JSR 223 defines to model a script context is the interface javax.script.ScriptContext. A bag would be messy if we put a lot of things in it without some type of organization. So to be neat and tidy, a script context (i.e., an instance of ScriptContext) partitions data it holds into scopes. The construct that JSR 223 defines to model the concept of scope is the interface javax.script.Bindings. Figure 12-3 illustrates context, its scopes, and data stored therein.
Figure 12-3. Context and scope in script engine managers and script engines
There are several important things to note in Figure 12-3:
- A script engine contains a script context.
- A script engine manager (i.e., an instance of
ScriptEngineManager) can be used to create multiple script engines. - A script engine manager contains a scope called global scope, but it does not contain a context.
- Each scope is basically just a collection of name-value pairs. Figure 12-3 shows that one of the scopes contains a slot whose name is x and a slot whose name is y. A scope is an instance of
javas.script.Bindings. - The context in a script engine contains a global scope, an engine scope, and zero or more other scopes.
- A script engine can be used to evaluate multiple scripts (i.e., separated code snippets written in the script language).
What do we mean by the global scope and the engine scope in Figure 12-3 and why do we need them? A global scope is a scope shared by multiple script engines. If you want some piece of data to be accessible across multiple script engines, a global scope is the place to put the data. Note that a global scope is not global to all script engines. It's only global to the script engines created by the script engine manager in which the global scope resides.
An engine scope is a scope shared by multiple scripts. If you want some piece of data to be accessible across multiple scripts, an engine scope is the place to put the data. For example, say we have two scripts like this:
(True & x) | y //Script A
(True & x) //Script B
If we want to share the same value for x across the two scripts, we can put that value in the engine scope held by the script engine we will use to evaluate the two scripts. And suppose we want to keep the value of y to only Script A. To do that, we can create a scope, remembering that this scope is visible only to Script A, and put the value of y in it.
As an example, Listing 12-7 shows the interpretBoolCodeExample method in BoolScriptHostApp.java. The method evaluates the BoolScript code x & y; True | y; using the variable bindings that exist in the script engine's scope.
Listing 12-7. Evaluating BoolScript Code in a Script Engine's Scope
private static void interpretBoolCodeExample() {
ScriptEngineManager scriptManager = new ScriptEngineManager();
List<Boolean> boolAnswers = null;
ScriptEngine bsEngine = scriptManager.getEngineByExtension("bool");
try
{
bsEngine.put("x", new Boolean(true));
bsEngine.put("y", new Boolean(false));
boolAnswers = (List<Boolean>) bsEngine.eval("x & y; True | y;");
printAnswers(boolAnswers);
}
catch (Exception ex)
{
System.out.println(ex.getMessage());
}
}
The code puts the values of both x and y in the engine scope, then it calls the eval method on the engine to evaluate the BoolScript code. If you look at the ScriptEngine interface, you'll see that the eval method is overloaded with different parameters. If we call eval with a string as we did in Listing 12-7, the script engine will evaluate the code in its context. If we don't want to evaluate the code in the script engine's context, we have to supply the context we'd like to use when we call eval. Listing 12-7 shows how to use the eval method of the BoolScriptEngine class to evaluate BoolScript code. Next we'll look at how the eval method is implemented.
BoolScript Engine
Listing 12-8 shows the code in the eval method of the BoolScriptEngine class. The BoolScriptEngine class is the JSR 223 wrapper around the BoolScript language so that BoolScript language code can be hosted in a Java program the JSR 223 way. The eval method implemented in the BoolScriptEngine class is responsible for taking in BoolScript code as a string, parsing it, and evaluating it. As you can see from Listing 12-8, the eval method calls the static parse method of BoolScriptParser to parse BoolScript code. The result of the parsing is a list of BoolExpression instances. This is in line with what I mentioned earlier about the syntax of the BoolScript language. In the section where we looked at the grammar definition of the BoolScript language, we saw that a BoolScript program consists of zero or more expressions. It therefore shouldn't be surprising that I chose to use a list of BoolExpression instances to represent the result of the syntax parsing. Once a BoolScript program is parsed into a list of BoolExpression instances, evaluating the program becomes a matter of evaluating each BoolExpression instance. In order to do this, the eval method in Listing 12-8 needs a scope that contains necessary variable bindings. In Listing 12-8, we get a reference to the engine scope by calling the getBindings method on the context that's passed to the eval method as a parameter. Because more than one scope might be in a context, we indicate that we want to get the engine scope by passing the constant ScriptContex.ENGINE_SCOPE to the getBindings method.
Listing 12-8. The eval Method of the BoolScriptEngine Class
public Object eval(String script, ScriptContext context) {
Bindings bindings = context.getBindings(ScriptContext.ENGINE_SCOPE);
List<BoolExpression> expressions = BoolScriptParser.parse(script);
List<Boolean> result = new ArrayList<Boolean>(expressions.size());
for (BoolExpression expression : expressions)
result.add(expression.eval(bindings));
return result;
}
Listing 12-9 shows the code of the BoolExpression interface. The interface defines an eval method that's supposed to evaluate a BoolScript expression when called. In the BoolScript Eclipse project, you can find several classes such as AndExpression, OrExpression and VarExpression that implement the BoolExpression interface. Each of those classes will implement its own specific logic for the eval method defined in the BoolExpression interface. The eval method implemented in the AndExpression class takes the left and right subexpressions of a Boolean AND operator and does the evaluation by performing a logical AND operation. Listing 12-10 shows how the eval method is implemented in the AndExpression class.
Listing 12-9. The BoolExpression Interface
public interface BoolExpression {
boolean eval(Map<String, Object> bindings);
Set<VarExpression> getVariables();
String toTargetCode();
}
Listing 12-10. The eval Method Implemented in the AndExpression Class
public class AndExpression implements BoolExpression {
private BoolExpression left;
private BoolExpression right;
public AndExpression(BoolExpression left, BoolExpression right) {
this.left = left;
this.right = right;
}
@Override
public boolean eval(Map<String, Object> bindings) {
return left.eval(bindings) & right.eval(bindings);
}
//other methods omitted.
}
The VarExpression class represents variables in the BoolScript language. Because BoolScript code relies on the script context to provide variable binding, evaluating a VarExpression instance means retrieving the variable's value from the script context. Listing 12-11 shows the eval method implemented in the VarExpression class. The code in the eval method simply calls the get method on the parameter bindings, which represents the variable bindings in the scope of the expression evaluation. The code in the eval method looks up the variable's value by the variable's name in the bindings parameter.
Listing 12-11. The eval Method Implemented in the VarExpression Class
public class VarExpression implements BoolExpression {
private String varName;
public VarExpression(String varName) {
this.varName = varName;
}
@Override
public boolean eval(Map<String, Object> bindings) {
return (Boolean) bindings.get(varName);
}
//other methods omitted.
}
Finally, I am ready to explain why a script engine manager initializes a script engine by calling the engine's setBindings method: When a script engine manager calls an engine's setBindings method, it passes its global scope as a parameter to the method. The engine's implementation of the setBinding method is expected to store the global scope in the engine's script context.
Before we leave this section, let's look at a few classes in the scripting API. I said that a ScriptEngineManager contains an instance of Bindings that represents a global scope. If you look at the javax.script.ScriptEngineManager class, you'll see that there is a getBindings method for getting the bindings and a setBindings method for setting the bindings that represent the global scope in a ScriptEngineManager.
A ScriptEngine contains an instance of ScriptContext. If you look at the javax.script.ScriptEngine interface, you'll see the methods getContext and setContext for getting and setting the script context in a ScriptEngine.
So nothing prevents you from sharing a global scope among several script engine managers. To do that, you just need to call getBindings on one script engine manager to get its global scope and then call setBindings with that global scope on other script engine managers.
If you look at our example script engine class BoolScriptEngine, you won't see it keeping a reference to an instance of ScriptContext explicitly. That's because BoolScriptEngine inherits from AbstractScriptEngine, which already has an instance of ScriptContext as its member. If you ever need to implement a script engine from scratch without inheriting from a class such as AbstractScriptEngine, you'll need to keep an instance of ScriptContext in your script engine and implement the getContext and setContext methods accordingly.
Compile BoolScript Code
By now, we've implemented the minimum for our BoolScript engine to qualify as a JSR 223 script engine. Every time a Java client program wants to use our script engine, it passes in the BoolScript code as a string. Internally, the script engine has a parser that parses the string into a tree of BoolExpression instances commonly called an abstract syntax tree, then it calls the eval method on each of BoolExpression instances in the tree to evaluate the BoolScript program. This whole process of evaluating BoolScript code is called interpretation, as opposed to compilation. And in this role, the BoolScript engine is called an interpreter, as opposed to a compiler. To be a compiler, the BoolScript engine would need to transform the textual BoolScript code into an intermediate form so that it wouldn't have to parse the code into an abstract syntax tree every time it wanted to evaluate it.
If you recall, the DLR Hosting API provides functionality for compiling dynamic code. With the DLR Hosting API, once the dynamic code is compiled, we can execute it multiple times in different script scopes. With JSR 223, once the dynamic code is compiled, we can also execute it multiple times in different script contexts. This section will show you how to compile BoolScript code into JVM bytecode and execute the bytecode in different script contexts. Java programs are compiled into an intermediate form called Java bytecode and stored in .class files. At runtime, .class files are loaded by classloaders, and the JVM executes the bytecode. Instead of defining our own intermediate form and implementing our own virtual machine, we'll simply stand on the shoulder of Java by compiling BoolScript code into Java bytecode.
The construct JSR 223 defines to model the concept of compilation is javax.script.Compilable, which is the interface BoolScriptEngine needs to implement. Figure 12-12 shows the runCompiledBoolScriptExample method in BoolScriptHostApp.java that demonstrates how to use the compilable BoolScript engine to compile and execute BoolScript code.
Listing 12-12. Compiling BoolScript Code into Java Bytecode
private static void runCompiledBoolScriptExample() throws ScriptException,
NoSuchMethodException {
ScriptEngineManager scriptManager = new ScriptEngineManager();
ScriptEngine engine = scriptManager.getEngineByExtension("bool");
CompiledScript compiledScript = ((Compilable) engine).compile("x & y;");
Bindings bindings = new SimpleBindings();
bindings.put("x", true);
bindings.put("y", false);
List<Boolean> result = (List<Boolean>) compiledScript.eval(bindings);
for (Boolean boolValue : result)
System.out.println("answer of boolean expression is: " + boolValue);
}
In Listing 12-12, the variable engine is an instance of BoolScriptEngine that we know also implements the Compilable interface. We cast it to an instance of Compilable and call its compile method to compile the code x & y. Listing 12-13 shows the implementation of the compile method in BoolScriptEngine.
Listing 12-13. The Compile Method of BoolScriptEngine
public CompiledScript compile(String script) throws ScriptException {
BoolScriptCompiler compiler = new BoolScriptCompiler(this);
compiledScript = compiler.compileSource(script);
return compiledScript;
}
The compile method of BoolScriptEngine creates an instance of BoolScriptCompiler and calls its compileSource method. Internally, the compileSource method transforms the BoolScript code x & y into the following Java code:
package boolscript.generated;
import java.util.*;
import java.lang.reflect.*;
class TempBoolClass {
public static List<Boolean> eval(boolean x, boolean y)
{
List<Boolean> resultList = new ArrayList<Boolean>();
boolean result = false;
result = x & y;
resultList.add(new Boolean(result));
return resultList;
}
}
The transformation converts BoolScript code into a Java method inside a Java class. The class name and method name are hard-coded to be TempBoolClass and eval respectively. Each variable in BoolScript code becomes a parameter in the Java eval method. You can find the code that performs the conversion from BoolScript code to Java code in the compileBoolCode method of the BoolScriptCompiler class.
Transforming BoolScript code to Java code is just half the story. The other half is about compiling the generated Java code into bytecode. I chose to compile the generated Java code in memory using JSR 199, the Java Compiler API, a feature that begins to be available in Java SE 6.0. Details of the Java Compiler API are beyond the scope of this chapter's discussion.
The Compilable interface dictates that the compile method must return an instance of CompiledScript. The class CompiledScript is the construct JSR 223 defines to model the result of a compilation. No matter how we compile our script code, after all is said and done, we need to package the compilation result as an instance of CompiledScript. In the example code, I defined a class CompiledBoolScript and derived it from CompiledScript to store the compiled BoolScript code. Listing 12-14 shows the code of the CompiledBoolScript class. Because the purpose of CompiledBoolScript is to store the result of compiling BoolScript code, I defined a member variable in CompiledBoolScript called generatedClass. The member variable generatedClass references the in-memory Java class generated from compiling the input BoolScript code. Besides keeping a reference to the generated in-memory Java class, I also defined the varList member variable to keep track of the variable expressions in the input BoolScript code. This way, the eval method of the CompiledBoolScript class can retrieve the variables' values from the script context when it is invoked to execute the compiled BoolScript code. The eval method of the CompiledBoolScript class uses Java reflection to call the eval method of the generated Java class.
Listing 12-14. CompiledBoolScript Class
public class CompiledBoolScript extends CompiledScript {
private BoolScriptEngine engine;
private Class generatedClass;
private List<VarExpression> varList;
public CompiledBoolScript(BoolScriptEngine engine,
Class generatedClass, List<VarExpression> varList) {
this.engine = engine;
this.generatedClass = generatedClass;
this.varList = varList;
}
@Override
public List<Boolean> eval(ScriptContext context) throws ScriptException {
Class[] parameterTypes = new Class[varList.size()];
Object[] parameters = new Object[varList.size()];
for (int i = 0; i < parameterTypes.length; i++) {
parameterTypes[i] = boolean.class;
String varName = varList.get(i).getName();
parameters[i] = context.getAttribute(varName);
}
Method evalMethod = getMethod(parameterTypes);
Object result = invokeMethod(evalMethod, parameters);
return (List<Boolean>) result;
}
private Object invokeMethod(Method evalMethod, Object[] parameters)
throws ScriptException {
try {
return evalMethod.invoke(null, parameters);
} catch (...) {
//exception handling code omitted.
}
}
private Method getMethod(Class[] parameterTypes) throws ScriptException {
try {
Method evalMethod = generatedClass.getMethod("eval", parameterTypes);
evalMethod.setAccessible(true);
return evalMethod;
} catch (...) {
//exception handling code omitted.
}
}
//other method omitted.
}
Once the script code is compiled, the client Java program can repeatedly execute the compiled code by calling the eval method on the CompiledBoolScript instance that represents the compilation result of the source BoolScript code. When we call the eval method on the CompiledBoolScript instance that represents the compiled result of the BoolScript code x & y;, we need to pass in a script context that contains the values for variables x and y.
Run BoolScript Code as Invocable Function
The eval method of CompiledScript is not the only way to execute compiled script code. If the script engine implements the Invocable interface, we can call the invoke method of the Invocable interface to execute compiled script code too, much like we used Invocable to invoke Python and Ruby functions earlier in this chapter. In our simple example, there might not seem to be any difference between using CompiledScript and using Invocable for script execution. However, practically, users of a script engine will use CompiledScript to execute a whole script file, and they'll use Invocable to execute individual functions (methods, in Java terms) in a script. And if we look at Invocable's invoke method, distinguishing between CompiledScript and Invocable is not difficult. Unlike CompiledScript's eval method, which takes an optional script context as a parameter, Invocable's invoke method takes as a parameter the name of the particular function you'd like to invoke in the compiled script.
Listing 12-15 shows BoolScriptEngine's very simple implementation of the Invocable interface. The code simply uses the member variable compiledScript to keep a reference to the CompiledBoolScript instance that represents the result of compiling the source BoolScript code. Then in the invokeFunction method, the code creates a script context from the input args parameter and calls the eval method on the compiledScript member variable with the script context. The implementation of the Invocable interface in Listing 12-15 is very simple because all it does is store the result of compilation and use that result when asked to invoke a function. Practically, we should store not only the result of the BoolScriptEngine's compile method but also the result of its eval method. This way, if a host Java program calls the eval method, the evaluated BoolScript code can be executed again by calling the invokeFunction method of BoolScriptEngine.
Listing 12-15. BoolScriptEngine's Implementation of the Invocable Interface
public class BoolScriptEngine
extends AbstractScriptEngine
implements Compilable, Invocable {
private CompiledBoolScript compiledScript = null;
@Override
public CompiledScript compile(String script) throws ScriptException {
BoolScriptCompiler compiler = new BoolScriptCompiler(this);
compiledScript = compiler.compileSource(script);
return compiledScript;
}
@Override
public Object invokeFunction(String name, Object... args)
throws ScriptException, NoSuchMethodException {
List<VarExpression> vars = compiledScript.getVarList();
ScriptContext context = new SimpleScriptContext();
for (int i = 0; i < args.length; i++)
context.setAttribute(vars.get(i).getName(), args[i], ScriptContext.ENGINE_SCOPE);
return compiledScript.eval(context);
}
//other methods omitted.
}
Once we extend the BoolScriptEngine class to implement the Invocable interface, we can use it to execute BoolScript code as if the BoolScript code were an invocable function. Listing 12-16 shows an example of such usage of the BoolScriptEngine class.
In Listing 12-16, the variable engine is an instance of ScriptEngine that we know also implements the Compilable and Invocable interfaces. We cast it to be an instance of Compilable and call the compile method to compile the BoolScript code “x & y;”. After the compilation, we cast engine to be an instance of Invocable and call its invokeFunction method. Invoking a compiled script function is much like invoking a Java method using Java reflection. You must tell invokeFunction the name of the function you want to invoke, and supply it with the parameters required by the function. We know that in our generated Java code, the method name is hard-coded to be eval. So we pass the string “eval” as the first parameter to invokeFunction. We also know that generated Java eval method takes two Boolean values as its input parameters. So we pass two Boolean values to invokeFunction as well.
Listing 12-16. Using BoolScriptEngine to Execute BoolScript Code as if It Were an Invocable Function
private static void invokeCompiledBoolScriptExample() throws ScriptException,
NoSuchMethodException {
ScriptEngineManager scriptManager = new ScriptEngineManager();
ScriptEngine engine = scriptManager.getEngineByExtension("bool");
CompiledScript compiledScript = ((Compilable) engine).compile("x & y;");
List<Boolean> result = (List<Boolean>) ((Invocable) engine).invokeFunction(
"eval", true, false);
for (Boolean boolValue : result)
System.out.println("answer of boolean expression is: " + boolValue);
}
Summary
This chapter covered several major areas of JSR 223, such as the script engine discovery mechanism, variable bindings, and the Compilable and Invocable interfaces. One part of JSR 223 not mentioned in this article is Web scripting. If we implemented Web scripting in the BoolScript engine, clients of our script engine would be able to use it to generate Web contents in a servlet container.
We discussed and compared in a general way the dynamic language support provided by .NET and JVM, then we focused on a more in-depth discussion and comparison between JSR 223 and the DLR Hosting API. We saw that both JSR 223 and the DLR Hosting API have a mechanism for discovering script engines in a deployment environment. Both also define an API contract for interpreting or compiling dynamic language code. Furthermore, they both have a way to pass objects between a host program and the hosted dynamic language code.
Developing a non-trivial language compiler or interpreter is a huge undertaking, let alone integrating it with Java or .NET. Depending on the complexity of the language you want to design, developing a compiler or interpreter can remain a daunting task. However, thanks to JSR 223 and the DLR Hosting API, the integration between your language and Java or .NET has never been easier.