As we saw in the last sections, a DSL can automatically refer to elements in other files thanks to global scoping. In particular, Xtext also takes imported namespaces into consideration; if we import smalljava.lang.*, then we can refer to, for example, Object directly, without its fully qualified name. The scope provider delegates this mechanism to the class ImportedNamespaceAwareLocalScopeProvider.

At this point, in order to use library classes like Object, we have to explicitly import smalljava.lang. In Java, you do not need to import java.lang, since that is implicitly imported in all Java programs. It would be nice to implement this implicit import mechanism also in SmallJava for the package smalljava.lang. All we need to do is to provide a custom implementation of ImportedNamespaceAwareLocalScopeProvider and redefine the method getImplicitImports (the technical details should be straightforward):

class SmallJavaImportedNamespaceAwareLocalScopeProvider
          extends ImportedNamespaceAwareLocalScopeProvider {
  override getImplicitImports(boolean ignoreCase) {
    newArrayList(new ImportNormalizer(
      QualifiedName.create("smalljava", "lang"),
      true, // wildcard
      ignoreCase
    ))
  }
}

In the previous code, the important part is the creation of ImportNormalizer, which takes a qualified name and interprets it as an imported namespace, with a wildcard when the second argument is true. This way, it is as if all SmallJava programs contained an import of smalljava.lang.*.

Now, we need to bind this implementation in the runtime module SmallJavaRuntimeModule. This is slightly different from other customizations in the Guice module we have seen so far; in fact, we need to bind the delegate field in the scope provider:

  override void configureIScopeProviderDelegate(Binder binder) {
    binder.bind(org.eclipse.xtext.scoping.IScopeProvider)
    .annotatedWith(
      com.google.inject.name.Names
      .named(AbstractDeclarativeScopeProvider.NAMED_DELEGATE))
     .to(SmallJavaImportedNamespaceAwareLocalScopeProvider);
  }

With this modification, we can simply remove the import statement import smalljava.lang.* and still be able to refer to the classes of the SmallJava library.

Being able to load the SmallJava library outside Eclipse is important both for testing and for implementing a standalone command-line compiler for the DSL.

As we saw in the previous sections, when we write unit tests with several dependent input programs, we need to load all the resources corresponding to input programs into the same resource set. Thus, we must load our library into the resource set as well to make the library available when running outside Eclipse.

We write a reusable class SmallJavaLib, which deals with all the aspects concerning the SmallJava library. We start with a method that loads the library in the passed the resource set:

class SmallJavaLib {

  public val static MAIN_LIB = "smalljava/lang/mainlib.smalljava"
  def loadLib(ResourceSet resourceSet) {
    val url = getClass().getClassLoader().getResource(MAIN_LIB)
    val stream = url.openStream
    val resource = resourceSet.createResource(URI.createFileURI(url.path))
    resource.load(stream, resourceSet.getLoadOptions())
  }
}

The important thing here is that we get the contents of mainlib.smalljava using the class loader. We use getResource, which returns an URL for the requested file and URL.openStream, which returns InputStream to read the contents of the requested file. The class loader will automatically search for the given file using the classpath. This will work both for JUnit tests and even when the program is bundled in a JAR as for the case of the standalone compiler. Then, we create an EMF resource and load it using the contents of the library file.

We are now able to write a test to verify that implicit imports work correctly. We use the version of the method parse that also takes the resource set as an argument, and we use the same resource set both for loading an input program and for loading the library using the method SmallJavaLib.loadLib shown previously:

@RunWith(XtextRunner)
@InjectWith(SmallJavaInjectorProvider)
class SmallJavaLibTest {
  @Inject extension ParseHelper<SJProgram>
  @Inject extension ValidationTestHelper
  @Inject extension SmallJavaLib
  @Inject Provider<ResourceSet> rsp
  @Test def void testImplicitImports() {
  '''
  class C extends Object {
    String s;
    Integer i;
    Object m(Object o) { return null; }
  }
  '''.loadLibAndParse.assertNoErrors
  }
  def private loadLibAndParse(CharSequence p) {
    val resourceSet = rsp.get
    loadLib(resourceSet)
    p.parse(resourceSet)
  }...

We also use SmallJavaLib to implement a standalone command-line compiler (see section Standalone command-line compiler of Chapter 5, Code Generation).

In the MWE2 workflow, we enable the following fragment:

generator = {
   generateXtendMain = true
}

We modify the generated Main Xtend class in order to load the library, load all the passed input files, and then validate all the resources in the resource set and run the generator for each resource. Remember that we must validate the resources only after all the resources have been loaded, otherwise the cross-reference resolution will fail. Here, we show the modified lines in the generated Main Xtend class:

class Main {
  
def static main(String[] args) {
    val injector =
      new SmallJavaStandaloneSetupGenerated().
        createInjectorAndDoEMFRegistration
    val main = injector.getInstance(Main)
    main.runGenerator(args)
  }


  ...
  @Inject SmallJavaLib smallJavaLib
  ...

  def protected runGenerator(String[] strings) {
    val set = resourceSetProvider.get
    // Configure the generator
    fileAccess.outputPath = 'src-gen/'
    val context = new GeneratorContext => [
        cancelIndicator = CancelIndicator.NullImpl
    ]
    // load the library
    smallJavaLib.loadLib(set)
    // load the input files
    strings.forEach[s | set.getResource(URI.createFileURI(s), true)]
    // validate the resources
    var ok = true
    for (resource : set.resources) {
      println("Compiling " + resource.URI + "...")
      val issues = validator.
        validate(resource, CheckMode.ALL,
          CancelIndicator.NullImpl)
      if (!issues.isEmpty()) {
        for (issue : issues) {
          System.err.println(issue)
        }
        ok = false
      } else {
         generator.generate(resource, fileAccess, context)
      }
    }
    if (ok)
      System.out.println('Programs well-typed.')
  }
}

After we finished adapting the type system to using the library, we can follow the same procedure illustrated in Chapter 5, Code Generation, to export a runnable JAR file together with all its dependencies. The file mainlib.smalljava will be bundled in the JAR, and the class loader will be able to load it.

Now that we have a library, we must update the type system and scope provider implementations in order to use the classes of the library. We declare public constants in SmallJavaLib for the fully qualified names of the classes declared in our library:

class SmallJavaLib {
  ...
  public val static LIB_PACKAGE = "smalljava.lang"
  public val static LIB_OBJECT = LIB_PACKAGE+".Object"
  public val static LIB_STRING = LIB_PACKAGE+".String"
  public val static LIB_INTEGER = LIB_PACKAGE+".Integer"
  public val static LIB_BOOLEAN = LIB_PACKAGE+".Boolean"

We use these constants to modify SmallJavaTypeConformance to define special cases as follows:

class SmallJavaTypeConformance {
  @Inject extension IQualifiedNameProvider
  def isConformant(SJClass c1, SJClass c2) {
    c1 === NULL_TYPE || // null can be assigned to everything
    c1 === c2 ||
    c2.fullyQualifiedName.toString == SmallJavaLib.LIB_OBJECT ||
    conformToLibraryTypes(c1, c2) ||
    c1.isSubclassOf(c2)
  }

  def conformToLibraryTypes(SJClass c1, SJClass c2) {
    (c1.conformsToString && c2.conformsToString) ||
    (c1.conformsToInt && c2.conformsToInt) ||
    (c1.conformsToBoolean && c2.conformsToBoolean)
  }

  
def conformsToString(SJClass c) {
    c == STRING_TYPE ||
    c.fullyQualifiedName.toString == SmallJavaLib.LIB_STRING
  }... similar implementations for int and boolean

Recall that in the type computer we have constants for some types, for example, for string, integer and boolean constant expressions. We now have to match such constant types with the corresponding SmallJava classes of the library. The type of string constant expression is type conformant to the library class String. The cases for boolean and integer expressions are similar.

Each class is considered type conformant to the library class Object, as in Java, even if that class does not explicitly extend Object. If we introduced basic types directly in SmallJava, for example, int and boolean, we would still have to check conformance with the corresponding library classes, for example, Integer and Boolean.

The fact that every SmallJava class implicitly extends the library class Object must be reflected in the scope provider so that any class is able to access the methods implicitly inherited from Object. For example, the following SmallJava class should be well typed even if it does not explicitly extend Object, but the current implementation rejects it, since it cannot resolve the references to members of Object:

class C {
  Object m(Object o) {
    Object c = this.clone();
    return this.toString();
  }
}

To solve this problem, we first add a method in SmallJavaLib that loads the EMF model corresponding to the library Object class:

class SmallJavaLib {
  @Inject extension SmallJavaIndex
  ...

  def getSmallJavaObjectClass(EObject context) {
    val desc = context.getVisibleClassesDescriptions.findFirst[
        qualifiedName.toString == LIB_OBJECT]
    if (desc == null)
      
return null
    var o = desc.EObjectOrProxy
    if (o.eIsProxy)
      o = context.eResource.resourceSet.
        getEObject(desc.EObjectURI, true)
    o as SJClass
  }

In the preceding code, we get the object description of the library class Object using SmallJavaIndex.getVisibleClassesDescriptions. If the EObject corresponding to Object is still a proxy, we explicitly load the actual EObject from the resource set of the passed context using the URI of the object description. The preceding code assumes that the library classes are visible in the current projects. It also assumes that the EObject context is already loaded, otherwise the resolution of the proxy will fail.

Since the scope provider uses the class hierarchy computed by SmallJavaModelUtil, we just need to modify the method SmallJavaModelUtil.classHierarchy, we developed in Chapter 9, Type Checking, section Checking method overriding, so that it adds the library Object SmallJava class at the end of the hierarchy, if not already present:

class SmallJavaModelUtil {
  @Inject extension SmallJavaLib
  ...
  def classHierarchy(SJClass c) {
    val visited = newLinkedHashSet()

    var current = c.superclass
    while (current != null && !visited.contains(current)) {
      visited.add(current)
      current = current.superclass
    }

    // new part
    val object = c.getSmallJavaObjectClass
    if (object != null)
      visited.add(object)

    visited

  }
}

Now, the scope provider will automatically retrieves also the methods defined in the library class Object.

Note that it is important to be able to easily modify the structure of the library classes in the future. We did hardcode as public constants the fully qualified names of library classes in SmallJavaLib, but not the library classes' structure, so if in the future we want to modify the implementation of the library classes, we will not have to modify the type system neither the scope provider.

The other interesting feature is that one could easily provide a different implementation of the library as long as the main library class names are kept. The current SmallJava implementation will seamlessly be able to use the new library without any change. This is another advantage of keeping the DSL and the library implementations separate.