The implementation consists of wrapping (scope) the two functions into traits and defining these functions as abstract values.

The mathematical formalism is as follows:

The Scala implementation is represented here:

type V = Vector[Double]
trait F{ val f: V => V}
trait G{ val g: V => Double }

The model is defined as the composition of the two functions. The stack of traits G, F describes the type of compatible functions that can be composed using the self-referenced constraint self: G with F:

Formalism   h = f o g

The implementation is as follows:

class H {self: G with F =>def apply(v:V): Double =g(f(v))}

The model is executed once the variable f and g are instantiated.

The formalism is as follows:

The implementation is as follows:

val h = new H with G with F {
val f: V=>V = (v: V) =>v.map(exp(_))
val g: V => Double = (v: V) =>v.sum

Clearly, Scala preserves the formalism of mathematical models, making it easier for scientists and developers to migrate their existing projects written in scientific oriented languages such as R.

Let's extend the concept preservation of mathematical formalism to the dynamic creation of workflows using traits. The design pattern described in the next section is sometimes referred to as the cake pattern.

Dependency injection is a fancy name for a reverse look-up and binding to dependencies. Let's consider the simple application that requires data preprocessing, classification and validation.

A simple implementation using traits looks like this:

val app = new Classification with Validation with PreProcessing { 
  val filter = ???
}

If, at a later stage, you need to use an unsupervised clustering algorithm instead of a classifier, then the application has to be re-wired:

val app = new Clustering with Validation with PreProcessing { 
  val filter = ???
}

This approach results in code duplication and lack of flexibility. Moreover, the class member, filter, needs to be redefined for each new class in the composition of the application. The problem arises when there is a dependency between traits used in the composition. Let's consider the case for which filter depends on the validation methodology.

Although you can define filter as an abstract value, it still has to be redefined each time a new validation type is introduced. The solution is to use self-type in the definition of the new composed trait PreProcessingWithValidation:

trait PreProcessiongWithValidation extends PreProcessing {
   self: Validation => val filter = ???
}

The application is built by stacking the PreProcessingWithValidation mixin against the class Classification:

val app = new Classification with PreProcessingWithValidation {
  val validation: Validation
}

trait Validator {val g = (n: Int) => ??? }
trait MyValidator extends Validator {def g(n: Int) = }//WRONG

Let's adapt and generalize this pattern to construct a boilerplate template in order to create dynamic computational workflows.

The first step is to generate different modules to encapsulate different types of data transformation.

The following simple example is realistic enough to illustrate the different component of the cake pattern.

Let's define a sequence of three parameterized modules that each define a specific data transformation using the explicit configuration of type Etransform:

The modules contain a single abstract value. One characteristic of the cake pattern is to enforce strict modularity by initializing the abstract values with the type encapsulated in the module. One of the objectives in building the framework is allowing developers to create data transformation (inherited from ETransform) independently from any workflow.

The next step is to write the different modules into a workflow. This is achieved by using the self reference to the stack of the three traits defined in the previous paragraph.

Here's the code:

class Workflow[T,U,V,W] {
self: Sampling[T,U] 
    with Normalization[U,V] with Aggregation[V,W] =>

  def |> (t: T): Try[W] = for {
   u <- sampler |> t
    v <- normalizer |> u
    w <- aggregator |> v
  } yield w
}

A picture is worth a thousand words; the following UML class diagram illustrates the workflow factory (or cake) design pattern:

UML class diagram of the workflow factory

Finally, the workflow is instantiated by dynamically initializing the abstract values, sampler, normalizer, and aggregator of the transformation as long as the signature (input and output types) matches the parameterized types defined in each module (line 1):

Type DblF = Double => Double
type DblVec = Vector[Double]
val samples = 100; val normRatio = 10; val splits = 4

val workflow = new Workflow[DblF, DblVec, DblVec, Int] 
      with Sampling[DblF, DblVec] 
        with Normalization[DblVec, DblVec] 
          with Aggregation[DblVec, Int] {
  val sampler = ???  //1
  val normalizer = ???
  val aggregator = ???
}

Let's implement the data transformation function for each of the three modules/traits by assigning a transformation to the abstract values.

The first transformation, sampler, samples a function f with frequency 1/samples over the interval [0, 1]. The second transformation, normalizer, normalizes the data over the range [0, 1] using the Stats class introduced in the next chapter.

The last transformation, aggregator, extracts the index of the large sample (value 1.0):

val sampler = new ETransform[DblF,DblVec](ConfigInt(samples)) {//2
  override def |> : PartialFunction[DblF, Try[DblVec]] = { 
    case f: =>
      Try(Vector.tabulate(samples)(n =>f(1.0*n/samples))) //5
  }
}

The transformation sampler uses a single model or configuration parameter sample (line 2). The type DblF of input is defined as Double=> Double (line 3) and the type of output as a vector of floating point values, DblVec (line 4). In this particular case, the transformation consists of applying the input function f to a vector of increasing normalized values (line 5).

The normalizer and aggregator transforms follow the same design pattern as the sampler:

val normalizer = new ETransform[DblVec, DblVec](         
   ConfigDouble(normRatio)
) {
  override def |> : PartialFunction[DblVec,Try[DblVec]] = { 
    case x: DblVec if(x.size > 0) =>    
      Try((Stats[Double](x)).normalize)
  }
}

val aggregator = new ETransform[DblVec, Int](ConfigInt(splits)) {
   override def |> : PartialFunction[DblVec, Try[Int]] = {
     case x:DblVec if(x.size> 0) =>
       Try((0 until x.size).find(x(_)==1.0).getOrElse(-1))
   }
}

The instantiation of the transformation function follows the template described in the Monadic data transformation section of Chapter 1, Getting Started.

The workflow is now ready to process any function as input:

val g = (x: Double) =>Math.log(x+1.0) + nextDouble
Try (workflow |> g)  //6

The workflow is executed by providing the input function g to the first mixin, sampler (line 6).

Scala's strong type checking catches any inconsistent data types at compilation time. It reduces the development cycle because runtime errors are more difficult to track down.