Supervised learning models are extracted from a training set. Transformations such as classification or regression use the implicit models to process input data, as illustrated in the following diagram:

Visualization of implicit models

trait ITransform[T, A] {  //1
self => 
   def |> : PartialFunction[T, Try[A]]   //2
   def map[B](f: A => B): ITransform[T, B] 
   def flatMap[B](f: A => ITransform[T, B]): ITransform[T, B] 
   def andThen[B](tr: ITransform[A, B]): ITransform[T, B] 
}

An implicit transformation has a type ITransform with two parameter types (line 1):

For instance, the moving average on a time series of a single variable is computed by a ITransform[Double, Double]. The input collection is the time series and the output is a smoothed time series.

The method |> declares the transformation that is defined by implementing the trait ITransform (line 2). Let's look at the monadic operators.

The map method applies a function to each element of the output of the transformation |>. It generates a new ITransform by overriding the |> method (line 3).

A new implementation of the data transformation |> returning an instance of PartialFunction[T, Try[B]](line 4) is created by overriding the methods isDefinedAt (line 5) and apply (line 6):

def map[B](f: A => B): ITransform[T,B] = new ITransform[T,B] {
   override def |> : PartialFunction[T, Try[B]] = //3
     new PartialFunction[T, Try[B]] {     //4
   override def isDefinedAt(t: T): Boolean =  //5
          self.|>.isDefinedAt(t)
   override def apply(t: T): Try[B] = self.|>(t).map(f) //6
}

The overridden methods for the instantiation of ITransform in flatMap follow the same design pattern as the map method. The argument f converts each output element into an implicit transformation of type ITransform[T, B](line 7), and outputs a new instance of ITransform after flattening (line 8).

As with the map method, it overrides the implementation of the data transformation |> returning a new partial function (line 9) after overriding the isDefinedAt and apply methods:

def flatMap[B](
  f: A => ITransform[T, B] //7
): ITransform[T, B] = new ITransform[T, B] { //8

  override def |> : PartialFunction[T, Try[B]] =
      new PartialFunction[T, Try[B]] { //9
  override def isDefinedAt(t: T): Boolean = 
      self.|>.isDefinedAt(t)
  override def apply(t: T): Try[B] = 
      self.|>(t).flatMap(f(_).|>(t)
}

The method andThen is not a proper element of a monad. Its meaning is similar to the Scala method Function1.andThen that chains a function with another one. It is indeed useful to create chains of implicit transformations. The method applies the transformation tr (line 10) to the output of this transformation. The output type of the first transformation is the input type of the next transformation, tr.

The implementation of the method andThen follows a pattern similar to the implementation of map and flatMap:

def andThen[B](
  tr: ITransform[A, B]  (line 10)
): ITransform[T, B] = new ITransform[T, B] {
   
  override def |> : PartialFunction[T, Try[B]] =
     new PartialFunction[T, Try[B]] {
  override def isDefinedAt(t: T): Boolean = 
     self.|>.isDefinedAt(t) &&
           tr.|>.isDefinedAt(self.|>(t).get)
  override def apply(t: T):Try[B] = tr.|>(self.|>(t).get)
}

The transformation on a dataset is performed using a model or configuration fully defined by the user, as illustrated in the following diagram:

Visualization of explicit models

The execution of a data transformation may depend on some context or external configuration. Such transformations are defined having the type ETransform parameterized by the type T of the elements of input collection and the type A of element of output collection (line 11). The context or configuration is defined by the trait config (line 12).

An explicit transformation is a transformation with the extra capability to use a set of external configuration parameters to generate an output. Therefore, ETransform inherits from ITransform (line 13):

abstract class ETransform[T,A]( //11
  config: Config//12
) extends ITransform[T,A] {  //13
self =>
  def map[B](f: A => B): ETransform[T,B] = 
    new ETransform[T,B](config) {
      override def |> : PartialFunction[T, Try[B]] = super.|>
     }

  def flatMap[B](f: A => ETransform[T,B]): ETransform[T,B] = 
    new ETransform[T, B](config) {
      override def |> : PartialFunction[T, Try[B]] = super.|>
    }

  def andThen[B](tr: ETransform[A,B]): ETransform[T,B] = 
    new ETransform[T, B](config) {
     override def|> :PartialFunction[T, Try[B]] = super.|>
   }
}

The client code is responsible for specifying the type and value of the configuration used by a given explicit transformation. Here are a few examples of configuration classes:

Trait Config
case class ConfigInt(iParam: Int) extends Config
case class ConfigDouble(fParam: Double) extends Config
case class ConfigArrayDouble(fParams: Array[Double) 
   extends Config

The supervised learning models described in future chapters, such as logistic regression, support vector machines, Naïve Bayes or multilayer perceptron are defined as implicit transformations and implement the ITransform trait. Filtering and data processing algorithms such as data extractor, moving averages, or Kalman filters inherit the ETransform abstract class.