The algorithms presented in this chapter are applied to time series with a single variable of type Double. Therefore we need a mechanism to convert implicitly a given type T to a Double. Scala provides developers with such design: context bounds [3:1]:

  trait ToDouble[T] { def apply(t: T): Double }
  implicit val str2Double = new ToDouble[String] {
     def apply(s: String): Double = s.toDouble
  }

The Defining primitives types section under Source code in Chapter 1, Getting Started introduced the types for time series of single variable, Vector[T], and multiple variables, Vector[Array[T]].

A time series of observations is a vector (type Vector) of observation elements:

A time series of labels or expected values is a single-variable vector for which elements may have a primitive type of Int for classification and Double for regression.

A time series of labeled observations is a pair of a vector of observations and a vector of labels:

Visualization of the single feature and multi features observations

The two generic types for time series, Vector[T] and Vector[Array[T]], will be used as the two primary classes for input data, from now on.

The class Stats introduced in the Profiling data section of Chapter 2, Data Pipelines, implements some basic statistics and normalization for single variable observations.

Let us create a singleton, TSeries, to compute statistics and normalize multi-dimensional observations:

type DblVec = Vector[Double]

Object TSeries { 
  def zipWithShift[T](v: Vector[T], n: Int): Vector[(T, T)] = 
     v.drop(n).zip(v.view.dropRight(n))  //1

  def statistics[T: ToDouble](vs: Vector[Array[T]]): 
      Vector[Stats[T]] = vs.transpose.map( Stats[T]( _ ))  //2
  
  def normalize[T: ToDouble](  //3
      vs: Vector[Array[T]], low: Double, high: Double
  )(implicit ordering: Ordering[T]): Try[DblVec] = 
    Try (Stats[T](vt).normalize(low, high) )
   ...
}

The first method of the singleton, TSeries, generates a vector of a pair of elements by zipping the last size –n elements of a time series with its first size- n elements (line 1). The methods statistics (line 2) and normalize (line 3) operate on both single- and multi-variable observations. These three methods are a subset of the functionality implemented in TSeries.

Here is a partial list of other commonly used operators:

sealed trait Transpose {
  type Result   //4
  def apply(): Result  //5
}

implicit def vSeries2Matrix[T: ClassTag](from: Vector[Array[T]]) = 
  new Transpose { type Result = Array[Array[T]]  //6
    def apply(): Result = from.toArray.transpose
}

def transpose(tpose: Transpose): tpose.Result = tpose()

sealed trait Difference[T] {
  type Result
  def apply(f: (Double, Double) => T): Result
}

implicit def vector2Double[T](x: DblVec) = new Difference[T] {
  type Result = Vector[T]
  def apply(f: (Double, Double) => T): Result =  //7
    zipWithShift(x, 1).collect{case(next,prev) =>f(prev,next)}
}

def difference[T](
   diff: Difference[T], 
   f: (Double, Double) => T): diff.Result = diff(f)

val diffDouble = (x: Double,y: Double) => y –x //8
val diffInt = (x: Double,y: Double) => if(y > x) 1 else 0 //9
val diffBoolean = (x: Double,y: Double) => (y > x) //10

A view in Scala is a proxy collection that represents a collection, but implements data transformation or higher-order method lazily. The elements of a view are defined as lazy values, which are instantiated on demand.

One important advantage of views over a strict (or fully allocated) collection is the reduced memory consumption.

Let's look at the data transformation, aggregator, introduced in the Instantiating the workflow section under Workflow computational model in Chapter 2, Data Pipelines. There is no need to allocate the entire set of x.size of elements: the higher-order method, find may exit after only a few elements have been read (line 11):

val aggregator = new ETransform[Int, Int](ConfigInt(splits)) { 
  override def |> : PartialFunction[Int, Try[Int]] = { 
    case x: U if(!x.isEmpty) => 
      Try( Range(0, x.size).view.find(x(_) == 1.0).get) //11
   }
}