Classifying multidimensional time series sensor data is inherently more complex than classifying traditional nominal data, as we saw in the previous chapters. First, each observation is temporally connected to the previous and following observations, making it very difficult to apply a straightforward classification of a single set of observations only. Second, the data obtained by sensors at different time points is stochastic, that is, unpredictable due to the influence of sensor noise, environmental disturbances, and many other factors. Moreover, an activity can consist of various sub-activities executed in a different manner and each person performs the activity a bit differently, which results in high intraclass differences. Finally, all these reasons make an activity recognition model imprecise, resulting in new data often being misclassified. One of the highly desirable properties of an activity recognition classifier is to ensure continuity and consistency in the recognized activity sequence.

To deal with these challenges, activity recognition is applied to a pipeline, as shown in the following diagram:

In the first step, we attenuate as much noise as we can, for example, by reducing the sensor sampling rate, removing outliers, applying high-or low-pass filters, and so on. In the next phase, we construct a feature vector. For instance, we convert sensor data from time domain to frequency domain by applying a discrete Fourier transform (DFT). DFT is a method that takes a list of samples as an input and returns a list of sinusoid coefficients ordered by their frequencies. They represent a combination of frequencies that are present in the original list of samples.

Next, based on the feature vector and set of training data, we can build an activity recognition model that assigns an atomic action to each observation. Therefore, for each new sensor reading, the model will output the most probable activity label. However, models make mistakes. Hence, the last phase smooths the transitions between activities by removing transitions that cannot occur in reality; for example, it is not physically feasible that the transition between the activities lying-standing-lying occur in less than half a second, hence such a transition between activities is smoothed as lying-lying-lying.

The activity recognition model is constructed with a supervised learning approach, which consists of training and classification steps. In the training step, a set of labeled data is provided to train the model. The second step is used to assign a label to the new unseen data by the trained model. The data in both phases must be preprocessed with the same set of tools, such as filtering and feature vector computation.

The post processing phase, that is, spurious activity removal, can also be a model itself and hence also requires a learning step. In this case, the preprocessing step also includes activity recognition, which makes such arrangement of classifiers into a meta-learning problem. To avoid overfitting, it is important that the dataset used for training the post processing phase is not the same as the one used for training the activity recognition model.