We introduced Trident in Chapter 5, Exploring High-level Abstraction in Storm with Trident, of this book. Trident-ML (GitHub repository: https://github.com/pmerienne/trident-ml) is an online machine-learning library written over Trident that can be used to implement machine-learning algorithms in Storm applications.
It supports the following algorithms out of the box:
- Linear classification
- Linear regression
- K-means clustering
- Feature normalization
- Text feature extraction
- Stream statistics (count, mean, variance, and standard deviation)
If the algorithm you are looking for is not implemented in Trident-ML, you can easily implement it. Trident-ML also comes with a very useful pretrained Twitter sentiment analyzer.
In Trident-ML, various parameters associated with the learned model is stored in a TridentState object. As more training data comes in, these model parameters can be updated. This TridentState object is then used in a DRPC server to retrieve the model parameters to compute or predict new features of the incoming data and enrich the stream to process further.
The following diagram illustrates a typical Trident-ML application:
The Trident-ML application
Next, we will look into the use case that we will be developing for in this chapter.