Now, we will combine the prediction power for all of these algorithms to generate a "super-learner" with the help of a neural network, which takes each model's prediction as input and then, tries to come up with a better prediction, given the guesses of the individually trained models. At a high level, the architecture would look something like this:
We will explain further the intuition behind building a "super-learner" and the benefits of this approach, and teach you how to build your Spark streaming application, which will take in your text (that is, a movie review that you will write) and run it through the prediction engine of each of your models. Using these predictions as input into your neural network, we will yield a positive or negative sentiment using the combined power of the various algorithms.