Automated stock analysis has gotten a lot of press recently. High-frequency trading firms are a flashpoint. People either believe that they're great for the markets and increasing liquidity, or that they're precursors to the apocalypse. Smaller traders have also gotten into the mix in a slower fashion. Some sites, such as Quantopian (https://www.quantopian.com/) and AlgoTrader (http://www.algotrader.ch/) provide services that allow you to create models for automated trading. Many others allow you to use automated analysis to inform your trading decisions.
Whatever your view of this phenomena, it's an area with a lot of data begging to be analyzed. It's also a nice domain in which to experiment with some analysis and machine learning techniques.
For this chapter, we're going to look for relationships between news articles and stock prices in the future.
In the course of this chapter, we will cover the following topics:
- Learn about financial data analysis
- Set up our project and acquire our data
- Prepare the data
- Analyze the text
- Analyze the stock prices
- Learn patterns in both text and stock prices with neural networks
- Use this system to predict the future
- Talk about the limitations of these systems
Finance has always relied heavily on data. Earnings statements, forecasting, and portfolio management are just some of the areas that make use of data to quantify their decisions. Because of this, financial data analysis and its related field, financial engineering, are extremely broad fields that are difficult to summarize in a short amount of space.
However, lately, quantitative finance, high-frequency trading, and similar fields have gotten a lot of press and really come into their own. As I mentioned, some people hate them and the added volatility that the markets seem to have. Others maintain that they bring the necessary liquidity that helps the market function better.
All of these fields apply statistical or machine learning methods to financial data. Some of these techniques can be quite simple. Others are more sophisticated. Some of these analyses are used to inform a human analyst or manager to make better financial decisions. Others are used as inputs to automated algorithmic processes that operate with varying degrees of human oversight, but perhaps with little to no intervention.
For this chapter, we'll focus on adding information to the human analyst's repertoire. We'll develop a simple machine learning system to look at past, current, and future stock prices, alongside the text of news articles, in order to identify potentially interesting articles that may indicate future fluctuations in stock price. These articles, with the possible future price vector, could provide important information to an investor or analyst attempting to decide how to shuffle his/her money around. We'll talk more about the purpose and limitations of this system toward the end of the chapter.