Sentiment

Simply speaking, sentiment measures the quality of news. The basic sentiment looks at the polarity of the news: good, bad, or neutral. More advanced sentiment analysis can further express more sophisticated emotional details, such as “anger,” “surprise,” or “beyond expectation.”

Construction of news sentiment usually involves natural language processing and statistical/machine learning algorithms (for example, naïve Bayes and supported vector machine).

Sentiment is usually normalized into scores (for example, in range 0–100) to make them cross-sectional comparable. Higher score means news is good, lower score means news is bad, and a score near 50 means the news is neutral. A typical trading strategy could be following the sentiment directly:

• If (stock A sentiment > 70) long stock A;
• If (stock B sentiment < 30) short stock B;
• Use the no-news stocks for neutral.

Sentiment is also useful in risk management. For example, a portfolio manager may directly cut size on stock because of the unexpected news, or estimate the portfolio covariance matrix taking into account the news sentiment score or news frequency.

Individual news sentiment may have exposure to market aggregate sentiment, seasonality, and other timing factors (e.g. before or after the earning season). In relative value strategy, it is useful to compare the relative sentiment.

Novelty

Novelty measures if the news is a brand new story or a successive story of some old news. Vendors may split one long report into several parts. In some other cases, news may be revised several times beyond the initial report. Less novelty news usually has smaller impact on the market, since the information delivered in the previous news may already be reflected in the market. If we view news as events on a time series, novelty is usually inversely proportional to the time-distance between the events.

Relevance

News can have an impact on multiple stocks. Relevance measures the focus of news on specific stocks. Some news is company specific, such as earnings or corporate actions. Relevance of such news is usually high. Some other news about industry or macro-economics usually has lower relevance to individual stocks. A general news item about the banking industry may affect lots of banking stocks; on the other hand, a news story about Apple’s new products may affect Apple and its competitors (like Samsung), with higher relevance on Apple and lower relevance on Samsung. In other words, relevance maps the news to the stocks.

News Categories

Besides classification of news into “good” or “bad,” further classification into more detailed categories is also very important for analyzing and making use of news. A category can be as broad as “earnings,” which may include all earnings-related reports, like earnings announcements, earnings forecasts, earnings revisions, guidelines, earnings conference calls, and earnings calendars. It can also be more specific, like “corporate legal issues.” There are several important aspects of using news categories. First, different categories of news may have different response times on the market. Some categories have longer effects on company valuations, while other categories can cause short-term price fluctuations. Second, markets have different flavors of news at different times. A category rotation strategy can take advantage of such flavors of news styles. Lastly, categories make different types of news easier to use together with other information to create alphas. For example, use the earnings news together with analyst earnings revisions.

“Expected” and “Unexpected” News

A seemingly good piece of news, if the information is already expected by the market and thus reflected by the price, will not cause positive price movement. For example, a piece of news reads: earnings have large growth, 150% compared to 2013. Analyzing this piece of news usually gives positive sentiment. However, if the previous market consensus is that the company will grow 200%, the value in the news is actually below expectations and will cause the price to go down. Therefore, it can be useful to use news together with market consensus and market expectation. Textual analysis and calendar analysis can also be helpful to find out if the news is a “routine update” or “something different.” Surprises and unexpected news and events usually result in larger price movement.

Headlines and Full Text

Headlines usually contain the most important information and are well formatted. It is easier to parse and analyze. On the other hand, full text provides more detailed information, but it is harder to work with. One academic research report shows an interesting result: most information in a paragraph is included in the first sentence and last sentence of the paragraph. Similarly, we can give more focus on the first paragraph and last paragraph, first few words or last few words. The paragraph structure and sentence structure can be also interesting to look at.

No News is Good News?

There is an old saying “no news is good news.” To some extent, it is true; news means change, events, or something unusual happening. Also, news is usually associated with higher volatility, higher volume, analyst revisions, and more news following up. This brings potential risk to the company. Since more and more firms are using news in risk management, companies that have lots of news might also be cut in size, or even removed out of the investment universe by institutional investors, which can induce lower returns.

News Momentum

If the information in the news is not fully reflected by the market instantly, the stock price may have drift or momentum afterward. The effect is much stronger on smaller stocks, since they are less observed, and on unexpected news. For large stocks and expected news, price reversal after initial overshoot can be observed.

Academic Research

Since 2000, news on stock returns has become a popular topic. Popular research topics cover: aggregation and dispersion of sentiment; “beta” calculations using news; leading news stocks; weighting schemes in textual analysis; news confirmation by day return earnings announcements; the idea that no news is good news; the notion that stocks that are sensitive to news outperform; confirmation of news by trading volume; bias in the news coverage on stocks; momentum, overshoot, and reversal after news; and the relationship of news to analyst revisions.

Related papers can be found by searching “news” and “stock return” on the Social Science Research Network (SSRN).

What Social Media Platforms Matter?

The most popular choice is Twitter, because it can be easily mapped to stock (by checking @ticker). There is also research based on online forums, blogs from professional investors/traders, Facebook, and even Wikipedia.

Academic Research

The first research paper on this topic was “Twitter mood predicts the stock market.” The paper argues an accuracy of 87.6% in predicting the daily up and down changes in the closing values of the DJIA. Since then, research was conducted on: the prediction power of various social media; social media applied on individual stocks; the discussion of noise in social media; finding valuable tweets by observing “retweets” and tweets from celebrities; social media sentiment with long-term firm value. Most papers can be found on SSRN as well.

Challenges in Using Social Media in Predictions

Social media is a hot area in quant research. There are several challenges in applying sentiment analysis of social media contents. First, social media has a larger number of records, and updates quickly. Second, social media content is usually casual in format; a Twitter posting can contain a lot of abbreviations and poorly formatted words. This increases the difficulty in language processing. Third, how do we find original and important records? A lot of social media content is in response to some news; these have smaller leads compared to the more “original” social media contents. Hence, there are many fake signals in social media, which is why it is difficult to use social media for predictions.