9.2.1 Price data

As part of the study, we use daily close‐to‐close commodity futures returns provided by Stevens Analytics.⁶ In addition, to modelling the next‐day (logarithmic) return, we use RavenPack data specific to each commodity up until 15 minutes before the settlement price of the given commodity futures. For example, the settlement price for crude oil futures is computed between 2:28 pm ET and 2:30 pm ET on CME Globex,⁷ meaning that we use RavenPack data available in the 24 hours up to 2:15 pm ET as input to our models.

Given that we are dealing with a basket of commodity futures with wildly varying volatilities – both across the spectrum and across time – we seek to volatility‐adjust the returns by a lagged rolling standard deviation. We do this to avoid over‐emphasizing those commodity futures with the highest volatility for each basket when estimating the models.

We define the log‐return, standard deviation and volatility‐adjusted log‐return as follows:

(9.1)

(9.2)

(9.3)

where n = 1, …, N represents the four commodity futures and t = 1, …, T is the time index identifying the day in which the price or return was observed, which can be missing for commodity‐specific non‐trading workdays. The parameter m defines the length of the window over which the standard deviation is calculated, while target defines the target volatility. Throughout this study, we use 21 trading days to calculate the standard deviation (m = 21) with an annualized target volatility of 20% . We have not optimized the parameter m, but we find that it provides a good tradeoff between stability and variability.

9.3.1 Feature creation

All the models presented herein make use of the same input: as target variable the volatility‐scaled log‐returns and as features, a matrix of continuous variables. The features are designed to capture the impact of an event category taking place at time t for commodity n as well as its relevance and is constructed as follows:

(9.5)

(9.6)

where i = 1, …, I is the number of events for category j.

In other words, if we detect at least one news story for a given event category for date t and commodity n, the variable switches to from 0. This implies that news stories are weighted based on their ERS – thereby giving higher importance to news stories where the event is featured prominently, for example in the headline.

In order for a given event category to be included in the modelling of the commodities basket, we require at least 50 days with one or more events across the in‐sample dataset. This requirement is not optimized and is simply introduced to remove very infrequent event categories from consideration. Furthermore, we remove perfectly correlated independent variables as appropriate.

Finally, we perform feature selection based on the in‐sample data by requiring that there is an absolute correlation of at least 0.5% between each feature and our target variable. This results in a reduction of the number of features of between 37% and 45% depending on the out‐of‐sample year, meaning that we are left with 34–37 predictors. During preliminary research we found that imposing this restriction on the features improved speed and, importantly, in‐sample robustness of the five machine learning algorithms.⁸

9.4.1 Model portfolios

Table 9.3 shows a set of in‐sample performance statistics⁹ for our five ML models across the commodities basket – resulting in five different portfolios.¹⁰ Overall, we find positive performance, with some rather big discrepancies across the models. In particular, we notice that the linear model is outperformed across all metrics by all the non‐linear ones, with the only exception of random forest (RF). In particular, the gradient boosted trees (GBN) and the ANN are the best models in‐sample, showing similar IRs of 2.40 and 2.39 respectively.

Having shown the in‐sample performance, in Table 9.4 we present the out‐of‐sample results. We find the RF model to be the best performer as it delivers an IR of 0.85 on an annualized return of 13.1%. The second‐best model is the KNN, which yields an IR of 0.83 and the lowest volatility (14.1%), followed by the GBN, which provides an IR of 0.76 and 12.3% annualized returns. The linear model (ELNET) is the second to last model, suggesting that modelling non‐linear relationships between our explanatory variable and commodities returns does indeed provide an edge that allows superior performance to be obtained.

Moreover, it is noteworthy that had we chosen a model based on the in‐sample evidence, this would have resulted in suboptimal out‐of‐sample performance as the best two models in‐sample are respectively the third and the last one out‐of‐sample, an evidence that warns against the risk of model selection.

Considering that there is only a moderate positive correlation between the predicted returns for RF and most of the better‐performing models, such as KNN (0.55), we may be able to benefit from combining the predictions of the various models and in this way control for the risk associated with model selection, something that we will explore in Section 9.4.3.

9.4.2 Variable importance

Up until now we have answered only the question of whether our framework can produce alpha, not the question of which variables and in turn which event categories drive that alpha‐generating performance. We choose RF to answer this question as it (i) is the best‐performing model with an out‐of‐sample IR of 0.85, and (ii) provides an elegant way of computing and analyzing variable importance.

In order to detect the most important categories, we rely on a measure based on residual sum of squares, i.e. the total decrease in node impurities from splitting on the variable.

In Figure 9.1, we show the top‐ten categories. In order to obtain a measure of relative importance by year, we first compute the total decrease in node impurities by year. We then rescale the measure of importance by this amount, in order to provide a measure of the relative importance of each category in each out‐of‐sample year.

The analysis of the most important variables provides a sensible picture. In particular, we find inventories‐related categories among the most important ones, for instance inventories‐down‐commodity. Inventories‐related news seems to play a relevant role in driving prices, as three out of the top ten variables belong to this event type.

Moreover, we also find supply‐related news to have a relevant role on price dynamics, as news related to resource discoveries, pointing to more supply of a commodity in the future, or spills, pointing to a decrease in the supply, also prove to be among the top predictors for future prices.

9.4.3 Ensemble portfolio

In Section 9.4.1 we demonstrated that the energy basket provides overall positive returns – both in absolute terms and on a risk‐adjusted basis. However, with five models to choose from, the question becomes, which one to select? A valid approach is to each year select the model which performed best the previous year. However, we have shown that relying on the in‐sample evidence for model selection might result in suboptimal out‐of‐sample performance: the ANN model had an in‐sample IR comparable to the best model – had we chosen this model, this would have resulted in the worst out‐of‐sample performance. An alternative approach is to implement an ensemble (Breiman 1994; Mendes‐Moreira et al. 2012) strategy whereby we combine the predicted returns across all five models via equal weight – thereby taking an agnostic view on which model is best.¹¹

In Table 9.5 we repeat the performance statistics for our energy basket with an additional column added for the ensemble strategy.

By combining the five models, we generate an IR of 0.65 with an annualized return of 9.8%. Without any prior knowledge about which of the five models perform best, we are able to construct an ensemble which is competitive in terms of returns – both in absolute and risk‐adjusted terms – and which allows for considerable risk reduction associated with model selection, therefore reducing the risk of potential overfitting. This highlights why ensemble methods can be strong performers despite relying on a mixed basket of models; in this particular case, we achieve an IR of 0.65 despite giving 40% weight to models with relatively poor performance (ELNET and ANN).

Since 2015, the ensemble basket has yielded a total cumulative return of 29.3%. In comparison, the equivalent long‐only daily‐rebalanced benchmark portfolio has yielded total returns of −9.0% with an IR of −0.11.

Analyzing the time series of returns, we note the high correlation between our best models (RF and KNN) and the ensemble, underlining once again the competitiveness of the model‐agnostic ensemble approach. Meanwhile, the correlation between the ensemble and the long‐only basket is negative.¹²

In Figure 9.2 we have plotted the cumulative returns profiles for the ensemble strategy against a long‐only basket. Overall, the ensemble strategy has performed reasonably well out‐of‐sample, though it is clear that it was more performant in the first half of the period when energy commodities, in general, plummeted across the globe. This indicates that there may have been a regime shift since the middle of 2016 – and commodities have indeed been trading sideways without much volatility since then – which the ensemble model struggles to fully capture. In Section 9.4.5, we investigate this in more detail.

9.4.4 Ensemble portfolio – marginal contributions

Having determined that our ensemble approach offers a simple yet well‐performing method for combining our models, in this section we evaluate the performance contribution of each commodity to the overall portfolio. In Figure 9.3 we show the dependency of the portfolio on any single commodity, presenting the IR when systematically leaving out one commodity at a time to capture the marginal contribution of each commodity. The label on the x‐axis makes reference to the commodity which is dropped from the basket.

By systematically removing one asset at a time from the basket we achieve IRs of 0.16–0.69 and annualized returns of 2.9–10.9%. Remarkably, the removal of natural gas hurts the portfolio the most, with the IR declining to 0.16, this commodity being the one contributing the most towards the performance of the portfolio.

These results also suggest that there may be additional performance to be had by modelling each commodity individually, but such a step drastically reduces the number of non‐zero entries in the explanatory variables, resulting in fewer features and potentially more biased estimates as well. For that reason we have chosen to model all four commodities together, implying that we assume that a news category has the same impact across the basket.

9.4.5 Regime detection in the ensemble portfolio

Up until now, we have traded all signals generated by our models. While such an approach is attractive from a signal diversification point of view, it may still be interesting to evaluate whether our signals perform particularly well during certain time periods. For example, volatile market regimes may provide more trading opportunities through stronger signals.¹³ Conversely, quiet markets may allow fundamental news to have a more pronounced impact on day‐to‐day returns without them being clouded by noisy fluctuations. By trading across all regimes, as is the case with the ensemble approach in Section 9.4.3, we may fail to take advantage of any regime dependency. In other words, by restricting ourselves to a subset of trades, we may be able to improve the per‐trade return, while reducing overall trading and the cost associated with it.

We seek to determine whether there are certain regimes in which the portfolio performance is particularly strong. Concretely, we illustrate this approach by creating various portfolios which only trade the underlying commodities when certain requirements related to the realized volatility are met.

We implement this by testing a volatility filter based on (i) the 1‐day lagged 21‐day volatility being above its annual average, or (ii) the 1‐day lagged 10‐day volatility being above its 21‐day equivalent. In other words, is volatility high relative to the last 12 months or rising?¹⁴ When at least one of the conditions is fulfilled we are in a high volatility environment, otherwise we are in a low volatility environment.

In Table 9.6 we present results of conditioning the ensemble portfolio from Table 9.5 on the two volatility regimes (high‐vol/low‐vol). As can be observed, by conditioning on the volatility regime, we are able to find discrepancies in performance. In particular, we notice that over the in‐sample period, the high‐vol signal outperformed the low‐vol one. Although both regimes provide positive returns, this observation, within the training period, raises the question whether we can achieve better out‐of‐sample performance by avoiding low‐vol regimes when trading the signal.

Looking at the out‐of‐sample performance confirms this intuition, as we find that periods of high volatility yield considerably higher returns, both in absolute and in risk‐adjusted terms. In particular, the out‐of‐sample period yields an IR of 1.27 with annualized return of 21.3% for the high‐vol regime compared with an IR of −0.20 and annualized return of −3.0% for the low‐vol regime. The discrepancy is further supported by Figure 9.4, where we show that the high‐vol strategy consistently yields superior returns over the full‐sample period, and particularly so during the second half of the out‐of‐sample period, where the low‐vol strategy yields negative performance.

In Figure 9.5 we compare the profiles of the out‐of‐sample cumulative returns from the ensemble and high‐vol strategies. In particular, we show that the high‐vol strategy not only yields higher returns but also has the advantage of being more robust, experiencing a positive trend across most of the out‐of‐sample period. Specifically, the high‐vol strategy provides more consistent performance, avoiding most of the negative trend showed by the ensemble during 2017, mostly due to the low‐vol signals. Moreover, even though the high‐vol strategy is characterized by a lower number of trades compared with the ensemble (1929 vs 2740), this is more than compensated by the boost observed in per‐trade returns. By only trading the high‐vol signals, we are able to more than double the per‐trade return compared with the ensemble, from 3.38 to 8.82 basis points.

The ensemble method is a good starting point for developing a trading portfolio, but the results in Table 9.5 and Figures 9.4 and 9.5 underline that it fails to fully take into account potential regime shifts seen in the market. This is not surprising, since our ensemble model only includes event relevance‐scaled dummy variables based on whether a RavenPack event category was triggered or not – and only for one day. More elaborate models could include information such as lagged category triggers, news volume, sentiment or novelty filters, or even market data such as asset volatility and returns (Hafez and Lautizi 2017).

Moreover, looking at the dynamics of cumulative returns over the full sample, we notice that they seem to be affected by seasonality. Preliminary research based on splitting the signals into spring–summer vs autumn–winter¹⁵ showed that the latter produced considerably higher returns, confirming our intuition about seasonality. Such evidence is consistent with the hypothesis that our predictive models work better during spikes in the expected and actual commodity demand occurring in colder months. Trying to exploit seasonality to further improve performance is yet another interesting direction to investigate in future research.