13.3.1 Multivariate time series analysis

One of the most important areas of financial modelling is the modelling of multivariate time series analysis. We have several modelling choices:

• Multivariate distributions.
• Copulas: mainly for risk management and regulatory purposes.
• Factor models: widely used for prediction, interpretation, dimension reduction, estimation, risk and performance attribution.
• Multivariate time series models.

Vector autoregression (VAR) models are one of the most widely used family of multivariate time series statistical approaches. These models have been applied in a wide variety of applications, ranging from describing the behaviour of economic and financial time series to modelling dynamical systems and estimating brain function connectivity. VAR models show good performance in modelling financial data and detecting various types of anomalies, outperforming the existing state‐of‐the‐art approaches. The basic multivariate time series models based on linear autoregressive, moving average models are:

• Vector autoregression VAR(p)
  
• Vector moving average VMA(q)
  
• Vector autoregression moving average VARMA (p, q)
  
• Vector autoregression moving average with a linear time trend VARMAL(p, q)
  
• Vector autoregression moving average with exogenous inputs VARMAX(p, q)
  
• Structural vector autoregression moving average SVARMA(p, q)
  

The following variables appear in the equations:

• y_t is the vector of response time series variables at time t. y_t has n elements.
• c is a constant vector of offsets, with n elements.
• Φ_i are n‐by‐n matrices for each i, where Φ_i are autoregressive matrices. There are p autoregressive matrices and some can be entirely composed of zeros.
• ɛ_t is a vector of serially uncorrelated innovations, vectors of length n. The _t are multivariate normal random vectors with a covariance matrix Σ.
• Θ_j are n‐by‐n matrices for each j, where Θ_j are moving average matrices. There are q moving average matrices and some can be entirely composed of zeros.
• δ is a constant vector of linear time trend coefficients, with n elements.
• x_t is an r‐by‐1 vector representing exogenous terms at each time t and r is the number of exogenous series. Exogenous terms are data (or other unmodelled inputs) in addition to the response time series y_t. Each exogenous series appears in all response equations.
• β is an n‐by‐r constant matrix of regression coefficients of size r. So the product βx_t is a vector of size n.

LSTMs provide a very interesting non‐linear version of these standard models. This paper uses multivariate LSTMs with exogenous variables in a stock picking prediction context in experiment 2.

13.3.2 Machine learning models in finance

Machine learning models have gained momentum in finance applications over the past five years following the tremendous success in areas like image recognition and natural language processing. These models have proven to be very useful to model unstructured data. In addition to that, machine learning models are able to model flexibly non‐linearity in classification and regression problems and discover hidden structure in supervised learning. Combinations of weak learners like XGBoost and AdaBoost are especially popular. Unsupervised learning is a branch of machine learning used to draw inferences from datasets consisting of input data without labelled responses – principal component analysis is an example. The challenges of using machine learning in finance are important as, like other models, it needs to deal with estimation risk, potentially non‐stationarity, overfitting and in some cases interpretability issues.

13.4.1 Deep learning and time series

Time series models in finance need to deal with autocorrelation, volatility clustering, non‐Gaussianity, and possibly cycles and regimes. Deep learning and RNNs in particular can help model these stylized facts. On this account, an RNN is a more flexible model, since it encodes the temporal context in its feedback connections, which are capable of capturing the time varying dynamics of the underlying system (Bianchi et al. n.d.; Schäfer and Zimmermann 2007). RNNs are a special class of neural networks characterized by internal self‐connections which in principle can approximate or model any nonlinear dynamical system, up to a given degree of accuracy.

13.5.1 Introduction

RNNs are suitable for processing sequences, where the input might be a sequence (x₁,x₂,…,x_T) with each datapoint x_t being a real valued vector or a scalar value. The target signal (y₁,y₂,…,y_T) can also be a sequence or a scalar value. The RNN architecture is different from that of a classical neural network. It has a recurrent connection or feedback with a time delay. The recurrent connections represent the internal states that encode the history of the sequence processed by the RNN (Yu and Deng 2015). The feedback can be implemented in many ways. Some common examples of the architecture of RNNs are taking the output from the hidden layer and feeding it back to the hidden layer together with new arriving input. Another form of recurrent feedback is taking the output signal from the network at time step t − 1 and feeding it as new input together with new input at time step t (Goodfellow et al. 2016).

An RNN has a deep architecture when unfolding it in time (Figure 13.1), where its depth is as long as the temporal input to the network. This type of depth is different from a regular deep neural network, where depth is achieved by stacking layers of hidden units on top of each other. In a sense, RNNs can be considered to have depth in both time and feature space, where depth in feature space is achieved by stacking layers of hidden units on top of each other. There even exist multidimensional RNNs suitable for video processing, medical imaging and other multidimensional sequential data (Graves 2012). RNNs have been successful for modelling variable length sequences, such as language modelling (Graves 2012; Sutskever 2013), learning word embeddings (Mokolov et al. 2013) and speech recognition (Graves 2012).

13.5.2 Elman recurrent neural network

The Elman recurrent neural network (ERNN), also known as simple RNN or vanilla RNN, is considered to be the most basic version of RNN. Most of the more complex RNN architectures, such as LSTM and gated recurrent units (GRUs), can be interpreted as a variation or as an extension of ERNNs. ERNNs have been applied in many different contexts. In natural language processing applications, ERNNs demonstrated to be capable of learning grammar using a training set of unannotated sentences to predict successive words in the sentence (Elman 1995; Ogata et al. 2007). Mori and Ogasawara (1993) studied ERNN performance in short‐term load forecasting and proposed a learning method, called ‘diffusion learning’ (a sort of momentum‐based gradient descent), to avoid local minima during the optimization procedure. Cai et al. (2007) trained an ERNN with a hybrid algorithm that combines particle swarm optimization and evolutionary computation to overcome the local minima issues of gradient‐based methods.

The layers in an RNN can be divided in an input layer, one or more hidden layers and an output layer. While input and output layers are characterized by feed‐forward connections, the hidden layers contain recurrent ones. At each time step t, the input layer process the component of a serial input x. The time series x has length T and it can contain real values, discrete values, one‐hot vectors and so on. In the input layer, each component x[t] is summed with a bias vector , where N_h is the number of nodes in the hidden layer, and then is multiplied with the input weight matrix .

The internal state of the network from the previous time interval is first summed with a bias vector and then multiplied by the weight matrix of the recurrent connections. The transformed current input and past network state are then combined and processed by the neurons in the hidden layers, which apply a non‐linear transformation. The difference equations for the update of the internal state and the output of the network at a time step t are:

(13.1)

(13.2)

where f(·) is the activation function of the neurons, usually implemented by a sigmoid or by a hyperbolic tangent. The hidden state h[t], which conveys the content of the memory of the network at time step t, is typically initialized with a vector of zeros and it depends on past network states and inputs. The output is computed through a transformation g(·), usually linear in a regression setting or non‐linear for classification problems using the matrix of the output weights applied to the current state h[t] and the bias vector . All the weight matrices and biases can be trained through gradient descent, with the back‐propagation through time (BPPT) procedure. Unless differently specified, in the following to compact the notation we omit the bias terms by assuming x = [x;1], h = [h;1], y = [y;1] and by augmenting , W_h^o with an additional column.

13.5.3 Activation function

Activation functions are used in neural networks to transform the input of a neural network, expressed as a linear combination of weights and bias, to an output in feature space

where T denotes the transpose of the weight matrix. In the forward pass these transformations are propagated forward and eventually reach the last layer of the network and become the output of the whole network. This transformation is what makes neural networks learn nonlinear functions. The rectified linear unit (ReLu) (Figure 13.2) is the most common type of hidden unit used in modern neural networks (Goodfellow et al. 2016). It is defined as g(z) = max{0,z}.

Another activation function is the logistic sigmoid, σ(x) = (1 + exp.(−x))⁻¹, which is a differentiable squashing function (see Figure 13.2). One of its drawbacks is that learning becomes slow due to saturations when its argument either becomes too negative or when it becomes too big. Nowadays its use is discouraged, especially in feed‐forward neural networks (Goodfellow et al. 2016). In RNNs the logistic sigmoid can be used as hidden as well as output units. The tanh(x) is very similar to the sigmoid but with a range in the interval (−1,1). It is employed as an activation function in all types of neural networks: FNN, RNN, LSTM.

13.5.4 Training recurrent neural networks

We start this section with a short introduction to the training procedure of an RNN. In order to train an RNN, we need to compute the cost function in the forward pass. Then we back‐propagate the errors committed by the network and use those errors to optimize the parameters of the model, with gradient descent.

The algorithm used to compute the gradients, in the context of RNNs, is called Backpropagation Through Time (BPTT). We introduce the loss and cost functions, then we show how the parameters of the model are updated and finally we present the BPTT algorithm.

13.5.5 Loss function

The loss function measures the discrepancy between the predictions made by the neural network and the target signal in the training data. To assess whether our model is learning we compute the cost (see below) during training and test its generalization power on a test set not seen by the model during training. Depending on the prediction task on which we want to apply our RNN, there are several loss functions to choose from. In what follows we make use of the following definitions. Let y be the target signal and f(x,θ) the output from the network. Then for a binary classification task the target belongs to the set y = {0,1}. In this case the loss function is given by (Bishop 2006):

Its derivation is as follows. An outcome from a binary classification problem is described by a Bernoulli distribution p(y ∣ x, θ) = f(x, θ)^y(1 − f(x, θ))^{1 − y}. Taking the natural logarithm on the Bernoulli distribution gives a likelihood function, which in this case is equal to the cost function, giving the stated result. In this case the output is given by f = σ(a) = (1 + exp(−a))⁻¹ satisfying 0 ≤ f(x,θ) ≤ 1. For multi‐class classification we often use the loss function

where the output of our model is given by the softmax function

subject to the conditions 0 ≤ f_k ≤ 1 and P_k f_k = 1. In regression estimation tasks we make use of the mean‐squared error as the loss function (Bishop 2006)

where f_n is the output of the network, x_n are input vectors, with n = 1,…,N, and y_n are the corresponding target vectors. In unsupervised learning, one of the main tasks is that of finding the density p(x,θ) from a set of densities such that we minimize the loss function (Vapnik 2000)

13.5.6 Cost function

Learning is achieved by minimizing the empirical risk. The principle of risk minimization can be defined as follows (Vapnik 2000). Define the loss function L(y,(x,θ)) as the discrepancy between the learning machine's output and the target signal y. The risk functional or cost function is then defined as

where F(x,y) is a joint probability distribution. The machine learning algorithm then has to find the best function f that minimizes the cost function. In practice, we have to rely on minimizing the empirical risk due to the fact that the joint distribution is unknown for the data generating process (Goodfellow et al. 2016). The empirical cost function is given by

where the expectation, E, is taken over the empirical data distribution ^∧p(x,y).

13.5.7 Gradient descent

To train RNNs we make use of gradient descent, an algorithm for finding the optimal point of the cost function or objective function, as it is also called. The objective function is a measure of how well the model compares to the real target. For computing gradient descent we need to compute the derivatives of the cost function with respect to the parameters. This can be achieved, for training RNNs, by employing BPTT, shown later in this section. As stated before, we are going to ignore the derivations for the bias terms. Similar identities can be obtained from the ones for the weights.

The name gradient descent refers to the fact that the updating rule for the weights chooses to take its next step in the direction of steepest gradient in weight space. To understand what this means, let us think of the loss function J(w) as a surface spanned by the weights w. When we take a small step w + δw away from w, the loss function changes as δJ 'δw^T ΔJ(w). Then the vector ΔJ(w) points in the direction of greatest rate of change (Bishop 2006). Optimization amounts to finding the optimal point where the following condition holds:

If at iteration n we don't find the optimal point, we can continue downhill the surface spanned by w in the direction −ΔJ(w), reducing the loss function until we eventually find the optimal point. In the context of deep learning, it is very difficult to find a unique global optimal point. The reason is that the deep neural networks used in deep learning are compositions of functions of the input, the weights and the biases. This function composition spanning many layers of hidden units makes the cost function to be a nonlinear function of the weights and biases, thus leaving us with a non‐convex optimization problem. Gradient descent is given by

where η is the learning rate. In this form, gradient descent processes all the data at once to do an update of the weights. This form of learning is called batch learning and refers to the fact that the whole training set is used at each iteration for updating the parameters (Bishop 2006; Haykin 2009). Batch learning is discouraged (Bishop 2006, p. 240) for gradient descent as there are better batch optimization methods, such as conjugate gradients or quasi‐Newton methods (Bishop 2006). It is more appropriate to use gradient descent in its online learning version (Bishop 2006; Haykin 2009). This means simply that the parameters are updated using some portion of the data or a single point at a time after each iteration. The cost function takes the form

where the sum runs over each data point. This leads to the online or stochastic gradient descent algorithm

Its name, stochastic gradient descent, derives from the fact that the update of parameters happens either one training example at a time or by choosing points at random with replacement (Bishop 2006).

13.5.7.1 Back‐Propagation Through Time The algorithm to compute the gradients used in gradient descent, in the case of the RNN, is called BPTT. It is similar to the regular back‐propagation algorithm used to train regular FNNs. By unfolding the RNN in time we can compute the gradients by propagating the errors backward in time. Let us define the cost function as the sum of squared errors (Yu and Deng 2015):

where τ_t represents the target signal and y_t the output from the RNN. The sum over the t variable runs over time steps t = 1,t,…,T and the sum over j runs over the j units. To further simplify notation let us redefine the equations of the RNN as:

(13.3)

(13.4)

Employing the local potentials or activation potentials u_t = W ^T h_t−1 + U^T x_t and v_t = V ^T h_t, by way of Eqs. (13.3) and (13.4) and using θ = {W,U,V}, we can define the errors (Yu and Deng 2015)

(13.5)

(13.6)

as the gradient of the cost function with respect to the units' potentials. The BPTT proceeds iteratively to compute the gradients over the time steps, t = T down to t = 1. For the final time step we compute

for the set of units j = 1,2,…,L. This error term can be expressed in vector notation as follows:

where is the element‐wise Hadamard product between matrices. For the hidden layers we have:

for j = 1,2,…,N. This expression can also be written in vector form as

Iterating for all other time steps t = T − 1,T − 2,…,1 we can summarize the error for the output as:

for all units j = 1,2,…,L. Similarly, for the hidden units we can summarize the result as

where is propagated from the output layer at time t and is propagated back from the hidden layer at time step t + 1.

13.5.7.2 Regularization Theory Regularization theory for ill‐posed problems tries to address the question of whether we can prevent our machine learning model from overfitting and therefore plays a big role in deep learning.

In the early 1900s it was discovered that solutions to linear operator equations of the form (Vapnik 2000)

for a linear operator A and a set of functions f ∈ Γ, in an arbitrary function space Γ were ill‐posed. That the above equation is ill‐posed means that a small deviation like changing F by F_δ satisfying kF − F_δk < δ for δ arbitrary small leads kf_δ − fk to become large. In the expression for the functional R(f) = kAf − F_δk², if the functions f_δ minimize the functional R(f), there is no guarantee that we find a good approximation to the right solution even if δ → 0.

Many problems in real life are ill‐posed, e.g. when trying to reverse the cause‐effect relations, a good example being to find unknown causes from known consequences. This problem is ill‐posed even though it is a one‐to‐one mapping (Vapnik 2000). Another example is that of estimating the density function from data (Vapnik 2000). In the 1960s it was recognized that if one instead minimizes the regularized functional (Vapnik 2000)

where Ω(f) is a functional and γ(δ) is a constant, then we obtain a sequence of solutions that converges to the correct solution as δ → 0. In deep learning, the first term in the functional, R^*(f), will be replaced by the cost function, whereas the regularization term depends on the set of parameters θ to be optimized. For L₁ regularization γ(δ)Ω(f) = λwhereas for L₂ regularization, this term equals (see Bishop 2006; Friedman et al. n.d.).

13.5.7.3 Dropout Dropout is a type of regularization which prevents neural networks from overfitting (Srivastava et al. 2014). This type of regularization is also inexpensive (Goodfellow et al. 2016), especially when it comes to training neural networks. During training, dropout samples from an exponentially number of different thinned networks (Srivastava et al. 2014). At test time the model is an approximation of the average of all the predictions of all thinned networks only that it is a much smaller model with less weights than the networks used during training. If we have a network with L hidden layers, then l ∈ {1,…,L} is the index of each layer. If z^(l) is the input to layer l, then y^(l) is the output from that layer, with y⁽⁰⁾ = x denoting the input to the network. Let W^(l) denote the weights and b^(l) denote the bias, then the network equations are given by:

(13.7)

(13.8)

where f is an activation function. Dropout is then a factor that randomly gets rid of some of the outputs from each layer by doing the following operation (Srivastava et al. 2014):

where * denotes element‐wise multiplication.

13.7.1 Return series construction

The returns are computed as:

where P_tⁱ is the price at time t for stock or commodity i and R_tⁱ₊₁ is its return at time t + 1. Our deep learning model then tries to learn a function G_θ(·) for predicting the return at time t + 1 for a parameter set θ:

where k is the number of time steps backward in time for the historical returns. We used a rolling window of 30 daily returns to predict the return for day 31 on a rolling basis. This process generated sequences of 30 consecutive one‐day returns , where t ≥ 30 for all of stocks i.

13.7.2 Evaluation of the model

One solution to machine learning‐driven investing with the use of deep learning would be to build a classifier with class 0 denoting negative returns and class 1 denoting positive returns. However, in our experiments we observed that solving the problem with regression gave better results than using pure classification. When learning our models, we used the mean squared error (MSE) loss as objective function. First, after the models were trained and validated with a validation set, we made predictions on an independent test set, or as we call it here, live dataset. These predictions were tested with respect to the target series of our stock returns (see experiments 1 and 2) on the independent set, with a measure for correctness called the hit ratio. In line with Lee and Yoo (2017), we chose the HR as a measure of how correct the model predicts the outcome of the stock returns compared to the real outcome. The HR is defined as:

where N is the total number of trading days and U_t is defined as:

where R_t in the realized stock returns and is the predicted stock returns at trading day t. Thus, HR is the rate of correct predictions measured against the real target series. By using the HR as a measure of discrepancy we could conclude that the predictions either moved in the same direction as the live target returns or moved in the opposite direction. If HR equals one, it indicates perfect correlation and a value of zero indicates that the prediction and the real series moved in opposite directions. A value of HR > 0.50 indicates that the model is right more than 50% of the time while a value of HR ≤ 0.50 indicates that the model guesses the outcome.

For the computations we used Python, as well as Keras and PyTorch deep learning libraries. Keras and PyTorch use tensors with strong GPU acceleration. The GPU computations were performed both on an NVIDIA GeForce GTX 1080 Ti and an NVIDIA GeForce GTX 1070 GDDR5 card on two separate machines.

13.7.3 Data and results

We conducted two types of experiments. The first was intended to demonstrate the predictive power of the LSTM using one stock at a time as input up to time t and as target the same stock's returns at time t + 1. From now on we refer to it as experiment 1. The second experiment was intended to predict the returns for all stocks simultaneously. This means that all of our 50 stock returns up to time t were fed as input to an LSTM which was trained to predict the 50 stock returns at time t + 1. Additionally, to the 50 stocks we fed to the model the returns from crude oil futures, silver and gold returns. We refer to this as experiment 2. All the stocks used in this chapter are from the S&P 500, while the commodity prices were from data provider Quandl (Quandl n.d.).

13.7.3.2 Experiment 2

In this experiment we used the returns of all the 50 stocks. Additionally, we used oil, gold and S&P 500 return series of the 30 previous days as input. The output from our model is the prediction of the 50 stock returns. To test the robustness of the LSTM we also performed experiments against a baseline model, which in this case consisted of an SVM (Friedman et al. n.d.), suited for regression. To test the profitability of the LSTM, we ran experiments on a smaller portfolio consisting of 40 of our initial stocks (see Table 13.4) and the return series from the S&P 500, oil and gold. This part of the experiment is intended to show that the LSTM is consistent in its predictions independent of the time periods. Especially we were interested to see whether the model was robust to the subprime financial crisis. These results are shown in Table 13.5.

Because this section consists of many experiments, we present the experiment performed on the 50 stocks plus commodities in the main experiment subsection. The baseline experiment is presented in the subsection above and the last experiment for time periods in the ‘Results in different market regimes’ subsection below.

13.7.3.2.1 Main Experiment All the features in this experiment were scaled with the min‐max formula:

where , and a, b is the range (a,b) of the features. It is common to set a = 0 and b = 1. The training data consisted of 560 days from the period 2014‐05‐13 to 2016‐08‐01. We used a validation set consisting of 83 days for choosing the meta parameters, from the period 2016‐08‐02 to 2016‐11‐25 and a test set of 83 days from the period 2016‐11‐28 to 2017‐03‐28. Finally, we used a live dataset of 111 days for the period 2017‐03‐29 to 2017‐09‐05.

We used an LSTM with one hidden layer LSTM and 50 hidden units. As activation function we used the ReLu; we also used a dropout rate of 0.01 and a batch size of 32. The model was trained for 400 epochs. The parameters of the Adam optimizer were a learning rate of 0.001, β₁ = 0.9, β₂ = 0.999,  ɛ = 10⁻⁹ and the decay parameter was set to 0.0 As loss function we used the MSE.

To assess the quality of our model and to try to determine whether it has value for investment purposes, we looked at the ‘live dataset’, which is the most recent dataset. This dataset is not used during training and can be considered to be an independent dataset. We computed the HR on the predictions made with the live data to assess how often our model was right compared to the true target returns. The hit ratio gives us information if the predictions of the model move in the same direction as the true returns. To evaluate the profitability of the model we built daily updated portfolios using the predictions from the model and computed their average daily return. A typical scenario would be that we get predictions from our LSTM model just before market opening for all 50 daily stock returns. According to the direction predicted, positive or negative, we open a long position in stock i if R_tⁱ > 0. If R_tⁱ < 0 we can either decide to open a short position (in that case we call it a long‐short portfolio) for stock i or do nothing and if we own the stocks we can choose to keep them (we call it a long portfolio). At market closing we close all positions. Thus, the daily returns of the portfolio on day t for a long‐short portfolio is .

Regarding the absolute value of the weights for the portfolio we tried two kinds of similar, equally‐weighted portfolios. Portfolio 1: at the beginning we allocate the same proportion of capital to invest in each stock, then the returns on each stock are independently compounded, thus the portfolio return on a period is the average of the returns on each stock on the period. Portfolio 2: the portfolio is rebalanced each day, i.e. each day we allocate the same proportion of capital to invest in each stock, thus the portfolio daily return is the average of the daily returns on each stock. Each of the portfolios has a long and a long‐short version. We are aware that not optimizing the weights might result in very conservative return profiles in our strategy. The results from experiment 2 are presented in Table 13.2.

Stock	HR	Avg Ret %(L)	Avg Ret %(L/S)
Portfolio 1	0.63	0.18	0.27
Portfolio 2	0.63	0.18	0.27
AAPL	0.63	0.24	0.32
MSFT	0.71	0.29	0.45
FB	0.71	0.31	0.42
AMZN	0.69	0.27	0.41
JNJ	0.65	0.12	0.19
BRK/B	0.70	0.19	0.31
JPM	0.62	0.22	0.38
XOM	0.70	0.11	0.25
GOOGL	0.72	0.31	0.50
GOOG	0.75	0.32	0.52
BAC	0.70	0.30	0.55
PG	0.60	0.60	0.90
T	0.61	0.80	0.22
WFC	0.67	0.16	0.38
GE	0.64	0.50	0.24
CVX	0.71	0.18	0.31
PFE	0.66	0.70	0.14
VZ	0.50	0.10	0.10
CMCSA	0.63	0.23	0.36
UNH	0.59	0.20	0.22
V	0.65	0.23	0.31
C	0.69	0.28	0.39
PM	0.64	0.17	0.28
HD	0.61	0.10	0.17
KO	0.61	0.80	0.90
MRK	0.61	0.90	0.16
PEP	0.60	0.80	0.11
INTC	0.63	0.16	0.31
CSCO	0.68	0.14	0.31
ORCL	0.52	0.80	0.40
DWDP	0.60	0.21	0.35
DIS	0.59	0	0.90
BA	0.57	0.23	0.16
AMGN	0.65	0.21	0.36
MCD	0.58	0.16	0.12
MA	0.66	0.25	0.34
IBM	0.54	−0.40	0.60
MO	0.59	0.10	0.13
MMM	0.63	0.16	0.24
ABBV	0.61	0.18	0.22
WMT	0.50	0.14	0.16
MDT	0.59	0.90	0.17
GILD	0.50	0.16	0.11
CELG	0.64	0.28	0.45
HON	0.66	0.15	0.20
NVDA	0.68	0.54	0.68
AVGO	0.65	0.37	0.59
BMY	0.57	0.13	0.19
PCLN	0.61	0.14	0.24
ABT	0.63	0.21	0.28

HR, average daily returns for long portfolio (L) and long‐short portfolio (L/S) in percent. The results are computed for the independent live dataset.

13.7.3.2.2 Baseline Experiments This experiment was designed first to compare the LSTM to a baseline, in this case an SVM, and second to test the generalization power between the models with respect to the look‐back period used as input to the LSTM and the SVM. We noticed that using a longer historic return series as input, the LSTM remains robust and we don't see overfitting, as is the case for the SVM. The SVM had very good performance on the training set but performed worse on the validation and test sets. Both the LSTM and the SVM were tested with a rolling window of days in the set: {1,2,5,10}.

The results of the baseline experiment are shown in Table 13.3. We can see from the results that the LSTM improves its performance in the HR and the average daily returns in both the long and long‐short portfolios. For the SVM, the opposite is true, i.e. the SVM is comparable to the LSTM only when taking into account the most recent history. The more historic data we use, the more the SVM deteriorates in all measures, HR, and average daily returns for the long and long‐short portfolios. This is an indication that the SVM overfits to the training data the longer backward in time our look‐back window goes, whereas the LSTM remains robust.

Model	HR	Avg Ret %(L)	Avg Ret %(L/S)
LSTM (1)	0.59	0.14	0.21
LSTM (2)	0.61	0.17	0.26
LSTM (5)	0.62	0.17	0.26
LSTM (10)	0.62	0.17	0.26
SVM (1)	0.59	0.14	0.21
SVM (2)	0.58	0.13	0.18
SVM (5)	0.57	0.12	0.16
SVM (10)	0.55	0.11	0.14

The table shows the HR and the daily average returns for each model; all computations are performed on the out‐of‐sample live dataset. The number in parentheses in the model name indicates the look‐back length of the return series, i.e. trading days.

13.7.3.2.3 Results in Different Market Regimes To validate our results for this experiment, we performed another experiment on portfolio 1. This time, instead of using all 50 stocks as input and output for the model, we picked 40 stocks. As before, we added the return series for the S&P 500, oil and gold in this portfolio. The data was divided as training set 66%, validation (1) 11%, validation (2) 11%, live dataset 11%. The stocks used for this portfolio are presented in Table 13.4 and the results are shown in Table 13.5. Notice that the performance of the portfolio (Sharpe ratio) reaches a peak in the pre‐financial crisis era (2005–2008) just to decline during the crisis but still with a performance quite high. These experiments are performed with no transaction costs and we still assume that we can buy and sell without any market frictions, which in reality might not be possible during a financial crisis.

AAPL	MSFT US	AMZN US Equity	JNJ US
BRK/B	JPM	XOM	BAC
PG	T	WFC	GE
CVX	PFE	VZ	CMCSA
UNH	C	HD	KO
MRK	PEP	INTC	CSCO
ORCL	DWDP	DIS	BA
AMGN	MCD	IBM	MO
MMM	WMT	MDT	GILD
CELG	HON	BMY	ABT

The 40 stocks used for the second part of experiment 2.

Training period	HR %	Avg. Ret % (L)	Avg. Ret % (L/S)	Sharpe ratio (L)	Sharpe ratio (L/S)
2000–2003	49.7	−0.05	−0.12	−0.84	−1.60
2001–2004	48.1	0.05	−0.02	2.06	−0.73
2002–2005	52.5	0.11	0.10	6.05	3.21
2003–2006	55.9	0.10	0.16	5.01	5.85
2004–2007	54.0	0.14	0.12	9.07	5.11
2005–2008	61.7	0.26	0.45	7.00	9.14
2006–2009	59.7	0.44	1.06	3.10	6.22
2007–2010	53.8	0.12	0.12	5.25	2.70
2008–2011	56.5	0.20	0.26	6.12	6.81
2009–2012	62.8	0.40	0.68	6.31	9.18
2010–2013	55.4	0.09	0.14	3.57	3.73
2011–2014	58.1	0.16	0.21	5.59	6.22
2012–2015	56.0	0.15	0.21	5.61	5.84

This table shows HR, average daily return for a long (L) portfolio, average daily return for a long‐short (L/S) and their respective Sharpe ratios (L) and (L/S). The results are computed for the independent live dataset. Each three‐year period is divided into 66% training, 11% validation (1), 11% validation (2) and 11% live set.

The LSTM network was trained for periods of three years and the test on live data was performed on data following the training and validation period. What this experiment intends to show is that the LSTM network can help us pick portfolios with very high Sharpe ratio independent of the time period chosen in the backtest. This means that the good performance of the LSTM is not merely a stroke of luck in the good times that stock markets are experiencing these times.