Nadayara–Watson Estimator for Semiparametric Regression

Let y_i be the ith observation of a response variable and be the K‐dimensional ith vector containing the observed predictor variables (i = 1, …, N), where T denotes the transposed vector. The constant unit level is introduced to take into account the interception component.

The simplest parametric regression model is the general linear model:

(6.27)

where denotes conditional statistical expectation, and 〈⋅, ⋅〉 is the dot product. Model coefficients β¹, …, β^K, and interception β₀, are estimated by ordinary LS.

Nonparametric kernel regression (Ruppert et al., 2003) computes a local weighted average of the criterion variable given the values of the predictors; that is:

(6.28)

where m(⋅) is a nonparametric function. For instance, the NW estimator consists of a constant‐kernel approximation given by

(6.29)

with

(6.30)

and

(6.31)

Parameter σ is called the bandwidth, and it represents the neighborhood of influence for each observation. A smaller bandwidth will give a lower bias but increased variance in the estimator, whereas a larger bandwidth will produce higher bias (model mismatch) despite a reduced variance estimator.

The SR model can be very useful when mixed nature predictor variables are available (for instance, metric and dichotomic), because different components can be modeling the contribution of each subset of variables. Also, sometimes we have some a priori knowledge about the model that we can assume for a subset of variables, but the remaining subset has a completely unknown nature. Without loss of generality, let us assume observation vectors that are composed of D dichotomic variables and M metric variables, . Let us use a parametric, linear component for the dichotomic variables, and a nonparametric, nonlinear component for the metric variables. The corresponding SR model is

(6.32)

The estimation method is described in Heerde et al. (2001) and Lee and Nelder (1996), and it is briefly presented here. The model can be written down as

(6.33)

where e_i denotes the ith residual. By taking the conditional average with respect to the parametric variables, we obtain

(6.34)

Then, by subtracting Equations 6.33 and 6.34, we have

(6.35)

Therefore, a three‐step procedure can be stated. First, averages conditional to the nonparametric variables are estimated for the response variable and for the parametric variables, given by

(6.36)

(6.37)

Note that bandwidths σ₁ and σ₂ must be properly chosen for the statistical estimation. Second, the new response variable and the new predictor variables are used to find by solving

(6.38)

by means of ordinary LS. Note that the intercept term disappears at this step because of the mean subtraction, so that it remains inside the nonparametric component. Third, the nonparametric component is obtained by nonparametric regression on ỹ; that is:

(6.39)

where bandwidth σ₃ must be also properly fixed. The choice of three different bandwidths must be addressed, with two of them coming from a nonparametric estimation of the pdf and the third one coming from the nonparametric regression process. Several bandwidth selection techniques are available, among which are Lee’s rule of thumb, cross‐validation, or subjective approach (Heerde et al. 2001 ; Lee and Nelder 1996).

Some limitations of the procedure could be present according to this formulation:

• Automatic selection procedures can deteriorate when low‐sized data sets are analyzed.
• The denominators in Equations 6.36, 6.37, and 6.39 can become very small for a newly tested point that is far enough from the observations, and this produces an unbounded predicted output. Again, this situation can be more present when reduced data sets are under analysis. A solution for this drawback can be the introduction of a small threshold parameter in the exponential exponent, which is, in fact, a regularization procedure. This threshold should be also found as a free parameter.
• Finally, all the available observations are used for building the solution in Equation 6.39, independently of their adequacy or noise level. This solution will not be operational for studies with high number of observations.

These limitations can be alleviated by the SVM formulation for SR, which is presented next.

Semiparametric Regression Approach using the Support Vector Machine

An alternative formulation of SR can be stated by using the RSM framework. Let us assume that the model can be expressed with two additive contributions: one from the parametric and another from the nonparametric predictor variables. Let us assume also that there exists a possibly nonlinear transformation of the metric predictor variables into a higher (possibly infinite) dimensionality space, , where ℱ_m is known as the feature space for the metric model component. The main property of this nonlinear transformation is that a linear regression operator can be found in the feature space, given by vector . Let us finally assume that, for the dichotomic model component, there exists another transformation of the dichotomic vectors to a different feature space _d where another linear regression operator can be properly adjusted.

In these conditions, the joint regression model is given by

(6.40)

Here, the reference interception term b_r can be previously fixed, in order to make easy the comparison to a given level. For instance, b_r could be the average sales level constrained to nonpromotional periods, so that the resulting model will provide us with information about the predictors either increasing or decreasing that level.

The SVM‐SR algorithm is stated as the minimization of the ɛ‐Huber cost plus a regularization term given by the ℓ₂ norm of the weights in the feature spaces; that is:

(6.41)

constrained to

(6.42)

(6.43)

(6.44)

where ξ_i and (in the following, denoted jointly by ) are the slack variables used to account for the residuals in the model; I₁ is the set of samples for which , and I₂ is the set of samples for which . Following the usual SVM formulation methodology, Lagrangian functional ℒ can be written down, and by making zero its gradient with respect to the primal variables we obtain

(6.45)

(6.46)

(6.47)

with α_i and denoting the Lagrange multipliers that correspond to Equation 6.45 and Equation 6.46 respectively. Matrix notation is introduced as follows:

(6.48)

(6.49)

(6.50)

(6.51)

Finally, the dual problem can be stated (Rojo‐Álvarez et al., 2004) as the maximization of

(6.52)

constrained to Equation 6.47, with respect to dual variables . After this quadratic programming problem is solved, and according to Equations 7.29, 6.45, and 6.46, the final expression of the solution can be easily shown to be given by the following expression:

(6.53)

where . Note that the estimated response variable ŷ is calculated from a weighted expansion of dot products in the feature spaces. With this expression for the solution, the explicit calculation of w^m and v^d is not required.

Here, we are mainly concerned about two properties of Mercer kernels: (1) the sum of two Mercer kernels is a Mercer kernel; and (2) the product of a Mercer kernel times a positive constant is a Mercer kernel. These simple properties allows us to propose the use of a scaled linear kernel that generates the parametric component:

(6.54)

with , plus a nonlinear kernel that generates the nonparametric component:

(6.55)

Constant δ can be chosen for giving a balance between the parametric and nonparametric components. Thus, the final solution of SVM‐SR can be readily expressed as

(6.56)

Taking Equations 6.53 and 6.56 into account, coefficients η_i determine completely both the parametric and the nonparametric components.

Bootstrap Resampling for Model Diagnosis

One of the main limitations of current SR methods is the difficulty in establishing clear cut‐off tests for the nonparametric variables of the model, and much effort is being done in this framework (Ruppert et al., 2003). Also, this aspect has not yet been completely solved in the SVM literature, and systematic procedures for establishing feature selection, significance levels, and confidence intervals (CIs) for model diagnosis have been developed (Lal et al., 2004). An interesting approach to the model diagnosis and feature selection issues in SVM for SR can be given by BRTs, which were first proposed as possibly nonparametric procedures for estimating the pdf of an estimator from a limited, but informative enough, set of observations (Efron and Tibshirani, 1998). BRTs have been successfully used before for fixing the free parameters of SVM classifiers (Rojo‐Álvarez et al., 2002) and as a feature selection strategy using the robust SVM linear maximum margin classifier (Soguero‐Ruiz et al., 2014a,b). We propose here to extend their use to model diagnosis and free parameter selection for SVM problems with the SR algorithm.

For a given set of N observations v, the dependence between the predictor variables and the response variable can be fully described by using their joint distribution:

(6.57)

In order to obtain the SVM‐SR model, Equation 7.33 is maximized. This estimation process is denoted by operator s(⋅), and it depends on observations v, and on the free parameters of the model that have been fixed a priori. Those free parameters can be grouped in a vector θ, that consists of the ε‐Huber cost parameters and of the kernel‐related parameters; that is, for the Gaussian kernel, θ = {ε, C, γ, δ, σ}. The SVM‐SR Lagrange multipliers obtained by using the observations and a given a priori fixed θ are

(6.58)

The model performance can be measured with the empirical risk, which can be defined as a merit figure of the model that is evaluated at the observations used for building the model. It can be expressed as

(6.59)

where t(⋅) represents an operator that stands for the empirical risk calculation. Two usual merit figures for the model are the coefficient of determination R² and the RMSE:

(6.60)

(6.61)

where m_y and m_ŷ are the averages of the observations and of the model predicted response respectively. Note that ρ should be as small (close to zero) as possible, whereas R² should be as high (close to + 1) as possible. As merit figures are random variables that depend on the observations, they are more accurately described in terms of CIs, which can be denoted by

(6.62)

(6.63)

where and P_ρ(⋅) are the pdfs of each merit figure; q ∈ (0, 1) is the confidence level; and subscripts “l” and “u” are the lower and upper limits respectively of the CI.

Given that the SVM‐SR model does not rely on any a priori distribution of the data, it is not easy to know the functional form of the pdf of the merit figures. Moreover, the sample merit figures’ estimators can be optimistically biased, especially for some degenerate choices of the free parameters; for instance, when too much emphasis is put on the cost of the residuals or when a too small bandwidth is used. Therefore, the empirical risk criterion is not a good criterion for fixing θ. Cross‐validation can be useful in some cases, but it requires one to reduce the training set, which can lead to important information loss in the model when a low number of observations are available. A method for estimating the joint pdf of the observations, necessary for a statistical characterization of the SVM‐SR model, is given by a BRT, and it can compensate the optimistic bias in the merit figures’ estimators (Rojo‐Álvarez et al., 2002).

A bootstrap resample is a data subset that is drawn from the observation set according to their empirical distribution . Hence, the true pdf is approximated by the empirical pdf of the observations, and the bootstrap resample can be seen as a sampling with replacement process of the observed data; that is:

(6.64)

where superscript * represents, in general, any observation, functional, or estimator that arises from the bootstrap resampling process. Therefore, resample v^* contains elements of v appearing none, one, or several times. The resampling process is repeated for b = 1, …, B times.

A partition of v in terms of resample v^*(b) is given by

(6.65)

where v_in^*(b) is the subset of observations that are included in resample b, and v_out^*(b) is the subset of nonincluded observations. The SVM‐SR coefficients for resample (b) are obtained by

(6.66)

A bootstrap replication of an estimator is given by its calculation constrained to the observations included in the bootstrap resample. The bootstrap replication of the empirical risk estimator is

(6.67)

The scaled histogram obtained from B resamples is an approximation to the pfd of the empirical risk. However, further advantage can be obtained by calculating the bootstrap replication of the risk estimator on the nonincluded observations. By doing so, rather than estimating the empirical risk, we are in fact obtaining the replication of the actual risk; that is:

(6.68)

The bootstrap replication of the averaged actual risk can be obtained by just taking the average of for b = 1, …, B. Simple search strategies can be used for finding the free parameters that minimize the averaged actual risk (Rojo‐Álvarez et al., 2003 ; Soguero‐Ruiz et al., 2014b). Moreover, the replications of the pdf of the model merit figures can provide CIs for the performance, by fulfilling

(6.69)

(6.70)

A typical range for B in practical applications can be from 50 to 500 bootstrap resamples. Other useful statistical characterizations can be readily achieved, for instance, CIs for the response variable, CIs for the parametric coefficients, and significance levels for the inclusion of nonparametric variables.

Confidence Intervals for the Response Variable

Frequently, it is not enough to report the predicted response variable, but it is also convenient to characterize the uncertainty on this prediction. This can be done by reporting the CI for the average of each prediction, if no strong statistical dependence is present in the response, or by reporting confidence bands, if output samples are strongly dependent. For the first case, the CI for each average output level can be readily obtained by calculating the pdf of the replications for each response variable in Equation 6.56 given by model in Equation 6.66 as

(6.71)

where denotes the pdf of the ith observation of the response, i = 1, …, N. When statistical independence of the response variable can no longer be assumed, confidence bands should be used instead of CIs (Politis et al., 1992). Prediction intervals for the output level can also be calculated.

Inference estimators can be obtained for each of the kth parametric coefficients of Equation 6.56 by obtaining the bootstrap replications of the parametric coefficients in each model in Equation 6.66 as follows:

(6.72)

where denotes the bootstrap estimated pdf of the kth parametric coefficient, k = 1, …, K. For a cut‐off test, a CI overlapping zero level corresponds to a nonsignificant parametric variable, and it can be excluded from the model.

Significance Level for Nonparametric Features

It is not possible, in general, to obtain CIs for coefficients related to the nonparametric variables, as these variables remain in a nonlinear, difficult to observe, equation. However, the performance of the complete model (all the nonparametric variables included) can be statistically compared with the performance of a reduced model (only a subset of them included). This can be made by comparing the CI of the merit figure of the reduced model with the bias‐corrected merit figure of the complete model. For instance, let denote the bootstrap bias‐corrected correlation coefficient for the complete model, and let S² be the correlation coefficient for the reduced model, with CI estimated by fulfilling

(6.73)

A possible test is the following:

(6.74)

That is, the null hypothesis ₀ states that the reduced model is sufficient, whereas the alternative hypothesis ₁ indicates that there is an important loss of fitness when considering only the reduced model.

Model Development

Model building requires tuning different free parameters depending on the SVM formulation (σ_ker, C, ε) and filter parameters (μ, P). A nonexhaustive iterative search strategy (T iterations) was used, and values of T = 3 and M = 20 exhibited good performance in our simulations in terms of the averaged normalized MSE:

(6.80)

where the is the estimated variance of the data. Most of MATLAB source code for the experiments is available and linked in the book’s web page.

Nonlinear Feedback System Identification

We now consider the system previously described and illustrated in Figure. 6.2. This system was used to generate 10 000 input–output sample pairs (x_n, f(g(x_n))), that were split into a training set (50) and a test set (following 500 samples). The experiment was repeated 100 times with randomly selected starting points. Table 6.2 shows the average results for all composite kernels. The best results are obtained with the summation kernel, followed by the kernel trick.

The Mackey–Glass Time Series

Our next experiment deals with the standard Mackey–Glass time series prediction problem, which is generated by the delay differential equation dx/dt = − 0.1x_n + , with delays Δ = 17 and Δ = 30, thus yielding the time series MG17 and MG30 respectively. We considered 500 training samples and used the next 1000 for free parameter selection (validation set), following the same approach as Mukherjee et al. (1997). The results are shown in Table 6.2 for both time series, suggesting that a more complex model is necessary to obtain good results in the prediction of this time series, which exhibits more complex dynamics.

Electroencephalogram Prediction

This additional and real‐life experiment deals with the EEG signal prediction four samples ahead. This is a very challenging nonlinear problem with high levels of noise and uncertainty. We used file “SLP01A” from the MIT‐BIH Polysomnographic Database.¹ The file contains 10 000 samples; hence, we used 100 as training samples, the next 1000 samples for free parameter selection (validation set), and the remainder for testing.

Average test results are shown in Table 6.2, showing that the kernel trick combined with the summation or cross‐terms kernel performs best, and suggesting that the high complexity of the underlying signal model has been retrieved by the data model.

On Model Complexity and Nonlinear Time Scales

Attending to the numerical results (nMSE) in Table 6.2, one could identify EEG and NLSYS as high‐complexity problems, and MG17 and MG30 as moderate‐complexity problems. However, different kernel structures may accommodate the problem difficulty better than others. Certainly, complexity and versatility is an important aspect for time‐series analysis. In this sense, Figure 6.3 reports the results for the four nonlinear time‐series problems in terms of machine complexity (SVs (%)), needed tap delays P, memory requirements μ, and its attendant memory depth M = P/μ, which quantifies the past information retained and it has units of time samples (Principe et al., 1993).

The memory depth M serves to uncover the efficiency in modeling the (nonlinear) time scales. On the one hand, it is worth noting that in complex problems (NLSYS and EEG) the kernel trick (KT) yields slightly higher values of M at the expense of poor numerical results (see Table 6.2).

The code used for the last three examples (MG17, MG30 and EEG) is shown in Listing 6.2. First, we select the problem and the method, and then we load the data and generate the input–output data matrices. Finally, we build the kernel using the code previously described and train the regression algorithm.

Network System and Network Model

In the Spanish electrical grid, power flows continuously from the power generators to the consumption centers (Feijoo et al., 2010). To achieve reliability, one of the main issues of this electrical grid consists of designing the telecontrol service with two redundant but physically different paths. Figure 6.4 shows an example of path calculation in which a simple description of the link availability based on an exponential failure probability with the distance link has been used to determine a reliable double path from a given origin node to the destination node assigned by the telecontrol system using Bayesian networks. The reliability of the telecommunication system can be estimated from data obtained and depends critically on the accuracy of the link and node availability estimates. In what follows, a method based on composite kernel and multiresolution is introduced and studied.

Composite Kernels and Multiresolution

In this application example, we used SVR for grouping and dealing with subsets of features of a different nature. This is usually addressed by using a Gaussian kernel, and in that case a free parameter σ is used for nonlinearly mapping input vectors. Input feature vectors x_i can be redefined into L disjoint feature groups: , with cardinality D_l, such that . In that case, we can start from a multiple or composite (multiple kernels) regression model, with a vector accounting for each subset of input variables:

(6.81)

Following Equation 6.11, a solution can be obtained for a new input vector:

(6.82)

A scaled kernel for each term in the sum can be used (see Camps‐Valls et al. (2006c) and Soguero‐Ruiz et al. (2016b)), according to

(6.83)

with λ_l ≥ 0. Thus, the final solution of the SVM can be readily expressed as

(6.84)

where λ_l represents the contribution of the lth group of input variables to the model. We can get mutiresolution by adjusting the widths σ_l for each kernel, so that it can adapt to different sources of different nature. Note also that weight vectors in feature spaces w^l have no physical meaning, but instead they are a mathematical tool for giving support to the kernel trick.

The methodology is evaluated in both a simulated and a real network. In the simulated network, the statistical distribution for the link failure – the rate consisting of geographical effects and link length effects – is known and allows us to generate failures on the links. Instead of using the simulated network to have a realistic link failure model, its final goal is to benchmark the performance of link failure estimators based on SVM and NN methods in a known scenario. Furthermore, it can be useful to obtain conclusions about the selection of the free model parameters, which were addressed in the HRCN power system.

Surrogate Data Model

A surrogate model was built for the underlying law of failure rate according to the real links and nodes of the network; that is, using the coordinates. Three different effects were considered:

• Smooth spatial variation, giving higher link failure probability P₁ for northern than for southern links (the relative difference when comparing northern and southern links is five times larger than when comparing eastern and western links (see Figure 6.5, left), following a linear trend (see Listing 6.3), as follows:
  (6.85)
  where c₁ is an offset constant.
• Fast spatial variation, yielding higher failure probabilities in network regions with higher grid density. This variation was given by a smoothed version (Gaussian kernel filtering) of the bidimensional histogram of the number of links in a region (see Matlab code in Listing 6.4). We denoted it as P₂(lat, lon) (see Figure 6.5, middle).
• Link failure probability, related to the link length. The fiber length increases exponentially to unity, following
  (6.86)
  and it is depicted in Figure 6.5 (right) (see Listing 6.5).

The final probability of a link failure happening in a given link was given by an additive mixture of P₁ and P₂ spatial effects, and a multiplicative mixture of the spatial and the fiber length, as follows:

(6.87)

where c is a numerical normalization constant. We use the pdfs for defining the link events in terms of its geometric enter and its length (see their smooth variation in Figure 6.5) as follows:

(6.88)

Annual and Asymptotic Data Generation

We generated an annual event series given by a list of events that have been observed s[n] during the nth year, with N_y the number of annual events, given by s_n, with n = 1, …, N_y, where for each event s_n consists of a three‐tuple including the coordinates and length of each link. The annual event can be accumulated as follows:

(6.89)

allowing us to use a frequentist estimation of the event rates at each link:

(6.90)

and it can be trivially shown to converge asymptotically to the actual event rate of the kth link; that is, . These asymptotic event probabilities were used as a benchmark for comparison of the estimated event probability given by learning from algorithms.

Simulations with Surrogate Data

We used SVM and NN algorithms to estimate obtaining ; that is, the frequentist estimation of the events for each link up to year n, given the inputs variables latitude, longitude, and fiber length, for each link (lat_k, lon_k, L_k). Three different approaches were built using SVM algorithms with a Gaussian kernel in all of them: (1) SVM‐1 K uses just a single kernel for a vector containing the three input variables; (2) SVM‐2 K has two kernels, one for coordinates and one for length; and (3) SVM‐3 K using three kernels, one for each input variable. The results obtained were benchmarked with a generalized regression NN (GRNN) for function approximation (Demuth and Beale, 1993), which is also a nonlinear method and only requires a free parameter (Gaussian width) to be previously fixed.

In the SVM models, a cross‐validation strategy (50% for training and 50% for validation) was applied to tune the free parameters; namely, the ones from the cost function (C, γ, ɛ), the width (σ_i) for each kernel and the relevance parameter λ_i. For each model, and using given free parameters, the MAE was calculated. We want to tune the set of free parameters that minimize the MAE. To that end, we started with fixed initial values for C, ε, σ_i, λ_i (i = 1, 2, or 3), we obtained the variation of MAE_n with γ. We then fixed parameter γ to the value minimizing MAE[n], and we obtained the variation of MAE_n as a function of C (while keeping the rest of the parameters to their fixed values). We continued this process until we explored all parameters.

The purpose of this surrogate data model consists of evaluating the methodology in a theoretic way, and comparing the estimated and the asymptotic results:

(6.91)

where is the estimated output after tuning the free parameters of the SVM with the training and validation sets, and is the asymptotic corresponding value (n = 10 000 for Equation 6.90, as previously described).

A similar procedure was followed for the GRNN, in which only the width parameter had to be searched in the rank σ ∈ (10⁻², 10). The same considerations for the training and test set were followed. MAE_n, and MAE were also calculated for comparison purposes.

Results on Observed and Asymptotic Data

In this section we show results in terms of observed error and asymptotic error, which are given by MAE_n, and MAE respectively, when evaluating SVM‐1 K, SVM‐2 K, and SVM‐3 K, and an NN scheme using the GRNN. We used 10 independent realizations. Listing 6.6 shows how to obtain both the asymptotic model and when considering 2–20 years.

Figure 6.6 shows the box plots for all the methods in the cases of including and not including the null events in the learning procedure. Panels (a) and (c) show the MAE_n that should be observed with the events available up to year n, and it can be compared with the gold standard given by MAE in panels (b) and (d) in terms of the box plots. Box plots use a box and whisker plot for the merit figures from each number of available years n in the conventional way, and have lines at the lower quartile, median, and upper quartile values. The plus sign represents the outliers.

The observed error (i.e., the MAE_n in n years) showed a too‐optimistic bias with very few years of events available, in particular, up to 5 years and 10 years respectively when considering and when not considering the null events in the training procedure. It also can be concluded that the asymptotic error (i.e., the MAE) clearly stabilized after an initial period of 10 years when considering the null events and of 5 years when not considering them. Finally, and overall, including the links with no events in the training procedure yielded lower asymptotic error in this case, which can be seen for instance in Figure 6.6b and d, with lower asymptotic error (MAE) being obtained for 2–20 years for this approach. The synthetic model created for the surrogate data model is shown in Listing 6.7.

It can be concluded that there are smooth partial relationships among variables, such as spatial or link fiber dependence. The methodology allows the study of complex interaction in HRCNs, increasing the accuracy with the number of availability events. We analyzed the effect of including or excluding the links with unobserved events in the training set. Finally, we benchmarked that better performance is obtained with SVM models, especially with few years of available data.

A Case Study

We also analyzed the historical data consisting of the events and the failures in the optical links from a real HRCN during 2 years. We used the same three input variables (longitude, latitude, and fiber length) to characterize each link. This real HRCN had a low number of observed events, 45 and 50 for S₁ and S₂ respectively. The same SVM algorithms were tested by using one, two, and three Gaussian kernels. Free parameters were tuned by using a cross‐validation technique, by randomly splitting the available data into training and validation subsets. We used MAE₂ on the validation to assess the performance of the model. To give a statistical description of the accuracy, the estimations were repeated 200 times with different randomization.

Figure 6.7 shows the histograms of MAE₂, in logarithmic units, for the three SVM algorithms. It is important to emphasize the bimodality in the error histograms, showing that there are two cluster of training subsets: one provides suboptimal results, whereas the other yields good performance. The MAE₂ values for SVM‐1 K, SVM‐2 K, and SVM‐3 K algorithms were − 0.84, − 3.41, and − 3.42 respectively. The conclusion is that using two kernels provides a significant improvement, whereas the inclusion of a third kernel did not further enhance performance.

Effect of the Number of Input Features

One of the main limitations of SR is that, due to the nonparametric component of the model, its performance sensibly deteriorates with the number of input features (curse of dimensionality). For classification and regression problems, the SVM has been shown to be robust when working with high‐dimensional input spaces, in part due to the sparsity enforced on the solution. We present a simple simulation example that compares the behavior of SVM‐SR with the SR using the NW estimator (NW‐SR), in terms of the dimension of the input space.

The following model was used to generate the observations:

(6.92)

(6.93)

(6.94)

where σ_m and σ_d are the standard deviations of the and processes respectively; Q is an M × M constant matrix, whose elements are i.i.d. and drawn from a rectified Gaussian pdf, N(0, 1); β is a D × 1 vector, whose elements are i.i.d. and drawn from a uniform pdf

U(−1, 1); x^m is an M × 1 random vector, drawn from an N(0, 2); x^d is a D × 1 random vector, drawn from a Bernoulli pdf with 1‐probability of 0.5; and e_n is an N(0, 0.2) perturbation. Performance is evaluated for SVM‐SR and for NW‐SR, over a rank M ∈ (5, 40), and M = D. The training and the validation sets consist of 50 and 500 observations respectively. The experiment was repeated for 10 runs.

Figure 6.8 shows the runs‐averaged ρ for both methods. Whereas the performance of NW‐SR gets worse with the number of input variables, the error in SVM‐SR remains almost at the same level. For M > 30, performance is significantly different for both methods.

Deal Effect Curve Estimation in Marketing

As a real application example, we describe next an approach to the analysis of the deal effect curve shape used in promotional analysis, by using the SVM‐SR in an available database (Soguero‐Ruiz et al., 2012). Our data set is constructed from store‐level scanner data of a Spanish supermarket for the period 2 January until 31 December 1999. The number of days in the data set is 304. To account for the effects of price discounts with promotional periods, data were represented on a daily basis. Ground coffee category is considered, as it is a storable product with a daily rate of sales. Brands occupying the major positions in the market (more than 80% of sales) and being sold on promotion were selected, leading to the selection of six brands: two national low‐priced brands (#1, #6), and four national high‐priced brands (#2 to #5), as seen in Table 6.3.

The predicted variable is the sales units sold, at day i, i = 1, …, 304, for a certain brand k, k = 1, …, 5, in the category. Brand #6 was neither modeled nor considered in the model for the other brands, because it had no promotional discounts.

To capture the influence of the day of the week on the sales obtained on each day of the promotional period, we introduce two groups of dichotomic variables: for brand k and day i, variables are the indicators of the day of week (Monday (1) to Saturday (6)) during promotional periods, whereas are the indicators of the day of week (Monday (7) to Saturday (12)) during nonpromotional periods in brand k. By distinguishing between both groups of variables, we can observe the gap in sales between promotional and nonpromotional periods due to the seasonal component.

One of the characteristics of price discounts that researchers have commonly analyzed is the influence of promotional prices ending in the digit 9 in the sales obtained by the retailer (Blattberg and Neslin, 1993). To capture the influence of 9‐ending promotional prices in the sales obtained in the category, we introduced an indicator variable that takes the unit value when the promotional price of brand k is 9‐ending.

We also considered in our model the influence of the promotional price. In order to remove the effect of the price, the amplitude of the promotional discount was considered instead of the actual price, as proposed in Heerde et al. (2001). The price index, or ratio between the discounted price and the regular price, was introduced using a metric variable for each brand, , with m = 1, …, 6. Although we know that the retailer had used some kind of feature advertising and displays during the period considered, we do not have any information referring to their usage, so these important effects could not be included in the model.

For each SVM‐SR model, the free parameters were found using B = 20 bootstrap resamples for the R² estimator. The observations were split into training (75%) and testing. For the best set of free parameters, the CI for merit figures R², ρ, and the CI for parametric variables were obtained (95% content). As collinearity between metric variables for price indexes and dichotomic variables for promotional day of the week was suspected, a test for the exclusion of the metric variables was performed. Interaction effects between pairs of price indexes were obtained for each model.

Models for all the brands reached a significant R² (see Table 6.4). Two examples of model fitting are shown in Figure 6.9a and d for a high‐quality and a low‐quality brand respectively. Their corresponding CIs for the parametric variables are depicted in Figure 6.9b and e. It can be observed that the weekly pattern is significantly different in both of them, with the amplitude of the amplitude of the promotional oscillations being greater than the nonpromotional. The highest rates of sale correspond to promotional weekend periods. This behavior is present in all the other models (not shown). With respect to the 9‐ending effect, it significantly increased the sales in brand #1, but not in brand #2. The self‐effects of the in brand #2. The self‐effects of the promotion are shown in Figure 6.9c and f. The increment of sales in brand #2 (high quality).

Table 6.4 shows the results for the test of significance of the price indexes in the model. In all but one (brand #5), nonparametric variables were relevant to explain the sales jointly with the weekly oscillations. This can be seen as two different effects due to the promotion: an increase in the average level of sales (function of the price index), and a fixed increase in the weekly oscillations amplitude.

Once the relevance of the price indexes has been established, it is worth exploring the complex, nonlinear relationship among them. We only describe here two examples. Figure 6.9 g shows the cross‐effects of brand #1 on brand #3 model. For the simultaneous promotion situation, brand in a stronger competence than brand #3, as sales of the later fall. However, Figure 6.9 h illustrates a weaker competence, as promotion in brand #4 increases the sales even despite the simultaneous promotion in brand #5.