2.2.1.1 Model Specification: Two Treatments

Eleven trials compared PTCA with Acc t-PA (data shown in the last 11 rows of Table 2.1). We will define Acc t-PA as our reference or ‘control’ treatment, treatment 1, and PTCA will be treatment 2. Defining r_ik as the number of events (deaths), out of the total number of patients in each arm, n_ik, for arm k of trial i, we assume that the data generation process follows a binomial likelihood, that is

(2.3)

where p_ik represents the probability of an event in arm k of trial i (i = 1, …, 11; k = 1, 2).

Since the parameters of interest, p_ik, are probabilities and therefore can only take values between 0 and 1, a transformation (link function) is used that maps these probabilities into a continuous measure that can take any value between plus and minus infinities. The most commonly used link function for the probability parameter of a binomial likelihood is the logit link function (see also Chapter 4). The probabilities of success p_ik are modelled on the logit scale as

(2.4)

In this setup, μ_i are trial-specific baselines, representing the log odds of the outcome in the ‘control’ treatment (i.e. the treatment in arm 1), and δ_i,1k are the trial-specific log odds ratios (LORs) of an event on the treatment in arm k compared with the treatment in arm 1 (which in this case is always treatment 1), which can take any value between plus and minus infinities. We set δ_i,11, the relative effect of treatment 1 compared with itself, to zero, , which implies that the treatment effects δ_i,1k are only estimated when k > 1. Equation (2.4) can therefore also be written as

For a random effects model, the LORs for trial i, δ_i,12, obtained as , come from the random effects distribution in equation (2.1). For a fixed effects model, equation (2.4) is replaced by

(2.5)

where d₁₁ = 0. The pooled LOR d₁₂ is assigned the prior distribution in equation (2.2), and the prior distribution for the between-trial heterogeneity standard deviation is

(2.6)

where the upper limit of 2 is quite large on the log odds scale (see Section 2.3.2). As noted previously, care is required when specifying prior distributions for this parameter as they can be unintentionally very informative if the upper bound is not sufficiently large. Checks should be made to ensure that the posterior distribution is not unduly influenced by the upper limit chosen (see Section 2.3.2).

Note that although data are available for each arm of each trial, we are only interested in pooling the relative treatment effects measured in each trial as this is what RCTs are designed to inform. Thus pooling occurs at the relative effect level (i.e. we pool the LOR). An important feature of all the meta-analytic models presented here is that the trial-specific baselines μ_i are regarded as nuisance parameters that are estimated in the model but are of no further interest (see also Chapters 5 and 6). Therefore they will need to be given (non-informative) unrelated prior distributions:

(2.7)

An alternative is to place a second hierarchical model on the trial-specific baselines, or to put a bivariate normal model on both the baselines and treatment effects (van Houwelingen et al., 1993, 2002; Warn et al., 2002). However, unless this model is correct, the estimated relative treatment effects will be biased (Senn, 2010; Dias et al., 2013c; Senn et al., 2013). Our approach is therefore more conservative, in keeping with the widely used frequentist meta-analysis methods in which relative effect estimates are treated as data and study-specific baselines eliminated entirely (see also Chapter 5).

2.2.1.2 WinBUGS Implementation: Two Treatments

The implementation in WinBUGS of the fixed effects model described in equations (2.2), (2.3), (2.5) and (2.7) is as follows:

WinBUGS code for pairwise meta-analysis: Binomial likelihood, logit link, fixed effect model.

The # symbol is used for comments – text after this symbol is ignored by WinBUGS. Note that WinBUGS specifies the normal distribution in terms of its mean and precision.

The full code with data and initial values is presented in the onlinefile Ch2_FE_Bi_logit_pair.odc.

# Binomial likelihood, logit link
# pairwise meta-analysis (2 treatments)
# Fixed effect model
model{                       # *** PROGRAM STARTS
for(i in 1:ns){              # LOOP THROUGH STUDIES
    mu[i] ~ dnorm(0,.0001)   # vague priors for all trial baselines
    for (k in 1:2) {         # LOOP THROUGH ARMS
        r[i,k] ~ dbin(p[i,k],n[i,k])  # binomial likelihood
        logit(p[i,k]) <- mu[i] + d[k] # model for linear predictor
   }
  }   
d[1]<- 0        # treatment effect is zero for reference treatment
d[2] ~ dnorm(0,.0001) # vague prior for treatment effect
}                     # *** PROGRAM ENDS 

As with all WinBUGS models, the likelihood describing the data generating process needs to be specified along with a ‘model’ and prior distributions. For each study i and for each study arm k, the data are in the form of events r[i,k] out of total patients n[i,k], which are described as coming from a binomial distribution with probability p[i,k]: r[i,k] ~ dbin(p[i,k],n[i,k]). Since multiple trials are available, each with two arms, the likelihood needs to be specified multiple times, and this is done by enclosing it within a loop where i goes from 1 to ns, the number of included studies (given as data), and a loop covering all treatment arms, k = 1 and 2. This specifies the appropriate likelihood for each arm of each study. Also within the i and k loops is the main part of the meta-analysis model, termed the linear predictor (see Chapter 4), which translates equation (2.5) into code. In the model, d[k] represents the fixed treatment effect of treatment k compared with treatment 1, that is, d[k] = d_1k. Prior distributions for the trial-specific baselines (nuisance parameters) mu[i] are specified for each trial within the i loop. Finally, d[1] is set to zero and the prior distribution for d[2] is specified (equation (2.2)).

In WinBUGS, the model must be checked for correct syntax by clicking on ‘check model’ in the Specification Tool obtained from the menu Model→Specification…. With the model checked, data must be loaded for the programme to compile. The data to load has two components: a list specifying the number of studies ns (in the example, ns = 11) and the main body of data in a column or vector format where r[,1] and n[,1] are the numerators and denominators for treatment 1 and r[,2] and n[,2] the numerators and denominators for treatment 2, respectively. For this example these values are taken from the last 11 rows of Table 2.1. Text is included after the hash symbol (#) for ease of reference to the original data source and to facilitate model diagnostics but is ignored by WinBUGS. Both sets of data need to be loaded for the model to run. This can be done in WinBUGS by clicking ‘load data’ in the Specification Tool for each type of data in turn:

# Data (Extended Thrombolytics example – 2 treatments)
list(ns=11)
r[,1]    n[,1]    r[,2]    n[,2]    #    Study ID
3        55       1        55       #    Ribichini 1996
10       94       3        95       #    Garcia 1997
40       573      32       565      #    GUSTO-2 1997
5        75       5        75       #    Vermeer 1999
5        69       3        71       #    Schomig 2000
2        61       3        62       #    LeMay 2001
19       419      20       421      #    Bonnefoy 2002
59       782      52       790      #    Andersen 2002
5        81       2        81       #    Kastrati 2002
16       226      12       225      #    Aversano 2002
8        66       6        71       #    Grines 2002
END

Once the data are loaded, the number of chains to run can be set in the appropriate box and the model can be complied by clicking ‘compile’. Appropriate initial values need to be given to all parameters with a distribution (except the data) for the model to run. We will run three chains with the following initial values:

# Initial values
#chain 1
list(d=c( NA, 0),   mu=c(0,0,0,0,0,     0,0,0,0,0,      0))
#chain 2
list(d=c( NA, -1),  mu=c(-3,-3,-3,-3,-3,   -3,-3,-3,-3,-3, -3))
#chain 3
list(d=c( NA, 2),   mu=c(-3,5,-1,-3,7,    -3,-4,-3,-3,0,  -7))

Note that parameter d is a vector with two components: d[1], which is fixed (set to zero) and therefore does not require initial values to be specified, and d[2], which is assigned a prior distribution and therefore requires an initial value. The initial value for the first component of vector d is therefore set as ‘NA’, which denotes a missing value in WinBUGS. The trial-specific baselines mu are also defined as vectors with length equal to the number of studies (in this case 11); therefore eleven initial values need to be specified. Initial values are loaded by clicking ‘load inits’ for each chain in turn. Any set of plausible initial values can be given and any number of chains can be run simultaneously. Plausible initial values are those that are allowed by each parameter’s prior distribution, but very extreme values should be avoided as they can produce numerical errors. Note that once the model has converged, the initial values will have no influence on the posterior distributions so they are not meant to reflect any prior knowledge on the likely values of the parameters.

We recommend running at least two chains with very different starting values in order to properly assess convergence and robustness of results to the starting values (Welton et al., 2012; Lunn et al., 2013). However, we often choose to run three chains to allow a greater variety of starting values to be tried. The model can then be run from the Model→Update… menu. Parameters can be monitored, convergence checked and outputs obtained using the Inference→Samples… menu. For further details see the WinBUGS help menu and Lunn et al. (2013).

In this example, convergence was achieved by 10,000 iterations. A further 20,000 iterations were run on three chains, giving a posterior sample of 60,000 values, on which all results are based. Posterior summaries for d[2], the pooled LOR of mortality on treatment 2 (PTCA) compared with treatment 1 (Acc t-PA), are given as follows:

The columns labelled ‘mean’ and ‘sd’ give the mean and standard deviation of the posterior distribution of d[2]. The column labelled ‘MC error’ shows the Monte Carlo standard error of the samples, which will decrease as the number of iterations increases and should typically be small. A common rule of thumb is to ensure that the MC error is less than 5% of the posterior standard deviation (Lunn et al., 2013). In the example, 5% of the standard deviation is 0.1178 × 0.05 = 0.00589, while the MC error = 0.0007668, which is considerably smaller. We can therefore be satisfied that sufficient posterior samples have been used for inference.

Bounds for the 95% credible interval (CrI) are given by the 2.5 and 97.5% quantiles, presented in the columns labelled ‘2.5%’ and ‘97.5%’, respectively. The median of the posterior distribution is given in the column labelled ‘median’, and finally the columns labelled ‘start’ and ‘sample’ give the starting iteration and the total number of values the results are based on. The full posterior distribution of d[2] is shown in Figure 2.2.

In general we summarise the posterior distribution by its median and 95% CrI, but note that for (approximately) symmetric distributions such as in Figure 2.2, the median and the mean will be very similar. In the example, using two decimal places, mean = median = −0.23. Hence we will say that the posterior median (and mean) of the LOR of mortality is −0.23 with 95% CrI (−0.47, −0.003), which is wholly negative, meaning that the odds of mortality on PTCA are lower than on Acc t-PA. In addition, Figure 2.2 shows that the probability that this LOR is positive is small.

However, usually we want to report the odds ratio (OR) of mortality with its credible interval. This can be easily achieved in WinBUGS by adding the following code before the last closing brace, which states that the OR is obtained by exponentiating the LOR:

or <- exp(d[2])

In addition, we may want to quantify the probability that the LOR is positive, that is, that PTCA does not reduce mortality compared with Acc t-PA. This can be done by adding the following code:

prob.harm <- step(d[2])

where prob.harm will hold the required probability using the step function, which returns the value 1 when its argument is greater than or equal to zero. Averaging the number of times d[2] ≥ 0 over all iterations (post-convergence) will give the probability that d[2] ≥ 0, that is, the probability that the OR for mortality is greater than 1.

Monitoring or will give posterior summaries for the OR of mortality. Monitoring prob.harm provides several summaries, but only the posterior mean is of interest as it represents the probability that PTCA actually increases mortality compared with Acc t-PA:

The posterior median OR is 0.79 with 95% CrI (0.63, 0.997), indicating that PTCA reduces mortality by about 20% compared with Acc t-PA, although the upper bound of the CrI is nearly 1, suggesting the possibility of no effect. The probability that PTCA actually increases mortality is small at 0.024 (note that only the column headed ‘mean’ is interpretable for this parameter).

The implementation of the random effects model described in equations (2.1), (2.2), (2.3), (2.4), (2.6) and (2.7) is as follows:

WinBUGS code for Binomial likelihood, logit link, random effects model, two treatments.

The # symbol is used for comments – text after this symbol is ignored by WinBUGS. Note that WinBUGS specifies the normal distribution in terms of its mean and precision.

The full code with data and initial values is presented in the online file Ch2_RE_Bi_logit_pair.odc.

# Binomial likelihood, logit link
# pairwise meta-analysis (2 treatments)
# Random effects model
model{                     # *** PROGRAM STARTS
for(i in 1:ns){            # LOOP THROUGH STUDIES
   delta[i,1] <- 0         # treatment effect is zero for control arm
   mu[i] ~ dnorm(0,.0001)  # vague priors for all trial baselines
   for (k in 1:2) {        # LOOP THROUGH ARMS
       r[i,k] ~ dbin(p[i,k],n[i,k]) # binomial likelihood
       logit(p[i,k]) <- mu[i] + delta[i,k]  # model for linear predictor
     }
    delta[i,2] ~ dnorm(d[2],tau)   # trial-specific LOR distributions
  }   
d[1]<- 0                   # treatment effect is zero for reference treatment
d[2] ~ dnorm(0,.0001)      # vague prior for treatment effect
sd ~ dunif(0,2)            # vague prior for between-trial SD
tau <- pow(sd,-2)          # between-trial precision = (1/between-trial variance)
}                          # *** PROGRAM ENDS                            

Most of the lines in the random effects code are the same as for the fixed effects model. In particular the likelihood and prior distributions for d and mu are the same, as are all the loops. Differences between the two sets of code are highlighted in bold. Since we are now using the model in equation (2.4), we need to set delta[i,1] to zero for each study and specify the random effects distribution in equation (2.1), as well as the prior distribution for the between-study standard deviation, sd. Note that WinBUGS describes the normal distribution in terms of its mean and precision; therefore we need to add the extra variable tau, which is defined as the inverse of the between-study variance, that is, the precision. Redundant subscripts have been dropped in the code.

The model needs to be checked, data loaded, compiled and initial values given, as before. The data structure is exactly the same as for the fixed effects model, but we now need to add an initial value for the between-study heterogeneity parameter sd, which is a single number (scalar):

# Initial values
#chain 1
list(d=c( NA, 0),  sd=1,   mu=c(0,0,0,0,0,      0,0,0,0,0,       0))
#chain 2
list(d=c( NA, -1), sd=0.1, mu=c(-3,-3,-3,-3,-3,   -3,-3,-3,-3,-3,   -3))
#chain 3
list(d=c( NA, 2),  sd=0.5, mu=c(-3,5,-1,-3,7,    -3,-4,-3,-3,0,   -7))

Any value can be chosen for sd as long as it is within the bounds of the prior distribution, in this case between zero and two (but not actually zero or two). Note that we have not specified initial values for the study-specific treatment effects delta. Although as a general rule initial values should be specified for all nodes assigned a distribution (except the data), in this case we can omit the initial values for delta and allow WinBUGS to generate them by clicking the ‘gen inits’ button after loading the initial values for all the chains. This is because the initial values that will be generated for delta will be chosen from the random effects distribution that is specified in terms of d and tau, to which we have given sensible (i.e. not very extreme) initial values, ensuring that values for delta will also not be very extreme. The same code can be added to calculate the OR and probability of harm.

Results from running both the fixed and random effects models are shown in Table 2.2. All results are based on 20,000 iterations on three chains, after a burn-in of 10,000.

Results for the fixed and random effects model are similar, although the latter produces wider credible intervals, since it allows for between-trial heterogeneity. However, the posterior median of the between-study heterogeneity is relatively small, and the lower bound for its 95% CrI is very close to zero. Its full posterior distribution is shown in Figure 2.3, and we can see that it does not resemble a uniform distribution, suggesting it has been suitably updated from the prior distribution, which was not too restrictive in this case (see Section 2.3.2 for more details).

In Chapter 3, we will discuss how to assess global model fit and how to choose between random and fixed effects models, but for now we note that the posterior distribution for the heterogeneity is mainly concentrated around small values.

2.2.2.1 Incorporating Multi-Arm Trials

Suppose we have a number of multi-arm trials involving some or all of the treatments of interest, 1, 2, 3, 4, and so on. Among commonly suggested stratagems for dealing with such trials are combining all active arms into one, splitting the control group between all relevant experimental groups or ignoring all but two of the trial arms (Higgins and Green, 2008). None of these are satisfactory. In the NMA framework set out previously, multi-arm trials are naturally incorporated in the synthesis. No special considerations are required when fitting fixed effects models as long as the data are set out correctly (see Section 2.2.3.1).

The question of how to conduct a random effects meta-analysis of multi-arms trials has been considered in a Bayesian framework by Lu and Ades (2004), and in a frequentist framework by Lumley (2002) and Chootrakool and Shi (2008). Based on the same exchangeability assumptions described previously, a single multi-arm trial will estimate a vector of random effects δ_i. For example, a three-arm trial will produce two random effects and a four-arm trial three. However, these multiple random effects are correlated and this must be correctly accounted for. Assuming, as before, that the relative effects all have the same between-trial variance, we have

(2.11)

where δ_i is the vector of random effects, which follows a multivariate normal distribution, a_i represents the number of arms in trial i (a_i = 2, 3, …) and is as given in equation (2.9). Then the conditional univariate distribution (Raiffa and Schlaiffer, 2000) for the random effect of arm k > 2, given all arms from 2 to k − 1, is

(2.12)

Either the multivariate distribution in equation (2.11) or the conditional distributions in equation (2.12) must be used to estimate the random effects for each multi-arm study so that the between-arm correlations between parameters are taken into account. We will use the formulation in equation (2.12) as it allows for a more generic code that works for trials with any number of arms. This general formulation is no different from the model presented by Higgins and Whitehead (1996) and provides another interpretation of the exchangeability assumptions. It is indeed another way of deducing the consistency relations: we may consider a connected network of M trials involving S treatments to originate from M S-arm trials, but with some of the arms missing at random, conditional on the trial design and choice of treatments, at randomisation (Section 1.5).

The WinBUGS code presented in Section 2.2.3.1 exactly instantiates the theory behind NMA that relates it to pairwise meta-analysis. Therefore it will analyse pairwise meta-analysis, indirect comparisons and NMA (including combined direct and indirect evidence) with or without multi-arm trials without distinction.

2.2.3.1 WinBUGS Implementation

The implementation of the NMA model with fixed treatment effects is given as follows:

WinBUGS code for NMA: Binomial likelihood, logit link, fixed effect model.

The # symbol is used for comments – text after this symbol is ignored by WinBUGS. Note that WinBUGS specifies the normal distribution in terms of its mean and precision.

The full code with data and initial values is presented in the online file Ch2_FE_Bi_logit.odc.

# Binomial likelihood, logit link
# Fixed effect model
model{                       # *** PROGRAM STARTS
for(i in 1:ns){              # LOOP THROUGH STUDIES
    mu[i] ~ dnorm(0,.0001)   # vague priors for all trial baselines
    for (k in 1:na[i])  {    # LOOP THROUGH ARMS
      r[i,k] ~ dbin(p[i,k],n[i,k])     # binomial likelihood
      logit(p[i,k]) <- mu[i] + d[t[i,k]] - d[t[i,1]]    # model for linear predictor
      }
   }   
d[1]<-0          # treatment effect is zero for reference treatment
for (k in 2:nt){  d[k] ~ dnorm(0,.0001) }    # vague priors for treatment effects
}                            # *** PROGRAM ENDS                             

Most of the lines in this fully generic (network) meta-analysis code are the same as in the previous code for pairwise meta-analysis with a fixed effect. The likelihood and prior distributions for mu[i] are the same, but now the loop covering all treatment arms needs to have k going from 1 to the total number of arms in that trial, which may be greater than 2, defined in na[i], loaded as data. The linear predictor now translates equation (2.14) into code where t[i,k] represents the treatment in arm k of trial i and d[k] = d_1k. As before d[1] is set to zero, and prior distributions for all other d[k] are specified, with nt representing the total number of treatments compared in the network, which is given as data. Note that no special consideration is needed for multi-arm trials in fixed effects models since no random effects are estimated, and therefore no within-trial correlation needs to be accounted for.

The data structure is presented in the succeeding text and has two components: a list specifying the number of studies ns and treatments nt (in this example ns = 36, nt = 7) and the main body of data. Both sets of data need to be loaded for the model to run (see Section 2.2.1.2). The main body of data is in a vector format, and we need to allow for a three-arm trial. Three column places are therefore required to specify the treatments, t, the number of events, r, and the number of patients, n, in each arm; ‘NA’ indicates that values are missing for a particular cell: r[,1] and n[,1] are the numerators and denominators for the first arm, r[,2] and n[,2] are the numerators and denominators for the second arm and r[,3] and n[,3] represent the numerators and denominators for the third arm of each trial, respectively. We specify t[,1], t[,2] and t[,3] as the treatment number identifiers for the first, second and third arms of each trial, respectively, that is, they identify the treatments compared in each arm to which r and n correspond. We also add a column with the number of arms in each trial, na[]. Only the first trial in this dataset has three arms; all other trials have two arms. Trial identifiers are added as comments, which are ignored by WinBUGS:

# Data (Thrombolytics example)
list(ns=36, nt=7)
na[]  t[,1]  t[,2]  t[,3]  r[,1]  n[,1]  r[,2]  n[,2]  r[,3]  n[,3]  #ID         year
3     1      3      4      1472   20251  652    10396  723    10374  #GUSTO-1     1993
2     1      2      NA     3      65     3      64     NA     NA     #ECSG       1985
2     1      2      NA     12     159    7      157    NA     NA     #TIMI-1      1987
2     1      2      NA     7      85     4      86     NA     NA     #PAIMS      1989
2     1      2      NA     10     135    5      135    NA     NA     #White      1989
2     1      2      NA     887    10396  929    10372  NA     NA     #GISSI-2     1990
2     1      2      NA     5      63     2      59     NA     NA     #Cherng     1992
2     1      2      NA     1455   13780  1418   13746  NA     NA     #ISIS-3      1992
2     1      2      NA     9      130    6      123    NA     NA     #CI         1993
2     1      4      NA     4      107    6      109    NA     NA     #KAMIT      1991
2     1      5      NA     285    3004   270    3006   NA     NA     #INJECT     1995
2     1      7      NA     11     149    2      152    NA     NA     #Zijlstra   1993
2     1      7      NA     1      50     3      50     NA     NA     #Riberio    1993
2     1      7      NA     8      58     5      54     NA     NA     #Grinfeld   1996
2     1      7      NA     1      53     1      47     NA     NA     #Zijlstra   1997
2     1      7      NA     4      45     0      42     NA     NA     #Akhras     1997
2     1      7      NA     14     99     7      101    NA     NA     #Widimsky   2000
2     1      7      NA     9      41     3      46     NA     NA     #DeBoer     2002
2     1      7      NA     42     421    29     429    NA     NA     #Widimsky   2002
2     2      7      NA     2      44     3      46     NA     NA     #DeWood     1990
2     2      7      NA     13     200    5      195    NA     NA     #Grines     1993
2     2      7      NA     2      56     2      47     NA     NA     #Gibbons    1993
2     3      5      NA     13     155    7      169    NA     NA     #RAPID-2     1996
2     3      5      NA     356    4921   757    10138  NA     NA     #GUSTO-3     1997
2     3      6      NA     522    8488   523    8461   NA     NA     #ASSENT-2    1999
2     3      7      NA     3      55     1      55     NA     NA     #Ribichini  1996
2     3      7      NA     10     94     3      95     NA     NA     #Garcia     1997
2     3      7      NA     40     573    32     565    NA     NA     #GUSTO-2     1997
2     3      7      NA     5      75     5      75     NA     NA     #Vermeer    1999
2     3      7      NA     5      69     3      71     NA     NA     #Schomig    2000
2     3      7      NA     2      61     3      62     NA     NA     #LeMay      2001
2     3      7      NA     19     419    20     421    NA     NA     #Bonnefoy   2002
2     3      7      NA     59     782    52     790    NA     NA     #Andersen   2002
2     3      7      NA     5      81     2      81     NA     NA     #Kastrati   2002
2     3      7      NA     16     226    12     225    NA     NA     #Aversano   2002
2     3      7      NA     8      66     6      71     NA     NA     #Grines     2002
END

In setting up the data structure, the maximum number of columns needed to define each variable is the maximum number of arms in the trials included in the dataset, that is, the maximum number specified in column na.

An important feature of the code presented is the assumption that the treatments are always presented in ascending (numerical) order and that treatment 1 is taken as the reference treatment. So, for example, the first row of data has the columns in order so that data for treatments 1, 3 and 4 are presented in the first, second and third columns, respectively, rather than having, for example, data for treatment 1, then 4 and then 3. Although this is not very important for the simple NMA models presented in this chapter and in Chapters 3 and 4 (the code will work regardless), it is crucial to have the data set up in this way to explore inconsistency (Chapter 7) and, when extending the model to include covariates (Chapter 8) or bias models (Chapter 9), to ensure the correct relative effects are estimated and appropriate assumptions are implemented.

We will run three chains using the following initial values:

# Initial values
#chain 1
list(d=c( NA, 0,0,0,0,   0,0), mu=c(0,0,0,0,0,  0,0,0,0,0,   0,0,0,0,0,   0,0,0,0,0,    0,0,0,0,0,  0,0,0,0,0,     0,0,0,0,0,  0))
#chain 2
list(d=c( NA, -1,-1,-1,-1, -1,-1),  mu=c(-3,-3,-3,-3,-3, -3,-3,-3,-3,-3,   -3,-3,-3,-3,-3,    -3,-3,-3,-3,-3,  -3,-3,-3,-3,-3,  -3,-3,-3,-3,-3,   -3,-3,-3,-3,-3,  -3))
#chain 3
list(d=c( NA, 2,5,-3,1,    -7,4), mu=c(-3,5,-1,-3,7,  -3,-4,-3,-3,9,-3,-3,-4,3,5,         -3, -2, 1, -3, -7,  -3,5,-1,-3,7,  -3,-4,-3,-3,0,           -3, 5,-1,-3,7,     -7))

Again note that d is a vector with seven components corresponding to the number of treatments being compared, but the initial value for the first component of vector d needs to be ‘NA’ since d[1] is set to zero and is therefore not estimated.

Running this model gives the following posterior summaries for the basic parameters, which represent the pooled LORs of mortality for all treatments compared with treatment 1 (SK). These are also summarised in Figure 2.4.

These LORs can be displayed in WinBUGS as a caterpillar plot, which can be obtained using the Inference→Compare… menu, by typing d in the ‘node’ box and clicking ‘caterpillar’. Treatment 7, PTCA, provides the largest reduction in mortality with posterior median of the LOR of −0.47 and 95% CrI (−0.67, −0.28).

Additional code can be added before the last closing brace to estimate all the pairwise LORs and ORs (i.e. for all treatments compared with every other), to generate ranking statistics and to calculate the probability that each treatment is the best treatment as well as the probabilities that each treatment is ranked 1st, 2nd, 3rd, and so on:

# pairwise ORs and LORs for all possible pairwise comparisons
for (c in 1:(nt-1)) {  
  for (k in (c+1):nt)  { 
       or[c,k] <- exp(d[k] - d[c])
       lor[c,k] <- (d[k]-d[c])
       }  
  }
# ranking on relative scale
for (k in 1:nt) { 
# rk[k] <- nt+1-rank(d[],k) # assumes events are “good”
  rk[k] <- rank(d[],k) # assumes events are “bad”
best[k] <- equals(rk[k],1) # calculate probability that treat k is best
for (h in 1:nt){ prob[h,k] <- equals(rk[k],h) }     # calculate probability that treat k is h-th best
}

In addition, given an assumption about the absolute effect of one treatment (see Chapter 5), it is possible to produce estimates of absolute effects on all treatments; to express the treatment effect on other scales such as the risk difference (RD), defined as the difference in probabilities of an event on treatment k compared with the reference treatment, and the relative risk (RR), defined as the ratio of the probabilities of an outcome on treatment k compared with treatment 1; or to calculate the number needed to treat (NNT), defined as 1/RD (Hutton, 2000; Deeks, 2002; Higgins and Green, 2008). An advantage of the Bayesian MCMC is that appropriate distributions, and therefore credible intervals, are automatically generated for all these quantities. This is achieved in WinBUGS by adding the following code:

# Provide estimates of treatment effects T[k] on the
# natural (probability) scale, given a mean effect, meanA,
# for ‘standard’ treatment 1, with precision precA
A ~ dnorm(meanA,precA)
for (k in 1:nt) { logit(T[k]) <- A + d[k]  }

In this case, meanA and precA will be the assumed mean and precision of the log odds of mortality on treatment 1, respectively, and T[k] will represent the absolute probabilities of mortality on each treatment k = 1, …, 7. Given the information on the absolute probabilities of mortality T[1], other quantities can be calculated using the following code:

# Provide estimates of number needed to treat NNT[k]
# Risk Difference RD[k] and Relative Risk RR[k],
# for each treatment relative to treatment 1
for (k in 2:nt) { 
#    NNT[k] <- 1/(T[k] - T[1])  # assumes events are “good”
    NNT[k] <- 1/(T[1]- T[k])    # assumes events are “bad”
    RD[k] <- T[k] - T[1]
    RR[k] <- T[k]/T[1]
 }

We will assume the absolute probability of mortality on SK is 8% with 95% CrI from 7 to 10%, which is approximately equivalent to assuming that the log odds of mortality on SK follows a normal distribution with a mean of −2.39 and precision of 11.9. For considerations on how to estimate or elicit these values, see Chapter 5. These values can be given as data or replaced in the aforementioned code. We will include them in the data by changing the list to read

list(ns=36, nt=7, meanA=-2.39, precA=11.9)

and loading that into WinBUGS.

The ORs of mortality for each treatment compared with every other can be obtained by monitoring node or. WinBUGS output for the fixed effects model is given in the succeeding text and summarised in Table 2.3 and Figure 2.5.

We can see that PTCA is better at reducing mortality than all other treatments, while, for example, Acc t-PA is slightly better than SK + t-PA, reducing the odds of mortality by about 12% with 95% CrI from 1 to 25%.

Note however that the estimates for the OR of mortality on PTCA compared with Acc t-PA are slightly different from those in Table 2.2 and the 95% CrI is narrower. This is expected since we are now using all the evidence in the network (both direct and indirect) to estimate this relative treatment effect.

A plot similar to Figure 2.5 can be obtained in WinBUGS by doing a ‘caterpillar’ plot of the parameter lor, although note that these will be the LORs. See Exercise 2.1.

The probabilities that each treatment is ranked 1st, 2nd, …, 7th are shown in Figure 2.6. Peaks in these figures indicate the most likely rank for that treatment, and the probabilities of each rank can be read on the vertical axis.

As expected, PTCA has a very high probability of being the best treatment, and so the probability that it is ranked 1st is close to 100%. Among the other treatments, Acc t-PA has probability of 42% being ranked 2nd and 48% of being ranked 3rd, and TNK also has about 43% probability of being ranked 2nd, suggesting these may be the best alternatives to PTCA, when it is not available. This agrees with the ORs in Figure 2.5, where apart from PTCA, Acc t-PA and TNK are the only other treatments that are favoured over the alternatives, and there seems to be no advantage of one over the other. The absolute probabilities, ranks and probabilities of being the best for all treatments are shown in Table 2.4.

The implementation of the random effects model accounting for the correlations in multi-arm trials is as follows:

WinBUGS code for NMA: Binomial likelihood, logit link, random effects model.

The # symbol is used for comments – text after this symbol is ignored by WinBUGS. Note that WinBUGS specifies the normal distribution in terms of its mean and precision.

The full code with data and initial values is presented in the online file Ch2_RE_Bi_logit.odc.

# Binomial likelihood, logit link
# Random effects model for multi-arm trials
model{                                               # *** PROGRAM STARTS
for(i in 1:ns){                                      # LOOP THROUGH STUDIES
   w[i,1] <- 0                                       # adjustment for multi-arm trials is zero for control arm
   delta[i,1] <- 0                                   # treatment effect is zero for control arm
   mu[i] ~ dnorm(0,.0001)                            # vague priors for all trial baselines
   for (k in 1:na[i])  {                             # LOOP THROUGH ARMS
     r[i,k] ~ dbin(p[i,k],n[i,k])                    # binomial likelihood
     logit(p[i,k]) <- mu[i] + delta[i,k]             # model for linear predictor
   }
   for (k in 2:na[i]) {                              # LOOP THROUGH ARMS
     delta[i,k] ~ dnorm(md[i,k],taud[i,k])           # trial-specific LOR distributions
     md[i,k] <- d[t[i,k]] - d[t[i,1]] + sw[i,k]      # mean of LOR distributions (multi-arm trial correction)
     taud[i,k] <- tau *2*(k-1)/k                     # precision of LOR distributions (with multi-arm trial correction)
     w[i,k] <- (delta[i,k] - d[t[i,k]] + d[t[i,1]])  # adjustment for multi-arm RCTs
     sw[i,k] <- sum(w[i,1:k-1])/(k-1)                # cumulative adjustment for multi-arm trials
   }
 }
d[1] <- 0                                            # treatment effect is zero for reference treatment
for (k in 2:nt){  d[k] ~ dnorm(0,.0001) }            # vague priors for treatment effects
sd ~ dunif(0,2)                                      # vague prior for between-trial SD.
tau <- pow(sd,-2)                                    # between-trial precision = (1/between-trial variance)
}                                                    # *** PROGRAM ENDS 

Readers should compare this code with the random effects code for pairwise meta-analysis and the fixed effects code for NMA and note the similarities. The main difference in this code is the implementation of the conditional univariate normal distributions for the trial-specific random effects given in equation (2.12).

We have coded the mean of the random effects normal distribution as md (WinBUGS does not allow algebraic expressions in distributions) and defined md as where sw_ik is the adjustment for the conditional normal distribution given as follows:

and taud is the adjusted precision for the conditional univariate normal, that is, the inverse of the aforementioned variance. We set w[i,1] to zero so that the code reduces to a simple univariate normal distribution (equation (2.8)) for two-arm trials.

Changes to the initial values are also required. Initial values need to be specified for the between-study heterogeneity sd, and WinBUGS can be left to generate initial values for the parameter delta, as before.

Results from the random effects model are given in Table 2.3. The posterior median of the between-study standard deviation is small compared with the size of the treatment effect of PTCA (Table 2.3), and its posterior distribution is concentrated around small values (Figure 2.7). We might therefore question whether a random effects model is really necessary here. Model comparison and model choice will be discussed in Chapter 3.

The posterior distribution of the between-study standard deviation (Figure 2.7) is far from the uniform distribution between zero and two that was set up as prior. This confirms that updating has taken place and that the prior distribution was not too restrictive (see Section 2.3.2 for more details). We should also note that the distribution is slightly different from that in Figure 2.3, since we are now using more evidence to estimate this parameter. In this case this has meant that the distribution is slightly narrower and concentrated around smaller values (see Section 2.3.2).