1 Introduction

Due to their intermittency and inter-temporal variations, the integration of renewable energy sources is very challenging [1]. A recent study in [2] has shown that significant wind-power curtailment may become inevitable if more renewable power-generation resources are installed without improving the existing infrastructure or using energy storage. other studies, e.g., in [3–5] have similarly suggested that energy storage can potentially help integrating renewable, in particular wind, energy resources. Although this basic idea has been widely speculated in the smart-grid community, it is still not clear how we can encourage major investment for building large-scale independently-owned storage units and how we should utilize the many different opportunities existing for these units. Addressing these open problems is the focus of this chapter.

The existing literature on integrating energy storage into smart grids is diverse. One thread of research, e.g., in [6–8], seeks to achieve various social objectives such as increasing the power-system reliability, reducing carbon emissions, and minimizing the total power-generation cost. They do not see the storage units as independent entities and rather assume that the operation of energy-storage systems is governed by a centralized controller. As a result, they do not address the profitability of investment in the storage sector and the possibility for storage units to participate in the wholesale market. Another thread of research, e.g., in [9–12], seeks to optimally operate a storage unit when it is combined and co-located with a wind farm. They essentially assume that it is the owner of the wind farm that must pay for the storage units. Clearly, this assumption may not always hold and it can certainly limit the opportunities to attract investors to build new energy-storage systems. Finally, there are some papers, such as [12–16], that aim to select optimal strategies for certain storage technologies, e.g., pumped hydro-storage units, to bid in the electricity market. However, they typically do not account for the uncertainties in the market prices, which can be a major decision factor if the amount of renewable power generation is significant. Moreover, they do not consider the opportunities for the energy-storage systems to participate not only in energy markets but also in reserve markets. Finally, the operation of large storage units such as pumped hydro are different from that of limited energy-storage units, which are of interest in this work. While pumped hydro units are mostly limited to the discharge rate or the capacity of turbines, batteries are limited by the available state of charge.

Therefore, the following question is yet to be answered: How can an energy-storage unit that is owned and operated by a private investor bid in both energy and reserve markets to maximize its profit, when there exists a significant penetration of unpredictable resources in the power grid? The storage unit may or may not be collocated with renewable or traditional generators. In fact, the location and size of the unit is decided by investors based on factors such as land availability and spot-price profile. In order to optimally operate the storage unit of interest, we propose a stochastic optimization approach to bid for energy and reserve in the day-ahead (DA) market and energy in the hour-ahead (HA) markets. Here, we assume a reserve-market structure similar to a simplified version of the day-ahead scheduling reserve market in the PJM (Pennsylvania, New Jersey, Maryland) inter-connection [17], where the exact utilization of the reserve bids is not decided by the storage unit; instead, it is decided by the market. As a result, finding the optimal charge and discharge schedules is particularly challenging when the storage unit participates in the reserve market. Another challenge is to formulate the bidding-optimization problem as a convex program to make it tractable and appropriate for practical scenarios. Compared to an earlier conference version of this work in [18], in this chapter, a more accurate solution approach is proposed to solve the formulated non-convex optimization problems. The new solution is more general and allows selling unused reserves in the hour-ahead energy market. Our contributions in this chapter can be summarized as follows:

• We propose a new stochastic optimization bidding mechanism for independent storage units in the DA and HA energy and reserve markets. Our design operates the charge and discharge cycles for the batteries such to assure meeting the future reserve commitments under different scenarios, regardless of the uncertainties that are present in the decision-making process.

• An important feature in our proposed market-participation model is that the power grid does not treat independent storage units any different from other energy and reserve resources. Therefore, our design can be used to encourage large-scale integration of energy-storage resources without the need for restructuring the market.

• We show through computer simulations that our proposed optimal energy and reserve-bidding mechanism is highly beneficial to independent storage units as it assures the profit gain of their investment. We also investigate the impact of various design parameters, such as the size and location of the storage unit on increasing the profit.

The rest of this chapter is organized as follows. The system model and the optimal-bidding problem formulation are explained in Section 2. Two different tractable design approaches to solve the formulated problems are presented in Section 3. Simulation results are presented in Section 4. The concluding remarks and future work are discussed in Section 5.

2 Problem Formulation

Consider a power grid with several traditional and renewable power generators as well as multiple independent energy-storage systems. We assume that not only the generators but also the storage units can bid and participate in the deregulated electricity market. As pointed out in Section 1, our key assumption is that the storage units are not treated any differently than other generators that participate in the energy or reserve markets. Since the energy-storage units in the system are owned and operated by private entities, they naturally seek to maximize their own profit. The stochastic wind generation, however, may create some extra benefits for storage units, considering that energy and reserve market prices may fluctuate significantly, giving them more opportunities to gain profit in the presence of high wind-power penetration. The assumption of high wind-power penetration adds to the load-forecast error component of the operating (DA scheduling) reserve,¹ since the wind generation is typically considered as part of the net load.

With more fluctuations in the net load, the operating reserves are more likely to be called up frequently. We also assume that the storage unit operates as a price-taker, i.e., it operates as a self-scheduling (must-run) unit and does not bid for price. Therefore, the storage unit will be compensated based on market-clearing prices. Of course, at the time of bidding in the day-ahead market, the storage unit does not know the actual prices, rather it only has an estimate of them. We also assume that the storage unit’s operation does not have impact on market prices due to its typically lower size (in megawatts) compared to traditional generators.

The main decision variables of the storage unit in our system model are the energy and reserve quantities for different hours in the DA market. However, the storage decisions and operation in the day-ahead market and hour-ahead (real-time) market are highly tied together. The storage unit’s bid in the DA market has a direct impact on the storage unit’s future profit in the HA market, since the commitments in the DA market will put some constraints in the charging and discharging profiles of the storage unit. An example of the charging and discharging cycles in the DA energy and reserve markets is shown in Figure 11.1, where the storage unit has committed to offer energy and reserve at three hours: h₁ = 7:00 AM, h₂ = 3:00 PM, and h₃ = 8:00 PM. In each case, the state-of-charge (SOC) of batteries must reach a level Cl_h ≥ P_h + R_h for all h ∈ {h₁, h₂, h₃}.

When a storage unit bids for reserve at a particular hour of the DA market, all, part of, or none of its committed reserve could be used. This will create uncertainty in the charging level of the storage unit; and depending on which value presumed for the utilization of reserve, the storage might have more or less charge available in real time. Therefore, even for the DA market bidding, the storage unit should have some information on the hour-ahead operation model.

When an independent storage unit submits a bid to the day-ahead market (DAM), it seeks to maximize its profit in the DA market plus the expected value of its profit in the next 24 hour-ahead markets (HAM). Therefore, prior to submitting the bids into the DA market, the storage unit should solve an operation-optimization problem in both the DA and HA markets for all the services that it aims to provide. The services that we aim to consider in this chapter are: DA power, DA reserve, and HA power.² Also the storage needs some estimation of the prices at hours h = 1 – 24 in both DA and HA markets. The prices of the HA markets on the next day will have more uncertainty due to availability variation and uncertainty in wind generation. The decision-making process of storage and the input information required is illustrated in Figure 11.2 This can be mathematically formulated as the following optimization problem³:

$\begin{array}{l}\begin{array}{cc}\hfill \underset{\mathrm{P},\mathrm{R},\mathrm{p}}{\mathbf{Maximize}}\hfill & \hfill \sum _{h=1}^{24}\left(P_h\cdot C{P}_h+R_h\cdot C{R}_h\right)+\mathbb{E}\left\{\mathit{HAM}\left(\mathrm{p},\mathrm{P},\mathrm{R};\mathrm{c}\tilde{\mathrm{p}},\mathrm{c}\tilde{\mathrm{r}},\tilde{\mathrm{r}}\right)\right\}\hfill \\ \hfill \underset{\forall h=1,\cdots ,24}{\mathbf{Subject}\mathbf{to}}\hfill & \hfill {\displaystyle \sum _{t=1}^h\left(P_t+{\tilde{r}}_t+p_t\right)\ge C{l}_{\mathit{init}}-C{l}_{\mathit{full}}}\hfill \end{array}\\ \begin{array}{cc}\hfill \hfill & \hfill {\displaystyle \sum _{t=1}^h\left(P_t+{\tilde{r}}_t+p_t\right)\le C{l}_{\mathit{init}}-C{l}_{\text{min}}}\hfill \end{array}\\ \begin{array}{cc}\hfill \hfill & \hfill R_h\ge 0\text{.}\hfill \end{array}\end{array}$

Note that P can take both positive and negative values while R is always positive. Negative values for P indicate purchasing power, i.e., charging. The expected value of the profit in the HA market, i.e., the second term in the objective function in (1), depends on not only the choices of P and R, but also the storage unit’s decision on the amount of power to be sold in the HA market p_h, the price of power in the HA market c$\tilde{\mathrm{p}}$, the price of reserve in the HA market c$\tilde{\mathrm{r}}$, the actual reserve utilization in the HA market $\tilde{\mathrm{r}}\text{,}$ and the fluctuations in wind generated. The third constraint assures that the reserve bid is non-negative. Note that, at the time of solving (1), $c{\tilde{p}}_h,c{\tilde{r}}_h,\mathrm{and}{\tilde{r}}_h\text{,}$ are unknown stochastic parameters.

Using the definition of mathematical expectation, we can rewrite the second term in (1) as a weighted summation of the aggregate HA profit terms, denoted by HAM, at many but finite scenarios, where the weight for each scenario is the probability for that scenario. That is, we can write:

$\mathbb{E}\left\{\mathit{HAM}\left(\tilde{\mathrm{p}},\mathrm{P},\mathrm{R};\mathrm{c}\tilde{\mathrm{p}},\mathrm{c}\tilde{\mathrm{r}},\tilde{\mathrm{r}}\right)\right\}={\displaystyle \sum _{k=1}^k{\mathit{\gamma }}_{\mathit{k}}\mathit{HA}{M}_k\text{,}}$

where HAM_k denotes the aggregate HA profit when scenario k occurs. We have ∑ _k = 1^Kγ_k = 1. It is worth clarifying that one of the main causes for profit uncertainty is the fluctuations in available wind power. Therefore, in our system model, each scenario is derived as a realization of available wind power at different wind-generation locations, given the wind-speed probability distribution functions, which is assumed to be available, e.g., by using the wind-forecasting techniques in [19–21]. For each scenario k, the corresponding aggregate HA profit can be calculated as follows:

$\begin{array}{l}\underset{{\mathrm{p}}_{\mathrm{k}}}{\mathbf{Max}}\mathit{HA}{M}_k\left(\mathrm{p},\mathrm{P},\mathrm{R};{\mathrm{cp}}_{\mathrm{k}},{\mathrm{cr}}_{\mathrm{k}},{\mathrm{r}}_{\mathrm{k}}\right)=\\ \underset{{\mathrm{p}}_{\mathrm{k}}}{\mathbf{Max}}{\displaystyle \sum _{h=1}^{24}\left(p_{k,h}\cdot c{p}_{k,h}+r_{k,h}\cdot c{r}_{k,h}\right)}\\ \underset{\forall h=1,\cdot \cdot \cdot ,24}{\mathbf{S}\mathbf{.}\mathbf{t}\mathbf{.}}{\displaystyle \sum _{t=1}^h{p}_{k,t}\le C{l}_{\mathit{init}}}-C{l}_{\mathit{min}}-{\displaystyle \sum _{t=1}^h\left(P_t+r_{k,t}\right)}\\ {\displaystyle \sum _{t=1}^h{p}_{k,t}\ge C{l}_{\mathit{init}}}-C{l}_{\mathit{full}}-{\displaystyle \sum _{t=1}^h\left(P_t+r_{k,t}\right)}\text{,}\end{array}$

where p_k is the adjustment to the power draw or power injection of the storage unit in the HA market for h = 1,…, 24, under scenario k. Here, cp_k, cr_k, and r_k are the actual realizations of the stochastic parameters $\mathrm{c}\tilde{\mathrm{p}},\mathrm{c}\tilde{\mathrm{r}},\mathrm{and}\tilde{\mathrm{r}}$ when scenario k occurs. We note that they are all set by the grid operator. The constraint in (3) indicates that the total generation bid up to hour h of the HA market has to be limited to the total charge available to the storage unit at hour h. Such total charge is calculated as the initial charge minus the sum of all the power drawn from the storage including the bid for power, i.e., P_h, and the reserve utilization in the HA market, i.e., r_k,_h, for all previous hours t = 1,…, h – 1.

Note that the actual reserve utilization r_k,_h may not always be as high as the committed reserve, as the grid may not need to utilize the entire reserve power offered by the storage unit. As a result, in the HA market, the storage unit needs to make corrective decisions to make the best use of any extra charge that is available due to different reserve utilizations caused by different wind availability and load scenarios. This makes dealing with parameter r_k,_h particularly complicated, as shown in Figure 11.3. Let r_h,k^max denote the maximum reserve power that the grid operator will need from the storage unit of interest at hour h if scenario k occurs. It is required that:

$r_{k,h}\le r_{k,h}^{\mathit{max}}h=1,\dots ,24\text{.}$

On the other hand, parameter r_k,_h also depends on the storage unit’s reserve commitment for each hour h based on its bid in the DA market. Therefore, it is further required that

$r_{k,h}\le R_{h}h=1,\dots ,24\text{.}$

From (4) and (5), at each hour h and scenario k, we have:

$r_{k,h}=min\left\{r_{k,h}^{\mathit{max}},R_h\right\}\text{.}$

Replacing (6) in the HA problem (3), it becomes:

$\begin{array}{l}\underset{{\mathrm{p}}_{\mathrm{k}}}{\mathbf{Max}}{\displaystyle \sum _{h=1}^{24}\left(p_{k,h}\cdot c{p}_{k,h}+min\left\{r_{k,h}^{\mathit{max}},R_h\right\}\cdot c{r}_{k,h}\right)}\\ \underset{\forall h}{\mathbf{S}\mathbf{.}\mathbf{t}\mathbf{.}}{\displaystyle \sum _{t=1}^h{p}_{k,t}+P_t+min\left\{r_{k,t}^{\mathit{max}},R_t\right\}}\le C{l}_{\mathit{init}}-C{l}_{\mathit{min}}\\ {\displaystyle \sum _{t=1}^h{p}_{k,t}+P_t+min\left\{r_{k,t}^{\mathit{max}},R_t\right\}\ge C{l}_{\mathit{init}}}-C{l}_{\mathit{full}}\text{.}\end{array}$

Next, we use the following equality [22]:

$\underset{x}{\mathit{sup}}\left(f\left(x\right)+\underset{y}{\mathit{sup}}\left(g\left(x,y\right)\right)\right)=\underset{x,y}{\mathit{sup}}\left(f\left(x\right)+g\left(x,y\right)\right)\text{,}$

and combine problems (1) and (3) into a single problem:

$\begin{array}{l}\underset{\mathrm{P},\mathrm{R},\mathrm{p}}{\mathbf{Max}}{\displaystyle \sum _{h=1}^{24}\left(P_h\cdot C{P}_h+R_h\cdot C{R}_h\right)+{\displaystyle \sum _{k=1}^K{\mathrm{\gamma }}_k}{\displaystyle \sum _{h=1}^{24}\left(p_{k,h}\cdot c{p}_{k,h}+min\left\{r_{k,h}^{\mathit{max}},R_h\right\}.c{r}_{k,h}\right)}}\\ \underset{\forall h,k}{\mathbf{S}\mathbf{.}\mathbf{t}\mathbf{.}}{\displaystyle \sum _{t=1}^h{p}_{k,t}+}{P}_t+min\left\{r_{k,t}^{\mathit{max}},R_t\right\}\le C{l}_{\mathit{init}}-C{l}_{\mathit{min}}\\ {\displaystyle \sum _{t=1}^h{p}_{k,t}+P_t+min\left\{r_{k,t}^{\mathit{max}},R_t\right\}\ge C{l}_{\mathit{init}}-C{l}_{\mathit{full}}}\\ R_h\ge 0\text{.}\end{array}$

However, optimization problem (9) is non-convex and hence difficult to solve. Note that the non-convexity is due to the way that the min function has appeared in the first constraint.

3 Solution Methods

In this section, we consider some practical assumptions in order to make problem (9) more tractable. In this regard, we take two approaches for choosing p_k,_h before solving the rest of the optimization problem. In both cases, we assume that the participation of the storage unit in the HA market is mainly to sell the unused charge from reserve bids. Therefore, for both approaches we always have p_k,_h ≥ 0.

5 Conclusions

Integration of large-scale storage systems in the power system is a key component of future smart grids. In this chapter, a novel approach was proposed to optimally operate such storage systems that are owned by independent private investors. In particular, we proposed an optimal-bidding mechanism for storage units to offer both energy and reserve in the day- ahead and the HA markets when significant fluctuation exists in the market prices due to high penetration of wind and intermittent renewable energy resources. Our design was based on formulating a stochastic programming framework to select different bidding variables. We showed that the formulated optimization problem can be transformed into convex-optimization problems that are tractable and appropriate for implementation. We showed that accounting for the unpredictable feature of market prices due to wind-power fluctuations can improve the decisions made by large storage units, hence increasing their profit. We also investigated the impact of various design parameters, such as the size and location of the storage unit on increasing the profit.

Appendix

First, we explain the new set of notations that we need in order to formulate and solve the standard stochastic unit-commitment problem. C(.) denotes the cost of a particular service. Comi_i,h denotes commitment of the ith unit in hour h. P_i,_h denotes generation of the ith unit in hour h. R_i,h denotes reserve commitment of the ith unit in hour h. P_i⁺ denotes the maximum capacity of the ith unit. P_i– denotes the minimum capacity of the ith unit. Ramp_i⁺ denotes the maximum ramp-up of the ith unit. Ramp_i^– denotes the maximum ramp down of the ith unit. G_f denotes the subset of the fast generators. p_i,k,h denotes the generation of the ith fast unit in the hour-ahead market for hour h and scenario k. r_i,k,_h denotes the reserve usage of the ith unit in the hour-ahead market for hour h and scenario k. NL_k,_h denotes the total net load in hour h of scenario k, which is the actual demand minus the wind-power generation. Given these notations and parameters, we can now formulate the standard SUC as follows, in which the realization scenarios are considered in order to minimize the expected value of the unit commitment cost in the system:

$\begin{array}{l}\begin{array}{c}\mathbf{min}\\ \mathit{Com},P,R,p_k,r_k\end{array}{\displaystyle \sum _h{\displaystyle \sum _i{C}_{\mathit{Co}{m}_i\cdot }}}\mathit{Co}{m}_{i,h}+C_{P_i}.P_{i,h}+C_{R_i}.R_{i.h}\\ +{\displaystyle \sum _k{\mathrm{\gamma }}_k}{\displaystyle \sum _h{\displaystyle \sum _i\left(c_{\mathit{pi}}\cdot p_{i,k,h}+c_{r_i}\cdot r_{i,k,h}\right)}}\\ \mathbf{s}\mathbf{.}\mathbf{t}\mathbf{.}{P}_{i,h}+R_{i,h}\le P_i^{+}\mathit{Co}{m}_{i,h}\forall i,h\\ P_{i,h}+R_{i,h}\ge P_i^{-}\mathit{Co}{m}_{i,h}\forall i,h\\ P_{i,h},R_{i,h}\ge 0\forall i,h\\ P_{i,h}-P_{i,h-1}\le \mathit{Ram}{p}_i^{+}\forall i,h\\ P_{i,h-1}-P_{i,h}\le \mathit{Ram}{p}_i^{-}\forall i,h\\ {\displaystyle \sum _i{P}_{i,h}+p_{i,k,h}+r_{i,k,h}}=N{L}_{k,h}\forall h,k\\ 0\le r_{i,k,h}\le R_{i,h}\forall i,k,h\\ 0\le p_{i,k,h}\le p_i^{+}\forall i\in G_f,k,h\\ p_{i,k,h}-p_{i,k,h-1}\le \mathit{Ram}{p}_i^{+}\forall i\in G_f,k,h\\ p_{i,k,h-1}-p_{i,k,h}\le \mathit{Ram}{p}_i^{-}\forall i\in G_f,k,h\\ \mathit{Co}{m}_{i,h}\in \left\{0,1\right\}\forall i,k,h\text{.}\end{array}$

The above SUC is a mixed-integer program. In general, mixed-integer programs are difficult to solve, although some classes of mixed-integer programs can be solved, e.g., using commercial optimization packages such as MOSEK and CPLEX software [2]. Alternatively, we can relax the binary constraint, solve problem (27), which is a convex-optimization problem after the binary constraints are relaxed, and then set Com_i,h = 1 for any i and h with the highest Com_i,h. If we repeat this operation until any Com_i,h in the solution is either zero or one, we can obtain a feasible but sub-optimal solution for the original problem (27). We used the latter approach. Given the solution and the Lagrange multipliers corresponding to each constraint, the needed system parameters are calculated accordingly, as we have already explained at the end of Section 2. Note that the above problem is solved in centralized fashion. We also note that since the storage units are assumed to have no impact on prices, this optimization problem does not include any variable from the storage units.