Full text: Power market operations and system reliability / Hogan, Michael

Power Market Operations and System Reliability
A contribution to the market design debate in the P ­ entalateral Energy Forum


Power Market Operations and System Reliability

IMPULSE Power Market Operations and System Reliability A contribution to the market design debate in the P ­ entalateral Energy Forum

STUDY ON BEHALF OF Agora Energiewende Rosenstrasse 2 | 10178 Berlin | Germany Project lead: Christian Redl

WRITTEN BY The Regulatory Assistance Project Rue de la Science 23 | 1040 Brussels | Belgium Michael Hogan Frederick Weston

Proof reading: Translationes, Berlin Typesetting: UKEX GRAPHIC, Ettlingen Cover: © Guillaume Le Bloas -

Please quote as: RAP (2014): Power Market Operations and System Reliability: A contribution to the market design debate in the Pentalateral Energy Forum. Study on behalf of Agora Energiewende.

058/08-I-2014/EN Published: December 2014

Dear reader, If we are to achieve a low carbon, competitive and secure power system in Europe, a refined market design that stresses increased system flexibility is essential. While baseload, midmerit and peakload power plants previously simply followed demand, today, renewable energy technologies – particularly wind and solar – require demand response, flexible generation capacities, smart grids and storage technologies reflecting the need for enhanced flexibility. In designing a refined power market regime, we must adequately take into account the interactions between the system’s various elements – whether technical, regulatory or political. Key elements that need to be considered include existing power markets, renewable energy support schemes, grid planning and operations and, potentially, additional ­instruments, such as capacity mechanisms. Huge welfare gains can be expected if the further optimisation of the regulatory regime for power markets were to be undertaken at a pan-European level. Regional cooperation is an important intermediate step towards achieving a pan-­ European solution. The Pentalateral Energy Forum – a regional initiative comprising seven countries  (Austria, ­Belgium, France, Germany, Luxembourg, the Netherlands and, as an observer, Switzerland) – is playing a leading role in this context. Based on their past history of regional cooperation, these countries intensify their activities with a view to regional security of supply. This paper aims to contribute to current discussions ­concerning optimal market design within and between the countries of the Pentalateral Energy Forum – and beyond. It reviews the available options for designing the dayahead, intraday, balancing and capacity markets while also taking into account the technical flexibility requirements of future power systems. We hope you find the discussion both stimulating and useful. Yours, Patrick Graichen Executive Director of Agora Energiewende

Key findings at a glance
Resource adequacy is not only about “how much?”, but also about “what kind?”. The power system of the future will require a mix of flexible resources in order to efficiently address resource adequacy. On the supply side, more peak and mid-merit plants and fewer inflexible baseload plants will be needed. In addition, the activation of flexibility potential on the demand side will be crucial.

1. 2. 3. 4.

Resource adequacy should be assessed on a regional level. Regional resource adequacy assessments lower the costs of achieving a reliable power system and mitigate the need for flexibility. For a given resource adequacy standard, the quantity of required resources decreases and the options for balancing the system expand as the market size increases.

A reformed energy-only market is a no-regret option. Making the energy-only market faster and larger is crucial to meeting the flexibility challenge. Further integrating short-term markets across borders as well as vertically linking the different segments (day-ahead, intraday and balancing markets) can help to reduce flexibility requirements. This will also allow markets to better reflect the real-time value of energy and balancing resources.

If resource adequacy is addressed through a capacity mechanism, resource capability rather than capacity needs to be the primary focus. Security of supply will increasingly become a dynamic issue. Future capacity mechanisms will need to consider this by focussing not just on capacity in a quantitative sense but also on operational capabilities. This will minimise price spillover effects of capacity mechanisms to energy-only markets while also fostering greater reliability at lower costs.




1	Introduction 2	 3	 	 	 	 4	 	 	 5	 	 	 	 Managing an Orderly, Low-Cost Transition to 2030 Evolving Solutions for Reliability during the Transition A. Resource adequacy is not, and never has been, only about capacity B. Capacity and capabilities C. Major consequences for “adequate” investment levels Addressing the Investment Challenge – Conventional Generation A. A “no regrets” approach B. The geography of adequacy Addressing the Investment Challenge – Renewable Energy Sources (RES) A. Re-think deployment support policies B. Recent EU policy direction C. Actively manage the impact on supply and demand

5 7 9 9 12 13 15 15 16 19 19 20 20 23 23 23 24 24 25 25 26 27 27 31

6	 Market Structure, Market Rules, Market Governance 	 A. Mitigating the need to increase resource flexibility 		 1. Larger balancing control areas 		 2. Faster markets 	 B. Tapping the potential for demand-side flexibility 	 C. Making the value of resource flexibility more visible 		 1. Fully price all energy market balancing decisions 		 2. Fully price scarcity in balancing services 		 3. Open balancing markets to non-traditional service providers 		 4. Locational Pricing 7	Conclusion


Agora Energiewende | Power Market Operations and System Reliability


IMPULSE | Power Market Operations and System Reliability

1.	Introduction
The European Union remains committed to economy-wide reductions in greenhouse gas emissions of 80-95 percent below 1990 levels by 2050, and in October 2014 they set the interim targets for 2030. A critical component for meeting those long-term goals is a decarbonised electric sector. But the power sector cannot sacrifice reliability in order to achieve those critical environmental objectives. It need not do so: multiple authoritative studies have described power sector decarbonisation pathways based on existing technologies that can meet or exceed current reliability standards. And, as importantly, a secure, reliable transition to a decarbonised power supply can be accomplished at a reasonable cost—indeed, at a cost little more than, and very possibly less than, the cost of “business as usual.” Framing solutions at a regional or even a pan-European level offers EU governments one of the best options for maintaining security of supply at a reasonable cost. As an example, the Pentalateral Energy Forum (PLEF) 1 represents precisely the sort of regional initiative that holds the key to many of the most important opportunities to make this a reality. Market design, market rules, and market operations 2 are at the centre of this process, particularly given the recent trajectory of European Union energy policy and legislation. The intersection of energy and climate policy has spawned a lively discussion about markets and security of supply, a discussion that has tended to focus too narrowly on traditional notions of, and solutions for, the reliability challenge. Rather than begin from the question of which market design is best able to deliver a given quantity of investment,
1	 The Pentalateral Energy Forum consists of six full members (Austria, Belgium, France, Germany, Luxembourg, and the Netherlands) and one observer (Switzerland). The operation of wholesale electricity markets in the EU is split between the system operators (who operate the balancing “markets,” to the extent that they are markets) and the power exchanges (who operate the day-ahead and intra-day energy markets). This introduces certain renewables integration challenges that will be discussed below.

recent studies have illuminated the importance of asking first what kind of investment is best suited to the needs of a low-carbon power system. The least-cost reliability solution will be delivered not by a market that perpetuates investment in “more of the same” but rather shifts investment from a legacy resource mix dominated by inflexible baseload generation to one that can efficiently complement production from a growing share of variable resources. The debate over energy-only vs. energy-plus-capacity markets is important, but to some extent it misses the point. Both models, when implemented well, can ensure reliability, and each carries significant risks. But whether the cost of the reliability solution is reasonable or dear will ultimately depend on whether the resulting mix of market instruments, market governance, and regulation adequately captures the need for an increasingly flexible mix of system resources. A more comprehensive discourse is needed about how best to structure markets and pricing mechanisms (including those for renewable resources) to achieve Europe’s climate, security, and economic goals. This paper is meant to offer a starting point for that discussion.



Agora Energiewende | Power Market Operations and System Reliability


IMPULSE | Power Market Operations and System Reliability

2.	 Managing an Orderly, Low-Cost Transition to 2030
A decarbonised power system must continue to be a reliable power system in order to meet the expectations of European industry and consumers. It should also continue to be an affordable power system for all and one that underpins European competitiveness. The drive toward a decarbonised power system is currently enshrined in policy and legislation but it cannot be taken for granted. If the costs of meeting established reliability expectations were to rise significantly, the entire undertaking could come under intense political pressure. Fortunately there is a growing body of expert analysis targeting these important challenges, a growing consensus about the steps that can be taken—immediately—to deal with them, and more and more real-world experience to justify confidence in their efficacy. While some of the strategies discussed here will involve a front-loading of costs with benefits accruing over time, each of them can be expected to contribute to reducing the overall cost of ensuring reliability in the transition to a low-carbon power system. Under any realistic scenario, decarbonisation will rely on a significant share of renewable energy sources (RES), dependent predominantly on wind and solar radiation, the availability of which are variable and uncontrolled. As variable resources become major players in the energy mix, the cost and complexity of maintaining reliability can vary greatly depending on how the design and operation of the overall portfolio of system resources evolves in response to the changing resource mix. It is now possible to envision a number of low-cost pathways based on different combinations of a set of mutually reinforcing options. One critical step is a deliberate investment shift towards more flexibility in the portfolio of generating resources – less inflexible baseload, more flexible mid-merit units. Other highly beneficial options include: the coordination of unit commitment, economic dispatch, and balancing over larger geographic areas; tighter integration of day-ahead, intra-day, and balancing market operations; incorporation of and frequent intra-day updating of state-of-theart wind and solar forecasts in unit commitment and grid operations; and full participation of dispatchable demandside flexibility (including demand-side energy storage) into balancing services, energy, and (if applicable) capacity markets. Every one of these steps is technically and economically feasible today and, to varying degrees, can be observed in operation in competitive markets in Europe and around the world. In fact, these measures interact with each other in important and positive ways. For instance, operation of balancing markets over wider areas reduces the need for added resource flexibility and vice versa. The question for decision-makers, then, is not whether the transition can be achieved reliably and at a reasonable cost, but rather which policy “levers” they choose to pull and in what combination to create such a pathway.


Agora Energiewende | Power Market Operations and System Reliability


IMPULSE | Power Market Operations and System Reliability

3.	 Evolving Solutions for Reliability during the Transition
The central challenge facing energy policy makers and system operators is that of ensuring the availability of sufficient resources to meet demand for service at virtually all times and at a reasonable cost. This requires investment in both a quantity of resources as well as a least-cost mix of resource capabilities, a function that throughout most of the last century was carried out by central planners. Over the last thirty years, forces of economic and technological change, and environmental and public health policy, have transformed the energy landscape. As a result, the power sector is transitioning to one whose mix of resources and means of operation will differ greatly from that of the last century. 3 This is leading to a re-assessment of how best to ensure a reliable, least-cost power system. In other words, in the 21 st Century power system, the question of ­reliability will remain in the forefront, but the nature of the solution must change as the penetration of variable4 production increases. system security dimension, placing greater emphasis on the ability of the remainder of system resources to complement renewable production efficiently and reliably. This can be seen clearly in Figures 1 and 2 below. Figure 1 shows gross demand on the Danish system in February 2012. Figure 2 shows net demand (gross demand less the contribution from zero-marginal-cost renewables) over the same period, net demand now representing the task facing dispatchable resources. Similarly, Figures 3 and 4 show gross and net demand prevailing in the German power system during December 2013. While the daily spread between peak and minimum load amounted to some 20 GW, the spread between peak and minimum net demand amounted to some 40 GW during the observation period. A more flexible mix of dispatchable resources, capable of shifting operations up and down in synch with the less controllable shifts in variable renewable production, will have far higher asset utilisation rates and require far less redundancy (and therefore far less investment) than a less flexible mix of thermal resources. 5 While resource adequacy has never been only a matter of the quantity of resources, now more than ever the answer to the question ‘how much?’ depends on the answer to the question ‘what type?’

A.	 Resource adequacy is not, and never has been, only about capacity
Service reliability is established in two dimensions: an operational dimension (typically referred to as system security) in which a combination of available resources is deployed to match expected demand in real time at the lowest reasonable cost; and an investment dimension (typically referred to as resource or generation adequacy), in which investment is required to maintain, refresh, expand, and transform the portfolio of resources so that they will continue to be available as needed to meet future demand at the lowest reasonable cost. The growing reliance on variable renewable resources fundamentally transforms the
3	 This is observed throughout Europe. Different specific cases will be discussed here, including cases specifically applicable to the Pentalateral Energy Forum region and their neighbours. The term “variable” used here refers to any generator whose ability to produce electricity – how much and when – is beyond the control of operators to a significant degree. The technical term often used for this is “intermittent.”



See e.g. The Power of Transformation (International Energy Agency, 2014); Renewable Energy Futures (U.S. Dept. of Energy/National Renewable Energy Laboratory, 2012); Accommodating High Levels of Variable Generation (North American Electric Reliability Corp., April 2009); Hinkle, Pedder, Stoffer et al., Contributions of Flexible Energy Resources for Renewable Energy Scenarios (GE Energy, ­8   March 2011); Power Perspectives 2030 (European Climate Foundation, McKinsey & Co., KEMA, Imperial College London, Regulatory Assistance Project, E3G, 2012).



Agora Energiewende | Power Market Operations and System Reliability

[MW] 6,000 5,000 7,000 4,000 3,000





6,000 7,000






01.02.2012 02.02.2012 03.02.2012 04.02.2012 05.02.2012 06.02.2012 07.02.2012 08.02.2012 09.02.2012 10.02.2012 11.02.2012 12.02.2012 13.02.2012 14.02.2012 15.02.2012 16.02.2012 17.02.2012 18.02.2012 19.02.2012 20.02.2012 21.02.2012 22.02.2012 23.02.2012 24.02.2012 25.02.2012 26.02.2012 27.02.2012 28.02.2012














Total electricity demand in Denmark during February 2012







Net demand (total demand minus wind power) in Denmark during February 2012











Figure 2



Figure 1

[MWh] 90,000 60,000 80,000 70,000 60,000 50,000 80,000



10,000 10,000 0








01.12.2013 02.12.2013 03.12.2013 04.12.2013 05.12.2013 06.12.2013 07.12.2013 08.12.2013 09.12.2013 10.12.2013 11.12.2013 12.12.2013 13.12.2013 14.12.2013 15.12.2013 16.12.2013 17.12.2013 18.12.2013 19.12.2013 20.12.2013 21.12.2013 22.12.2013 23.12.2013 24.12.2013 25.12.2013 26.12.2013 27.12.2013 28.12.2013 29.12.2013 30.12.2013






Agora Energiewende (2014), own analysis

Agora Energiewende (2014), own analysis









Total electricity demand in Germany during December 2013








Net demand (total demand minus wind and PV) in Germany during December 2013











IMPULSE | Power Market Operations and System Reliability

Figure 4

Figure 3




Agora Energiewende | Power Market Operations and System Reliability

B.	 Capacity and capabilities
Where generation and supply are provided competitively, decisions about when to invest, how much to invest, and what to invest in are, in principle, left to the market. A debate is underway in many parts of Europe and North America about whether to rely solely on energy markets or to adopt some combination of energy markets and capacity remuneration mechanisms. Where capacity mechanisms are proposed they are often designed as “single product” mechanisms addressing only the quantity of capacity, based primarily on the claim that the need to invest in resource flexibility can be left to the energy market. The problem with this is that the price signals missing from the energy market to drive investment in capacity are the same price signals the energy market is meant to use to remunerate investments in greater resource flexibility. In both cases investors in energy markets are meant to rely on the expression of “scarcity value” in the pricing of energy and balancing services to do so. That is, as demand approaches the limits of supply, the value of energy increases beyond the short-run marginal cost of generation, reflecting a combination of supply scarcity and the value to consumers of uninterrupted service.6 The principles underlying the theory of competitive wholesale electricity markets rely on individual instances of scarcity pricing to express a shortage of responsive resources in the short term and the accumulation of scarcity pricing incidents, when they become sufficiently frequent, to express a shortage of investment in such resources in the long term. However, in many markets scarcity value is suppressed through administrative interventions 7 or

poor market implementation. Where this is the case, the absence of proper scarcity pricing devalues investment in both firm capacity and increased operational flexibility (or “capabilities”). Where such administrative distortions are allowed to persist, some form of supplemental mechanism in support of investment may be appropriate. The European Commission recently provided guidance on this issue 8 (paraphrasing): →→ 1) A properly functioning energy market can deliver  the investment needed to ensure reliable service and should be given the opportunity to do so. →→ 2) In parallel, member state authorities should regu larly conduct an “objective, facts-based” assessment of “generation adequacy 9 […] fully taking account of developments at regional and Union level” as required under the Electricity Security of Supply Directive. →→ 3) If a concern with resource adequacy arises, the  causes should be identified and, where possible, remedied. →→ 4) If, despite compliance with the foregoing, legitimate concerns over resource adequacy remain, a decision can be taken to intervene in support of investment; if so, the form of intervention should be one that “least distorts cross-border trade and the proper functioning of the internal energy market.” In discussing item 2 (resource adequacy assessments), the Commission added that “the rules contained in the Electricity Security of Supply Directive and its transposition and implementation may be insufficient to tackle the challenges of the future in a fully satisfactory way.” This
nation that there is effective competition and the introduction of effective market monitoring are necessary pre-conditions for the removal of these market power mitigation measures. 8	 9	 Generation Adequacy in the internal electricity ­market - guidance on public interventions (5 Nov 2013). The use of the term “generation adequacy” is an unfortunate convention in the European discussion given the growing realisation of dispatchable demand response as a competitive alternative to generation. We have used the more accurate, widely accepted “resource adequacy” throughout this document.


Until we can enable more active involvement of customers in purchasing decisions, the security constraints employed by system operators tend to serve as a proxy for the value of lost load; system operators in the more advanced energy markets are translating these security constraints into real-time scarcity pricing in the energy and balancing markets. In some cases such interventions are appropriate to mitigate the potential for abuse of market power, and where market power continues to be a legitimate concern it will limit the scope for reliance on unconstrained energy market pricing. A determi-



IMPULSE | Power Market Operations and System Reliability

appears to be a reference to the growing need to consider resource attributes beyond simple quantity when assessing “generation adequacy.” To make the point more directly, the North American Electric Reliability Corporation, a leading authority on power system reliability, has stated explicitly that resource adequacy cannot be determined by measuring capacity as an undifferentiated commodity, but rather the adequacy of system resources can only be determined with reference to their operational characteristics. 10 Similarly, the Council of European Energy Regulators has recently issued recommendations for adequacy assessments including the necessity to explicitly consider flexibility, resource needs disaggregated by time period, and demand side flexibility. 11 That this is crucial is borne out repeatedly in power system experience, where the great majority of generation-related system reliability events occur during periods when total de-rated (“firm”) generating capacity on the system comfortably exceeds total demand. 12 In short, one cannot ensure resource adequacy by intervening in the market to support investment in capacity indiscriminately without also addressing the fact that the very same “missing” scarcity value also distorts the rela10	NERC, Balancing and Frequency Control (26 Jan 2011), pages 40-41. 11	CEER, Recommendations for the assessment of electricity generation adequacy, Ref: C13-ESS-33-04, 08 October 2014. 12	 Compare the German situation in February 2012. Though ample capacity was available on the day to meet demand, for several hours security of supply could only be guaranteed by deploying virtually all contracted balancing reserve capacities. Many balancing responsible parties had bought and scheduled less electricity on the day-ahead market than was required to meet their submitted demand schedules, meaning that capacity that could have been committed was not and was therefore unable to respond as needed (BMWi, 2014: Ein Strommarkt für die Energiewende. Diskussionspapier des Bundesministeriums für Wirtschaft und Energie (Grünbuch)). At that time imbalance prices in Germany could still be lower than day-ahead and intraday prices, yielding an incentive for being structurally “short”.

tive value of more flexible capacity. On the contrary, as the share of variable renewable production increases, a single-product capacity mechanism will only reinforce the mismatch between the inflexibility of the current portfolio and what will be needed to ensure least-cost system security going forward.

C. 	 Major consequences for “adequate” ­investment ­levels
Leaving this problem to be addressed later will lead to poor asset utilisation and an unstable investment environment, necessitating additional investment costs for consumers that could have been avoided. The consequences of failing to value resource flexibility fully and in a timely fashion were addressed in the recent IEA study on renewables integration.13 In analysing a system similar to the one that is shared by the PLEF countries, the study highlighted dramatic differences between a system in which the mix of resources shifts in response to the growing role of variable renewables and one that continues to invest in “more of the same.” The results are captured in the graphs in Figure 5 below for a system undergoing a transition from 0 percent to 45 percent variable renewables. The graphs depict a system in which the share of variable RES has grown to 45 percent, under two scenarios. In the “Legacy” scenario the incumbent mix of thermal generation capacity (baseload, mid-merit and peak) has remained essentially unchanged through the transition. Most of the non-renewable energy production comes from inflexible baseload plants, with flexible mid-merit plants producing a much smaller amount. Baseload plants that traditionally saw capacity factors in the 90 percent range are now running only 62 percent of the time, while mid-merit plants that typically ran about 40 percent of the time are now seeing only 11 percent capacity factors, in both cases insufficient to support investment without some form of supplemental assistance. This is just the sort of dire picture often

13	IEA, The Power of Transformation: Wind, Sun and the Economics of Flexible Power Systems (Feb 2014), in particular pages 162-164.


Agora Energiewende | Power Market Operations and System Reliability

Impact of thermal plant mix on investment and plant utilization rates 
Legacy (MWh produced) Annual Capacity (MWh)

Figure 5

Plant utilisation (Legacy) 90 80 70 60 50 40 30 20 10 0 Baseload Mid-merit Peak Idle Used





Transformed (MWh produced) Annual Capacity (MWh)

Plant utilisation (Transformed) 90 80 70 60 50 40 30 20 10 0 Baseload Mid-merit Peak

Baseload RES Mid-merit

Adapted from IEA report in footnote 11

painted of a system with high shares of variable RES. What the IEA analysis and other recent analyses demonstrate is that there is nothing inevitable about this outcome. In the “Transformed” scenario the mix of thermal resource types has been re-balanced in response to the growth in variable resources, with investment shifting to more flexible plant. Slightly more energy is now produced by flexible mid-merit plants than by baseload plants. The remaining baseload plants are back at over 90 percent of capacity whilst the mid-merit plants are back to about 40 percent of capacity. Taking the analysis further, these results imply much different levels of investment. Using conservative assump-

tions for the cost to build these types of plant, 14 the Transformed scenario delivers the same amount of energy to the same reliability standard but with over 40 percent less investment required to do so. In short, a more flexible mix of dispatchable resources, capable of shifting operations up and down in synch with the less controllable shifts in variable renewable production, will have far higher asset utilisation rates and require far less redundancy (and therefore far less investment) than a less flexible mix of thermal resources.

14	 We assumed an average of €3,500/kW new-build costs for baseload, €1,300 for mid-merit and €350/kW for peaking.


IMPULSE | Power Market Operations and System Reliability

4.	 Addressing the Investment Challenge – Conventional Generation
Reliability, then, rests on a foundation of investment in system resources. The adequacy of that investment derives from both an adequate quantity of resources and an adequate mix of resource capabilities (or operational attributes). In competitive electricity markets there are broadly two approaches to delivering an adequate portfolio of resource investment: “energy-only” markets that rely on the outturn prices for energy and balancing services to drive investment, and energy markets to which an administrative mechanism has been appended that is meant to set a value for capacity over some future period of time. Energy-only markets are capable in theory of delivering an economically efficient resource adequacy solution, but there are risks inherent in relying solely on energyonly markets and often, in practice, shortcomings in the implementation of such markets. Implementation problems can include price-distorting market power mitigation measures necessitated by a failure to adequately limit and monitor market power, or a structural disconnect between energy market pricing and supply-demand conditions in the balancing market. Capacity mechanisms can create greater certainty around the delivery of a given level of resource investment, but as administrative mechanisms they carry their own implementation risks. Particularly in the context of the low-carbon transformation, the most important of those risks centre on the ability or the willingness of market administrators to capture sufficiently the multi-faceted nature of the resource adequacy challenge these mechanisms are intended to address. Put simply, given the practical constraints on administrative solutions and the human and political factors that come into play, a capacity mechanism can lock in too much investment, and in the wrong mix of resources, which in turn can lead to yet more overinvestment and needless escalation of the costs of the transition. 15
15	 While beyond the scope of this paper, there is a growing body

Reasonable people can disagree about the wisdom of and need for adopting capacity mechanisms. We do not take a position on that question here. Whichever route is ultimately chosen, the important question remains the same: Is the market driving investment toward a portfolio of resources best suited to provide an acceptable level of reliability at least cost as we decarbonise the power sector? There is now sufficient experience in markets around the world with both approaches to conclude that there are no simple answers to that question.

A.	 A “no regrets” approach
Frustration with the practical and political challenges of perfecting the operation of energy-only markets has led to the adoption of capacity mechanisms in a number of market areas as a way of reducing the risk of under-investment. In virtually every instance, these have initially been designed as “single product” mechanisms that treat capacity as an undifferentiated commodity, both in order to reduce complexity and in the belief that the energy market will direct investment toward the right mix of resources. ­ Reducing unnecessary complexity is laudable when designing administrative mechanisms, especially ones as inherently complex as these, but as we have already discussed there are fundamental flaws in the assumption that an energy market deemed incapable of delivering the right quantity of resource investments can nonetheless be relied upon to ­deliver the right mix of resource investments.
of academic analysis demonstrating that reliability standards traditionally applied in many market regions have little or no objective basis in economic analyses of the value of reliability (See e.g. Brattle Group, ERCOT Investment Incentives and Resource Adequacy, 1 June 2012). As a result resource adequacy assessments have often led to questionable conclusions about the required level of investment and the performance of energy-only markets in delivering it. The resulting tendency to use capacity markets to lock in uneconomic capacity investments is of particular concern in a low-carbon power system.


Agora Energiewende | Power Market Operations and System Reliability

As a consequence, in markets that have accumulated the longest experience with capacity mechanisms authorities are finding it necessary to re-visit this issue. Several have recently begun to introduce differential levels of compensation for different types of resources based on their operational capabilities, even in markets where the penetration of variable resources remains relatively modest. 16 As the share of production from variable resources becomes more significant, with the attendant transformation in the optimal mix of conventional resource capabilities, the shortcomings of single-product capacity mechanisms become more and more apparent. As various markets consider the merits of adding capacity mechanisms to their power markets at the same time as they incorporate more and more variable resources, capacity will become an increasingly differentiated product and capacity mechanisms will have to reflect that in some fashion if they are to continue to serve their intended purpose of ensuring resource adequacy at a reasonable cost. Whilst the relative simplicity of single-product capacity mechanisms is ultimately unsustainable, there will be practical constraints, such as the need for adequate liquidity, on the degree of complexity that can or should be designed into these administrative mechanisms. Design complexity should also be limited by the recognition that the search for precision will quickly overtake the capacity of administrators to determine with any certainty what the right future mix of resources will be. For this reason, the responsibility for shaping investment toward the optimal mix of resource capabilities can never be left entirely to capacity mechanisms. However beneficial a well-designed capacity mechanism might be in reducing uncertainty, it will always be a comparatively crude administrative tool. Whilst it is important to harness such mechanisms in support of the need to shift investment toward more flexible system resources, they cannot fully
16	 For information on ISO New England’s January 2014 proposal see spi-news-iso-ne-submits-proposal-to-strengthen-performance-i.html; for a description of PJM’s August 2014 proposal see reports/20140820-pjm-capacity-performance-proposal.ashx.

replicate the role energy-only markets can and should play in doing so. Once again, in many of the more advanced wholesale electricity markets there is evidence that this is the case. Aggressive reforms designed to improve scarcity pricing in both energy and balancing services markets have recently been introduced in several large competitive markets, not only in energy-only markets, but also more notably in markets firmly committed to the use of capacity mechanisms.17 Just as a desire to ensure a given quantity of investment motivates some to seek the belts-and-braces comfort afforded by a capacity mechanism, the need to ensure a least-cost mix of resources drives the need to improve the effectiveness of energy and balancing market price signals. The “no regrets” option is to re-double efforts to bring the operation of energy markets more into line with their theoretical potential, regardless of what decision might be taken about capacity mechanisms. A current example of this in Europe is in the Great Britain market, where reforms designed to greatly improve the expression of scarcity pricing in intra-day markets have been adopted in parallel with the adoption of a capacity market. Should a capacity mechanism be adopted, the hard-won lessons of real-world experience tell us that it must sooner or later be designed to recognise the difference in value between different types of resources in delivering least-cost reliability. As more variable resources enter the system, the value of discriminating explicitly in favour of operational flexibility will be impossible to ignore.


See fact-sheets/shortage-pricing-fact-sheet.ashx for a brief description of shortage pricing improvements in the PJM market; for a description of recent shortage pricing improvements to the UK’s GB market see electricity/wholesale-market/market-efficiency-reviewand-reform/electricity-balancing-significant-code-review; for details of new scarcity pricing mechanisms introduced by ERCOT in June 2014 see content/wcm/training_courses/107/ordc_workshop.pdf


IMPULSE | Power Market Operations and System Reliability

B.	 The geography of adequacy
When considering how much of a margin in capacity resources over and above demand is enough, the issue of cross-border integration must also be mentioned. It is well established that, all else being equal, the quantity of resources required to meet a given resource adequacy standard is reduced as the size of the market (in terms of both area and demand) in which it is applied is increased. The broad benefits of true cross-border market integration to low-cost decarbonisation will be dealt with below, but it is worth considering here that member-state-by-member-state assessments of resource adequacy will inevitably lead to the need for more investment than if security of supply and resource adequacy were managed over larger geographic footprints. While this has always been true to some extent, recent studies have demonstrated that the benefits of geographic aggregation become overwhelming where there are significant shares of variable resources in the relevant market areas. 18 The greatest benefit accrues where responsibility for system security and resource adequacy is vested in a single balancing area authority across the subject footprint. Where that is not possible for whatever reasons, much of the benefit can be realised through the adoption of market mechanisms designed to integrate unit commitment, dispatch and balancing operations in real time among multiple balancing control areas. This latter option will effectively have been implemented if and when balancing markets across the EU are fully coupled, complementing the progress already made in coupling energy markets.

Early action by groups of member states such as the PLEF countries19 would do much to demonstrate the benefits and accelerate progress in other regions. As the share of variable resources continues to grow, the alternative becomes increasingly unattractive. Continuing the practice of member-state-by-member-state management of resource adequacy will only exacerbate over-investment and raise the cost of reliability over the course of the transition and thereafter.

18	 See e.g. Booz & Co et al., Benefits of an Integrated European Energy Market, September 2013 (; NREL, Combining Balancing Areas’ Variability, 2010 (, NREL, Examination of Potential Benefits of an Energy Imbalance Market in the Western Interconnection, 2013 (; and IEA, The Power of Transformation, 2014 (available at

19	 See the discussion of the IGCC initiative in Section V.A.1 below.


Agora Energiewende | Power Market Operations and System Reliability


IMPULSE | Power Market Operations and System Reliability

5.	 Addressing the Investment Challenge – Renewable Energy Sources (RES)
The challenge – and imperative – of continued investment in RES presents a special case. All prudent power sector decarbonisation pathways rely on sustained commercial deployment of renewable generation at some significant level. Amongst the decarbonisation technology options, renewables are unique in the extent of progress made in recent years in addressing performance and cost challenges. Were deployment of renewables to grind to a halt simply because it is incompatible with current market conditions, the dramatic and hard-won progress of recent years in building a commercial industry, so crucial to the long-term competitiveness of a low-carbon energy system, will have been for naught. The European Commission’s statement on 2030 energy and climate policy affirms this by proposing a (minimum) 27 percent EU target for RES penetration against the 2020 target of 20 percent. Recently, the European Council endorsed this target, which is binding at the EU level. This is estimated to translate to a share of 45-53 percent 20 renewables in the power sector compared to the estimated share of approximately 34 percent derived from the 2020 target. Yet despite the dramatic progress made over the past two decades, investment in renewables – in particular variable renewables – still faces challenges in the current market environment. In many cases the total cost of renewable power is at or near (in some cases even below) the level of the average prices with which they should be expected to compete, but other factors come into play. Variable renewables selling into the energy market tend to earn less than the average market price because prices tend to be lower during periods when renewable energy is most available and higher when renewable energy is less available. In contrast, more dispatchable resources have the opportunity to earn higher prices during periods when variable renewable production is low and demand is high. Known changes in market operation and the overall portfolio of system resources – which could reduce overall price volatility and balancing costs to the benefit of all investment, not just variable renewables – have been slow in coming and will take years to fully materialise, where they are currently being pursued at all. (See Section 6 below for a brief discussion of some of these opportunities.) Therefore as the “booster stages” of deployment support near the end of their role for many well-developed renewable technologies, but with some external hurdles yet to be overcome, there are still unanswered questions as to what forms the “intermediate stages” of support will or should take. Two key areas to explore are examined below.

A.	 Re-think deployment support policies
Because of the variable nature of most of the primary renewable sources, they will not benefit significantly from capacity-based interventions, at least not in any of the forms currently employed or under serious consideration. As for support specifically for renewables technology investments, most recent proposals are still essentially subsidy mechanisms designed to bring market signals more strongly into play, including auctions and “feed-in premiums” as well as variations on quota-based tradable certificate programs. It may be better to look at moving more clearly beyond a subsidy paradigm. As the costs of many key RES technologies have declined the need for actual subsidies has declined as well and, with the expectation of higher carbon prices and yet further cost reductions, should continue to decline. “Priority dispatch” was intended to protect variable RES from curtailment for convenience, but with RES now well established in most markets, its very low short-

20	 EC (2014). Impact Assessment Accompanying the Communication A policy framework for climate and energy in the period from 2020 up to 2030. The RES-E range mentioned refers to the scenarios with a GHG reduction of 40%.


Agora Energiewende | Power Market Operations and System Reliability

run marginal cost means it is likely to be dispatched virtually whenever it is available. Despite arguments for and against priority dispatch on both sides of the RES debate it is not entirely clear what incremental impact (in either direction) priority dispatch has at this stage. Improved market operations and more responsive demand should also reduce the need for insulation of variable RES from curtailment and balancing risk, which will be a critical factor supporting the trend in policy towards increasing RES exposure to balancing responsibility. But as described above, the challenge of earning disproportionately low energy prices will continue for some time into the future, essentially setting the bar for RES investment higher today and through the medium term than it is likely to be once the system is better able to absorb the swings in variable renewable production. During this transition period the support required for RES investment will increasingly have less to do with subsidy paradigms and more to do with alternative revenue mechanisms that enable variable RES to realise close-to-average market prices. There is creative thinking yet to be done on this front.

Recent policy direction in the EU is also favouring restrictions on the payment of production premiums during pricing periods when energy prices are negative, in order to avoid offering additional incentives to produce at times when there is no demand for additional energy. To the extent this becomes established policy it will have implications for the design of renewables support mechanisms, including perhaps the adoption of some non-energy component of support or dynamic shifting of support premiums toward periods of energy shortage. As noted above, EU and member state support policy is also moving toward increasing exposure of RES to balancing responsibility and, by implication, away from the insulated status afforded to RES under measures such as feed-in tariffs and priority dispatch.

C.	 Actively manage the impact on supply and demand
One additional challenge facing investment in all resource categories, including RES as they become more exposed to market price levels, is the persistent overhang of surplus production capacity across most of Europe. RES support policies, particularly feed-in tariff schemes, have until now tended to promote investment in new renewable capacity without regard to the resulting impact on the balance between the demand for productive capacity and the supply of productive capacity and with no obvious plan or policy to deal with any surplus or stranded capacity that might be created as a result. Also the investment programs of many incumbent European utilities initiated prior to the recession appear to have been based on the rather remarkable assumption that new conventional generation would be needed to meet nearly all of the then-projected growth in demand. The current substantial oversupply of capacity, the result of the combined effects of the deep recession, successful efficiency measures, ill-timed investment in new fossil-fired plants, and aggressive renewable support mechanisms, is one of the primary causes of the instability

B.	 Recent EU policy direction
Some concrete new direction has also been emerging from Brussels regarding future support for renewables investment. Whilst the future direction of EU policy in this area is still in formative stages, a few specific dimensions of that policy direction have emerged. There has long been some ambiguity around the state aid status of certain renewables support policies. Recent action by the Commission provides guidance on how such policies may be treated in the future. It would appear that the use of auctions for the setting of premiums for renewable energy production would constitute the clearest litmus test for whether support constitutes state aid. Related to this, and in accord with Competition law and the objectives of the Internal Energy Market, the Commission will be pursuing efforts to increase cross-border cooperation in the design and implementation of renewable investment support mechanisms.


IMPULSE | Power Market Operations and System Reliability

currently plaguing the European power sector, though it is certainly not the only cause. 21 This should have been an opportunity. As policy-driven investment in RES has continued to grow at a rapid pace in many markets, faster than the growth in demand, there should have been a plan to remove the resulting surplus in baseload generating capacity from the market in an orderly and equitable fashion. This would have created a more stable marketplace for those resources remaining on the system and facilitated the transition to a more appropriate mix of resources. Instead the onset of oversupplied markets seems to have come as a surprise, and the belated response from some governments has been to consider supporting redundant generation financially through fixed capacity-related payments.22 This may create challenges for continued support for RES investment. Artificially depressed energy prices inflate the perceived gap between average wholesale power market prices and the prices paid for RES supply, generating unwanted political blowback against RES support policies. The failure to assist redundant baseload generation in exiting the market also impedes the critical transformation of the conventional generation portfolio into a smaller, more nimble fleet dominated by flexible mid-merit generation.

The future policy framework around renewable deployment, whatever form it takes, needs to do a better job of dealing with this issue. In particular, member state governments and the EU need to consider how best to deliberately and selectively remove surplus baseload thermal generating capacity from the market.

21	 Among other causes one could point to is the fragmented and incomplete implementation of the Third Energy Package across member states; ongoing uncertainty around the future of the ETS; and the influx of cheap coal into the European market driven at least in part by the boom in natural gas production in North America. 22	 Some individual member states, including some PLEF countries, have suggested that on a purely national basis they may be facing generation shortages in the near future. Whether or not this is the case, the direction of ENTSO-E reliability activities, consistent with EU energy policy and with what is widely recognised as best practice, is to assess adequacy on a regional basis reflecting the scope of resources actually available to ensure reliability.


Agora Energiewende | Power Market Operations and System Reliability


IMPULSE | Power Market Operations and System Reliability

6.	 Market Structure, Market Rules, Market Governance
Section 4.A above addresses the importance of improving the effectiveness of wholesale electric energy markets regardless of whether a capacity mechanism is appended to the market. If and when introduced, capacity mechanisms need to be “smart” – they need to recognise that flaws in the implementation of energy markets that risk underinvestment in capacity will also risk underinvestment in operational flexibility – but they cannot fully substitute for improvements in the functioning of energy markets. This section will look at practical options for making energy markets more effective. The basic failing of many energy markets is often referred to as “missing money,” referring to income required to support needed investment that is not available in the energy market. Typical causes of “missing money” are various forms of price suppression, either through administrative interventions (such as price caps), lock-in of an oversupply of production capacity (for example via capacity markets with excessive reserve margin requirements) or poorly designed market rules (such as balancing services mechanisms that do not reflect the real-time value of the service). This “missing money” problem, where it exists, affects both the quantity and the capabilities of capacity resources. There are other obstacles to the effectiveness of energy markets that are not so much failings as they are missed opportunities. These missed opportunities include the failure to exploit non-traditional resources such as dispatchable demand-response that can compete very favourably with generation, as well as advances in market administration (discussed below) that can more effectively absorb the impacts of variable production. Measures are available to governments, regulators and system operators to redress these obstacles. Some are designed to restore the expression of scarcity value and, at the same time, to make it more reliable, less volatile and less extreme. In so doing, they provide a more attractive basis for investment and give more robust expression to the value of flexibility. Other measures are designed to activate the potential for demand as a flexible resource alongside flexible supply-side resources, critical to enabling consumers to mitigate the impact of more dynamic scarcity pricing. And some measures are designed to reduce the overall need for more flexible resources in the first place. We discuss some of the more prominent examples of each type of measure here.

A.	 Mitigating the need to increase resource flexibility
1. Larger balancing control areas Increasing the size of balancing control areas reduces the need for more resource flexibility. Larger control areas are beneficial in any case, but where the share of variable production is significant, the benefit can be especially large. In most cases the size and the frequency of swings between resource surplus and resource scarcity can be reduced dramatically. 23 The benefit derives from three main sources: →→ (a) increasing the size of the control area reduces the impact of any single system event and affords the control area authority a more diverse portfolio of resource options with which to maintain system balance; →→ (b) demand across large geographic areas is generally not well correlated and thus the natural variability of demand cancels out to some extent; and →→ (c) the variability of variable renewable resources is generally not well correlated over large geographic areas, reducing the variability of supply. The most direct way to access these benefits, and the one that maximises the benefits available, is simply to consolidate multiple contiguous control areas under a single balancing authority. The regional independent transmission system operator model found in parts of North America
23	 NREL, Energy Imbalance Markets (2012) at


Agora Energiewende | Power Market Operations and System Reliability

and Australia are good examples. Where full integration of area control under a single regional authority is not feasible for whatever reason, much of the benefit can be accessed through virtual consolidation. The integration of the balancing markets in Europe as proposed under the Target Model would be a major step toward the real-time consolidation of balancing markets across national borders. An important step toward this objective is the International Grid Control Cooperation. The IGCC is an initiative led by the four German TSOs 24 to integrate markets for certain types of reserves across multiple control areas, “… to exploit synergies [as] in a single fictitious control area, without giving up the proven structure of control areas. It also enables a flexible response in case of network bottlenecks.” 25 Another example of this is the emerging Energy Imbalance Market in the Western Interconnect of North America. 26 2. Faster markets Yet another way that energy and balancing services markets can be structured to reduce the need for additional flexibility is to make them “faster.” Fast energy markets are those in which the dispatching of system resources takes place as close to real time as possible, and where dispatch schedules are updated at multiple points throughout the day based on updated weather forecasts. Frequent rescheduling at shorter market intervals reduces the range of uncertainty about real-time outcomes between dispatch schedules and thereby reduces the need for system reserves. Resources can be dispatched in smaller increments during periods when system net demand is ramping up or down to a significant extent. Uncertainty is further
24	 In addition to the four German control areas the IGCC currently includes Denmark, Switzerland, Austria, the Netherlands, Belgium and the Czech Republic. 25	 IGCC, 2014. Information on grid control cooperation and international development. Version: 10/04/2014 at https://www. 26	 The EIM is an initiative to integrate balancing markets across nearly 40 contiguous control areas in the Western US and Canada without going to full control area consolidation. It currently includes California ISO, Pacificorp, Sierra Pacific and Nevada Power.

reduced by frequent and more sophisticated weather forecasts. In the most advanced energy markets the system is dispatched at five-minute intervals based on state-ofthe-art weather forecasts that are at most a few hours old, whereas in many traditional markets the dispatching of market resources takes place only once an hour and is based on day-ahead weather forecasts. The need for more flexible system resources acting in fast-response mode can be reduced dramatically by adopting faster market processes. Centrally dispatched energy markets, common in many places outside of Europe, are particularly well suited to adopting state-of-the-art fast market processes. Comparably fast markets are more difficult to implement in the decentralised dispatch model favoured in Europe, but it is still possible to do so. One of the key challenges in decentralised markets is to ensure that the quality and timeliness of information flows between the power exchanges and the system operator is sufficient to allow grid constraints and changes in trading positions to be resolved in the shortest possible amount of time.

B.	 Tapping the potential for demand-side flexibility
A number of measures can effectively reduce the need for increased generator flexibility by increasing the opportunity for demand to respond in real time to uncontrolled swings in supply. The keys to accessing this potential are to offer dynamic pricing (preferably real-time pricing) to those wishing to participate and to remove barriers to participation by demand in day-ahead and intra-day energy markets. In many energy markets it has long been possible for large industrial customers to participate directly, though often in very rudimentary ways. But pushing this direct market participation model to the much larger and more diverse pool of residential and small commercial customers is challenging on a number of levels, including the fact that such customers will in most cases have neither the capacity nor the willingness to take action themselves in real time to respond in any reliable or enforceable fashion.


IMPULSE | Power Market Operations and System Reliability

To access this much larger potential it is essential to open energy market access to demand aggregation, in which consumption by a number of individual consumers, or more effectively by individual loads at consumer premises, is managed under contract to a single service provider in return for whatever form of compensation the aggregator and consumers agree. The aggregator then uses the demand under contract to sell the equivalent of energy production into the market. (Aggregators can and do also use demand response to supply various balancing services, a source of flexibility that will be discussed below, as well as the capacity value of demand response, which can often compete successfully with the cost of an equivalent amount of generating capacity.27 ) Aggregation can be carried out by any qualified commercial entity, including competitive electricity suppliers. Experience demonstrates the clear benefits of ensuring that the opportunity to manage consumers’ energy services be fully open to competition from both traditional and nontraditional enterprises, meaning that market power must be strictly regulated. One valuable step in this direction, particularly where actual or virtual vertical integration is still a market reality, would be to separate the roles of electricity supplier and Balancing Responsible Party. Demand aggregation is a separate service, entered into at the customer’s discretion, in which the service provider essentially steps into the customer’s shoes and manages the interface between primary energy supply and the provision of various energy services. There is no good reason why suppliers must play this role – though where there is effective competition they should have the right to do so – nor is there a reason they should have to retain responsibility for the balancing issues that may arise as a result. In assuming the role of managing energy services, the aggregator can and should also assume balancing responsibility for the supply procured on behalf of that customer. The current bundling of the roles creates an unnecessary barrier to market entry by placing service providers (i.e., demand aggregators) in a position of either being in direct competi27	 See Hurley, Peterson and Whited Demand Response as a Power System Resource, Synapse and RAP 2013.

tion with incumbent suppliers or being required to negotiate balancing exposure remedies with such suppliers.28 Another form of responsive demand can be accessed by making combined heat and power facilities more flexible in response to the needs of the power system. This typically involves the incorporation of thermal energy storage systems so that the provision of heating (or cooling) when demanded by customers can be physically decoupled from the operation of the CHP plant for the production of electricity. This same application of distributed energy storage is technically feasible and can be applied inexpensively directly to thermal appliances at customer premises, yet another source of demand flexibility that could be dispatched to match the needs of the power system.

C.	 Making the value of resource flexibility more visible
1. Fully price all energy market balancing decisions It is a common misconception that energy markets are meant to set prices based on the short-run production cost of the marginal generating resource. In fact, they are meant to set prices based on the short-run value of whatever marginal action is required to balance supply with demand. A major cause of “missing money” is that many of the actions taken to maintain the balance between demand and supply, in particular actions with the highest marginal value, take place outside of the energy market. These are actions taken by the system operator (or by market stakeholders at the request of the system operator) within the system balancing mechanism, the market operating regime that begins once control over market operations is turned over to the system operator (in Europe, typically one hour before real time). System operators typically deploy these resources at a long-term contract cost that has

28	 For a discussion of the roles of suppliers, BRPs, customers and grid operators in the German context see RAP 2013, Nachfragesteuerung im deutschen Stromsystem – die unerschlossene Ressource für die Versorgungssicherheit.


Agora Energiewende | Power Market Operations and System Reliability

no relationship to the instantaneous value of the resource when it’s needed, or they require system resources to provide them at no charge, as a supplement to the resources deployed in the energy market. In this way the true shortterm value of providing critical balancing services such as reserves at times when resources are constrained are often obscured. This in turn suppresses the relative value of supplying energy rather than providing reserves. Energy markets would be far more efficient at signalling the need for investment, and in particular the need for investment in greater resource flexibility, if the true cost of balancing the system at times of scarcity were reflected in clearing energy prices. This is accomplished by co-optimising energy and balancing markets, which forces the value of energy and the value of balancing services to converge more dynamically in real time. Some markets have already begun to move in this direction. In North America, the PJM, NYISO and ERCOT markets have all adopted measures that will allow energy market clearing prices to be set by an expanded set of balancing actions including deployment of demand-side resources. In Europe, the UK’s Ofgem has recently adopted similar measures as part of their Electricity Balancing Significant Code Review. If a sufficiently large and diverse portfolio of end-use loads can be activated (including via demand aggregation) and factored into the setting of energy market clearing prices the demand curve will become more sloped than vertical. In other words, consumers will begin to acquire the ability to express, directly or indirectly, the actual value of realtime access to electricity to power the various energy services they consume. This is an important step in making scarcity pricing more real, less volatile, less extreme, and more predictable and therefore a more robust investment signal. It is also an important step in making consumers less vulnerable to the abuse of generator market power, in turn making it possible for governments and regulators to be more amenable to the relaxation of energy market price caps. The ability to relax price caps is a critical step in enhancing the efficiency with which energy markets support investment, lowering consumer energy prices overall.

For the purposes of this paper, the important point is that it will make much more visible the value of investments in more flexible resources capable of efficiently complementing production from variable renewables. 2. Fully price scarcity in balancing services Given historical limitations on consumers’ ability to express more fully and accurately the true, more granular value of reliability, that value has traditionally been expressed by proxy via the security constraints adopted by system operators to keep the system in balance after gate-closure. Put simply, system operators determine for each scheduling interval how much and what type of fastresponding resources must be kept in reserve in order to reduce the likelihood of failure to a level that reflects the accepted standard for service reliability. As described above, in most cases the market value of these reserves is set by a long-term contract price that bears no relationship to their value at the times they are deployed, or they are available to the system operator at no charge and their value is set accordingly. In reality, at times when demand is approaching the limits of supply and more and more resources are scheduled to produce energy, the supply of these balancing services available to the system operator, whilst still sufficient to meet demand, can fall below the level required to meet security constraints. In those moments the value of additional supply can rise dramatically. If this value is not made visible by the balancing mechanism it deprives the market of one of its most important and reliable means of expressing scarcity value. In addition to distorting balancing service price signals directly, the failure to price the demand for balancing services correctly deprives the energy market of the information it needs to gauge the opportunity cost of selling energy rather than selling reserve services, thereby distorting price signals in the energy market. This situation can be remedied and several markets have moved or are moving to do so. Markets including PJM, ISO New England and the GB market in the UK have adopted different versions of an administrative remedy sometimes referred to as an Operating Reserve Demand Curve. The ERCOT market in Texas is in the process of adopting such a


IMPULSE | Power Market Operations and System Reliability

mechanism. A detailed explanation is beyond the scope of this paper, but in short, this mechanism establishes a value for balancing reserves based on the same “value of lost load” methodology that lies at the heart of the resource adequacy process. The value runs along on a continuum that rises from a variable cost based value when reserves are plentiful, up to the maximum value of energy in the market once all reserves are exhausted. As with other measures described here, this mechanism is intended not just to restore scarcity pricing to the energy market, but also to do so in a way that reveals scarcity value in a continuous manner whenever scarcity begins to emerge. In so doing these measures reflect more realistically how resource scarcity manifests itself and reveal incremental market signals for responses to scarcity long before demand reaches the limit of available resources. 3. Open balancing markets to non-traditional service  providers Improving the real-time price signals for energy and balancing services will be of only limited value, indeed it may simply increase costs, if the market for those services is effectively closed to the most flexible resource options available. Many balancing services mechanisms lock in the resources they expect to need well in advance of real time under procurement processes that, not surprisingly, reflect the characteristics of the conventional supply-side resources from which they’ve traditionally been obtained. In so doing they exclude resources that don’t fit that particular template. An example is the fact that many system operators conduct solicitations for primary, secondary or tertiary reserves for terms of months or even years at a time. Many resources, such as various demand response opportunities, capable of providing balancing reserves as well as or better than traditional generators and at a lower price, are excluded because they are seasonal rather than annual resources. There are numerous similar examples of what is often unintentional discrimination. Aggregators of such non-traditional sources are excluded from markets, in some cases explicitly so. In other cases procurement processes effectively exclude variable renewable resources

from providing the service by imposing unnecessary requirements on suppliers. Many markets, for example some of the large organised markets in North America, have successfully adopted procurement for certain reserves categories that replicate the rhythm of the daily energy markets, obtaining the reserves they need from the most effective and economic resources available at the time. As the share of variable resources grows on the system, regulators and market operators can avoid unnecessarily high integration costs by combining proper price signals for the value of balancing services with open access for the most cost-effective balancing service providers available. 4. Locational Pricing As noted earlier, the diversity of loads and supplies that comes with actually or virtually increasing the size of a balancing area will, all else being equal, reduce the quantity of resources required to meet a given resource adequacy standard and also reduce the amount of flexibility needed to integrate a specified amount of variable resources. This does not mean that the energy clearing price across the enlarged market is, or even should be, uniform. Limitations in the ability of the system to move lowest marginal-cost power throughout the system at any and all times – that is, “congestion” – will necessarily result in differences in the costs to serve load at different times and places. Those cost differences (the “costs of congestion”), if visible, can have very real and important effects on decisions to invest – on both the types of resources to be deployed and where to site them. As production from variable renewables increases, these differentiated local impacts will increase as well. As renewables become more exposed to market conditions, the benefits of addressing locational issues head-on will outweigh any short-term disadvantages that might arise from doing so. Locational price differences in and of themselves are of limited value in driving investment due to their transitory nature, but making them visible makes more tangible and urgent the case for proceeding with the investments needed to resolve them.


Agora Energiewende | Power Market Operations and System Reliability

Generally speaking, locational pricing is the cost of most efficiently supplying an increment of load at a particular place, while satisfying all operational constraints. 29 Put another way, it is the means by which least-cost system operation (i.e., merit order dispatch) is achieved when the bulk power grid is congested. As competitive wholesale electricity markets evolved over the past several decades, two general approaches to locational pricing emerged. The first, nodal pricing (also referred to as locational marginal pricing or LMP), calls for the calculation of prices at every “node” on the transmission grid. A node denotes a place where supply (generation or an import) is injected onto the grid or where demand is withdrawn from it. Depending on the size of the grid, there can be many hundreds or even thousands of nodes. Nodal pricing is in effect in Argentina, Chile, New Zealand, Russia, Singapore, and several regions of the US (New England, New York, PJM, and Texas). 30 The second is zonal pricing or market splitting, which reduces the number of locational prices to be determined by aggregating nodes into larger areas (zones) of uniform pricing. Ideally, the zones are configured so as to minimise intra-zone congestion and thereby minimise the congestion costs that are hidden in the zonal price. In much of Europe today, each country is effectively its own zone. Countries that have several zones include Australia (each state is a zone), Denmark, Norway, and Sweden. 31 In principle, the concept of locational prices is straightforward. In practice, their calculation is complicated. They consist of at least two elements, the cost of energy and the cost of congestion; the eastern US markets recognise a third (and very real) cost, that of line losses. The system operator calculates prices first for the day-ahead market. They derive from the clearing of supply and demand at

every node or zone, subject to meeting reliability criteria. In a system without constraints (and ignoring line losses), the prices will equalise across all nodes or zones. The market clearing price will equal the bid price of the marginal unit (the last unit to clear in the market), since its price represents the marginal cost to serve the next increment of demand.32 Where there are constraints, however, the locational prices will diverge. In the constrained areas, the prices will rise because higher cost resources will be called on to serve the local load. That increase in price is the “congestion component” of the locational price. The prices in the unconstrained areas will again equalise, but they may in fact be lower than they would have otherwise been, if there is now an excess of generation caused by the inability to serve across the constraints (i.e., the congestion component is negative).33 It is on the basis of these day-ahead prices that the financial obligations of sellers and buyers are set and the next day’s dispatch determined. Buyers pay the locational prices at their nodes or zones and sellers are paid the prices at theirs. If there is no congestion, the total payments of the buyers will equal the total receipts of the sellers (including the costs of line losses). If there is congestion, the buyers’ payments will exceed the amounts paid to sellers; the difference is the cost of congestion and is typically used to compensate those who purchased hedging instruments (sometimes called “financial transmission rights”) to manage the risk of exposure to these costs. In the US, locational prices are also calculated in real time, typically at five-minute intervals, to re-optimise dispatch and to determine the value of incremental supply for balancing and other services (including flexibility). These

29	 Eugene Litvinov, Locational Marginal Pricing, ISO-NE, WEM-301 (2011), at 70-71. 30	 Pär Holmberg and Ewa Lazarczyk, Congestion management in electricity networks: Nodal, zonal and discriminatory pricing, University of Cambridge, Electric Policy Research Group, CWPE 1219 & EPRG 1209, April 2012, at 4. 31	 Id.

32	 Strictly speaking, the LMP at a location is defined as a change in the total cost of production associated with meeting an increment of load at that location. 33	 Note that this description has ignored line losses. In fact, in unconstrained systems, LMPs will vary from node to node, to the extent that line losses vary on different parts of the transmission system. New England, New York, and the PJM systems adjust their LMPs to reflect line losses. Texas does not (which means that the costs of line losses are shared equally among all nodes).


IMPULSE | Power Market Operations and System Reliability

real-time prices are also used to determine, for the purpose of financial settlements, whether committed supply is performing as required. This has to do with protecting against the exercise of market power (which resources in constrained areas often possess) and making sure that prices reflect the true costs of congestion. Locational pricing is seen to have two sets of benefits. In the short term, it improves economic efficiency by revealing the cost of congestion and thus ensuring that, given the physical limitations of the network, demand is met at the lowest total operating cost. In the longer term, it reveals the value of solving congestion problems and thereby lowers obstacles to efficient new investment—generation, transmission, demand response, and energy efficiency—in the constrained areas. In this way, overall economic efficiency is enhanced. And, insofar as the locational prices reflect the value of lost load, investment in flexibility will also be encouraged. This in turn transforms the debate over grid expansion, and the alternatives to it, from one about cause to one about effect, because it exposes where congestion can be found and how much one is willing to pay to get rid of it. There might still be arguments about the best ways to solve the problems, but there will be little question about their existence and cost.


Agora Energiewende | Power Market Operations and System Reliability


IMPULSE | Power Market Operations and System Reliability

7.	Conclusion
We set out to describe how Europe’s wholesale electricity markets can be adapted to match the needs of a decarbonising power system to the prevailing expectations for power system reliability. Our argument is that a secure, reliable transition to a decarbonised power supply can be accomplished at a reasonable cost. Indeed, preserving reliability at a reasonable cost through the transition is essential to sustaining strong political support for the project. Market rules, market design and market operations are at the centre of this process. We looked at some of the impacts of high shares of variable production on the nature of the resource adequacy challenge and on how markets determine the quantity and quality of investment that will be required to meet it. We addressed in particular the need to expand the investment problem set to include not just the quantity of capacity but also the operational capabilities needed to deliver a least-cost reliability solution as the power system is decarbonised. We looked at the rationale for adopting capacity markets and discussed why they will not deliver resource adequacy at least cost if capacity is valued as an undifferentiated commodity. We showed that even “smart” capacity mechanisms would need to rely on the improved operation of energy markets. As such, improving energy markets constitutes a “no regrets” measure. We considered the special case of continued investment in renewable capacity. Given the commitment to a decarbonised power sector, there are sound security of supply and economic reasons why investment in the deployment of key renewable technologies must continue, but the nature of support for that investment, as with investment in conventional resources, is currently the subject of considerable discussion. We looked at the evolving nature of the challenges facing investment in renewables, particularly variable renewables, and offered some suggestions for what should drive renewable support policy going forward, including the need to be more deliberate and selective in dealing with oversupply of baseload generation as the share of variable RES grows. Looking beyond the various debates about energy market interventions we returned to the underlying energy market itself. We reviewed a range of practical options available to governments, regulators, and system operators to better match the structure of the market to the needs of a decarbonised power sector, and to restore the functioning of energy and balancing services markets as close as possible to their full potential. A decarbonised power sector is at the heart of delivering on the EU’s climate policy goals. The challenges in delivering on this objective in a secure and affordable manner are real. So are the many options available to overcome those challenges. Making good choices begins with sound fundamentals and a holistic approach. We have attempted to provide some of that foundation here as a context within which to evaluate a number of key choices facing policy makers and regulators.


Publications of Agora Energiewende
In Englisch 12 Insights on Germany’s Energiewende A radically simplified EEG 2.0 in 2014
Concept for a two-step process 2014-2017 An Discussion Paper Exploring Key Challenges for the Power Sector

Benefits of Energy Efficiency on the German Power Sector
Final report of a study conducted by Prognos AG and IAEW

Comparing Electricity Prices for Industry
An elusive task – illustrated by the German case

Comparing the Cost of Low-Carbon Technologies: What is the Cheapest Option? Cost Optimal Expansion of Renewables in Germany

An analysis of new wind, solar, nuclear and CCS based on current support schemes in the UK and Germany

A comparison of strategies for expanding wind and solar power in Germany

Load Management as a Way of Covering Peak Demand in Southern Germany The German Energiewende and its Climate Paradox In German 12 Thesen zur Energiewende

Final report on a study conducted by Fraunhofer ISI and Forschungsgesellschaft für Energiewirtschaft

An Analysis of Power Sector Trends for Renewables, Coal, Gas, Nuclear Power and CO 2 Emissions, 2010-2030

Ein Diskussionsbeitrag zu den wichtigsten Herausforderungen im Strommarkt (Lang- und Kurzfassung)

Auf dem Weg zum neuen Strommarktdesign: Kann der Energy-only-Markt 2.0 auf ­Kapazitätsmechanismen verzichten? Ausschreibungen für Erneuerbare Energien
Welche Fragen sind zu prüfen?

Dokumentation der Stellungnahmen der Referenten für die Diskussionsveranstaltung am 17. September 2014

Das deutsche Energiewende-Paradox. Ursachen und Herausforderungen

Eine Analyse des Stromsystems von 2010 bis 2030 in Bezug auf Erneuerbare Energien, Kohle, Gas, Kernkraft und CO 2-Emissionen

Der Spotmarktpreis als Index für eine dynamische EEG-Umlage

Vorschlag für eine verbesserte Integration Erneuerbarer Energien durch Flexibilisierung der Nachfrage

Effekte regional verteilter sowie Ost-/West-ausgerichteter Solarstromanlagen Ein radikal vereinfachtes EEG 2.0 und ein umfassender Marktdesign-Prozess
Konzept für ein zweistufiges Verfahren 2014-2017

Eine Abschätzung systemischer und ökonomischer Effekte verschiedener Zubauszenarien der Photovoltaik


Publications of Agora Energiewende
Ein robustes Stromnetz für die Zukunft

Methodenvorschlag zur Planung – Kurzfassung einer Studie von BET Aachen

Entwicklung der Windenergie in Deutschland Erneuerbare-Energien-Gesetz 3.0

Eine Beschreibung von aktuellen und zukünftigen Trends und Charakteristika der Einspeisung von Windenergieanlagen

Konzept einer strukturellen EEG-Reform auf dem Weg zu einem neuen Strommarktdesign

Kapazitätsmarkt oder Strategische Reserve: Was ist der nächste Schritt?
Eine Übersicht über die in der Diskussion befindlichen Modelle zur Gewährleistung der Versorgungssicherheit in Deutschland ­

Klimafreundliche Stromerzeugung: Welche Option ist am günstigsten?

Stromerzeugungskosten neuer Wind- und Solaranalagen sowie neuer CCS- und Kernkraftwerke auf Basis der Förderkonditionen in Großbritannien und Deutschland

Kostenoptimaler Ausbau der Erneuerbaren Energien in Deutschland

Ein Vergleich möglicher Strategien für den Ausbau von Wind- und Solarenergie in Deutschland bis 2033

Lastmanagement als Beitrag zur Deckung des Spitzenlastbedarfs in Süddeutschland
Endbericht einer Studie von Fraunhofer ISI und der Forschungsgesellschaft für Energiewirtschaft

Negative Strompreise: Ursache und Wirkungen

Eine Analyse der aktuellen Entwicklungen – und ein Vorschlag für ein Flexibilitätsgesetz

Positive Effekte von Energieeffizienz auf den deutschen Stromsektor

Endbericht einer Studie von der Prognos AG und dem Institut für Elektrische Anlagen und Energiewirtschaft (IAEW)

Power-to-Heat zur Integration von ansonsten abgeregeltem Strom aus Erneuerbaren Energien
Handlungsvorschläge basierend auf einer Analyse von Potenzialen und energiewirtschaftlichen Effekten

Reform des Konzessionsabgabenrechts
Gutachten vorgelegt von Raue LLP

Stromspeicher für die Energiewende

Untersuchung zum Bedarf an neuen Stromspeichern in Deutschland für den Erzeugungsausgleich, Systemdienstleistungen und im Verteilnetz

Stromverteilnetze für die Energiewende

Empfehlungen des Stakeholder-Dialogs Verteilnetze für die Bundesrepublik – Schlussbericht

Vergütung von Windenergieanlagen an Land über das Referenzertragsmodell

Vorschlag für eine Weiterentwicklung des Referenzertragsmodells und eine Anpassung der Vergütungshöhe

Vorschlag für eine Reform der Umlage-Mechanismen im Erneuerbare Energien Gesetz (EEG)
Studie des Öko-Instituts im Auftrag von Agora Energiewende

Zusammenhang von Strombörsen und Endkundenpreisen
Studie von Energy Brainpool All publications are available on our website:



How do we accomplish the Energiewende? Which legislation, initiatives, and measures do we need to make it a success? Agora Energiewende helps to prepare the ground to ensure that Germany sets the course towards a fully decarbonised p ­ ower sector. As a think-&-do-tank, we work with key stakeholders to enhance the knowledge basis and facilitate c ­ onvergence of views.

Agora Energiewende Rosenstrasse 2 | 10178 Berlin | Germany T +49. (0)30. 284 49 01-00 F +49. (0)30. 284 49 01-29
Agora Energiewende is a joint initiative of the Mercator Foundation and the European Climate Foundation.
Top of page

Note to user

Dear user,

In response to current developments in the web technology used by the Goobi viewer, the software no longer supports your browser.

Please use one of the following browsers to display this page correctly.

Thank you.