Irrational Agents and the Power Grid

Irrational Agents and the Power Grid
September 26, 2018
Sean Meyn
Department of Electrical and Computer Engineering — University of Florida
Based in part on joint research with
In-Koo Cho, Matias Negrete-Pincetic, Gui Wang, Uday Shanbhag, Robert Moye
Prabir Barooah, Ana Buˇsi´c, Yue Chen, Neil Cammardella, Joel Mathias & Matthew Kiener
Thanks to to our sponsors: NSF, DOE, ARPA-E

Irrational Agents and the Power Grid
1 Goals of our Research
2 Genius of Control
3 Genius of the Market
4 Eﬃcient Outcome
5 Conclusions
6 References

Goals
Creation of Virtual Energy Storage

Balancing Reserves for my GRID
Comfort for me and my owner
Goals
Happy Grid, Loads and Customers

80
100
120
140
80
100
120
140
0
MWMW
-10
0
10
Tracking Typical Load Response
temp(F)temp(F)
rt≡0Noreg:|rt|≤10MW
LoadOnLoadOn
BPA Reference:
Power Deviation
rt
Genius of Control

Genius of Control Happy grid
What does the Balancing Authority Want?
Ancillary services to match supply and demand:
• Balancing Reserves
Sun
0
1
-1
GW
AGC/Secondary control +
1 / 22

• Peak shaving
1 / 22

• Peak shaving
Modified Prices with Demand Dispatch
1 / 22

• Peak shaving
• Ramp service
GW
Forecasted peak: 29,549Forecasted peak: 29,549
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
17
22
27
32
1 / 22

• Peak shaving
• Ramp service
GW
Forecasted peak: 29,549Forecasted peak: 29,549
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
17
22
27
32
Modified Load with Demand Dispatch
1 / 22

• Peak shaving
• Ramp service
• Contingency support
59.915 Hz
60.010 Hz Modified Load with Demand Dispatch
1 / 22

Genius of Control Happy consumers
What do the Consumers Want?
Rational agent wants a hot shower
http://www.onsetcomp.com/learning/application_stories/multi-channel-data-loggers-improve-forensic-analysis-complex-domestic-hot-water-complaints
Θ(t)
G(t)
Ambient
Temperature
Inlet Water
Temperature
3 kW
Water heater trajectories
Θ(t): Temperature
G(t): Power consumption
2 / 22

Θ(t)
G(t)
Ambient
Temperature
Inlet Water
Temperature
3 kW
Mismatch:
G(t) (of interest to BA) Spike Train
Θ(t) (of interest to home) Smooth
Θ(t): Temperature
2 / 22

Θ(t)
G(t)
Ambient
Temperature
Inlet Water
Temperature
3 kW
Mismatch:
G(t) (of interest to BA) Spike Train
Θ(t) (of interest to home) Smooth
Θ(t): Temperature
Mismatch = gift to the control engineer
2 / 22

Genius of Control Distributed control
Tracking with 100,000 Water Heaters
80
100
120
140
0 5 10 15 200 5 10 15 20
80
100
120
140
80
100
120
140
0
50
100
-50
0
50
MWMWMW
-10
0
10
Nominal power consumption
Tracking
Tracking
Typical Load Response
temp(F)temp(F)temp(F)
rt≡0Noreg:|rt|≤40MW|rt|≤10MW
LoadOnLoadOnLoadOn
(hrs)t (hrs)t
BPA Reference:
Power Deviation
rt
Tracking results with 100,000 water heaters, and the behavior of a single
water heater in three cases, distinguished by the reference signal [12]a
Theoretical power capacity is approx 8 MW (with no flow)
a
Buˇsić & M. – summary of five year program
3 / 22

Genius of Control Distributed control
Tracking with 100,000 Water Heaters
Energy Limits – Ramps and Contingencies
-8
-6
-4
-2
0
2
4
6
8
Powerdeviation(MW)
-6
-5
-4
-3
-2
-1
0
1
2
0 5 10 15 20
hours
ζ
ζ
Every water heater OFF
ReferencePower Deviation
Powerdeviation(MW)
-8
-6
-4
-2
0
2
4
6
8
-6
-4
-2
0
2
0 5 10 15 20
hours
ζ
ζ
Tracking a sawtooth wave with 100,000 water heaters:
Average power consumption = 8MW
Quality of Service = temperature limits
By design, QoS violation is not possible
4 / 22

Purchase Price $/MWh
Previous week
Spinning reserve prices PX prices $/MWh
100
150
0
50
200
250
10
20
30
40
50
60
70
Texas: February2,2011
California: July2000Illinois:July1998
Ontario: November,2005
0
1000
2000
3000
4000
5000
Mon Tues Weds Thurs Fri Mon Tues WedsWeds Thurs Fri Sat Sun
Tues Weds Thurs
Time3 6 9 12 15 18 213 6 9 12 15 18 213 6 9 12 15 18 21
Demand in MW Last Updated 11:00 AM Predispatch 1975.11 Dispatch 19683.5
Hourly Ontario Energy Price $/MWh Last Updated 11:00 AM Predispatch 72.79 Dispatch 90.82
2000
21000
18000
15000
1500
1000
500
0
ForecastPricesForecastDemand
5am 10am 3pm 8pm
−500
0
1000
2000
3000
$/MWh
Average price
is usually $30
$/MWh
Genius of the Market

Genius of the Market Real time markets
RTM Model
The dream
"The active participation of final demand in the wholesale market is essential to managing
the greater intermittency of renewable resources and in limiting the ability of wholesale
electricity suppliers to exercise unilateral market power. A demand that is able to reduce its
consumption in real-time in response to higher prices limits the ability of suppliers
to exercise unilateral market power in a formal wholesale market such as the California ISO"
(http://www.stanford.edu/group/fwolak/cgi-
bin/sites/default/files/files/little_hoover_testimony_wolak_sept_2011.pdf) -F. Wolak
Low-cost information and communications technologies
and advanced metering
enable more cost-reflective prices and charges for
electricity services that can finally animate the“demand
side”of the power system and align myriad decisions
"Virtually all economists agree that the outcome [of the California crisis] was exacerbated by the inability of the demand side of the
market to respond to real or artificial supply shortages. This realization prompted my research stream on." real-time electricity pricing.”
- S. Borenstein
5 / 22

RTM Model
The dream
- S. Borenstein
reduce its
higher prices
reduce
consumption in real-time
5 / 22

RTM Model
The dream
- S. Borenstein
"Virtuallyalleconomistsagreethattheoutcome[oftheCaliforniacrisis]wasexacerbatedbytheinabilityofthedemandsideofthe
markettorespondtorealorartificialsupplyshortages.Thisrealizationpromptedmyresearchstreamon." real-timeelectricitypricing.”
-S.Borenstein
demand side
respond
real-time electricity pricing
5 / 22

Electricity Markets Today
Two coupled markets
Day-ahead market (DAM):
Cleared one day prior to the production and delivery of energy: The ISO
generates a schedule of generators to supply speciﬁc levels of power for
each hour over the next 24 hour period.
6 / 22

Electricity Markets Today
Two coupled markets
Day-ahead market (DAM):
Cleared one day prior to the production and delivery of energy: The ISO
generates a schedule of generators to supply speciﬁc levels of power for
each hour over the next 24 hour period.
Real-time market (RTM):
As supply and demand are not perfectly predictable, the RTM plays the
role of ﬁne-tuning this resource allocation process
RTM is the focus here
6 / 22

Genius of the Market Competitive equilibrium
RTM Model
Dynamic model for reserves
Simplest model of Cho & M:
R(t) = Available power − Demand = G(t) − D(t)
D(t) = Actual demand − Forecast
G(t): Deviation in on-line capacity from day-ahead market
7 / 22

RTM Model
Dynamic model
Generation cannot increase instantaneously:
For all t ≥ 0 and t > t,
G(t ) − G(t)
t − t
≤ ζ
7 / 22

RTM Model
Dynamic model
Generation cannot increase instantaneously:
For all t ≥ 0 and t > t,
G(t ) − G(t)
t − t
≤ ζ
Later work: lower bounds on generation, as well as network constraints [9]
7 / 22

Market Analysis
Second Welfare Theorem
Self-Interested Agents
max
GS
E e−γt
WS(t) dt max
GD
E e−γt
WD(t) dt
8 / 22

Market Analysis
max
GS
E e−γt
WS(t) dt max
GD
E e−γt
WD(t) dt
Welfare functions deﬁned with a nominal price function P(t):
WS(t) = P(t)GS(t) − cS(GS(t))
WD(t) = wD(GD(t)) − P(t)GD(t)
8 / 22

Market Analysis
max
GS
E e−γt
WS(t) dt max
GD
E e−γt
WD(t) dt
Key component of equilibrium theory:
Perfect competition
The price of power P(t) in the RTM is assumed to be exogenous;
prices do not depend on the decisions of the market agents.
8 / 22

Market Analysis
max
GS
E e−γt
WS(t) dt max
GD
E e−γt
WD(t) dt
Perfect competition
The price of power P(t) in the RTM is assumed to be exogenous;
prices do not depend on the decisions of the market agents.
“price-taking assumption”
8 / 22

Market Analysis
Eﬃcient Equilibrium
max
GS
E e−γt
WS(t) dt max
GD
E e−γt
WD(t) dt
The market is eﬃcient if G∗
S
= G∗
D
Perfect competition
The price of power P(t) in the RTM is assumed to be exogenous (it
does not depend on the decisions of the market agents).
“price-taking assumption”
9 / 22

Market Analysis
Social Planner’s Problem
An eﬃcient equilibrium is optimal for the social planner:
max K(G) = E e−γt
WS(t) + WD(t) dt
s.t. GS(t) = GD(t) for all t
Welfare functions deﬁned with a nominal price function P(t)
10 / 22

Market Analysis
Social Planner’s Problem
An eﬃcient equilibrium is optimal for the social planner:
WS(t) + WD(t) dt
s.t. GS(t) = GD(t) for all t
Welfare functions deﬁned with a nominal price function P(t)
Price is irrelevant when GS(t) = GD(t):
10 / 22

Market Analysis
Second Welfare Theorem ⇐⇒ Lagrangian Decomposition
WS(t) + WD(t)
+ λ(t) GS(t) − GD(t) dt
11 / 22

Market Analysis
max K(G) = max
GS
E e−γt
WS(t) + λ(t)GS(t) dt
+ max
GD
E e−γt
WD(t) − λ(t)GD(t) dt
11 / 22

Market Analysis
max K(G) = max
GS
E e−γt
+ max
GD
E e−γt
Assume: Social planner’s problem has a solution,
and there is no duality gap.
11 / 22

Market Analysis
max K(G) = max
GS
E e−γt
+ max
GD
E e−γt
Assume: Social planner’s problem has a solution,
and there is no duality gap.
Then
P∗(t) = P(t) + λ∗(t) provides an eﬃcient equilibrium.
Price is marginal value: P∗
(t) = wD(G∗
D(t))
11 / 22

Market Analysis
(t) = wD(G∗
D(t))
12 / 22

Market Analysis
(t) = wD(G∗
D(t))
Average price is average marginal cost:
E e−γt
P∗
(t) dt = E e−γt
cS(G∗
D(t)) dt
Economist Nirvana!
12 / 22

Market Analysis
(t) = wD(G∗
D(t))
Average price is average marginal cost:
E e−γt
P∗
(t) dt = E e−γt
cS(G∗
D(t)) dt
Economist Nirvana!
With transmission constraints, equilibrium prices are nodal: they
can be negative, or above marginal value [8, 9]
See bibliography: [8, 3, 2, 9]
12 / 22

Real-world price dynamics
Marginal value? Obviously not marginal cost
Purchase Price $/MWh
Previous week
Spinning reserve prices PX prices $/MWh
100
150
0
50
200
250
10
20
30
40
50
60
70
Texas: February2,2011
California: July2000Illinois:July1998
Ontario: November,2005
0
1000
2000
3000
4000
5000
Mon Tues Weds Thurs Fri Mon Tues WedsWeds Thurs Fri Sat Sun
Tues Weds Thurs
Time3 6 9 12 15 18 213 6 9 12 15 18 213 6 9 12 15 18 21
Demand in MW Last Updated 11:00 AM Predispatch 1975.11 Dispatch 19683.5
Hourly Ontario Energy Price $/MWh Last Updated 11:00 AM Predispatch 72.79 Dispatch 90.82
2000
21000
18000
15000
1500
1000
500
0
ForecastPricesForecastDemand
5am 10am 3pm 8pm
−500
0
1000
2000
3000
$/MWh
Average price
is usually $30
$/MWh
13 / 22

Genius of the Market Rational agents?
But where are the rational agents?
An imperfect but reasonable RTM model:
14 / 22

What about this?
14 / 22

What about this?
Irrational Agents
14 / 22

What about this?
What is the “value of power” to consumers?
Irrational Agents
Power is NOT the commodity of interest!
14 / 22

12 181 6 24
26
22
18
14
10
6
2
24
20
16
12
8
4
0
Renewables
Thermal
Imports
Nuclear
Hydro
hours
Generation(GW)
Generation at CAISO March 4, 2018
Eﬃcient Outcome

Eﬃcient Outcome Price signals
Control and Price Signals
TotalPower(GW)
0
5
10
15
20
hrs
P nominal
P desired
P delivered
1 2 3 4 5
Example: Aggregator has contracts with consumers
7 million residential ACs
700,000 water heaters
700,000 commercial water heaters
17 million refrigerators
All the pools in California
Promises strict bounds on QoS for each customer
15 / 22

TotalPower(GW)
Temperature,cycling,energy
0
5
10
15
20
hrs
P nominal
P desired
P delivered
1 2 3 4 5
hrs1 2 3 4 5
ACs
Small WHs
Commercial WHs
Refrigerators
Pools
QoS
15 / 22

TotalPower(GW)
0
5
10
15
20
hrs
P nominal
P desired
P delivered
1 2 3 4 5
Balancing authority desires power reduction over 2 hours
Sends PRICE SIGNAL: 10% increase
Aggregator optimizes subject to QoS constraints
15 / 22

TotalPower(GW)Power(GW)
0
5
10
15
20
hrs
0
2
4
6
P nominal
P desired
P delivered
ACs FWHs
SWHs Fridges
Pools
1 2 3 4 5
Price event: 10% increase
15 / 22

0
5
10
15
20
hrs
0
2
4
6
P nominal
P desired
P delivered
ACs FWHs
SWHs Fridges
Pools
1 2 3 4 5
15 / 22

0
5
10
15
20
hrs
0
2
4
6
P nominal
P desired
P delivered
ACs FWHs
SWHs Fridges
Pools
1 2 3 4 5
No QoS promises to Balancing Authority!
15 / 22

Problem with Price Signals
Automatic
Generation Control
Real-time
Market
Day Ahead
Market
Desired behavior
Desired behavior
Predictions
GRID
Millions of Residential
and commercial electric loads
Generators with their own
local control loops (DROOP)
Distributed generation,
possibly not grid-friendly
Dynamics of transmission
Brains
Brawn
Disturbance Voltage,
Frequency,
Phase
Conjecture: We could create a price signal P∗(t) that would induce the
behavior we want.
16 / 22

Automatic
Generation Control
Real-time
Market
Day Ahead
Market
Desired behavior
Desired behavior
Predictions
GRID
Brains
Brawn
Frequency,
Phase
behavior we want.
The price is necessarily non-causal and device-dependent:
functional of the nonlinear dynamics of each collection of loads
16 / 22

Automatic
Generation Control
Real-time
Market
Day Ahead
Market
Desired behavior
Desired behavior
Predictions
GRID
Brains
Brawn
Frequency,
Phase
behavior we want.
The price is necessarily non-causal and device-dependent:
functional of the nonlinear dynamics of each collection of loads
This intuition can be justiﬁed based on
Lagrangian decomposition / solution to Euler-Lagrange equations.
16 / 22

Efficient Outcome Distributed control
Efficient Outcome
Efficient Outcome 2018
Example: Aggregator has contract with consumers, and with BA.
Promises QoS constraints to all parties
Aggregator’s optimization problem: Demand Dispatch
17 / 22

Eﬃcient Outcome
Example: Aggregator has contract with consumers, and with BA.
Promises QoS constraints to all parties
Aggregator’s optimization problem: Demand Dispatch
Formulate as a convex program [11]
17 / 22

Eﬃcient Outcome
12
14
16
18
20
22
24
26
-5
0
5
10
2 4 6 8 10 12 14 16 18 20 22 24
-2
-1
0
1
2
DemandDispatch(GW)
GWGW
Net Load Generation (without help from loads)
hrs
ACs fWHs sWHs Fridges Pools
17 / 22

Eﬃcient Outcome
12
14
16
18
20
22
24
26
-5
0
5
10
2 4 6 8 10 12 14 16 18 20 22 24
-2
-1
0
1
2
DemandDispatch(GW)
GWGW
Net Load Generation
Demand Dispatch
hrs
17 / 22

Eﬃcient Outcome
2 4 6 8 10 12 14 16 18 20 22 24
GWGW
Net Load Generation
Demand Dispatch
hrs
12
14
16
18
20
22
24
26
-3
-2
-1
0
1
2
3
-5
0
5
10
DemandDispatch(GW)
17 / 22

Eﬃcient Outcome
GW
Net Load Generation
Demand Dispatch
DemandDispatch(GW)
and
Irrigation
Cow cooling
Plug-in electric vehicles
Commercial HVAC
17 / 22

Power GridControl WaterPump
Batteries
Coal
GasTurbine
BP
BP
BP C
BP
BP
Voltage
Frequency
Phase
HC
Σ
−
Actuator feedback loop
A
LOAD
Conclusions

Conclusions
Conclusion
Rational agent in Bordeaux wants a shower
Θ(t)
G(t)
Ambient
Temperature
Inlet Water
Temperature
3 kW
Typical water heater trajectories
Θ(t): Temperature
Not-so rational agent: max
G
T
0
U(G(t)) − p(t)G(t) dt
18 / 22

Conclusions
Conclusion
Θ(t)
G(t)
Ambient
Temperature
Inlet Water
Temperature
3 kW
Θ(t): Temperature
G
T
0
Markets are awesome
Control is cool
18 / 22

Conclusions
Conclusion
Θ(t)
G(t)
Ambient
Temperature
Inlet Water
Temperature
3 kW
Θ(t): Temperature
G
T
0
Markets are awesome
Control is cool
Real time prices are irrational
18 / 22

Conclusions Thank You
Thank You
19 / 22

References
Control Techniques
FOR
Complex Networks
Sean Meyn
Pre-publication version for on-line viewing. Monograph available for purchase at your favorite retailer
More information available at http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521884419
References
20 / 22

References
Selected References I
[1] M. Chen, I.-K. Cho, and S. Meyn. Reliability by design in a distributed power transmission
network. Automatica, 42:1267–1281, August 2006. (invited).
[2] I. K. Cho and S. Meyn. Dynamics of ancillary service prices in power distribution systems.
In Proc. of the 42nd IEEE CDC, volume 3, 2003.
[3] I.-K. Cho and S. P. Meyn. Eﬃciency and marginal cost pricing in dynamic competitive
markets with friction. Theoretical Economics, 5(2), 2010.
[4] S. Robinson. Math model explains volatile prices in power markets. SIAM News, Oct.
2005.
[5] R. Moye and S. Meyn. Redesign of U.S. electricity capacity markets. In IMA volume on
the control of energy markets and grids. Springer, 2018.
[6] R. Moye and S. Meyn. The use of marginal energy costs in the design of U.S. capacity
markets. In Proc. 51st Annual Hawaii International Conference on System Sciences
(HICSS), 2018.
[7] R. Moye and S. Meyn. Scarcity pricing in U.S. wholesale electricity markets. In Proc. 52nd
Annual Hawaii International Conference on System Sciences (HICSS) (submitted), 2018.
[8] M. Negrete-Pincetic. Intelligence by design in an entropic power grid. PhD thesis, UIUC,
Urbana, IL, 2012.
21 / 22

References
Selected References II
[9] G. Wang, M. Negrete-Pincetic, A. Kowli, E. Shafieepoorfard, S. Meyn, and U. V.
Shanbhag. Dynamic competitive equilibria in electricity markets. In A. Chakrabortty and
M. Illic, editors, Control and Optimization Methods for Electric Smart Grids, pages
35–62. Springer, 2012.
[10] R. A¨ıd, D. Possama¨ı, and N. Touzi. Electricity demand response and optimal contract
theory. SIAM News, 2017.
[11] N. Cammardella, J. Mathias, M. Kiener, A. Buˇsić, and S. Meyn. Balancing California’s
grid without batteries. IEEE Conf. on Decision and Control (submitted), Dec 2018.
[12] Y. Chen, U. Hashmi, J. Mathias, A. Buˇsić, and S. Meyn. Distributed Control Design for
Balancing the Grid Using Flexible Loads. In IMA volume on the control of energy markets
and grids Springer, 2018.
[13] J. Mathias, A. Buˇsić, and S. Meyn. Demand dispatch with heterogeneous intelligent
loads. In Proc. 50th Annual Hawaii International Conference on System Sciences, 2017.
[14] S. Meyn, P. Barooah, A. Buˇsić, Y. Chen, and J. Ehren. Ancillary service to the grid using
intelligent deferrable loads. IEEE Trans. Automat. Control, 60(11):2847–2862, Nov 2015.
[15] Coase, R.H. The marginal cost controversy. Econometrica 13(51), 169–182 (1946)
22 / 22

Irrational Agents and the Power Grid

More Related Content

What's hot (20)

Similar to Irrational Agents and the Power Grid (20)

More from Sean Meyn (20)

Recently uploaded (20)

Irrational Agents and the Power Grid