Producing and validating small area estimates
of household electricity demand

Dr Ben Anderson
Sustainable Energy Research Centre
University of Southampton
@dataknut

IMA 2013
Canberra, December 12th 2013
Small Area Estimates of Electricity Consumption

Contents
 What & Why

 How?
 Results
– Overall consumption
– Consumption inequalities

 Conclusions & future Directions
@dataknut

2
Small Area Estimates of Electricity Consumption

Contents
 What & Why

 How?

?

 Results
– Overall consumption
– Consumption inequalities

 Conclusions & future Directions
@dataknut

3
Small Area Estimates of Electricity Consumption

Digression: Geography
 Southampton (UK)

@dataknut

4
Small Area Estimates of Electricity Consumption

Digression: What’s a small area?
 In this case…
– English Lower Layer
Super Output Areas
– Census 2001 LSOAs
– c. 630 households each
– 148 in Southampton City

@dataknut

5
Small Area Estimates of Electricity Consumption

What & Why
 Basically we want something for nothing
– Small area estimates of energy demand
– Without a bespoke energy census

 Why?
– Infrastructure planning
– Energy efficiency intervention analysis
– Politics!
@dataknut

6
Small Area Estimates of Electricity Consumption

The problem:
 Small area summaries exist
 But they are aggregates
– Or averages

 And we want a micro-level model
– To micro-simulate change…

@dataknut

7
Small Area Estimates of Electricity Consumption

What can we do?
 A bespoke energy census
– ££££££££££££££££££££

@dataknut

8
Small Area Estimates of Electricity Consumption

What can we do?
 A bespoke energy census
– ££££££££££££££££££££

 A large sample energy survey covering all
LSOAs
– ££££££££££

@dataknut

9
Small Area Estimates of Electricity Consumption

What can we do?
 A bespoke energy census
– ££££££££££££££££££££

 A large sample energy survey covering all LSOAs
– ££££££££££

 Small Area Estimation
– Take existing area level data
– Take (ideally) an existing large n survey
– Combine £
@dataknut

10
Small Area Estimates of Electricity Consumption

Small Area Estimation
 Econometric approaches

Income, income deprivation,
income inequality
smoking prevalence,
obesity,
consumption expenditure,
CO2, water…

– Well known
– Multi-level Models
– Usually requires census microdata for anything other
Innovation Network:
than means
“Evaluating and improving small area
estimation methods”

 Re-weighting (and other) approaches
– Increasingly http://eprints.ncrm.ac.uk/3210/
well known
– 'Spatial microsimulation'
– Does not require census microdata
@dataknut

11
Small Area Estimates of Electricity Consumption

Contents
 What & Why

Estimation

 How?
 Results
– Overall consumption
– Consumption inequalities

 Conclusions & Future Directions
@dataknut

12
Small Area Estimates of Electricity Consumption

Data
 Data
– Living Costs and Food Survey 2008-2010
Consumption proxies (reported energy expenditure)

– Census 2001 (2011)

 Projection
– Projected ‘surveys’
– Projected ‘census’

 So far…
@dataknut

13
Small Area Estimates of Electricity Consumption

Conceptually…
LSOA census ‘constraint’ tables

Survey data cases
If Region = 1

LSOA 1.1
(Region1)

If Region = 2

@dataknut

Weights

LSOA 2.1
(Region2)

Iterative proportional fitting

Ballas et al (2005)
14
Small Area Estimates of Electricity Consumption

Key First Job:
 Choose your constraints
Census data

Survey data

 You may have little choice

@dataknut

16
Small Area Estimates of Electricity Consumption

Key First Job:
 The constraints
– Selected by stepwise regression
Expenditure
Number of persons

Employment Status

Accommodation type

Number of earners

Age of HRP

Age of HRP

Employment Status

Most important

Share of expenditure

Composition

Number of rooms
Number of children
Least important
R sq
@dataknut

Ethnicity (non-white)
0.136

0.01
19
Small Area Estimates of Electricity Consumption

IPF…
 Well known!
 Deming and Stephan 1940
– Fienberg 1970; Wong 1992

 A way of iteratively adjusting statistical tables
– To give known margins (row/column totals)

 In this case
– Create weights for each case so LSOA totals ‘fit’
constraints
– Weighting ‘down’
@dataknut

20
Small Area Estimates of Electricity Consumption

Internal Validation methods
 Use of constraints to re-create the Census
tables
 Difference = Absolute Error
–
–

Total Absolute Error (TAE) = sum of all
errors
Standardised AE = TAE/(n persons x n
constraint categories)

 Smith et al:
–

SAE of less than 20% and ideally less
than 10%

–

Consumption

Mean SAE

p90

Ethnicity

2.18%

3.05%

Number of children

0.11%

0.22%

Number of rooms

0.05%

0.10%

Employment status
(HRP)

0.88%

1.22%

Age (HRP)

0.34%

0.75%

Tenure

0.07%

0.14%

Accomodation type

0.21%

0.51%

Number of persons

0.00%

0.00%

in 90% of the areas is desirable.

@dataknut

21
Small Area Estimates of Electricity Consumption

Preliminary results: Electricity
 Mean weekly £

 Modelled
 Census 2001
 LC&F Survey 20082010

@dataknut

22
Small Area Estimates of Electricity Consumption

Validation: Electricity
 Mean weekly £

 Observed @LSOA
 DECC 2010
 Spearman: 0.317

@dataknut

23
Small Area Estimates of Electricity Consumption

Preliminary results: Electricity
 Total weekly £

 Modelled
 Census 2001
 LC&F Survey 2008-2010

@dataknut

24
Small Area Estimates of Electricity Consumption

Validation: Electricity
 Total weekly £

 Observed @LSOA
 DECC 2010
 Spearman: 0.509

@dataknut

25
Small Area Estimates of Electricity Consumption

What is causing the error?
 Heating!
–

@dataknut

2011 data

26
Small Area Estimates of Electricity Consumption

What is causing the error?
 Heating!
–

2011 data

 Housing growth
 Combined

@dataknut

27
Small Area Estimates of Electricity Consumption

Consumption inequality
 Area level gini

 R = -0.413
–

@dataknut

(p < 0.001)

28
Small Area Estimates of Electricity Consumption

Consumption inequality
 Area level gini

 R = 0.463
–

@dataknut

(p < 0.001)

29
Small Area Estimates of Electricity Consumption

My big worry
 Data quality

@dataknut

30
Small Area Estimates of Electricity Consumption

Contents
 What & Why

 How?
 Results
– Overall consumption
– Consumption inequalities

 Conclusions & Future Directions
@dataknut

31
Small Area Estimates of Electricity Consumption

Conclusions
 Outliers and errors are informative
 Reported consumption data
– Could be dangerous

 Census 2011 central heating
– Critical new constraint

@dataknut

32
Small Area Estimates of Electricity Consumption

Future directions
 Update for 2011 data
 Census projection 1981 -> 2021
 Use measured energy consumption
 Contact:
– b.anderson@soton.ac.uk

@dataknut

33

More Related Content

PPTX
Small Area Estimation as a tool for thinking about spatial variation in energ...
PDF
MaddieDrake_ ResearchPoster
PDF
Prediction of the Power Output of Solar Cells Using Neural Networks: Solar Ce...
PDF
Modeling and Simulation of Grid Connected PV System
PPTX
questionnaire on electricity conservation
PPTX
Problem with the Increase of Electric Consumption
PDF
Wind and Solar Power Can Supply 80 Percent of Electricity Needs in the U.S.
PDF
Design of Hybrid Solar-Wind Power System for a Coaster Area in Lagos State, S...
Small Area Estimation as a tool for thinking about spatial variation in energ...
MaddieDrake_ ResearchPoster
Prediction of the Power Output of Solar Cells Using Neural Networks: Solar Ce...
Modeling and Simulation of Grid Connected PV System
questionnaire on electricity conservation
Problem with the Increase of Electric Consumption
Wind and Solar Power Can Supply 80 Percent of Electricity Needs in the U.S.
Design of Hybrid Solar-Wind Power System for a Coaster Area in Lagos State, S...

Similar to Producing and validating small area estimates of household electricity demand (20)

PPTX
Small Area Estimation as a tool for thinking about temporal and spatial varia...
PPTX
Technology Option and Cost of Increasing Electricity Access in Taraba, Nigeria
PPTX
energy consumption calculator
PPTX
Hunting for (energy) demanding practices using big & medium sized data
PPTX
Making the South African TIMES (SATIM) Energy System Model ‘Water Smart’, by ...
PDF
DANIEL_OWEN_POSTER
PPTX
WESCML: A Data Standard for Exchanging Water and Energy Supply and Consumptio...
PPTX
Energy Efficiency Workshop - Powering Sydney
PPTX
The Distribution of Domestic Energy-Tech in Great Britain: 2008 – 2011
PPTX
Visualize & analyze energy data
PPTX
Survey on use of datamining
PDF
Do ‘eco’ attitudes & behaviours explain the uptake of domestic energy product...
PDF
Ph d thesis_seminar_on the design of energy efficient wireless access_sibelto...
PPTX
MREA Energy Fair 2015 - Midwest Solar Market Potential
PPSX
Load Forecasting II
PDF
Design needs of a solar pv system for winton – queensland, australia
PDF
Control of an Autonomous Hybrid Microgrid as Energy Source for a Small Rural ...
PDF
Control of an Autonomous Hybrid Microgrid as Energy Source for a Small Rural ...
PPTX
Monitoring solar with internet of things
PPTX
Michigan IPL: Solar Powering Michigan
Small Area Estimation as a tool for thinking about temporal and spatial varia...
Technology Option and Cost of Increasing Electricity Access in Taraba, Nigeria
energy consumption calculator
Hunting for (energy) demanding practices using big & medium sized data
Making the South African TIMES (SATIM) Energy System Model ‘Water Smart’, by ...
DANIEL_OWEN_POSTER
WESCML: A Data Standard for Exchanging Water and Energy Supply and Consumptio...
Energy Efficiency Workshop - Powering Sydney
The Distribution of Domestic Energy-Tech in Great Britain: 2008 – 2011
Visualize & analyze energy data
Survey on use of datamining
Do ‘eco’ attitudes & behaviours explain the uptake of domestic energy product...
Ph d thesis_seminar_on the design of energy efficient wireless access_sibelto...
MREA Energy Fair 2015 - Midwest Solar Market Potential
Load Forecasting II
Design needs of a solar pv system for winton – queensland, australia
Control of an Autonomous Hybrid Microgrid as Energy Source for a Small Rural ...
Control of an Autonomous Hybrid Microgrid as Energy Source for a Small Rural ...
Monitoring solar with internet of things
Michigan IPL: Solar Powering Michigan
Ad

More from Ben Anderson (20)

PPTX
Using Time Use Data To Trace 'Energy Practices' Through Time
PPTX
Modeling Water Demand in Droughts (in England & Wales)
PPTX
Modeling Water Demand in Droughts (in England & Wales)
PDF
A Social Practices-based Microsimulation Model for Estimating Domestic Hot Wa...
PPTX
SAVE: Lightning Talk
PDF
SAVE: A large scale randomised control trial approach to testing domestic ele...
PPTX
Electricity consumption and household characteristics: Implications for censu...
PPT
The Time and Timing of UK Domestic Energy DEMAND
PPT
Modeling Electricity Demand in Time and Space
PPT
Developing insight from commercial data to support #Census2022
PPTX
PRACTICE HUNTING: Time Use Surveys for a quantification of practices distribu...
PPT
Census2022: Extracting value from domestic consumption data in a post­census era
PPTX
The Rhythms and Components of ‘Peak Energy’ Demand
PDF
Tracking Social Practices with Big(ish) data
PDF
Modes of commuting, workplace choice and energy use at home
PPT
Patterns of Water: Thinking about diversity, demand and consumption
PPT
Census 2022: An overview
PPT
On the social scientific value of transactional data
PPT
Practice hunting with British telephone call records
ODP
Practices by Proxy: Climate, Consumption and Water (and troubles with data)
Using Time Use Data To Trace 'Energy Practices' Through Time
Modeling Water Demand in Droughts (in England & Wales)
Modeling Water Demand in Droughts (in England & Wales)
A Social Practices-based Microsimulation Model for Estimating Domestic Hot Wa...
SAVE: Lightning Talk
SAVE: A large scale randomised control trial approach to testing domestic ele...
Electricity consumption and household characteristics: Implications for censu...
The Time and Timing of UK Domestic Energy DEMAND
Modeling Electricity Demand in Time and Space
Developing insight from commercial data to support #Census2022
PRACTICE HUNTING: Time Use Surveys for a quantification of practices distribu...
Census2022: Extracting value from domestic consumption data in a post­census era
The Rhythms and Components of ‘Peak Energy’ Demand
Tracking Social Practices with Big(ish) data
Modes of commuting, workplace choice and energy use at home
Patterns of Water: Thinking about diversity, demand and consumption
Census 2022: An overview
On the social scientific value of transactional data
Practice hunting with British telephone call records
Practices by Proxy: Climate, Consumption and Water (and troubles with data)
Ad

Recently uploaded (20)

PPTX
operations management : demand supply ch
PPTX
Transportation in Logistics management.pptx
PDF
Susan Semmelmann: Enriching the Lives of others through her Talents and Bless...
DOCX
Center Enamel Powering Innovation and Resilience in the Italian Chemical Indu...
DOCX
Emerging Dubai Investment Opportunities in 2025.docx
DOCX
Center Enamel A Strategic Partner for the Modernization of Georgia's Chemical...
PDF
Communication Tactics in Legal Contexts: Historical Case Studies (www.kiu.ac...
PDF
Vinod Bhatt - Most Inspiring Supply Chain Leader in India 2025.pdf
PPTX
IITM - FINAL Option - 01 - 12.08.25.pptx
PPT
Retail Management and Retail Markets and Concepts
DOCX
80 DE ÔN VÀO 10 NĂM 2023vhkkkjjhhhhjjjj
PDF
533158074-Saudi-Arabia-Companies-List-Contact.pdf
PPTX
BUSINESS CYCLE_INFLATION AND UNEMPLOYMENT.pptx
PPTX
33ABJFA6556B1ZP researhchzfrsdfasdfsadzd
PDF
Immigration Law and Communication: Challenges and Solutions {www.kiu.ac.ug)
PDF
Chapter 2 - AI chatbots and prompt engineering.pdf
PPTX
TRAINNING, DEVELOPMENT AND APPRAISAL.pptx
PPTX
CTG - Business Update 2Q2025 & 6M2025.pptx
PPTX
Market and Demand Analysis.pptx for Management students
PPTX
Understanding Procurement Strategies.pptx Your score increases as you pick a ...
operations management : demand supply ch
Transportation in Logistics management.pptx
Susan Semmelmann: Enriching the Lives of others through her Talents and Bless...
Center Enamel Powering Innovation and Resilience in the Italian Chemical Indu...
Emerging Dubai Investment Opportunities in 2025.docx
Center Enamel A Strategic Partner for the Modernization of Georgia's Chemical...
Communication Tactics in Legal Contexts: Historical Case Studies (www.kiu.ac...
Vinod Bhatt - Most Inspiring Supply Chain Leader in India 2025.pdf
IITM - FINAL Option - 01 - 12.08.25.pptx
Retail Management and Retail Markets and Concepts
80 DE ÔN VÀO 10 NĂM 2023vhkkkjjhhhhjjjj
533158074-Saudi-Arabia-Companies-List-Contact.pdf
BUSINESS CYCLE_INFLATION AND UNEMPLOYMENT.pptx
33ABJFA6556B1ZP researhchzfrsdfasdfsadzd
Immigration Law and Communication: Challenges and Solutions {www.kiu.ac.ug)
Chapter 2 - AI chatbots and prompt engineering.pdf
TRAINNING, DEVELOPMENT AND APPRAISAL.pptx
CTG - Business Update 2Q2025 & 6M2025.pptx
Market and Demand Analysis.pptx for Management students
Understanding Procurement Strategies.pptx Your score increases as you pick a ...

Producing and validating small area estimates of household electricity demand

  • 1. Producing and validating small area estimates of household electricity demand Dr Ben Anderson Sustainable Energy Research Centre University of Southampton @dataknut IMA 2013 Canberra, December 12th 2013
  • 2. Small Area Estimates of Electricity Consumption Contents  What & Why  How?  Results – Overall consumption – Consumption inequalities  Conclusions & future Directions @dataknut 2
  • 3. Small Area Estimates of Electricity Consumption Contents  What & Why  How? ?  Results – Overall consumption – Consumption inequalities  Conclusions & future Directions @dataknut 3
  • 4. Small Area Estimates of Electricity Consumption Digression: Geography  Southampton (UK) @dataknut 4
  • 5. Small Area Estimates of Electricity Consumption Digression: What’s a small area?  In this case… – English Lower Layer Super Output Areas – Census 2001 LSOAs – c. 630 households each – 148 in Southampton City @dataknut 5
  • 6. Small Area Estimates of Electricity Consumption What & Why  Basically we want something for nothing – Small area estimates of energy demand – Without a bespoke energy census  Why? – Infrastructure planning – Energy efficiency intervention analysis – Politics! @dataknut 6
  • 7. Small Area Estimates of Electricity Consumption The problem:  Small area summaries exist  But they are aggregates – Or averages  And we want a micro-level model – To micro-simulate change… @dataknut 7
  • 8. Small Area Estimates of Electricity Consumption What can we do?  A bespoke energy census – ££££££££££££££££££££ @dataknut 8
  • 9. Small Area Estimates of Electricity Consumption What can we do?  A bespoke energy census – ££££££££££££££££££££  A large sample energy survey covering all LSOAs – ££££££££££ @dataknut 9
  • 10. Small Area Estimates of Electricity Consumption What can we do?  A bespoke energy census – ££££££££££££££££££££  A large sample energy survey covering all LSOAs – ££££££££££  Small Area Estimation – Take existing area level data – Take (ideally) an existing large n survey – Combine £ @dataknut 10
  • 11. Small Area Estimates of Electricity Consumption Small Area Estimation  Econometric approaches Income, income deprivation, income inequality smoking prevalence, obesity, consumption expenditure, CO2, water… – Well known – Multi-level Models – Usually requires census microdata for anything other Innovation Network: than means “Evaluating and improving small area estimation methods”  Re-weighting (and other) approaches – Increasingly http://eprints.ncrm.ac.uk/3210/ well known – 'Spatial microsimulation' – Does not require census microdata @dataknut 11
  • 12. Small Area Estimates of Electricity Consumption Contents  What & Why Estimation  How?  Results – Overall consumption – Consumption inequalities  Conclusions & Future Directions @dataknut 12
  • 13. Small Area Estimates of Electricity Consumption Data  Data – Living Costs and Food Survey 2008-2010 Consumption proxies (reported energy expenditure) – Census 2001 (2011)  Projection – Projected ‘surveys’ – Projected ‘census’  So far… @dataknut 13
  • 14. Small Area Estimates of Electricity Consumption Conceptually… LSOA census ‘constraint’ tables Survey data cases If Region = 1 LSOA 1.1 (Region1) If Region = 2 @dataknut Weights LSOA 2.1 (Region2) Iterative proportional fitting Ballas et al (2005) 14
  • 15. Small Area Estimates of Electricity Consumption Key First Job:  Choose your constraints Census data Survey data  You may have little choice @dataknut 16
  • 16. Small Area Estimates of Electricity Consumption Key First Job:  The constraints – Selected by stepwise regression Expenditure Number of persons Employment Status Accommodation type Number of earners Age of HRP Age of HRP Employment Status Most important Share of expenditure Composition Number of rooms Number of children Least important R sq @dataknut Ethnicity (non-white) 0.136 0.01 19
  • 17. Small Area Estimates of Electricity Consumption IPF…  Well known!  Deming and Stephan 1940 – Fienberg 1970; Wong 1992  A way of iteratively adjusting statistical tables – To give known margins (row/column totals)  In this case – Create weights for each case so LSOA totals ‘fit’ constraints – Weighting ‘down’ @dataknut 20
  • 18. Small Area Estimates of Electricity Consumption Internal Validation methods  Use of constraints to re-create the Census tables  Difference = Absolute Error – – Total Absolute Error (TAE) = sum of all errors Standardised AE = TAE/(n persons x n constraint categories)  Smith et al: – SAE of less than 20% and ideally less than 10% – Consumption Mean SAE p90 Ethnicity 2.18% 3.05% Number of children 0.11% 0.22% Number of rooms 0.05% 0.10% Employment status (HRP) 0.88% 1.22% Age (HRP) 0.34% 0.75% Tenure 0.07% 0.14% Accomodation type 0.21% 0.51% Number of persons 0.00% 0.00% in 90% of the areas is desirable. @dataknut 21
  • 19. Small Area Estimates of Electricity Consumption Preliminary results: Electricity  Mean weekly £  Modelled  Census 2001  LC&F Survey 20082010 @dataknut 22
  • 20. Small Area Estimates of Electricity Consumption Validation: Electricity  Mean weekly £  Observed @LSOA  DECC 2010  Spearman: 0.317 @dataknut 23
  • 21. Small Area Estimates of Electricity Consumption Preliminary results: Electricity  Total weekly £  Modelled  Census 2001  LC&F Survey 2008-2010 @dataknut 24
  • 22. Small Area Estimates of Electricity Consumption Validation: Electricity  Total weekly £  Observed @LSOA  DECC 2010  Spearman: 0.509 @dataknut 25
  • 23. Small Area Estimates of Electricity Consumption What is causing the error?  Heating! – @dataknut 2011 data 26
  • 24. Small Area Estimates of Electricity Consumption What is causing the error?  Heating! – 2011 data  Housing growth  Combined @dataknut 27
  • 25. Small Area Estimates of Electricity Consumption Consumption inequality  Area level gini  R = -0.413 – @dataknut (p < 0.001) 28
  • 26. Small Area Estimates of Electricity Consumption Consumption inequality  Area level gini  R = 0.463 – @dataknut (p < 0.001) 29
  • 27. Small Area Estimates of Electricity Consumption My big worry  Data quality @dataknut 30
  • 28. Small Area Estimates of Electricity Consumption Contents  What & Why  How?  Results – Overall consumption – Consumption inequalities  Conclusions & Future Directions @dataknut 31
  • 29. Small Area Estimates of Electricity Consumption Conclusions  Outliers and errors are informative  Reported consumption data – Could be dangerous  Census 2011 central heating – Critical new constraint @dataknut 32
  • 30. Small Area Estimates of Electricity Consumption Future directions  Update for 2011 data  Census projection 1981 -> 2021  Use measured energy consumption  Contact: – b.anderson@soton.ac.uk @dataknut 33

Editor's Notes

  • #31: Data from SPRG linked water demand survey2 implications:Error in the estimates (spurious correlation with constraints)Error in any policy microsimulation