TIMES-NZ 3.0: automating upstream data processing for an open-source workflow
0 likes62 views
TIMES-NZ 3.0: automating upstream data processing for an open-source workflow Luke Searle, EECA, New Zealand
TIMES-NZ 3.0: automating upstream data processing for an open-source workflow
- 1. TIMES-NZ 3.0 automating upstream data processing for an open-source workflow Luke Searle ETSAP Workshop, June 9 2025
- 2. • TIMES-NZ: current workflow overview • Upstream workflow: current and future states • Proof-of-concept: automatic excel file generation • New system design and electricity generation example
- 3. TIMES-NZ workflow Excel files (manual) TIMES model generator Veda TIMES-NZ webtool ??? Available to users Not available to users
- 4. Hundreds of Veda tags, dozens of spreadsheets Derivation methods or source data often not included Labour-intensive to update and vulnerable to team turnover Not trackable via git Upstream processing: current state
- 5. Upstream processing: current state TIMES-NZ CI/CD: XL2TIMES and the times_excel_reader Results (not methods) now trackable via git
- 6. We want upstream to be: • Replicable • Maintainable • Transparent Which means: • Clear delineation of data, assumptions, or upstream derivations • Automated workflow from inputs to excel files for Veda • Version controlled and centrally available on github for our users Upstream processing: future state
- 7. Proof of concept: Auto-generated excel files for TIMES-NZ 2.1 Excel files (auto- generated) TIMES model generator Veda TIMES-NZ webtool raw_tables.txt
- 8. Stage 0: Set config Raw data Stage 1: Ingest raw data Stage 2: Base year data Stage 3: Scenario and subRES data Stage 4: Format for Veda Excel files (auto-generated) Assumptions Veda format rules User settings New system design
- 9. VT_TIMESNZ_ELC.toml Stage 0: Config metadata parse_tomls.py Case study: base year electricity generation (Stage 0) Automatic tag documentation
- 10. Case study: base year electricity supply (Stage 1-4) Stage 1: Input Data Stage 1 scripts Raw data Assumptions Fuel delivery costs Peak contribution Capacity factors Plant lifetimes Distribution network costs Transmission network costs etc Rooftop solar capacity Existing grid assets Transmission losses Generation levels Operating costs Heat rates Wind availability Geothermal emission factors etc baseyear_electricity_supply.py base_year_electricity_supply.csv create_baseyear_ELC_files.py Stage 4: csv files for Veda VT_NZ_ELC.xlsx write_excel.py Stage 0: Config metadata