FME for Climate Data: Turning Big Data into Actionable Insights

FME for Climate
Data:
GHD
The Peak of Data
and AI 2025
Turning Big Data into
Actionable Insights

2025
The
Peak
of
Data
and
AI
Vishal Kehri
GIS Developer
GHD
Vishal.Kehri@ghd.com

Agenda
1. Introduction
2. Project Overview
3. Problem
4. Methodology
5. Solution
6. Unlocked Opportunities
7. Questions

Objective
Update current IDF curves and develop
future climate-informed curves’
Tasks
Collect and process rainfall data, analyze
trends, integrate climate change
Challenges
Limited data coverage, older radar
formats, uncertainty in data
Outcome
Updated, reliable IDF curves to support
future-ready stormwater planning

2025
The
Peak
of
Data
and
AI
"The rainfall dataset lacked consistent structure, making
it difficult to accurately isolate and analyze rainfall events
for individual stations."

2025
The
Peak
of
Data
and
AI
Challenges

2025
The
Peak
of
Data
and
AI
Challenges
• Unique processing methods were needed due to data
structure inconsistencies before and after 2007.
• Varying data mapping and inconsistent time intervals
made event identification challenging.
• Single CSV file for all radar stations.
• Identifying events for each station was crucial.

Methodology
Section 4
“The goal is to turn data into information,
and information into insight.”
— Carly Fiorina, Former CEO of Hewlett-Packard

Feature
Manipulation
Engine (FME)
Workﬂow

Methodology
Raw Data Input
• Extracted rainfall CSVs and station locations
• Divided the annual data from the initial point
Data Cleaning
• Cleaned raw data to eliminate errors and inconsistencies
• Ensured temporal consistency across datasets
Geographic
Correlation
• Linked rainfall data to radar locations
• Built a spatial dataset for regional frequency analysis
Data Enrichment
• Enhanced datasets with additional fields
• Enabling detailed analysis and visualization capabilities.

2025
The
Peak
of
Data
and
AI
How do we do this?

2025
The
Peak
of
Data
and
AI
The Starting Point
We began with two massive datasets —
● Local rain gauge ﬁles (20+ years)
● USGS NEXRAD radar data

2025
The
Peak
of
Data
and
AI
Data Prep &
Pivot
● Removed Null/Missing
● Removed unnecessary ﬁelds
● Data Pivot
○ Pivoted to obtain data for
○ How hard it rained
○ How often it reached
certain intensities

2025
The
Peak
of
Data
and
AI
Attach Spatial
Info
● Join tabular data with Radar
Locations
● Assign Location IDs

2025
The
Peak
of
Data
and
AI
Spatial Data
Preparation
● Join tabular data with Radar
Locations
● Assign Location IDs

2025
The
Peak
of
Data
and
AI
Data Querying
● Remove no rainfall events data
● Query data for
○ Start Date
○ End Date
○ Station Name

2025
The
Peak
of
Data
and
AI
Calculations
● Calculate time differences
● Remove non consequence data
● Calculate Rainfall Depth
● Calculate Rainfall Statistics

2025
The
Peak
of
Data
and
AI
Calculations
Explained
Attribute: timeDiff (Difference > 0)
Attribute: timeDiff (Difference < 0)

2025
The
Peak
of
Data
and
AI
Calculations
Explained
Time Difference per Hour
Rainfall Depth in mm

2025
The
Peak
of
Data
and
AI
Final Output

Unlocked
Opportunities
Section 6

2025
The
Peak
of
Data
and
AI
Repeatable Use Case

2025
The
Peak
of
Data
and
AI
• Spatial Analysis
• Temporal Analysis
• Data Interpolation
• Data Prediction

2025
The
Peak
of
Data
and
AI
“We’re in a new world in which
data may be more important
than software.”
— Tim O’Reilly, Tech Publisher and Media Innovator

2025
The
Peak
of
Data
and
AI
ThankYou
Vishal.Kehri@ghd.com
Vishal Kehri
GIS Developer
GHD

FME for Climate Data: Turning Big Data into Actionable Insights

More Related Content

Similar to FME for Climate Data: Turning Big Data into Actionable Insights (20)

More from Safe Software (20)

Recently uploaded (20)

FME for Climate Data: Turning Big Data into Actionable Insights