Python in the Atmospheric sciences

ENVIRONMENTAL
SCIENCE
DIVISION
PYTHON IN THE ATMOSPHERIC AND OCEANOGRAPHIC
SCIENCES… AND WHY YOU SHOULD CARE
SCOTT COLLIS
Python Nerd
Argonne National Laboratory and
Northwestern University
Name of Facility, Division, etc

ENVIRONMENTAL SCIENCE DIVISION
Snakes and people….
 I might be rehashing a little of what
you know…
 I want to give some perspective of
why the Python programming
language is “Special”.
 Provide some context of my own
journey into open source community
code.
 Perhaps indoctrinate a few of you into
a cult.
THIS TALK….

 I did a PhD in Physics, in a lab setting. I made
heavy use of IDL and FORTRAN.
 Lived near the mountains so made a hobby of
using model output to pick the best day to go
skiing (my PhD took some time…).
 Made my hobby my profession joining the
Australian Bureau of Meteorology as a
forecaster in 2007. No IDL license meant I
taught myself Python.
 Really got into it when I had to design retrieval
systems for DOE-ARM at Argonne National
Laboratory.
MY PYTHON JOURNEY

A language built on the philosophy of readability.
 Python is an interpreted, object-oriented,
high-level programming language.
 Its high-level built in data structures,
combined with dynamic typing and
dynamic binding, make it very attractive
for Rapid Application Development, as well
as for use as a scripting or glue language
to connect existing components together.
 Python's simple, easy to learn syntax
emphasizes readability and therefore
reduces the cost of program maintenance.
WHAT IS PYTHON?

Python is so ubiquitous it tends to be the go-to scripting language
 As the Python ecosystem has grown
and due to the readability, openness
and ubiquities of the language.
 If it can’t be done in a shell script
Python is the natural next step.
 In terms of data analytics it has seen
a particularly dramatic uptake in the
financial sector.
WHERE IS PYTHON USED?
https://www.capitalone.com/tech/software-engineering/building-python-user-
community-among-capital-ones-data-analysts/

Sustainable Software Engineering. Things for your CV
 Version Control. Learn Git.
 Social coding. Git and VC is the first
step.. There is a culture to pull
requests etc.. And then there is
GitHub and Bitbucket which exposes
your code to the community and
allows interaction.
 Test Driven Design and unit testing.
Ideally write the test before the
code… ideally…
 Continuous integration and family..
SO I USE PYTHON? WHAT ELSE DO I NEED TO
BE AWESOME?

Python is Python.. Needs modules.
 Just firing up Python only gives
you the core modules. Python is
not a scientific programming
language.
 Scientific Python is the term given
to the ecosystem of packages
enabling science in Python.
 The modularity and namespace
abilities of Python allows users
and developers to “roll their own”
stack.
SCIENTIFIC PYTHON STACK

Master the art of vectorization!
 At the core of Numpy (nee Numerical
Python) is a powerful array object.
 This builds on Python’s List object but
endows it with some highly optimized
computations under the hood in C.
 Avoid loops in Numpy! If you can
break down your code into a series of
array operations things can be nice
and fast.
 MASSIVE community effort.
NUMPY – MAKES PYTHON FAST

Highly customizable publication quality graphics.
 Matplotlib is a highly scriptable
comprehensive library for creating
plots and visualizations.
 Acts as a backbone to many other
libraries! Can take some wrestling to
get what you want due to the focus on
flexibility.
 Great fonts.
 Recent focus on CVD friendly
colormaps.
MATPLOTLIB – MAKES PYTHON PRETTY

Master the art of the GroupBy!
 Pandas adds some smarts to the
array object.
 It works with ”labels”.
 Great for time series analysis.
 Great for data exploration with
visualization methods built in.
 Gateway for those who have used
Stata, S and R.
PANDAS – MAKES PYTHON SMART

N-Dimensional labelled arrays!
 Xarray makes n-dimensional arrays
label aware.
 This is extremely useful for CF
compliant data files. Xarray can
detect axes and slicing becomes
intuitive and code becomes readable.
 Easy bindings to Matplotlib and
cartopy.
XARRAY – MAKES PYTHON AWESOME

By Scientists, for Scientists.
 The Atmospheric and Oceanic
community have really come together
to build a AOS specific stack.
 This includes Radar, Meteorology,
Satellite, retrievals, visualization R2O
(eg scheduling) etc…
 Yearly symposium at the AMS annual
meeting. Abstracts OPEN!
PY-AOS ECOSYSTEM

It is what it says! Python
for Meteorologists.
 Loaded with great meteorological
goodness. Lots of calculations etc…
 Great mapping… Layer on top of
cartopy.
 Aims to replace Gempack.
METPY

ENVIRONMENTAL SCIENCE DIVISION 15
Philosophy: It’s all about the data model.
https://github.com/ARM-DOE/pyart
 Py-ART’s central core is a data model for gated
data with pointing information.
 Py-ART created a way of representing radar
data in the Python programming language that
mirrors the CF-Radial standard.
 Py-ART has a cloud functions to correct,
retrieve and grid radar data.
 By keeping a limited scope Py-ART aims to “do
less better”.
 There is now a rich ecosystem of packages
that interact: ART-View, CSU tools, PyTDA,
Multidop to name a few. Oh and we are working
on TINT!
THE PYTHON ARM RADAR TOOLKIT
Animation courtesy of users Marcus van Lier-Walqui
and Sara E. Lytle
Data: Andrei Lindenmaeir – ARM Mentor

Python in the Atmospheric sciences

The power of community software.. But, you need standards, unit testing and
continuous integration.
Only scollis, jjhelmus, zssherman, rcjackson and rumpkie
have received ARM funds. The other 28 have not…

We are all in this together…
 Python efforts in Radar Meteorology nucleated
in parallel in Europe and the USA.
 In addition the long standing TITAN software
cloud received NSF support for a modernization
surge.
 A short course at ERAD 2014 led to a co-written
paper which lead to a rotating course between
ERAD and the AMS radar conference.
 We now have a web presence and a growing
ecosystem of identified packages.
OPEN RADAR PARTNERSHIP

2014 Germany2016
Turkey
2018 Netherlands
2017 Chicago
2018 Melbourne
2019 Melbourne
2019 Japan
2015 Oklahoma

PyDDA, RainyMotion,
CSU Radar Tools… etc…
AND MORE…

Python is the community
the breaks down the barriers
between domains.
 Python is a programming language.
 Scientific Python and the Py-AOS
stack represents the sum of the
knowledge given by those who have
been kind enough to work on them.
 Using Python is great and a great
way of increasing your employability.
 Developing Python is better and will
all but ensure you get a job 
SO, IN THE END, WHAT IS PYTHON?

FINAL WORDS OF ADVICE….

THANK YOU FOR YOUR TIME
– scollis@anl.gov - Email
– cyclogenesis_au -Twitter
– scollis – GitHub

Python in the Atmospheric sciences

More Related Content

Similar to Python in the Atmospheric sciences (20)

Recently uploaded (20)

Python in the Atmospheric sciences