Observational Science With Python and a Webcam

Observational Science
With Python
And a Webcam
By: Eric Floehr

Observational Science With Python and a Webcam by Eric Floehr is licensed under a
Creative Commons Attribution-ShareAlike 3.0 Unported License.

and I made a time-lapse video.
http://bit.ly/ospw-1

I wanted to do better.

Longer

Inside location

Automated

Once a minute

So this is what I did.

Second floor window
Little-used room
Pointing west
Pull pictures down with cronjob
That runs once per minute

I have...
● 896,309 image files
● 11,809,972,880 bytes

● 10.9989 gigabytes

● From 2:14pm on August 29, 2010

● To 4:11pm on July 25, 2012

● Still going...

Tenet #1:

Don't be afraid of big data.

What can we do?
● Moar Time-lapse!
● Explore phenomenon that occurs

over long periods
● Unique visualizations

● Have lots of fun!

First...

We need to organize the data.

How will we access the data?
Let's use Django!

Why Django?
● It has a good ORM
● It has a command framework

● Makes extending to the web later

easy
● I already am familiar with it

● Django isn't just for web :-)

Quick setup
● Create your virtualenv (with site packages)
● PIL is a pain to compile

● Install Django

● django-admin.py startproject

● Edit settings.py

● createdb pics

● django-admin.py startapp pics

● django-admin.py syncdb

Tenet #2:

Don't let small things keep you
from your big picture.

What do we need to store?
● Image file location and filename
● Time the image was taken

● Interesting information about the

image
● Is it a valid image?

class Picture(models.Model):

filepath = models.CharField(max_length=1024)
filename = models.CharField(max_length=1024)

timestamp = models.DateTimeField(db_index=True)
file_timestamp = models.DateTimeField(null=True)

# Auto-populated, don't manually enter
hour = models.SmallIntegerField(null=True, db_index=True)
minute = models.SmallIntegerField(null=True, db_index=True)
file_hour = models.SmallIntegerField(null=True)
file_minute = models.SmallIntegerField(null=True)
# End auto-populated fields

size = models.IntegerField(default=0)

center_color = models.IntegerField(null=True)
mean_color = models.IntegerField(null=True)
median_color = models.IntegerField(null=True)

stddev_red = models.IntegerField(null=True)
stddev_green = models.IntegerField(null=True)
stddev_blue = models.IntegerField(null=True)

min_color = models.IntegerField(null=True)
max_color = models.IntegerField(null=True)

valid = models.BooleanField(default=False)

class Meta:
ordering = ['timestamp']


filename = models.CharField(max_length=1024) <-- File location and name








class Meta:




hour = models.SmallIntegerField(null=True, db_index=True) <-- Image Timestamp






class Meta:







stddev_red = models.IntegerField(null=True) <-- Interesting image data



class Meta:









<-- Is it valid? How do we order?
class Meta:

How do we populate the table?
● Iterate through directories of
image files
● Parse the file name to get

timestamp
● Get file timestamp to compare

● Load image to collect information

about the image

Find the image files
import os
import fnmatch
import datetime

for dirpath, dirs, files in os.walk(directory):
for f in sorted(fnmatch.filter(files, 'image-*.jpg')):
# Pull out timestamp
timestamp = f.split('.')[0].split('-')[1]
date = datetime.datetime.fromtimestamp(int(timestamp), utc)

Use PIL to get image info
import Image, ImageStat
From util.color import rgb_to_int

try:
im = Image.open(pic.filepath)
stats = ImageStat.Stat(im)
pic.center_color = rgb_to_int(im.getpixel((320,240)))
pic.mean_color = rgb_to_int(stats.mean)
pic.median_color = rgb_to_int(stats.median)

if im.size <> (640,480):
pic.valid = False

except:
pic.valid = False

It took a an hour or two using
six threads on my desktop

Tenet #3:

You have a 1990's
supercomputer on your lap or
under your desk.
Use it!

Size Indicates Complexity in jpeg
pics=# select timestamp, filepath, size from pics_picture where size=(select
max(size) from pics_picture);
timestamp | filepath | size
------------------------+--------------------------------------+-------
2011-10-10 18:26:01-04 | /data/pics/1318/image-1318285561.jpg | 64016


High Stddev Means Color Variation
pics=# select timestamp, filepath, size from pics_picture where
stddev_red+stddev_green+stddev_blue =
(select max(stddev_red+stddev_green+stddev_blue) from pics_picture);

Timestamp | filepath | size
------------------------+--------------------------------------+-------


About the Dataset
pics=# select min(timestamp) as start, max(timestamp) as end from pics_picture;
start | end
------------------------+------------------------
2010-08-29 14:14:01-04 | 2012-07-25 16:11:01-04
(1 row)

pics=# select count(*), sum(case when valid=false then 1 else 0 end) as invalid
from pics_picture;
count | invalid
--------+---------
896309 | 29555
(1 row)
pics=# select timestamp, filepath, size from pics_picture where size>0 and
valid=false order by size desc limit 1;
timestamp | filepath | size
------------------------+--------------------------------------+-------
(1 row)


Yuck! This Data Sucks!
● 29,555 invalid image files
● That's 3.3% of all image files

● Worse yet, there isn't a file every minute

● Based on start and end times, we should

have 1,002,358 images
● We are missing 10.6% of all minutes

between start and end times

It's Worse...I Forgot The Bolts!




* Interestingly, I was listening to the Serious Business remix of “All Is Not
Lost” by OK Go from the Humble Music Bundle as I made the previous slide.

Tenet #4:

Real world data is messy.
That's ok. Just work around it.

How we can work around it?
● Create table of all minutes
● Accept images “near” missing

minutes
● Use empty images when

acceptable images can't be found

How do we make movies?
● Collect a set of images in frame
order.
● Send that list of images to a movie

maker.
● Wait for movie to be made.

● Watch movie!

The previous bullet points in code
from movies import ImageSequence
from pics.models import NormalizedPicture
import datetime
from pytz import timezone

ims = ImageSequence()

ettz = timezone('US/Eastern')

start = datetime.datetime(2011,4,7).replace(tzinfo=ettz)
end = start + datetime.timedelta(days=1)

pics = NormalizedPicture.objects.filter(timestamp__gte=start,
timestamp__lt=end)


for pic in pics:
if pic.picture is not None:
ims.add_picture(pic.picture)
else:
ims.add_image('/tmp/no_image.png')

ims.make_timelapse('/tmp/{0}'.format(start.strftime("%Y%m%d")), fps=24)

Collect a set of Images
import datetime




timestamp__lt=end)


for pic in pics:
else:


Send Images to Movie Maker
import datetime




timestamp__lt=end)


for pic in pics:
else:


Wait For Movie To Be Made
import datetime




timestamp__lt=end)


for pic in pics:
else:


Wait For Movie To Be Made
class ImageSequence(object):
def __init__(self):
self.images = []

def add_picture(self, picture):
self.images.append(picture.filepath)

def write_to_file(self, fileobj):
for image in self.images:
fileobj.write(image)

def make_timelapse(self, name, fps=25):
tmpfile = tempfile.NamedTemporaryFile()
self.write_to_file(tmpfile)

bitrate = int(round(60 * fps * 640 * 480 / 256.0))

execall = []
execall.append(mencoder_exe)
execall.extend(mencoder_options)
execall.extend(["-lavcopts",
"vcodec=mpeg4:vbitrate={0}:mbd=2:keyint=132:v4mv:vqmin=3:lumi_mask=0.07:dark_mask=0.2
:mpeg_quant:scplx_mask=0.1:tcplx_mask=0.1:naq".format(bitrate)])
execall.append("mf://@{0}".format(tmpfile.name))
execall.extend(["-mf", "type=jpeg:fps={0}".format(fps)])
execall.extend(["-o", "{name}.mp4".format(name=name)])

return subprocess.call(execall)

Watch Movie!
http://bit.ly/ospw-8-raw

We aren't limited to
consecutive minutes...

Movie of a Specific Time


Hour = 22 # UTC Time
minute = 0

last_pic = None

pics = NormalizedPicture.objects.filter(hour=hour, minute=minute)

last_pic = None

for pic in pics:
last_pic = pic.picture
else:
# Use yesterdays' image
if last_pic is not None:
ims.add_picture(last_pic)
else: # Give up
ims.add_image('/tmp/no_image.jpg')

ims.make_timelapse('/tmp/time_{0:02d}{1:02d}'.format(hour,minute),fps=12)

Movie of a Specific Time


hour = 22 # UTC time
minute = 0

last_pic = None

pics = NormalizedPicture.objects.filter(hour=hour, minute=minute)

last_pic = None

for pic in pics:
last_pic = pic.picture
else:
else: # Give up

ims.make_timelapse('/tmp/time_{0:02d}{1:02d}'.format(hour,minute),fps=12)

Watch Movie!

Now what if we want the Sun
the same height above the
horizon, so we can better see
the seasonal progression of
the Sun?

We can anchor on sunset. At a
given time before sunset, the
Sun will be at relatively the
same height above the
horizon.

Where I thank Brandon Craig
Rhodes for pyephem

Movie at Specific Time Before Sunset
import sky

obs = sky.webcam()

while current < end:
# Get today's sunset time, but 70 minutes before
pic_time = sky.sunset(obs, current) - datetime.timedelta(minutes=70)
pic_time = pic_time.replace(second=0, microsecond=0)

pic = NormalizedPicture.objects.get(timestamp=pic_time)

else:
piclist = NormalizedPicture.objects.get(
timestamp__gte=pic_time - datetime.timedelta(minutes=20),
timestamp__lte=pic_time + datetime.timedelta(minutes=20),
picture__id__isnull=False)

if len(piclist) > 0:
ims.add_picture(piclist[0].picture)
else:
else: # Give up

current = current + datetime.timedelta(days=1)

Our sky module that uses pyephem
import ephem
from pytz import timezone, utc

sun = ephem.Sun()
moon = ephem.Moon()
est = timezone('EST')

def observer():
location = ephem.Observer()
location.lon = '-83:23:24'
location.lat = '40:13:48'
location.elevation = 302
return location

def sunrise(observer, date):
observer.date = date.astimezone(utc)
return observer.next_rising(sun).datetime().replace(tzinfo=utc).astimezone(est)

def sunset(observer, date):
return observer.next_setting(sun).datetime().replace(tzinfo=utc).astimezone(est)

def moonset(observer, date):
return observer.next_setting(moon).datetime().replace(tzinfo=utc).astimezone(est)

Watch Movie!

August 29, 2010
6:59pm EDT


October 22, 2010
5:33pm EDT


Ancient Astronomical Alignments

Photo credits:
http://en.wikipedia.org/wiki/File:ChichenItzaEquinox.jpg
http://www.colorado.edu/Conferences/chaco/tour/images/dagger.jpg

Tenet #5:

Once you have a foundation
(data or code), however messy,
you can build higher.

We Also Take Pictures At Night
● Could I have captured a UFO in an
image?
● Or a fireball?

● Some other phenomenon?

● What track does the moon take

through the sky?
● How about Venus or Jupiter?

Moon Tracks and UFOs
● We want to find interesting things
● How do we easily identify “interesting things”

● At night, bright things are interesting things

● Brightness is a good proxy for “interesting”

● It makes it easy to identify interesting things

● As it is easier to spot black spots on white images,

we'll invert the images

Making Night Tracks
● Process each image
● Stack images into a single “all

night” image
● From one hour after sunset to one

hour before sunrise the next day
● Black splotches are interesting

things

def make_nighttrack(rootdir, day):
obs = sky.observer()

# One hour after sunset
start_time = sky.sunset(obs, day) + datetime.timedelta(hours=1)

# Until one hour before sunrise the next day
end_time = sky.sunrise(obs, tomorrow) - datetime.timedelta(hours=1)

pics = NormalizedPicture.objects.filter(
timestamp__gte=start_time, timestamp__lt=end_time, picture__id__isnull=False)

print "Drawing image with {0} images".format(len(pics))
im = Image.new("L", (640,480), background_color)

for picture in pics:
source = Image.open(picture.picture.filepath)

# Get the negative
source_gray = ImageOps.grayscale(source)
source_neg = ImageOps.invert(source_gray)

# Threshold white
source_thresh = Image.eval(source_neg, lambda x: 255*(x>224))

# Merge in the new image
im = ImageChops.multiply(im, source_thresh)

# Put a date on the image
canvas = ImageDraw.Draw(im)
canvas.text((10,460), day.strftime("%Y-%m-%d"))

# And save
im.save('{root}/{date}_mt.png'.format(root=rootdir,date=day.strftime("%Y%m%d")))

Venus, Jupiter, and the Moon


Lighting Up the Sky


Lightning Bug? Aircraft? UFO!

Venus

?
Moon


Shows up 3 weeks later


Last Oddity: 9/2/2011


Last Oddity: 9/2/2011

Single-Frame Oddity Crescent Moon That Night

http://bit.ly/ospw-30 http://bit.ly/ospw-32

Daystrips
● So far we've been using whole
images...let's change that
● Instead of using a whole image,

let's just use one line
● Kind of like a scanner

● Start at midnight, stacking lines

Daystrip Scanner

12:06pm

11:40pm http://bit.ly/ospw-24

Daystrips
● There are 1440 minutes in a day
● Images will be 1440 pixels high

● We place image line at the current

minute in the day
● HOUR * 60 + MINUTE

This is beginning to look familiar
def make_daystrip(rootdir, day):
end = day + datetime.timedelta(days=1)

print "Getting data for {0}".format(day)
timestamp__gte=day, timestamp__lt=end, picture__id__isnull=False)

im = Image.new('RGB', (640,1440), background_color)


# Get row 1/3 of the way down, painting on proper row of canvas
timestamp = picture.timestamp.astimezone(esttz)

y = timestamp.hour * 60 + timestamp.minute

im.paste(source.crop((0,160,640,161)),(0,y))

im.save('{root}/{date}_daystrip.png'.format(root=rootdir,date=day.strftime("%Y%m
%d")))

Create Our New Image







%d")))

Iterate Though Our Images







%d")))

And Paste The Single Row







%d")))

Finally, save the image







%d")))

Daystrip Example – March 17, 2011
Midnight

Moon Crossing

Sunrise

More cloudy (gray)

Less cloudy (blue)

Sun Crossing
Sunset

Midnight

Shortest and Longest Day

Dec 22, 2011 June 20, 2012


Tenet #6:
Don't be afraid to be creative.
Science can be beautiful.

Everything we've made so far
spans a day or less.

What we've done so far
● Viewed full images of interest
● Combined full images in movies

● Stacked full images to find UFOs

● Took a full line from a day's worth

of images
● Everything is a day or less of data

Instead of an image row being
a single minute...

it was a whole day...

And each pixel in the row was
a minute in the day.

Therefore, each of our 896,309
webcam images would
comprise a single pixel in our
über-image.

A Visual Representation

Midnight
Midnight

Noon
Minutes in a day (1440)
Start Day
Days (each row is one day)

Each pixel is one
minute in the day
Image

End Day

pics = NormalizedPicture.objects.all()

im = Image.new('RGB', (1440,num_days), background_color)
Canvas = ImageDraw.Draw(im)

# Get XY
timestamp = picture.timestamp.astimezone(est)

y_timedelta = timestamp - start
y = y_timedelta.days
x = timestamp.hour * 60 + timestamp.minute

# Paint pixel
if picture.picture is not None:
color = int_to_rgb(picture.picture.median_color)
else:
color = background_color

canvas.point((x,y), fill=color)

# All done, save
im.save(filepath)

The Result


What Exactly does DST Do?


Basically It Normalizes Sunrise Time
By Making Summer Sunrise Later


At The Expense Of Wider Sunset Time
Variation, Because Of Later Summer Sunset


Python makes it easy to
answer “What If?”

So...

Tenet #7:
Don't be afraid to ask
“What if?”, and don't be afraid
of where it takes you.

Presentation:
http://bit.ly/ospw-talk-pdf
http://bit.ly/ospw-talk (raw)
Code:
http://bit.ly/ospw-code
Picture Set (9.1GB):
http://bit.ly/ospw-pics
Links and more (soon):
http://intellovations.com/ospw

Observational Science With Python and a Webcam

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to Observational Science With Python and a Webcam (20)

Recently uploaded (20)

Observational Science With Python and a Webcam