Color naming 65,274,705,768 pixels

Color naming 65,274,705,768 pixels

Nathan Moroney and Giordano Beretta

HP Labs

Electronic Imaging 2013: Color Imaging XVIII

Outline
 Motivation
 More (pixel) data
 Finding and processing 65 billion pixels
 Hint: Wikipedia & a dual core Open MP color namer
 What did you learn?
 The most frequent non-achromatic color term is…
 What’s next?
 Other than a trillion pixels


Motivation
 Previous work in crowd-sourcing color training data
and experimental efforts
 Related work in the area of big (image) data
 A. Torralba, R. Fergus, W. T. Freeman, "80 million tiny images: a
large dataset for non-parametric object and scene recognition",
IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol.30(11), pp. 1958-1970, 2008.
 Ben Shneiderman, "Extreme Visualization: Squeezing a Billion
Records into a Million Pixels", SIGMOD Conference, pp. 3-12,
(2008).
 Steven Seitz, “A Trillion Photos”, EI’13 Keynote (2013).


Motivation

0 1 2 3 4 5 6
Log Number of Images


Source Data
 ImageClef 2010 snapshot
 Adrian Popescu, Theodora Tsikrika and Jana Kludas, "Overview
of the wikipedia retrieval task at ImageCLEF 2010", In the
Working Notes for the CLEF 2010 Workshop, 20-23 September,
Padova, Italy, 2010.
 250,000 images plus associated wikipedia data
 20 gigabytes
 65,000,000,000 pixels uncompressed


Source Data: At 200 PPI


Processing
 Basic single dual-core (but Open MP threaded) script
to process over all image files
 Simple stuff like getting image dimensions can be
done over lunch
 Uncompressing all the JPEG files to memory can
take hours
 Goal was a color naming algorithm that could be run
in less than a day


Processing
 Some testing done using HP Cloud Services and
compute clusters
 But majority of focus on single computing device
 Antony Rowstron, Dushyanth Narayanan, Austin Donnelly, Greg
O'Shea, and Andrew Douglas. "Nobody ever got fired for using
hadoop on a cluster", In HotCDP 2012 - 1st International
Workshop on Hot Topics in Cloud Data Processing, (2012).


Processing
 Won’t describe the specifics of the color naming
algorithm (throw produce if you have it) but generally
 Input single RGB pixel and output is a single color term
 Size of vocabulary or number of color terms is a parameter
 Relative range of chroma values corresponding to an achromatic
values is also a parameter
 Also currently testing a completely revised model
 Finally, in the Future directions section note that the
best option for formal publication is to make use of
currently available open source machine learning
toolboxes.


Results: Aspect Ratios
 Wide range of
image types
 Most basic test
of processing
scripts


Results: Median
 Additional test and
visualization of
basic color
properties of images
 Large enough data
set was worthwhile
to write custom
HTML5 2d canvas
renderer


Results: Median
 So much data, that
as noted by
Shneiderman the
density plot "uses a
spatial substrate
organizing
principle, but shows
concentrations of
markers” is maybe a
better idea
 Data, alpha=0.05


Results: Max
 Max of R+G+B for
the images
 Final test of basic
scripting code


Results
 Color terms
across all images
 Majority pixels
achromatic
 Top chromatic
colors are
arguably natural
tones
 Higher chroma
terms relatively
infrequent

Results
Color Terms for 200,000+ images
60000

 Color terms per
image 50000

 Peak at 5 are all 40000

achromatic terms
Number of Images
30000
or images
 Gradual then 20000

rapid usage of 10000

chromatic terms
0
0 5 10 15 20 25 30 35
Number of Color Terms. Maximum Vocabulary of 30


Results
Color Terms for 200,000+ images
60000

 Sudden drop off
at 30 is a model 50000

failure 40000

 Term added to
Number of Images
30000
vocabulary based
on previous 20000

limited
10000

optimization
0
0 5 10 15 20 25 30 35
Number of Color Terms. Maximum Vocabulary of 30


Current Work
 Repeated entire process adjusting the model
parameters
 Processing to fill SQL databases
 Query the database to validate all of the steps and
explore specific


Current Work
 SELECT * from
cntable order by
skyblue desc limit 40


Future Directions
 Image collections as “pixel
corpora” for algorithm
design, testing and optimization.
 Similar to the role that written and spoken
corpora fill for NLP and corpus linguistics
 Useful to formalize for citation and
repeatability
 Additional analysis features
 Testing with more public domain
machine learning algorithms for
repeatability


Summary
 Algorithm optimization, like machine color
naming, with 200,000 images is different than with
200.
 Based on Wikipedia, majority of visual content or
pixels are achromatic
 Based on Wikipedia, higher chroma named pixels are
less frequent
 Based on Wikipedia, there is a gradual then sudden
transition in color term usage


Color naming 65,274,705,768 pixels

More Related Content

What's hot (10)

Viewers also liked (20)

Similar to Color naming 65,274,705,768 pixels (18)

Recently uploaded (20)

Color naming 65,274,705,768 pixels