Antispam Image Filtering Technologies

Image-Filtering
Technologies
Michael Lamont
Senior Software
Engineer
Process Software

Overview
• Role of image filtering in anti-spam
filtering
• Two popular image filtering methods:
– Shape recognition
– Skin detection
• Example image filtering
• Image filtering issues
• Tools you can play with on your own

What Isn’t Covered
• Anything requiring advanced math
• Optical character recognition (OCR)

Spam Images
• A picture is worth 1000 words…
• …and it’s a lot harder to filter than
1000 words.
• Especially when spamvertizing
pornography, photos are essential
marketing tools.

Spam Images
• Right now, a spam filter can be very
effective without looking at images.
• This is going to change when the
majority of sites start installing more
accurate filters, and spammers are
forced to adapt.

90-Second Image Review
• To understand how image filtering
technologies work, you need a basic
understanding of how computers
represent images.
• Images are broken into square dots,
which correspond to pixels on a
monitor.

• Example image:

• Each dot’s color is represented by 3
components: red, green, and blue.
• Each of the three color components
has a value of 0 to 255.
• If all three are 0, then the pixel is black.
If all three are 255, then the pixel is
white.

• The higher the number, the more
intense the color component.
• Example: Increasing red value from 0
to 255 while leaving other components
at 0:

Shape Recognition
• Identifies objects in an image using
posterization and edge finding.
• Extracts interesting objects and
searches for similar objects in a
database of “bad” objects.
• For our application, the objects are
human body parts.

Posterization
• Dramatically reduces the number of
colors in an image.
• Has the side effect of lumping most of
an object’s pixels together.
• Called “posterization” because the
same kind of color reduction used to
be done for images printed on posters.

Posterization - Method
• A number of color bins are created.
• The number of bins is a lot less than
the ~16m colors that are possible.
• Each bin holds several hundred colors
that are closely related.
• Every color in the bin is represented by
the average color.

Posterization - Method
• Example: If a bin contained every
shade of red from light pink to dark
blood, every color in the bin would be
represented by plain old red.
• The posterization process itself
consists of replacing the color of every
pixel in the image with its bin’s
representative color.

Edge Finding
• After posterizing the image, edge
finding is used to identify individual
objects.
• Edge finding determines the
boundaries between different patches
of color and contrast.

Edge Finding - Method
• The edge finding program scans the
image looking for pixels that are very
different from their neighbors.
• When it finds a radically different pixel,
it marks it as part of an edge.
• Good edge finding algorithms look at
lots of neighboring pixels to help
reduce noise.

Object Extraction
• Once objects have been identified with
posterization and edge finding, they’re
easy to extract.

Object Extraction
• Leg, midriff, and upper torso objects
are being searched in the case of
people wearing swimsuits.

Object Extraction
• A database of known objects is
searched for matches to the extracted
objects.
• Both object shape and color are used
in the search.
• Comparisons are done with a fuzzy
logic algorithm, since it’s unlikely two
objects will be exactly alike.

Skin Detection
• Subset of an image classification
method called color histogram
matching.
• Finds patches of skin tone in an image.
• Calculates the overall percentage of
the image that is skin.
• If more than a specified amount of the
image is skin, it’s filtered.

Skin Tones
• Almost all human skin is the same hue
- saturation differences result in
different skin colors.
• Human skin tones don’t often appear
in other photographed objects, so color
alone can be used to identify skin.
• Skin tones are primarily red, without
any blue and little if any green.

Skin Color Model
• To identify skin tones in an image, a
filter needs to know what colors are
skin tones.
• You could hardcode every skin color,
but there are tens of thousands of
them.
• Much more accurate to identify skin
patches in an image and “train” the
filter.

Skin Color Training
• Works almost like Bayesian filter
training, but with image colors instead
of message tokens.
• Filter maintains one database of skin
colors, and another database of non-
skin colors.
• If a color appears more often in the
skin color database, it’s treated as a
skin color.

Skin Color Training
• This system has the nice side-effect of
dropping out most skin colors that also
appear in non-skin areas of photos.

Skin Identification
• To analyze an image, the filter
examines the color of each pixel.
• If the color is a skin tone, the filter
marks the pixel as skin.
• When every pixel has been examined,
the % of the image that is skin is
calculated.
• If the % is over a specified threshold,
the image is filtered.

Correctly Filtered Images - Shape

Correctly Filtered Images - Skin

Shape Recognition Problems
• Following are examples of images that
shape recognition doesn’t handle
correctly.
• Skin detection handles them correctly,
but only because it’s biased to filter
images with a lot of skin.

• Unusual angle obscures shapes

• Skin detection works

• Shapes are too broken up for the filter
to work

• Not enough “swimsuit” objects

• Image is so noisy that edge detection
goes crazy

• Amazingly, skin detection still works

Skin Detection Problems
• Following are examples of images that
skin detection incorrectly filters.
• Shape recognition works for most of
these, mainly because it can’t extract
any useful shapes.

• Baby photos tend to show lots of skin

• Shape recognition doesn’t filter the
image

• Portraits have the same problem as
babies.

• Shape recognition ignores the image.

• In the right light, sand can be the same
color as skin.

• That’s fairly rare - usually skin color
models exclude sand colors.

• Black & white images can’t be filtered

• It also makes life rough on shape
recognition filters.

Wedding Photos
• Wedding photos are guaranteed to
make a mess of image filters.
• Skin fades into the background
because of soft lighting, soft filters, and
retouching.
• Turns out that brides get upset if the
image is crystal clear with good
contrast - it shows off skin flaws.

Wedding Photos
• Skin detection filters start identifying
everything as skin (false positive).
• Shape recognition filters give up and
don’t filter the message (accurate, but
not for the right reasons).
• Porn tends not to be shot with soft
lighting - good contrast makes skin
“pop” in photos.

“Art Porn”
• Usually shot with the same lighting
effects as wedding photos.
• Rarely seen in email.
• In this case, skin detection is accurate
for the wrong reasons while shape
recognition lets the image pass.

“Art Porn” Example - Shape

“Artistic” Example - Shape

Things I Can’t Show You
• S & M
– Skin tends to be covered with “clothing”
– Shapes are broken up by all of the
paraphernalia
• Simpson’s shocker
• Still images from “interesting” videos
– Images are badly pixelated
– Colors are muddy and smudged

Image Filtering Issues
• Accuracy:
– Shape recognition misses lots of images it
shouldn’t (false negatives)
– Skin detection filters lots of images it
shouldn’t (false positives)
– Best skin detection systems are about
80% accurate
– Best shape recognition systems are about
40% accurate

• Performance:
– Image filtering requires huge amounts of
memory, CPU time, and disk bandwidth.
– Unacceptably slows down most site’s
email servers/filtering systems.
– DL380 benchmark:
• ~1.2 million messages/hour with no filtering
• ~195,000 messages/hour with skin detection
• ~69,000 messages/hour with shape recognition

• Diminishing returns on accuracy - most
spam filters won’t see a noticeable
increase in accuracy with the addition
of image filtering.
• That’s likely to change in the future as
spammers discover it’s one of the
better options for circumventing current
solutions.

I Wanna Play!
• Shape recognition:
– UC Berkeley’s blobworld
• Open source
• http://elib.cs.berkeley.edu/
– Skin detection
• No good open-source examples
• Trivial to write your own using ImageMagick
• http://www.imagemagick.org/

Quick Review
• We covered:
– How and why images appear in spam
– Why the use of images in spam is likely to
increase
– Two methods for filtering images
– Examples of how the two methods work
and don’t work
– Why image filtering isn’t widely used at
this point.

Antispam Image Filtering Technologies

Antispam Image Filtering Technologies

More Related Content

Similar to Antispam Image Filtering Technologies (20)

More from Michael Lamont (15)

Recently uploaded (20)

Antispam Image Filtering Technologies