Large Scale Image Forensics using Tika and Tensorflow [ICMR MFSec 2017]
2 likes1,022 views
The document presents a methodology for large-scale image forensics using deep learning, focused on monitoring illegal weapons sales through the DARPA Memex program. It describes the integration of TensorFlow with Apache Tika for analyzing 1.4 million weapon images using the Inception-v3 model, optimized for efficient performance. The evaluation indicates successful image recognition, aiding the classification of weapons and enhancing data retrieval capabilities.
Large Scale Image Forensics using Tika and Tensorflow [ICMR MFSec 2017]
- 1. An Approach for Automatic and Large Scale Image Forensics Thamme Gowda*# , Kyle Hundman # , and Chris Mattmann *# * Computer Science Department, University of Southern California, Los Angeles, CA, USA # Jet Propulsion Laboratory California Institute of Technology Pasadena, CA, USA Presenter: Paul Ramirez #
- 2. OVERVIEW • Abstract • Motivation • Data • Image Recognition • Inception Net • Integration • Evaluation • Conclusion
- 3. ABSTRACT • Applications of deep learning-based image recognition in the DARPA Memex program • Integration of Tensorflow with Apache Tika for automatic image forensics • Evaluation of model performance on weapons dataset
- 4. MOTIVATION DARPA Memex: • Monitor online weapons sale in the United States • Goal 1: Retrieve ads and relevant multimedia such as images, videos • Goal 2: Forensics • Classify illegal weapons • Sale trends • Goal 3: Discoverable / Searchable
- 5. DATA COLLECTION • Used web crawlers specialized for retrieving data • Crawlers that can login to web sites and run javascript in pages • Crawlers that can work with Onion protocol • Example: Apache Nutch, Sparkler, Scrapy, … by various teams • Large repository of web pages and multimedia documents • 1.4 M images from weapons domain
- 6. IMAGE RECOGNITION TASK • Image Recognition: Detect real word entities in the digital images • ImageNet dataset: • Large visual dataset of annotated images • ImageNet Large Scale Visual Recognition Challenge (ILSVRC) • Annual competition organized by Stanford and Princeton Universities • Challenge: How accurately can your model identify 1000 classes • From 2010 to Now • Since 2012, Deep ConvNets ruled the competition • Goto place to see state-of-the-art models for image recognition
- 7. INCEPTION NET • Developed by Google Research Team • Sergey et al,2015 - Originally GoogleNet, winner of ILSVRC 2014 • Code named Inception, multiple version V1, V2, V3, V4,.. • Google open sourced Tensorflow with Inception-V3 and its model trained on ImageNet dataset • Inception-V3 is optimized to run with less memory and fewer CPU cycles (like Android devices) • We have used Inception-V3 for our forensics
- 8. SOFTWARE STACK • Apache Tika - universal parser for parsing files over a thousand file types • Primarily written in Java; available for free via Apache License • Meta data analysis • Semantic analysis - detect names of people, locations etc in text • And more - OCR in images • One of the key technology for content analysis in DARPA Memex • Had been useful for others too - heard of Panama Papers? • Tensorflow • Written in C++ with Python bindings; available free via Apache License • Developed by google and now one of the popular deep learning frameworks
- 9. INTEGRATION METHODS • Challenge: • Make use of C++/Python code from a Java Client • Techniques • Command Line Invocation (CLI) • Java Native Interface (JNI) • gRPC Remote Procedure Call (gRPC) • REpresentation State Transfer (REST) API • REST API integration was the best among the above four
- 10. RESULTS Labeled the 1.4 million images in Memex Weapons Dataset 1000 target classes in training data (ImageNet)
- 11. HANDLING THE SCALE • 1.4 million images in the dataset • REST integration took 36 hours to run on 32 Core CPUs, no GPUs used • TensorFlow automatically parallelized the load on all CPU cores in a single node • Wiki https://wiki.apache.org/tika/TikaAndVision • Recent work: We have hadoop/spark distributable framework powered by Deeplearning4j • https://wiki.apache.org/tika/TikaAndVisionDL4J • https://github.com/thammegowda/tika-dl4j-spark-imgrec
- 12. RIFLES
- 13. REVOLVERS
- 14. EVALUATION Our evaluation dataset: • Consists of gun images • Law enforcement officers manually labelled them Observations: • Some Rifles mislabeled based on surrounding objects - small size • Top - 5 measure is a reasonable measure
- 15. CONCLUSION • We have made image recognition easy for Apache Tika users • We have tested that Inception-V3 model was successful in detecting weapon images • Image labels helped to build a better web page classifier for Memex ACKNOWLEDGEMENT: This effort was supported in part by JPL, managed by the California Institute of Technology on behalf of NASA, and additionally in part by the DARPA Memex/XDATA/D3M programs and NSF award numbers ICER-1639753, PLR-1348450 and PLR-144562 funded a portion of the work