Image Classification and Retrieval on Spark
1 like1,119 views
This document discusses using Spark Mbuto to build machine learning pipelines for image classification and retrieval. It provides an overview of classification and retrieval problems and the logic behind building pipelines for each. Key steps in the pipelines include feature extraction, building a codebook/dictionary, training and evaluating classifiers. Classification models discussed include KNN and neural networks. The document outlines building image pipelines in Spark Mbuto through sequential Spark jobs that share a context.
Image Classification and Retrieval on Spark
- 1. SPARK MBUTO Design & Engineering Machine Learning Pipelines Gianvito Siciliano Use Case: Image Classification and Retrieval
- 2. OUTLINE 1. Spark ‘Mbuto intro 2. ML problems overview 3. Classification & retrieval logic 4. Classification Models 5. Image Pipeline
- 3. OUTLINE 1. Spark ‘Mbuto intro • Abstractions • Basic Examples 2. ML problems overview 3. Classification & retrieval logic 4. Classification Models 5. Image Pipeline
- 4. SPARK MBUTO • Spark poc to (easy) create, run and test pipelines and workflow • Pipelines are made by sequential steps in a SparkJobApp • Each steps is a SparkJob • Each job share the same Spark/SQL context • Jobs are consecutively run by JobRunner
- 5. SPARKJOB
- 6. JOBRUNNER
- 7. SPARKJOBAPP
- 10. READABLE APP App .main JobRunner .run Job Job .execute .execute next job
- 12. OUTLINE 1. Spark ‘Mbuto intro 2. ML problems overview • Classification • Retrieval 3. Classification & retrieval logic 4. Classification Models 5. Image Pipeline
- 13. IMAGE CLASSIFICATION • Multiclass image classification: 1. Choose model (NN, SVM,TREE…) 2. Train/test model (with labeled images) 3. Predict the label of new images 4. Tune the model
- 14. IMAGE RETRIEVAL • Multiclass image classification: 1. Choose metric (Euclidean, cosine…) 2. Build dictionary 3. Train/test the model 4. Query and search 5. Tune the model
- 15. WHAT CHANGES? • Pipelines architecture • Classification logic • How to update the model?
- 18. OUTLINE 1. Spark ‘Mbuto intro 2. ML problems overview 3. Classification & retrieval logic 4. Classification Models 5. Image Pipeline
- 19. CLASSIFICATION & RETRIEVAL • Keypoints extraction from each images • Clustering on the keypoints universe • Represent each image with weighted cluster vector • Train &Test the model • Query the model (finding the most similar images) Features Engineering Build the Dictionary Build the classifier Query the model
- 20. C. & R. JOBS • Load whole dataset • Extract keypoints • Reduce the keypoints universe • Transform the features space • Create the dictionary (aka Codebook) • Train, test & evaluate the classifier • Query and get prediction DATA TRAIN CLASSIFIER MODEL PREDICTION
- 23. KNN IMPLEMENTATION • Is a comparison model: the similarity metric is crucial! • Nearest Neighbour search (in the codebook) is the panic point: • KDTree: not parallel (anche se…) • LSH: hyperparams difficult to tune • MetricTree: disjoint features points area • Spill tree: too many shared points => HybridTree
- 24. HYBRIDTREE • TopTree is a Metric tree • SubLeaf Tree are Spill tree, trained in parallel • Nodes can be: • OVERLAP => defeatist search • NON OVERLAP => backtracking
- 25. NEURAL NETWORK • Convolutional works well with images • Hyperparameters tuning is the panic point, but can be automatised (guarda il nuovo algo) • Training is not trivial, update the model is easy to complain
- 26. WHAT MORE? • Features engineering • Hyperparameters tuning • Parallel optimizations • Persist/update steps • Ensemble models DATA Combiner PREDICTION Normalizer pipelineModel Cross Validator