Computer Vision: Coming to a Store Near You
1 like229 views
The document discusses the integration of computer vision in retail, highlighting trends, use cases, and business drivers that improve operational efficiency and customer experience. Key opportunities include enabling frictionless commerce using technologies like deep learning and machine learning, as showcased by examples from Amazon and Alibaba. The document emphasizes the economic value that AI and IoT will add to the retail sector by 2025, projected to be $410 billion.
Computer Vision: Coming to a Store Near You
- 1. © Cloudera, Inc. All rights reserved. COMPUTER VISION: COMING TO A STORE NEAR YOU Brent Biddulph, Managing Director, Retail & CG, Cloudera Florian Muellerklein, Data Scientist, Miner & Kasch
- 2. © Cloudera, Inc. All rights reserved. 2 AGENDA Industry Trends Business Drivers Use Cases How it Works Bringing It All Together
- 3. © Cloudera, Inc. All rights reserved. 3 86% …OF RETAIL SALES WILL STILL OCCUR IN- STORE IN 2021. (STATISTA, 2018)
- 4. © Cloudera, Inc. All rights reserved. 4 Mobile Apps ESLs, Robots Autonomous Vehicles Magic Mirrors+ Shelf, Bin, & Rack Sensors Surveillance CamerasCompetitive PricingSatellite Images Logistics & Asset TelematicsIn Home Devices Frictionless Checkout RFID Streams SINGLE VIEW OF CUSTOMER FRICTIONLESS COMMERCE UBIQUITOUS FULFILLMENT RELEVANT INTERACTIONS OPERATIONAL EFFICIENCY Streaming Capabilities Enable New Insights,Right-Time Response New Data Sources Extend Retail Intelligence = video, image data
- 5. © Cloudera, Inc. All rights reserved. 5 BUSINESS IMPROVEMENT OPPORTUNITIES FOR COMPUTER VISION IN-STORE • Enabling Frictionless Commerce • Improving Operational Efficiencies • Improving Customer Experiences • Reducing Fraud & Shrink $410B “AI / IOT ECONOMIC VALUE- ADD IN RETAIL BY 2025, WITH SOME OF THE MOST VALUABLE USE CASES DRIVEN BY STREAMING VIDEO ANALYTICS.” (MCKINSEY, 2017) REAL-TIME PROCESSING –IMPROVED ACCURACY – DEEPER INSIGHTS
- 6. © Cloudera, Inc. All rights reserved. 6 ENABLING FRICTIONLESS COMMERCE Amazon Go Store • Computer Vision (100s of cameras on shelves, ceiling) • Sensor Fusion (cameras, shelf scales) • Deep Learning (advanced pattern recognition, 100s of algorithm’s)
- 7. © Cloudera, Inc. All rights reserved. 7 ENABLING FRICTIONLESS COMMERCE Alibaba’s HEMA store • Robots, apps, facial recognition and overhead conveyor belts for fulfillment delivery • Using facial recognition customers can pay using their face - no QR code nor credit card swipe required
- 8. © Cloudera, Inc. All rights reserved. 8 IMPROVING OPERATIONAL EFFICIENCIES Out-of-Stock & Merchandising Execution • Out-of-Stock Notifications • Price and Promotion Compliance • Schematic & Display Compliance Photo Source: Retail Technology Corp.
- 9. © Cloudera, Inc. All rights reserved. 9 24% …OF AMAZON REVENUE CAN BE ATTRIBUTED TO BRICK & MORTAR STORE OUT-OF-STOCKS. (IHL, 2018) OPPORTUNITY: IMPROVED OUT OF STOCK RESPONSE
- 10. © Cloudera, Inc. All rights reserved. 10 IMPROVING OPERATIONAL EFFICIENCIES Advertising & Promotional Execution: • Day-Part Promotional Execution • Audience and Conversion Insights • Proximity Marketing
- 11. © Cloudera, Inc. All rights reserved. 11© Cloudera, Inc. All rights reserved. $174B IN CPG ANNUAL TRADE PROMO SPEND. OF THAT, 17% ($300M) SPENT ON SHELF, YET 68% OF CONSUMER DECISIONS ARE MADE AT THE SHELF. IRI, 2016 OPPORTUNITY: IMPROVED MERCHANDISING EXECUTION INSIGHTS
- 12. © Cloudera, Inc. All rights reserved. 12 IMPROVING CUSTOMER EXPERIENCES • Identify VIPs, ensure high-touch CX (‘clienteling’) • Understand, respond (anonymously) with proactive customer service opportunities
- 13. © Cloudera, Inc. All rights reserved. 13 • Simplifying the dressing room experience • Assist to complete the ‘look’ via cross- sell of matching shoes, accessories • Reduce likelihood of returns IMPROVING CUSTOMER EXPERIENCES
- 14. © Cloudera, Inc. All rights reserved. 14© Cloudera, Inc. All rights reserved. $260B IN MERCHANDISE IS RETURNED TO RETAILERS EACH YEAR. AS MUCH AS 40% OF ONLINE CLOTHING PURCHASES ARE RETURNED. NRF, 2016 OPPORTUNITY: IMPROVED CX + MERCHANDISE RETURNS
- 15. © Cloudera, Inc. All rights reserved. 15 The use cases for CV are promising…driving meaningful business impact, even creating new revenue models Frictionless Checkout Loss Prevention Neighborhood Insights Personalization Customer Insights Customer Engagement Dynamic Merchandising Autonomous DeliverySource: Allure Source: Ahold/Delhaize Source: Caper Source: Orbital Insight Source: Aura visionSource: Trigo-vision
- 16. © Cloudera, Inc. All rights reserved. 16 HOW IT WORKS AND HOW YOU CAN DO IT
- 17. © Cloudera, Inc. All rights reserved. 17 HOW DOES IMAGE RECOGNITION WORK? Detect & Score Convolutional neural network analyzes images and produces actionable representations Compare Representation manifolds can be mapped to find matches, similar, or complementary images Match & Alert Users alerted and shown matches
- 18. © Cloudera, Inc. All rights reserved. 18© Cloudera, Inc. All rights reserved. IMPLEMENTING THESE TECHNIQUES IN YOUR COMPANY • Options for implementing these functionalities in your store • Purchase a point service for the desired functionality • Whether it’s the whole solution or just part of the solution • Hire a company to design and implement a custom solution • Shameless plug for my company • Implement the solutions yourself!
- 19. © Cloudera, Inc. All rights reserved. 19© Cloudera, Inc. All rights reserved. IMPLEMENTING THESE TECHNIQUES IN YOUR COMPANY • Need to define the problem and break it down into smaller parts • There are a few CV modalities where machine learning shines • Think of a modality as a smaller subproblem in machine perception • May have multiple modalities within one problem • A few different models may need to run • Design supporting infrastructure for ML models • Method to feed data into model(s) • Methods to interpret model output(s) • System to chain together multiple models • Translate from model output to human interpretable results
- 20. © Cloudera, Inc. All rights reserved. 20 MACHINE LEARNING AND COMPUTER VISION
- 21. © Cloudera, Inc. All rights reserved. 21 DEEP LEARNING • Many computer vision applications are now becoming successful due to advances in deep learning • A deep learning model for computer vision is specifically designed to exploit the spatial dependencies in image data • Flexible architecture designs to produce desired output for given problem domain
- 22. © Cloudera, Inc. All rights reserved. 22 COMPUTER VISION PROBLEMS • Computer vision problems can often be broken down into several problem types • Can make global inferences about an image • Multiple spatially aware inferences • Pixel level inference for very fine grained output needs • Learning latent representations
- 23. © Cloudera, Inc. All rights reserved. 23 GLOBAL INFERENCE • We may want a model to produce only a single output for a given image • Identify the subject of the image (type of inventory, type of object) • Identify affect of a person’s face • Qualitative assessment of scene or object (does shelf need organization, is inventory damaged) • Infer distance between two objects Gender Affect Eyeglasses Headwear
- 24. © Cloudera, Inc. All rights reserved. 24 SPATIALLY AWARE INFERENCE • We may want a model to produce many outputs for a given image • Describes the locations and classifications for objects in an image • Locate customers in your stores (counts, locations, heatmaps, time lingering, …) • Locate products on a shelf Aura Vision, https://auravision.ai
- 25. © Cloudera, Inc. All rights reserved. 25 PIXEL LEVEL INFERENCE • We may want to produce an output for every pixel in an image • Classify each pixel as what object it’s a part of • Produce mapping or overlay for the entire image • Probability map of object location • Depth map of a room
- 26. © Cloudera, Inc. All rights reserved. 26 LATENT REPRESENTATION LEARNING • Finally, we may want generate some kind of representation of an image that allows us to perform some useful math • Representations capture semantic information • Quantify similarity metrics between two images • Train a model on a closely related task that forces it to learn good representations
- 27. © Cloudera, Inc. All rights reserved. 27© Cloudera, Inc. All rights reserved. EXAMPLE USE CASE
- 28. © Cloudera, Inc. All rights reserved. 28 EXAMPLE USE CASE • Some combination of these tools and pre/post model logic can be used to build a real retain computer vision use-case • Lets assume that we want to build a tool to help our customers by providing clothing recommendations with CV • They come into our store looking for a specific style of clothing • Arrive with image of desired style • Set up kiosk for clothing recommendations • Augment magic mirror
- 29. © Cloudera, Inc. All rights reserved. 29 FASHION PRODUCT RECOMMENDATION EXAMPLE • In order to create this application we’ll need two different machine learning models and some logic around the model outputs • Some capability to interpret the image and find the targeted piece of clothing • Using a segmentation model • Extract targeted article of clothing from image • Create representation of clothing that allows it to be compared to others (model b) • Create representations of entire catalog • Matching algorithm to find closest match to query article
- 30. © Cloudera, Inc. All rights reserved. 30 Segmentoutthevariousarticlesofclothing
- 31. © Cloudera, Inc. All rights reserved. 31 Createsemanticrepresentationsforourentirecatalog
- 32. © Cloudera, Inc. All rights reserved. 32 Extractthetypeofclothingthatwewanttomatchon
- 33. © Cloudera, Inc. All rights reserved. 33 Createsemanticrepresentationsfromthoseextracted Numeric vector representing visual patterns and attributes
- 34. © Cloudera, Inc. All rights reserved. 34 Comparethoserepresentationstothoseofourcatalog A B Best Match
- 35. © Cloudera, Inc. All rights reserved. 35 Puttingittogether
- 36. © Cloudera, Inc. All rights reserved. 36 DEMO
- 37. © Cloudera, Inc. All rights reserved. 37
- 38. © Cloudera, Inc. All rights reserved. 38 BRINGING IT ALL TOGETHER
- 39. © Cloudera, Inc. All rights reserved.39 WHATINDUSTRIALIZED“EDGETOAI”LOOKSLIKE Streaming Ingest Batch Ingest Machine Learning Tools BI Tools and SQL Editors Data Products DATA, METADATA, SECURITY, GOVERNANCE, WORKLOAD MANAGEMENT MACHINE LEARNING DATA ENGINEERING DATA WAREHOUSE OPERATIONAL DATABASE
- 40. © Cloudera, Inc. All rights reserved. THANK YOU Let’s Keep the Conversation Going… Brent Biddulph MD, Retail & CG, Cloudera bbiddulph@cloudera.com +1 425.273.6851 www.linkedin.com/in/brentbiddulph/ @brentbiddulph Florian Muellerklein Data Scientist, Miner & Kasch fmuellerklein@minerkasch.com +1 410.564.1720 www.linkedin.com/in/florian-muellerklein/ @mllrkln