Machine Learning & Data Science come to DAM
1 like429 views
The document discusses the integration of machine learning and data science in digital asset management (DAM), emphasizing the need for automation to handle the growing complexity and volume of digital content. It highlights practical applications of machine learning, such as automated tagging and analytics, to enhance asset performance and searchability. The content also outlines challenges companies face in managing assets and provides recommendations for improving metadata strategies while leveraging AI technologies.
Machine Learning & Data Science come to DAM
- 1. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Machine Learning & Data Science come to DAM Tuesday November 29th , 2016 Sponsored by: Elliot Sedegah Strategy & Product Marketing Adobe Experience Manager Adobe Dr. Jonas Dahl Product Manager Machine Learning & Innovation Adobe
- 2. © 2016 Adobe Systems Incorporated. All Rights Reserved. Machine Learning & Data Science come to DAM November 29, 2016
- 3. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Agenda 3 Understanding Content Velocity Challenges Demystify Machine Learning & Data Science 5 Practical Applications with DAM Tips & Tricks
- 4. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . It’s very getting more costly & complex. 4 85% SLOW TIME TO MARKET 71% HIGHER COST & COMPLEXITY IDC InfoBrief, Proving the Value of Digital Asset Management Forrester, Benchmark Your B2B Content Marketing Strategy And Maturity Creative Team (In-house) Partner Stock Content Employees Content Community & User Generated Content Creative Agency (External) Creative Agency (External) Creative Agency (External) Your Organization Under pressure to create assets and deliver more campaigns, more quickly Creating over 10x the assets today to support increasing channels
- 5. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . We’ve all heard the challenges. “eCom is growing and current process cannot support it” “everything about assets is manual, slow and tedious” “I can’t seem to find anything digital asset that I KNOW exist” “we need a better process to manage assets” “multiple teams are involved in the creation of content with different storage media” “I wish I knew which assets were being used and how they were performing” “cannot scale personalization across channels” “we have no idea how much we spend on creating assets”
- 6. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Maintaining Content Velocity is hard –we will need some help. 6© 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe .
- 7. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . “We are surrounded by data, But starved for insights” 7 - Jay Baer
- 8. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Digital Asset Data is the bedrock of DAM We need lots of Metadata: • Descriptive metadata • Structural metadata • Technical metadata • Administrative metadata • Rights management metadata • Preservation metadata…
- 9. © 2016 Adobe Systems Incorporated. All Rights Reserved.© 2016 Adobe Systems Incorporated. All Rights Reserved. Analytics & Data can enrich every digital asset. DAM ANALYTICS Performance & Usage data: • How many times, where an asset has been used? • Which assets are being used the most? • Which assets have not been used? • Are the top assets being deployed across all channels? • Which assets are resonating the most, creating max impact?
- 10. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . The DAM of the future will be different. ANALYTICS INTELLIGENCE
- 11. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . The DAM of the future will be different. ANALYTICS INTELLIGENCE Scale & Automation Data & Machine learning will automate some manual tasks. Insights Every single asset is a ‘signal’ that can provide insights & recommendations to your business”
- 12. © 2016 Adobe Systems Incorporated. All Rights Reserved.© 2016 Adobe Systems Incorporated. All Rights Reserved. How did we get here? 12 Source: Nvidia Blog: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/ ARTIFICIAL INTELLIGENCE Early artificial intelligence stris excitement. 1950’s 1960’s 1970’s 1980’s 1990’s 2000’s 2010’s MACHINE LEARNING Machine learning begins to flourish Deep Learning Deep Learning breakthroughs Drive Al boom. 0010101010001001001 0010101010001001001 0010101010001001001 0010101010001001001 0010101010001001001 0010101010001001001
- 13. © 2016 Adobe Systems Incorporated. All Rights Reserved.© 2016 Adobe Systems Incorporated. All Rights Reserved. Haven’t we tried this before? What’s changed? 13 CLOUD Highly available and scalable compute DATA REVOLUTION We have much more data to use ALGORITHMS Faster and more intelligent
- 14. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . The old approach: image recognition was difficult and less accurate 14 You must tell the system what to look for…. “This is face because…..” A face has: q 2 eyes q 1 Nose q 1 Mouth q Teeth q 2 Ears Is this a Face? ü2 eyes ü1 Nose ü1 Mouth üTeeth X Ears This method has a higher error rate & is more labor intensive: • Higher Error Rate • Model Creation: Manual No?
- 15. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Deep Learning is more accurate and can generalize concepts 15 Instruction to AI Platform: (1) Here are many faces… (2) Teach yourself to to “see” This is a Face. DAM Yes (95%) model
- 16. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Deep Learning “trains” itself to learn concepts 16 Source: http://www.andrewng.org/publications/ Faces Source: www.eecs.umich.edu/~honglak/icml09-ConvolutionalDeepBeliefNetworks.pdf
- 17. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Study: Deep Learning Drops Error Rate for Breast Cancer Diagnoses by 85% 17 Source: https://blogs.nvidia.com/blog/2016/09/19/deep-learning-breast-cancer-diagnosis/ 350.00% 290.00% 50% 0.00% 50.00% 100.00% 150.00% 200.00% 250.00% 300.00% 350.00% 400.00% Study Pathologist All Model All Model + Study Pathologist (Al + Pathologist)> Pathologist Study Pathologist All Model All Model + Study Pathologist
- 18. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . How Deep Learning classifies images with tags • How it works for images • Takes a Low Res image, pushes it to a neural network (likely in the cloud) • The Cloud returns tags with predictions 18 Deep Learning model
- 19. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . What type of predictive “smart” Tags can be added? 19 Deep learning technology can analyze and extract the semantic content of images based on a training set § Automatically tag images with keywords : § Photo type (macro, portrait, etc.) § Activities (running, skiing, hiking), § Emotions (smiling, crying) § Objects (car, roads, people, etc.) § Locations, Animals, primary colors, and more
- 20. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Machine-generated and User-generated metadata § Existing metadata is preserved § Smart tags are stored separately from user-generated tags 20 Digital Asset User-generated metadata Machine- Generated Digital Asset User- Generated, ADD APPEND Digital Asset User-generated metadata Machine- Generated REPLACE Machine- Generated § Smart tags are added to existing metadata entries § Existing metadata is replaced with machine generated
- 21. © 2016 Adobe Systems Incorporated. All Rights Reserved . Machine-generated and User-generated metadata Use confidence scores to drive search results Sunset (88) Sunset (75) Sunset (71) IMAGE 17.jpg 5 days ago 45.8 KB 612 x 320 IMAGE 15.jpg 5 days ago 33.4 KB 612 x 320 IMAGE 27.jpg 5 days ago 732 KB 1600 x 1067
- 22. © 2016 Adobe Systems Incorporated. All Rights Reserved . Machine-generated and User-generated metadata How to best set confidence threshold • Confidence threshold • Max. number of Smart Tags per asset 22 Balance between search performance and richness Typically 10-30 tags per asset Coffee (93) Café (75) Cup (92) Tea (74) Drink (83) Mug (72) Beverage (76) White (72) Breakfast (75) Expresso (71) Table (75) Brown (69) Black (67) Caffeine (67) Morning (66) Aroma (66) Close-up (65) Saucer (65) Spoon (63) Break (63) Food (62) Italian Culture (32) Agile (32)
- 23. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Example 1: Stock Photography contribution 23 Benefits: - Reduces under-tagged assets - Search Relevancy - Boost search If manual tag matches generated tag How it works: 1. Upload an image 2. Algorithm will analyze it 3. Generating keywords from top images 4. Reorder keywords by relevancy 5. Display the top 5 to review, edit & reorder
- 24. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Example 2: Sourcing Community Content with Filtering & Moderation Moderation during bulk ingest Auto-populate social media content with tags 24 Filter: NSWF images, Off-Brand DISCOVER SOCIAL CONTENT ORGANIZE TOPIC CAMPAIGN LOCATION APPROVED ADMIN MODERATOR RIGHTS
- 25. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . We are creating more content than ever before. 25 3,500 3,000 2,500 2,000 1,500 1,000 500 0 #ofPhotosSharedperDay(MM) Facebook Owned properties 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Daily Number of Photos Shared on Select Platforms, Global, 2005-2015 Source: Snapchat, Company disclosed information, KPCB estimates Note: Snapchat data includes images and video. Snapchat stories are a completion of images and video. WhatsApp data estimated based on average of photos shared disclosed in Q115 and Q1:16 Instagram data per Instagram press release. Messenger data per Facebook (~9.5B photos per months). Facebook shares ~2B photos per day across Facebook, Instagram, Messenger, and WhatsApp (2015) @KPCB Snapchat Facebook Messenger (2015 only) Instagram WhatsApp (2013 onward only) Facebook
- 26. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Example 3: Global Hotel 26 Challenges with content: • Unable to keep up with new content (up to 1,000) per day • Incomplete metadata – “taxonomy doesn’t naturally match users search terms” – Nobody adds metadata for ‘subjective’ descriptions. Such as “calm” • Duplicate/Similar content • Unable to easily filter Customer & User Generated Content Sources content from: • Internal & External Agencies – Photoshoots of global owned- hotel properties, Lifestyle content, Global marketing campaigns • Franchise Partners – Publish local content • Commercial Partners • Branded Products (e.g. credit cards, events,) • Local Tour Guides – Submitting “What’s happening this week” content for approval Users/Customers – Shots from destinations, vacations, conference events
- 27. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Example 4: Global Publisher Sources content from: • Content Producers • Licensed Content • Shared Content from internal brands • Users/Readers – Social Feeds 27 Future Need to Repurpose Example: Tags you need are: “Woman”, “Calm”, “Fitness” Challenges • Not enough resources to tag - Too much content (200-400) per day • Undertagged assets - Giant pool of content that was built up over the years that were never tagged. • Incomplete metadata – “taxonomy doesn’t naturally match users search terms” – No metadata for ‘subjective’ descriptions. (e.g “calm”) Example: Original Asset List Tags: Article #123 Studio, Rights, “Yoga Studio”
- 28. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Example 5: Consumer Electronics Company Assets are used in multiple places • Report Asset usage on Main Website, Mobile Apps, Regional Websites & 3rd- Party & Partner Sites Assets commissioned from multiple agencies • Need an understanding of the performance of the assets as attributed to the creating entities. 28 Opportunities with Asset Insights • Tracking usage & performance of assets • use data to drive search results • Understand agency and subcontractor performance • Compare Campaign performance • Provide feedback to content creators • Discover which assets are being reused/repurposed/high-performing
- 29. © 2016 Adobe Systems Incorporated. All Rights Reserved . Other applications for this technology Custom Smart tags • User tags a small number of images • System is trained based on these images and can now tag similar images Similarity Search • Deep learning based similarity search is much better than traditional similarity search because it is based on concepts Recommendations • Recommendations will be a more important part of how marketers discover assets Video • Smart tagging for video will also account for the temporal aspect of video • Analyze and Identify key frames, audio, and more.
- 30. © 2016 Adobe Systems Incorporated. All Rights Reserved.© 2016 Adobe Systems Incorporated. All Rights Reserved. Artificial Intelligence & Data Science will be an essential tool to increase the value of the assets we manage 30 MOVE FASTER MORE EFFICIENT MORE INTELLIGENCE
- 31. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Audience Participation: Can you predict the tags? 31
- 32. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Audience Participation: What tags would you assign to this image? 32
- 33. © 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe . Next Steps & Practical Advice Build the Business Case • Think productivity, content reuse/savings 33 Determine what data would make a difference for your stakeholders • Performance data? Search data? Don’t abandon your metadata & taxonomy strategy ! • Instead, think about how to add and improve it! Start Small & Take a Phased Approach • Start with a Focused POC that will expand in scope over time Don’t create another data silo with your DAM § Connect to your broader organization’s data sources
- 34. © 2016 Adobe Systems Incorporated. All Rights Reserved. Questions? 34
- 35. © 2016 Adobe Systems Incorporated. All Rights Reserved. 35 @AdobeExpMgr facebook.com/AdobeMarketingCloud adobe.com/go/dam 877-722-7088 enterpriseADM@adobe.com Stay connected
- 37. © 2016 Adobe Systems Incorporated. All Rights Reserved.