The road to pragmatic application of AI.pdf
1 like225 views
The document discusses the evolution and application of AI and big data across various industries, highlighting the shifts from initial hype to practical adoption and the associated challenges in data usage and innovation. It emphasizes the importance of effective data governance, collaboration, and the need for organizations to embrace a product-driven approach to harness the full potential of AI and data. The text concludes by advocating for a focus on grassroots business value and iterative development processes to drive successful data and AI initiatives.
The road to pragmatic application of AI.pdf
- 1. www.scling.com The road to pragmatic application of AI Lars Albertsson, Founder, Scling Nordic Software Summit 2024-08-20 1
- 2. www.scling.com Hypes disrupt some fields 2 Hype Enabled / boosted Accessible to anyone Disrupted Personal computer Corporate digital tools Yes Office work, design Internet Machine communication Finance, telecommunications Expert systems Rule-based AI, anomaly detection, classification Manufacturing, operations, fraud prevention, games, medicine Web Broadcast human communication Yes Media, marketing, commerce Semantic web Machine communication? Social web Peer human communication Yes Media, marketing, culture, politics IoT Small machine communication Manufacturing, energy, security Smartphones Personal digital tools Yes Point-of-sales, field work, travel, eating Big data Basic AI, personalisation, classification, patterns Media, marketing, commerce, operations, research Deep learning Object recognition, robot control, data extraction Control systems, vehicles, medicine, translation Blockchain Cryptocurrencies Yes Criminal finance Generative AI Text & media generation Yes Media, games, product user interfaces. Fraud, malicious politics. Impact most businesses
- 3. www.scling.com Big data adoption 3 ● 2003-2007: Only Google ● 2007-2014: Hadoop era. Highly technical companies succeed and disrupt. ● 2015-2019: Enterprise adoption. Big data gone from Gartner hype cycle. “New normal” ● 2019: Many enterprises in production, but big data and machine learning ROI still confined to high-tech.
- 4. www.scling.com Enabling innovation 4 "The actual work that went into Discover Weekly was very little, because we're reusing things we already had." https://youtu.be/A259Yo8hBRs https://youtu.be/ZcmJxli8WS8 https://musically.com/2018/08/08/daniel-ek-would-have-killed-discover-weekly-before-launch/ "Discover Weekly wasn't a great strategic plan and 100 engineers. It was 3 engineers that decided to build something." "I would have killed it. All of a sudden, they shipped it. It’s one of the most loved product features that we have." - Daniel Ek, CEO
- 5. www.scling.com Data not acted upon - value left on table 5 The car has sensors for precipitation, temperature, and window state. I would have liked to receive a mobile app warning when all windows were down during a snowy night.
- 6. www.scling.com Data not acted upon - value left on table 6 Changing to winter tyres at Bilia. When I arrived, the staff could see that mechanics were currently not fully booked, and offered me a discounted front wheel adjustment, which I accepted. Data value taken from table. Our Volvo incorrectly activates the automatic safety brake again. This is a known problem since years back. I have no efficient channel to report time and circumstances for this occasion, allowing Volvo to get more data on the problem, and me to feel confident that the issue is worked on. After a repair at Bilia of the rear view mirror, the car does not properly detect cars in front. Camera near mirror is suspected. Bilia cannot obtain camera measurements or car detection statistics from Volvo, so a cycle of trial and error repairs follows, taking me to the mechanic multiple times. The car has sensors for precipitation, temperature, and window state. I would have liked to receive a mobile app warning when all windows were down during a snowy night. I make the effort to submit a detailed bug report regarding Volvo's known over-aggressive automatic safety brake problem, in order to provide more data for debugging. I receive no feedback from the process, and lose interest. I send a design suggestion to Volvo with pics of a hacked solution for a luggage problem. A simple hook could solve it. I do not receive "We have assigned your suggestion id X, and will get back if it is implemented." I will not send another, and feel less connected to the brand. Wife unlocks Volvo. Driver seat switches to her position. I take the driver's seat, pull it back, and change to my profile. Car switches seat back to her position, and I have to pull it back again. When seat heating was automatically activated, "Increase seat heating" audio command turns it off. I make additional commands to get to the desired state. Measurements could identify repeated audio or UX commands. The Volvo phone app warns about many things: interrupted charging, car parked but not being charged, etc. But not about the roof and windows left open in the rain. Rear view mirror electronics now need a repair. I change the audio balance in the Volvo with six screen clicks. Measurements could indicate that long click sequences at high speed indicates poor user interface. Wife has driven our Volvo. Mirrors and seats automatically move to my position, except for the right mirror, which I need to correct again, as I have for years. Volvo could have measured and detected irregular changes of settings, and found this bug. The car keeps informing me that there is a software upgrade, but no matter what I do, nothing gets upgraded. Map updates fail to auto install. Support cannot obtain diagnose information. Manual installation attempts fail without error messages. Navigation suggests a road that eventually becomes so narrow and close to the beach that it is dangerous to drive a 2 ton SUV. With speed statistics from other vehicles, a better route could have been suggested. Neighbour jumps in their Volvo, talks to husband through window, and drives off. On arrival she discovers that she has no car key - husband's key was close enough to start car. An early warning that key is no longer in proximity would have been valuable.
- 7. www.scling.com 1. Log in, search for product X ○ Popular items first 2. Find X in product catalog ○ Take me to shop 3. Put in cart, delivery? ○ I am logged in 4. ... An e-shopping tale 7 Full story: “Avoid artificial stupidity” @ mapflat.com/blog 1. Log in, search for product X ○ X + 100s of accessories, random order 2. Find X in product catalog ○ No link to web shop 3. Put in cart, delivery? ○ Ask for address ○ Ask for customer club number 4. …
- 8. www.scling.com Data-centric innovation ● Need data from teams ○ willing? ○ backlog? ○ collected? ○ useful? ○ quality? ○ extraction? ○ data governance? ○ history? 8
- 9. www.scling.com Big data - a collaboration paradigm 9 Data platform Data lake Data democratised Value stream-aligned teams - few handoffs
- 10. www.scling.com From craft to process 10
- 11. www.scling.com From craft to process 11 Multiple time windows Assess ingress data quality Repair broken data from complementary source Forecast based on history, multiple parameter settings Assess outcome data quality Assess forecast success, adapt parameters
- 12. www.scling.com IT craft to factory 12 Security Waterfall Application delivery Traditional operations Traditional QA Infrastructure DevSecOps Agile Containers DevOps CI/CD Infrastructure as code
- 13. www.scling.com Security Waterfall Data factories 13 Application delivery Traditional operations DevSecOps Traditional QA Infrastructure DB-oriented architecture Agile Containers DevOps CI/CD Infrastructure as code Data factories, data pipelines, DataOps
- 15. www.scling.com Sustainable production machine learning 15 Multiple models, parameters, features Assess ingress data quality Repair broken data from complementary source Choose model and parameters based on performance and input data Benchmark models Try multiple models, measure, A/B test
- 16. www.scling.com AI & data priorities 16 Application AI / data-powered feature Probe Data platform Probe Probe Data sources Fast data flow development iterations and feedback loops Fast integration iterations Minimal operational overhead Ways of working & processes Data platform & tech stack Importance Supporting
- 17. www.scling.com Capability KPIs DORA research / State of DevOps report: ● Lead time for changes ● Deployment frequency ● Change failure rate ● Time to restore service Small elite ~1000x span 17 Observed differences in data organisations: ● Lead time from idea to production ● Time to mend / change pipeline ● Number of pipelines / developer ● Number of datasets / day / developer Small elite 100 - 10000x span
- 18. www.scling.com Efficiency gap, data cost & value ● Data processing produces datasets ○ Each dataset has business value ● Proxy value/cost metric: datasets / day ○ S-M traditional: < 10 ○ Bank, telecom, media: 100-1000 18 2014: 6500 datasets / day 2016: 20000 datasets / day 2018: 100000+ datasets / day, 25% of staff use BigQuery 2021: 500B events collected / day 2016: 1600 000 000 datasets / day Disruptive value of data, machine learning Financial, reporting Insights, data-fed features effort value
- 19. www.scling.com Myth: ● We are all doing quite ok ● 2-10x leader-to-rear span The great capability divide 19 capability in X # orgs
- 20. www.scling.com Myth: ● We are all doing quite ok ● 2-10x leader-to-rear span The great capability divide 20 capability in X # orgs capability in X # orgs Reality: ● Few leaders in each area ● 100-10000x leader-to-rear span
- 21. www.scling.com Size = effort Credits: “Hidden Technical Debt in Colour = code complexity Machine Learning Systems”, Google, NIPS 2015 Machine learning product efforts 21 Configuration Data collection Monitoring Serving infrastructure Feature extraction Process management tools Analysis tools Machine resource management Data verification ML
- 22. www.scling.com Boring, valuable data-powered product Data-driven products vs business value 22 Configuration Data collection Monitoring Serving infra Feature extraction Process management tools Analysis tools Exciting prototype without business value Machine resource mgmt Data verification ML < Insert meme with drooling person here >
- 23. www.scling.com Generative AI → more complex data engineering 23 Input LLM Output Generative AI is simple? Want multi-modal? Correct facts too? Safety mechanisms to avoid undesirable use? Add relevance? → Retrieval augmented generation (RAG) Input LLM Output Search engine Document corpus Document snippets Nvidia: Nemo Guardrails Langchain: Multi-modal RAG USTC + UCLA + Google: Corrective RAG
- 24. www.scling.com Data mature company product portfolio 24
- 25. www.scling.com Future data engineering - "data factory engineering" 25 DW ~10 year capability gap Enterprise big data failures "Modern data stack" - traditional workflows, new technology 4GL / UML phase of data engineering Data engineering education
- 26. www.scling.com 10 failure / success factors - flow 26 Friction creating Flow enabling Description Organisation by functionality Aligned with the value stream Whether product development that requires multiple skill sets demands engagement from multiple teams or can be done within a single team. Human processes Technical processes Whether there is bias towards changing the way that humans work vs the way that machines work when facing a challenge. Push Pull Whether work is initiated and prioritised driven by decisions to implement capabilities or milestones, or by specific business value needs. More things Less things Whether technical systems and processes are seen as assets to grow and nurture or as liabilities to remove unless they bring sufficient value. Project Product Whether activities are seen as time-limited efforts or as team-owned artefacts that evolve in an iterative manner.
- 27. www.scling.com 10 failure / success factors - governance 27 Friction creating Flow enabling Description Denied by default Trusted by default Whether a request needs to be sufficiently motivated before being granted. Opaque by default Transparent by default Whether information about systems and products is accessible on a need-to-know basis or generally available. Siloed by default Accessible by default Whether data is locked up in decentralised, heterogeneous systems well guarded by owners or available in centralised, homogeneous systems. Risk management by rituals Risk management by golden paths and guard rails Whether risks (security, compliance, reliability, etc) are managed by deciding on a single way to do things and forcing teams to stick to it, or by decentralising responsibility but providing suggested, paved paths and warning systems. Direction & control Thrust Whether more effort is spent figuring out in detail what to do and how as opposed to improving speed forward so more things can be tested with trial and error.
- 28. www.scling.com Don't build. Grow. 28 ● Every data / AI project has either ○ failed ○ cost a fortune ● Every leading data company has ○ solved product challenges at hand ○ improved process / ways of working ○ had data / AI success through enabled teams ● "CorpOps" - start where you are ○ Data & automation for current challenges ○ Data collection grows … ○ Data gradually refined and prepared … ○ … while value is created Technology change Application change Negative ROI Positive ROI We must be AI first! Solve today's challenges with data & automation Sustainable data+AI feature development
- 29. www.scling.com 29 Taking the CorpOps path ● Let grassroots business value drive automation ○ Seek low-hanging fruit ○ E.g. things done with spreadsheets ○ "What if you had 50 junior assistants?" ● Everything is an engineering challenge ○ Software engineers in majority ○ Product, data, QA Remember the priorities: 1. Fast development iterations 2. Fast integration iterations 3. Minimal operational cost 4. Processes that support 1-3 5. Platform that supports 1-4 Avoid the 10 frictions: 1. Org by value stream, not functionality 2. Change technology, not humans 3. Pull, not push 4. Less, not more 5. Product, not project 6. Trust, don't deny 7. Transparent, not opaque 8. Accessible, not siloed 9. Guard rails, not rituals 10. Thrust, not control
- 30. www.scling.com 30 The AI success paradox Remember the priorities: 1. Fast development iterations 2. Fast integration iterations 3. Minimal operational cost 4. Processes that support 1-3 5. Platform that supports 1-4 ● Let grassroots business value drive automation ○ Seek low-hanging fruit ○ E.g. things done with spreadsheets ○ "What if you had 50 junior assistants?" ● Everything is an engineering challenge ○ Software engineers in majority ○ Product, data, QA Avoid the 10 frictions: 1. Org by value stream, not functionality 2. Change technology, not humans 3. Pull, not push 4. Less, not more 5. Product, not project 6. Trust, don't deny 7. Transparent, not opaque 8. Accessible, not siloed 9. Guard rails, not rituals 10. Thrust, not control By focusing more on your product and less on AI, you are more likely to succeed with AI.
- 31. www.scling.com ● Other disciplines? ○ Shared innovation ○ E.g. Foxconn, Autoliv ● Tech leaders ○ Grow, don't build ○ Product-driven 31 Data factory adoption Observations: ● Industrial success requires veterans ● Not enough factory experience around ● Minimal flow of veterans to incumbents / consultants ● Artisanal tools (Data warehouse / low-code / …) inadequate for domain-specific AI ● Industrial tools not helpful without veterans Belief: We need new ways of collaboration. Customer Data factory Data platform & lake data domain expertise Value from data! Data innovation as leaders do it Learning by doing, in collaboration
- 32. www.scling.com What we learnt 32 Success under good circumstances ● Match Spotify's numbers per developer ● 10-1000x client's own capability ● Data + AI innovation on par with leaders Challenge: Data capability → innovation ● Domain-specific competence ● "Innovation building blocks" needed ○ Agile, pull vs push, iterative ○ Digital thinking ○ Aligned, cross-functional teams ○ Product focus ○ Value chain alignment Greater challenge: "How hard can it be?" ● Unawareness of data divide
- 33. www.scling.com What we learnt, what we need 33 Humble, but competent clients ● Data with value potential ● Domain experts ● Innovation building blocks ● Humble regarding data divide Other ways to slice the challenge? Partners that bridge innovation gap ● Digitalisation in some vertical ● Interested in new business models Success under good circumstances ● Match Spotify's numbers per developer ● 10-1000x client's own capability ● Data + AI innovation on par with leaders Challenge: Data capability → innovation ● Domain-specific competence ● "Innovation building blocks" needed ○ Agile, pull vs push, iterative ○ Digital thinking ○ Aligned, cross-functional teams ○ Product focus ○ Value chain alignment Greater challenge: "How hard can it be?" ● Unawareness of data divide