AI and Machine Learning for the Lean Start Up

calculation | consulting
artiﬁcial intelligence
(TM)
c|c
(TM)
charles@calculationconsulting.com

calculation|consulting
Artiﬁcial Intelligence
(TM)

calculation | consulting data science leadership
Who Are We?
c|c
(TM)
Dr. Charles H. Martin, PhD
University of Chicago, Chemical Physics
NSF Fellow in Theoretical Chemistry
Over 20 years experience in Applied Machine Learning (ML) and AI
Developed ML algos for Demand Media; the ﬁrst $1B IPO since Google
Lean Start Ups: Aardvark (acquired by Google), Swoop, …
Internet Giants: eHow, eBay, GoDaddy, …
Wall Street: BlackRock, GLG, …
Fortune 500: Big Pharma, Telecom, …
www.calculationconsulting.com
(TM)
3

the third wave: of AI companies
c|c
(TM)
(TM)
4
What is: an AI Startup?
2010 2014 2018

c|c
(TM)
Overview: ML for Startups
calculation | consulting data science leadership(TM)
5
1. Lean vs Fat: Aardvark vs eHow
2. ML Engineering Models
3. Enterprise AI
4. Lessons from Consulting
5. Q&A

c|c
(TM)
Your funding environment informs your process
Startup Models: Lean or Fat ?
6
2008 Market crashed
Money is tight
Lean startup time
2018 Market booming
Lots of $1B companies
More funding ; ambitious

c|c
(TM)
Your funding environment informs your process
Part 1: Lean vs Fat Startups
7
Aardvark
ex-Googlers
prototype lean startup
Acquired fast
ex-Chairman, MySpace
Huge funding
$1B Unicorn enabled by AI

c|c
(TM)
How you implement AI depends on your funding model
8
2010 Google bought Aardvark for $50 Million
Machine learning enabled social search engine
Featured in Lean Start Up

c|c
(TM)
9
Lean Startup: Aardvark /
featured lean startup acquired by Google ($50M)

c|c
(TM)
eHow / Demand Media ﬁrst $1B IPO since Google
From Startup to IPO: Demand Media
10

c|c
(TM)
eHow / Demand Media ﬁrst $1B IPO since Google
11
2011 Rich Rosenblatt’s Yacht
‘The AdSense’
Bad press for Google
‘

c|c
(TM)
eHow ML Algos caused Google Panda
12
• eHow ML algos soared
• Google adapted (Panda)
• Search changed forever
IPO
Panda
stock price 2011-2012
DMD

c|c
(TM)
Panda-Induced ‘Market Crash’
Google CPC dropped just after Panda
13

c|c
(TM)
ML came later, enabling the business model
14
ML to find users to
answer questions.
Drive adoption
ML predict what questions
people ask on Google.
Drive SEO

c|c
(TM)
Lean or Fat: modern funding models
15
https://techcrunch.com/2016/08/26/growing-up-in-the-intelligence-era/

c|c
(TM)
16
3. Enterprise AI
5. Q&A

c|c
(TM)
Part 2: ML Engineering
17
• Engineering Management Models
• Lean Startup & Learning Circles
• Aardvark case Study
• Wizard of Oz Testing
• From humans to AI
• eHow comparison

c|c
(TM)
Management: what for ML ?
(TM)
18

c|c
(TM)
Engineering Models: V for ML ?
(TM)
19
V model: Very large projects

c|c
(TM)
Engineering Models: Lean Startup
(TM)
20

c|c
(TM)
ML Product Design Cycle: Learning Circles
21
Peter Senge Fifth Discipline SystemsThinking

c|c
(TM)
ML Product Design Cycle: Learning Circles
22
Weekly team learnings Rapid turnaround & feedback
vark !?

c|c
(TM)
Lean ML: In Practice
23
Still within a speciﬁc space; Broader search space using new tools
Early ‘customer interactions’ were not ML based
Fake it till you make it:
ML Things you can prototype, and get feedback from users
Early prototypes used Humans in the loop
People pretend to be products
2010 vs 2018:
ML was hard in 2010
Today, ML is easier, can make simple products

c|c
(TM)
Lean ML: Wizard of Oz Testing
24
Test unimplemented AI with Human in the Loop
Spoliler Alert !

c|c
(TM)
Lean ML: Wizard of Oz Testing
25
Intercept all communications
between users and system
Simulate the system's responses
in real-time
Run live experiments
Learn from user feedback
“Fake it till you make it”
Test unimplemented AI with Human in the Loop

c|c
(TM)
Lean ML: up to full automation
26
Humans
act on
predictions
Full
Human judgements
Actions
w/human
approval
Full
Automation
Logging
12-18 months of runway (funding $$$)

c|c
(TM)
eHow comparison ML SEO
27
Huge ML Risk other peoples data
Generate huge number pages
Collect inbound search trafﬁc
Predict out what people will ask
Create evergreen content $$$
all results are eHow results now

c|c
(TM)
28
3. Enterprise AI
5. Q&A

c|c
(TM)
Part 3: Enterprise ML
29
• Types of AI Startups
• Vertical Startups
• Enterprise B2B Challenges
• Lessons from Consulting

c|c
(TM)
Types of AI Startups: Trade-Offs
30
Full Stack Tech Enablers
Vertical Horizontal
Incremental Disruptive
Full value chain Existing budgets Existing budgets Unique data
Larger TAM Faster sale cycle Easier to sell First mover
Longer training Cold start Existing players Adoption
Bear user costs Partnerships Commoditizes Market size ?
+
-
+
-

c|c
(TM)
31
Full stack products : entire value chain
SME: solve a business problem directly
And SELL into that domain
Proprietary data: defensible IP
AI delivers the core value
Key components of aVertical AI startup
Enterprise AI: Vertical startups

c|c
(TM)
32
Key funding withVertical AI sectors
Enterprise AI: Funding forVerticals

c|c
(TM)
33
Enterprise exits come in cohorts

c|c
(TM)
34
Enterprise exits have less large outliers

c|c
(TM)
35
Where to get data ?
IP sharing: no way
Systems Integrations vs Algo Research
Pilots vs Product Development
Repeat Products vs the Consulting Trap
Overcoming technical debt
Key challenges of doing AI in an Enterprise
Enterprise AI: B2B Challenges

c|c
(TM)
36
3. Enterprise AI
5. Q&A

c|c
(TM)
37
Early wins come fast Improvements very hard
crawl the web and label all that data
or … a new car !
Consulting Lessons: Fast & Slow Gains

c|c
(TM)
38
Recommenders (Amazon, Netﬂix)
Big, fast revenue drivers 30% gains!
Commoditization of algos in 3-5 years
Not always plug-n-play
Core business is not AI; AI is an optimization
Consulting Lessons: Incrementals

c|c
(TM)
39
In 30 days 30% revenue gains
Back of the Envelope Calculations
Simple experiments
Get into production fast
Don’t plan — test
Repeat until it works
Consulting Lessons: Rapid Prototyping
Not guaranteed, results may vary. See your data scientist for details

c|c
(TM)
Part 4: ML Soft Skills
40
• Data Scientists are Different
• French vs Americans

Hiring: Data Scientists are Different
c|c
(TM)
theoretical physics
machine learning / AI specialist
(TM)
41
experimental physics
data scientist - interfaces with business
engineer
software, browser tech, dev ops, …
not all techies are the same

c|c
(TM)
Working Styles: American vs French
42

c|c
(TM)
43
the French are very logical

c|c
(TM)
Working Styles: US vs French
44
in the US, a fight is communication !!!

c|c
(TM)
45
in the US, a fight is communication !!!

c|c
(TM)
Working together: American and French
46
we really want to work with you

c|c
(TM)
47
3. Enterprise AI
5. Q&A

c|c
(TM)
Q&A: ML and AI for Startups
48
http://www.calculationconsulting.com
https://www.youtube.com/calculationconsulting
email: charles@calculationconsulting.com

(TM)
c|c
(TM)
c | c

AI and Machine Learning for the Lean Start Up

More Related Content

What's hot (20)

Similar to AI and Machine Learning for the Lean Start Up (20)

More from Charles Martin (13)

Recently uploaded (20)

AI and Machine Learning for the Lean Start Up