AV Combinator H4D 2020 Lessons Learned

AV Combinator
Original problem statement
In-Q-Tel wants to better understand the current
state of autonomous technology development and
where it will be in five years so that it can be a
conduit of industry expertise to the US Government
and the Defense/Intelligence Community
Current problem statement
In-Q-Tel wants to know which set of recommendations for
regulations, initiatives, test regions, smart infrastructure,
information platforms or services can help accelerate the
development of, and authorization for Level 5 fully
autonomous driving. How might those recommendations be
delivered to the USG (e.g., policy guidelines, proposals,
incentives, future investments)?
Advisor and Mentor Team
Mark Breier, Problem Sponsor (IQT) - LtCol Jason Campbell, Defense Advisor - LtCol Donnie Hasseltine, Defense Advisor
Teresa Briggs, Business Advisor - Mark Rosekind, Chief Safety Innovation Officer @ Zoox
Total interviews
102
Michael Nehmad Richard Sun Julius Niehaus Troy Lawrence
Strategy / Product
(Technology Due Diligence)
Government and
Regulatory Affairs
Strategy / Product
(Technology Due Diligence) R&D Engineer
Jenn Hu
Government
and Regulatory Affairs
Combined 18 years of
experience in mobility
& transportation
Experience from 3
continents
Background in 7
Stanford programs

AV Development is a Hype Cycle Technology
Quick Wins Are Easy But A Complete Solution Is Hard
While AV companies managed to solve many situations fairly quickly, the “edge cases” take
a lot longer to solve

Our Emotional “Hype Cycle” Journey
We followed a similar journey throughout the course
Teammorale

Week 0: Team Formation
“There is magic in every beginning”
Teammorale
Met with Program
Sponsor, In-Q-Tel
Learned more about the
problem statement and IQT’s AV
priorities

Teammorale
Met with Program
Sponsor, In-Q-Tel
Interviewed AV
startups, robotics
companies, OEMs,
policymakers
priorities
Key takeaway: AV industry’s
relationship with regulation
(frustrating) and defense
(sparse)

Teammorale
Met with Program
Sponsor, In-Q-Tel
Conducted team
huddle + discussed
Zoom Norms
Interviewed AV
startups, robotics
companies, OEMs,
policymakers
priorities
Key takeaway: AV industry’s
relationship with regulation
(frustrating) and defense
(sparse)
Clarified individual teammate’s
goals for H4D

KEY PARTNERS
1) Full Stack AV Companies
2) Ride Hailing Companies
3) Automakers
4) AV-related Startups
5) US Policy Makers
6) In-Q-Tel
-------------------------------------------
Partners 1, 2, 3, and 4
- They provide: information on
technology readiness and
“answers” to policymaker Q’s
- We provide: connection w/ &
influence over policymakers
Partners 5 and 6
- They Provide: Their time and
Q’s that matter most to them
- We Provide: Easy to digest
answers and tech due diligence
KEY RESOURCES
- MN and JN: Broad rolodex at
nearly every OEM & AV startup
- TL: Engineering know-how
- RS: Deep understanding of
state and local govt policy
- JH: Int’l context & experience
In-Q-Tel: Contacts
VALUE PROPOSITIONS
(Policymakers):
- Pain: not informed enough to
advocate for productive policies
- Pain: time consuming to
understand complex technology
- Pain: AV tech players are
regulatory entrepreneurs and
avoid sharing competitive info
→ Gain: Easy, quick way to
increase knowledge and get Q’s
answered
(AV Players):
- Pain: Slow/destructive policy
making hinders AV deployment
→ Gain: Understand the needs
of policymakers and efficient
comms channel to them
KEY ACTIVITIES
- Discussions with policy makers
to craft AV information template
- Conversations with private
sector players to show
advantage of working with
rather than around government
- 2-sided pipeline development
(mobile app, email digest, tbd…)
MISSION ACHIEVEMENT/IMPACT FACTORS
1) # of active platform users over time
2) Anecdotal evidence from policy makers once(if) successful legislation occurs
3) How long after (if any lag at all) the tech is deemed ready, are US consumers
able to finally take rides at scale
* Ultimately hard to compare to how long legislation would have taken without
platform
Designed by: The H4Di Team Date: Apr 7
2020
Version:1.1
DEPLOYMENT (TBD)
(Ideas Include):
- 2-sided mobile app
- 2-sided website
- Weekly email digest
BUY-IN & SUPPORT
- 2-sided market requires buy-in
from both sides (Policymakers
and AV Players)
- Both need to contribute to
pipeline for value to be accrued
- Our research will seed version
1.0
→ Will be up to both sides to
keep platform up-to-
date/ongoing
MISSION BUDGET/COST
- Costs: Should be very minimal. The most valuable resources we’re leveraging are
time and information. Potentially small supporting server costs ($1,000s)
- Timeline: Develop working base model by end of spring quarter. Project will be
refined as technology matures. Initial focus will be on components, building up to
complete vehicles over time.
BENEFICIARIES
1) US Consumers: The quicker
that the developers and policy
makers can coordinate, the
quicker the public will be able to
take advantage of AV
technology (improved safety,
more free time, less pollution,
etc…)
2) AV Players: The quicker that
supportive US policy is rolled
out, the quicker they can start
reaping the economic rewards
of their development
3) US Defense Department / In-
Q-Tel: Upfront comms with
policymakers & innovators
ensures faster, more effective
application of AV tech to
defense sector

KEY PARTNERS
1) Full Stack AV Companies
2) Ride Hailing Companies
3) Automakers
4) AV-related Startups
5) US Policy Makers
6) In-Q-Tel
-------------------------------------------
Partners 1, 2, 3, and 4
- They provide: information on
technology readiness and
“answers” to policymaker Q’s
- We provide: connection w/ &
influence over policymakers
Partners 5 and 6
- They Provide: Their time and
Q’s that matter most to them
- We Provide: Easy to digest
answers and tech due diligence
KEY RESOURCES
In-Q-Tel: Contacts
VALUE PROPOSITIONS
(Policymakers):
- Pain: not informed enough to
advocate for productive policies
- Pain: time consuming to
understand complex technology
→ Gain: Easy, quick way to
increase knowledge and get Q’s
answered
(AV Players):
- Pain: Slow/destructive policy
making hinders AV deployment
→ Gain: Understand the needs
of policymakers and efficient
comms channel to them
KEY ACTIVITIES
- Discussions with policy makers
to craft AV information template
- Conversations with private
sector players to show
advantage of working with
rather than around government
- 2-sided pipeline development
(mobile app, email digest, tbd…)
1) # of active platform users over time
2) Anecdotal evidence from policy makers once(if) successful legislation occurs
* Ultimately hard to compare to how long legislation would have taken without
platform
Designed by: The H4Di Team Date: Apr 7
2020
Version:1.1
DEPLOYMENT (TBD)
(Ideas Include):
- 2-sided mobile app
- 2-sided website
- Weekly email digest
BUY-IN & SUPPORT
from both sides (Policymakers
and AV Players)
- Both need to contribute to
pipeline for value to be accrued
- Our research will seed version
1.0
→ Will be up to both sides to
keep platform up-to-
date/ongoing
MISSION BUDGET/COST
- Timeline: Develop working base model by end of spring quarter. Project will be
refined as technology matures. Initial focus will be on components, building up to
complete vehicles over time.
BENEFICIARIES
1) US Consumers: The quicker
that the developers and policy
makers can coordinate, the
quicker the public will be able to
take advantage of AV
technology (improved safety,
more free time, less pollution,
etc…)
2) AV Players: The quicker that
3) US Defense Department / In-
Q-Tel: Upfront comms with
policymakers & innovators
ensures faster, more effective
application of AV tech to
defense sector
MMC Hypothesis #1:
Our core goal was to close the
communication gap between
policymakers and private AV
companies

Week 1: Excitement and Expectation
Our expectations peaked when we received time and insight
from experts above and beyond our expectations
Teammorale

Week 1: Excitement and Expectation
Our expectations peaked when we received time and insight
from experts above and beyond our expectations
Teammorale
Robert Grant, VP of
Government Affairs
Tudor Achim, CTO
Oliver Cameron, CEO
Paul Perrone, CEO
AV testing engineer
Product manager
ADAS
Jonathan Tame,
FMCSA
consultant
Daimler/Mercedes-
Benz Manager
Marc Berman, CA
State Assembly
Member:

Weeks 2-3: Coming Back to Earth
But we also started to realize the breadth of the problem
Teammorale

In-Q-Tel wanted us to boil the sea, the land, and the air
Original problem statement
In-Q-Tel wants to better understand the current
state of autonomous technology development
and where it will be in five years so that it can
be a conduit of industry expertise to the US
Government and the Defense/Intelligence
Community
How can each layer
of the AV stack be
leveraged for defense
applications?
Land
Air
Sea

We Preferred to “Stay Grounded” on Land
Current problem statement
In-Q-Tel wants to know which set of recommendations for
regulations, initiatives, test regions, smart infrastructure,
information platforms or services can help accelerate the
development of, and authorization for Level 5 fully
autonomous driving. How might those recommendations be
delivered to the USG (e.g., policy guidelines, proposals,
incentives, future investments)?
Provide
Recommendations to
Accelerate Level 5
Full Autonomous
Driving
Land

We responded by whiteboarding two targeted tools one
Federal (Report to Congress), one state (State Website)
Autonomous driving L4/L5:
How to supercharge
innovation and safeguard US
leadership
Report to Congress
June 2020
Human Driven Vehicles
1) Miles Drive: ______
2) Crashes/100K miles: ______
3) Occupant Deaths/100K miles:
______
4) External Deaths/100K miles
______
5) Property Damage $/100K
miles: ______
6) Bodily Damage $/100K miles:
Autonomous Vehicles
______
______
miles: ______
Updated Daily by the State of Arizona
Trial Days Elapsed: ____

We responded by whiteboarding two targeted tools one
Federal (Report to Congress), one state (State Website)
Autonomous driving L4/L5:
How to supercharge
innovation and safeguard US
leadership
Report to Congress
June 2020
Human Driven Vehicles
______
______
miles: ______
Autonomous Vehicles
______
______
miles: ______
Updated Daily by the State of Arizona
Trial Days Elapsed: ____
Tool 2: State
State Based Website
Tool 1: Federal
Report to Congress

But we were not really happy with yet another report...

Week 4-6: Rapid Ascent
As we started week 4 things improved… dramatically
Teammorale

Week 4-6: Rapid Ascent
We onboarded Dr. Mark Rosekind as a mentor!
Chief Safety Innovation Officer, Zoox
15th Administrator of NHTSA (2014-2016)
Board Member at NTSB (2010-2014)
Stanford Grad and Guest Lecturer
Nobody Better

Legislators need
data on safety to
allow AVs on the
road for testing and
eventually, for
commercialization
AV innovators need
access to real-
world testing
environments to
gather data.
Gathering this data
inherently risks lives
Goal: Break the chicken and egg cycle by formulating proactive safety standards, by identifying effective
measures to gather this data in a safe, and demonstrative manner, and by communicating it effectively
Week 4: Rapid Ascent Part 1
Eureka! PIVOT from regulation to safety

Week 5 & 6: Rapid Ascent Part 2
Evolved into a modular approach to safety
Aggregated Safety
Assessment
VS.
Human vs. AV Driving Test
Supporting Industry
Consortium Safety
Standards (ISO SOTIF, UL
4600)
Traditional Safety Metrics
(Crashes, Injuries,
Fatalities)
Traditional Vehicle Crash
Testing
Anonymized Database of
Best Practices
Anonymized Database of
Safety Metrics
Holy Grail:
De-anonymized
linking of best
practices to
corresponding
safety metrics
Proactive Safety Metrics

KEY PARTNERS
1) AV Companies: Full Stack/Startup
2) Automakers
3) US Policy Makers
4) In-Q-Tel
5) Insurance Companies
6) Safety Advocates
-------------------------------------------
Partners 1 and 2
- They provide: info on tech readiness
and “answers” to policy Q’s
- We provide: Accelerated AV
commercialization timeline
Partners 3 and 4
- They Provide: Their time and Q’s that
matter most to them
- We Provide: Confident, quick, effective,
policy-making
Partners 5 and 6
- They provide: pressing safety
concerns:
- We provide: policy memo for solutions
KEY RESOURCES
- In-Q-Tel: Contacts
- MR: Contacts + Expertise
VALUE PROPOSITIONS
(Policymakers):
- Pain: need more data in order
to conduct cost-benefit analysis,
allow AVs on the road for testing
- Pain: Balancing interests of
beneficiaries and saboteurs (eg
trial lawyers, safety advocates)
→ Gain: Accelerate safety and
economic benefits of AVs
(AV Players):
- Pain: Need testing to gather
data for testing (chicken-egg
problem).
→ Gain: Accelerated testing,
followed by commercialization
KEY ACTIVITIES
- Understand pain points in
safety regulation and collection
of data to conduct cost-benefit
analysis on policy side
- Formulate proactive safety
metrics for NHTSA, FMCSA,
Congress, OEM/AV developers
1) Leaders in policy, AV industry co-sponsoring/backing policy memo
2) # of beneficiary groups that adopt X# of our modules
3) Convergence vs. divergence on modules over time across groups
Designed by: The H4Di Team Date: May 26
2020
Version:1.2
DEPLOYMENT
- Industry + Government: Policy
memo outlining full details of
modular safety suite + tracking
website
- Public: Disseminate abstract
through PAVE, AAA, etc
BUY-IN & SUPPORT
from all sides (Policymakers,
Organized interests, and AV
Players)
- Modular nature enables all
groups to buy-in earlier than
requiring agreement. Leads to
faster development and roll-out
MISSION BUDGET/COST
- Timeline: Weeks 2-4) Information gathering and defining the problem. Weeks 5-7)
Ideate on and propose potential MVP solutions to the problems. Week 8) Finalize top-
3 policy suggestions and delivery method. Weeks 9-TBD) Propose/Deliver a way to
enact at least one of our top-3 policy suggestions
BENEFICIARIES
1) Policy-Makers: (Jack
Danielson, Executive Director
NHTSA & Heads of state DMVs,
Bernard Soriano) Identify and
accelerate the most relevant
benefits of AVs to their
constituents
2) Head of Govt Affairs of AV
Companies: The quicker that
3) US Consumers (Suburban
parents): The quicker that the
developers and policy makers
can coordinate, the quicker the
public will be able to take
advantage of AV technology
(improved safety, more free
time, less pollution, etc…)

Week 7-9: Enhance and Refine
We started receiving great feedback for our proposed solution
Teammorale
Dr. Mark
Rosekind:
“Nothing like
this exists
today. You guys
are now on the
right path.”
Director of CA
DMV:
“This is
interesting. Flesh
it out more and I’d
like to review it
again in two
weeks.”
VP of Comms &
Advocacy:
“This is unique.
I’d love to see
the final report.
PAVE could
disseminate.”

Aggregated Safety Assessment
Vehicle-related
metrics
Anonymized dataAudit Proactive safety
metrics
Physical
certification
Traditional Vehicle
Crash Testing
Traditional Safety
Metrics (Crashes,
Injuries, Fatalities)
Anonymize
d Database
of Safety
Metrics
Industry Consortium
Safety Standards (ISO
SOTIF, UL 4600)
Internal safety
process certification /
audit
Human vs. AV
Driving Test in critical
scenarios
Real world test drive
Anonymize
d Database
of Best
Practices
Holy Grail:
De-anonymized linking
of best practices to
safety metrics
Proactive safety
metrics (e.g. based
on simulation)
VS.
“Traditional” safety metrics New safety metrics
We continued to revise our MVP

But we were not the first to come to this realization
End-to-end safety
framework centered
around a database that
stores driving scenarios
Data center that
neutrally collects AV
data, making it available
to third-parties
UL4600 describes safety
principles and
processes for evaluating
fully autonomous
vehicles

We also realized there are significant non-market challenges
Consumer Support
Legal Support
Financial Support

Collecting data Using data Supplying data
Collection of safety data from
multiple sources incl. traditional
safety metrics and new sources
1 2 3
Federal level
State level
Preparation, evaluation,
analysis, and processing of
safety data to extract
actionable insights
Communication of safety data
in digestible format to public
and federal and state
regulators
To overcome these challenges, we needed to refine our value
proposition

Policy
Week 11: The Future
No More Chicken or the Egg
Partners
Parallel Process 3Ps: Policy, Partners, and Product
Federal level
State level
Product
Team AV
Combinator
AV Safety
Systems
LAB

Thank you!
With special thanks to our mentor, advisor, and sponsor team:
Mark Breier, Problem Sponsor (IQT)
LtCol Jason Campbell, Defense Advisor
LtCol Donnie Hasseltine, Defense Advisor
Teresa Briggs, Business Advisor
Mark Rosekind, Chief Safety Innovation Officer @ Zoox

AV Combinator H4D 2020 Lessons Learned

More Related Content

What's hot (20)

Similar to AV Combinator H4D 2020 Lessons Learned (20)

More from Stanford University (20)

Recently uploaded (20)

AV Combinator H4D 2020 Lessons Learned

Editor's Notes