Racing with Artificial Intelligence

Wednesday, July 31, 2019
1. Intro & Activity Update
2. Community Open Mic
3. Alex Barbosa Coqueiro - Public
Sector SA Manager @ AWS
Canada: “Racing with AI”
4. Networking
1
ServerlessToronto.org Meetup Agenda

Why we do what we do?
2
Serverless is New Agile
Serverless Dev (gluing
other people’s APIs
and managed services)
We're obsessed
helping Startups (and
creating meaningful
MVPs & products)
We build bridges
between Serverless
Community (“Dev leg”),
and Front-end & Voice-
First folks (“UX leg”),
and empower UX
developers
Achieve agility NOT by
“sprinting” faster (like in
Scrum), but by working
smarter (by using
bigger building blocks
and less Ops)

Why AI/ML topics at Serverless meetup?
3
AWS Machine Learning Stack

Manning Publications 2019 giveaways:
1. www.manning.com/books/serverless-applications-with-nodejs
2. www.manning.com/livevideo/production-ready-serverless
3. www.manning.com/livevideo/production-ready-serverless
4. www.manning.com/livevideo/serverless-applications-with-AWS
5. www.manning.com/livevideo/serverless-applications-with-AWS
6. www.manning.com/books/serverless-architectures-on-aws
7. www.manning.com/books/http2-in-action
8. www.manning.com/books/event-streams-in-action
9. www.manning.com/books/the-design-of-everyday-apis
10. www.manning.com/livevideo/graphql-in-motion
11. www.manning.com/books/voice-applications-for-alexa-and-google-assistant
12. www.manning.com/livevideo/machine-learning-for-mere-mortals
13. www.manning.com/books/classic-computer-science-problems-in-python
14. www.manning.com/books/getting-mean-with-mongo-express-angular-and-node
5Check out MEAP program from our Learning Sponsor!

Venue Sponsor
6
As Certified B Corporation, Myplanet is purpose-driven and
creates benefit for all stakeholders, not just shareholders!

Catering Sponsor
7
Also an active Serverless Community member – check out
their “Data Pipelines using Serverless Architectures” talk!

Community Open Mic
10
10 seconds of freedom
to pitch yourself, or
your company

Feature Talk
Racing with AI
11

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Racing with Artificial Intelligence
Alex Coqueiro
Head of Public Sector Solutions Architecture for Canada, Latin America and Caribbean
AWS
@alexbcbr

Rubik’s cube challenge

43,252,003,274,489,856,000
43 QUINTILLION
UNIQUE COMBINATIONS

Don’t code the patterns, let the
system learn through data

F2 U' R' L F2 R L' U'
ModelData

F2 U' R' L F2 R L' U'
Confidence
1%
accuracy
R U r U R U2 r U2%
accuracy
Training Models
Model

Confidence
20%
accuracy
40%
accuracy
60%
accuracy
80%
accuracy
95%
accuracy
2%
accuracy
Training Models
Model

Confidence
95%
accuracy
?
F2 R F R′ B′ D F D′ B D F
Inference
Model

SOLVED IN 0.9 SECONDS

Let’s apply it into the business

https://youtu.be/xC-tikvEvzo

10,000+ customers | 2x customer references | 85% of TensorFlow projects in the cloud happen on
AWS

LEA (Robot Care Systems)
https://youtu.be/r-US8rs8EY0

tuSimple (Autonomous Vehicle)

Show me how to do it

Our problem re-formulation

Different problems require different learning strategies
labeled training data
Complexityofdecisions
Supervised learning
Non-labeled training data

Autonomous Driving Development

Robocar (Donkey Car Project)
Donkey Car Project
https://github.com/sunilmallya/donkey/tree/master/sagetrain
http://awsrobocar.s3-website-us-east-1.amazonaws.com/
https://github.com/tescal2/donkey

SSD MultiBox — Real-Time Object Detection +
Behavioral Cloning

Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale
1
2
3

Different problems require different learning strategies
labeled training data
Complexityofdecisions
Supervised learning
Unsupervised
learning
Reinforcement
Learning
Non-labeled training data

Reinforcement learning in the real world
Reward positive
behavior
Don’t reward
negative
behavior
The result!

• Build machine learning models in Amazon
SageMaker
• Train, test, and iterate on the track using the AWS
DeepRacer 3D racing simulator
• Compete in the world’s first global autonomous
racing league, to race for prizes and a chance to
advance to win the coveted AWS DeepRacer Cup
AWS DeepRacer
A fully autonomous 1/18th-scale race car designed to help you learn about
reinforcement learning through autonomous driving

Robotic autonomous
race car
DeepRacer: An exciting way for developers to get hands-on experience with
Reinforcement Learning
Racing LeagueVirtual simulator, to
train and experiment

Track components
TRACK CENTER
TRACK WALL
TRACK SURFACE aka ON-TRACK
FIELD aka OFF-TRACK
TRACK BOUNDARIES

Action space

The reward function in a race grid
S G = 2
GOALAGENT

Incentivizing centerline behavior
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
S 2 2 2 2 2 2 G = 2
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
REWARD FUNCTION
8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9
S 10.4 9.4 8.2 6.9 5.4 3.8 G = 2
8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9
MAX VALUE OF EACH STATE
AFTER LOTS OF EXPLORING
Discount per step
0.9

Programming your own reward function

Let’s go deeper
Let’s go deeper…

AWS DeepRacer Neural Network Architecture
An overview of the network architecture that AWS DeepRacer uses:
Output

Amazon
Sagemaker RL

Reinforcement Learning Algorithms Compared
Value Approximation Policy Approximation
Advantages
More stable performance when it works, and tends to
converge on global optimum
Effective in continuous action spaces, can learn stochastic policies,
and faster convergence
Disadvantages
Difficult to converge if too many (state, action)
combinations, slower convergence in general, and can’t
learn stochastic properties
Typically converges to a local rather than global optimum, high
variance in estimating the gradient adversely affects stability, and
evaluating a policy is generally inefficient
Examples Q-Learning, Deep Q Network, Deep Double Q Network Policy Gradient, Proximal Policy Optimization (PPO)

Hyper parameters control the training algorithm

Coordinate system and track waypoints
OUTER BOUNDARY WAYPOINTS
TRACK CENTER WAYPOINTS
INNER BOUNDARY WAYPOINTS
X
Y
TRACK WIDTH
CAR DIRECTION

AWS DeepRacer Car Specifications
CAR 18th scale 4WD with monster truck chassis
CPU Intel Atom™ Processor
MEMORY 4GB RAM
STORAGE 32GB (expandable)
WI-FI 802.11ac
CAMERA 4 MP camera with MJPEG
DRIVE BATTERY 7.4V/1100mAh lithium polymer
COMPUTE BATTERY 13600mAh USB-C PD
SENSORS Integrated accelerometer and gyroscope
PORTS 4x USB-A, 1x USB-C, 1x Micro-USB, 1x HDMI
SOFTWARE Ubuntu OS 16.04.3 LTS, Intel® OpenVINO™
toolkit, ROS Kinetic

Robotic Arms
International Space Station
Drones
Education
Water
Home
Self-Driving Vehicles
Autonomous Walker
Rover
Robot landscape

Robotics trends
in 2018
Robotics is undergoing fundamental
change in collaboration, autonomous
mobility, and increasing intelligence
Source: IDTechEx
• Logistics
• Construction
• Retail
• Hospitality
• Healthcare
Robots are being put to work every
day across many industries
• Agriculture
• Energy Management
• Oil and Gas
• Facilities Management
• Household chores
By 2023, it’s estimated that mobile autonomous robots will
emerge as the standard for logistic and fulfillment processes
By 2030, 70% of all mobile material
handling equipment will be autonomous

Robotic development cycle
2) Develop
robotics
application
1) Select robotics
software
framework
1) Deploy and
manage
application
3) Test and
simulate
application
New application release and update

Most widely used software framework for teaching and learning about robotics – over 16 million .deb (Linux Debian)
packages downloaded in 2018, a 400% increase since 2014
Founded in Stanford labs over 10 year ago, now managed by the Open Source Robotics Foundation (OSRF)
Global open-source community supports two products—Robot Operating System (ROS) and Gazebo
ROS
A set of software libraries and tools, from drivers to algorithms,
that help developers build robot applications
Gazebo
Robust physics engine, high-quality graphics, and programmatic
and graphical interfaces to help developers simulate robots
Robot Operating System (ROS) primer

Introducing AWS RoboMaker
A service that makes it easy for
developers to develop, test, and
deploy robotics applications, as
well as build intelligent robotics
functions using cloud services

AWS RoboMaker service suite
Development
Environment
SimulationCloud Extensions for
ROS
Fleet
Management

AWS RoboMaker
Sample Robot Applications
Hello
World
Navigation
and Person
Recognition
Voice
Commands
Robot
Monitoring
Object-
following using
RL
Self-
driving
using RL

AWS Cloud
AWS
DeepRacer
NAT gateway
VPC
AWS DeepRacer
Models
Simulation
video
Metrics
AWS DeepRacer Simulation Architecture

ROS Msg Node
Stored File
ROS Nodes
Model
Optimizer
Video
M-JPEG
Web Server
Video
Inference
Results
Web
Server
Publisher
Autonomous
Drive
Control
Node
Optimized
Model
Media engine
Camera
Model
Inference
engine
Manual
Drive
Navigation
Node
Servo & Motor
AWS DeepRacer Software Architecture

DATA

We are not spectators,
but actors of the future
Herb Simon,
2000

ml.aws
@alexbcbr

Upcoming Serverless Toronto Meetups
2
1. August 12, 2019: 1. A look at Google Cloud Functions
– Matt Welke // 2. Firebase Crash Course – Kudz Murefu
2. September 12, 2019: Serverless Design Patterns and
Best Practices – Mike Apted, Startup SA at AWS
3. October, 2019: Serverless CI/CD pipelines with AWS
CodePipeline and CodeBuild, vs CircleCI, vs Travis, vs
Seed – Frank Wang & Jay V
4. November or December: Serverless Heroes (authors
of “Serverless Applications with Node.js”) Slobodan
Stojanović & Aleksandar Simović will be here!
5. December or January, 2020: re:Invent recap –
Jonathan Dion, Senior Technical Evangelist at AWS

Racing with Artificial Intelligence

More Related Content

Similar to Racing with Artificial Intelligence (20)

More from Daniel Zivkovic (20)

Recently uploaded (20)

Racing with Artificial Intelligence