SlideShare a Scribd company logo

Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Autonomous Robots

Pooyan Jamshidi, profile picture
Pooyan Jamshidi

The document discusses the integration of machine learning with quantitative planning to enable self-adaptation in autonomous robots, focusing on the challenge of managing the vast configuration space in highly-configurable systems. The approach includes using machine learning to identify pareto-optimal configurations, making real-time planning more efficient while maintaining high-quality outcomes. Evaluation shows that this method allows for effective optimization even in dynamic environments, laying groundwork for future developments in adaptive robotics.

Technology
1 of 43
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Pooyan Jamshidi, Javier Cámara, Bradley Schmerl, Chris3an Kästner, David Garlan Machine Learning Meets Quan0ta0ve Planning: Enabling Self-Adapta1on in Autonomous Robots https://arxiv.org/abs/1903.03920
Outline • Self-adapta*on of Highly-Configurable Systems • Mobile robo)cs domain • Challenges with quan)ta)ve planning and scale of search space • Our approach: use machine learning to iden*fy interes*ng configura*ons • Evalua-on: third party evalua*on of highly-configurable robot naviga*ng internal space • Results: machine learning to limit configura*on search space leads to tractable high quality plans synthesized at run *me • Future work 214th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Self-adapta.on of Highly-Configurable Systems • Many cyberphyscial systems have many alterna2ve components with hundreds of configura2on op2ons • Many different kinds of sensors • Alterna3ve so?ware for different robot func3ons • Abundant configura3on op3ons • E.g., AMCL, a component for robot localiza8on, has ~40 configura8on parameters • Understanding effect of parameters on behavior, power consump3on, memory, etc. is hard • Self-adapta2on required to handle dynamic situa2ons 314th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Challenges • How does self-adapta2on deal with this? • Fixed set of plans developed at design 3me • Restricted to a manageable set of condi8ons, pre-known condi8ons • Run 3me planning that needs to search large planning space • Need to simplify the problem to deal with large s earch space • Cyberphysical components à intractable to completely define ground truth model Desire a solu2on that can deal with large configura,on space and highly dynamic environments 414th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Scenario: Autonomous Service Robot Power 12 Go to a series of locations in a building to deliver packages and messages. Objectives: • Timeliness (time to completion) • Success rate (number of targets reached) 5 Adapta/on space: • Instruc8on graph (move, charge, etc.) • Robot’s configura8on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 3 Adapt! Find new plan Choose configuration Sensitive to power model.
Our approach 1. Off-line machine learning finds Pareto-op2mal configura2ons 2. Planning space restricted to only these configura2ons H1: Machine learning can find sufficiently op2mal configura2ons with limited sampling budget. H2: Restric2ng planning to pareto-op2mal solu2ons makes run2me planning tractable while maintaining high quality plans. 614th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Approach to machine learning 7 Offline Learning Polynomial regression model Query Value Hidden Power Model Exhaus8ve search 𝑓 ⋅ = 1.2 + 3𝑜! + 5𝑜" + 0.9𝑜# + 0.8𝑜" 𝑜# +4𝑜! 𝑜" 𝑜# 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Background: Configura/on Representa/on ℂ = 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ Kinect Configuration Space thermometer 𝑐! = 0×0× ⋯×0×1𝑐! ∈ ℂ Localization Lidar GPS 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 8
Our learning approach A typical approach for understanding the performance behavior is sensi2vity analysis 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') 𝑓 ∼ 𝑓(⋅) ⋯ Learn TrainingSet ^ 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 9
Our learning approach: Step-wise linear regression 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') ⋯ TrainingSet Learn power model 1. Fit an ini#al model 2. Forward selec#on: Add terms itera0vely 3. Backward elimina#on: Removes terms itera0vely 4. Terminate: When neither (2) or (3) improve the model Source (Execution time of Program X) 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 10 𝐋𝐞𝐚𝐫𝐧𝐞𝐝 𝐦𝐨𝐝𝐞𝐥: 𝑓(⋅) = 1.2 + 3𝑜$ + 5𝑜% + 0.9𝑜& + 0.8𝑜% 𝑜& + 4𝑜$ 𝑜% 𝑜&
Planning: Approach overview The set of Pareto op2mal configura2ons reduces the search space • But not enough to do planning all in one model Approach: Divide and conquer 1. Determine valid paths 2. Find best configura2on for each path 3. Pick path/config combina2on with best score Approach that comes up with the best combina2on configura2on/path to sa2sfy a preference func2on over quality aPributes 1114th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Planning: Mul.ple Models Planner requires informa2on from mul2ple models Each stage updates some of the models 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 12 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models
Planning: Machine learned models Machine learning produces models for: • Configura2on space to search • Power consump2on of robot opera2ons in those configura2ons 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 13 Configura)on Machine Learning Pipeline (Offline) Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Offline
Planning: Find legal paths Use Dijkstra's algorithm Considers current knowledge of location, target, and environment. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 14 Task Planning Pipeline (Online) Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology Online
Planning: Quan.ta.ve Planning All models combined into Prism models Prism synthesizes plan that… 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019 15 Task Planning Pipeline (Online) Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Task Planner Task Plan Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Model-ViewTranslationandAggregation
Evalua.on: H1 Want to know how accurate a learned model is: • Sampling ground truth model through physical experimenta3on • Power model, which is a set of func3ons, one for each configura3on Approach: Learn from a set of synthe2c models • 100 synthe3cally generated power models each with 1000000 configura3ons • Pick 100 samples from every model and try to learn that model 16 H1: ML finds Pareto-op3mal configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Results: H1 We are able to learn an accurate model that is highly likely to iden2fy Pareto op2mal configura2ons 17 H1: ML finds Pareto-op3mal configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Evalua.on: H2 A range of condi2ons: - different missions - sequences of waypoints - different adapta2on-causing perturba2ons - obstacle placement and baRery deple3on - different learning budgets - how much machine learning is done Actual experiments chosen and executed by a third party - Lincoln Laboratories( ) as part of a DARPA project 18 H2: Good adapta3ons with just Pareto configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Docker Container Evalua.on Implementa.on: H2 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 19 Start: - mission - power model - learning budget Perturb: - obstacle - baRery REST Test Adapter Robot Software Planning Path plan Configuration Test Driver Gazebo Simulator H2: Good adapta3ons with just Pareto configura3ons. Offline Learning Find Pareto- opt Models Analysis
Docker Container Evalua.on Implementa.on: H2 20 Start: - power model - learning budget - mission Perturb: - obstacle - baRery REST Test Adapter Models Analysis Planning Test Driver Robot Software Gazebo Simulator H2: Good adapta3ons with just Pareto configura3ons. Offline Learning Find Pareto-opt Choose modelLearn modelStart missionPerturb system Path plan Configuration 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Evaluation Design: H2 Baseline A: No Perturba2ons, no learning, reac2ve planning Baseline B: Perturba2ons, no learning, reac2ve planning Challenge: Perturba2ons, learning, quan2ta2ve planning 280 Test triples (840 runs total) 120 Valid triples (Where successful mission in A and unsuccessful in B) 21 H2: Good adapta3ons with just Pareto configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Results: H2 Verdicts: Pass: C completes mission Degraded: C completes more tasks in the mission Fail: B bePer than C 22 H2: Good adaptations with just Pareto configurations. Path obstruc,on Power deple,on
Results: Summary H1: Machine learning can find op2mal configura2ons without exploring the en2re state space • Pareto configura-ons learned even when observing 10-4% of the configura-on space H2: Restric2ng planning to pareto-op2mal solu2ons makes run2me planning tractable while maintaining high quality plans • Planning was able to be done in real -me in a robot simula-on that beat reac-ve adapta-on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 23
Limita.ons Miscommunication in test design led to poor test cases by independent evaluators: - Multiple battery perturbations drain battery completely - Did not combine battery and obstacle perturbations - Only one domain (service robots) and one learned model (power, polynomial) 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 24 Future work - On-line transfer learning to learn and adapt models at run 2me - Incorpora2on of mul2ple learned models - More principled approach to model integra2on
14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 25
Approach to machine learning 26LL Specify Query ValueMARS DAS Learn Polynomial regression model 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Background: Configura/on Representa/on ℂ = 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ Kinect Configuration Space thermometer 𝑐! = 0×0× ⋯×0×1𝑐! ∈ ℂ Energy Localization Robot Compiled Code Instrumented Binary Hardware Compile Deploy Configure 𝑓!"(𝑐#) = 100𝑚𝑤ℎ Non-func/onal measurable/quan/fiable aspect Lidar GPS 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 27
Our learning approach Performance model could be in any appropriate form of black-box models 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') 𝑓 ∼ 𝑓(⋅) ⋯ Learn TrainingSet ^ 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 28
Our learning approach: Measuring Accuracy 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') ⋯ TrainingSet Source (Execution time of Program X) 𝑓 ̂ ∼ 𝑓(⋅) Learn Evaluate Accuracy 𝐴𝑃𝐸(𝑓 ̂ , 𝑓) = |𝑓 ̂ (𝑐) − 𝑓(𝑐)| 𝑓(𝑐) ×100 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019 29
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Planning Architecture 30 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Aggregator Task Planning Model Generator Task Planner Task Plan Prism Spec Robot loca)on Target loca)on Space Topology Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Planning: Mul.ple Models 31 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTransla)onandAggrega)on Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Different models capture facets of the domain Each model includes a model-view translator that enables retrieving and inserting information 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Configura)on Machine Learning Pipeline (Offline) Planning: Mul.ple Models 32 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTransla)onandAggrega)on Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs The set of Pareto- optimal configurations into the configuration model The energy consumption of robot operations in those configurations into the power model 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Configuration Machine LearnerSystem Observa2ons Pareto-optimal configs The Configuration Machine Learner incorporates...
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Legal Paths 33 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTranslationandAggregation Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot loca)on Target location Space Topology The Aggregator gathers... The robot’s and target location from the task model The topological information of the physical space 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Legal Paths 34 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTransla)onandAggrega)on Configuration Machine LearnerSystem Observations Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology The Path Preprocessor generates the legal paths between those locations... ...and Legal Paths are inserted back into the task model Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 35 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot location Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers...
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 36 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs AdaptationPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 37 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observations Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target location Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model Distances from the physical env. model
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 38 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-optimal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model Distances from the physical env. model Robot operations from the operations model
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 39 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task attribute quantifiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTranslationandAggregation 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model Distances from the physical env. model Pareto-optimal configs from the config model Robot operations from the operations model
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 40 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot loca)on Target location Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Task Planning Model Generator creates a Prism Specification using all the former elements as building blocks
Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Planning 41 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot location Target loca)on Space Topology Aggregator Task Planning Model Generator Task Planner Task Plan Prism Spec Task attribute quantifiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTranslationandAggregation 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Task Planner Task Plan Prism Spec The Task Planner use probabilistic model checking (MDP policy synthesis) in the backend to generate a Task Plan
Our learning approach: Op0on analysis A power model contains useful informa2on about influen2al op2ons and interac2ons 𝑓(⋅) = 1.2 + 3𝑜! + 5𝑜& + 0.9𝑜( + 0.8𝑜& 𝑜( + 4𝑜! 𝑜& 𝑜( 𝑓: ℂ → ℝ 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 42
4314th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019

More Related Content

PDF
D space magazin_2019_2_iupui_english
Sree Shruthi
 
PDF
NUAV - A Testbed for Development of Autonomous Unmanned Aerial Vehicles
adil raja
 
PDF
Simulators as Drivers of Cutting Edge Research
adil raja
 
PDF
Planning/Scheduling with CP Optimizer
Philippe Laborie
 
PDF
AICamp - Dr Ramine Tinati - Making Computer Vision Real
Ramine Tinati
 
PDF
CP Optimizer pour la planification et l'ordonnancement
Philippe Laborie
 
PDF
Deep AutoViML For Tensorflow Models and MLOps Workflows
Bill Liu
 
PDF
D space magazin-2019-02-iupui_03_190529_e
Sree Shruthi
 
D space magazin_2019_2_iupui_english
Sree Shruthi
 
NUAV - A Testbed for Development of Autonomous Unmanned Aerial Vehicles
adil raja
 
Simulators as Drivers of Cutting Edge Research
adil raja
 
Planning/Scheduling with CP Optimizer
Philippe Laborie
 
AICamp - Dr Ramine Tinati - Making Computer Vision Real
Ramine Tinati
 
CP Optimizer pour la planification et l'ordonnancement
Philippe Laborie
 
Deep AutoViML For Tensorflow Models and MLOps Workflows
Bill Liu
 
D space magazin-2019-02-iupui_03_190529_e
Sree Shruthi
 

Similar to Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Autonomous Robots (20)

PDF
Integrated Model Discovery and Self-Adaptation of Robots
Pooyan Jamshidi
 
PDF
Robotics Science And Systems Vii Hugh F Durrantwhyte Nicholas Roy Pieter Abbeel
ajpuaconardo55
 
PPTX
COMP 775 Motion planning paper presentation
Ravikiran J
 
PPTX
Data-driven AI for Self-adaptive Information Systems
Andreas Metzger
 
PPT
Machine Learning and Robotics
butest
 
PDF
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
AutonomyIncubator
 
PDF
PRM-RL: Long-range Robotics Navigation Tasks by Combining Reinforcement Learn...
Dongmin Lee
 
PPT
AI Robotics
Yasir Khan
 
PDF
Transfer Learning for Improving Model Predictions in Highly Configurable Soft...
Pooyan Jamshidi
 
PDF
Racing with Artificial Intelligence
Daniel Zivkovic
 
PDF
Transfer Learning for Improving Model Predictions in Robotic Systems
Pooyan Jamshidi
 
PPTX
Feature Model-Guided Online Reinforcement Learning for Self-Adaptive Services
Andreas Metzger
 
PDF
Towards Systematically Engineering Ensembles - Martin Wirsing
FET AWARE project - Self Awareness in Autonomic Systems
 
PDF
AutoML lectures (ACDL 2019)
Joaquin Vanschoren
 
PDF
Motion Planning
Alaa Khamis, PhD, SMIEEE
 
PDF
DESIGN, IMPLEMENT AND SIMULATE AN AGENT MOTION PLANNING ALGORITHM IN 2D AND 3...
csandit
 
PPT
This is a heavily data-oriented
butest
 
PPT
This is a heavily data-oriented
butest
 
PDF
A simulated motion planning algorithm in 2 d
ijaia
 
PPT
16355694.ppt
shohel rana
 
Integrated Model Discovery and Self-Adaptation of Robots
Pooyan Jamshidi
 
Robotics Science And Systems Vii Hugh F Durrantwhyte Nicholas Roy Pieter Abbeel
ajpuaconardo55
 
COMP 775 Motion planning paper presentation
Ravikiran J
 
Data-driven AI for Self-adaptive Information Systems
Andreas Metzger
 
Machine Learning and Robotics
butest
 
Autonomy Incubator Seminar Series: Tractable Robust Planning and Model Learni...
AutonomyIncubator
 
PRM-RL: Long-range Robotics Navigation Tasks by Combining Reinforcement Learn...
Dongmin Lee
 
AI Robotics
Yasir Khan
 
Transfer Learning for Improving Model Predictions in Highly Configurable Soft...
Pooyan Jamshidi
 
Racing with Artificial Intelligence
Daniel Zivkovic
 
Transfer Learning for Improving Model Predictions in Robotic Systems
Pooyan Jamshidi
 
Feature Model-Guided Online Reinforcement Learning for Self-Adaptive Services
Andreas Metzger
 
Towards Systematically Engineering Ensembles - Martin Wirsing
FET AWARE project - Self Awareness in Autonomic Systems
 
AutoML lectures (ACDL 2019)
Joaquin Vanschoren
 
Motion Planning
Alaa Khamis, PhD, SMIEEE
 
DESIGN, IMPLEMENT AND SIMULATE AN AGENT MOTION PLANNING ALGORITHM IN 2D AND 3...
csandit
 
This is a heavily data-oriented
butest
 
This is a heavily data-oriented
butest
 
A simulated motion planning algorithm in 2 d
ijaia
 
16355694.ppt
shohel rana
 
Ad

More from Pooyan Jamshidi (20)

PDF
Learning LWF Chain Graphs: A Markov Blanket Discovery Approach
Pooyan Jamshidi
 
PDF
A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn...
Pooyan Jamshidi
 
PDF
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
Pooyan Jamshidi
 
PDF
Transfer Learning for Performance Analysis of Machine Learning Systems
Pooyan Jamshidi
 
PDF
Transfer Learning for Performance Analysis of Configurable Systems: A Causal ...
Pooyan Jamshidi
 
PDF
Machine Learning meets DevOps
Pooyan Jamshidi
 
PDF
Learning to Sample
Pooyan Jamshidi
 
PDF
Transfer Learning for Performance Analysis of Highly-Configurable Software
Pooyan Jamshidi
 
PDF
Architectural Tradeoff in Learning-Based Software
Pooyan Jamshidi
 
PDF
Production-Ready Machine Learning for the Software Architect
Pooyan Jamshidi
 
PDF
Transfer Learning for Software Performance Analysis: An Exploratory Analysis
Pooyan Jamshidi
 
PDF
Architecting for Scale
Pooyan Jamshidi
 
PDF
Learning Software Performance Models for Dynamic and Uncertain Environments
Pooyan Jamshidi
 
PDF
Sensitivity Analysis for Building Adaptive Robotic Software
Pooyan Jamshidi
 
PDF
Machine Learning meets DevOps
Pooyan Jamshidi
 
PDF
An Uncertainty-Aware Approach to Optimal Configuration of Stream Processing S...
Pooyan Jamshidi
 
PDF
Configuration Optimization Tool
Pooyan Jamshidi
 
PDF
Fuzzy Self-Learning Controllers for Elasticity Management in Dynamic Cloud Ar...
Pooyan Jamshidi
 
PDF
Configuration Optimization for Big Data Software
Pooyan Jamshidi
 
PDF
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Pooyan Jamshidi
 
Learning LWF Chain Graphs: A Markov Blanket Discovery Approach
Pooyan Jamshidi
 
A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn...
Pooyan Jamshidi
 
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
Pooyan Jamshidi
 
Transfer Learning for Performance Analysis of Machine Learning Systems
Pooyan Jamshidi
 
Transfer Learning for Performance Analysis of Configurable Systems: A Causal ...
Pooyan Jamshidi
 
Machine Learning meets DevOps
Pooyan Jamshidi
 
Learning to Sample
Pooyan Jamshidi
 
Transfer Learning for Performance Analysis of Highly-Configurable Software
Pooyan Jamshidi
 
Architectural Tradeoff in Learning-Based Software
Pooyan Jamshidi
 
Production-Ready Machine Learning for the Software Architect
Pooyan Jamshidi
 
Transfer Learning for Software Performance Analysis: An Exploratory Analysis
Pooyan Jamshidi
 
Architecting for Scale
Pooyan Jamshidi
 
Learning Software Performance Models for Dynamic and Uncertain Environments
Pooyan Jamshidi
 
Sensitivity Analysis for Building Adaptive Robotic Software
Pooyan Jamshidi
 
Machine Learning meets DevOps
Pooyan Jamshidi
 
An Uncertainty-Aware Approach to Optimal Configuration of Stream Processing S...
Pooyan Jamshidi
 
Configuration Optimization Tool
Pooyan Jamshidi
 
Fuzzy Self-Learning Controllers for Elasticity Management in Dynamic Cloud Ar...
Pooyan Jamshidi
 
Configuration Optimization for Big Data Software
Pooyan Jamshidi
 
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Pooyan Jamshidi
 
Ad

Recently uploaded (20)

PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PPTX
Spectroscopy.pptx food analysis technology
nawahassan45
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PDF
gpt5_lecture_notes_comprehensive_20250812015547.pdf
Chinchu40
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PPTX
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
Spectroscopy.pptx food analysis technology
nawahassan45
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
Approach and Philosophy of On baking technology
30anthuong17
 
gpt5_lecture_notes_comprehensive_20250812015547.pdf
Chinchu40
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
Teaching material agriculture food technology
LiaRayya
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 

Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Autonomous Robots

  • 1. Pooyan Jamshidi, Javier Cámara, Bradley Schmerl, Chris3an Kästner, David Garlan Machine Learning Meets Quan0ta0ve Planning: Enabling Self-Adapta1on in Autonomous Robots https://arxiv.org/abs/1903.03920
  • 2. Outline • Self-adapta*on of Highly-Configurable Systems • Mobile robo)cs domain • Challenges with quan)ta)ve planning and scale of search space • Our approach: use machine learning to iden*fy interes*ng configura*ons • Evalua-on: third party evalua*on of highly-configurable robot naviga*ng internal space • Results: machine learning to limit configura*on search space leads to tractable high quality plans synthesized at run *me • Future work 214th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 3. Self-adapta.on of Highly-Configurable Systems • Many cyberphyscial systems have many alterna2ve components with hundreds of configura2on op2ons • Many different kinds of sensors • Alterna3ve so?ware for different robot func3ons • Abundant configura3on op3ons • E.g., AMCL, a component for robot localiza8on, has ~40 configura8on parameters • Understanding effect of parameters on behavior, power consump3on, memory, etc. is hard • Self-adapta2on required to handle dynamic situa2ons 314th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 4. Challenges • How does self-adapta2on deal with this? • Fixed set of plans developed at design 3me • Restricted to a manageable set of condi8ons, pre-known condi8ons • Run 3me planning that needs to search large planning space • Need to simplify the problem to deal with large s earch space • Cyberphysical components à intractable to completely define ground truth model Desire a solu2on that can deal with large configura,on space and highly dynamic environments 414th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 5. Scenario: Autonomous Service Robot Power 12 Go to a series of locations in a building to deliver packages and messages. Objectives: • Timeliness (time to completion) • Success rate (number of targets reached) 5 Adapta/on space: • Instruc8on graph (move, charge, etc.) • Robot’s configura8on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 3 Adapt! Find new plan Choose configuration Sensitive to power model.
  • 6. Our approach 1. Off-line machine learning finds Pareto-op2mal configura2ons 2. Planning space restricted to only these configura2ons H1: Machine learning can find sufficiently op2mal configura2ons with limited sampling budget. H2: Restric2ng planning to pareto-op2mal solu2ons makes run2me planning tractable while maintaining high quality plans. 614th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 7. Approach to machine learning 7 Offline Learning Polynomial regression model Query Value Hidden Power Model Exhaus8ve search 𝑓 ⋅ = 1.2 + 3𝑜! + 5𝑜" + 0.9𝑜# + 0.8𝑜" 𝑜# +4𝑜! 𝑜" 𝑜# 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 8. Background: Configura/on Representa/on ℂ = 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ Kinect Configuration Space thermometer 𝑐! = 0×0× ⋯×0×1𝑐! ∈ ℂ Localization Lidar GPS 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 8
  • 9. Our learning approach A typical approach for understanding the performance behavior is sensi2vity analysis 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') 𝑓 ∼ 𝑓(⋅) ⋯ Learn TrainingSet ^ 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 9
  • 10. Our learning approach: Step-wise linear regression 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') ⋯ TrainingSet Learn power model 1. Fit an ini#al model 2. Forward selec#on: Add terms itera0vely 3. Backward elimina#on: Removes terms itera0vely 4. Terminate: When neither (2) or (3) improve the model Source (Execution time of Program X) 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 10 𝐋𝐞𝐚𝐫𝐧𝐞𝐝 𝐦𝐨𝐝𝐞𝐥: 𝑓(⋅) = 1.2 + 3𝑜$ + 5𝑜% + 0.9𝑜& + 0.8𝑜% 𝑜& + 4𝑜$ 𝑜% 𝑜&
  • 11. Planning: Approach overview The set of Pareto op2mal configura2ons reduces the search space • But not enough to do planning all in one model Approach: Divide and conquer 1. Determine valid paths 2. Find best configura2on for each path 3. Pick path/config combina2on with best score Approach that comes up with the best combina2on configura2on/path to sa2sfy a preference func2on over quality aPributes 1114th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 12. Planning: Mul.ple Models Planner requires informa2on from mul2ple models Each stage updates some of the models 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 12 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models
  • 13. Planning: Machine learned models Machine learning produces models for: • Configura2on space to search • Power consump2on of robot opera2ons in those configura2ons 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 13 Configura)on Machine Learning Pipeline (Offline) Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Offline
  • 14. Planning: Find legal paths Use Dijkstra's algorithm Considers current knowledge of location, target, and environment. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 14 Task Planning Pipeline (Online) Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology Online
  • 15. Planning: Quan.ta.ve Planning All models combined into Prism models Prism synthesizes plan that… 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019 15 Task Planning Pipeline (Online) Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Task Planner Task Plan Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Model-ViewTranslationandAggregation
  • 16. Evalua.on: H1 Want to know how accurate a learned model is: • Sampling ground truth model through physical experimenta3on • Power model, which is a set of func3ons, one for each configura3on Approach: Learn from a set of synthe2c models • 100 synthe3cally generated power models each with 1000000 configura3ons • Pick 100 samples from every model and try to learn that model 16 H1: ML finds Pareto-op3mal configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 17. Results: H1 We are able to learn an accurate model that is highly likely to iden2fy Pareto op2mal configura2ons 17 H1: ML finds Pareto-op3mal configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 18. Evalua.on: H2 A range of condi2ons: - different missions - sequences of waypoints - different adapta2on-causing perturba2ons - obstacle placement and baRery deple3on - different learning budgets - how much machine learning is done Actual experiments chosen and executed by a third party - Lincoln Laboratories( ) as part of a DARPA project 18 H2: Good adapta3ons with just Pareto configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 19. Docker Container Evalua.on Implementa.on: H2 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 19 Start: - mission - power model - learning budget Perturb: - obstacle - baRery REST Test Adapter Robot Software Planning Path plan Configuration Test Driver Gazebo Simulator H2: Good adapta3ons with just Pareto configura3ons. Offline Learning Find Pareto- opt Models Analysis
  • 20. Docker Container Evalua.on Implementa.on: H2 20 Start: - power model - learning budget - mission Perturb: - obstacle - baRery REST Test Adapter Models Analysis Planning Test Driver Robot Software Gazebo Simulator H2: Good adapta3ons with just Pareto configura3ons. Offline Learning Find Pareto-opt Choose modelLearn modelStart missionPerturb system Path plan Configuration 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 21. Evaluation Design: H2 Baseline A: No Perturba2ons, no learning, reac2ve planning Baseline B: Perturba2ons, no learning, reac2ve planning Challenge: Perturba2ons, learning, quan2ta2ve planning 280 Test triples (840 runs total) 120 Valid triples (Where successful mission in A and unsuccessful in B) 21 H2: Good adapta3ons with just Pareto configura3ons. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 22. Results: H2 Verdicts: Pass: C completes mission Degraded: C completes more tasks in the mission Fail: B bePer than C 22 H2: Good adaptations with just Pareto configurations. Path obstruc,on Power deple,on
  • 23. Results: Summary H1: Machine learning can find op2mal configura2ons without exploring the en2re state space • Pareto configura-ons learned even when observing 10-4% of the configura-on space H2: Restric2ng planning to pareto-op2mal solu2ons makes run2me planning tractable while maintaining high quality plans • Planning was able to be done in real -me in a robot simula-on that beat reac-ve adapta-on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 23
  • 24. Limita.ons Miscommunication in test design led to poor test cases by independent evaluators: - Multiple battery perturbations drain battery completely - Did not combine battery and obstacle perturbations - Only one domain (service robots) and one learned model (power, polynomial) 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 24 Future work - On-line transfer learning to learn and adapt models at run 2me - Incorpora2on of mul2ple learned models - More principled approach to model integra2on
  • 25. 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 25
  • 26. Approach to machine learning 26LL Specify Query ValueMARS DAS Learn Polynomial regression model 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 27. Background: Configura/on Representa/on ℂ = 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ Kinect Configuration Space thermometer 𝑐! = 0×0× ⋯×0×1𝑐! ∈ ℂ Energy Localization Robot Compiled Code Instrumented Binary Hardware Compile Deploy Configure 𝑓!"(𝑐#) = 100𝑚𝑤ℎ Non-func/onal measurable/quan/fiable aspect Lidar GPS 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 27
  • 28. Our learning approach Performance model could be in any appropriate form of black-box models 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') 𝑓 ∼ 𝑓(⋅) ⋯ Learn TrainingSet ^ 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 28
  • 29. Our learning approach: Measuring Accuracy 𝑂!×𝑂"× ⋯×𝑂!#×𝑂"$ 0×0× ⋯×0×1 0×0× ⋯×1×0 0×0× ⋯×1×1 1×1× ⋯×1×0 1×1× ⋯×1×1 ⋯ 𝑐! 𝑐% 𝑐& 𝑐' 𝑦! = 𝑓(𝑐!) 𝑦% = 𝑓(𝑐%) 𝑦& = 𝑓(𝑐&) 𝑦' = 𝑓(𝑐') ⋯ TrainingSet Source (Execution time of Program X) 𝑓 ̂ ∼ 𝑓(⋅) Learn Evaluate Accuracy 𝐴𝑃𝐸(𝑓 ̂ , 𝑓) = |𝑓 ̂ (𝑐) − 𝑓(𝑐)| 𝑓(𝑐) ×100 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019 29
  • 30. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Planning Architecture 30 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Aggregator Task Planning Model Generator Task Planner Task Plan Prism Spec Robot loca)on Target loca)on Space Topology Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 31. Planning: Mul.ple Models 31 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTransla)onandAggrega)on Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Different models capture facets of the domain Each model includes a model-view translator that enables retrieving and inserting information 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 32. Configura)on Machine Learning Pipeline (Offline) Planning: Mul.ple Models 32 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTransla)onandAggrega)on Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs The set of Pareto- optimal configurations into the configuration model The energy consumption of robot operations in those configurations into the power model 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Configuration Machine LearnerSystem Observa2ons Pareto-optimal configs The Configuration Machine Learner incorporates...
  • 33. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Legal Paths 33 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTranslationandAggregation Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot loca)on Target location Space Topology The Aggregator gathers... The robot’s and target location from the task model The topological information of the physical space 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 34. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Legal Paths 34 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Adapta)onPlanningModel-ViewTransla)onandAggrega)on Configuration Machine LearnerSystem Observations Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology The Path Preprocessor generates the legal paths between those locations... ...and Legal Paths are inserted back into the task model Aggregator Path Preprocessor Legal paths Robot loca)on Target loca)on Space Topology 14th Symposium on Software Engineering for Adaptive and Self-Managing Systems, Montreal, CA, 24-25 May 2019
  • 35. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 35 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot location Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers...
  • 36. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 36 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs AdaptationPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model
  • 37. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 37 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observations Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target location Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model Distances from the physical env. model
  • 38. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 38 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-optimal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model Distances from the physical env. model Robot operations from the operations model
  • 39. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 39 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Task Planner Task Plan Robot loca)on Target loca)on Space Topology Aggregator Task Planning Model Generator Prism Spec Task attribute quantifiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTranslationandAggregation 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Aggregator gathers... Legal paths, task preferences, and task attribute quantifiers from the task model Distances from the physical env. model Pareto-optimal configs from the config model Robot operations from the operations model
  • 40. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Prism Gen 40 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot loca)on Target location Space Topology Aggregator Task Planning Model Generator Prism Spec Task a4ribute quan)fiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTransla)onandAggrega)on 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Aggregator Task Planning Model Generator Prism Spec The Task Planning Model Generator creates a Prism Specification using all the former elements as building blocks
  • 41. Task Planning Pipeline (Online) Configura)on Machine Learning Pipeline (Offline) Planning: Planning 41 Task Model Physical Env. Model Power Model Operations Model Configuration Model Model-View Translator Model-View Translator Model-View Translator Model-View Translator Model-View Translator Problem Domain Models Configuration Machine LearnerSystem Observa5ons Pareto-optimal configs Aggregator Path Preprocessor Legal paths Robot location Target loca)on Space Topology Aggregator Task Planning Model Generator Task Planner Task Plan Prism Spec Task attribute quantifiers Legal paths Preferences Distances Robot opera)ons’ energy consump)on Robot opera)ons Pareto-op)mal configs Adapta)onPlanningModel-ViewTranslationandAggregation 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 Task Planner Task Plan Prism Spec The Task Planner use probabilistic model checking (MDP policy synthesis) in the backend to generate a Task Plan
  • 42. Our learning approach: Op0on analysis A power model contains useful informa2on about influen2al op2ons and interac2ons 𝑓(⋅) = 1.2 + 3𝑜! + 5𝑜& + 0.9𝑜( + 0.8𝑜& 𝑜( + 4𝑜! 𝑜& 𝑜( 𝑓: ℂ → ℝ 14th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019 42
  • 43. 4314th Symposium on So/ware Engineering for Adap9ve and Self-Managing Systems, Montreal, CA, 24-25 May 2019