Team29-Presentation Final
- 1. 交大密西根学院 UM-SJTU Joint Institute Low Cost SLAM Realization on Smart Phone Group 29 Qian Shiyi, Wei Shupeng, He Qinyue Liu Zhuohe, Xu Zhentao Aug. 12, 2016 Sponsor:
- 2. 交大密西根学院 UM-SJTU Joint Institute Major: Mechanical Engineering Interest: Mechanical Design Work Experience: Intern in Siemens and IMS Health Skills: MATLAB, UG, LabVIEW Hobby: Piano Qian Shiyi (钱诗怡) Using: Huawei G7
- 3. 交大密西根学院 UM-SJTU Joint Institute Dual degree student from Shanghai (Biomedical Engineering (BME) + ECE) Interests: Neural Engineering, Space Technology, Synthetic Biology (Research Experience) Skills: C++/VB, Mathematica, LabVIEW, Graphic Design Hobbies: Collect Coins/Banknotes, Astronomical Photography Liu Zhuohe (刘卓和) Using: Huawei Mate 7
- 4. 交大密西根学院 UM-SJTU Joint Institute Majors: ECE+ME Interests: Robotics, Automatic Control, Programming Work Experience: UM HaptiX Lab Skills: C/C++ Hobbies: Badminton, Programming Zhentao Xu (徐镇涛) Using: Huawei P9 Plus
- 5. 交大密西根学院 UM-SJTU Joint Institute Major: Mechanical Engineering Interest: Mechanical Design and Simulation Work Experience: SJTU racing team member, Intern in BOSCH Skills: CATIA, MATLAB, C Hobbies: Cycling, watching Le Tour de France Wei Shupeng (魏树鹏)
- 6. 交大密西根学院 UM-SJTU Joint Institute Major: Mechanical Engineering Interest: Product Design Work Experience: Intern in Fiat Chrysler Automobiles and BorgWarner Skills: CAD, MATLAB, LabVIEW Hobbies: Cycling, Running, Piano He Qinyue (何沁悦) Using: Huawei P8
- 7. 交大密西根学院 UM-SJTU Joint Institute Problem Driverless car: GPS not accurate Solution Introduction SLAM + Smart Phone More accurate way to locate car Smart Phone SLAM camera, accelerometer, magnetometer, gyroscope simultaneous localization and mapping
- 8. 交大密西根学院 UM-SJTU Joint Institute Quality Function Development (QFD)
- 9. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Integrate and Display on Large Screen Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline
- 10. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Matlab Display on Large Screen Get Location & Orientation Esteblish3D Model Small Car Carry Phone Outline Basic part: APP
- 11. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Integrate and Display on Large Screen
- 12. 交大密西根学院 UM-SJTU Joint Institute Get Package Decode Matrix Operation Get Location & Orientation Outline Highlight 1: Combine sensor information
- 13. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Integrate and Display on Large Screen
- 14. 交大密西根学院 UM-SJTU Joint Institute Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Highlight 2: Mini-campus & small car
- 15. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Integrate and Display on Large Screen
- 16. 交大密西根学院 UM-SJTU Joint Institute Outline Integrate and Display on Large Screen Highlight 3: Large Screen Display
- 17. 交大密西根学院 UM-SJTU Joint Institute Video
- 18. 交大密西根学院 UM-SJTU Joint Institute Video
- 19. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Integrate and Display on Large Screen
- 20. 交大密西根学院 UM-SJTU Joint Institute Five Activities – Welcome Screen, Settings – Camera Calibration* – File Chooser* – SLAM *: Using templates Dependencies – Compiling: OpenCV 3.1, NDK v. 12 – Runtime: OpenCV Manager Android Application
- 21. 交大密西根学院 UM-SJTU Joint Institute Welcome Screen –JI Logo –Credits –References Android GUI Design
- 22. 交大密西根学院 UM-SJTU Joint Institute Settings – Preference auto save – Designate two files required by ORB-SLAM Android GUI Design
- 23. 交大密西根学院 UM-SJTU Joint Institute File Chooser – From: aFileChooser Camera Calibration – From: OpenCV Package Examples Android GUI Design
- 24. 交大密西根学院 UM-SJTU Joint Institute Loading takes about 20 s – ORB Vocabulary 138 MB – Prompt user if the OpenCV Manager is not ready Android GUI Design: SLAM
- 25. 交大密西根学院 UM-SJTU Joint Institute UDP IP address UDP IP port Pause/Resume SLAM Start/Stop UDP communication Mode notification text “High Performance” Android GUI Design: SLAM Control Panel
- 26. 交大密西根学院 UM-SJTU Joint Institute 3D Graphics (OpenGL) – Map points – Key frames – Key frame relations (path) Turned off in High Performance Mode Android GUI Design: SLAM SLAM Result Visualization
- 27. 交大密西根学院 UM-SJTU Joint Institute SLAM Result View – Show feature points Camera Preview – Support four sizes of frame capture – Same resolution but different field of view Android GUI Design: SLAM SLAM Processed Image View OpenCV Camera Preview
- 28. 交大密西根学院 UM-SJTU Joint Institute Drawbacks – Mapping data can not be recovered if reload the application – Accumulated error of ORB-SLAM only corrected if detects close-loop Android GUI Design: SLAM
- 29. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Integrate and Display on Large Screen
- 30. 交大密西根学院 UM-SJTU Joint Institute Highlight: SLAM + IMU Gyroscope Compass Accelerator Built-in Filter Functions: – Initialization – SLAM Alternative Inertia Measurement Sensor
- 31. 交大密西根学院 UM-SJTU Joint Institute Highlight: Car - phone connection UDP/TCP protocol Location: X,Y,(Z), angle (Rotational Matrix) Data Visualization: Mathematica Data Transfer
- 32. 交大密西根学院 UM-SJTU Joint Institute IMU & UDP & Matlab
- 33. 交大密西根学院 UM-SJTU Joint Institute Pure SLAM & 3D Visualization
- 34. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Integrate and Display on Large Screen
- 35. 交大密西根学院 UM-SJTU Joint Institute 2*3 meters 6 buildings 5 environment Painted walls Mini Campus
- 36. 交大密西根学院 UM-SJTU Joint Institute Visual SFM (Picture comparison) PMVS 2 (Sparse contruction) Mapping
- 37. 交大密西根学院 UM-SJTU Joint Institute 3D takes up too much space, so we change to 2D map Mapping
- 38. 交大密西根学院 UM-SJTU Joint Institute The Car 1st prototype 2nd prototype Minimize size: horizontal design vs. vertical design Decrease velocity: DC motor vs. step motor
- 39. 交大密西根学院 UM-SJTU Joint Institute Car Design Holder Front wheel Rear wheel Actuator board Arduino (with Bluetooth board) Battery Motor
- 40. 交大密西根学院 UM-SJTU Joint Institute Manufacture
- 41. 交大密西根学院 UM-SJTU Joint Institute Orientation Initialization Realtime SLAM & Sensor Data SLAM Available ? SLAM Data Sensor Data Send Package Get Package Decode Matrix Operation Get Location & Orientation Build Mini- Campus Esteblish3D Model Small Car Carry Phone Outline Integrate and Display on Large Screen
- 42. 交大密西根学院 UM-SJTU Joint Institute Display
- 43. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 44. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 45. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 46. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 47. 交大密西根学院 UM-SJTU Joint Institute Program Efficiency
- 48. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 49. 交大密西根学院 UM-SJTU Joint Institute Translational error: actual length of trajectory compared to 1 meter Rotational error: the angle of actual trajectory deviated Program Accuracy 1 m Translational error: 23% Rotational error: -6.26 deg/m
- 50. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 51. 交大密西根学院 UM-SJTU Joint Institute Program Complexity (CPU Usage)
- 52. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 53. 交大密西根学院 UM-SJTU Joint Institute Probability of Bugs Probability of bugs: Sensor Mode Total Time = 3.8% ± 11.4%
- 54. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation 10.5 min
- 55. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 56. 交大密西根学院 UM-SJTU Joint Institute Proportion of Image Size (App)
- 57. 交大密西根学院 UM-SJTU Joint Institute Proportion of Image Size (App) 73.38%
- 58. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation
- 59. 交大密西根学院 UM-SJTU Joint Institute Gantt Chart
- 60. 交大密西根学院 UM-SJTU Joint Institute Cost Analysis
- 62. 交大密西根学院 UM-SJTU Joint Institute Qian Shiyi qsy@sjtu.edu.cn (18817818361) Wei Shupeng weishupeng@163.com (18817818655) He Qinyue hqy0213@sjtu.edu.cn (18818273021) Liu Zhuohe liuzhuohe@sjtu.edu.cn (13764826899) Xu Zhentao xuzhentaottup@sjtu.edu.cn (13918842481) Contact Information
- 63. 交大密西根学院 UM-SJTU Joint Institute Video (First Car)
- 64. 交大密西根学院 UM-SJTU Joint Institute More Demo Video
- 65. 交大密西根学院 UM-SJTU Joint Institute Program Efficiency
- 66. 交大密西根学院 UM-SJTU Joint Institute Engineering Specification Target Value Camera Precision 12M pixels GPS Precision 10 m Gyroscope Precision 17.5 mdps Additional Hardware Price 0 $ Program Efficiency 0.1 s Program Accuracy Translation: 1.15% Rotation: 0.0027 deg/m Program Complexity (CPU Usage) 50% Probability of Bugs 5% Average User Learning Time 30 min Proportion of Image Size (App) 80% Validation Memory
- 67. 交大密西根学院 UM-SJTU Joint Institute Memory