Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
1 like671 views
The document discusses the development of a biohybrid robotic jellyfish to enhance ocean monitoring capabilities due to over 80% of the ocean being unexplored. It outlines the advantages of using jellyfish as a model organism for creating energy-efficient, minimally invasive robots that can collect data with reduced environmental impact. The research focuses on understanding jellyfish swimming mechanics and using technological advancements to improve control and adaptability for ocean exploration.
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
- 1. Biohybrid robotic jellyfish for future applications in ocean monitoring Nicole W. Xu, James Townsend, Jack Costello, Sean Colin, John O. Dabiri Stanford University HPC-AI Stanford Conference April 21, 2020
- 2. 2 Nicole W. Xu Stanford University April 21, 2020 How much do we know about the ocean? More than 80% of our ocean is unmapped, unobserved, and unexplored.
- 3. 3 Importance of ocean monitoring Track changes in temperature, acidity, concentrations of nutrients Discovery of new species or behaviors National GeographicMBARI Toby Hudson Greg Torda Nudibranch Coral Bleaching Algal Blooms Nicole W. Xu Stanford University April 21, 2020
- 4. 4 Current methods of ocean monitoring Remotely operated vehicles (ROVs) Autonomous underwater vehicles (AUVs) Slocum Glider, AUVACRJE International Challenges: Costly Bulky High power consumption Vehicle wakes Yoda Productions Nicole W. Xu Stanford University April 21, 2020
- 5. 5 How can we expand ocean monitoring tools? Nicole W. Xu Stanford University April 21, 2020 Goal: Use technology to collect data about ocean environments Needs Able to explore new locations Minimal power Minimal cost Minimal disturbance to aquatic life How we can achieve this: Combine technological advances, experimental work, and computational tools National Geographic
- 6. 6 How can we expand ocean monitoring tools? Nicole W. Xu Stanford University April 21, 2020 Goal: Use technology to collect data about ocean environments Needs Able to explore new locations Minimal power Minimal cost Minimal disturbance to aquatic life How we can achieve this: Combine technological advances, experimental work, and computational tools National Geographic
- 7. 7 Bioinspired robots offer advantages Energy-efficient, bioinspired aquatic vehicles that leave natural wakes to minimize environmental disturbances Collect images, data, samples from new locations Tadesse et al., Smart Materials and Structures, 2012Katzschmann et al., Science Robotics, 2018 MantaDroid Fish Manta ray Jellyfish Nicole W. Xu Stanford University April 21, 2020 Which animal model do we choose?
- 8. 8 Jellyfish as a model organism Nicole W. Xu Stanford University Despite 500 million years of evolutionary pressure, the body structure of moon jellyfish has remained largely unchanged. Aurelia aurita • Simple body structure • Swimming is tied into feeding, escape behaviors, etc. • Ubiquitous 5 cm April 21, 2020
- 9. Nicole W. Xu Stanford University 9 COT is a metric of energy efficiency Opposite of mpg COT = Energy Mass x Distance April 21, 2020 Tesla
- 10. Nicole W. Xu Stanford University 10 What can we learn from jellyfish? Jellyfish have a low cost of transport compared to other animals. Jellyfish are energy efficient. Gemmell et al., PNAS, 2013 Flying Running Swimming COT = Energy Mass x Distance April 21, 2020
- 11. Spectrum of bioinspired jellyfish constructs Nicole W. Xu Stanford University 11 Nawroth et al, Nature Biotech, 2012 Biological Construct Limitation: Survival in a few specific environments Villanueva et al, PLoS ONE, 2014 Robotic Construct Limitation: High power consumption April 21, 2020
- 12. 12 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University April 21, 2020
- 13. 13 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). April 21, 2020
- 14. 14 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). April 21, 2020
- 15. Nicole W. Xu Stanford University Pacemakers: sensory organs that innervate the swim muscle exumbrellar surface (top) subumbrellar surface (underside) 5 cm 5 cm 15 Swim anatomy April 21, 2020
- 16. Swim motion Power stroke (active contraction and thrust) Recovery stroke (passive relaxation and feeding current) All-or-nothing muscle activation Nicole W. Xu Stanford University 16 5 cm April 21, 2020
- 17. 17 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). April 21, 2020
- 18. External control of swimming Electrodes act as “artificial pacemakers” to activate the muscles. 18 V t Input(s): Signal voltage Signal pulse width Signal period (inverse of frequency) Electrode location on the bell Output: Pulses (tag displacement) Nicole W. Xu Stanford University April 21, 2020
- 19. In-dish experiments Nicole W. Xu Stanford University 19 2.5 cm April 21, 2020
- 20. Natural pulses 20 Nicole W. Xu Stanford University 2.5 cm April 21, 2020 TagDisplacement(pixels)
- 21. Natural vs. stimulated pulses 21 Nicole W. Xu Stanford University April 21, 2020 Natural TagDisplacement(pixels) TagDisplacement(pixels) 0.25 Hz TagDisplacement(pixels) 2 Hz TagDisplacement(pixels) 1 Hz
- 22. Frequency spectrum of natural pulses 22 Unstimulated cases, N = 12 animals Natural motion includes a bout frequency and successive pulses per bout (a spread between 0.4-1 Hz). 0.16 Hz 0.4-1 Hz Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020 Amplitude TagDisplacement(pixels) Example displacement, N = 1
- 23. Frequency spectrum of stimulated pulses 23 Stimulated at 1 Hz, N = 10 animals 1.02 Hz Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020 Amplitude TagDisplacement(pixels) Example displacement, N = 1
- 24. N = 10 N = 10 N = 10 N = 10 N = 10 N = 10 N = 9 Stimulated frequency response (0.25 – 2 Hz) 24 N = 10 animals PowerSpectralDensity Frequency Output (Hz) Nicole W. Xu Stanford University Successful one-to-one response Max pulse frequency according to a proposed bio limit (Bullock, J Cell Comp Physio, 1943) Peak frequency of unstimulated animals (natural behavior) Xu and Dabiri, Science Advances, 2020 April 21, 2020 Successful frequency responses between 0.25 to 1 Hz
- 25. Electrode placement (1 Hz) Where is the optimal location for electrode placement? Stomach Midway Pacemaker Margin Sensor placement will also be relevant when we place temperature/salinity/pH probes on jellyfish in the ocean Nicole W. Xu Stanford University 25April 21, 2020
- 26. Electrode placement (1 Hz, N = 10) Nicole W. Xu Stanford University 26 Stomach Midway Margin Rhopalia . . . . . . . . . . . . x x x x Electrodes can be placed anywhere for consistent muscle responses April 21, 2020
- 27. Using CFD to determine optimal sensor placement Nicole W. Xu Stanford University 27 Stomach Midway Margin Rhopalia . . . . . . . . . . . . x x x x Future work: computational fluid dynamics (CFD) to determine optimal sensor placement for untethered experiments April 21, 2020 Hoover et al., JFM, 2017 Gemmell et al., J. Royal Soc. Interface, 2015
- 28. Using CFD to determine optimal sensor placement Nicole W. Xu Stanford University 28 Stomach Midway Margin Rhopalia . . . . . . . . . . . . x x x x Future work: computational fluid dynamics (CFD) to determine optimal sensor placement for untethered experiments April 21, 2020 Hoover et al., JFM, 2017
- 29. 29 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). April 21, 2020
- 30. Swim controller design 1 cm 1 cm 1 cm 30 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 Design choices: Small Neutrally buoyant Low power Inexpensive Housing Cap Microcontroller Battery Electrodes April 21, 2020
- 31. Swim controller design 31 Nicole W. Xu Stanford University April 21, 2020
- 32. 32 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). April 21, 2020
- 33. Frequency vs. swimming speed 0.9 m 1.8 m swim controller jellyfish camera Two electrodes (straight swimming) Frequency (Hz) 0.25-1.00 Hz 33 Nicole W. Xu Stanford University April 21, 2020
- 34. Frequency vs. swimming speed 34 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020
- 35. Frequency vs. swimming speed 35 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020
- 36. Robotic control can increase speeds up to 3x Swim controller frequency (Hz) Speed(bodydiameterss-1) 36 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 AR (aspect ratio) = height diameter AR = 0.3 Enhancementfactor =stimulatedspeed/unstimulatedspeed April 21, 2020
- 37. Robotic control can increase speeds up to 3x Swim controller frequency (Hz) Enhancementfactor 37 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 AR = 0.2 AR = 0.3 Speed(bodydiameterss-1) AR (aspect ratio) = height diameter Enhancementfactor =stimulatedspeed/unstimulatedspeed April 21, 2020
- 38. Robotic control can increase speeds up to 3x Swim controller frequency (Hz) 38 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 AR = 0.2 AR = 0.3 Speed(bodydiameterss-1) AR (aspect ratio) = height diameter Enhancementfactor =stimulatedspeed/unstimulatedspeed April 21, 2020 We can increase jellyfish swimming up to 2.8 times. Enhancement depends on body morphology/shape.
- 39. Mechanistic model 39 Nicole W. Xu Stanford University November 26, 2019 åF = mj du dt Daniel, Canadian Journal of Zoology, 1983 Xu and Dabiri, Science Advances, 2020 Theoretical model that incorporates body kinematics in thrust, drag, and acceleration reaction forces T - D - AR = mj du dt T = rw Asub æ è çç ö ø ÷÷ dVsub dt æ è ç ö ø ÷ 2 D = 1 2 Cd rw Aj u2 AR =arj Vj du dt a = 2ht dt æ è çç ö ø ÷÷ 1.4 Thrust Drag Acceleration Reaction mj mass of the jellyfish u speed ⍴w density of saltwater Asub area of the jellyfish subumbrella Vsub volume of the jellyfish subumbrella Cd drag coefficient Aj area of the jellyfish ht height of the jellyfish dt diameter of the jellyfish ⍴w density of the jellyfish Vj volume of the jellyfish
- 40. Mechanistic model 40 Nicole W. Xu Stanford University November 26, 2019 åF = mj du dt Frequency (Hz)Frequency (Hz) Speed(bodydiameterss-1) Daniel, Canadian Journal of Zoology, 1983 Xu and Dabiri, Science Advances, 2020 Theoretical model that incorporates body kinematics in thrust, drag, and acceleration reaction forces Speed(bodydiameterss-1)
- 41. 41 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). April 21, 2020 We can increase speeds, but at what cost to the animal?
- 42. Measuring frequency vs. respiratory rate 25 cm 10 cm oxygen probe jellyfish 2 electrodes power supply sealed top O2Concentration(μmolO2L-1) Time (min) Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 42April 21, 2020
- 43. Measuring frequency vs. respiratory rate O2Consumption(μmolO2L-1hr-1g-1) 0 Hz 0.25 Hz 0.50 Hz 0.88 Hz 43 Nicole W. Xu Stanford University April 21, 2020 25 cm 10 cm oxygen probe jellyfish 2 electrodes power supply sealed top
- 44. Metabolic costs 44 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020
- 45. Metabolic costs 45 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020
- 46. Metabolic costs 46 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 P ~ v3 April 21, 2020
- 47. Metabolic costs 47 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020 åF = mj du dt
- 48. Metabolic costs: efficient swimming 48 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 Robotic control can enhance swimming speed up to 3x, with only a 2x increase in cost of transport. April 21, 2020
- 49. 49 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). April 21, 2020 But the electronic part ALSO consumes power.
- 50. Low power compared to other swimming robots Biohybrid (this work) 50 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020
- 51. Low power compared to other swimming robots Biohybrid (this work) 51 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020 Frame et al, Bioinsp Biomim, 2019
- 52. Low power compared to other swimming robots Biohybrid (this work) 52 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020 Park et al, Science, 2016
- 53. Low power compared to other swimming robots Biohybrid (this work) 53 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020 Hydroid
- 54. Low power compared to other swimming robots Biohybrid (this work) This biohybrid robotic jellyfish uses 10-1000x less external power per mass compared to existing swimming robots. 54 Nicole W. Xu Stanford University Xu and Dabiri, Science Advances, 2020 April 21, 2020
- 55. Spectrum of bioinspired jellyfish constructs Nicole W. Xu Stanford University 55 Biological Construct Limitation: Survival in a few specific environments Robotic Construct Limitation: High power consumption Biohybrid Robot Advantages: Low power & cost Adaptable to many environments April 21, 2020 Nawroth et al, Nature Biotech, 2012Villanueva et al, PLoS ONE, 2014
- 56. 56 Building a biohybrid robotic jellyfish Nicole W. Xu Stanford University 1. Understand how jellyfish naturally swim. 2. Characterize how to control jellyfish muscle contraction (tethered in-dish experiments). 3. Build a portable swim controller. 4. Characterize the artificial control of jellyfish swimming (free-swimming experiments). 5. Improve robotic controllability toward ocean monitoring (ongoing work). Robustness and maneuverability in real environments April 21, 2020
- 57. Robustness: robot survivability in the ocean Marine Biological Laboratory in Woods Hole, MA Scuba divers: Jack Costello, Sean Colin, James Townsend 57 Nicole W. Xu Stanford University April 21, 2020
- 58. Lab work 0.9 m 1.8 m swim controller jellyfish camera 58April 21, 2020
- 59. Field work swim controller jellyfish 5.5 mcamera rope markersN = 4 animals f = 0, 0.50, 0.75 Hz
- 60. 0.50 Hz, two electrodes Nicole W. Xu Dabiri Lab, Stanford University 60October 4, 2019
- 63. Field work swim controller jellyfish rope markersExternally driven jellyfish can swim faster than normal jellyfish, similar to lab experiments 0 Hz: 1.8-3.5 cm/s 0.50 Hz: 3.1-4.4 cm/s 0.75 Hz: 2.8-4.9 cm/s
- 64. 64 How can AI help biohybrid jellyfish research? Nicole W. Xu Stanford University April 21, 2020
- 65. 65 Using computer vision to track animals Nicole W. Xu Stanford University April 21, 2020 Google AI
- 66. 66 Using computer vision to track animals Nicole W. Xu Stanford University April 21, 2020
- 67. 67 How can we expand ocean monitoring tools? Nicole W. Xu Stanford University April 21, 2020 Goal: Use technology to collect data about ocean environments Needs Able to explore new locations Minimal power Minimal cost Minimal disturbance to aquatic life How we can achieve this: Combine technological advances, experimental work, and computational tools National Geographic
- 68. Next steps toward using biohybrid jellyfish robots for ocean monitoring 68 Nicole W. Xu Stanford University Technology Sensors Science Lab experiments to control more complex maneuverability (turning) Test various jellyfish species Field experiments and ocean monitoring April 21, 2020 Technology Science Computation
- 69. Next steps toward using biohybrid jellyfish robots for ocean monitoring 69 Nicole W. Xu Stanford University Computational tools AI for image segmentation and object tracking in various environments CFD for optimal sensor placement and predicting animal trajectories for closed loop feedback April 21, 2020 Technology Science Computation
- 70. Acknowledgements • Members of the Dabiri lab • Cabrillo Marine Aquarium • Marine Biological Laboratory - Jack Costello - Sean Colin - James Townsend - Brad Gemmell • NSF Graduate Research Fellowship Weiland Family Fellowship Timothy G. Shi Graduate Fellowship Contact info: nicolexu@stanford.edu web.stanford.edu/~nicolexu 70 Nicole W. Xu Stanford University April 21, 2020