Creating Digital Twins Using Rapid Development Techniques.pdf
0 likes246 views
Integrated Computer Solutions (ICS) provides innovative software solutions for developing next-gen products, with a focus on medical devices. The document discusses the creation of digital twins representing real-world devices and emphasizes the use of virtual and augmented reality (VR/AR) for enhanced training methods, particularly in healthcare. It outlines methodologies for integrating digital twins into training systems, highlighting benefits over traditional training methods.
Creating Digital Twins Using Rapid Development Techniques.pdf
- 1. Creating Device-Accurate Digital Twins Using ICS’ Rapid Development Techniques Jeff LeBlanc Director of Solutions Engineering jleblanc@ics.com
- 2. Established in 1987, Integrated Computer Solutions, Inc. (ICS) delivers innovative software solutions with a full suite of services to accelerate development of successful next-gen products. ICS is headquartered outside Boston in Waltham, MA with offices in California, Canada and Europe. ● UX and UI design services ● Full stack software development ● Cloud-native application development ● Full end-to-end product realization ● Focus on medical device development In 2018, the success of our in-house UX and Visual design studio has led us to convert it into a full-service digital agency called Boston UX. About ICS
- 3. A Few Of Our Customers
- 4. What is AR / VR / MR / XR ? AR - Augmented Reality, a mix of the real and the virtual world VR - Virtual Reality, a purely digital world MR - Mixed Reality, meaning the whole spectrum; also XR - Extended Reality,
- 5. What is a Digital Twin? “A digital twin is a digital representation of a real-world entity or system. The implementation of a digital twin is an encapsulated software object or model that mirrors a unique physical object, process, organization, person or other abstraction.” - Gartner Digital twins are used in many domains, including urban planning, aerospace, manufacturing and healthcare Many uses, such as testing, monitoring, simulation, and training
- 6. Training is critical for new and returning workers On-the-job training not always viable in critical areas ● Aerospace ● Defense ● Healthcare The Need for Training
- 7. Traditional training methods include: ● Reading ● Lectures ● Workshops VR/AR can provide an alternative to these methods These methods have severe limitations: ● Expensive, both time and money ● Geographically constrained ● May require actual hardware ● Often have low retention rates Traditional Training Methods
- 8. According to PWC (2022), students who used VR for training were: 4X faster to train than in a classroom 3.75X more emotionally connected to content than classroom learners 275% more confident in applying skills after training 4X more focused on learning than peers using e-learning methods Training in VR
- 9. Specific Case: Medical Devices Training for medical devices is particularly challenging ● Very expensive, so access is limited ● Operators have a critical need to understand them before using them in the field Good fit for creating trainer systems
- 10. 1 Create a device-accurate digital twin 2 Connect to simulators (also used during development) 3 Add scaffolding within virtual environment for lessons and analytics 4 At this point, you have a solid foundation for a trainer system Methodology for Creating Training Systems
- 11. Digital Twins in Extended Reality (XR)
- 12. Case 1: Creating an XR training system during development of the device Case 2: Recreating an existing device in XR Use Cases for XR-based Trainers
- 13. (Re)Creating Project RED in VR Project RED - Respiratory Emergency Device ICS worked on this open source low-cost ventilator during the pandemic https://redventilator.org/
- 14. Figma UX & UI Project RED screens Software Development Project RED screens in code App in Qt / QML Running software on device ICS Best Practice for Creating Embedded Devices ICS uses our Rapid Development Solution as part of a 3-step pipeline for creating embedded devices. This allows us to create Qt-based UIs in about half the time of hand coding. Rapid asset generation Rapid development Proven software architecture
- 15. Figma UX & UI Project RED screens Software Development Project RED screens in code ICS Best Practice for Creating Digital Twins Unity Running software in VR The same Qt code base can run in VR simply by re-compiling. Often, no code changes are needed!
- 16. ICS uses a standard layered architecture for many of our projects Real Hardware Simulator Communication Application Logic Presentation Qt / QML Hardware Simulator Software Architecture Signals Function calls
- 17. Unity WebGL Qt gets us to WebGL with little effort. This gets us to a digital twin of the device in VR Software Architecture Simulator Communication Application Logic Presentation Qt / QML Hardware Simulator
- 18. Project RED - Physical and Virtual Real VR
- 19. (Re)Creating an Existing Device in VR Figma UX & UI Existing Device screens Software Development Existing Device screens in code Recreating an existing device in VR is essentially the same process. ● UX designer recreates existing screens in Figma ● Software engineer connects the UI to simulators and implements navigation to mimic the device workflow ● Hardware can be done either in Figma or VR environments Rapid asset generation Rapid development Unity Running software in VR Rapid deployment
- 20. Recreating an Existing Device in XR This is easier than making the original device as all hardware interactions are simulated. Application logic, and even data output, can all be simulated as needed. UX team recreates the screens based on a reference device and manuals. Unity WebGL Simulator Communication Application Logic Presentation Qt / QML Hardware Simulator Signals Function calls
- 21. Now, we add scaffolding around the twin for lesson support From Twin to Trainer Directions Run a lesson in the VR environment Analytics How did the lesson go? Reporting Progress over time
- 22. Now, we add scaffolding around the twin for lesson support Building a Trainer - The Device in XR Directions Run a lesson in the VR environment Analytics How did the lesson go? Reporting Progress over time
- 23. VR Trainer Software Architecture The XR environment needs to communicate with the twin ● Respond to a hardware control ○ Physical button push or knob turn ● Display a hardware response ○ Turn on a light ● Respond to simulated UI controls ○ Virtual menu and prompts ● Convey information in XR
- 24. Simulator Communication Application Logic Presentation Qt / QML Hardware Simulator API (Unity Trainer Software) API (Unity Trainer Software) Communicate with simulators through API VR Trainer Software Architecture The XR environment needs to communicate with the twin Signals
- 25. VR Trainer Software Virtual menu for patient vitals
- 26. With a working digital twin and training environment, creating an Augmented Reality (AR) version in Unity becomes easy (mostly) Next Steps: Continue Training by Extending to AR
- 27. Next Steps: Continue Training by Extending to AR
- 28. Thanks For Listening! Questions? Jeff LeBlanc, Director of Solutions Engineering jleblanc@ics.com http://www.ics.com