![Related Works
• Researches related to clothes fabrication
– Stefan et.al. (ISMAR2011)：Fitting simulation
based pre-captured clothes with various poses
– Igarashi et.al.(UIST2002)：Interactive simulation
of sewing the parts and fitting on CG doll
• MR-based virtual fitting systems
– EON Reality Eon Interective Mirror
simple cloth simulation
– MR Fitting Simulation (Our previous work)
• Superimposing scanned clothes CG model in real-time
• Did not adopt measurement of target’s body and
rigorous cloth simulation
6
[Igarashi 2002]
[Stefan 2011]
[Eon Interactive Mirror]
[Ichikari 2014]](https://image.slidesharecdn.com/csun2018jounaltrack-180323162726/85/Fitting-Simulation-Based-on-Mobile-Body-Scanning-for-Wheelchair-Users-6-320.jpg)
Fitting Simulation Based on Mobile Body Scanning for Wheelchair Users
- 1. Fitting Simulation Based on Mobile Body Scanning for Wheelchair Users Ryosuke Ichikari, Masaki Onishi, Takeshi Kurata National Institute of Advanced Industrial Science and Technology (AIST), Japan CSUN AT Conf. 2018＠San Diego Session:IND-081 March 23rd, 2018 (8:00 AM)
- 2. Fashion for wheelchair users • Fashion: Important for the wheelchair users to improve quality of life (QoL) and return to society • Problems of making their garments: – lack of items, costs (order-made or modified from normal garments) – market size, difficulty for keeping business – consumers have highly individualized needs Fashion show at Koku-Reha collection 2012 2
- 3. Projects for people with physical disabilities and special needs • Fashion-able (2011-, EU) – Fashionable footwear for diabetic feet – Fashionable clothing for wheelchair users – High-performing textile compression bandages • Our projects (2012-, Japan) – Sustainable research project for manufacturing ideal clothes for wheelchair users – Simultaneously improving functionality, design, and economic efficiency – Research institutes, a fashion school, apparel-CAD companies, garment manufacturers are involved. 3
- 4. Research elements in our project 4
- 5. Virtual Fitting App. • Virtual trial fitting anywhere-anytime with smartphone. • Simulation of garment functionality and its design • Mobile 3D body measurement by RGB-D camera 5
- 6. Related Works • Researches related to clothes fabrication – Stefan et.al. (ISMAR2011)：Fitting simulation based pre-captured clothes with various poses – Igarashi et.al.(UIST2002)：Interactive simulation of sewing the parts and fitting on CG doll • MR-based virtual fitting systems – EON Reality Eon Interective Mirror simple cloth simulation – MR Fitting Simulation (Our previous work) • Superimposing scanned clothes CG model in real-time • Did not adopt measurement of target’s body and rigorous cloth simulation 6 [Igarashi 2002] [Stefan 2011] [Eon Interactive Mirror] [Ichikari 2014]
- 7. Overview of virtual fitting app. 7 Runtime B. Deforming Standard Model by Measurement C1.Virtual Fitting for Standard Model Runtime A. Mobile Body Scanning Runtime C2. Virtual Fitting for Deformed Standard Model Measured Data Runtime D. Visualization Only Clothes Pattern data Standard model Deformed Standard Model Fitting Result for Standard Model Propagating the fitting result to the deformed standard model, collision simulation with the seat, and fitting adjustment Visualizing fitting result with target body in mobile app. Matching correspondence of body parts, posture, and deformation Fitting Preliminarily Operation
- 8. Functionalities of virtual fitting app. (1)Ability to allow wheelchair users to try clothes virtually while they are in a seated position (i.e., on the wheelchair) (2)Ability to realize automated and accurate fitting simulation using examples (3)Ability to realize three-dimensional physical measurement function using a common mobile device (4)Ability to verify fitting results by three-dimensional visualization using a common mobile device (5)Ability to develop a form database by collecting the results of three- dimensional measurements on a server (6)Ability to provide reliable data required in the garment manufacturing industry (CAD), and support CAD data format. 8
- 9. System Configuration Microsoft Azure Virtual Windows Machine PostgreSQL (on Linux Virtual Machine) Apache (Web Server) Server-sideClient-side Android (for smartphone with Tango) Unity (Visualization & User Interface ) Client-side application Server-side Script (PHP Script) HTTPS DS Manager (IF for DS Body) DS Body (Fitting Engine) Body Shape DB Garment DB User DB Fitting DB Tango SDK (Body Scanning) Windows (for tablet) Unity (Visualization & User Interface ) Client-side application RealSense SDK (Body Scanning) RealSense Camera (e.g. R200, SR300) USB HTTPS 9
- 10. Device (Tango) • Google’s AR(Augmented Reality) Platform • Enables camera tracking and 3D reconstruction by a depth camera • Can be utilized as a mobile body scanner 10 Lenovo’s phablet Phab 2 pro
- 11. Step A 11 Runtime B. Deforming Standard Model by Measurement C1.Virtual Fitting for Standard Model Runtime A. Mobile Body Scanning Runtime C2. Virtual Fitting for Deformed Standard Model Measured Data Runtime D. Visualization Only Clothes Pattern data Standard model Deformed Standard Model Fitting Result for Standard Model Propagating the fitting result to the deformed standard model, collision simulation with the seat, and fitting adjustment Visualizing fitting result with target body in mobile app. Matching correspondence of body parts, posture, and deformation Fitting Preliminarily Operation
- 12. Mobile body measurement without contact 12
- 13. Step B 13 Runtime B. Deforming Standard Model by Measurement C1.Virtual Fitting for Standard Model Runtime A. Mobile Body Scanning Runtime C2. Virtual Fitting for Deformed Standard Model Measured Data Runtime D. Visualization Only Clothes Pattern data Standard model Deformed Standard Model Fitting Result for Standard Model Propagating the fitting result to the deformed standard model, collision simulation with the seat, and fitting adjustment Visualizing fitting result with target body in mobile app. Matching correspondence of body parts, posture, and deformation Fitting Preliminarily Operation
- 14. Correspondence between a standard model and the measured model • Scanned geometry is converted to a homology model – Matching correspondence between a homology model and scanned geometry. – Fitting simulation engine assumes that a homology model is used as a target body – Scanned geometries are noisy • AIST’s Dhaiba model is used as a homology model 14
- 15. Matching joint angles of bodies with images • User can changing joint angles while checking appearance from various viewing angles (semi-automated process) ⇒ Supposed to be fully-automated in the future 15
- 16. Body shape fitting by blend shape deformation • Deforming Dhaiba model by changing blend weights between body with various heights and weights • Checking the similarity of shape by calculating distance between the closest pair of points on the surfaces 16
- 17. Step C1 17 Runtime B. Deforming Standard Model by Measurement C1.Virtual Fitting for Standard Model Runtime A. Mobile Body Scanning Runtime C2. Virtual Fitting for Deformed Standard Model Measured Data Runtime D. Visualization Only Clothes Pattern data Standard model Deformed Standard Model Fitting Result for Standard Model Propagating the fitting result to the deformed standard model, collision simulation with the seat, and fitting adjustment Visualizing fitting result with target body in mobile app. Matching correspondence of body parts, posture, and deformation Fitting Preliminarily Operation
- 18. Fitting to Standard Model (Preliminarily Operation) • Adopts Dressing Sim Body (DSBody) – Fitting simulation engine developed by Digital Fashion Ltd. • Simulation results for the human model can be propagated to another human model with homology Fitting for standing standard model (Semi-automatic process with GUI) Fitting for sitting model with different pose and shape (Automatic) Automatic propagation 18 Step C1 Step C2
- 19. Detailed operation of fitting for standard model 19
- 20. Step C2 20 Runtime B. Deforming Standard Model by Measurement C1.Virtual Fitting for Standard Model Runtime A. Mobile Body Scanning Runtime C2. Virtual Fitting for Deformed Standard Model Measured Data Runtime D. Visualization Only Clothes Pattern data Standard model Deformed Standard Model Fitting Result for Standard Model Propagating the fitting result to the deformed standard model, collision simulation with the seat, and fitting adjustment Visualizing fitting result with target body in mobile app. Matching correspondence of body parts, posture, and deformation Fitting Preliminarily Operation
- 21. Interface for selecting garments • User can select the garments to be fitted on their body from choices with thumbnails • Can be connected to web-site of manufacturers • User can trigger starting simulation after garment selection 21
- 22. Step D 22 Runtime B. Deforming Standard Model by Measurement C1.Virtual Fitting for Standard Model Runtime A. Mobile Body Scanning Runtime C2. Virtual Fitting for Deformed Standard Model Measured Data Runtime D. Visualization Only Clothes Pattern data Standard model Deformed Standard Model Fitting Result for Standard Model Propagating the fitting result to the deformed standard model, collision simulation with the seat, and fitting adjustment Visualizing fitting result with target body in mobile app. Matching correspondence of body parts, posture, and deformation Fitting Preliminarily Operation
- 23. Visulization of the virtual fitting results • Fitting results are displayed on the client smartphone – Full-CG visualization: Fitting simulation result on to deformed Dhaiba model – AR visualization: Simulation results are superimposed onto real images captured during body scanning 23 Full-CG visualization AR visualization
- 25. Remarkable Result 1 • Results of simulation using specially designed pattern for sitting position 25 Results with normal pattern Results with dedicated pattern for sitting position
- 26. Remarkable Result 2 26 • Comparison between with or without collision detection (a) Fitting without collision simulation with the seat (b) Fitting with collision simulation with the seat
- 27. Numerical Evaluation Correct Estimated Chest girth 97 cm 98.8 cm Waist 90 cm 86.3 cm Thigh length 56 cm 53.3 cm 27 • Comparison between estimated lengths (yellow lines) and actually measured lengths of the chest girth, waist, thigh length • Will gather more data of body after releasing the app. – Plan for holding the experiment for collecting the body data has been approved from AIST.
- 28. Conclusion & Future Works • Proposed virtual fitting app for wheelchair users – The user can check fitting result they remain sitting on the wheelchair – The app can collect the body shape data, which are required for designing the garments for them. • Future works – Releasing app. – Statistical analysis of the body data collected by the released app – Automation of the body pose and shape matching procedures (AI might be utilized for this) 28
- 29. Questions? • Contact: Ryosuke Ichikari, Ph.D. r.ichikari@aist.go.jp • This presentation is on Journal Track, you can download the draft paper from the CSUN web-site. 29