User Friendly Virtual Clothes System Based on Simulation and Visualization using RGBD Camera

68 International Journal for Modern Trends in Science and Technology
Volume: 2 | Issue: 09 | September 2016 | ISSN: 2455-3778IJMTST
User Friendly Virtual Clothes System Based
on Simulation and Visualization using RGBD
Camera
Vishruti Patel1
| Varsha Satpute2
| Shivani Kuhikar 3
| Arun Hattarge4
1,2,3 Student, Computer Dept., Jaywantrao Sawant College of Engineering, Pune, Maharashtra, India
4Professor, Computer Dept., Jaywantrao Sawant College of Engineering, Pune, Maharashtra, India
Now a days due to the commercial strength, virtual try-on of different things such as shirts, pants, skirts
etc has received much attention. The paper represents a system for 3d virtual clothes try-on that enables a
user to see himself/herself wearing virtual clothes on the screen. it is possible by just looking at a screen,
without taking off his/her actual clothes. It is helpful for online shopping or fashion recommendation to nail
down the selections to few designs and sizes. The user can select various virtual clothes for trying on the
screen. The major contribution of this project is that we automatically sets an invisible (or partially visible)
avatar based on the user’s body size and the skin color and use it for proper clothes fitting, alignment and
clothes simulation in our virtual try-on system. The system physically characterize the selected virtual
clothes on the user’s body which help the user to see virtual clothes fitting from multiple angles as he/she
moves.
KEYWORDS: 3D-2D alignment, body customization, mixed reality, skin color matching and real-time cloth
simulation, virtual try-on.
Copyright © 2016 International Journal for Modern Trends in Science and Technology
All rights reserved.
I. INTRODUCTION
Virtual try-on applications have become
popular in re- cent years because they allow users
to see themselves wearing different clothes without
the eﬀort of changing them physically. This helps
users to quickly judge whether they like a
garment or not, and also garment is fit or not
which in turn allows retail shops to sell more in
less time.In some previous solution display
garment pictures as retexture images from a single
camera. These approaches are not directly capable
of producing arbitrary viewing angles. Moreover,
garments do not adapt their size and shape to the
user due to their static nature. Other solutions
address these issues by reconstructing 3D clothes
models.
Fig.1. The Virtual Mirror system replaces color and
texture of a t-shirts.
These approaches can produce the desired
output, but require motion capture to track the
user’s position over time. One method is proposed
ABSTRACT

User Friendly Virtual Clothes System Based on Simulation and Visualization using RGBD Camera
where a user can be displayed wearing previously
recorded garments. To achieve this by creating a
garment database that stores the appearance of a
worn garment over time. These recordings are not
required to be from the same user and are
performed using the same multi-camera device
that is used for augmentation.In virtual clothes
system, users stand infront of camera can see
themselves on a screen which shows them
wearing different clothes. User can see
themselves from arbitrary viewpoints. The system
therefore needs to capture and render the user at
interactive rates while augmenting his or her body
with garments.
Fig2.Structure Of Virtual Dressing
II. RELATED WORK
A. Body Customization
Avatar customization can potentially be a strong
tool in modeling and simulation applications.
Rather than having to create models with different
body shapes and sizes for a specific application, it
is much economical in terms of time and effort, if a
generic model can be modified realistically based
on some parameters. measurements such as
height, shoulder width, waist height and arm
lengths can be obtained directly from a RGB-D
camera, and can be used to customize an avatar
which is suitable for virtual try-on[1].
B. Height Estimation
Virtual clothes need to be rescaled according to
users’ body size, for good fitting and try-on
experiences.In virtual try on sytem pepar used two
methods to estimate a user’s shoulder height[2].
The first one simply uses the neck to feet height
difference, when both the neck and the feet joints
are detected by Kinect skeletal tracking SDKs.
sometimes the feet are not located within the field
of view of Kinect. In such scenarios, estimate the
neck height from the tilting angle of the Kinect
sensor,the depth of the neck joint in the Kinect
depth image,and the offset of the neck joint with
respect to the center point of the depth image. After
the shoulder/neck height is estimated, then
uniformly resize the digital clothes in three
dimensions for a better fit to the user’s body.
C. Cloths Segmentation and Image Preprocessing
Hilsmann et al. created a databases of different
body positions of different levels of details to allow
interactive visualization and retexturing of cothes.
Here two modes are used for the clothes
segmentations.One is the Detection Mode and
another is Tracking Mode .During the Detection
Mode the algorithm searches for the clothes in the
image using the apriori knowledge of the color of
the clothes and the assumption that it contains a
rectangular highly textured region. Additionally, in
this mode the mesh that is used as motion model to
estimate the deformation of the texture in image
plane. During the Tracking Mode here segments
are used the clothes region in order to give it a new
color. Assume that if the clothes is in the image, it
is the largest area of that color and all parts of the
shirt are connected. With these assumptions can
use a very simple but efficient approach to detect
and segment the clothes in the image that is robust
against illumination changes and changing
background.
First,author Hilsmann et.al. transform all pixels
in the image into the normalized RGB-colorspace
which is more invariant to changes of surface
orientation relatively to the light source. then find
the largest blob by evaluating the normalized pixel
values and fill all holes in that region. Using this
very general method for segmentation ensures that
all parts of the shirt are segmented, including dark
shadows or highlighted regions. During the
Tracking Mode user can use this segmentation
mask to recolor the untextured part of the shirt.

Volume: 2 | Issue: 09 | September 2016 | ISSN: 2455-3778IJMTST
In the Detection Mode proceed further by
searching for a highly textured region in the
detected green region to initialize the mesh used for
tracking. Then find and sample the contours of the
texture defining the positions of the border mesh
vertices and interpolate all inner vertex position
points. By this method we assure that if the texture
is already deformed in the model frame.
D. Size Estimation
Size Estimation[5][7] method was indroduced by
Shreya Kamani et al. when the user calibrates
himself/herself to the Kinect sensor[6], his/her
size and girth is estimated. This information is
used to achieve a better fit of the virtual clothing.
The length of each limb is taken by computing the
distance between each joint from the skeleton. The
size of the body is taken by estimating[ the girth of
the chest on a number of points.The girth of the
user can be measured by two methods:
1. Computing the distance between each point
on the line .
2. Taking 3 points (outer left, center and
outer right) and compute the distance .
Although the first solution seems more
appropriate and easy to calculate, it is susceptible
to a noisy sensor and cloth folding. Even the
slightest fold will drastically hamper the users
estimated girth. The second option, although not
ideal, proved more accurate than the first method.
To calculate the estimated size of the user, an
average over 20 frames is taken for better
approximation.
E. Deformable Surface Tracking
Once the clothes has been detected and the mesh
has been initialized track the textured region of
the shirt in an optical-flow based approach. In
exploit the optical flow constraint quation and
regularize the optical flow field[3] with a predefined
mesh- based motion model[4]. The best
transformation can then be determined by
minimizing a quadratic.
𝐸 = (𝛻𝐼(𝑥𝑖, 𝑦𝑖)𝑛
𝑖=1 .d(xi,yi)+
𝑑𝑖
𝑑𝑡
(xi,yi)2
Fig. Skeleton Tracking
(xi, yi) denotes the spatial derivatives of the image I
at pixel position [xi, yi]T and ?I ?t (xi, yi) denotes the
temporal gradient between two images. d (xi, yi)
denotes the displacement vector at position [xi,
yi]T and is defined by the motion model. n is the
number of pixels selected for contribution to the
errorfunction, i.e. pixels where the gradient is
non-zero.
F. Skin-Tone Matching
In paper Virtual Try-0N System[2][1] author
presented SKIN-TONE MATCHING[4] Besides
editing the mesh geometry according to the user's
sizes,skin color matching is the another
requirement for creating a tool alike avatar. In this
method they are using the user's face skin color to
adaptively change the avatar's body skin color.This
transformation consist of three steps .In first step
facial features are located using the active shape
model(ASM)technique. In the second step,they are
using piecewise linear curves represent the cheek
areas and the extract cheep patches.finally they
are applying global color transfering method to
shift the color of the avatar body face patches. In
given fig they are showing original face image and
face jointed lines and this image is presented by[1].

User Friendly Virtual Clothes System Based on Simulation and Visualization using RGBD Camera
To capture face skin patches[1], one common
technique is applied for transfer the RGB images to
YCbCr color space and threshold the chrominance
components. The pixels with their chromatic value
falling in a certain range are classified as skin-color
pixels. However, in most uncontrolled
environments, some background clothes or hair
may have similar colors as face. Moreover, the lips
and the areas between eyebrows and upper eyelids
(or even the eyebrows in some cases) are easily
misclassified as skin areas which are not perfect
skin patches to be used for skin-tone transfer.
Another problem in skin tone extraction is that the
appearance of skin is usually quite different under
different viewing and lighting conditions.
Additionally, due to different scattering properties
of skin[1], cellular and 3D topographic structure of
the face[7], there are usually highlights in the
forehead, nose and chin areas.
G. Virtual Clothes
Virtual cothes[4][10] step is to dress the
tracked user. 3-D meshes[5][7] of clothing are
designed in Blender and imported to Unity. As
per our research, Unity provides two different
cloth components that can be added to a mesh:
interactive cloth and skinned cloth . Both
components have features such as stretch,
damping and thickness to give a real clothing
experience.
III. CONCLUSION
In this study, a virtual mirror system is designed
for the purpose of cloth changing room. Our
motivation here is to increase the time efficiency
and improve the accessibility of clothes try-on by
creating a virtual dressing room environment. The
system exchanges the colour and the texture of a
shirt while the person wearing the shirt can move
freely in front of the mirror and even perform
elastic deformations of the cloth like stretching and
bending or move toward or away from the camera.
Besides these results, one important characteristic
of our system is that it uses very simple
application. These algorithm achieves realism by
using image-based rendering, which makes it a
good alternative to manually model garments.
REFERENCES
[1] Miaolong Yuan, A Mixed Reality Virtual Clothes
Try-On System,IEEE TRANSACTIONS ON
MULTIMEDIA, VOL. 15, NO. 8, DECEMBER
2013.
[2] A. Divivier, R. Trieb, and A. e. a. Ebert, “Virtual
try-on using kinetic camera: Topics in realistic,
individualized dressing in virtual reality,” in
Proc. of Virtual and Augmented Reality Status
Conference, Leipzig, Germany.
[3] A. Hilsmann and P. Eisert, “Tracking and
retexturing cloth for real- time virtual clothing
applications,” in Proc. Mirage 2009—Comput.
Vis./Comput.Graph.Collab.Technol.andApp.,R
ocquencourt,France, May 2009.
[4] VIDEO-DRIVEN ANIMATION OF HUMAN
BODY SCANS Edilson de Aguiar, Rhaleb Zayer.
[5] International Journal of Advanced Computer
Technology (IJACT) ISSN:2319-7900 VIRTUAL
TRIAL ROOM USING AUGMENTED REALITY.
[6] RGB-D datasets using microsoft kinect or
similar sensors: a survey
Ziyun Cai, Jungong Han, 19 March 2016.
[7] Xiao Hu Liu A 3D Display System for Cloth
Online Virtual Fitting Room,2016.
[8] Andres Traumann A New Retexturing Method
for Virtual Fitting Room Using Kinect 2
Camera,Multimed Tools Appl DOI
10.1007/s11042-016-3374-6.

User Friendly Virtual Clothes System Based on Simulation and Visualization using RGBD Camera

More Related Content

What's hot (20)

Viewers also liked (19)

Similar to User Friendly Virtual Clothes System Based on Simulation and Visualization using RGBD Camera (20)

Recently uploaded (20)

User Friendly Virtual Clothes System Based on Simulation and Visualization using RGBD Camera