Testing and Verification through Virtual Product Models

Testing and Verification through Virtual Product
Models: A survey and look ahead
Andrea Serester∗, Gergely Hidas∗, Gábor Szögi∗, Péter Galambos∗†
∗Antal Bejczy Center for Intelligent Robotics, Óbuda University, Bécsi út 96/b, H-1034 Budapest, Hungary
†Institute for Computer Science and Control, Hungarian Academy of Sciences, Kende u. 13-17, H-1111 Budapest, Hungary
‡T ÜV Rheinland InterCert Ltd., Váci út 48/A-B, H-1132 Budapest, Hungary
E-mail: peter.galambos@irob.uni-obuda.hu
Abstract—Modern engineering design and simulation software
(CAD, CAM, CAE) allow for extensive use of virtual prototypes
along the product and production development reducing the cost
and time of physical prototyping. These software technologies in
turn has also important role in testing and verification against
the various regulations. This paper reviews the recent progress
of virtual verification (ViVer) technology and the underlying
emerging concepts with special focus on the role of certification
service providers. The paper covers multiple aspects of the ViVer
concept serving as a conceptual guideline for the development of
virtual verification systems of the future.
Index Terms—product certification, product testing, virtual
verification, virtual prototyping, computer aided engineering
(CAE), virtual reality
I. INTRODUCTION
Direct use of computers in product development dates back
to the 1960’s when the ancestor of all modern CAD system
called Sketcpad developed by MIT’s young researcher Ivan
Sutherland [1], [2]. Today, computers and extremely powerful
engineering software pervade every aspects of product and
production design covering the full product lifecycle. Compa-
nies like Siemens, Dassault Systéms and PTC among many
others deliver advanced PLM frameworks for all industrial
segments [3], [4]. Recent research activity in PLM focuses
on the improvement of concurrent engineering paradigm and
widening the applicable range of virtual prototyping [5], [6],
[7], [8], [9] and virtual product development.
Over the mere engineering aspects of product innovation,
due to the strict and complicated regulations on product
compliance, testing and verification processes has a significant
role in product lifecycle in terms of cost and time requirement
[10], [11], [12]. In practice, every new product undergoes
certification/verification, which requires the transportation of
a prototype (manufactured immediately after the completion
of the design process) to a suitably equipped verification lab-
oratory. Whenever the verification procedure fails, the product
is re-designed, its prototype is manufactured once again, and
re-submitted for a new round of verification.
The latest engineering software technology allows for
checking and optimizing products through virtual prototypes
with continuously improving certainty reducing the number
of physically built prototypes [13]. However, product certifi-
cation requires certain standard-defined test cases that need
the special knowledge of various regulations to define the
test scenarios in generic engineering tools. In practice, the
product development and product certification is usually sep-
arated, compliance is investigated by specialized certification
companies.
Test, inspection and verification tasks are relevant to a very
wide range of consumer products, hence, virtual verification
has the potential to influence a large sector of world economy.
Virtual design and testing is a key concept in cutting-edge
R&D, and companies are seeing a strong incentive to move in
this direction by e.g., integrating the design and verification
procedures into a coherent whole starting from the very begin-
ning of product development, and by combining existing tools
that support these procedures with virtualization infrastructure
[14]. More specifically, the sub-area of non-destructive testing
alone is estimated to generate about e 1.23 billion world-wide
(and e 370 million within Europe), which translates into a
growth rate of about 7.2% 1
.
This study surveys various aspects of this evolution towards
a visionary software infrastructure that integrates the testing,
verification and certification functions into PLM through spe-
cific means of Virtual Reality.
The paper is structured as follows: Section II recalls the his-
tory and the motivation for model-based testing. A visionary
methodology of Virual Verification is discussed in III, while
section IV gives a short outlook on the ViVer aspects of safety-
related verification of modern robotized production systems.
Section V identifies the special role of test and verification-
specific engineering knowledge in use of engineering simula-
tion software. Finally, section VI concludes the paper.
II. EARLY ADOPTERS: AEROSPACE AND AUTOMOTIVE
INDUSTRIES
In the past 10 years, advances in informatics have brought
about the incentive within the domain of cost-intensive indus-
trial production to minimize the costs associated with proto-
type development. An important part of these incentives is
to provide facilities for the internal, virtual quality assurance,
testing and verification of prototypes [14], [13], [6]. The first
segment in which these incentives were formulated were the
1Source: Frost & Sullivan 2014

aerospace and automotive industries [15], [16], [17], in which
companies developed a set of quality assurance procedures
which were to be observed and applied by all actors of
the product supply chain. However, the implementation of
all of these procedures significantly increased the costs of
individual suppliers, which reduced the incentives for further
co-operation. In parallel, ideas for virtualized verfication also
appeared in the testing and validation of close-to-finalized
product models (i.e., prototype vehicles), as a means to reduce
the costs associated with the maintenance of a physical labora-
tory environment, along with the tools necessary to physically
build prototypes. One of the more rudimentary steps in this
direction involved a set of quality control guidelines which
were to be met in advance before any kind of prototype
would be produced. However, the first generation of such
solutions still incurred considerable costs, and were therefore
not available to smaller suppliers within the sector. In this case,
smaller functional components (i.e., various car parts) were
instead sent to specialized companies such as T ÜV Rheinland
for product verification.
The standards and guidelines, as well as the standard
toolsets used for their implementation in complex industries
(such as the automotive industry) are under constant change
and improvement. The standards and guidelines used today
adopt a completely different approach from those used a
decade ago. Verification and quality assurance procedures have
been developing in a direction where the prior measurement
data and quality control data collected by the manufacturer are
used directly to obtain a highly accurate estimate of the quality
of the end product (this can be referred to as the family of pre-
calculation based methods). The many examples of application
domains in which virtual verification procedures are used on
a daily basis and have legally binding effects include e.g.,
automotive crash tests, switching gears (EN 61439), etc. Not
all standards allow the use of pre-calculation based methods,
but standardization practices are inevitably evolving in this
direction. As a result, manufacturers (as opposed to companies
specializing in validation) have growing responsibilities in
terms of quality assurance, and are therefore increasingly fo-
cusing on quality management and risk assessment procedures
that can be integrated into the manufacturing process from the
early stages of product design.
III. INTEGRATED ONLINE ECOSYSTEM FOR AGILE
DESIGN-VERIFICATION LOOP
The economic impact can be increased through an emerging
paradigm that combines virtual product design and testing
based on cloud-based online services [18], [19], which is
currently and dominantly based on offline engineering toolsets.
This integration, which will occur both at the level of product
development and at the level of verification/certification stan-
dards can enhance the entire chain of R&D&I extended with
verification (R&D&I&V) behind the entire PLM workflow.
This will in turn radically improve the capacity of various
branches of industry to flexibly develop new products, and at
the same time it will decrease the costs and time associated
with development and verification procedures, as well as
reduce environmental risks by helping to reduce material and
transportation costs in the design and prototyping phases.
This recently forming approach is often referred to as
Virtual Verification, thus this terminology is used along this
study. The rest of this section discusses the practical aspects
of Virtual Verification on a usecase level. Figure 1 briefly
illustrates the functions that should be covered in such a
service ecosystem supporting the agile product development.
Fig. 1. Conceptual overview of the Virtual Verification process illustrating the
relationship of the product developer, the certification company and the 3rd-
party entities providing various analytic tools built into a commercial cloud
ecosystem.
The essence of the new paradigm is to allow for the parallel
execution of tasks relevant to the design, manufacturing and
certification/verification of products - starting from the level
of formulation of conceptual ideas all the way to practical
realization - based on an Internet based 3D virtual collabo-
ration platform such as VirCA (Virtual Collaboration Arena)
[20], [21], [22]. The use of 3D virtual collaboration as an
underlying technological framework yields several benefits:
1) Leads to an increase in manufacturing throughput, while
reducing costs and environmental risks.
2) Enables the engineering design process to continuously
follow the results of verification stages, thus minimizing
risks of encountering unanticipated pitfalls which would
otherwise result in a waste of considerable time and
resources.
3) Eventually pave the way for the deployment of almost
entirely automated virtual verification procedures.
The appearance of the new integrated tight-loop software
services is timely and in line with current trends in engineering
and manufacturing [23], [24], [25]. Software capable of sim-
ulating various physical phenomena (e.g., stress, heat transfer,
acoustics, hydrodynamics, dynamics, durability etc. ) are in
widespread use in engineering design, and the design process
itself is primarily executed using 3D design systems. However,
in contrast to this high degree of automation in engineering
design, the technology behind verification procedures is not

nearly as technology-driven (it is mostly manually performed
on real prototypes). Hence, the goal of this is to introduce the
appropriate virtual toolset (based on the existing simulation
engines) into the context of product test and verification. The
integration of current intelligent systems with smart virtual
spaces will further allow for the verification process to be
managed in networked settings. In summary, the paradigm
behind this approach will allow for the enhancement of
verification procedures, thus offering a radically new aspect
to product management in a way that is in line with the best
engineering practices.
Fig. 2. Activity of different stakeholders in a collaborative verification session.
IV. VERIFICATION OF THE PRODUCTION PROCESS
Not only the product itself but the production process also
very important concerning safety issues. Standards e.g., BS
EN 292:1991, BS EN 294:1992, BS EN 1050:1997, BS EN
60204-1:2006+A1:2009 regulates the machinery safety and
occupational safety. Safety requirements for industrial robots
and robotic devices are covered by the BS EN ISO 10218-
1:2011 standard.
These matured regulations are well applicable for conven-
tional machinery and production lines, however, the newest
generation of collaborative industrial robots require radically
new concepts. Real-life scenarios considering different means
of collaboration between human and robots are under inves-
tigation for the purpose of standardization but there is no
published draft up till today. This topic has been touched by
the scientific community and several influential works have
been published (e.g., [26], [27], [28], [29], [30]).
Main goal of the standardization process is to introduce
a set of mandatory safety functions that prevent the injury
of the human co-working with collaborating industrial robots.
Validation of these safety functions renders new challenges
for the testing and certification companies familiar with the
former generation of safety standards. Realtime simulations
in merged physical and virtual environments give a promising
toolset along the conceptual framework of virtual verification.
The latest Virtual Reality (VR) and Augmented Reality (AR)
devices and recently spreading spatial recognition technologies
together allows for testing scenarios where some components
of the system (e.g., workpiece, robot, gripper) are represented
only virtually while other elements and the safety control logic
are real. This concept is discussed in [31] and [20] considering
virtual commissioning of production systems.
V. INTERDISCIPLINARY UNDERPINNINGS OF VIRTUAL
VERIFICATION
Model-based product testing in general requires several
types of resources in terms of digital description of the prod-
ucts and computational methods. Execution of a test case also
needs the precise parametrization of the concrete investigation.
Parametrization usually means the definition of initial and/or
boundary conditions with additional constraints to mathemati-
cally describe a test case. In conclusion, parametrization makes
the connection between the product model and the available
mathematical methods so that the input data and the simulation
process together lead to an interpretable result in terms of pass
or fail a compliance check.
It is also noteworthy that the parametrization of an engi-
neering simulation requires special skill in case of product
testing purpose: While a product developer sets up a simulation
in order to optimize a product concerning cost and quality
aspects, a test engineer interprets a large set of regulations
and formalize a synthetic simulation that helps to judge the
product’s compliance for given standards.
Applying the currently available simulation software, virtual
test synthesis are viable in a wide range of engineering fields.
Table I collects some relevant academic disciplines and the
corresponding investigations and software categories. These
fields are considered as main direction where the Virtual
Verification concept can spread in the upcoming years.
VI. CONCLUSION
Based on the previously analysed situation the following
challenges need to be overcome:
• Production line and/or production machines and verifica-
tion procedures must be adapted to and integrated into
the virtual verification process
• Potentially hostile and aggressive production environ-
ments must be adapted to, and consequences for cali-
bration, accuracy, aging, drift, etc. must be taken into
consideration
• In-line test, inspection and verification procedures have to
be developed to a level where they can be accepted as an
alternative to end-of-line test, inspection and verification
procedures for certification purposes
• Tests have to be formulated at the levels of product,
system and subsystem integration
The predictable global impact of virtual verification tech-
nologies can be summarized in the following points:
• The Virtual testing and verification service will decrease
the time-to-market duration. This only one effect has huge

TABLE I
A POSSIBLE CATEGORIZATION OF SCIENTIFIC DISCIPLINES CONCERNING THE APPLICABILITY WITHIN THE VIVER CONCEPT
Scientific/technical discipline Observable features / phenomena Related class of engineering software
Geometry
Length/distance
Curvature
Angle
Area/surface
Volume
CAD
CAM
Mechanics
Balance
Motion
Stress
Deformations
Vibrations
Acoustics
Multibody dynamics
Rigid body simulations
Finite Element Analysis (FEA)
CTR and SPPS-based acoustic software
Optics
Color
Luminescence
Contrast
Reflections
Refractions
Diffraction
Interference
Geometric optics / Ray optics simulators
Wave optics simulators
Electrotechnics
Voltage
Current
Circuit simulation software
Electromagnetism
Electromagnetic emission
Wave propagation
Simulators based on computational electromagnetics
Transport processes (in general sense)
Heat and mass transfer
Conduction and diffusion
Radiative heat transfer
Conductive transport
Finite Volume Analysis (FVA)
Chemistry
Chemical reactions
Solvation
Corrosion
Electrodeposition
Chemical process simulators
Human Factors
Ergonomic / Anthropometric measures
Psychologycal effects
Ergonomics databases
Ergonomics analysis software
and several further effects and one of the most important
development factors of the industry.
• The use of 3D and simulation software as well as the
development of these software will increase.
• The testing and verification based on digital models will
be introduced many additional areas of the standardis-
ation. The special procedures of the virtual testing and
verification should be developed by international stan-
dardization bodies together with industrial partners. This
will also enhance the distribution of the new technology.
• Legislation shall introduce new aspects: the concept
of purely virtually verified products together with the
aspects of product liability. This process has started
however the new technology on a new area will boost the
legislative change that will make further development in
the introduction of new technologies of the early stage of
product development.
Advanced products increase the complexity of test & in-
spection tasks, which extend to the validation of tools, the
calibration and control of manufacturing processes. The pro-
fessional test and verification laboratories has not yet made
Virtual Verification to be an everyday practice, but smart and
adaptive verification has already started to be integrated into
every phase of product design and manufacturing.
ACKNOWLEDGMENT
The research was supported by the Ministry of Human
Capacities of Hungary.
REFERENCES
[1] I. Sutherland, “Sketchpad-a graphical man-machine interface,” PhD
dissertationMIT, Cambridge, MA, 1963.
[2] D. T. Ross and J. E. Rodriguez, “Theoretical Foundations for the
Computer-aided Design System,” in Proceedings of the May 21-23,
1963, Spring Joint Computer Conference, ser. AFIPS ’63 (Spring). New
York, NY, USA: ACM, 1963, pp. 305–322.
[3] F. Ameri and D. Dutta, “Product Lifecycle Management: Closing the
Knowledge Loops,” Computer-Aided Design and Applications, vol. 2,
no. 5, pp. 577–590, Jan. 2005.
[4] J. Stark, “Product Lifecycle Management,” in Product Lifecycle Man-
agement, ser. Decision Engineering. Springer London, 2011, pp. 1–16.
[5] B. Liu, R. I. Campbell, and E. Pei, “Real-time integration of prototypes
in the product development process,” Assembly Automation, vol. 33,
no. 1, pp. 22–28, Feb. 2013.
[6] M. Bordegoni and F. Ferrise, “Designing interaction with consumer
products in a multisensory virtual reality environment,” Virtual and
Physical Prototyping, vol. 8, no. 1, pp. 51–64, 2013.
[7] B. H. Li, X. Chai, X. Yan, and B. Hou, Eds., Multi-Disciplinary Virtual
Prototype Modeling and Simulation Theory and Application. Nova
Science Pub Inc, Nov. 2012.
[8] K. Tahera, C. Earl, and E. Claudia, “Integrating virtual and physical
testing to accelerate the engineering product development process,” In-
ternational Journal of Information Technology and Management, vol. 13,
no. 2-3, pp. 154–175, Jan. 2014.
[9] J. Huang, S. Ong, and A. Nee, “Real-time finite element structural
analysis in augmented reality,” Advances in Engineering Software,
vol. 87, no. 0, pp. 43 – 56, 2015.

[10] P. Maropoulos and D. Ceglarek, “Design verification and validation in
product lifecycle,” {CIRP} Annals - Manufacturing Technology, vol. 59,
no. 2, pp. 740 – 759, 2010.
[11] K. Tahera, C. Earl, and C. Eckert, “The role of testing in the engineering
product development process,” Karlsruhe, Germany, May 2012.
[12] K. Tahera, C. Eckert, and C. Earl, “Highlighting the importance of
testing in the product development process,” in 20th International
Conference on Engineering Design, ICED15., July 2015.
[13] F. Ferrise, M. Bordegoni, and U. Cugini, “Interactive Virtual Prototypes
for Testing the Interaction with new Products,” Computer-Aided Design
and Applications, vol. 10, no. 3, pp. 515–525, 2013.
[14] D. Talaba and A. Amditis, Eds., Product Engineering: Tools and
Methods Based on Virtual Reality, softcover reprint of hardcover 1st
ed. 2008 ed. Springer, Dec. 2010.
[15] F. Mas, J. L. Menndez, M. Oliva, A. Gmez, and J. Ros, “Collaborative
Engineering Paradigm Applied to the Aerospace Industry,” in Product
Lifecycle Management for Society, ser. IFIP Advances in Information
and Communication Technology, A. Bernard, L. Rivest, and D. Dutta,
Eds. Springer Berlin Heidelberg, Jan. 2013, no. 409, pp. 675–684.
[16] F. Mas, J. L. Menndez, M. Oliva, and J. Ros, “Collaborative Engineering:
An Airbus Case Study,” Procedia Engineering, vol. 63, pp. 336–345,
2013.
[17] J. L. Menndez, F. Mas, J. Servn, and J. Ros, “Virtual verification
of the AIRBUS A400m final assembly line industrialization,” in AIP
Conference Proceedings, vol. 1431. AIP Publishing, Apr. 2012, pp.
641–648.
[18] I. Ari and N. Muhtaroglu, “Design and implementation of a cloud
computing service for finite element analysis,” Advances in Engineering
Software, vol. 6061, pp. 122–135, Jun. 2013.
[19] S. Ma and L. Tian, “A web service-based multi-disciplinary collabo-
rative simulation platform for complicated product development,” The
International Journal of Advanced Manufacturing Technology, vol. 73,
no. 5-8, pp. 1033–1047, May 2014.
[20] P. Galambos, d. Csap, P. Zentay, I. M. Flp, T. Haidegger, P. Baranyi, and
I. J. Rudas, “Design, programming and orchestration of heterogeneous
manufacturing systems through VR-powered remote collaboration,”
Robotics and Computer-Integrated Manufacturing, vol. 33, pp. 68–77,
Jun. 2015.
[21] P. Galambos, C. Weidig, Pter Zentay, d. Csap, P. Baranyi, J. C. Aurich,
B. Hammann, and O. Kreylos, “Future Internet-based Collaboration in
Factory Planning,” Acta Polytechnica Hungarica, vol. 11, no. 7, 2014.
[22] P. Galambos, C. Weidig, P. Baranyi, J. C. Aurich, B. Hamann, and
O. Kreylos, “VirCA NET: A case study for collaboration in shared
virtual space,” in 2012 IEEE 3rd International Conference on Cognitive
Infocommunications (CogInfoCom), Dec. 2012, pp. 273 –277.
[23] S. Choi, K. Jung, and S. Do Noh, “Virtual reality applications in
manufacturing industries: Past research, present findings, and future
directions,” Concurrent Engineering, vol. 23, no. 1, pp. 40–63, 2015.
[24] D. Chen, “A methodology for developing service in virtual manufactur-
ing environment,” Annual Reviews in Control, vol. 39, no. 0, pp. 102 –
117, 2015.
[25] W. Zhu, X. Peng, and M. C. Leu, Engineering Applications of Virtual
Reality. John Wiley & Sons, Inc., 2014.
[26] A. Bicchi, M. A. Peshkin, and J. E. Colgate, “Safety for Physical
HumanRobot Interaction,” in Springer Handbook of Robotics, B. S. Prof
and O. K. Prof, Eds. Springer Berlin Heidelberg, 2008, pp. 1335–1348.
[27] B. S. Dhillon, Robot Reliability and Safety. Springer Science &
Business Media, Dec. 2012.
[28] J. Fryman and B. Matthias, “Safety of Industrial Robots: From Con-
ventional to Collaborative Applications,” in Robotics; Proceedings of
ROBOTIK 2012; 7th German Conference on, May 2012, pp. 1–5.
[29] M. Vasic and A. Billard, “Safety issues in human-robot interactions,”
in 2013 IEEE International Conference on Robotics and Automation
(ICRA), May 2013, pp. 197–204.
[30] S. Haddadin, “Physical Safety in Robotics,” in Formal Modeling and
Verification of Cyber-Physical Systems, R. Drechsler and U. Khne, Eds.
Springer Fachmedien Wiesbaden, 2015, pp. 249–271.
[31] P. Galambos, P. Baranyi, and I. Rudas, “Merged physical and virtual
reality in collaborative virtual workspaces: The virca approach,” in
Industrial Electronics Society, IECON 2014 - 40th Annual Conference
of the IEEE, Oct 2014, pp. 2585–2590.

Testing and Verification through Virtual Product Models

More Related Content

What's hot (20)

Similar to Testing and Verification through Virtual Product Models (20)

Recently uploaded (20)

Testing and Verification through Virtual Product Models