Data Quality meets SOA

WHITE PAPER: SOA

WHITE PAPER /
Data Quality meets SOA –
Making Data Quality available
for all Business Processes

Data quality functions were already being
provided as services for Unix, Windows
and Linux, before the analysts of Gartner
had invented the term SOA. For the most
part, technical reasons were the decisive
factor for this architecture. In addition to
this, the implementation of service-orien-
ted architectures results in new and chan-
ged requirements for data quality servi-
ces and also increases the opportunities
and benefits which they can create.

__________________________

All company and product names and logos used in this document are trade names

and/or registered trademarks of the respective companies.

Page 1

WHITE PAPER: SOA

The starting point
for data quality services
In order to consider the importance of service-oriented architectures for the pro-
vision and use of data quality functions, it is first of all useful to look at the typi-
cal data quality functions themselves.

APPLICATION A: A classic application is the validation of a postal address on the basis of reference data,
which includes street and place names and the dependencies of the postcodes on places, streets and
house numbers. In contrast to simple database access, the correct address should also be found here if
the input is incomplete or contains recording or hearing errors. The goal is a high matching accuracy,
in order to be able to correct the greatest possible number of incorrect addresses automatically.

APPLICATION B: Another application is searching for duplicates in the in-house database. Here too the
goal is to quickly and reliably identify a business object, a business partner, a product or a sales oppor-
tunity in spite of incomplete, divergent or incorrect input, in order to (a) simplify the search and thereby
increase the productivity of the users, and (b) to prevent the creation of duplicates, i.e. multiple entries
which refer to the same object in the real world. Consistent, complete and unambiguous mapping of the
real objects in the database will be achieved as a result.

An important goal in the context of improving Apart from the response time, the integration in a
the data quality consists in preventing incomplete wide range of environments already played an
or incorrect data from being stored in the data- important role in data quality services. Decoupling
base. Possible problems should be detected at data the data quality services from the service consum-
entry and then cleaned up either automatically or ers and utilization via a client/server protocol is
manually by the user after appropriate feedback. important for reaching this goal. This function had
Specialized search indices ensure that a search therefore already been provided and used for data
in databases with 1,000,000 to 100,000,000 quality, at least for more sophisticated applications,
data records normally only requires a fraction of a before the „invention“ of service-oriented architec-
second, even with divergent spelling. Nevertheless, tures as a service. As a result, both the high require-
these response times require intelligent caching of ments for the response times could be met and the
the indices. This can be provided very efficiently provision of functions guaranteed for a wide range
by implementing the software as a central service of environments.
which is made available from an in-house server.

© UNISERV GmbH / +49 7231 936-0 / All rights reserved. Page 2

WHITE PAPER: SOA

From the „fat client“
to the 3-layer architecture
Layered Architecture LAYERED ARCHITECTURE
The typical architecture in the early days of the cli- FAT CLIENT
ent/server world consisted of a database which,
in addition to the storage of transaction and mas- ROLE

ter data, enabled asynchronous communication Name Customer

between different system components, thereby Street Supplier

allowing these components to be decoupled. Postcode Reseller

Messages were written in the database by the
sender and read there by the receiver. For this pur-
pose, however, regular «polling» of the respective
table was required, in order to establish whether
new unprocessed messages were available. The
business logic was mainly implemented in so-
called «fat» clients. Tasks which were executed
via batch processes were implemented via back-
ground processes which accessed the database.

Interactive functions for the validation of addresses
or for the detection of duplicates directly at data
entry were normally integrated in the graphical user
interface. They were usually called via proprietary
interfaces. Specifications such as DCE1 or CORBA2,
whose goal was the standardization of interfaces
for the communication of distributed components,
had nothing more than a niche existence.

1 http://en.wikipedia.org/wiki/Distributed_Computing_Environment

2 http://en.wikipedia.org/wiki/CORBA


LAYERED ARCHITECTURE WHITE PAPER: SOA

FAT CLIENT CLIENT
This situation has fundamentally changed in the past
few years. The starting point for this was mainly the
ROLE ROLE
Name Customer Name Customer
establishment of standards within the framework of
JEE3 (Java Enterprise Edition) which resulted in the
Street Supplier Street Supplier

provision of high-performance implementations of
Postcode Reseller Postcode Reseller

these standards both as commercial products and
open source solutions.
APPLICATION SERVER
SIMPLE INTEGRATION IN THE
APPLICATION SERVER – EITHER WITH A
JEE OR A .NET ARCHITECTURE – NOW
CAME TO THE FORE.

For the Windows world, Microsoft followed with the
development of .NET4 as a language-independent
platform and the .NET Enterprise Services. As a
result, a high-performance infrastructure software
was available irrespective of the selected platform
(JEE, .NET), in order to largely detach the business
logic from the presentation layer and implement it in
its own layer on the server. This also changed the
requirements for data quality services, which were
now executed mainly from the business logic on the
server side.

3 http://en.wikipedia.org/wiki/Java_Platform,_Enterprise_Edition

4 http://en.wikipedia.org/wiki/.NET_Framework


WHITE PAPER: SOA

SOAP as an
Enabler for SOA
SOA does not become really effective until a stand- Proprietary Protocol
ard protocol for the provision and use of services
has been established. This gap was closed by the
Web Service protocol SOAP5. It is supported by
practically all middleware and infrastructure com-
ponents, thereby enabling interoperability between
service providers, middleware and service con-
sumers. As a result, the provision of connectors
for the use of proprietary protocols in proprietary
middleware is no longer necessary. This therefore
lays the basis for the establishment of powerful mid-
dleware components. It includes the concept of the
Enterprise Service Bus6 (ESB), which enables the
loose coupling of different components which play
a significant role in the routing of messages, as
well as engines which can directly execute defined
workflows (BPEL)7 in a business process language.
SOAP-based Web Services have therefore devel-
oped into a central instrument for the provision of
interactive data quality services in modern enter-
prise architectures. These mainly concern services
which run according to the request/response pat-
tern, in which the service consumer initiates a
request, e.g. «validate the specified address», and
the service makes a direct response with a confir-
mation, a correction suggestion or a selection of
possible correct addresses.

5 http://en.wikipedia.org/wiki/SOAP

6 http://en.wikipedia.org/wiki/Enterprise_service_bus

7 http://en.wikipedia.org/wiki/BPEL


WHITE PAPER: SOA

This procedure corresponds to the interactive char- SOAP Protocol
acter of the validation, which should support the
user directly at entry of a business object and offer
options for intervention in the event of problems.
However, this function is no longer implemented
in isolation in the presentation layer but normally
takes place in the context of a higher-ranking busi-

THE CORRELATION BETWEEN THE
IMPLEMENTATION OF BUSINESS
PROCESSES AND DATA QUALITY
FUNCTIONS IMMEDIATELY BECOMES
OBVIOUS AND THEREFORE ALSO THE
CONTRIBUTION WHICH THEY
MAKE TO THE SUCCESS OF THE
RESPECTIVE BUSINESS PROCESS.

ness process, e.g. the implementation of the order-
ing process in an e-business application, the imple-
mentation of a process for lead conversion and
qualification in a CRM application or a compa-
rable process. The correlation between the imple-
mentation of business processes and data quality
functions immediately becomes obvious and there-
fore also the contribution which they make to the
success of the respective business process.


WHITE PAPER: SOA

Customer Cases
Orange
The customers of the telecommunications company This service-oriented approach makes it possible
Orange in France can contact the company via to use the same services in different processes and
various channels. They can visit different channels. Irrespective of whether it con-
the web portal of Orange in cerns the creation of a new Orange customer or
the Internet, call the call center the change of address of an existing customer and
of Orange and visit the mobile irrespective of which contact channel a process is
phone business of a partner of initiated, the underlying service-oriented architecture
Orange. However, irrespective always ensures that the executed processes are
of which contact channel the customer chooses, it configured consistently and can access the same
must be ensured that the respective processes are services. It is thereby possible to guarantee a con-
executed with the same quality. An important com- sistently high quality standard of the address data
ponent of this process quality consists in ensuring across the company.
that customer addresses are correct.
The technical basis is the open source Java appli-
cation server JONAS. This JEE server is the central
point for the provision of services which are required
to implement the business processes of Orange.
The Uniserv data quality service for the valida-
tion, restructuring and standardization of customer
addresses is also provided in this environment.


WHITE PAPER: SOA

Customer Cases
WinGroup AG
The German WinGroup AG, a service network
for sales and marketing, offers its customers an
extensive range of services in
the areas of call center, letter-
shop, dialog marketing and IT
services. In order to guarantee
a consistently high quality level of the underly-
ing processes in all service areas and customer-
specific applications, the subsidiary company,
WinLogic, has developed a service-oriented archi-
tecture based on an Enterprise Service Bus. This
represents the central middleware for docking all
applications in the company in the central services.
The address validation and the duplicate check of
Uniserv are linked here to secure the data quality.


WHITE PAPER: SOA

Lightweight REST –
when SOAP is too unwieldy
Even if the SOAP-based protocol for integration The call is via the http protocol, and the call argu-
in typical enterprise middleware seems to be the ments in the URL are encoded as a result. In the
ideal solution, there are various applications where case of a call from JavaScript, the result is output
alternatives such as RESTful8 Services are advan- in the JSONformat9 in the ideal case. This results
tageous. This is particularly the case when data in a JavaScript code which can be directly inter-
quality services are to be directly activated in a
presentation layer which is HTML/AJAX-based. SERVICES CONFIGURED IN THIS
Input aids which automatically complete a partial MANNER CAN BE IDEALLY USED IN
input by the user or offer suggestions for comple- TYPICAL WEB 2.0 APPLICATIONS.
tion of the input based on the partial input are
a typical scenario for this case. Input aids are
located in the presentation layer by their nature. preted by the JavaScript interpreter of the browser,
However, they make considerably higher demands in order to provide the result of the call directly
for the response time, since they are called more as JavaScript objects. Services configured in this
frequently during the input, usually after each input manner can be ideally used in typical Web 2.0
character. applications.

Although the same services used for address valida-
tion present themselves for the address input in this
implementation, SOAP-based Web Services are
not always suitable for this application scenario on
account of the overhead which the SOAP protocol
entails. RESTful Web Services are extremely lean
in comparison to SOAP-based Web Services.

8 http://en.wikipedia.org/wiki/REST

9 http://en.wikipedia.org/wiki/JSON


WHITE PAPER: SOA

SOAP -
the fine differences
A learning curve also has to be overcome for However, this criterion is essential in a data qual-
adapting proprietary interfaces to a SOAP-based ity service which must useable in a great variety
communication. The packets exchanged in the of environments which may not even be known
framework of a SOAP-based communication are beforehand.
described in a metaformat, the so-called WSDL In addition to this, the SOAP specification offers
(Web Service Description Language).10 two basic options for defining in the WSDL the
linkage between the packet structure in XML and
DURING THE DEVELOPMENT OF the constructs of the programming language which
THE WSDL, IT MUST FIRST OF ALL BE provides or interprets the packet. As the acronym
ENSURED WITHOUT FAIL THAT THE DATA suggests, the so-called «rpc-style» corresponds more
TYPES USED ARE ACTUALLY SUPPORTED to the conventional Remote Procedure Call (RPC)
BY ALL THE TARGET LANGUAGES AND and models operations as method calls which do
SYSTEMS. not differ from local calls. They are ideal when
the Web Service is to be called from an object-
oriented programming language such as Java or
During the development of the WSDL, it must first C#. The so-called «document-style» is more suitable
of all be ensured without fail that the data types for modelling complex contents as a result which
used are actually supported by all the target lan- is represented as an XML document with its own
guages and systems, in which the service is to be XML scheme. Validation against the XML scheme
consumed. This aspect is relatively non-critical in a using standard XML means is therefore possible on
pure in-house development with a homogeneous the one hand, and the result document can be eas-
software infrastructure, e.g. JEE or .NET. ily further processed, transformed or processed for
the presentation layer using standard XML means
or suitable frameworks on the other. Both variants
have their advantages and disadvantages.

10 http://en.wikipedia.org/wiki/Web_Services_Description_Language


WHITE PAPER: SOA

Both variants may also have to be made avail- Web Services must be scalable. A prerequisite
able to services which need to be called from any for this is the described statelessness. In addition
context. to this, the Web Service should access global
resources as little as possible or not at all. If this
is necessary, however, the administration and
WEB SERVICES ARE STATELESS BY THEIR
synchronization of the access to these resources
NATURE. IN THE IDEAL CASE, THIS
should never be implemented ad-hoc for the
MEANS NO PARTIAL RESULTS; INSTEAD
respective WEB service. Resource pools such as
THE OVERALL RESULT IS OUTPUT AS A
are offered by most application servers or exist as
RESULT OF THE CALL, AND TWO SUCCES-
open source extensions should be used instead.
SIVE CALLS ARE TOTALLY INDEPENDENT
Effective and configurable pooling is thereby
OF EACH OTHER.
enabled.
The last two points in particular normally require at
Web Services are stateless by their nature. In the least a partial redesign if the existing functionality
ideal case, this means no partial results; instead is to be made available as a Web Service.
the overall result is output as a result of the call,
and two successive calls are totally independent
of each other.


WHITE PAPER: SOA

SOA blurs the distinction between
«on premise» and «on demand»
The provision of software services and applica- The reference data required for the service must
tions via the Internet, referred to by the buzzwords be regularly updated. This means both regularly
«Software on Demand», «Software as a Service»11 recurring manual work as well as regular subscrip-
or «Cloud Computing», is a theme which is grow- tion charges which have to be paid to the data
ing in importance. In many cases, a fundamen- provider. In the case of country-specific postcode
tal and profound contradiction between locally directories, this work and expense are incurred for
installed software («on premise») and software each country for which addresses are checked.
used via the Internet («on demand») is depicted.
This is not the case from the SOA perspective, AS A RESULT, AN OPTIMUM SOLUTION,
because within the framework of a service-oriented WHICH CAN ALSO BE FLEXIBLY ADAPTED
architecture it is not critical how the respective IN RETROSPECT TO CHANGED BASIC
service is provided. The provision of a service via CONDITIONS, CAN BE FOUND FOR THE
the Internet as an alternative to a locally installed RESPECTIVE USER.
server presents interesting new application possi-
bilities, particularly in the area of data quality serv- This only makes the use of such solutions interesting
ices which carry out validations and corrections by for larger quantities of data. This restriction is not
matching and merging against reference data. applicable if the service is provided as an SaaS
offer, in which invoicing is based exclusively on
the executed transactions. Locally installed services
and services used via the Internet can be also com-
bined as required or exchanged by using them in
a service-oriented architecture. As a result, an opti-
mum solution, which can also be flexibly adapted
in retrospect to changed basic conditions, can be
found for the respective user.

11 http://en.wikipedia.org/wiki/Software_as_a_Service


WHITE PAPER: SOA

Conclusions
The following conclusions about two aspects can be drawn from the above
described experience:

ASPECT ONE: What are the requirements for data
quality functions which are to be used in an SOA
environment?

– The functions must be accessible via SOAP as – It should be checked whether an alternative use
Services. Integration in practically any infra- scenario, in which the service is provided via
structure for the service-oriented implementation the Internet, provides commercial or technical
of the business processes is thereby guaran- advantages, and whether the service provider
teed. However, care should be taken in the supports such a scenario.
detail that mapping between the XML elements
of the service description and the respective
constructs of the respective server environment
can be represented.

– If use in the presentation layer is foreseeable,
it must be checked whether a RESTful Service
implementation is necessary.


WHITE PAPER: SOA

ASPECT TWO: Which aspects have to be considered
if functions for the validation, enhancement and
processing of data are to be made SOA-capable?

– The use scenarios must be clearly defined. The – The scalability of the service must be provided:
question as to whether the application of the if the Web Service requires global resources,
services takes place within the framework of e.g. a database connection, these should be
the business logic or the presentation logic is administered by means of a suitable resource
particularly important. In the first case, integra- pool in the server container. Otherwise, these
tion takes place in the application server, the resources quickly become a bottleneck which
Enterprise Services Bus or a BPEL engine, in prevents genuine scalability of the service.
the second case in a graphical user interface,
which can in turn make its own demands. The – The service should be internet-capable, i.e. it
decision as to whether SOAP-based and/or should be irrelevant for the functionality of the
RESTful services are more appropriate is made service whether it is provided in the local net-
depending on this. work or via the Internet. The possible applica-
tions are extended enormously as a result.
– The target systems and languages in which the
service is to be used must be specified. The
decision on the degree of complexity of the
modelling with respect to the XML structures For further information
used and XML data types of the call results is please visit our web page
made depending on this. www.uniserv.com or contact us directly:

– The service must be designed, so that it is state-
less, i.e. it must function without the storage of
an internal state between two calls. If a state
between the calls is required, it must be trans- We are looking forward for advising and sup-
ferred for the follow-up call or made persistent porting you through your project.
in a suitable manner.


WHITE PAPER: SOA

Uniserv
Uniserv is the largest specialised supplier of data quality solutions in Europe
with an internationally usable software portfolio and services for the quality as-
surance of data in business intelligence, CRM applications, data warehousing,
eBusiness and direct and database marketing.

With several thousand installations worldwide, Uniserv
supports hundreds of customers in their endeavours to
map the Single View of Customer in their customer data- DATA
base. Uniserv employs more than 110 people at its head- MIGRATION
PROJECTS
quarters in Pforzheim and its subsidiary in Paris, France, E-COMMERCE
and serves a large number of prestigious customers in all
sectors of industry and commerce, such as ADAC, Al- ERP

lianz, BMW, Commerzbank, DBV Winterthur, Deutsche
Bank, Deutsche Börse Group, France Telecom, Green-
COMPLIANCE
peace, GEZ, Heineken, Johnson & Johnson, Nestlé,
CRM
Payback, PSA Peugeot Citroën as well as Time Life and
Union Investment.

Further information is available SOA

at www.uniserv.com DIRECT
MARKETING MDM/CDI

ON-PREMISE/
ON-DEMAND

BI/BDW

CPM

Experience: Market position: Employees:
OVER 40 YEARS LARGEST MORE THAN 110 PEOPLE
EUROPEAN SUPPLIER

UNISERV GmbH Contact:
Rastatter Straße 13 • 75179 Pforzheim • Germany • T +49 7231 936-0 • +49 7231 936-0
F +49 7231 936-3002 • E info@uniserv.com • www.uniserv.com
© Copyright Uniserv • Pforzheim/Germany • All rights reserved.


Data Quality meets SOA

More Related Content

What's hot (20)

Similar to Data Quality meets SOA (20)

More from Uniserv (16)

Recently uploaded (20)

Data Quality meets SOA