TUW-ASE Summer 2015: Data as a Service - Models and Data Concerns

Data as a Service – Models and Data
Concerns
Hong-Linh Truong
Distributed Systems Group,
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://dsg.tuwien.ac.at/staff/truong
1ASE Summer 2015
Advanced Services Engineering,
Summer 2015 – Lectures 4 & 5
Advanced Services Engineering,
Summer 2015 – Lectures 4 & 5

Outline
 Data provisioning and data service units
 Data-as-a-Service concepts
 Data concerns for DaaS
 Evaluating data concerns
ASE Summer 2015 2

Data versus data assets
ASE Summer 2015
3
Data
Data
Assets
Data
management
and
provisioning
Data concerns
Data
collection,
assessment
and
enrichment

Data provisioning activities and
issues
ASE Summer 2015 4
Collect
• Data sources
• Ownership
• License
• Quality
assessment
and
enrichment
Store
• Query and
backup
capabilities
• Local versus
cloud,
distributed
versus
centralized
storage
Access
• Interface
• Public versus
private
access
• Access
granularity
• Pricing and
licensing
model
Utilize
• Alone or in
combination
with other
data sources
• Redistribution
• Updates
Non-exhausive list! Add your own issues!
Provisioning Models

Stakeholders in data provisioning
ASE Summer 2015 5
Data
Data Provider
• People
(individual/crowds/org
anization)
• Software, Things
Data Provider
• People
(individual/crowds/org
anization)
• Software, Things
Service Provider
• Software and people
Service Provider
• Software and people
Data Consumer
• People, Software,
Things
Data Consumer
• People, Software,
Things
Data Aggregator/Integrator
• Software
• People + software
Data Aggregator/Integrator
• Software
• People + software
Data Assessment
• Software and
people
Data Assessment
• Software and
people
Stakeholder classes can be further divided!
Domain-specific versus domain-independent functions

Data service unit
ASE Summer 2015 6
Service
model
Unit
Concept
Data
service
unit
Data
 Can be used for private
or public
 Can be elastic or not
What about the
granularity of
the unit?
What about the
granularity of
the unit?
„basic
component“/“basic
function“ modeling
and description
Consumption,
ownership,
provisioning, price, etc.

Data service units in clouds/internet
 Provide data capabilities rather than provide
computation or software capabilities
 Providing data in clouds/internet is an increasing
trend
 In both business and e-science environments
 Now often in a combination of data + analytics
atop the data  to provide data assets
7ASE Summer 2015

Data service unitData service unit
8
Data service units in
clouds/internet
datadata
Internet/CloudInternet/Cloud
People
data
Things
ASE Summer 2015
data data

Data as a Service -- characteristics
 On-demand self-service
 Capabilities to provision data at different granularities
 Resource pooling
 Multiple types of data, big, static or near-realtime,raw data and
high-level information
 Broad network access
 Can be access from anywhere
 Rapid elasticity
 Easy to add/remove data sources
 Measured service
 Measuring, monitoring and publishing data concerns and usage
ASE Summer 2015 9
Let us use NIST‘s definition

Data-as-a-Service – service modelsData-as-a-Service – service models
Data as a Service – service models
and deployment models
ASE Summer 2015 10
Storage-as-a-Service
(Basic storage functions)
Storage-as-a-Service
(Basic storage functions)
Database-as-a-Service
(Structured/non-structured
querying systems)
Database-as-a-Service
(Structured/non-structured
querying systems)
Data publish/subcription
middleware as a service
Data publish/subcription
middleware as a service
Sensor-as-a-ServiceSensor-as-a-Service
Private/Public/Hybrid/Community CloudsPrivate/Public/Hybrid/Community Clouds
deploy

Examples of DaaS
ASE Summer 2015 11
Xively Cloud Services™
https://xively.com/
Xively Cloud Services™
https://xively.com/

DaaS design & implementation –
APIs
 Read-only DaaS versus CRUD DaaS APIs
 Service APIs versus Data APIs
 They are not the same wrt data/service
concerns
 SOAP versus REST
 Streaming data API
ASE Summer 2015 12

service provider vs data provider
 The DaaS provider is separated from the data
provider
13
DaaS
Consumer
DaaS
Sensor
DaaS
Consumer DaaS provider Data
provider
ASE Summer 2015

Example: DaaS provider =! data
provider
14ASE Summer 2015

structures
 DaaS and data providers have the right to
publish the data
ASE Summer 2015 15
DaaS
• Service
APIs
• Data APIs
for the
whole
resource
Data
Resource
• Data APIs
for
particular
resources
• Data APIs
for data
items
Data Items
• Data APIs
for data
items
Three levels

16
structures (2)
Data
items
Data
items
Data
items
Data resourceData resource
Data
assets
Data resourceData resource Data resourceData resource
Data resourceData resourceData resourceData resource
Consumer
Consumer
DaaS
ASE Summer 2015

patterns for „turning data to DaaS“ (1)
ASE Summer 2015 17
DaaSDaaSdatadata Build Data
Service
APIs
Deploy
Data
Service
Examples: using WSO2 data service

Storage/Database
-as-a-Service
Storage/Database
-as-a-Service
ASE Summer 2015 18
datadata
Examples: using
Amazon S3
DaaSDaaS

Storage/Databa
se/Middleware
Storage/Databa
se/Middleware
ASE Summer 2015 19
datadata
Examples:
using Crowd-
sourcing with
Pachube (the
predecessor of
Xively)
Things
One Thing  10000... Things
DaaSDaaS

Storage/Database/
Middleware
Storage/Database/
Middleware
ASE Summer 2015 20
datadata
Examples: using Twitter
People
DaaSDaaS

........
not just „functional“ aspects (1)
ASE Summer 2015 21
datadata DaaSDaaS.... data assetsdata assets
Data
concerns
Quality of
data
Ownership
Price
License ....
Enrichment
Cleansing
Profiling
Integration ...
Data Assessment
/Improvement
APIs, Querying, Data Management, etc.

not just „functional“ aspects (2)
ASE Summer 2015 22
Understand the DaaS ecosystem
Specifying, Evaluating and Provisioning Data
concerns and Data Contract

Example -
http://www.strikeiron.com/
ASE Summer 2015 23

DATA CONCERNS
ASE Summer 2015 24

........
What are data concerns?
Located
in US?
free?
price?
redistribution?
Service
quality?
25ASE Summer 2015
Quality of data? Privacy
problem?

........
DaaS concerns
ASE Summer 2015 26
Data
concerns
Quality of
data
Ownership
Price
License ....
DaaS concerns include QoS, quality of data (QoD),
service licensing, data licensing, data governance, etc.
DaaS concerns include QoS, quality of data (QoD),
service licensing, data licensing, data governance, etc.

Why DaaS/data concerns are
important?
 Too much data returned to the
consumer/integrator are not good
 Results are returned without a clear usage and
ownership causing data compliance problems
 Consumers want to deal with dynamic changes
27
Ultimate goal: to provide relevant data with
acceptable constraints on data concerns in
different provisioning models
Ultimate goal: to provide relevant data with
acceptable constraints on data concerns in
different provisioning models
ASE Summer 2015

DaaS concerns analysis and
specification
 Which concerns are important in which
situations?
 How to specify concerns?
28ASE Summer 2015
Hong Linh Truong, Schahram Dustdar On analyzing and specifying concerns for data as a service. APSCC 2009: 87-
94
Hong Linh Truong, Schahram Dustdar On analyzing and specifying concerns for data as a service. APSCC 2009: 87-
94

The importance of concerns in
DaaS consumer‘s view – data
governance
ASE Summer 2015 29
Important factor, for example, the security and
privacy compliance, data distribution, and auditing
Storage/Database
-as-a-Service
Storage/Database
-as-a-Service
datadata DaaSDaaS
Data governance

The importance of concerns in DaaS
consumer‘s view – quality of data
Read-only DaaS
 Important factor for the
selection of DaaS.
 For example, the
accurary and
compleness of the data,
whether the data is up-to-
date
CRUD DaaS
 Expected some support
to control the quality of
the data in case the data
is offered to other
consumers
30 30ASE Summer 2015

DaaS consumer‘s view– data and
service usage
Read-only DaaS
 Important factor, in
particular, price, data
and service APIs
licensing, law
enforcement, and
Intellectual Property
rights
CRUD DaaS
 Important factor, in
paricular, price, service
APIs licensing, and law
enforcement
ASE Summer 2015 31

DaaS consumer‘s view – quality of
service
Read-only DaaS
Important factor, in
particular availability and
response time
CRUD Daas
Important factor, in
particular, availability,
response time,
dependability, and security
ASE Summer 2015 32

The importance of concerns in DaaS
consumer‘s view– service context
Read-only DaaS
Useful factor, such as
classification and service
type (REST, SOAP),
location
CRUD DaaS
Important factor, e.g.
location (for regulation
compliance) and versioning
ASE Summer 2015 33

Conceptual model for DaaS
concerns and contracts
34ASE Summer 2015

35
Implementation (1)
Check http://www.infosys.tuwien.ac.at/prototyp/SOD1/dataconcernsCheck http://www.infosys.tuwien.ac.at/prototyp/SOD1/dataconcerns
ASE Summer 2015

36
Implementation (2)
 Data privacy concerns are annotated with WSDL
and MicroWSMO
ASE Summer 2015

37
Implementation (3)
 Joint work with
Michael Mrissa, Salah-Eddine Tbahriti, Hong Linh
Truong: Privacy Model and Annotation for
DaaS. ECOWS 2010: 3-10
Michael Mrissa, Salah-Eddine Tbahriti, Hong Linh
Truong: Privacy Model and Annotation for
DaaS. ECOWS 2010: 3-10
ASE Summer 2015

38
Populating DaaS concerns
DaaS
Concerns
evaluate, specify,
publish and manage
specify, select,
monitor, evaluate
monitor and
evaluate
The role of stakeholders in the most trivial view
Data
Aggregator/Integrator
Data
Consumer
Data
Assessment
Service Provider
Data Provider
ASE Summer 2015

Data concerns in multi-dimensional
elasticity
Simple
dependency
flows (increase nr. of services)
(increase) (increase response time)
(increase cost)
How do we maintain
our systems to deal
with such complex
dependencies?
How do we maintain
our systems to deal
with such complex
dependencies?
ASE Summer 2015 39

HOW TO EVALUATE DATA
CONCENRS FOR DATA
ASSETS IN DAAS?
ASE Summer 2015 40

Patterns for „turning data to DaaS“
ASE Summer 2015 41
Storage/Database
-as-a-Service
Storage/Database
-as-a-Service
datadata DaaSDaaS
Storage/Databa
se/Middleware
Storage/Databa
se/Middleware
datadata
Things
DaaSDaaS
Storage/Database/
Middleware
Storage/Database/
Middleware
datadata
People
DaaSDaaS
DaaSDaaSdatadata Build Data
Service
APIs
Deploy
Data
Service

Data-related activities
ASE Summer 2015 42
Wrapping
data
Publishing DaaS
interface
Typical activities for data wrapping and publishing
Typical activities for data updating & retrieval
Updating
data
Selecting
data
datadata
Provisioning
data

Typical data concern evaluation
ASE Summer 2015 43
Evaluating data
concerns
Evaluating data
concerns
Describing data
concerns
Describing data
concerns
Data Concerns
Evaluation Tools
Data Concerns
Representation Models
Populating data
concerns
Populating data
concerns
Publishing services
What do we need in order to perform these activities?

44
Data concern-aware DaaS
engineering process Typical activities
for data wrapping
and publishing
Typical activities
for data updating &
retrieval
ASE Summer 2015
Hong Linh Truong, Schahram Dustdar: On Evaluating and Publishing
Data Concerns for Data as a Service. APSCC 2010: 363-370
Hong Linh Truong, Schahram Dustdar: On Evaluating and Publishing
Data Concerns for Data as a Service. APSCC 2010: 363-370

Evaluating data concerns – the
three important points
45
• At which level the
evaluation is performed?
evaluation
scope
• When the evaluation is
done?
evaluation
modes
• How the evaluation tool
is invoked?
integration
model
ASE Summer 2015
Hong Linh Truong, Schahram Dustdar: On Evaluating and Publishing Data Concerns for Data as a Service. APSCC
2010: 363-370
Hong Linh Truong, Schahram Dustdar: On Evaluating and Publishing Data Concerns for Data as a Service. APSCC
2010: 363-370

Evaluating data concerns – some
patterns (1)
46
Pull, pass-by-referencesPull, pass-by-references
ASE Summer 2015

patterns (2)
47
Pull, pass-by-valuesPull, pass-by-values
ASE Summer 2015

patterns (3)
48
Push, pass-by-values (1)Push, pass-by-values (1)
ASE Summer 2015

patterns (4)
49
Push, pass-by-values (2)Push, pass-by-values (2)
ASE Summer 2015

Evaluation Tool – Internal Software
components
 Self-developed or third-party software
components for evaluation tool
 Advantages
 Tightly couple integration  performance, security,
data compliance
 Customization
 Disadvantages
 Usually cannot be integrated with other features
(e.g., data enrichment)
 Costly (e.g., what if we do not need them)
ASE Summer 2015 50

Evaluation tool – using cloud
services
 Evaluation features are provided by cloud
services
 Several implementations
 Informatica Cloud Data Quality Web Services, StrikeIron,
 Advantages
 Pay-per-use, combined features
 Disadvantages
 Features are limited (with certain types of data)
 Performance issues with large-scale data
 Data compliance and security assurance
ASE Summer 2015 51

Evaluation Tool -- using human
computation capabilities
 Professionals and Crowds can act as data
concerns evaluators
 For complex quality assessment that cannot be done by
software
 Issues
 Subjective evaluation
 Performance
 Limited type of data (e.g., images, documents, etc.)
ASE Summer 2015 52
Michael Reiter, Uwe Breitenbücher, Schahram Dustdar, Dimka Karastoyanova, Frank Leymann, Hong Linh Truong: A Novel
Framework for Monitoring and Analyzing Quality of Data in Simulation Workflows. eScience 2011: 105-112
Maribel Acosta, Amrapali Zaveri, Elena Simperl, Dimitris Kontokostas, Sören Auer, Jens Lehmann: Crowdsourcing Linked
Data Quality Assessment. International Semantic Web Conference (2) 2013: 260-276
Óscar Figuerola Salas, Velibor Adzic, Akash Shah, and Hari Kalva. 2013. Assessing internet video quality using
crowdsourcing. In Proceedings of the 2nd ACM international workshop on Crowdsourcing for multimedia (CrowdMM '13).
ACM, New York, NY, USA, 23-28. DOI=10.1145/2506364.2506366 http://doi.acm.org/10.1145/2506364.2506366
Michael Reiter, Uwe Breitenbücher, Schahram Dustdar, Dimka Karastoyanova, Frank Leymann, Hong Linh Truong: A Novel
Framework for Monitoring and Analyzing Quality of Data in Simulation Workflows. eScience 2011: 105-112
Maribel Acosta, Amrapali Zaveri, Elena Simperl, Dimitris Kontokostas, Sören Auer, Jens Lehmann: Crowdsourcing Linked
Data Quality Assessment. International Semantic Web Conference (2) 2013: 260-276
Óscar Figuerola Salas, Velibor Adzic, Akash Shah, and Hari Kalva. 2013. Assessing internet video quality using
crowdsourcing. In Proceedings of the 2nd ACM international workshop on Crowdsourcing for multimedia (CrowdMM '13).
ACM, New York, NY, USA, 23-28. DOI=10.1145/2506364.2506366 http://doi.acm.org/10.1145/2506364.2506366

53
QoD framework: publishing
concerns (1)
 Off-line data concern
publishing, e.g.
 a common data concern
publication specification
 a tool for providing data concerns
according to the specification
 supported by external service
information systems
ASE Summer 2015

concerns (2)
 On-the-fly querying data concerns associated with data
resources, e.g.,
 Using REST parameter convention
 Based on metric names in the data concern
specification
ASE Summer 2015 54
Hong Linh Truong, Schahram Dustdar, Andrea Maurino, Marco Comerio: Context, Quality and Relevance:
Dependencies and Impacts on RESTful Web Services Design. ICWE Workshops 2010: 347-359
Hong Linh Truong, Schahram Dustdar, Andrea Maurino, Marco Comerio: Context, Quality and Relevance:
Dependencies and Impacts on RESTful Web Services Design. ICWE Workshops 2010: 347-359

concerns (3)
 Specifying requests by using utilizing query parameters
the form of metricName=value
55
 Obtaining contex and quality by using context and quality
parameters without specifying value conditions
GET/resource?crq.accuracy="0.5"&crq.location=’’Europe”GET/resource?crq.accuracy="0.5"&crq.location=’’Europe”
curl http://localhost:8080/UNDataService/data/query/Population annual growth rate
(percent)?crq.qod
{”crq.qod” : {
”crq.dataelementcompleteness ”: 0.8654708520179372,
”crq.datasetcompleteness”: 0.7356502242152466,
...
}}
curl http://localhost:8080/UNDataService/data/query/Population annual growth rate
(percent)?crq.qod
{”crq.qod” : {
”crq.dataelementcompleteness ”: 0.8654708520179372,
”crq.datasetcompleteness”: 0.7356502242152466,
...
}}
ASE Summer 2015

Exercises
 Read mentioned papers
 Check characteristics, service models and
deployment models of mentioned DaaS (and
find out more)
 Identify services in the ecosystem of some DaaS
 Write small programs to test public DaaS, such
as Xively, Microsoft Azure and Infochimps
 Turn some data to DaaS using existing tools
ASE Summer 2015 56

Exercises (2)
 Identify and analyze the relationships between
data concerns evaluation tools and types of data
 Analyze trade-offs between on-line and off-line
evaluation and when we can combine them
 Analyze how to utilize evaluated data concerns
for optimizing data compositions
 Analyze situations when software cannot be
used to evaluate data concerns
ASE Summer 2015 57

58
Thanks for
your attention
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://dsg.tuwien.ac.at/staff/truong
ASE Summer 2015

TUW-ASE Summer 2015: Data as a Service - Models and Data Concerns

More Related Content

What's hot (20)

Viewers also liked (18)

Similar to TUW-ASE Summer 2015: Data as a Service - Models and Data Concerns (20)

More from Hong-Linh Truong (20)

Recently uploaded (20)

TUW-ASE Summer 2015: Data as a Service - Models and Data Concerns