Data modelling where did it all go wrong?

Data Modelling

Where did it all go wrong?
DAMA London, 15th June 2007
Christopher Bradley
1
I

Contents

1. Background
2. Seven deadly sins
3. Our part in fixing this

2
I

Audience Poll
What’s your role within your organization?

Data Architect
DBA
Manager or Executive Sponsor
Business Analyst
Consultant
Marketing
Other
3 Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+

1. Background


Background:
Data Management growth:
1950-1970 1970-1990 1990-2000 1990-2000

Database development
Database operation
Data requirements analysis
Data modelling
Enterprise data management coordination
Enterprise data integration
Enterprise data stewardship
Enterprise data use
Explicit focus on data quality
Security
Compliance
Other responsibilities

Background:
Data Modelling’s promise ….
"a single consistent definition of data"
"master data records of reference"
“reduced development time”
“improved data quality”
“impact analysis”
…….
No brainers?

So why is it that in many organisations the
benefits of data modelling still need to be
“sold” and in others the big benefits simply
fail to be delivered?

2. Seven
deadly sins


i: Not focusing on benefits

Project requirements vs Big picture
Reward drives behaviour
WIIFM
Metrics
Evidence
Sustained improvement


What’s the value X Data M odelling to BP?
Company of benefits

x body of know ledge - m odels repository.
Consistency of cross dom ain data concepts.
Eases M aster Data Take-on, Legacy M igration, M I/BI, Application
interoperability
Reuse of com m on m odels & definitions (including standard
industry m odels)
Interoperability, & efficiency through com m on approaches
Reduction in m aintenance.


Company X: User Survey; Benefits
What benefits are you gaining from the Data m odelling service?
80% 79%
77%
70% 70%
We are obtaining
benefit through use of a 60%
common modelling tool 50% 55% 60%
40%
We are obtaining benefit
through utilisation of a 30%
common repository
20%
10%
We are obtaining 0%
benefit through use of 4%

Disagree
common standards,
guidelines & processes

Stongly Agree
We are obtaining
benefit through re-use
of models & artefacts
We are obtaining benefit We are not
through provision of obtaining any
10 central support & help benefits
Complete keyboard char set so that all ordinary characters of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+

Company Y metrics
What’s the $ value of Data M odelling to BP?

A) Complete representation of requirements
M easures
• Number of definitions the client takes ow nership of. If the client is w illing to assume responsibility for the maintenance of the
definitions, then it is safe to assume the definitions are accurate.
• Number of modifications to the model after each review . This is more of a rolling "how w ell is the modelling process going"
measure than an end-state measure of how complete the model is. A low er number of post-review modifications is an indicator of a
higher degree of completeness.

B) Retention of collected information (including re-use)
M easures
• Number of times portions of a model are referenced (on a w eb page for example). If the model has been published (w hich all
should be) and the repository information is easily accessible, the "number of hits" on each entity (for example) can be a gauge of the
usefulness of the originally collected information.
• Number of entities re-used in subsequent projects. This is as much a measure of the quality of the original analysis (and potentially
design) as it is a measure of the amount of re-use. Costs savings for this measure can be calculated based on a "days per entity"
number. Total time savings (and related cost savings) w ould be equal to the "days per entity" multiplied by the number of entities re-
used
• Time to market for projects. Assuming w e w ere able to re-use an existing database for a second application, the time savings could
simply be "days per entity" multiplied by the number of tables in the existing database.

C) Consistent interface
M easures
• Review time by entity. The time required to review each entity (or definition) should decrease as the review ers become familiar
w ith the consistent style of the model. A side benefit to follow ing a consistent style is that subsequent projects w ill be able to
accurately reflect the amount of time required to review a data model in project plans based on the results of past review s.
• Amount of time spent during subsequent referral to the model. Just as the number of times the model is subsequently referenced
is a measure of the retention theme, the amount of time spent w hen referencing a specific portion of the model is a measure of the
consistency. If the model has follow ed a consistent interface, subsequent users of the model should be able to find the required
information quickly.


Value of Data Modelling - Company Z

• Increased reuse & development efficiency >>> Reduced
developm ent tim e (* based upon £10k per new Entity & 46% re-use)
$300m
• Increased consistency >>> Decreased maintenance (* based upon
22% reduction in # bespoke tables & messages)

$75m


ii: Forgetting the purpose

Top down only?
Bottom up & middle out
It’s not simply for RDBMS development


Why Produce a Data Model?
Company Z Top Ten Reasons

1. Capturing Business Requirements
2. Promotes Reuse, Consistency, Quality
3. Bridge Betw een Business and Technology
Personnel
4. Assessing Fit of Package Solutions
5. Identify and M anage Redundant Data
6. Sets Context for Project w ithin the Enterprise
7. Interaction Analysis: Compliments Process M odel
8. Pictures Communicate Better than Words
9. Avoid Late Discovery of M issed Requirements
14
10. Critical in M anaging Integration Betw een Systems

Not only for “new” Data Base Systems?
SOA:
Important in an SoA World.
Definition of data & consequently calls to / results from services is vital.
Straight through processing can exacerbate the issue
• what does the data mean?
• which definition of X (e.g. “cost of goods”)?
• need to utilise the logical model and ERP models definitions

Data Lineage:
Repository based Data migration design - Consistency
Source to target mapping
Reverse engineer & generate ETL
Impact analysis
ERP:
Model Data requirements – aid configuration / fit for purpose evaluation
Data Integration
Legacy Data take on
Master Data integration
BI / DW:
Model Data requirements in Dimensional Model
Reverse engineer BW Info Cubes, BO Universes, …….
15 Generate Star / Snowflake / Starflake schemas

iii: Language & intellectual snobbery

The term “ M odelling” often has baggage
associated w ith it
Use appropriate language & terms for different
audiences
Banish methodology bigots & dogma
Barker / ERD /UM L / OR / etc etc

Banish methodology bigots & dogma
NEVER air methodology issues in front of users


iv: Discipline
Dumbing dow n - It’s not just about picture
draw ing!
Don’t forget the metadata
Training & appropriate NASA Mars Climate Orbiter
personnel
Identify relevant
standards & guidelines
Communicate
Honesty – it’s not easy!


v: Inappropriate positioning

Don’t do it just for modelling's sake!

v: Inappropriate positioning

Data modelling performed in isolation – silos DM , PM , DBA ...
Left until too late in the lifecycle
Speed – too much focus on final 20% to be “ theoretically
perfect”
DM considered an overhead
Charging for M odelling infrastructure
Hidden / unpublished models – w hat’s the point!
Limited re-use
Projects left to ow n devices – “ the train has departed”
DM function not resourced appropriately thus models not
subject to peer / cross-domain review


vi: Failing to adapt
Plethora of tools – good usage is more important
than choosing the “ best”
Forgetting the overall information architecture
M aster Data, Transaction data, M I/BI, Unstructured, BDD …

Disservice by ERP package vendors
COTS Logical Data M odel w ith package?

Lack of soft skills
Hero seeking
cow boys


vii: Square pegs & round holes

TLA factory – DM , M DM , EDM , EII, CDI, SOA …….
The right people in the role?
Is being a good modeller enough?
Certification coming at last 
Engaging w ith the business
Nobody ow es us a living
Communicating our successes
Do people know w hy this is undertaken?
Creating communities of interest
Lack of “ Selling” skills

3. Our part in
fixing this

22
I

Industry Culture
DBAs, Data Architects and Executives are different creatures

DBA Data Architect Business Executive
• Cautious • Analytical • Results-Oriented
• Analytical • Structured • “Big Picture” focused
• Structured • Passionate • Little Time
• Doesn’t like to • “Big Picture” focused • “How is this going to help
talk • Likes to Talk me?”
• “Just let me • “Let me tell you about • “I don’t care about your
code!” my data model!” data model.”
• “I don’t have time.”

3NF


Role of the Data Architect
How to gain Traction, Budget and Executive buy-in
• Be Visible about the program:
• Identify key decision-makers in your organization and update them on your
project and its value to the organization
• Focus on the most important data that is crucial to the business first! Publish
that and get buy in before moving on. (e.g. start small with a core set of data)

•Monitor the progress of your project and show its value:
• Define deliverables, goals and key performance indicators (KPIs)
• Start small—focus on core data that is highly visible in the organization. Don’t
try to “boil the ocean” initially.
• Track and Promote progress that is made
• Measure Metrics where possible
“Hard data” is easy (# data elements, #end users, money saved, etc.)
“Softer data” is important as well (data quality, improved decision-making, etc.)
Anecdotal examples help with business/executive users
“Did you realize we were using the wrong calculation for Total Revenue?”
(based on data definitions)

Communicate Effectively
Provide Information to uses in their “Language”
• Repurpose information into various tools: BI, ETL, DDL, etc.
• Publish to the Web
• Exploit collaboration tools / SharePoint / Wiki …….
• Business users like Excel, Word, Web tools
Document Metadata
• Data in Context (by Organization, Project, etc.)
• Data with Definitions
Provide the Right Amount of Information
• Don’t overwhelm with too much information. For business users, terms and
definitions, might be enough.
• Cater to your audience. Don’t show DDL to a business user or Business
definitions to a DBA.
Market, Market, Market!
• Provide Visibility to your project.
• Talk to teams in the organization that are looking for assistance
• Provide short-term results with a subset of information, then move on.

Model publishing


Case Study: Web-based information
sharing


Company X
Data Management Maturity Model Obtaining Limited
Delivering broad
Quality & Re-use
Ideal, Obtaining
Optimal Value from Data

Operating in “Fire Benefits Level 5 - Optimised
Fighting” Mode Level 4 - Managed
Undesirable
Level 3 - Defined
Level 2 - Repeatable Aspiration
Data Principles Level 1 - Initial

Recognized Data Ownership Model Data Ownership Model Defined Data Data Ownership Model is Data Ownership Model has
does not exist. Data does not exist. Owners Ownership Model implemented for the key data been extended such that
Ownership Owners, if any, evolve commissioned in the exists. Ownership entities. Governance the majority of data entities
on their own during short-term for specific Model is loosely process regularly reviews are now governed in a
project rollouts (i.e. self projects & initiatives. applied to key data this model and its consistent manner.
appointed data owners).As-IsOwnership tends to be entities. application, updating and
in form of “Data Teams” To-Be
improving as needed.
or “Super Users” that
manage “all” data.

Unique Data definitions Key data defined in the Key data definitions Single set of data definitions Data definitions extended
unknown and/or short-term for specific exist to those who exist for the key data beyond just “key” data
Definitions inconsistent across the projects & initiatives. know where to look. entities. Definitions are entities. Common data
business(s). Definitions are not Multiple sets of published to a central definitions used throughout
leveraged from project definitions exist location that is accessible to the businesses & functions.
to project and changeAs-Is because no other programs, projects and
To-Be
often. rationalization/standar users in secure manner.
dization occurs.

Accessible Data repository(s) does Disparate set of data Multiple data A single integrated data Central data repository is
not exist. repositories exist as a repositories that repository houses the optimized via standard
Repositories result of specific synchronize and/or “record of reference” (single data collection &
projects & initiatives. communicate via version of the truth). Other distribution mechanisms.
Little or no bespoke interfaces. systems access the RoR Data accessible to other
As-Is
synch/communication from To-Be
the central integrated programs, projects and
across these tools. repository. users in secure manner.

Lifecycle Complete lack of Short term procedures Limited procedures or Defined & consistent set of Defined & consistent set of
procedures or controls or controls for key data controls for key data procedures & ctrls for key procedures & ctrls extend
Management for key data operations operations of create, operations of create, data operations of create, beyond just key data. End-
of create, read, update read, update & delete. read, update & delete. read, update & delete. Key to-end automated “create
& delete. No warehouse Ltd warehouse & Warehouse/archiving
As-Is data is proactively monitored to archive/warehouse”
To-Be
and/or archiving of IPL Title Fontget embedded in file zxcvbnm,./asdfghjkl;’#qwertyuiop[]1234567890-=`|ZXCVBNM<>?ASDFGHJKL:@~QWERTYUIOP{}¬!”£$%^&*()_+ that arch’ing/warehousing
archiving driven only by defined only for key so processes optimize the life-
29 Complete keyboard char set so that all ordinary characters
processes in place. space constraints. data entities. occurs at optimal times. cycle mgmt. of all data.

Make it sustainable:

Current position
Avoid the abyss via
investment in “ sustain”
activities
Visibility

Typical Gartner
“ hype cycle”

Technology Peak of inflated Trough of
Slope of enlightenment Plateau of productivity
Trigger expectations disillusionment

M aturity @ your company


Thank you

Contact details:
Email: chris.bradley@ipl.com
Tel: +44 (0)7973 184475
MSN: chrisbradley@bigfoot.com
Web: www.ipl.com

I

Data modelling where did it all go wrong?

More Related Content

Viewers also liked (19)

Similar to Data modelling where did it all go wrong? (20)

More from Christopher Bradley (20)

Recently uploaded (20)

Data modelling where did it all go wrong?