BIML: BI to the next level

BI Markup Language:
BI to the next level
Davide Mauri
SolidQ – Founder & Mentor
@mauridb
www.solidq.com
dmauri@solidq.com
13.12.2014

Thank you to all our SPONSORS!
13.12.2014

Davide Mauri
 20 Years of experience on the SQL Server Platform
 Specialized in Data Solution Architecture, Database Design,
Performance Tuning, Business Intelligence
 Projects, Consulting, Mentoring & Training
 Regular Speaker @ SQL Server events
 Microsoft SQL Server MVP
 President of UGISS (Italian SQL Server UG)
 Mentor @ SolidQ
 Find me here:
 Blog: http://sqlblog.com/blogs/davide_mauri/default.aspx
 Twitter: @mauridb
13.12.2014

Agenda
 BI, DWH & ETL today
 BIML – What’s that?
 Tools
 BIDS Helper
 MIST
 BIML Kick Off
 Basic Concepts
 Automation With “Embedded” C#
 BIML & PowerShell
13.12.2014

Building a BI/DWH/ETL in 2014
Is still a (almost) manual process
A *lot* of repetitive low-value work
No (or very few) standard tools available
13.12.2014

How it should be
Semi-automatic process
 “develop by intent”
Define the mapping logic from a semantic
perspective
 Source to Dimensions / Measures
 (Metadata anyone?)
Design the model and let the tool build it for you
13.12.2014

How it should be
Semi-automatic process
 “develop by intent”
Define the mapping logic from a semantic
perspective
 Source to Dimensions / Measures
 (Metadata anyone?)
Design the model and let the tool build it for you
13.12.2014
CREATE DIMENSION Customer
FROM SourceCustomerTable
MAP USING CustomerMetadata
ALTER DIMENSION Customers
ADD ATTRIBUTE LoyaltyLevel
AS TYPE 1
CREATE FACT Orders
FROM SourceOrdersTable
MAP USING OrdersMetadata
ALTER FACT Orders
ADD DIMENSION Customer

The perfect BI process & architecture
13.12.2014
Iterative!

Design Pattern
“A general reusable
solution to a
commonly occurring
problem within a
given context”
13.12.2014

No Monkey Work!
13.12.2014
Let the people think
and let the machines
do the «monkey»
work.

BIML – What’s that?
 Business Intelligence Markup Language
 XML-Based
 BI Object-Generator
 SQL Server
 Integration Services
 Analysis Services
13.12.2014

 Allows a declarative approach to define BI
objects
 Favor automation
 Human-readable
 Compatible with SCC system
 Allows merge of files
 Born from “Project Vulcan”
 Still available on CodePlex
13.12.2014

 Once objects are generated, BIML is not needed
anymore
 No additional runtime/components required
 Support for SQL Server 2008 & 2012 objects
 2014 is coming very soon
 Anyway generated packages are automatically
converted into 2014 packages by Visual Studio 2013,
without problems 
13.12.2014

 BIMLScript is based on the T4 templating engine.
Allows the generation of BIML using
 C#
 VB.NET
 BIML + BIMLScript = HTML + ASP Classic
13.12.2014

Tools
 BIDS Helper
 http://bidshelper.codeplex.com/
 Visual Studio Plugin
 Free
 Source Code Available
 Not all BIML features are supported
 “Limited” to SSIS generation only
13.12.2014

Tools
 MIST
 Full-Featured BIML IDE
 Buy or Rent
 Made by BIML creator: Varigence
13.12.2014

DEMO
 First Contact with BIML
13.12.2014

Basic Concepts
 A single BIML file can generate more than one
SSIS Package
 All SSIS object and related properties are exposed
via BIML elements
 Additional specific attribute: ConstraintMode
 Linear or Parallel
13.12.2014

Basic Concepts
 Parallel Constraint Mode
 “Inputs” connected to “Outputs”
 SSIS Variables: “dot” notation
13.12.2014

Basic Concepts
 Control Flow and Data Flow 100% supported
 In Data Flow it’s mandatory to define input and
outputs
 Since they aren’t constraints, but data streams 
13.12.2014

DEMO
 BIML is your friend!
13.12.2014

Automation with BIMLScript
 BIML Script (T4) Directives: <#@ and #>
13.12.2014

 BIML Script Tags: <# e #>
 Allows usage of C# / VB.Net within BIML
 The “=” symbol allows “in-line” code injection
13.12.2014

 Metadata is needed in order to define what to
generate
 Metadata can be extracted by
 BIML Engine (Hadron)
 T-SQL Queries
13.12.2014

 Metadata via BIML Engine (Hadron)
13.12.2014

 Metadata via SQL Query
13.12.2014

DEMO
 BIMLScript RULES!
13.12.2014

Debugging BIML
 With complex BIML Script it can be difficult to
“see” what’s happening behind the scenes
 With MIST you can see the resulting BIML
generated from BIML Script
 But with BIDS Helper you’re on your own.
 NLog can help a lot here!
13.12.2014

Debugging BIML
 Nlog: free, powerful & flexible logging
infrastructure
 http://nlog-project.org/
 Stand-Alone Assembly
 Just reference the assembly in the BIML Script (T4)
template and then use Nlog objects & methods
 http://sqlblog.com/blogs/davide_mauri/default.aspx
13.12.2014

DEMO
 BIMLScript & NLog
13.12.2014

BIML & PowerShell
 What if you don’t like C#...
 Or if BIMLScript looks to complex?
 Especially without MIST?
 One option is to generate BIML using PowerShell
 Is just XML after all….
13.12.2014

DEMO
 BIML & PowerShell
13.12.2014

Conclusions
 Use BIML!!!!
 It helps to
 Make better use of your time
 Create your own ETL patterns
 http://www.slideshare.net/davidemauri/automating-dwh-patterns-
13.12.2014
through-metadata
 Support Refactoring & Enables Agility
 Favor use of metadata

References
 BIML Script
 http://bimlscript.com/
 Stairway to BIML
 http://www.sqlservercentral.com/stairway/100550/
 Automating DWH Patterns Through Metadata
 http://www.slideshare.net/davidemauri/automating-dwh-
13.12.2014
patterns-through-metadata

BIML: BI to the next level

More Related Content

What's hot (20)

Viewers also liked (8)

Similar to BIML: BI to the next level (20)

More from Davide Mauri (20)

Recently uploaded (20)

BIML: BI to the next level

Editor's Notes