SQL Server Data Tools

1 | 17/11/2012 | SQL Saturday #176 – Pordenone, Italy

SQL Saturday #176
Pordenone, Italy
SQL Server Data Tools

17/11/2012 | SQL Saturday #176 – Pordenone, Italy

#sqlsat176


Marco Parenzan

 .NET Developer, SQL User
 1nn0va Speaker (www.innovazionefvg.net)
 Trainer and consultant on application
development in companies and in University
(of Trieste)
 Head of Development in CGN S.P.A.
(www.cgn.it)


The Developer‟s experience

DATABASE DEVELOPMENT


Database Development

 Often not integrated into application life cycle
management and processes
 What is a database development
experience?
 Install SSMS into developer machine….
 It‟s an «admin» tool
 Outside Visual Studio (it‟s Visual Studio yes)
 Third party tools…
 What does »deployment» means?


The premise

IMPERATIVE DATABASE
DEVELOPMENT

Database Development Challenges

 Databases are inherently stateful
 Focus is on ALTER instead of CREATE
 Dependencies complicate scripts
 Errors are often found only when scripts are
executed
 Synchronizing application and database
versions
 Targeting different SQL Server and SQL
Azure versions

Simple Schema Versioning Example
-- version 1
CREATE TABLE dbo.Speakers
(
SpeakerId INT NOT NULL,
FirstName NVARCHAR(64) NOT NULL,
LastName NVARCHAR(64) NOT NULL,

)

-- version 2 -- version 3
ALTER TABLE dbo.Speakers ALTER TABLE dbo.Speakers
WITH CHECK ADD CONSTRAINT WITH CHECK
Speaker_PK ADD CONSTRAINT Speaker_PK UNIQUE
(LastName, FirstName)

V1 V2 V3 Revision History


Imperative

 Imperative scripts hard codes knowledge about:
 The state of the target system:
 Presence of objects, dependencies, unbinding, rebinding
 Required ordering of operations
 Cumulative changes need to be serialized
 v1v2v3v4v5 instead of v1v4v5
 Validating the end-result against expected end-state
is hard
 No easy way to compare expected vs. actual
 Batching or separating multiple changes
 There are third party tools


Imperative Versioning
 -- version 1 Add table dbo.Auction
 IF OBJECT_ID (N'dbo.Auction', N'U') IS NULL
 BEGIN
 CREATE TABLE dbo.Auction
 (`
 id INT NOT NULL,
 name VARCHAR(25) NOT NULL,
 start DATETIME NULL,
 len INT NULL
 )
 END
 -- version 2 Add PK Au_PK
 IF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')
 BEGIN
 ALTER TABLE Auction
 WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)
 END
 -- version 3 Add UC Au_SK
 IF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = „UQ')
 BEGIN
 ALTER TABLE Auction
 WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)
 END


Missing/Manual

 Object Dependencies
 Intercept errors before deployment
 Don‟t loose data
 Sync DB
 Managing different versions

The foundation

DECLARATIVE DATABASE
DEVELOPMENT

Declarative

 Define what you want in the form of a model
 Think about a table without thinking about the
specific SQL
 Fill the model using a DSL (domain specific
language)
 T-SQL
 Use the model as the starting point “blueprint”
for all operations


Declarative Versioning

-- version 1
CREATE TABLE dbo.Auction
-- version 2
( CREATE TABLE dbo.Auction
id ( INT NOT NULL,3
-- version
name id CREATE NOT NOTdbo.Auction
TABLE NULL,
VARCHAR(25) NULL PRIMARY KEY,
INT
(
start DATETIME NULL, NOT NULL,
name VARCHAR(25)
len start NULL INT NOT NULL PRIMARY KEY,
INTidDATETIME NULL,
) name VARCHAR(25) NOT NULL UNIQUE,
len INT NULL
start DATETIME NULL,
)
len INT NULL
)

V1 V2 V3 Revision History


Model as the starting point “blueprint”

 Deployment/publish, start by comparing the
current state of the target with the required
state (blueprint)
 Use the resulting difference knowledge to
programmatically generate an deployment
plan
 Use plan to create sequence of (SQL)
statements required to make the target state
become like the blueprint state

Declarative Schema Deployment
17
Source Target
Model Reverse
Schema Schema
Compare Engineer
Model Model
Diff List

Plan
Optimizer
Additional
schema
Deploy Plan
artifacts
Script
Generator

Script Target
Executor Incremental target update DB


Imperative vs. Declarative

 Manual vs. generated / programmatic
 Point in time vs. always current
 Sequential vs. differential


Model Based
.SQL SqlScript Schema
Parse Dom Interpret Validate
Source Model

 All schema objects are represented inside a
model
 What is in the model is defined by the provider
 To load/hydrate a model instance
 Parse  SqlCodeDom, based on parsers Abstract
Syntax Tree (AST)
 Interpretation Schema Model
 Symbol list
 Object References (hard and soft dependencies)
 Validation


Script Fundamentals

 Requirement to be
able to round trip DSL A

artifacts S T

 Script  Model
 SQL script Parse Script
Parser
Script fragment T-SQL
Fragment

 Script fragment Script
Gen  SQL script
 Script fragment is AST
Script
Script
Gen T-SQL
Fragment
(Abstract Syntax Tree)


ScriptDom
 SQL Server 2012 managed
parser
 Supports SQL Server 2005+
 Class TSqlXXXParser
 XXX = [80, 90, 100, 110]
 SQLDom.msi (redist x86/x64)
 Microsoft.SqlServer.TransactSql.S
criptDom.dll
 C:Program Files (x86)Microsoft
SQL Server110SDK
AssembliesMicrosoft.SqlServer.Tra
nsactSql.ScriptDom.dll
 GAC


Basic ScriptDom loop

static void Main(string[] args)
{
bool initialQuotedIdentifiers = false;
TSqlParser parser = new TSql110Parser(initialQuotedIdentifiers);

StreamReader sr = new StreamReader(@".test.sql");
IList<ParseError> errors;

TSqlFragment fragment = parser.Parse(sr, out errors);

StreamWriter sw = new StreamWriter(Console.OpenStandardOutput());

Sql110ScriptGenerator scriptGen = new Sql110ScriptGenerator();
scriptGen.GenerateScript(fragment, sw);
}


23

SQL SERVER DATA TOOLS
OVERVIEW

History of the “DataDude” Project
 Project funded April 2005
 Project started July 1st 2005
 Visual Studio 2005 Team System for Database
Professionals
Professionals
 Re-architected to be declarative model based system
Professionals GDR R2
Professionals
 Transitioned to SQL Server 2009
 SQL Server Data Tools


Database Project
 Evolution of VS database project
 We have more things
 We don‟t have
 Data generation
 Unit testing
 Data compare
 Use a DB Project for this
 http://social.msdn.microsoft.com/Forums/en-US/ssdt/thread/33664902-
a4e2-415a-ad4e-a480fbc56158/
 Not available in Professional or Express Edition
 Migration
 http://geekswithblogs.net/80n/archive/2012/09/11/vsdb-to-ssdt-
series--introduction.aspx
 Comparison

DataDude Vs SSDT Comparison (1)

http://blogs.msdn.com/b/ssdt/archive/2011/11/21/sql-server-data-tools-ctp4-vs-
vs2010-database-projects.aspx


DataDude Vs SSDT Comparison (2)

http://blogs.msdn.com/b/ssdt/archive/2011/11/21/sql-server-data-tools-ctp4-vs-
vs2010-database-projects.aspx


Supported OS Platforms

 Windows Vista SP2+
 Windows 7 SP1+
 Windows Server 2008 SP+
 Windows Server 2008 R2 SP1+

 Note: this is different then the Visual Studio
2010 platforms


Availability

 Visual Studio 2010 Integrated Shell
 SQL-CLR requires Visual Studio 2010 Standard
 Visual Studio 2010 SP1
 Visual Studio 2012
 SQL Server 2012


Features of SQL Server Data Tools

03/29/2012

31

CONNECTED DEVELOPMENT


Connected Development

 SQL Server Object Explorer (SSOX)
 T-SQL Editor/Debugger/IntelliSense (New Query)
 Power-Buffer (New Object/View Code)
 Table Designer (View Designer)
 Data Editor (View Data)
 Execute Procedure/Function
 Schema Compare

32 | 03/29/2012

SQL Server Object Explorer (SSOX)
 The SQL Server navigation tool window
 Modeled after the SSMS Object Explorer
 Server scoped vs. Server Explorer which is database scoped
 Launch point for T-SQL Editor, Debugger, Object
Execution, Designers, …
 Supports connecting directly to SQL Server
 SQL Server {2005, 2008, 2008R2, 2012} and SQL Azure
 Supports connecting directly to a (contained) database
 Constructs a partial model in the background and under the
covers
 Supports:
 Cascading Drops and Renames
 Drift detection
 Automatically updates the tree/source as changes occur on the server


T-SQL Editor/Debugger/IntelliSense
 T-SQL IntelliSense
 SQL Server 2005+ incl. SQL Azure
 Result to Grid | Text | File
 Execution modes
 Execute
 Execute with debugger
 Reusable debug script generation


Power-Buffer/Table Designer
 Declarative deferred online schema management
 Brings “project-oriented” features to online / connected
development
 Live errors and warnings as changes are made against code
 “Project-oriented” validation during editing
 Table designer
 Code-behind designer
 Works on top of Power-Buffer (partial model) or project model
 Strives to be the same experience online as experienced in the
project system
 Coordinated execution of buffers against the same target
 Diff scripts by default are transactional
(ToolsOptionsSQL Server ToolsOnline EditingInclude
transactional scripts)


Data Editor

 View and Edit Data via grid editor
 Copy-Paste support
 Generate as INSERT to file or query window


Execute Procedure/Function

 Dialog based input for parameter collection
 Generates SQL script for execution/debugging


38

PROJECT BASED
DEVELOPMENT

Creating and Importing
 Single project template
 VSDB note:
 No distinction between server or user projects
 Server projects are projects with target database set to
master
 Import from:
 Database
 DACPAC
 SQL Script (multi-file import enabled)
 Selective import by using Schema Compare
 “Create New Project” from SSOX
 Inherits version from source database


Folder Structure

 Options on import:
 None
 Schema
 Object type
 SchemaObject type


Project Directory Guidelines…

 Do not host more then one table inside script
file
 Table designer will not see additional tables
inside script


Additional Language Services
 Projects enable additional language services
 Go To Definition
 Find All References
 Refactoring
 Rename (sp_rename)
 Move to Schema (ALTER SCHEMA schema_name
TRANSFER)
 Wildcard Expansion
 Fully Quality Name
 Invoked from Editor, Table Designer and SQL
Server Object Explorer for project based schema
objects


T-SQL Static Code Analysis

 Rule set
 Suppression
 File level
 Project level
 MSBuild support


Multi-Targeting

 Target version aware:
 SQL Server 2005
 SQL Server 2008 & SQL Server 2008 R2
 SQL Server 2012
 SQL Azure


Isolated Sandbox

 F5 – Run by default configure to use the
LocalDB instance associated with the project
 Data Source=(localdb)<project name>;Initial
Catalog=<project name>;Integrated
Security=True;Pooling=False;Connect
Timeout=30
 To retarget to a different SQL Server version
and/or shared SQL Server change:
 Project propertiesDebugTarget Connection
String

SQL Server 2012 Express LocalDB
 Simplify embedded usage of SQL Server
 Simple installation
 Zero-admin database
 Same programming surface as User Instances of SQL Express
 The same sqlservr.exe as in service-based SQL Server with the same language and features
 Connection through the same client-side APIs, using a new connection string option
 SQL Server programming symmetry
 Installation similar to .NET Framework model
 One set of installed binaries (per major version), no global configuration, no constantly-
running service
 Activation model similar to RANU (Access, SQL Compact, …)
 The application connects to a database file, SQL activates automatically
 No service: SQL process starts in the context of current user
 Default connection string
 (localdb)v11.0
 http://msdn.microsoft.com/en-us/library/hh510202(v=SQL.110).aspx


Build Time Validation

 Build validates the complete model
 Platform specific validation
 Example – Enforces clustered index requirement on
tables in SQL Azure
 Integration with Compiler Services
 Engine-level build-time validation without schema
deployment
 Pre & Post Deployment scripts
 Build validates syntactical correctness of pre- and
post- deployment scripts using the SQLCMD
variables


Build Actions
 Build - This script will be interpreted as T-SQL (or an XML
Schema in the case of XML Schema Collections) and reflected in
the T-SQL Model
 Compile - C# or VB (depending on your project properties) to be
deployed as a SQLCLR assembly
 None - Use this for any loose artifacts (like text files)
 PreDeploy - The pre deployment master script (there can only be
one)
 PostDeploy - The post deployment master script (there can only
be one)
 RefactorLog - This is a refactor log which notifies the DaxFx
deployment engine that particular changes should be interpreted
as refactoring operations and not drop/adds
 Build Extension Configuration - Reserved for future use, for this
is interpreted as None


Pre & Post Deployment Scripts
 Can be used for any “arbitrary” SQL actions that you
need to perform “before” and “after” deployment of
the schema objects
 The pre deployment block comes after the generic
database context validation block
 The post deployment block comes after the schema
objects block
 Pre and post deployment scripts must be
idempotent
 They are “executed” each deployment!
 http://blogs.msdn.com/gertd/archive/2007/01/25/idempoten
t-scripts-required.aspx


Database References
 3-part name usage:
 SELECT * FROM
[LiteralDBRef].[Schema1].[Table1]
 SELECT *
FROM [$(DBRef)].[Schema1].[Table1]
 4-part name usage:
 SELECT *
FROM [$(ServerRef)].[$(DBRef)].[Schema1].[Table1]
 SELECT *
FROM
[$(ServerRef)].[LiteralDBRef].[Schema1].[Table1]


Project Deployment

 Build serializes the model in to model.xml
stream inside .dacpac file


What is a Data-Tier Application
(dacpac)?
 Data-Tier Application Component (DAC)
 SQL Server database registered as Data-Tier Application
Component
 DAC is a self contained entity of a database used by an
application
 Unit of deployment and management
 Authored along side your application code
 Compiled into a DAC package (DACPAC)
 Extracted for existing databases
 Database extracted (DACPAC) or exported (BACPAC) into
packages
 Package is deployable to SQL Server and SQL Azure

53
Copyright © 2012 Gert Drapers - All Rights Reserved. 03/29/2012

DACFX Packages

 .dacpac - Packaged Schema representing the declarative model
of the database
 Not SQL, but a declarative Schema Model representation of the
database
 Compiled from source code or Extracted from existing databases
 Deploy as new database or Incremental Upgrade an existing database
 Supports all SQL Server Objects and Types

 .bacpac - Packaged Schema and Data representing the state of
the database
 Composed of Schema Model with stateful properties and Data Streams
 Exported from existing databases
 Imported as a new database or populates an empty database
 Supports most SQL Server Objects and Types (SQL Azure parity)

54

DAC Framework v3.0 (DACFX)

 Core SQL Server redist component providing
modeling, reverse engineering and
deployment capabilities
 Managed Public API
 Exposes verbs for DACPAC and BACPAC
package operations
 Command-line tool (SqlPackage.exe)
 Executable over the managed public API

55

Database Schema Snapshots

 Point-in-time representation of schema model
 Model must be “error-free” state
 Produces same file format (*.dacpac) as build or
extract
 To be used for:
 Versioning
 Comparison
 Sharing schema definitions or changes


Schema Compare

 Performs a model comparison between:
 Databases
 Projects
 DACPAC files (snapshots, build outputs, extracts)
 Supports:
 Refactoring log
 SQLCMD variables

57 | 03/29/2012

Schema Definition

Script
Model
Builder
Fragment

Source
DB Scaffolding Schema
Model

Schema Model
Deserializer
Package


Deployment & Schema Compare

Script Script
Model Model
Builder Builder
Fragment Fragment

Source Target
DB Scaffolding Schema Schema Scaffolding DB
Model Model

Model
Schema Model Compare Model Schema
Deserializer Deserializer
Package Package
Diff List

Plan Diff
Optimizer Visualizer

Deploy Plan

59 | 17/11/2012 | SQL Saturday #176 – Pordenone, Italy Script
Generator

Schema Compare Next and Previous move between top-
level objects, no longer expand the tree.
Simpler set of columns are aligned around the central
action column echoes alignment of schemas above

60
Group-by Action is the Same
default column order
is used for all
groupings;
cleaner and
easier to
Schema no longer parse
used as a parent for regardless of
top-level objects; only grouping
shown when changed

Improved script diff’
algorithm for tables
emphasizes columns
as unit of change.
Gives much improved
alignment. Also
improved look


Schema Compare


Deployment Options

 Options to influence
deployment behavior
 Ignore*
 Deployment validation
options
 Do not alter Change
Data Capture objects
 Do not ALTER replicated
objects
 Verify deployment

Copyright © 2012 Gert Drapers - All Rights
62 03/29/2012
Reserved.

63

SQL CLR


03/29/2012

SQL-CLR Support

 Embedded C# or VB.NET code artifacts
 Deployed as SQL-CLR objects
 Binary assembly deployment and SQL-CLR
object association
 Example:
 CREATE TYPE [ClrSchema].[IPv4]
EXTERNAL NAME [IPAssembly].[IPv4];


CLR Needs

 SQL Server CLR usage
 Geographical support
 Your needs
 Web communication
 Custom function
 String management
 Aggregate functions
 «extreme calculus» in sp
 Don‟t abuse

66

SQL AZURE


03/29/2012

SQL Azure

 It‟s Cloud Computing!
 SQL Server e SQL Azure sono diversi
 Diverse esigenze
 Connected development on SQL Azure
 Change targetFix & Build


Conclusions





 Already third party tools
 First release
 Many step forward
 Some step back (compared to DataDude)


Conclusions

 Modeling infrastructure
 One development environment
 More checks over actions
 Use targets
 A tool for SQL Azure
 Debug and schema deployment
 Go under TFS


Resources

 Articles
 MSDN Magazine Sept 2011 The "Juneau"
Database Project
 SQL Server Data Tools Online Installation
 http://msdn.microsoft.com/data/tools
 Team Blog
 http://blogs.msdn.com/b/ssdt/


References

 twitter: @marco_parenzan
 email: marco.parenzan@libero.it
 blog: http://codeisvalue.wordpress.com/
 facebook: http://www.facebook.com/parenzan.marco
 linkedin: http://it.linkedin.com/in/marcoparenzan
 slideshare: http://www.slideshare.net/marco.parenzan/


GRAZIE! Commenta la sessione che hai
appena seguito su Twitter
#sqlsat176


SQL Server Data Tools

More Related Content

Similar to SQL Server Data Tools (20)

More from Marco Parenzan (20)

Recently uploaded (20)

SQL Server Data Tools

Editor's Notes