SlideShare a Scribd company logo
SQL Type Provider

Ross McKinlay, January 2014
@pezi_pink

www.pinksquirrellabs.com
github.com/pezipink/sqlprovider
But….. We already have a SQL type
provider!!!
• Only supports SQL Server

• Supports any number of database vendors

• Uses SQLMetal.exe to generate code

• No code generation step

• Is not really an erasing type provider,
doesn’t scale

• Fully erasing type provider, can scale to
massive schemas

• Is closed source

• Completely open source!

• Has very limited extensibility points

• Pluggable back end, and open source. Go
WILD!
• Supports a limited set of LINQ functions
(currently)
• Constraint navigation and querying, “dot
individuals”

• Supports almost all LINQ functions
• Limited schema exploration mechanics
SqlProvider Features
•

Instant access to a variety of database vendors with no code generation step

•

Lazily evaluated metadata enabling scaling to large schemas

•

LINQ queries, presently with the following supported keywords :
–

select, select many(for), join, distinct, where, take (top), exactlyOne (single), sortBy, sortByDescending, thenBy, thenByDescending

•

“Dot Individuals” allowing you to directly view the rows of the database as static types via
intelliense

•

Schema navigation – you can directly enumerate over the constraints of an entity – no need to
understand the relational design of the database up front to perform joins

•

Custom operators suc as in In |=| and Like =%

•

F# Option types can be used for nullable columns

•

Stored procedures (well, sort of, support for this isn’t great yet!)

•

Currently support for MS SQL Server SQLite, Postgresql

•

Debugging hooks
Compile Time

Schema
Metadata

Provided Types

Execution Time

ISqlProvider

Quotations

Type Provider

LINQ

SQL Entity

DataContext
Entity Attributes
(Columns)

Schema
Metadata

Provided Types

Employee
Id, Name, etc

Company
Id, Address, etc

Erased Type
SqlEntity
KeyValue Pairs

Order
Id, Amount, etc
Entity Relationships
(Constraints)

Schema
Metadata

Provided Types

Employee
{ FK Company –> Employee }

Company
{ FK Company –> Employee }
{ FK Company -> Order }

Constraint information

Static
DataContext

Order
{FK Company -> Order }
Am I a compiler? Yes. Pretty much.
RUNTIME

Project Results

LINQ AST

Execute Query

SQL AST

Generate SQL

SQL Query Type

Optimise

ISqlProvider

LINQ Expression
LINQ Expression Trees!
Nested Active Patterns!
| MethodCall(None, (MethodWithName "Join"),
[createRelated
Convert(MethodCall(_, (MethodWithName "_CreateEntities"), [String destEntity] ))
OptionalQuote (Lambda([ParamName sourceAlias],SqlColumnGet(sourceTi,sourceKey,_)))
OptionalQuote (Lambda([ParamName destAlias],SqlColumnGet(destTi,destKey,_)))
OptionalQuote (Lambda(projectionParams,_))]) ->

| OptionalOuterJoin(outerJoin,
MethodCall(None,(MethodWithName "_CreateRelated"),
[param; _;
String PE; String PK;
String FE; String FK;
String IE; RelDirection dir;])) ->
| MethodCall(Some(ParamName name | PropertyGet(_,PropName name)),
(MethodWithName "GetColumn"),[String key]) ->
Projection
Analysis and transformation steps are applied to the LINQ projection expression;
1)

To extract fields that have been explicitly been selected so they can be used to construct a sensible
query where possible

2)

Entity access nodes are replaced with calls to GetSubEntity() which performs the magic allowing it to
seem as if using an ORM, when in reality each row’s data is stored in one master SqlEntity

SqlEntity A

SqlEntity C

GetSubEntity(a)

DataRow
(SqlEntity)

SqlEntity B
GetSubEntity(b)

GetSubEntity(c)

a.Name

b.Addr1

b.Addr2

a.Title

c.Id

c.Amount

a.Phone

b.Addr2
Extensibility via ADO.NET
All you need to do is implement most of this interface! WHAT COULD BE SIMPLER! 
type internal ISqlProvider =
abstract CreateConnection : string -> IDbConnection
abstract CreateCommand : IDbConnection * string -> IDbCommand
abstract CreateCommandParameter : string * obj * SqlDbType option -> IDataParameter
abstract CreateTypeMappings : IDbConnection -> Unit
abstract ClrToEnum : (string -> SqlDbType option) with get
abstract SqlToEnum : (string -> SqlDbType option) with get
abstract SqlToClr : (string -> Type option)
with get
abstract GetTables : IDbConnection -> Table list
abstract GetColumns : IDbConnection * Table -> Column list
abstract GetPrimaryKey : Table -> string option
abstract GetRelationships : IDbConnection * Table -> (Relationship list * Relationship list)
abstract GetSprocs : IDbConnection -> Sproc list
abstract GenerateQueryText : SqlQuery * string * Table *
Dictionary<string,ResizeArray<string>> -> string * ResizeArray<IDataParameter>
^ Of course, this last one is the kicker… but thankfully you can largely just copy and
paste from an existing provider 
SQL Provider
Questions

¿

More Related Content

PDF
Type-safe front-end development with Scala
PDF
Killing Shark-Riding Dinosaurs with ORM
PPTX
Hibernate Tips ‘n’ Tricks - 15 Tips to solve common problems
PDF
Scala Frameworks for Web Application 2016
PDF
Develop realtime web with Scala and Xitrum
PDF
Scala Abide: A lint tool for Scala
PPTX
2017.09.09 Orbit - Seattle Code Camp
PDF
ORM Pink Unicorns
Type-safe front-end development with Scala
Killing Shark-Riding Dinosaurs with ORM
Hibernate Tips ‘n’ Tricks - 15 Tips to solve common problems
Scala Frameworks for Web Application 2016
Develop realtime web with Scala and Xitrum
Scala Abide: A lint tool for Scala
2017.09.09 Orbit - Seattle Code Camp
ORM Pink Unicorns

What's hot (20)

PPTX
PPTX
CodeFest 2014. Christopher Bennage — Semantic Logging. Avoiding the log chaos
PDF
Scala for java developers 6 may 2017 - yeni
ODP
Introduction to Scala JS
PPTX
Hibernate Performance Tuning @JUG Thüringen
PDF
How to write a web framework
PPTX
Introduction to GraphQL in Scala (ScalaMatsuri 2017)
PDF
OCCIware
ODP
Xitrum @ Scala Conference in Japan 2013
ODP
Scal`a`ngular - Scala and Angular
PDF
Xitrum HOWTOs
PDF
ITB2017 - Slaying the ORM dragons with cborm
PDF
State of Search, Solr and Facets in Drupal 8 - Drupalcamp Belgium 2015
PDF
Java DataBase Connectivity -JDBC Part-1
PPTX
Getting Started with Javascript
KEY
Rails 3.1 - Grzegorz Witek & Krzysztof Jablonski (PRUG 3.0)
PDF
How i wrote my own bdd framework
PPTX
Scala elegant and exotic part 1
PDF
Scala in a wild enterprise
PDF
Coscup
CodeFest 2014. Christopher Bennage — Semantic Logging. Avoiding the log chaos
Scala for java developers 6 may 2017 - yeni
Introduction to Scala JS
Hibernate Performance Tuning @JUG Thüringen
How to write a web framework
Introduction to GraphQL in Scala (ScalaMatsuri 2017)
OCCIware
Xitrum @ Scala Conference in Japan 2013
Scal`a`ngular - Scala and Angular
Xitrum HOWTOs
ITB2017 - Slaying the ORM dragons with cborm
State of Search, Solr and Facets in Drupal 8 - Drupalcamp Belgium 2015
Java DataBase Connectivity -JDBC Part-1
Getting Started with Javascript
Rails 3.1 - Grzegorz Witek & Krzysztof Jablonski (PRUG 3.0)
How i wrote my own bdd framework
Scala elegant and exotic part 1
Scala in a wild enterprise
Coscup
Ad

Similar to SQL Provider (20)

PPTX
Entity Framework v1 and v2
PDF
In-memory ColumnStore Index
PPTX
Dapper: the microORM that will change your life
PPTX
Entity Framework V1 and V2
PPT
MySQL, LINQ and the ADO_NET Entity Framework Presentation.ppt
PPTX
In memory databases presentation
PPT
LINQ 2 SQL Presentation To Palmchip And Trg, Technology Resource Group
PDF
Entity Framework Interview Questions PDF By ScholarHat
PPTX
111111112222223333335555555666Unit-4.pptx
PPT
L2s 090701234157 Phpapp02
PPT
Object Relational Mapping with LINQ To SQL
PDF
What Is Super Key In Dbms
PDF
Database Management System
PPT
What's New for Data?
PPTX
Entity Framework Overview
PDF
The LINQ Between XML and Database
PPT
Sql Summit Clr, Service Broker And Xml
PPTX
Object relational database management system
PPTX
Introduction to Database Management Systems (DBMS)
PPTX
Being RDBMS Free -- Alternate Approaches to Data Persistence
Entity Framework v1 and v2
In-memory ColumnStore Index
Dapper: the microORM that will change your life
Entity Framework V1 and V2
MySQL, LINQ and the ADO_NET Entity Framework Presentation.ppt
In memory databases presentation
LINQ 2 SQL Presentation To Palmchip And Trg, Technology Resource Group
Entity Framework Interview Questions PDF By ScholarHat
111111112222223333335555555666Unit-4.pptx
L2s 090701234157 Phpapp02
Object Relational Mapping with LINQ To SQL
What Is Super Key In Dbms
Database Management System
What's New for Data?
Entity Framework Overview
The LINQ Between XML and Database
Sql Summit Clr, Service Broker And Xml
Object relational database management system
Introduction to Database Management Systems (DBMS)
Being RDBMS Free -- Alternate Approaches to Data Persistence
Ad

Recently uploaded (20)

PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
A comparative analysis of optical character recognition models for extracting...
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Network Security Unit 5.pdf for BCA BBA.
PPTX
A Presentation on Artificial Intelligence
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PPTX
Machine Learning_overview_presentation.pptx
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Approach and Philosophy of On baking technology
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
cuic standard and advanced reporting.pdf
PPT
Teaching material agriculture food technology
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Assigned Numbers - 2025 - Bluetooth® Document
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
DOCX
The AUB Centre for AI in Media Proposal.docx
Programs and apps: productivity, graphics, security and other tools
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Reach Out and Touch Someone: Haptics and Empathic Computing
A comparative analysis of optical character recognition models for extracting...
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Network Security Unit 5.pdf for BCA BBA.
A Presentation on Artificial Intelligence
Agricultural_Statistics_at_a_Glance_2022_0.pdf
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Machine Learning_overview_presentation.pptx
MYSQL Presentation for SQL database connectivity
Approach and Philosophy of On baking technology
Mobile App Security Testing_ A Comprehensive Guide.pdf
cuic standard and advanced reporting.pdf
Teaching material agriculture food technology
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Assigned Numbers - 2025 - Bluetooth® Document
Dropbox Q2 2025 Financial Results & Investor Presentation
Building Integrated photovoltaic BIPV_UPV.pdf
The AUB Centre for AI in Media Proposal.docx

SQL Provider

  • 1. SQL Type Provider Ross McKinlay, January 2014 @pezi_pink www.pinksquirrellabs.com github.com/pezipink/sqlprovider
  • 2. But….. We already have a SQL type provider!!!
  • 3. • Only supports SQL Server • Supports any number of database vendors • Uses SQLMetal.exe to generate code • No code generation step • Is not really an erasing type provider, doesn’t scale • Fully erasing type provider, can scale to massive schemas • Is closed source • Completely open source! • Has very limited extensibility points • Pluggable back end, and open source. Go WILD! • Supports a limited set of LINQ functions (currently) • Constraint navigation and querying, “dot individuals” • Supports almost all LINQ functions • Limited schema exploration mechanics
  • 4. SqlProvider Features • Instant access to a variety of database vendors with no code generation step • Lazily evaluated metadata enabling scaling to large schemas • LINQ queries, presently with the following supported keywords : – select, select many(for), join, distinct, where, take (top), exactlyOne (single), sortBy, sortByDescending, thenBy, thenByDescending • “Dot Individuals” allowing you to directly view the rows of the database as static types via intelliense • Schema navigation – you can directly enumerate over the constraints of an entity – no need to understand the relational design of the database up front to perform joins • Custom operators suc as in In |=| and Like =% • F# Option types can be used for nullable columns • Stored procedures (well, sort of, support for this isn’t great yet!) • Currently support for MS SQL Server SQLite, Postgresql • Debugging hooks
  • 5. Compile Time Schema Metadata Provided Types Execution Time ISqlProvider Quotations Type Provider LINQ SQL Entity DataContext
  • 6. Entity Attributes (Columns) Schema Metadata Provided Types Employee Id, Name, etc Company Id, Address, etc Erased Type SqlEntity KeyValue Pairs Order Id, Amount, etc
  • 7. Entity Relationships (Constraints) Schema Metadata Provided Types Employee { FK Company –> Employee } Company { FK Company –> Employee } { FK Company -> Order } Constraint information Static DataContext Order {FK Company -> Order }
  • 8. Am I a compiler? Yes. Pretty much. RUNTIME Project Results LINQ AST Execute Query SQL AST Generate SQL SQL Query Type Optimise ISqlProvider LINQ Expression
  • 10. Nested Active Patterns! | MethodCall(None, (MethodWithName "Join"), [createRelated Convert(MethodCall(_, (MethodWithName "_CreateEntities"), [String destEntity] )) OptionalQuote (Lambda([ParamName sourceAlias],SqlColumnGet(sourceTi,sourceKey,_))) OptionalQuote (Lambda([ParamName destAlias],SqlColumnGet(destTi,destKey,_))) OptionalQuote (Lambda(projectionParams,_))]) -> | OptionalOuterJoin(outerJoin, MethodCall(None,(MethodWithName "_CreateRelated"), [param; _; String PE; String PK; String FE; String FK; String IE; RelDirection dir;])) -> | MethodCall(Some(ParamName name | PropertyGet(_,PropName name)), (MethodWithName "GetColumn"),[String key]) ->
  • 11. Projection Analysis and transformation steps are applied to the LINQ projection expression; 1) To extract fields that have been explicitly been selected so they can be used to construct a sensible query where possible 2) Entity access nodes are replaced with calls to GetSubEntity() which performs the magic allowing it to seem as if using an ORM, when in reality each row’s data is stored in one master SqlEntity SqlEntity A SqlEntity C GetSubEntity(a) DataRow (SqlEntity) SqlEntity B GetSubEntity(b) GetSubEntity(c) a.Name b.Addr1 b.Addr2 a.Title c.Id c.Amount a.Phone b.Addr2
  • 12. Extensibility via ADO.NET All you need to do is implement most of this interface! WHAT COULD BE SIMPLER!  type internal ISqlProvider = abstract CreateConnection : string -> IDbConnection abstract CreateCommand : IDbConnection * string -> IDbCommand abstract CreateCommandParameter : string * obj * SqlDbType option -> IDataParameter abstract CreateTypeMappings : IDbConnection -> Unit abstract ClrToEnum : (string -> SqlDbType option) with get abstract SqlToEnum : (string -> SqlDbType option) with get abstract SqlToClr : (string -> Type option) with get abstract GetTables : IDbConnection -> Table list abstract GetColumns : IDbConnection * Table -> Column list abstract GetPrimaryKey : Table -> string option abstract GetRelationships : IDbConnection * Table -> (Relationship list * Relationship list) abstract GetSprocs : IDbConnection -> Sproc list abstract GenerateQueryText : SqlQuery * string * Table * Dictionary<string,ResizeArray<string>> -> string * ResizeArray<IDataParameter> ^ Of course, this last one is the kicker… but thankfully you can largely just copy and paste from an existing provider 