SlideShare a Scribd company logo
Advanced Data Access
with Dapper
@Dave_Paquette
Microsoft MVP (ASP.NET/IIS)
contactme@davepaquette.com
http://www.davepaquette.com
Advanced .NET Data Access with Dapper
Data Access
How did we get here?
Why is this so hard?
public class Foo
{
//…some properties
}
SQL Server
Data Access in .NET
• In the beginning there was System.Data
• IDbConnection
• IDbCommand
• IDataReader
• IDbTransaction
Sytem.Data - Reading Records
var results = new List<Employee>();
using (SqlConnection connection = new SqlConnection(Settings.ConnectionString))
{
SqlCommand command = new SqlCommand(“SELECT * FROM Employees”, connection);
connection.Open();
IDataReader reader = command.ExecuteReader();
while (reader.Read())
{
results.Add(ReadSingleRow(reader));
}
reader.Close();
}
return results;
The ORMs will save us
• Hide details of database
• Generate SQL based on object model and configuration
• ChangeTracking
• Complex mapping strategies
• Many-to-many relationships
• Inheritance
Popular ORMs
•Entity Framework
•nHibernate?
•Linq2Sql?
ORM Pain Points
• Black box code generation –What is going on?
• Performance Problems
• Eager Loading vs Lazy Loading
• Complex Inheritance Chains
• Dealing with disconnected entities (in a web context)
The Micro-ORMs will save us!
• Query --> Objects
• Usually focused on speed
• Light on features
Popular Micro-ORMs
• Dapper
• PetaPOCO
• Massive
• Simple.Data
Dapper
• Used in production at Stack Exchange
• Super fast and easy to use
Performance of SELECT mapping over 500
iterations - POCO serialization
https://github.com/StackExchange/Dapper#performance
ORM Method Mean Allocated
LINQ to DB 'First (Compiled)' 78.75 us 2.66 KB
LINQ to DB Query<T> 80.38 us 6.87 KB
Hand Coded SqlCommand 87.16 us 12.24 KB
Dapper QueryFirstOrDefault<dynamic> 87.80 us 13.5 KB
Dapper QueryFirstOrDefault<T> 91.51 us 13.46 KB
Massive 'Query (dynamic)' 96.18 us 14.19 KB
PetaPoco 'Fetch<T> (Fast)' 96.57 us 13.65 KB
EF 6 SqlQuery 143.86 us 27.86 KB
EF Core 'First (Compiled)' 148.42 us 16.08 KB
NHibernate Get<T> 196.88 us 32.5 KB
EF Core First 197.91 us 20.25 KB
NHibernate HQL 207.84 us 35 KB
EF Core 'First (NoTracking)' 213.58 us 21.36 KB
How can it be that fast?
• Dapper dynamically writes code for you
• Emits IL code for tasks like loading a data
reader into an object
• https://github.com/StackExchange/Dapper/blo
b/master/Dapper/SqlMapper.cs#L3078
Dapper works on Database Connections
A set of extension methods on IDbConnection
Aircraft aircraft;
using (var connection = new SqlConnection(_connectionString))
{
await connection.OpenAsync(); //Optional
var query = "SELECT * FROM Aircraft WHERE Id = @Id";
Aircraft aircraft = await connection.QuerySingleAsync<Aircraft>(query, new {Id = id});
}
Querying Simple Objects
Aircraft aircraft = await connection.QuerySingleAsync<Aircraft>(query, new {Id = id});
IEnumerable<Aircraft> aircraft = await connection.QueryAsync<Aircraft>(query);
Loading Related Objects – Multi-Mapping
• Write a single query that returns all the data in a single row
scheduledFlights =
await connection.QueryAsync<ScheduledFlight, Airport, ScheduledFlight>(query,
(flight, airport) => {
flight.Airport = airport;
return flight;
},
new{FromCode = from} );
Multi-Mapping Caveats
• Data duplication
• Query returns a bloated data set
• Multiple instances of an object that represent the same thing
• This is totally fine forOne-to-One relationships
• No duplication here
Loading Related Objects – Multiple Queries
• Get data using multiple queries and wire them up yourself
• Still executed in a single command that returns multiple results sets
using (var multi = await connection.QueryMultipleAsync(query, new{FromCode = from} ))
{
scheduledFlights = multi.Read<ScheduledFlight>();
var airports = multi.Read<Airport>().ToDictionary(a => a.Id);
foreach(var flight in scheduledFlights)
{
flight.Airport = airports[flight.AirportId];
}
}
Multi-Mapping vs Multiple Queries
100 ScheduledFlight Records 1,000 ScheduledFlight Records
Method Mean
MultiMapping 926.5 us
MultipleResultSets 705.9 us
Method Mean
MultiMapping 5.098 ms
MultipleResultSets 2.809 ms
https://www.davepaquette.com/archive/2018/04/10/loading-related-entities-many-to-one-part-2.aspx
AirportSchedule
Date
Airport
Departures
Arrivals
Airport
Id
Code
City
Province
Country
Flight
Id
ScheduledFlight
ScheduledDeparture
ActualDeparture
ScheduledArrival
ActualArrival
ScheduledFlight
Id
FlightNumber
DepartureAirport
ArrivalAirport
…
1
1
1
**
1
1
1
1
1
* *
Loading More Complex Object Graphs
Loading More Complex Object Graphs
Paging through large collections
• Use an OFFSET FETCH query
• Include a total count in the result
SELECT * FROM Flight f
INNER JOIN ScheduledFlight sf
ON f.ScheduledFlightId = sf.Id
ORDER BY Day, FlightNumber
OFFSET 0 ROWS
FETCH NEXT 10 ROWS ONLY
SELECT COUNT(*) FROM Flight f
INNER JOIN ScheduledFlight sf
ON f.ScheduledFlightId = sf.Id
Insert / Update / Delete
• Use the ExecuteAsync method
• Built-in support for batch inserts
await connection.ExecuteAsync(
@"INSERT INTO Flight(ScheduledFlightId, Day, ScheduledDeparture,
ScheduledArrival)
VALUES(@ScheduledFlightId, @Day, @ScheduledDeparture, @ScheduledArrival)",
flights);
Buffering
•Working with really large result sets?
• Consider setting buffering=false for queries
Dapper Contrib
• Generates SQL for simple CRUD operations
Questions?
Check out my blog posts
http://bit.ly/dapperseries
www.davepaquette.com
westerndevs.com
aspnetmonsters.com
Blog(s)
Twitter @Dave_Paquette
Email contactme@davepaquette.com
Thanks!

More Related Content

PPTX
Advanced data access with Dapper
PPTX
Dapper performance
PPTX
PPTX
Angular JS deep dive
PDF
Dapper & Dapper.SimpleCRUD
PDF
Log analytics with ELK stack
PDF
Building data flows with Celery and SQLAlchemy
PDF
Time Series Data with InfluxDB
Advanced data access with Dapper
Dapper performance
Angular JS deep dive
Dapper & Dapper.SimpleCRUD
Log analytics with ELK stack
Building data flows with Celery and SQLAlchemy
Time Series Data with InfluxDB

What's hot (20)

PDF
Faceting Optimizations for Solr: Presented by Toke Eskildsen, State & Univers...
PDF
Easy, scalable, fault tolerant stream processing with structured streaming - ...
PDF
Influxdb and time series data
PDF
Spark Summit EU talk by Ted Malaska
PPT
SQL on Big Data using Optiq
PPTX
ELK Elasticsearch Logstash and Kibana Stack for Log Management
PDF
Ali Asad Lotia (DevOps at Beamly) - Riemann Stream Processing at #DOXLON
PDF
From Overnight to Always On @ Jfokus 2019
PPTX
To scale or not to scale: Key/Value, Document, SQL, JPA – What’s right for my...
PPTX
Elastic Stack Introduction
PDF
Introduction to Mongodb execution plan and optimizer
PPTX
Orms vs Micro-ORMs
PDF
Making Structured Streaming Ready for Production
ODP
PDF
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLON
PDF
ElasticES-Hadoop: Bridging the world of Hadoop and Elasticsearch
PPTX
DZone Java 8 Block Buster: Query Databases Using Streams
PDF
Automating Workflows for Analytics Pipelines
PDF
Thunderstruck
PDF
SQL on everything, in memory
Faceting Optimizations for Solr: Presented by Toke Eskildsen, State & Univers...
Easy, scalable, fault tolerant stream processing with structured streaming - ...
Influxdb and time series data
Spark Summit EU talk by Ted Malaska
SQL on Big Data using Optiq
ELK Elasticsearch Logstash and Kibana Stack for Log Management
Ali Asad Lotia (DevOps at Beamly) - Riemann Stream Processing at #DOXLON
From Overnight to Always On @ Jfokus 2019
To scale or not to scale: Key/Value, Document, SQL, JPA – What’s right for my...
Elastic Stack Introduction
Introduction to Mongodb execution plan and optimizer
Orms vs Micro-ORMs
Making Structured Streaming Ready for Production
Paul Dix (Founder InfluxDB) - Organising Metrics at #DOXLON
ElasticES-Hadoop: Bridging the world of Hadoop and Elasticsearch
DZone Java 8 Block Buster: Query Databases Using Streams
Automating Workflows for Analytics Pipelines
Thunderstruck
SQL on everything, in memory
Ad

Similar to Advanced .NET Data Access with Dapper (20)

PPT
JDBC Connecticity.ppt
PDF
Play framework productivity formula
PPTX
Spark etl
PDF
UEMB200: Next Generation of Endpoint Management Architecture and Discovery Se...
PPT
nodejs_at_a_glance.ppt
PDF
Hw09 Sqoop Database Import For Hadoop
PPT
nodejs_at_a_glance, understanding java script
PDF
NoSQL meets Microservices - Michael Hackstein
PPTX
Keeping Spark on Track: Productionizing Spark for ETL
PPT
Zend Con 2008 Slides
PDF
Complex Made Simple: Sleep Better with TorqueBox
PPTX
Sherlock Homepage - A detective story about running large web services (VISUG...
PPTX
Sherlock Homepage (Maarten Balliauw)
KEY
Writing robust Node.js applications
PDF
Breaking Parser Logic: Take Your Path Normalization Off and Pop 0days Out!
PPTX
Sherlock Homepage - A detective story about running large web services - NDC ...
ODP
Web program-peformance-optimization
PPT
OWB11gR2 - Extending ETL
PDF
Nodejs - A quick tour (v6)
PPTX
Eagle from eBay at China Hadoop Summit 2015
JDBC Connecticity.ppt
Play framework productivity formula
Spark etl
UEMB200: Next Generation of Endpoint Management Architecture and Discovery Se...
nodejs_at_a_glance.ppt
Hw09 Sqoop Database Import For Hadoop
nodejs_at_a_glance, understanding java script
NoSQL meets Microservices - Michael Hackstein
Keeping Spark on Track: Productionizing Spark for ETL
Zend Con 2008 Slides
Complex Made Simple: Sleep Better with TorqueBox
Sherlock Homepage - A detective story about running large web services (VISUG...
Sherlock Homepage (Maarten Balliauw)
Writing robust Node.js applications
Breaking Parser Logic: Take Your Path Normalization Off and Pop 0days Out!
Sherlock Homepage - A detective story about running large web services - NDC ...
Web program-peformance-optimization
OWB11gR2 - Extending ETL
Nodejs - A quick tour (v6)
Eagle from eBay at China Hadoop Summit 2015
Ad

Recently uploaded (20)

PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PPTX
Programs and apps: productivity, graphics, security and other tools
PPT
Teaching material agriculture food technology
PPTX
Cloud computing and distributed systems.
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PPTX
Spectroscopy.pptx food analysis technology
PDF
Spectral efficient network and resource selection model in 5G networks
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
Encapsulation theory and applications.pdf
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
20250228 LYD VKU AI Blended-Learning.pptx
Programs and apps: productivity, graphics, security and other tools
Teaching material agriculture food technology
Cloud computing and distributed systems.
Understanding_Digital_Forensics_Presentation.pptx
NewMind AI Weekly Chronicles - August'25 Week I
Per capita expenditure prediction using model stacking based on satellite ima...
Spectroscopy.pptx food analysis technology
Spectral efficient network and resource selection model in 5G networks
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
MIND Revenue Release Quarter 2 2025 Press Release
Encapsulation theory and applications.pdf
Encapsulation_ Review paper, used for researhc scholars
Mobile App Security Testing_ A Comprehensive Guide.pdf
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Building Integrated photovoltaic BIPV_UPV.pdf
The Rise and Fall of 3GPP – Time for a Sabbatical?
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton

Advanced .NET Data Access with Dapper

  • 1. Advanced Data Access with Dapper @Dave_Paquette Microsoft MVP (ASP.NET/IIS) contactme@davepaquette.com http://www.davepaquette.com
  • 3. Data Access How did we get here? Why is this so hard? public class Foo { //…some properties } SQL Server
  • 4. Data Access in .NET • In the beginning there was System.Data • IDbConnection • IDbCommand • IDataReader • IDbTransaction
  • 5. Sytem.Data - Reading Records var results = new List<Employee>(); using (SqlConnection connection = new SqlConnection(Settings.ConnectionString)) { SqlCommand command = new SqlCommand(“SELECT * FROM Employees”, connection); connection.Open(); IDataReader reader = command.ExecuteReader(); while (reader.Read()) { results.Add(ReadSingleRow(reader)); } reader.Close(); } return results;
  • 6. The ORMs will save us • Hide details of database • Generate SQL based on object model and configuration • ChangeTracking • Complex mapping strategies • Many-to-many relationships • Inheritance
  • 8. ORM Pain Points • Black box code generation –What is going on? • Performance Problems • Eager Loading vs Lazy Loading • Complex Inheritance Chains • Dealing with disconnected entities (in a web context)
  • 9. The Micro-ORMs will save us! • Query --> Objects • Usually focused on speed • Light on features
  • 10. Popular Micro-ORMs • Dapper • PetaPOCO • Massive • Simple.Data
  • 11. Dapper • Used in production at Stack Exchange • Super fast and easy to use
  • 12. Performance of SELECT mapping over 500 iterations - POCO serialization https://github.com/StackExchange/Dapper#performance ORM Method Mean Allocated LINQ to DB 'First (Compiled)' 78.75 us 2.66 KB LINQ to DB Query<T> 80.38 us 6.87 KB Hand Coded SqlCommand 87.16 us 12.24 KB Dapper QueryFirstOrDefault<dynamic> 87.80 us 13.5 KB Dapper QueryFirstOrDefault<T> 91.51 us 13.46 KB Massive 'Query (dynamic)' 96.18 us 14.19 KB PetaPoco 'Fetch<T> (Fast)' 96.57 us 13.65 KB EF 6 SqlQuery 143.86 us 27.86 KB EF Core 'First (Compiled)' 148.42 us 16.08 KB NHibernate Get<T> 196.88 us 32.5 KB EF Core First 197.91 us 20.25 KB NHibernate HQL 207.84 us 35 KB EF Core 'First (NoTracking)' 213.58 us 21.36 KB
  • 13. How can it be that fast? • Dapper dynamically writes code for you • Emits IL code for tasks like loading a data reader into an object • https://github.com/StackExchange/Dapper/blo b/master/Dapper/SqlMapper.cs#L3078
  • 14. Dapper works on Database Connections A set of extension methods on IDbConnection Aircraft aircraft; using (var connection = new SqlConnection(_connectionString)) { await connection.OpenAsync(); //Optional var query = "SELECT * FROM Aircraft WHERE Id = @Id"; Aircraft aircraft = await connection.QuerySingleAsync<Aircraft>(query, new {Id = id}); }
  • 15. Querying Simple Objects Aircraft aircraft = await connection.QuerySingleAsync<Aircraft>(query, new {Id = id}); IEnumerable<Aircraft> aircraft = await connection.QueryAsync<Aircraft>(query);
  • 16. Loading Related Objects – Multi-Mapping • Write a single query that returns all the data in a single row scheduledFlights = await connection.QueryAsync<ScheduledFlight, Airport, ScheduledFlight>(query, (flight, airport) => { flight.Airport = airport; return flight; }, new{FromCode = from} );
  • 17. Multi-Mapping Caveats • Data duplication • Query returns a bloated data set • Multiple instances of an object that represent the same thing • This is totally fine forOne-to-One relationships • No duplication here
  • 18. Loading Related Objects – Multiple Queries • Get data using multiple queries and wire them up yourself • Still executed in a single command that returns multiple results sets using (var multi = await connection.QueryMultipleAsync(query, new{FromCode = from} )) { scheduledFlights = multi.Read<ScheduledFlight>(); var airports = multi.Read<Airport>().ToDictionary(a => a.Id); foreach(var flight in scheduledFlights) { flight.Airport = airports[flight.AirportId]; } }
  • 19. Multi-Mapping vs Multiple Queries 100 ScheduledFlight Records 1,000 ScheduledFlight Records Method Mean MultiMapping 926.5 us MultipleResultSets 705.9 us Method Mean MultiMapping 5.098 ms MultipleResultSets 2.809 ms https://www.davepaquette.com/archive/2018/04/10/loading-related-entities-many-to-one-part-2.aspx
  • 21. Loading More Complex Object Graphs
  • 22. Paging through large collections • Use an OFFSET FETCH query • Include a total count in the result SELECT * FROM Flight f INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id ORDER BY Day, FlightNumber OFFSET 0 ROWS FETCH NEXT 10 ROWS ONLY SELECT COUNT(*) FROM Flight f INNER JOIN ScheduledFlight sf ON f.ScheduledFlightId = sf.Id
  • 23. Insert / Update / Delete • Use the ExecuteAsync method • Built-in support for batch inserts await connection.ExecuteAsync( @"INSERT INTO Flight(ScheduledFlightId, Day, ScheduledDeparture, ScheduledArrival) VALUES(@ScheduledFlightId, @Day, @ScheduledDeparture, @ScheduledArrival)", flights);
  • 24. Buffering •Working with really large result sets? • Consider setting buffering=false for queries
  • 25. Dapper Contrib • Generates SQL for simple CRUD operations
  • 26. Questions? Check out my blog posts http://bit.ly/dapperseries

Editor's Notes

  • #4: Data access has consumed such a huge amount of our time. - Why is that? - What is so hard about this?
  • #5: These 4 interfaces were in the original .NET framework. They have been the basis for ALL database access since then. Each database provider has an implementation of each of these interfaces (eg. SqlConnection) Every ORM or Micro-ORM written in .NET is based on these 4 interfaces.
  • #6: Show sample code for reading and writing data using the raw SQL
  • #7: Swinging the pendulum
  • #21: Demo Start with loading simply the first level (Airport + Arrival and Departure Flights) Using multi-mapping to load the ScheduledFlight info Use multi queries to also load each ScheduledFlight’s Arrival and Departure airports