Building SSRS 2008 large scale solutions

Denny Lee, Lukasz Pawlowski
SQL Customer Advisory Team
SQL Server Reporting Services

Building SSRS 2008 Large
Scale Solutions PASS Community Summit 2008
November 18 – 21, 2008 Seattle WA

SQL Server Customer Advisory Team
(SQLCAT)
 Works on the largest, most complex SQL Server projects worldwide
– US: NASDAQ, Progressive, Premier Bankcard, Hilton Hotels
– Europe: Barclays Capital, Danske Bank, McLaren, Bwin
– Asia/Pacific: Korea Telecom, GMarket, Japan Railways East, China
Mobile
– LATAM: Banco Itau, Oi
– Strategic ISVs: SAP, Siebel, JDE, PeopleSoft, GE Healthcare, SunGard,
Siemens, Dynamics and more
 Drives product requirements back into SQL Server from our customers
and ISVs
 Shares deep technical content with SQL Server community
– SQLCAT.com
– http://blogs.msdn.com/sqlcat
– http://blogs.msdn.com/mssqlisv
– http://technet.microsoft.com/en-us/sqlserver/bb331794.aspx

PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 2

SQL Server Design Win Program

 Target the Most Challenging and Innovative
Applications on SQL Server
 Investing in Large Scale, Referenceable SQL Server
Projects Across the World
– Provide SQLCAT technical & project experience
– Conduct architecture and design reviews covering performance,
operation, scalability and availability aspects
– Offer use of HW lab in Redmond with direct access to SQL
Server development team
 Work with Marketing Team Developing Case Studies


Session Objectives and Takeaways

 Session Objective(s):
– Provide guidance on how to scale out your Reporting Services
environment
– Provide RS best practices on RS catalogs, scale out deployment,
and performance optimizations

 Agenda:
– Reporting Services Scale Out Architecture
– Report Catalog Best Practices
– Scale Out Deployment Best Practices
– Performance Optimization Configurations


Reporting Services Scale Out
Architecture


Scale Out Architecture:
Overall Architecture
Report Server

RS Scale Out Deployment
Clients
RS Server

Report Catalog

Reporting Data
Flat Files,
OLE DB,
ODBC

NLB
Clients
RS Server RSDB

SQL, AS,
DB2, Oracle,
RS Server Teradata, etc.

Clients


Scale Out Architecture
Read the manuals!

 A lot of documentation on SSRS available online
 Many mistakes in implementation could have been avoided
 Read these:
– Planning for Scalability and Performance with Reporting Services
– Upgrading Reporting Services (SQL Books Online)
– Configuring a Report Server Scale-Out Deployment
 On sqlcat.com
– Building and Deploying Large Scale SQL Server Reporting Services
Environments Series


Enterprise Rent-A-Car Customer Scenario

Report Server

RS Server

AS Server

Report Catalog

Reporting Data
RS Server

Teradata
RSDB

RS Server

AS Server

RS Server


Enterprise Rent-A-Car Customer Scenario

 Build test system that accurately represented production
– Goal: 1800 concurrent users
 using VS test
 10s think time
 Mean 33-36s txn time
– Testing allowed them to identify blocking issues
 drop down parameter lists of thousands of rows for areas and branches
 Developed accurate workload representation (e.g. Proclarity and SSRS
clients)
 Currently in production
 This presentation incorporates lessons learned from this
and other customers


Importance of Performance Testing

 Need to understand your scenarios and reports
– Scenarios are defined by user personas & usage patterns
– Reports are either test reports or actual reports
– Tests should isolate Report Server from other systems
 Need tools to automate the testing
– See white paper: Using Visual Studio 2005 to Perform Load Testing
on a SQL Server 2005 Reporting Services Report Server
– Make single incremental changes between tests
– Do not use SQL trace inside VSTE


Customer Performance Testing

RS User Mean Tx Mean
Servers s Time CPU%
Max # of Concurrent Users
1 Server 608 36.9 99
2500
2 servers 1218 36.8 96
2300
2000 4 servers 2300 30.5 80

1500

 8GB RAM, 2 dual core RS
1218
1000
servers, Windows 2003
500  Graph is max # of users
608
reached for sustained time
0 period (>=15 min)
1 server 2 servers 4 servers
 2x RAM and CPU core, only
Users
1/3 increase in load


RS 2008 vs. RS 2005: Lessons Learned

 RS 2008 Front-End Server Scales Up Much Better than RS
2005
– Able to respond to 3–4 times the total number of users and their
requests without errors on the same hardware for all renderers
– RS 2008 consistently outperformed RS 2005 with the PDF and XLS
renderers on the four-processor, quad-core hardware platform
 See: Scaling Up RS 2008 vs. RS 2005: Lessoned Learned


RS 2008 vs. RS 2005: Lessoned Learned

Avg. Response Time (lower is better)
250

4x4 2008 Mix
4x4 2005 Mix
200
4x2 2008 Mix
4x2 2005 Mix
2x2 2008 Mix
150
Avg. Response
Time (ms) 2x2 2005 Mix

100

50

0

User Load


Scaling Up and Scaling Out with RS 2008 (cont)

 RS 2008
– Scale up front-end server to four-processor, quad-core servers for
performance
– Scale out to a two-node deployment for high availability
– Optimize disk I/O subsystem on all RS 2008 boxes for maximum
performance


Reporting Catalog Best
Practices


Report Catalog Best Practices
Report Server

Clients
RS Server

Report Catalog

Reporting Data
Flat Files,
OLE DB,
ODBC

NLB
Clients
RS Server RSDB

SQL, AS,
DB2, Oracle,

Clients


Report Server Catalog Breakdown

Report Server Catalog (RSDB)
Stores all report metadata including report
Report Catalog

definitions, report / history snapshots,
scheduling, etc.

RSDB RS Temp DB
Stores temporary snapshots while running
reports

 These databases can be a bottleneck
 Optimize by applying standard SQL DB techniques
 Catalog has a lot of I/O and transactions
– RS2005: Many inserts to ChunkData, SnapshotData, and SessionData tables
– RS2008: Many inserts Segment; takes majority of transactions of RSTempDB


Use a dedicated server
Common scenarios
 Same server as SSRS Server
– Great for small environments
RSDB – In enterprise environments, too much resource contention
 Same server as data source database
– SQL resource contention (TempDB, plan cache, memory
buffer pool) between data source and RS catalogs
– As load increases need to monitor CPU, I/O, network
resources, and buffer pool
 Reduce resource contention by having a dedicated RS
catalog server you can tune.
 Apply standard high availability and disaster recovery
(e.g. clustering, mirroring, log shipping) to protect the
RSDB

High Performance Disk

 Check out Predeployment I/O Best Practices
 Have more smaller size disks with faster rotation speeds
(>=15k RPM) vs. fewer larger disks with slower rotations
RSDB  Maximize/balance I/O across ALL available spindles
 Separate disks between RSDB and RSTempDB
– RSDB a lot of small transactions (report metadata)
– RSTempDB has more (not as many) larger transactions
 Pre-grow your databases
 Stripe dB files to number of cores (0.25 – 1.0)
– Minimize allocation contention
– Easier to rebalance database when new LUNs are available
 Use RAID 10, not RAID 5


Operations Best Practices

 Data in RSTempDB is highly volatile
– Report lifetime policy of data = SessionTimeout value (10min)
– CleanupCycleMinutes guides background cleanup thread
RSDB – Once session timeout reached, cleanup temporary snapshot from tempDB
– This is done every CleanupCycleMinutes
 Data is RSDB is long lived; should be backed up
– Backing Up and Restore Databases in SQL Server
– Optimizing Backup and Restore Performance in SQL Server
– Backing Up and Restore Encryption Keys
 Maintain your RS catalogs
– Remember, these are SQL databses
– E.g. Re-indexing catalog tables or updating stats may improve query
performance


Report Catalog Sizing
 RSDB database size
– Varies by number of reports published and number of history snapshots
– General rule of thumb:
 Moderate size report definition takes 100-200KB of disk space
RSDB  This is larger than the actual RDL
 SSRS persists both RDL and compiled binary
 Assume 5:1 compression ratio (e.g. 10MB of data, snapshot is 2MB in size)

 RSTempDB database size
– Varies by number of users whom are concurrently using the Report Servers
– Each live report execution generates report snapshot persisted in the
RSTempDB
– General rule of thumb:
 10-20% concurrency of user base
 E.g. 1000 users, then max 200 concurrent users.
 If most users are accessing 10MB reports, then you will need 400MB of storage
– 200 users x 10MB reports / 5:1 compression ratio= 400MB
 Want to calculate for the maximum number of concurrent users


Disaster Recovery

Primary Data Center Content Switch

Automatic Failover
SSRS SSRS

Manual Failover
Failover Cluster

RSDB Async RSDB

RSDB Mirroring

Scale Out Deployment Best
Practices


Scale Out Deployment Best Practices
Report Server

Clients
RS Server

Report Catalog

Reporting Data
Flat Files,
OLE DB,
ODBC

NLB
Clients
RS Server RSDB

SQL, AS,
DB2, Oracle,

Clients


RS 2005: File System Snapshots

 RS TempDB has a lot of transactions to keep
report consistency (i.e. cached reports)
 Reduce RS Catalog I/O with File System
RS Server Snapshots
– It will store data on file system
– Unlike RS/IIS setup, will require more disk space
RS Server  To enable, update RSReportServer.config file:
– <Add Key="WebServiceUseFileShareStorage"
Value="true" />
RS Server – <WindowsServiceUseFileShareStorage>True</Wi
ndowsServiceUseFileShareStorage>


RS 2008: Why not File System Snapshots?

 SSRS 2005
– Advantage for SSRS 2005 because enabling feature
allowed less hits to RSTempDB
RS Server
– Entire report was calculated when requesting first page
 SSRS 2008 caches a lot of this data into memory
– Data continually persisted in report catalogs
RS Server
– Local file system acts as a write-through cache
– Does not pre-calculate everything on initial request
– On-demand engine retrieves all of the data and places into
RSTempDB for consistency
RS Server
– But many calculations are done on-demand as needed vs.
pre-calculated and stored.
 Still want to test in your environment

Cache Execution

 Recurring theme on effective user of memory and
minimal I/O
 To help reduce I/O further, enable cache execution
RS Server on your reports.
 By default, reports are live execution
 Turn on cache execution for each report so the
RS Server report is stored in memory (thus reduced disk I/O)
 E.g. Even if you cache report every 5 minutes,
potentially a 80% reduction in I/O
RS Server
– If report is hit every minute, now only I/O hit every 5
minutes, i.e. 20% of the time
 No global setting for cache execution

Load Balance your Network

 Load balancing important for many
client connections to RS servers

Clients  Recommend: Use cookie
RS Server persistence to preserve SSRS-to-
client connection
– IP affinity can work but may be
NLB overload for browser-based
Clients connections
RS Server
– Makes use of SSRS file cache
– Keep round-robin for initial
connections
RS Server  Recommend: dual NIC for RS
Clients – Split browser and AS/DB traffic


Isolate your workloads
Report Server
Interaction

RS Scale Out
Deployment
NLB
Clients
RS Server

Report Catalog

Reporting Data
Scheduling
Benefits:
RSDB
Predictable Workloads
RS Scale Out
Deployment
Helps with Security Model
Isolate Performance Issues

RS Server


Report Data Performance Considerations

 Scale out works for RS but may not work for
underlying Report Data (data source)
 Reporting loads Report Data, limit impact of
large numbers of users
Reporting Data

Flat Files,
OLE DB, – Limit data set size using report filters
ODBC
– SSIS limited data from Operational data sources
– Do not let all users access all of the reports
– E.g. Report Builder against Analysis Services results in
many queries being executed.
SQL, AS,
DB2, Oracle,
Teradata, etc.


Report Data Performance Considerations

Additional resources to scale out SQL and SSAS
 Deploying a Scalable Shared Database
 SQL Server Replication: Providing High
Availability using Database Mirroring
Reporting Data

Flat Files,
OLE DB,
ODBC  Database Mirroring and Log Shipping
 SQL Server Replication Features
 Scale-Out Querying with Analysis Services
SQL, AS,  Scale-Out Querying with Analysis Services Using
DB2, Oracle,
Teradata, etc. SAN Snapshots
 Scaling out an Analysis Services Solution


Performance Optimization
Configurations


Handling Large Workloads

 Control Size of reports
– Do you need them?
– Is this really a data feed?
RS Server
– Aggregate reports and remove unused columns
 Recommendations
– Cache Execution
– Report Execution Timeouts
– Scheduled snapshots for large reports with data processing
bottlenecks
– Delivered Rendered reports for non-browser formats
– Pre-populate report cache using data driven subscriptions


Large Workload Tuning

 Analyze your reports
– Use ExecutionLog2 View
 Back to Report Catalogs
RS Server
– Increase size of your report catalogs to store more snapshot
data
 Tune the web service
– SSRS 2005: Tune IIS
– SSRS 2008: Tune HTTP.sys
 Windows 2003
 Windows 2008


ExecutionLog2 Analysis Checklist

 Sort by ElapsedSec or RowCount  Sort by Instance
for long running reports – Determine if NLB is handling request in
– TimeDataRetrieval: If high, need to balanced fashion
optimize data source  Sort by Report Path & Timestart
– High RowCount: A lot of data to determine report pattern
aggregated by SSRS, have SQL do
this – E.g. Expensive report (takes 5 minutes
to run) running every 10 minutes
 Sort by Request Type
 Sort by Status
– A lot of subscriptions, can determine
bottlenecks and stagger reports – Failures occur before (e.g. incorrect
RDL) or after (e.g. subscription delivery
 Sort by source error) report is processed
– To determine if live data or snapshot – Outdated information or settings (e.g.
– If report can be snapshot (e.g. last expired passwords, missing
week’s report), create snapshot to subreports, etc.)
avoid query execution, report  Data Driven Subscriptions
processing, and rendering
– Errors > 5%
– Continual Scale Mode

Monitoring Reporting Services by the
ExecutionLog2 View

Should we use 64-bit?

Yes!
 Report Server Catalog
– Standard Database techniques for optimization
RS Server
– Since SQL 2005, database written natively for 64-bit
 Report Server Service
– Most reports memory intensive
– Note, some workloads (e.g. many small reports) 32-bit can
execute faster
– Handle more concurrent report users or more large reports
– Able to more effectively use memory in SSRS 2008
– Will spill to file system if hits memory pressure
– Exceptions:
 Certain data provides not available for 64-bit

SSRS 2008 Memory Configurations

 Uses memory more efficiently; under intensive workload pressure, it
uses the file system cache. E.g., small requests will continue to stay
in memory while long running request will go to disk
 Therefore, before looking at the file system, check these memory
RS Server
configurations:
– WorkingSetMinimum / WorkingSetMaximum:
 Minimum / Maximum amount of physical memory that RS will make available to
perform its task;
 KB value within RSReportServer.config
 Increase value to process more requests in memory
 After WorkingSetMaximum is reached and exceeded for a period of time, recycle
app domains to reduce memory utlization
– MemorySafetyMargin:
 Defines boundary between low/medium pressure scenarios
 Default 80% value in RSReportServer.config
– MemoryThreshold:
 Defines boundary between medium/high pressure scenarios
 Default 90% value in RSReportServer.config


SSRS 2005 Memory Configurations

Recall, SSRS 2005 does not scale up as well as SSRS 2008
 MemoryLimit Configuration
• Default 60% of physical memory
RS Server
• Increase help process more requests
• Once threshold hit, no new requests are accepted
 MaximumMemoryLimit Configuration
• Default 80% of physical memory
• If this threshold is met, processing is aborted
 Changing values may solve RS only to bring up other contentions
 Recommendation: If constantly hitting memory thresholds, consider
scaling up and then scaling out


Thank you
for attending this session and the
PASS Community Summit 2008
PASS Community Summit 2008
November 18 – 21, 2008 Seattle WA

Building SSRS 2008 large scale solutions

More Related Content

What's hot (19)

Viewers also liked (8)

Similar to Building SSRS 2008 large scale solutions (20)

More from Denny Lee (20)

Building SSRS 2008 large scale solutions

Editor's Notes