Accelerate SQL Server Migration to the AWS Cloud
Download as PPTX, PDF0 likes176 views
The document outlines the approach and methodology for accelerating SQL Server migration to AWS Cloud, providing insights into cloud services like IaaS, PaaS, and SaaS. It details the migration process, comparing Amazon EC2 and RDS for SQL Server, along with considerations for high availability, disaster recovery, and cost management. The document emphasizes the DBA's evolving role in cloud environments and includes various case studies and recommendations for best practices in migration and ongoing database management.
Accelerate SQL Server Migration to the AWS Cloud
- 1. Accelerate SQL Server Migration to the AWS Cloud Approach & Methodology Jeff Duncan, DBA Manager, SQL Server, Datavail
- 2. Everything PASS Has To Offer Unlock exclusive training & networking Free 1-day local training events Local user groups around the world Back-to-back live webinar events Online special interest user groups Get involved Free Resources Online PASS.org Explore
- 3. Submit before Friday, Nov. 20 at 5pm EST to win prizes Your feedback is important to us! Please visit the “Session Evaluation” link in the left hand side navigation bar to rate this session. Session Evaluation
- 4. Jeff Duncan DBA Manager, SQL Server, Datavail /jeffreyaaronduncan/ Jeff.Duncan@datavail.com SQL Server Team Manager with 20+ years’ experience Expertise with database operations, cloud migrations, database development, report writing & project management Responsible for a team of 14 DBAs that deliver complex custom solutions for Datavail’s customers
- 5. Accelerate SQL Server Migration to the AWS Cloud Approach & Methodology Jeff Duncan, DBA Manager, SQL Server, Datavail
- 6. About Datavail Databases • Windows Workloads • Open Source Workloads • Oracle Workloads 100+ Cloud SA’s and Engineers 16+ Years Database Services 700+ Customers 8+ Years Cloud Experience 200,000+ Databases Managed 100+ Cloud Migrations Expertise Experience Outcome Data Integration & Analytics
- 7. Agenda • Cloud: What, Who & Why • What’s the Difference Between IaaS, SaaS & PaaS? • SQL Server in the Cloud • SQL Server High Availability & Disaster Recovery • DBA Role in the Cloud: Now What?
- 8. Who is the Cloud? AWS | Microsoft Azure | Google Cloud
- 9. Why Go to the Cloud? • Business Strategy • Reduced Expense • Security • Data Ownership & Access • Agility
- 10. On-Prem v. IaaS, PaaS and SaaS On-Premise Servers Storage Networking Virtualization OS Middleware Runtime Apps Data IaaS Servers Storage Networking Virtualization OS Middleware Runtime Apps Data PaaS Servers Storage Networking Virtualization OS Middleware Runtime Apps Data SaaS Servers Storage Networking Virtualization OS Middleware Runtime Apps Data • Infrastructure as a Service • Platform as a Service • Software as a Service
- 11. Migration Methodology • Involve stakeholders • Calculate your TCO (Total Cost of Ownership) • Discover and evaluate apps Assess • Select a migration strategy • Apply the migration strategy • Find recommended tools Planning & Migration • Analyze your costs • Save with offers • Reinvest to do more Optimize • Security • Data protection • Monitoring Secure and Manage Workloads On-Premise AWS
- 12. EC2 RDS SQL Server on Amazon EC2 vs. RDS • Elastic Compute Cloud (EC2) is the AWS version of a virtual machine in the cloud • Can develop & deploy applications faster • Launch virtual servers as needed • Scales up and down to handle changes • Relational Database Service (RDS) offers Database as a Service (DBaaS) • Easy to set up, operate, and scale a relational database in the cloud. • Cost-efficient and resizable capacity • Automated administration tasks
- 13. Amazon EC2 Instance Types General Purpose Compute Optimized Memory Optimized Accelerated Computing Storage Optimized A1 C5 R5 P3 I3 T3 C5n R5a P2 I3en T3a C4 R5n Inf1 D2 T2 R4 G4 H1 M6g X1e G3 M5 X1e F1 M5a High Memory M5n Z1d M4
- 14. Amazon EBS Volume Types Solid-State Drives (SSD) Hard Disk Drives (HDD) Volume Type General Purpose SSD (gp2) Provisioned IOPS SSD (io1) Throughput Optimized HDD (st1) Cold HDD (sc1) Description General purpose SSD volume that balances price and performance for a wide variety of workloads Highest- performance SSD volume for mission- critical low-latency or high-throughput workloads Low-cost HDD volume designed for frequently accessed, throughput- intensive workloads Lowest cost HDD volume designed for less frequently accessed workloads
- 15. Amazon EBS Volume Types Solid-State Drives (SSD) Hard Disk Drives (HDD) Use Cases Recommended for most workloads System boot volumes Virtual desktops Low-latency interactive apps Development and test environments Critical business applications that require sustained IOPS performance, or more than 16,000 IOPS or 250 MiB/s of throughput per volume Large database workloads, such as: • MongoDB • Cassandra • Microsoft SQL Server • MySQL • PostgreSQL • Oracle Streaming workloads requiring consistent, fast throughput at a low price Big data Data warehouses Log processing Cannot be a boot volume Throughput-oriented storage for large volumes of data that is infrequently accessed Scenarios where the lowest storage cost is important
- 16. Amazon RDS doesn't support running these services on the same server as your Amazon RDS DB instance: Limitation of RDS over SQL Server on EC2 • SQL Server Analysis Services • SQL Server Integration Services • SQL Server Reporting Services • Data Quality Services • Master Data Services • We can create up to 30 databases on each of your DB instances running Microsoft SQL Server. The Microsoft system databases, such as master and model, don’t count toward this limit. • Some ports are reserved for Amazon RDS use and you can’t use them when you create a DB instance. • Amazon RDS for SQL Server does not support importing data into the msdb database. • We can’t rename databases on a DB instance in a SQL Server Multi-AZ with Mirroring deployment.
- 17. Limitation of RDS over SQL Server on EC2 The following Microsoft SQL Server features are not supported on Amazon RDS: • Stretch database • Backing up to Microsoft Azure Blob Storage • Buffer pool extension • Data Quality Services • Database Log Shipping • Database Mail • Distribution Transaction Coordinator (MSDTC) • File tables • FILESTREAM support • Maintenance Plans • Performance Data Collector • Policy-Based Management • PolyBase • Machine Learning and R Services (requires OS access to install it) • Replication • Resource Governor • Server-level triggers • Service Broker endpoints • T-SQL endpoints (all operations using CREATE ENDPOINT are unavailable) • WCF Data Services
- 18. SQL Server HA & DR on AWS EC2 • Use Multiple Availability Zones (AZs) • Implement instance-level and AZ-level failure tolerance • Synchronous replication across physical data centers • Enterprise Edition (2012+) • Always On Availability Groups • Shared nothing architecture (multi-subnet failover) • Standard Edition • Basic Always On Availability Groups (2016+) • Failover Cluster Instances (FCI) using block storage replication Availability Zone C AWS Availability Zone A Availability Zone B
- 19. Multi-Region Always On Availability Group • High Availability and Disaster recovery • High Performance EBS Volume • Instance Store • Backups to S3 Bucket • Horizontal and Vertical Scaling EC2 Primary Replica VPC Availability Zone 1 Availability Zone 2 Synchronous Commit Automatic Failover Availability Zone 1 Asynchronous Commit Manual Failover VPC Elastic IP Elastic IP AWS Region A AWS Region B MS SQL EC2 Secondary Replica MS SQL EC2 Secondary Replica MS SQL
- 20. Feature Comparison: RDS & EC2 Features RDS SQL on EC2 Control AWS In house DBA Installation Automatic installation Manual Installation Disk Provisioning and management Automatic Manually Patching Automatic Manually Minor version upgrades Automatic Manually Failed instance replacement Automatic N/A Backup and recovery of your SQL Server databases Automatic In House DBA Automated Multi-AZ (Availability Zone) Automatic Manual Configuration required
- 21. Feature Comparison: RDS & EC2 Features RDS SQL on EC2 Cost Cost effective More Control over cost Transparent Data Encryption Available Available CDC Available Available On-Lined Indexing Available Available Data-at-rest protection Available Available Data-in-transit protection Available ( Amazon RDS creates an SSL certificate for your SQL Server DB instance when the instance is created.) Need to configure Manually BD access Controls Through IAM Through SQL Server
- 22. Choosing Between SQL Server RDS & EC2 • Consider RDS first • Focus on business value tasks • High-level tuning tasks • Schema optimization • No in-house database expertise • Need full control over DB instance • Backups • Replication • Clustering • Options that are not available in RDS Amazon RDS for SQL Server SQL Server on Amazon EC2 AWS managed Scaling High Availability Database Backups DBMS Patching DBMS Install / Maintenance OS Patching OS Install / Maintenance Power, HVAC, net Scaling High Availability Database Backups DBMS Patching DBMS Install / Maintenance OS Patching OS Install / Maintenance Power, HVAC, net Customer managed
- 23. Cost Reduction on AWS Cloud for Running SQL Server Don’t pay for a EC2 when all you need is storage You don’t need to pay for a processor if you simply want to store some files. Use AWS Storage S3 instead; you’ll get storage space without the costs of a EC2. Analyze your spending Use the AWS billing portal to track your usage. Then use the information to identify where you’re spending on unneeded resources and shut them down to save money. Leverage AWS RDS and AMI You’ll add to your costs by creating a fully custom environment and loading on lots of third-party products. Instead, leverage AWS’s Platform as a Service features and the configurations in the Amazon Marketplace to buy services designed to work together.
- 24. SQL Server Performance Baselining Memory • MemoryAvailable Mbytes • Page Life Expectancy • User Connections • Memory Grants Pending • Total Server Memory (KB) Disk • Avg. Disk sec/Read • Avg. Disk sec/Write • Avg. Disk Queue Length CPU • Processor Time • Processor Queue Length SQL Statistics • Batch Requests/sec • SQL Compilations/sec • SQL Re-Compilations/sec Minimum, maximum and an average values for each counter has been calculated using the values that were collected over a period of days.
- 25. Sony Case Study – Summary • Very large I/O Demands • Scalability and flexibility • Enhance HA & DR • Storage 10 + TB Data • Infrastructure Modernization • Migration strategy • Performance baseline • Highly available database design (99.999%) • Data transfer automation • Testing and Go Live • Improved performance (30 % increase) • Increased scalability • Higher availability • Data warehouse build in AWS • Analytics – AWS Glue, Amazon Quicksight Challenges Accelerated Migration Outcome Amazon EC2Amazon RDSAmazon S3Auto Scaling Accelerated Migration
- 26. Post-migration Activities • Performance baseline: Performed on a continual basis • Ongoing performance tuning: query tuning, wait statistics analysis, tempdb optimization • Database monitoring and alerting • Index and statistics maintenance
- 27. DBA Responsibility in the Cloud Category DBA Responsibility IaaS PaaS Software Installation and Maintenance Plan, install, and configure new SQL Server, Oracle, MySQL, and any other database management system. On-going updates and patches. If new server is needed, transfer of data from existing system to new platform. Database backup and Recovery Create backup plans to backup the databases. Develop recovery plans and procedures to recover the databases in case of disaster. Test and verify the recovery plans to meet business recovery objectives. High Availability and Disaster Recovery (HADR) Plan, design, and implement High Availability and Disaster Recovery (HADR) solutions based on business requirements. E.g.: SQL AOAG, Clustering, Database Mirroring, and Log Shipping. Test and verify the HADR solutions on set schedules as per business requirements. Infrastructure as a Service (IaaS): AWS EC2 Platform as a Service (PaaS): AWS RDS
- 28. DBA Responsibility in the Cloud Category DBA Responsibility IaaS PaaS Performance Monitoring and Database Tuning Monitor databases for performance issues. Tune the database based on application and usage. Re-balance workloads across database servers based on monitoring. Security and Authentication Implement best practices to minimize security risk. Setup and grant employee and application access based on the Principle of Least Privileges (PoLP). Capacity Planning Workload (I/O) capacity planning. Storage and disk space usage capacity planning. Data Extract, Transformation, and Loading Import large volumes of data that have been extracted from multiple sources into a data warehouse or into a centralized repository. Troubleshooting Quickly understand and respond to problems when they occur and restore data or correct issues to minimize damages. Infrastructure as a Service (IaaS): AWS EC2 Platform as a Service (PaaS): AWS RDS
- 29. DBA Scope of Work on Cloud Category Scope of work Amazon SQL on Premises VMSQL On AWS (EC2) (IaaS) SQL RDS ( PaaS) Installation SQL Server installation DBA perform the SQL assessment and may need to correct the configuration (4-6 hour) Built In Automation DBA Installation of SSIS, SSRS, SSAS , MDS, other SQL features DBA Not available DBA Patching Upgrade DBA Built In Automation (Minor upgrades there is check box) DBA Configuration Login Creation (DB User) DBA DBA DBA Server level configuration (Memory Capping, Tempdb and others) DBA DBA DBA Configure linked server DBA Not available DBA Configure DB Mail DBA Not available DBA
- 30. DBA Scope of Work on Cloud Category Scope of work Amazon SQL on Premises VMSQL On AWS (EC2) (IaaS) SQL RDS ( PaaS) Security Data Masking Not Available DBA Not Available Advanced Threat protection Not Available DBA Not Available Security Auditing DBA DBA DBA Administration SQL Server START STOP DBA DBA DBA Database capacity planning DBA DBA DBA Database Backup DBA DBA DBA Snapshot backup DBA Built In Automation DBA Database Restore DBA DBA DBA Database configuration DBA DBA DBA Database HA DBA Built In Automation (Geo- replication, Auto failover) DBA (Always on, clustering) Database Index maintenance DBA DBA DBA Database corruption DBA Built In Automation DBA Deployment, maintenance, and scaling DBA (Always on, clustering) DBA DBA