The Key to Effective Analytics: Fast-Returning Queries
0 likes185 views
The document discusses database management and performance tuning, highlighting the prevalence of relational databases like MS SQL Server and Oracle, while mentioning challenges such as database entropy and performance bottlenecks. Key insights include the need for effective analytics tools like SQL Diagnostic Manager to help DBAs identify issues and enhance performance through proactive monitoring and tuning. It also emphasizes the importance of flexibility and the ability to review historical data for troubleshooting.
The Key to Effective Analytics: Fast-Returning Queries
- 1. Grab some coffee and enjoy the pre-show banter before the top of the hour!
- 7. Database Tuning Robin Bloor Ph.D.
- 8. The DBA For sophisticated databases (Oracle, SQL Server, DB2, etc.) database tuning is a tough job, but a SECURE job
- 9. file://localhost/Users/robinbloor2/Dropbox/ ~Desktop/bottlenecks.jpg The Database Landscape u Relational databases still dominate in popularity (> 95%) u MS SQL Server is second in popularity to Oracle u Used both for OLTP and query workloads u Alternatives (NoSQL & Graph) used mainly for analytics u Most companies standardize and use alternatives only in exceptional circumstances u Databases are “strategic” and they evolve u Neither Hadoop nor Spark is a database
- 10. file://localhost/Users/robinbloor2/Dropbox/ ~Desktop/bottlenecks.jpg DBA Activities u Install, config, upgrade u License management u ETL & Replication u Storage & Caplan u Troubleshooting u Performance monitoring & tuning u Security u BU & Recovery u S/W Test systems u Data lifecycle
- 11. file://localhost/Users/robinbloor2/Dropbox/ ~Desktop/bottlenecks.jpg Database Performance Bottlenecks u CPU: CPU overload, context switches, I/O waits, CPU cache, Backplane throughput u Memory: Thrashing, fragmentation u Disk: Caching, I/O saturation, fragmentation, SSD performance u Network Overhead (SANs, NAS, local disk) u Locking
- 12. file://localhost/Users/robinbloor2/Dropbox/ ~Desktop/bottlenecks.jpg Hardware Factors u CPUs, GPUs & FPGAs u Cross breeding u 3D Xpoint and PCM u SSDs & parallel access u Parallel hardware architectures At the hardware level PERFORMANCE is accelerating and COSTS continue to fall.
- 13. file://localhost/Users/robinbloor2/Dropbox/ ~Desktop/bottlenecks.jpg Database Issues u Database entropy (proliferating instances) is very common – it’s the norm, with databases being used as “cupboards” u Few databases are self-tuning or properly tuned (especially the ones that perform) u SQL needs to be tuned (particularly for JOINS) u Most companies employ few DBAs (they’re expensive)
- 18. @dez_blanchfield Key challenges impac/ng Database Analy/cs performance § Infrastructure Performance Monitoring § Keeping on top of Query Monitoring § Monitoring response 6mes to pla8orm & user § Addressing poorly structured queries § Itera6ve query / review & fine tuning workflows § Keeping on top of deadlocks and blocks § ShiF from Historical to Predic6ve aler6ng Typical day to day performance hits..
- 20. @dez_blanchfield Performance issues which regular users can’t easily resolve § Performance of complex code you didn’t write § Pre-staging of data for performance where possible § Dele6ng indexes before bulk imports then re-index § Op6mize indexes regularly or aFer a bulk import § Limit queries to just number of columns you need § Knowing how & when to use temp tables § Batch up delete & update workloads efficiently § Detec6ng & addressing foreign key constraints Issues invisible to most mere mortals..
- 23. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. IDERA - SQL DIAGNOSTIC MANAGER Hot Technology, The Key to Effective Analytics: Fast-Returning Queries Bullett Manale Director, Sales Engineering
- 24. © 2015 IDERA, Inc. All rights reserved. Proprietary and confidential. 2
- 25. © 2015 IDERA, Inc. All rights reserved. Proprietary and confidential. 3 § FOCUS: SQL DIAGNOSTIC MANAGER
- 26. © 2015 IDERA, Inc. All rights reserved. Proprietary and confidential. 4 § Data Manager § DBA § Acting DBA (or most technical person in the room) TYPICAL USER OF SQL DIAGNOSTIC MANAGER
- 27. © 2015 IDERA, Inc. All rights reserved. Proprietary and confidential. 5 RESPONSIBILITIES OF THE DBA (ACCORDING TO WIKIPEDIA)
- 28. 6© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. USUALLY MORE TO A QUESTION THAN MEETS THE EYE ‘Loaded’ Questions: What is my worst performing query? | (Reads, Writes, CPU, Frequency, Waits, Duration) Show me blocking incidents within my production environment? | (Blocking by count, by blocking time) When will I run out of space? | (Drive space, Database space, VM Host Space, Partition) What is my worst instance of SQL Server running? | (CPU, Waits, I/O, Memory, Alerts, etc..) What is my worst performing application? | (CPU, I/O, by user, by client, by database) What caused my problem that happened last night? | (What time last night, CPU, I/O, Non-responsive) What can I do proactively to improve my performance? (Totally Open Ended)
- 29. 7© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. TYPICAL REQUIREMENTS OF A GOOD ANALYTICS TOOL BE ABLE TO….. …collect the information at little to no cost …collect the appropriate data and leave out the unnecessary data …once collected, access the data quickly and efficiently …easily understand the data or provide additional resources that help educate ...easily share the information with others using reports, exporting, and integration …easily drill even deeper into the data if it is called for
- 30. 8© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. SQL DIAGNOSTIC MANAGER CORE DELIVERABLES § Manage at the Enterprise Level – DBAs typically manage a lot of database instances. Having a view that represents the health of each instance in a single window is vital. This provides peace of mind for the DBA in that they can quickly glance at a screen to know if everything is running as it should be; if it’s not, then to have the ability to drill down and identify and troubleshoot. The tool also needs to be presented in a way that is familiar to the day to day operations of the DBA (DBA Centric View).
- 31. 9© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. SQL DIAGNOSTIC MANAGER CORE DELIVERABLES § Ability to Look Into the Past – A DBA can’t continuously stare at a console and wait for problems to happen. Problems happen during meetings, in the middle of the night and off hours. Having a tool to help see the state of a server when a problem happened in the past is vital. Additionally, the historical data collected can be used for other purposes than just troubleshooting. For example: What is my fastest growing database over the last year?
- 32. 10© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. SQL DIAGNOSTIC MANAGER CORE DELIVERABLES § Query Performance and Accountability – The DBA is often left holding the bag whenever there is a performance related issue. The perception is that if the database is the slowdown (or calls to the database) then it must be the DBAs fault. Having the ability to identify performance issues down to the command level is a vital part of what SQL Diagnostic Manager does.
- 33. 11© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. SQL DIAGNOSTIC MANAGER CORE DELIVERABLES § Ability to be Proactive – Proactive means different things to different people. For DBAs, in most cases, it means taking corrective action before the user experience is diminished; or even going so far as to automate the remediation of known issues automatically. SQL Diagnostic Manager can help with all of these.
- 34. 12© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. SQL DIAGNOSTIC MANAGER CORE DELIVERABLES § Overall Flexibility – Every environment is different. Out-of-the-box SQL Diagnostic Manager has canned metrics that are monitored and collected historically for review. The same holds true for reports. Having the ability to extend this monitoring footprint is paramount, especially for larger enterprise shops. Obscure monitoring requirements will be necessary. Having a tool that adapts to the environment is crucial.
- 35. 13© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. ARCHITECTURE OF SQL DIAGNOSTIC MANAGER SQLdm Thick Consoles SQLdm Services SQLdm Repository (SQL server 2005+) Management Service Collection Service Active Android iOS Windows • Nothing installed on the monitored SQL instance • No agents, stored procedures, etc. • Monitor physical and virtual databases IDERA core services SQL Diagnostic Manager SQLSafe Backup*SQL Enterprise Job Manager* SQLdm & Other Web Consoles * Sold separately* Sold separately SQLdm Mobile Licensed per instance Now Supports 2016!
- 36. 14© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. INTRODUCING SQL WORKLOAD ANALYSIS (PART OF SQL DIAGNOSTIC PRO BUNDLE)
- 37. 15© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. QUICK DEMONSTRATION SQL DIAGNOSTIC MANAGER
- 39. The Archive Trifecta: • Inside Analysis www.insideanalysis.com • SlideShare www.slideshare.net/InsideAnalysis • YouTube www.youtube.com/user/BloorGroup THANK YOU!