Managing Exadata in the Real World
- 1. Managing Exadata in the Real World Arup Nanda
- 2. What is this About? • How Exadata is different? • Who manages it? • Who does what? • What you need to know about managing it • What must you learn about potential issues 2
- 3. What is Exadata • Looks like an appliance • But is not an appliance. Why? – It contains additional software to make it a better database machine – The components are individually administered • That’s why Oracle calls it a Database Machine (DBM) • DMA – Database Machine Administrator 3
- 4. iDB CPU iDB Memory Network I/O Controller Disk 4
- 5. Storage Cell Server iDB • Cells are Sun Blades • Run Oracle Enterprise Linux • Software called Exadata Storage Server (ESS) which understands iDB Disk1 Disk2 Disk3 5
- 6. Storage Indexes Storage Indexes store in memory of the Cell Server the areas on the disk and the MIN/MAX value of the column and whether NULL Disk1 Disk2 Disk3 exists. They eliminate disk I/O. SELECT … MIN = 3 MIN = 4 MIN = 1 MIN = 3 FROM TABLE MAX = 5 MAX = 5 MAX = 2 MAX = 5 WHERE COL1 = 1 Storage Index Disk4 6
- 7. Put Together: One Full Rack RAC Cluster Database Database Node 1 Node 8 Database Node 7 Clients InfiniBand Network Switch connect to the Switch database nodes. Cell 1 Cell 14 Cell 1 Cell 1 Cell 1 7
- 8. Disk Layout • Disks (hard and flash) are connected to the cells. • The disks are partitioned at the Compute Nodes cell • Some partitions are presented as filesystems Storage • The rest are used for ASM Cell diskgroups • All these disks/partitions are presented to the compute nodes 8
- 9. Disk Presentation Node filesystem filesystem Cell 9
- 10. Disk Failures Datafile block1 Cell 1 Cell 2 block1 block1 10
- 11. One Cluster? One Cluster QA1 QA2 QA3 Prod1 Prod2 Prod3 Dev1 Int1 QA1 QA2 Prod4 Prod1 Prod2 Prod3 Dev1 Int1 QA1 QA2 QA3 Prod1 Prod2 Prod3 Dev1 Int1 Dev2 11
- 12. Many Clusters? QA Cluster Prod Cluster Dev Int QA1 QA2 QA3 Prod1 Prod2 Prod3 Dev1 Int1 QA Cluster Prod Cluster Dev Int 12
- 13. Playing Nice • Database Resource Manager • I/O Resource Manager • Cell Fencing Compute QA Prod Nodes Storage Cells 13
- 14. Component Management Linux Commands – vmstat, mpstat, fdisk, etc. Compute Nodes ASM Commands – SQL*Plus, ASMCMD, ASMCA Database Commands – startup, alter database, etc. Clusterware Commands – CRSCTL, SRVCTL, etc. Storage Cell Linux Commands – vmstat, mpstat, fdisk, etc. CellCLI – command line tool to manage the Cell 5-part Linux Commands article series http://bit.ly/k4mKQS 4-part Exadata Command Reference article series http://bit.ly/lljFl0 14
- 15. Server Management • Sun Blades and Oracle Enterprise Linux • Normal Sysadmin Work – Shutdown, fdisk, etc. • ILOM – Integrated Lights Out Management • KVM allows physical access – But you can use ILOM for virtual console • Needs Pure Linux Skills 15
- 16. Database, Cluster Management • Cluster – crsctl, srvctl • ASM – asmcmd, SQL*Plus • Database – srvctl, SQL*Plus 16
- 17. Storage Management • Two ways to manage the storage – Traditional Storage Administrators not quite iDB helpful here – Enterprise Manager – CellCLI Disk1 Disk2 Disk3 17
- 18. Network Management • Two types of network – Ethernet – Infiniband • Tools – ibstatus – iblinkinfo – verify-topology 18
- 19. Oracle Provided Tools • All tools are found at /opt/oracle.SupportTools • CheckHWnFWProfile – to check the HW profile • Directory ibdiagtools /opt/MegaRAID/MegaCli/MegaCli64 # ipmitool -H prolcel01-ilom -U root chassis power on # imageinfo # imagehistory 19
- 20. Skills for Administration Skill Needed System Administrator 10% Storage Administrator 0% Network Administrator 5% Database Administrator 60% Cell Administration 25% DBA Sys Admin Network Admin Cell Admin 20
- 21. Divide and Conquer Database DBA Machine System Admin Network Network Admin Storage ?? DBA Sys Admin Network Admin Cell Admin 21
- 22. Combined Skills Database Database Machine Administrator Machine (DMA) New Role Network Storage DBA Sys Admin Network Admin Cell Admin 22
- 23. Resources • My Papers – 5-part Linux Commands article series http://bit.ly/k4mKQS – 4-part Exadata Command Reference article series http://bit.ly/lljFl0 • OTN Page on Exadata – http://www.oracle.com/technetwork/database/exadata/in dex.html • Tutorials – http://www.oracle.com/technetwork/tutorials/index.html 23
- 24. Pros and Cons of DMA • Pros – Natural progression from DBA skills – Very little non-DBA activities – Patching becomes simpler – Performance diagnosis becomes easier • Cons – Skills – Security? • Appliance model comes in here 24
- 25. Backup and DR • No SAN connectivity • Only NAS Exadata – Infiniband – Tape , Disk Pool • DR Infiniband – No Storage Level Replication – Only Data Guard Backup – Supplemental Logging Device – Force Logging – http://www.oracle.com/technetwork/database/features/availa bility/maa-wp-dr-dbm-130065.pdf • Golden Gate 25
- 26. ETL and Reporting OLTP Exadata Golden Gate Infiniband Informatica MicroStrategy Exalytics 26
- 27. Summary • Exadata is an Oracle Database on RAC with a specialized storage • A single role helps in administration: Database Machine Administrator (DMA) • DBA -> DMA is the natural progression • Use many clusters in the same Exadata frame • Use IB for ETL, Reporting and Backups • Data Guard only physical DR solution 27
- 28. Thank You! 28