Oracle RAC 12c Best Practices Sanger OOW13 [CON8805]

9/30/13&
1&
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
Oracle Real Application
Clusters (RAC) 12c
Best Practices
Markus Michalewicz, Director of Product
Management, Oracle Real Application Clusters
Kalyan Kallepally, Principal Database
Administrator, Wellcome Trust Sanger Institute

9/30/13&
2&
Safe Harbor Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.
Agenda
!  Oracle RAC 12c and Standardization
!  Preparation of the Servers for Oracle RAC 12c
!  Installation of Oracle Grid Infrastructure 12c
!  Installation of the Oracle Database Home
!  Oracle (RAC) Database Creation using DBCA
!  Post-install Steps and Configuration
!  Customer Example: Wellcome Trust Sanger Institute
!  Summary and Q&A

9/30/13&
3&
Standardize on Clusters
Clusters increase scalability and HA regardless of DB types used later
Oracle GI
Oracle GI
Oracle GI for a Cluster
Oracle GI
for a
Cluster
High Availability
Scalability
License Costs
Oracle GI
Standalone
Oracle GI
Re-configuration
to enable cluster
MOS note: 1584742.1
Support Impact of the
Deprecation Announcement
of Oracle Restart with Oracle
Database 12c
Standardize on Oracle RAC
For all the flexibility you need
Oracle GIDBA
DBA
Oracle GIDBA
DBA
DBA
Oracle RAC One Node
Oracle RAC
Flexibility
Scalability
Online
upgrade
DBA
Oracle Restart
Oracle GI
Online (rolling) Patch and PSU Application

9/30/13&
4&
Standardize on Oracle RAC and Oracle Multitenant
The new standard for Oracle Database Consolidation
Oracle GI
Oracle RAC One Node
Consolidation
Agility
Oracle GI
Oracle RAC
Preparation of the Servers
!  Setup of the Servers include:
–  Setting up network connections
!  Public and private with switch
–  Setting up shared storage for all servers
!  Shared storage required
–  Setting up the OS image – here OL 6.4
!  Suggestion:
–  Use cloning to deploy OS images fastervixencomet
dancerdasher
OS OL 6.4 OS OL 6.4
OS OL 6.4OS OL 6.4
OS image setup and more

9/30/13&
5&
Preparation of the Servers
!  Recommendations:
–  Use OL 6.4+ UEK
–  Use the Oracle preinstall package
!  Available for
–  Oracle Database 11g Rel. 2
–  Oracle Database 12c Rel. 1
–  Additional steps to perform after using the
pre-install package for an Oracle RAC cluster:
!  Add a “grid” user & “dba” group
!  Align UIDs / GUIDs across servers
!  Set SELinux to “permissive”, if possible
!  Check large / huge pages settings
Kernel and Oracle preinstall package
[root@dasher ~]# uname –a
Linux dasher 2.6.39-400.17.1.el6uek.x86_64 #1 SMP Fri Feb 22
18:16:18 PST 2013 x86_64 x86_64 x86_64 GNU/Linux
#Get the pre-install package
[root@dasher Desktop]# yum list oracle-*
oracle-rdbms-server-11gR2-preinstall.x86_64 1.0-7.el6 ol6_latest
oracle-rdbms-server-12cR1-preinstall.x86_64 1.0-8.el6 ol6_latest
Install Oracle Grid Infrastructure 12c
What is the default when keeping the preselection in steps 1-4…?
!  GI for a Cluster Installation
!  Standard Cluster
!  Typical Installation

9/30/13&
6&
Step 3: Double-check the cluster topology to deploy!
Oracle Flex Cluster
The new standard for resource management in the cluster
DBA
DBA
DBA
DBB
DBB
Hub Nodes
Leaf Nodes
!  Oracle Flex Cluster:
–  The new standard for flexible
resource management in the cluster
!  Use it as demand requires it
!  Inherits all Flex ASM HA capabilities
!  Will support more use cases in future
–  Efficient High Availability and
resource management across tiers
!  Lightweight cluster stack on leaf nodes
–  Benefit from virtualization and
physical deployments in one cluster

9/30/13&
7&
For now, “Standard Cluster” is the default given the preselection
!  Oracle Flex Cluster
–  Can consist of only HUB-nodes
!  If a Leaf-node has access to shared storage,
it can perform a role change to a HUB-node
–  A Standard Cluster can be converted to a
Flex Cluster at any time after the installation
!  Once a cluster is configured as a Flex Cluster
this configuration is irreversible without a re-
configuration of the cluster.
!  Recommendation:
–  An all-HUB-node Flex Cluster behaves the
same as a Standard Cluster using Flex ASM.
–  Either can be used, but if in doubt, you can
choose standard Cluster and convert later.
Step 4: Do not accept the default – choose “Advanced Installation”
!  Typical Installation
–  Does not provide an option to use “Flex ASM”
–  Does not provide an option to create the “Mgmt
DB / Grid Infrastructure Management Repository”
!  Enhancement Requests for later
releases have been filed already
!  Advanced Option
–  Recommended for all configurations

9/30/13&
8&
“Advanced Installation” – SCAN & GNS requirement
–  SCAN remains to be required for the cluster
–  Choose cluster-specific unique SCAN names
–  OUI will try to enforce it as much as possible
!  http://www.oracle.com/technetwork/products/
clustering/overview/scan-129069.pdf
–  Choose “Configure GNS” if:
!  You want to use dynamic IP management
(DHCP assigned / IPv6 Stateless Address
Autoconfiguration) for client connectivity
–  In this case, GNS requirements remain as
with previous version for standard cluster
“Advanced Installation” – Network Interface Usage
–  Use “Redundant Interconnect Usage” feature
!  Bonding can be used underneath if required
!  MOS notes explain further configuration – e.g.:
–  MOS note 1210883.1 - 11gR2 Grid
Infrastructure Redundant Interconnect
and ora.cluster_interconnect.haip
• It is recommended to align the
ASM network with the private
interconnect.
• Using different subnets for the
private interconnect is required.

9/30/13&
9&
“Advanced Installation” – Grid Infrastructure Management Repository
!  Grid Infrastructure Management Repository (GIMR)
–  A Single Instance Oracle Database 12c
–  It will be installed on one of the nodes
!  It is managed as a failover database
–  It contains the Cluster Health Monitor (CHM) data
–  It is stored in the first ASM disk group created
–  As part of the configuration, Quality of Service
Management (QoS) required resources are
added to the cluster configuration (e.g. OC4J)
!  Recommendation: configure the GIMR!
“Advanced Installation” – Storage Options
!  For Standard Cluster Installations
three storage options are available:
1.  Standard ASM
!  ASM configuration as known
from previous releases
2.  Oracle Flex ASM
!  Recommended option available
with Oracle Grid Infrastructure 12c
–  Flex Cluster requires Flex ASM
3.  Shared File System
!  Alternative

9/30/13&
10&
Oracle ASM 12c – Overview
Oracle ASM 12c Standard Deployment
ASM Cluster Pool of Storage
Disk Group BDisk Group AShared Disk
Groups
Wide File Striping
One to One
Mapping of ASM
Instances to
Servers
ASM Instance
Database Instance
ASM Disk
RAC Cluster
Node4Node3Node2Node1 Node5ASM ASM ASM ASM ASM
ASM Instance
Database Instance
DBA DBA DBCDBB DBBDBB
Introducing Oracle Flex ASM
Removal of One to One Mapping and HA
ASM Cluster Pool of Storage
Disk Group BDisk Group AShared Disk
Groups
Wide File Striping
Databases share
ASM instances
ASM Instance
Database Instance
ASM Disk
RAC Cluster
Node5Node4Node3Node2Node1
Node5
runs as
ASM Client
to Node4
Node1
runs as
ASM Client
to Node2
Node1
runs as
ASM Client
to Node4
Node2
runs as
ASM Client
to Node3
ASM ASM ASM
ASM Instance
DBA DBA DBCDBB DBBDBB

9/30/13&
11&
Use Oracle Flex ASM for storage
–  Use Oracle Flex ASM for storage
!  For better HA of database instances
!  Reserve per-node spare memory
[GRID]> ps -ef |grep pmon
grid 3822 1 0 08:05 ? 00:00:00 asm_pmon_+ASM1
grid 4865 1 0 08:06 ? 00:00:00 mdb_pmon_-MGMTDB
grid 9592 1 0 08:09 ? 00:00:00 apx_pmon_+APX1
grid 19114 19003 0 09:22 pts/0 00:00:00 grep pmon
“Advanced Installation” – Create ASM Disk Group
–  Create Infrastructure Disk Group (DG) first
!  Use it to host OCR / Voting Disk
–  Choose Redundancy accordingly
–  Normal (+) redundancy is recommended
–  Create Data and FRA disk groups
post installation and as required.
!  Optional: Use it to host shared DB homes
by creating an ACFS file system on top of it
–  http://www.oracle.com/technetwork/products/cloud-storage/
oop-patching-of-acfs-shared-oh-1885763.pdf
–  ADVM volumes can be of lower
redundancy than the underlying DG

9/30/13&
12&
“Advanced Installation” – Prerequisite Checks Part 1
“Advanced Installation” – Prerequisite Checks Part 2
Do fix
later!

9/30/13&
13&
Agenda
Install the Oracle Database Home
Nothing special – just a quick walk through
Not the
default…

9/30/13&
14&
Install the Oracle Database Home
A few things to point out
ACFS based
shared home …
Before Creating Databases
Create Server Pools and additional disk groups as required
[GRID]$ srvctl add srvpool –serverpool backoffice
–min 1 –max 2 –i 1
[GRID]$ srvctl add srvpool –serverpool frontoffice
–min 1 –max 4 –i 2
Create additional disk
groups as required.

9/30/13&
15&
Agenda
Create an Oracle Database
What is the default when keeping the preselection in steps 1-4…?
!  Policy-managed
!  Oracle RAC-based
!  Non-CDB Database

9/30/13&
16&
Create an Oracle RAC Database
A few tips and tricks…
Create an empty
multitenant CDB.
Create an Oracle RAC Database
A few tips and tricks…
This is the default…,
but not recommended
–  Use a different disk group than the “database file
location”-disk group as the Fast Recovery Area.
–  Automatic Storage Management (ASM)
recommendation remains unchanged
!  Use it for any current production database
!  New in Oracle Database 12c:
ACFS can be used to host database files
–  Use ACFS for database files, if you want to
!  manage “all data” on a file system
!  use advanced functionality such as

9/30/13&
17&
Milestone
What has been installed and configured so far?
vixen srvpool: frontofficecomet srvpool: backoffice
dancer srvpool: backofficedasher srvpool: frontoffice
OS OL 6.4 OS OL 6.4
OS OL 6.4OS OL 6.4
Oracle GI for a cluster Oracle GI for a cluster
Oracle GI for a clusterOracle GI for a cluster
Oracle RAC Oracle RAC
Oracle RACOracle RAC
raccdb1
raccdb1_2
raccdb1_1
[RAC]> srvctl status database -d raccdb1
Instance raccdb1_1 is running on node comet
Instance raccdb1_2 is running on node dancer
[RAC]>srvctl status srvpool -serverpool backoffice -detail
Server pool name: backoffice
Active servers count: 2
Active server names: comet,dancer
NAME=comet STATE=ONLINE
NAME=dancer STATE=ONLINE
[RAC]> srvctl status srvpool
Server pool name: Free
Server pool name: Generic
Server pool name: backoffice
Server pool name: frontoffice
Create Pluggable Databases
Create HR PDB on running instances
raccdb1
raccdb1_2
raccdb1_1
Create a PDB on a node
that hosts a multintenant
CDB instance

9/30/13&
18&
Create Pluggable Databases
Create CRM PDB on running instances
raccdb1
raccdb1_2
raccdb1_1
Agenda

9/30/13&
19&
Post-Creation Steps
Create services to balance the workload – part 1
raccdb1
raccdb1_2
raccdb1_1 [RAC]> srvctl add service -d raccdb1 -s hrsvc -pdb hr -
serverpool backoffice
[RAC]> srvctl add service -d raccdb1 -s crmsvc -pdb crm -
serverpool frontoffice
[RAC]> srvctl start service -d raccdb1 -s hrsvc
[RAC]> srvctl start service -d raccdb1 -s crmsvc
!  Dynamic Database Services
–  For easier management of multitenant CDBs
–  For better workload management in the cluster
!  Pluggable Databases
–  By default are in mount status after creation
–  They need to be opened in either r/o or w/r
–  Startup and distribution of Pluggable databases
can be automated and optimized using services
Post-Creation Steps
Create services to balance the workload – part 2
raccdb1
raccdb1_3
raccdb1_4 raccdb1_2
raccdb1_1
[RAC]> srvctl status database -d raccdb1
Instance raccdb1_1 is running on node comet
Instance raccdb1_2 is running on node dancer
Instance raccdb1_3 is running on node vixen
Instance raccdb1_4 is running on node dasher
[RAC]> srvctl status service -d raccdb1
Service crmsvc is running on nodes: vixen,dasher
Service hrsvc is running on nodes: comet,dancer

9/30/13&
20&
Post-Install Steps
Check your environment regularly using CVU and RACcheck
#CVU is run regularly by default
[GRID]> srvctl status cvu
CVU is enabled and running on node comet
!  CVU is run regularly as part of the GI stack
!  Use RACcheck - RAC Configuration Audit Tool
–  For details see MOS note 1268927.1
–  New features in 2.2.2 include:
!  Execution as the root user for sysadmin
profile e.g. ./raccheck -profile sysadmin
!  RACcheck daemon feature to automate
execution of RACcheck non-interactively
at predefined interval
!  Support for Solaris Sparc 11
!  Upgrade best practices for 11.2.0.3, 11.2.0.4
and Oracle Database 12c
!  Standard health check is now
included in post upgrade
More information attached
Post-Install Steps
Monitor best practices notes and announcements
!  RACcheck - RAC Configuration Audit Tool
–  For details see MOS note 1268927.1
!  Great, but as any tool, RACcheck has a release cycle
!  Best practices My Oracle Support (MOS) notes:
–  MOS note 810394.1 - RAC and Oracle Clusterware Best Practices and Starter Kit (Platform Independent)
!  MOS note 811306.1 - RAC and Oracle Clusterware Best Practices and Starter Kit (Linux)
!  MOS note 811280.1 - RAC and Oracle Clusterware Best Practices and Starter Kit (Solaris)
–  Note: Engineered Systems with the same OS will have additional recommendations
!  MOS note 811271.1 - RAC and Oracle Clusterware Best Practices and Starter Kit (Windows)
!  MOS note 811303.1 - RAC and Oracle Clusterware Best Practices and Starter Kit (HP-UX)
!  MOS note 811293.1 - RAC and Oracle Clusterware Best Practices and Starter Kit (AIX)
!  MOS note 1452965.2 –
–  Information Center: Oracle Scalability Grid Infrastructure / Clusterware and Real Application Clusters (RAC)

9/30/13&
21&
!
!
!
Oracle!Database!12c"
at!
Wellcome!Trust!Sanger!Ins7tute!(WTSI)!
Oracle Open World - 2013
!
!
!
!
!
!
!
Kalyan!Kallepally!
Principal!Database!Administrator!
About!me!
•  Working!as!Principal!Database!Administrator!at!
–  Wellcome!Trust!Sanger!Ins7tute!
•  12!years!of!experience!in!Oracle!(8E11gr2)!
•  6!years!of!experience!in!MySQL,!SQL!Server!&!PostgreSQL!
•  Ac7ve!member!in!12C!beta!tes7ng!program!
•  Recently!started!blogging!!
–  www.dbakalyan.wordpress.com!
–  You!can!reach!me!on!dba_kkk@yahoo.co.uk!
!

9/30/13&
22&
!
!Wellcome!Trust!Sanger!Ins7tute!
•  One!of!the!leading!and!most!published!genome!ins7tu7ons!in!the!world!
•  Located!in!Hinxton,!south!of!Cambridge,!UK!!!
•  Founded!in!1993!by!the!Wellcome!Trust!and!the!UK!Medical!Research!Council!(MRC)!
•  Primarily!funded!by!the!Wellcome!Trust!!
•  The!ins7tute!is!named!aZer!Dr!Fred!Sanger!–!The!double!Nobel!prizeEwinning!biochemist!
•  Main!aim!is!to!ensure!scien7ﬁc!data!is!freely!available!
•  Responsible!for!the!comple7on!of!1/3!of!the!human!genome!
!
Data!Centre!at!WTSI!
•  1000m2!of!Datacentre!space!split!into!!
4!quadrants!
•  Current!computa7onal!capacity!!E!
16.5K!cores!and!20PB!of!storage!
•  24/7!reliability!with!DR!setup!
!
•  Going!Green!ini7a7ve!–!Implemented!Virtualiza7on,!MAID!storage,!data!
compression!using!algorithms!and!storage!solu7ons!
•  Power!consump7on!constant!at!1.2MW!since!May!2009!despite!20!fold!data!
storage!increase!

9/30/13&
23&
WTSI!Oracle!Grid!Infrastructure!E!11gR2!
•  Primary!Site!
! 2!X!4!Node!RAC!clusters!
! 2!Node!RAC!Cluster!
! Oracle!Restart!Nodes!
!
•  DR!Site!
! 4!Node!RAC!cluster!
! Oracle!Restart!
Cancer!Genome!Project!E!CGP!
•  Sanger!Ins7tute!is!a!world!leader!in!using!DNA!!
sequencing!to!iden7fy!muta7ons!that!causes!cancer.!
•  Make!the!scien7ﬁc!data!available!to!!the!world!!
via!our!websites!24x7!
•  Mul7ple!database!deployments!to!support!both!!
laboratory!and!!research!websites!

9/30/13&
24&
Dataguard
4 Node
4 Node
WAN TRAFFIC
Clients
Dedicated Network
Primary Site DR Site
+ASM1 +ASM2 +ASM3 +ASM4 +ASM1 +ASM2 +ASM3 +ASM4
Cancer Genome Project -
Current Oracle RAC / Grid
Infra. Layout (11gR2)
!
Current!Challenges!–!Cancer!Genome!Project!
•  Databases!in!consolidated!environments!!
compete!for!shared!compu7ng!resources!!
"  CPU,!memory,!sessions,!parallel!execu7on!servers,!I/O!
•  Provisioning!and!management!of!many!discrete!databases!
"  Cloning!
"  Managing!large!number!of!standbys!for!HA!
"  Backups!
•  Patching!and!upgrading!Oracle!database/soZware!binaries!
"  Many!dependencies!!

9/30/13&
25&
4 Node
WAN TRAFFIC
Clients
Primary Site
+ASM1 +ASM2 +ASM3 +ASM4
&Loss!of!Service!
•  If!ASM!on!any!node!fails!
•  Database!service!is!down!
•  Not!easy!to!
•  Patch/Upgrade!
•  Move/Clone!Databases!
•  Non!Commijed!Transac7ons!
•  Captured!&!replayed!
Current Challenges –
Oracle RAC 11gR2
Op7mal!Solu7ons!to!our!Current!Challenges!
(Sanger!Projects)!!
!
Oracle!Database!12c!New!Features:!
!

9/30/13&
26&
!Business!Con7nuity!and!High!Availability!
!!!Flex!ASM!
!  No!need!to!have!an!ASM!instance!!
per!node!in!the!cluster!
!  Enables!an!interEnode!ASM!instance!failover!
!  Lowers!impact!of!secondary!failure!
!!!Mul7tenant!Architecture!with!RAC!
!  High!consolida7on!density!
!  Rapid!provisioning!and!cloning!using!SQL!
!  Rapid!patching!and!upgrades!
!  Manage!many!databases!as!one!
!  Dynamic!!database!resource!management!
!!!Applica7on!Con7nuity!
!  Captures!last!inEﬂight!transac7on!
!  Masks!outages!from!end!users!
!  Replay!ini7a7on!7me!and!replay!reten7on!
!!!Global!Data!Services!!
!  ConnectE7me!Load!balancing!
!  InterEdatabase!Service!failover!
!  RoleEbased!global!Services!!
for!Ac7ve!Data!Guard!
Dataguard
4 Node
4 Node
WAN TRAFFIC
Clients
Dedicated Network
+ASM1 +ASM2 +ASM3 +ASM4 +ASM1 +ASM2 +ASM3 +ASM4
•  Enable!Flex!ASM!
•  Mul7tenant!Architecture!
•  Enable!Global!Data!Services!
•  Conﬁgure!Applica7on!Con7nuity!
Global Data Services
Application Continuity

9/30/13&
27&
Dataguard
4 Node
4 Node
WAN TRAFFIC
Clients
Dedicated Network
+ASM1 +ASM2 +ASM3 +ASM1 +ASM2 +ASM3
Manage!Many!!Databases!As!One!
•  Fewer!databases!to!manage!
•  Faster!Upgrades!and!Patching!
•  Backed!up!as!one!!
•  PDBs!are!replicated!automa7cally!
Global Data Services
Application Continuity
WAN TRAFFIC
Clients
Primary Site
+ASM1 +ASM2 +ASM3
PolicyEbased!management!
beneﬁts:!!
•  Flexible!and!more!eﬃcient!
hardware!u7liza7on!!
•  Higher!density!consolida7on!
and!greater!scalability!

9/30/13&
28&
Summary!
Looking!forward!to!life!with!Oracle!Database!!
12c!at!WTSI!#!
Agenda

9/30/13&
29&
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template57
Graphic Section Divider
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template58

Oracle RAC 12c Best Practices Sanger OOW13 [CON8805]

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to Oracle RAC 12c Best Practices Sanger OOW13 [CON8805] (20)

More from Markus Michalewicz (20)

Recently uploaded (20)

Oracle RAC 12c Best Practices Sanger OOW13 [CON8805]