Abstract image of a woman sitting on books and studying on a laptop

Effective Oracle Home Management in the new Release Model era

Ludovico Caldara, profile picture
Ludovico Caldara

The document presents strategies for effective Oracle home management, emphasizing the importance of patch management and automation in maintaining Oracle environments. It discusses common pitfalls such as inconsistent patch levels and manual installation processes, advocating for the use of golden images and automated scripts to improve efficiency. Additionally, it highlights the complexities involved in patching and the necessity for clear naming conventions and systematic tracking of patches.

Technology
1 of 49
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
Effective Oracle Home Management in the new Release Model era
Effective Oracle Home Management with the New Release Model Ludovico Caldara Computing Engineer IT Department, CERN ludovico.caldara@cern.ch @ludodba 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 2
About Ludovico Caldara ■ 18+ Years DBA (Not Only Oracle) ■ ITOUG co-founder ■ OCP (11g, 12c, MySQL) & OCE ■ Italian living in Switzerland ■ http://www.ludovicocaldara.net ■ @ludodba ■ ludovicocaldara 05.12.2018 3 Effective Oracle Home Management #UKOUG_Tech18
CERN • CERN - European Laboratory for Particle Physics • Founded in 1954 by 12 Countries for fundamental physics research in the post-war Europe • Today 22 member states and world-wide collaborations • About ~1100 MCHF yearly budget • 2’300 CERN personnel • 10’000 users from 110 countries 405.12.2018 Effective Oracle Home Management #UKOUG_Tech18
Fundamental Research • What is 95% of the Universe made of? • Why do particles have mass? • Why is there no antimatter left in the Universe? • What was the Universe like, just after the "Big Bang"? 505.12.2018 Effective Oracle Home Management #UKOUG_Tech18
Effective Oracle Home Management #UKOUG_Tech18 6 The Large Hadron Collider (LHC) Largest machine in the world 27km, 6000+ superconducting magnets Emptiest place in the solar system High vacuum inside the magnets Hottest spot in the galaxy During Lead ion collisions create temperatures 100 000x hotter than the heart of the sun Fastest racetrack on Earth Protons circulate 11245 times/s (99.9999991% the speed of light)
“Patch soon, patch often” vs reality 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 7
Frequent patching? • Patching is time consuming • It is complex • It introduces some risks • It is not always really necessary • It leads to human errors 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 8
How many bugs could you encounter? $ opatch lspatches 27923320;Database PSU 12.1.0.2.180717, Oracle JavaVM Component (JUL2018) 22243983; 27762253;OCW PATCH SET UPDATE 12.1.0.2.180717 (27762253) 27547374;Database Bundle Patch : 12.1.0.2.180717 (27547374) OPatch succeeded. $ cd $ORACLE_HOME/inventory $ grep -r "bug description" * | wc -l 2132 $ grep -r "bug description" * | grep -i "wrong result" | wc -l 66 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 9
Quick overview of the New Release Model 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 10
What does it change? Jan 18 Apr Jul Oct Jan 19 Apr Jul Oct RU 18.1.0 18.2.0 18.3.0 18.4.0 18.5.0 18.6.0 18.7.0 18.8.0 RUR1 18.2.1 18.3.1 18.4.1 18.5.1 18.6.1 18.7.1 RUR2 18.2.2 18.3.2 18.4.2 18.5.2 18.6.2 RU 19.1.0 19.2.0 19.3.0 19.4.0 RUR1 19.2.1 19.3.1 RUR2 19.2.2 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 11 2
What does it change? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 12 Short Development cycle Frequent Upgrades And Updates
Which patches should I apply? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 13
Brave vs Fearful 12.1 >= 12.2 Effective Oracle Home Management #UKOUG_Tech18 14 How do I feel? PSU Bundle Patch BraveFearful How do I feel? Release Update Revision Release Update BraveFearful • 05.12.2018
Brave vs Fearful 12.1 >= 12.2 Effective Oracle Home Management #UKOUG_Tech18 15 How do I feel? PSU Bundle Patch BraveFearful How do I feel? Release Update Revision Release Update BraveFearful Ludovico(*) Ludovico(*) (*) yes, it’s Comic Sans https://mikedietrichde.com/2017/10/24/differences-psu-bp-ru-rur/ • 05.12.2018
Common patching patterns 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 16
Common worst practices in maintaining homes • Installing software by hand • Different patch levels for Oracle Homes with the same path • Not tracking the installed patches • Oracle Home paths hard-coded in the operational scripts • Not minding about naming conventions 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 17
Worst approach (big bang) 1. Stop everything 2. In-place binaries patching 3. Database patching, all at once 4. Start everything Effective Oracle Home Management #UKOUG_Tech18 18 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP UP DOWN UP STOP ALL START ALLPATCH BINARIES PATCH DBs • 05.12.2018
Worst approach (big bang) 1. Stop everything 2. In-place binaries patching 3. Database patching, all at once 4. Start everything Effective Oracle Home Management #UKOUG_Tech18 1905.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP UP DOWN UP STOP ALL START ALLPATCH BINARIES PATCH DBs No easy rollback Long downtime Complex planning
Different approach (still bad) 1. Install new binaries manually 2. Patch the new binaries 3. Stop, change OH, patch databases one by one 4. Decommission old binaries Effective Oracle Home Management #UKOUG_Tech18 2005.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP PATCH BINARIES STOP, MOVE, PATCH ONE BY ONE
Different approach (still bad) 1. Install new binaries manually 2. Patch the new binaries 3. Stop, change OH, patch databases one by one 4. Decommission old binaries Effective Oracle Home Management #UKOUG_Tech18 2105.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP PATCH BINARIES STOP, MOVE, PATCH ONE BY ONE Complex to do manually Repetitive task when many servers
Automate the installation! • Automated creation 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 22 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply
Image repo Automate the installation! • Automated creation • Software cloning 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 23 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply 18.3.1 18.4.0 Server 18.3.1
Image repo Automate the installation! • Automated creation • Software cloning 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 24 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply 18.3.1 18.4.0 Server 18.3.1 RUNDECK ANSIBLE RHP OTHER
Less paths Effective Oracle Home Management #UKOUG_Tech18 2505.12.2018 18.3.0 18.4.0 18.4.0 + one-off X 18.5.0 + one-off X 18.5.0 + one-off X + one-off Y 18.3.0 18.4.0 18.4.0 + one-off X 18.5.0 + one-off X 18.5.0 + one-off X + one-off Y 18.4.1 18.4.1 + one-off Y GOOD: BAD:
Workarounds vs one-offs • Sometimes through underscore parameters • Mitigate the number of one-offs • Less merge requests • Document it! (why, when, who) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 26 1 2 3 alter system set "_px_groupby_pushdown"=off comment='Ludo, 03.05.16: W/A for bug 18499088' scope=both sid='*‘;
Rapid Home provisioning • It might be the perfect product… • Requires Lifecycle Management Pack • The technology stack is complex (Clusterware, ASM, ACFS, NFS, GNS…) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 27
Custom approach • Simple script • Golden images • Solid naming conventions • I blogged about it! http://www.ludovicocaldara.net/dba/oh-mgmt-[1..7]/ 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 28
Implementation at Etat de Vaud 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 29
Naming Conventions • Central Inventory contains OH name • OH name matches OH location • No dots (.) allowed (inventory limitation) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 30 Central Inventory <HOME NAME=“18_3_0" LOC="/u01/app/oracle/product/18_3_0" TYPE="O" IDX="2"> • 1 to 1 mapping: easy to code • 18c read-only OH uses the name as subdir in `orabasehome`: consistent naming!
Golden images 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 31 Golden Images Store 18_3_0.zip 18_4_0.zip 18_3_1.zip SERVER1 18_3_0.zip 18_3_1.zip SERVER2 18_3_1.zip SERVER3 18_4_0.zip Golden Image Metadata Name Path ------- ----------------- 18_3_0 /store/18_3_0.zip 18_3_1 /store/18_3_1.zip 18_4_0 /store/18_4_0.zip
Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 33 SERVER1 18_3_0 SERVER2… First Install
Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 34 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -c -n 18_3_0
Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 35 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_3_018_3_0
Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 36 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_3_0 -n 18_4_0 18_3_0 18_4_0
Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 37 18_3_0.zip SERVER1 18_3_0 SERVER2… Manual patching 18_3_0 18_4_0
Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 38 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -c 18_3_0 18_4_0 18_4_0.zip
Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 39 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_4_0 18_3_0 18_4_0 18_4_0.zip 18_4_0
What it actually does Creation workflow • Copy the Home • Clean it up • Create the zip • Store the zip in the repo • Store metadata Installation workflow • Security checks • Create new path • Unzip from the store • Attach home • Relink (opt. with RAC) • Run setasmgid • Other stuff (e.g. TNS) Effective Oracle Home Management #UKOUG_Tech18 4005.12.2018
Effective Oracle Home Management #UKOUG_Tech18 4105.12.2018 DEMO
Further development at CERN 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 42
Read-only Oracle Home $ roohctl –enable Enabling Read-Only Oracle home. Update orabasetab file to enable Read-Only Oracle home. Orabasetab file has been updated successfully. Create bootstrap directories for Read-Only Oracle home. Bootstrap directories have been created successfully. Bootstrap files have been processed successfully. Read-Only Oracle home has been enabled successfully. Check the log file /u01/app/oracle/cfgtoollogs/roohctl/roohctl- 180822AM102911.log. 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 43
AGILE INFRASTRUCTURE SERVER Insertion in the Agile Infrastructure 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 44 Image Store Scripts REST client Oracle Home
Grid Infrastructure Installation • Puppet + Rundeck for automatic GI install • Patched Grid Infrastructure versions beside the first one https://www.markusdba.net/?p=294 • Out-of-place patching with 18c independent local-mode automaton 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 45 $ rhpctl move gihome –sourcehome old_path -destinationhome new_path
Independent Local-mode Automaton (18c) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 46 # $GI_HOME/srvctl enable volume -volume GHCHKPT -diskgroup mgmt # $GI_HOME/srvctl enable filesystem -volume GHCHKPT -diskgroup mgmt # $GI_HOME/srvctl start filesystem -volume GHCHKPT -diskgroup mgmt $ $GI_HOME/srvctl start rhpserver Local Resources --------------------------------------------------------------- ora.MGMT.GHCHKPT.advm ora.mgmt.ghchkpt.acfs ora.helper Cluster Resources --------------------------------------------------------------- ora.rhpserver
Independent Local-mode Automaton (18c) • Damn-easy move of GI HOME • Automatic resource relocation • Enable Grid Infrastructure Golden Images • Patching (apply and rollback) is done automatically 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 47
Back to the demo… 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 48
Questions? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 49
Thank you! Ludovico Caldara Computing Engineer IT Department, CERN ludovico.caldara@cern.ch @ludodba 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 50

More Related Content

PDF
Effective Oracle Home Management - UKOUG_Tech18
Ludovico Caldara
 
PDF
Oracle Drivers configuration for High Availability
Ludovico Caldara
 
PPTX
Oracle Active Data Guard 12cR2. Is it the best option?
Ludovico Caldara
 
PPTX
How to bake a Customer Story with With Windows, NVM-e, Data Guard, ACFS Snaps...
Ludovico Caldara
 
PDF
Oracle Client Failover - Under The Hood
Ludovico Caldara
 
PDF
Oracle Active Data Guard and Global Data Services in Action!
Ludovico Caldara
 
PPTX
Get the most out of Oracle Data Guard - POUG version
Ludovico Caldara
 
PPTX
Database Migration Assistant for Unicode (DMU)
Ludovico Caldara
 
Effective Oracle Home Management - UKOUG_Tech18
Ludovico Caldara
 
Oracle Drivers configuration for High Availability
Ludovico Caldara
 
Oracle Active Data Guard 12cR2. Is it the best option?
Ludovico Caldara
 
How to bake a Customer Story with With Windows, NVM-e, Data Guard, ACFS Snaps...
Ludovico Caldara
 
Oracle Client Failover - Under The Hood
Ludovico Caldara
 
Oracle Active Data Guard and Global Data Services in Action!
Ludovico Caldara
 
Get the most out of Oracle Data Guard - POUG version
Ludovico Caldara
 
Database Migration Assistant for Unicode (DMU)
Ludovico Caldara
 

What's hot (20)

PDF
Rapid Home Provisioning
Ludovico Caldara
 
PPTX
Get the most out of Oracle Data Guard - OOW version
Ludovico Caldara
 
PDF
Oracle GoldenGate for Oracle DBAs
Guatemala User Group
 
PPTX
Adaptive Features or: How I Learned to Stop Worrying and Troubleshoot the Bomb.
Ludovico Caldara
 
PDF
GoldenGate and Oracle Data Integrator - A Perfect Match...
Michael Rainey
 
PDF
Migration to Oracle Multitenant
Jitendra Singh
 
PDF
Oracle Rac Performance Tunning Tips&Tricks
Zekeriya Besiroglu
 
PPTX
Examining Oracle GoldenGate Trail Files
Bobby Curtis
 
PPT
Real-Time Data Replication to Hadoop using GoldenGate 12c Adaptors
Michael Rainey
 
PDF
Oracle RAC, Data Guard, and Pluggable Databases: When MAA Meets Multitenant (...
Ludovico Caldara
 
PDF
Maximum Availability Architecture - Best Practices for Oracle Database 19c
Glen Hawkins
 
PDF
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Nelson Calero
 
PDF
Oracle12c data guard farsync and whats new - Nassyam Basha
pasalapudi123
 
PDF
Oracle RAC, Oracle Data Guard, and Pluggable Databases: When MAA Meets Oracle...
Ludovico Caldara
 
PDF
Oracle Active Data Guard 12c: Far Sync Instance, Real-Time Cascade and Other ...
Ludovico Caldara
 
PDF
GoldenGate and ODI - A Perfect Match for Real-Time Data Warehousing
Michael Rainey
 
PDF
GoldenGate and Oracle Data Integrator - A Perfect Match- Upgrade to 12c
Michael Rainey
 
PPTX
Oracle Database 12.1.0.2: New Features
Deiby Gómez
 
PPTX
Enable GoldenGate Monitoring with OEM 12c/JAgent
Bobby Curtis
 
PDF
OOW 17 - database consolidation using the oracle multitenant architecture
Pini Dibask
 
Rapid Home Provisioning
Ludovico Caldara
 
Get the most out of Oracle Data Guard - OOW version
Ludovico Caldara
 
Oracle GoldenGate for Oracle DBAs
Guatemala User Group
 
Adaptive Features or: How I Learned to Stop Worrying and Troubleshoot the Bomb.
Ludovico Caldara
 
GoldenGate and Oracle Data Integrator - A Perfect Match...
Michael Rainey
 
Migration to Oracle Multitenant
Jitendra Singh
 
Oracle Rac Performance Tunning Tips&Tricks
Zekeriya Besiroglu
 
Examining Oracle GoldenGate Trail Files
Bobby Curtis
 
Real-Time Data Replication to Hadoop using GoldenGate 12c Adaptors
Michael Rainey
 
Oracle RAC, Data Guard, and Pluggable Databases: When MAA Meets Multitenant (...
Ludovico Caldara
 
Maximum Availability Architecture - Best Practices for Oracle Database 19c
Glen Hawkins
 
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Nelson Calero
 
Oracle12c data guard farsync and whats new - Nassyam Basha
pasalapudi123
 
Oracle RAC, Oracle Data Guard, and Pluggable Databases: When MAA Meets Oracle...
Ludovico Caldara
 
Oracle Active Data Guard 12c: Far Sync Instance, Real-Time Cascade and Other ...
Ludovico Caldara
 
GoldenGate and ODI - A Perfect Match for Real-Time Data Warehousing
Michael Rainey
 
GoldenGate and Oracle Data Integrator - A Perfect Match- Upgrade to 12c
Michael Rainey
 
Oracle Database 12.1.0.2: New Features
Deiby Gómez
 
Enable GoldenGate Monitoring with OEM 12c/JAgent
Bobby Curtis
 
OOW 17 - database consolidation using the oracle multitenant architecture
Pini Dibask
 
Ad

Similar to Effective Oracle Home Management in the new Release Model era (20)

PDF
2024_05_16_doag_out-of-place_patching.pdf
Abilash868456
 
PDF
Out-of-Place Oracle Database Patching and Provisioning Golden Images
Alireza Kamrani
 
PDF
upgrade_migrate_consolidate_19999c (1).pdf
tricantino1973
 
PPTX
SAOUG 2018 - Rapid Home Provisioning
Ian Baugaard
 
PPTX
Meetup Oracle Database MAD_BCN: 1.3 Gestión del ciclo de vida de Oracle Datab...
avanttic Consultoría Tecnológica
 
PDF
Streamline it management
DLT Solutions
 
PDF
Data Base Upgrade
guest362312
 
PPTX
Database Upgrades Automation using Enterprise Manager 12c
Hari Srinivasan
 
PDF
Release and patching strategy
Jitendra Singh
 
PDF
11158682984719608417 emea11 fallback
Locuto Riorama
 
PDF
11158682984719608417 emea11 fallback
Locuto Riorama
 
PPT
Less18 support
Amit Bhalla
 
PDF
Suffering from Chronic Patching Pain? Get Relief with Fleet Maintenance [CON6...
Timothy Schubert
 
PPTX
OEM13c_PPT.pptx
SDPL Technologies
 
PPTX
Hyderabad Mar 2019 - Autonomous Database
Connor McDonald
 
PDF
ILOUG 2019 - Autonomous, what does it mean for DBAs
Connor McDonald
 
PPTX
Perth APAC Groundbreakers tour - The Autonomous Database
Connor McDonald
 
PDF
Mike_Salt.pdf
SayedMahfouz3
 
PDF
Fujitsu Services _Large Scale EBS 12.2 Upgrade Licking the Wounds_Mike_Salt.pdf
Kommaneni Sreenivasulu
 
PDF
Living with the Oracle Database Appliance
Simon Haslam
 
2024_05_16_doag_out-of-place_patching.pdf
Abilash868456
 
Out-of-Place Oracle Database Patching and Provisioning Golden Images
Alireza Kamrani
 
upgrade_migrate_consolidate_19999c (1).pdf
tricantino1973
 
SAOUG 2018 - Rapid Home Provisioning
Ian Baugaard
 
Meetup Oracle Database MAD_BCN: 1.3 Gestión del ciclo de vida de Oracle Datab...
avanttic Consultoría Tecnológica
 
Streamline it management
DLT Solutions
 
Data Base Upgrade
guest362312
 
Database Upgrades Automation using Enterprise Manager 12c
Hari Srinivasan
 
Release and patching strategy
Jitendra Singh
 
11158682984719608417 emea11 fallback
Locuto Riorama
 
11158682984719608417 emea11 fallback
Locuto Riorama
 
Less18 support
Amit Bhalla
 
Suffering from Chronic Patching Pain? Get Relief with Fleet Maintenance [CON6...
Timothy Schubert
 
OEM13c_PPT.pptx
SDPL Technologies
 
Hyderabad Mar 2019 - Autonomous Database
Connor McDonald
 
ILOUG 2019 - Autonomous, what does it mean for DBAs
Connor McDonald
 
Perth APAC Groundbreakers tour - The Autonomous Database
Connor McDonald
 
Mike_Salt.pdf
SayedMahfouz3
 
Fujitsu Services _Large Scale EBS 12.2 Upgrade Licking the Wounds_Mike_Salt.pdf
Kommaneni Sreenivasulu
 
Living with the Oracle Database Appliance
Simon Haslam
 
Ad

More from Ludovico Caldara (12)

PDF
Oracle Fleet Patching and Provisioning Deep Dive Webcast Slides
Ludovico Caldara
 
PDF
Oracle Drivers configuration for High Availability, is it a developer's job?
Ludovico Caldara
 
PDF
Long live to CMAN!
Ludovico Caldara
 
PDF
Let your DBAs get some REST(api)
Ludovico Caldara
 
PDF
ADAPTIVE FEATURES OR: HOW I LEARNED TO STOP WORRYING AND TROUBLESHOOT THE BOMB
Ludovico Caldara
 
PPTX
Migrating to Oracle Database 12c: 300 DBs in 300 days.
Ludovico Caldara
 
PDF
Oracle Database on ACFS: a perfect marriage?
Ludovico Caldara
 
PDF
ORACLE 12C DATA GUARD: FAR SYNC, REAL-TIME CASCADE STANDBY AND OTHER GOODIES
Ludovico Caldara
 
PDF
Oracle RAC 12c and Policy-Managed Databases, a Technical Overview
Ludovico Caldara
 
PDF
Boost your Oracle RAC manageability with Policy-Managed Databases
Ludovico Caldara
 
PDF
RACAttack 12c Advanced Lab: Server Pools and Policy-managed databases
Ludovico Caldara
 
PDF
Galera Cluster: Synchronous Multi-Master Replication for MySQL HA
Ludovico Caldara
 
Oracle Fleet Patching and Provisioning Deep Dive Webcast Slides
Ludovico Caldara
 
Oracle Drivers configuration for High Availability, is it a developer's job?
Ludovico Caldara
 
Long live to CMAN!
Ludovico Caldara
 
Let your DBAs get some REST(api)
Ludovico Caldara
 
ADAPTIVE FEATURES OR: HOW I LEARNED TO STOP WORRYING AND TROUBLESHOOT THE BOMB
Ludovico Caldara
 
Migrating to Oracle Database 12c: 300 DBs in 300 days.
Ludovico Caldara
 
Oracle Database on ACFS: a perfect marriage?
Ludovico Caldara
 
ORACLE 12C DATA GUARD: FAR SYNC, REAL-TIME CASCADE STANDBY AND OTHER GOODIES
Ludovico Caldara
 
Oracle RAC 12c and Policy-Managed Databases, a Technical Overview
Ludovico Caldara
 
Boost your Oracle RAC manageability with Policy-Managed Databases
Ludovico Caldara
 
RACAttack 12c Advanced Lab: Server Pools and Policy-managed databases
Ludovico Caldara
 
Galera Cluster: Synchronous Multi-Master Replication for MySQL HA
Ludovico Caldara
 

Recently uploaded (20)

PPTX
O2C Customer Invoices to Receipt V15A.pptx
ssuserbfa915
 
PDF
sustainability-14-14877-v2.pddhzftheheeeee
VenkateshGovindaraja9
 
PPTX
observCloud-Native Containerability and monitoring.pptx
anupsingh76937
 
PDF
Five Habits of High-Impact Board Members
OnBoard
 
PPT
What is a Computer? Input Devices /output devices
GulJan8
 
PDF
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
PDF
Unlock new opportunities with location data.pdf
Precisely
 
PPTX
Benefits of Physical activity for teenagers.pptx
Nokwanda Thabethe
 
PDF
DP Operators-handbook-extract for the Mautical Institute
LeonelMejiaLarios
 
PDF
1 - Historical Antecedents, Social Consideration.pdf
tuazon2030627
 
PDF
A contest of sentiment analysis: k-nearest neighbor versus neural network
IAESIJAI
 
PDF
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
IAESIJAI
 
PPTX
Chapter 5: Probability Theory and Statistics
RandyFin
 
PDF
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
PDF
Univ-Connecticut-ChatGPT-Presentaion.pdf
ry7r52mzb4
 
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
ssuser28425f
 
PDF
Zenith AI: Advanced Artificial Intelligence
Biz Preneurs
 
PDF
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
IAESIJAI
 
PDF
NewMind AI Weekly Chronicles – August ’25 Week III
NewMind AI
 
O2C Customer Invoices to Receipt V15A.pptx
ssuserbfa915
 
sustainability-14-14877-v2.pddhzftheheeeee
VenkateshGovindaraja9
 
observCloud-Native Containerability and monitoring.pptx
anupsingh76937
 
Five Habits of High-Impact Board Members
OnBoard
 
What is a Computer? Input Devices /output devices
GulJan8
 
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
Unlock new opportunities with location data.pdf
Precisely
 
Benefits of Physical activity for teenagers.pptx
Nokwanda Thabethe
 
DP Operators-handbook-extract for the Mautical Institute
LeonelMejiaLarios
 
1 - Historical Antecedents, Social Consideration.pdf
tuazon2030627
 
A contest of sentiment analysis: k-nearest neighbor versus neural network
IAESIJAI
 
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
IAESIJAI
 
Chapter 5: Probability Theory and Statistics
RandyFin
 
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
Univ-Connecticut-ChatGPT-Presentaion.pdf
ry7r52mzb4
 
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
ssuser28425f
 
Zenith AI: Advanced Artificial Intelligence
Biz Preneurs
 
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
IAESIJAI
 
NewMind AI Weekly Chronicles – August ’25 Week III
NewMind AI
 

Effective Oracle Home Management in the new Release Model era

  • 2. Effective Oracle Home Management with the New Release Model Ludovico Caldara Computing Engineer IT Department, CERN ludovico.caldara@cern.ch @ludodba 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 2
  • 3. About Ludovico Caldara ■ 18+ Years DBA (Not Only Oracle) ■ ITOUG co-founder ■ OCP (11g, 12c, MySQL) & OCE ■ Italian living in Switzerland ■ http://www.ludovicocaldara.net ■ @ludodba ■ ludovicocaldara 05.12.2018 3 Effective Oracle Home Management #UKOUG_Tech18
  • 4. CERN • CERN - European Laboratory for Particle Physics • Founded in 1954 by 12 Countries for fundamental physics research in the post-war Europe • Today 22 member states and world-wide collaborations • About ~1100 MCHF yearly budget • 2’300 CERN personnel • 10’000 users from 110 countries 405.12.2018 Effective Oracle Home Management #UKOUG_Tech18
  • 5. Fundamental Research • What is 95% of the Universe made of? • Why do particles have mass? • Why is there no antimatter left in the Universe? • What was the Universe like, just after the "Big Bang"? 505.12.2018 Effective Oracle Home Management #UKOUG_Tech18
  • 6. Effective Oracle Home Management #UKOUG_Tech18 6 The Large Hadron Collider (LHC) Largest machine in the world 27km, 6000+ superconducting magnets Emptiest place in the solar system High vacuum inside the magnets Hottest spot in the galaxy During Lead ion collisions create temperatures 100 000x hotter than the heart of the sun Fastest racetrack on Earth Protons circulate 11245 times/s (99.9999991% the speed of light)
  • 7. “Patch soon, patch often” vs reality 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 7
  • 8. Frequent patching? • Patching is time consuming • It is complex • It introduces some risks • It is not always really necessary • It leads to human errors 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 8
  • 9. How many bugs could you encounter? $ opatch lspatches 27923320;Database PSU 12.1.0.2.180717, Oracle JavaVM Component (JUL2018) 22243983; 27762253;OCW PATCH SET UPDATE 12.1.0.2.180717 (27762253) 27547374;Database Bundle Patch : 12.1.0.2.180717 (27547374) OPatch succeeded. $ cd $ORACLE_HOME/inventory $ grep -r "bug description" * | wc -l 2132 $ grep -r "bug description" * | grep -i "wrong result" | wc -l 66 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 9
  • 10. Quick overview of the New Release Model 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 10
  • 11. What does it change? Jan 18 Apr Jul Oct Jan 19 Apr Jul Oct RU 18.1.0 18.2.0 18.3.0 18.4.0 18.5.0 18.6.0 18.7.0 18.8.0 RUR1 18.2.1 18.3.1 18.4.1 18.5.1 18.6.1 18.7.1 RUR2 18.2.2 18.3.2 18.4.2 18.5.2 18.6.2 RU 19.1.0 19.2.0 19.3.0 19.4.0 RUR1 19.2.1 19.3.1 RUR2 19.2.2 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 11 2
  • 12. What does it change? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 12 Short Development cycle Frequent Upgrades And Updates
  • 13. Which patches should I apply? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 13
  • 14. Brave vs Fearful 12.1 >= 12.2 Effective Oracle Home Management #UKOUG_Tech18 14 How do I feel? PSU Bundle Patch BraveFearful How do I feel? Release Update Revision Release Update BraveFearful • 05.12.2018
  • 15. Brave vs Fearful 12.1 >= 12.2 Effective Oracle Home Management #UKOUG_Tech18 15 How do I feel? PSU Bundle Patch BraveFearful How do I feel? Release Update Revision Release Update BraveFearful Ludovico(*) Ludovico(*) (*) yes, it’s Comic Sans https://mikedietrichde.com/2017/10/24/differences-psu-bp-ru-rur/ • 05.12.2018
  • 16. Common patching patterns 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 16
  • 17. Common worst practices in maintaining homes • Installing software by hand • Different patch levels for Oracle Homes with the same path • Not tracking the installed patches • Oracle Home paths hard-coded in the operational scripts • Not minding about naming conventions 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 17
  • 18. Worst approach (big bang) 1. Stop everything 2. In-place binaries patching 3. Database patching, all at once 4. Start everything Effective Oracle Home Management #UKOUG_Tech18 18 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP UP DOWN UP STOP ALL START ALLPATCH BINARIES PATCH DBs • 05.12.2018
  • 19. Worst approach (big bang) 1. Stop everything 2. In-place binaries patching 3. Database patching, all at once 4. Start everything Effective Oracle Home Management #UKOUG_Tech18 1905.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP UP DOWN UP STOP ALL START ALLPATCH BINARIES PATCH DBs No easy rollback Long downtime Complex planning
  • 20. Different approach (still bad) 1. Install new binaries manually 2. Patch the new binaries 3. Stop, change OH, patch databases one by one 4. Decommission old binaries Effective Oracle Home Management #UKOUG_Tech18 2005.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP PATCH BINARIES STOP, MOVE, PATCH ONE BY ONE
  • 21. Different approach (still bad) 1. Install new binaries manually 2. Patch the new binaries 3. Stop, change OH, patch databases one by one 4. Decommission old binaries Effective Oracle Home Management #UKOUG_Tech18 2105.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP PATCH BINARIES STOP, MOVE, PATCH ONE BY ONE Complex to do manually Repetitive task when many servers
  • 22. Automate the installation! • Automated creation 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 22 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply
  • 23. Image repo Automate the installation! • Automated creation • Software cloning 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 23 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply 18.3.1 18.4.0 Server 18.3.1
  • 24. Image repo Automate the installation! • Automated creation • Software cloning 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 24 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply 18.3.1 18.4.0 Server 18.3.1 RUNDECK ANSIBLE RHP OTHER
  • 25. Less paths Effective Oracle Home Management #UKOUG_Tech18 2505.12.2018 18.3.0 18.4.0 18.4.0 + one-off X 18.5.0 + one-off X 18.5.0 + one-off X + one-off Y 18.3.0 18.4.0 18.4.0 + one-off X 18.5.0 + one-off X 18.5.0 + one-off X + one-off Y 18.4.1 18.4.1 + one-off Y GOOD: BAD:
  • 26. Workarounds vs one-offs • Sometimes through underscore parameters • Mitigate the number of one-offs • Less merge requests • Document it! (why, when, who) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 26 1 2 3 alter system set "_px_groupby_pushdown"=off comment='Ludo, 03.05.16: W/A for bug 18499088' scope=both sid='*‘;
  • 27. Rapid Home provisioning • It might be the perfect product… • Requires Lifecycle Management Pack • The technology stack is complex (Clusterware, ASM, ACFS, NFS, GNS…) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 27
  • 28. Custom approach • Simple script • Golden images • Solid naming conventions • I blogged about it! http://www.ludovicocaldara.net/dba/oh-mgmt-[1..7]/ 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 28
  • 29. Implementation at Etat de Vaud 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 29
  • 30. Naming Conventions • Central Inventory contains OH name • OH name matches OH location • No dots (.) allowed (inventory limitation) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 30 Central Inventory <HOME NAME=“18_3_0" LOC="/u01/app/oracle/product/18_3_0" TYPE="O" IDX="2"> • 1 to 1 mapping: easy to code • 18c read-only OH uses the name as subdir in `orabasehome`: consistent naming!
  • 31. Golden images 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 31 Golden Images Store 18_3_0.zip 18_4_0.zip 18_3_1.zip SERVER1 18_3_0.zip 18_3_1.zip SERVER2 18_3_1.zip SERVER3 18_4_0.zip Golden Image Metadata Name Path ------- ----------------- 18_3_0 /store/18_3_0.zip 18_3_1 /store/18_3_1.zip 18_4_0 /store/18_4_0.zip
  • 32. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 33 SERVER1 18_3_0 SERVER2… First Install
  • 33. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 34 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -c -n 18_3_0
  • 34. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 35 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_3_018_3_0
  • 35. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 36 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_3_0 -n 18_4_0 18_3_0 18_4_0
  • 36. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 37 18_3_0.zip SERVER1 18_3_0 SERVER2… Manual patching 18_3_0 18_4_0
  • 37. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 38 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -c 18_3_0 18_4_0 18_4_0.zip
  • 38. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 39 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_4_0 18_3_0 18_4_0 18_4_0.zip 18_4_0
  • 39. What it actually does Creation workflow • Copy the Home • Clean it up • Create the zip • Store the zip in the repo • Store metadata Installation workflow • Security checks • Create new path • Unzip from the store • Attach home • Relink (opt. with RAC) • Run setasmgid • Other stuff (e.g. TNS) Effective Oracle Home Management #UKOUG_Tech18 4005.12.2018
  • 40. Effective Oracle Home Management #UKOUG_Tech18 4105.12.2018 DEMO
  • 41. Further development at CERN 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 42
  • 42. Read-only Oracle Home $ roohctl –enable Enabling Read-Only Oracle home. Update orabasetab file to enable Read-Only Oracle home. Orabasetab file has been updated successfully. Create bootstrap directories for Read-Only Oracle home. Bootstrap directories have been created successfully. Bootstrap files have been processed successfully. Read-Only Oracle home has been enabled successfully. Check the log file /u01/app/oracle/cfgtoollogs/roohctl/roohctl- 180822AM102911.log. 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 43
  • 43. AGILE INFRASTRUCTURE SERVER Insertion in the Agile Infrastructure 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 44 Image Store Scripts REST client Oracle Home
  • 44. Grid Infrastructure Installation • Puppet + Rundeck for automatic GI install • Patched Grid Infrastructure versions beside the first one https://www.markusdba.net/?p=294 • Out-of-place patching with 18c independent local-mode automaton 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 45 $ rhpctl move gihome –sourcehome old_path -destinationhome new_path
  • 45. Independent Local-mode Automaton (18c) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 46 # $GI_HOME/srvctl enable volume -volume GHCHKPT -diskgroup mgmt # $GI_HOME/srvctl enable filesystem -volume GHCHKPT -diskgroup mgmt # $GI_HOME/srvctl start filesystem -volume GHCHKPT -diskgroup mgmt $ $GI_HOME/srvctl start rhpserver Local Resources --------------------------------------------------------------- ora.MGMT.GHCHKPT.advm ora.mgmt.ghchkpt.acfs ora.helper Cluster Resources --------------------------------------------------------------- ora.rhpserver
  • 46. Independent Local-mode Automaton (18c) • Damn-easy move of GI HOME • Automatic resource relocation • Enable Grid Infrastructure Golden Images • Patching (apply and rollback) is done automatically 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 47
  • 47. Back to the demo… 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 48
  • 48. Questions? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 49
  • 49. Thank you! Ludovico Caldara Computing Engineer IT Department, CERN ludovico.caldara@cern.ch @ludodba 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 50