Experiences of SOACS

Copyright © 2014, eProseed and/or its affiliates. All rights reserved. | Confidential
Implementing the UK’s 1st Production
SOA Cloud Service
Kiran Tailor
Global Lead DBA / BI Solutions Architect, CIMA
Simon Haslam
Technical Director, eProseed
1

© CIMA 2
Me and CIMA
• Kiran Tailor , Global Lead DBA/BI Solutions Architect
• Blog : http://blog.puredba.co.uk/
•Twitter : @KiranTailorUK
CIMA and the AICPA are joining forces to create a new
association to represent the entire breadth of the
accounting profession.
*
600,000 members and students worldwide.
*Offices and Staff across the world.
*

Simon Haslam
Technical Director, eProseed UK
• Platform / Infrastructure Architect
• Using Oracle products since ~1995 (Oracle7)
• Formerly UKOUG App Server & Middleware SIG Chair

Copyright © 2016 eProseed and its affiliates. All rights reserved.
© CIMA 4
• Our Previous and New Architectures
• Provisioning DBCS and SOACS
• Tailoring Cloud Services
• Experiences and Oracle Cloud Tips
• Monitoring in the Cloud and Reporting

Our Previous System – All Running On-Premises
Oracle
Databases
CRM
Biztalk
GP
SQL Server

Hybrid Cloud
Public Cloud
Private Cloud
New System
Siebel
CRM
Oracle
Databases
Exalytics
Oracle ERP
Oracle
DBCS
SOACS

Oracle Database Cloud Service
Oracle SOA Cloud Services
Financials
Our Integration
SOA Composites
Fusion Cloud
Services
ConnectivityServices
SOA
Infra
Integration
Tables
Customer Processes Invoice Processes
Activity Services
On-Premises
Payment Processes
ConnectivityServices
SQL
Server
© CIMA
Batch
UpdateNotifications  Fault
Handling

WHAT THIS PRESENTATION IS IS NOT
• This is not a step-by-step guide about provisioning SOACS
or DBCS manually through console
• We focus on some of the decisions you have to make,
lessons learnt and tips
• You will notice some of these are same as for on-prem…
many of your old skills are still useful 

TOPOLOGIES
• Before eProseed CIMA were in development as
single instance SOACS:
–WebLogic Managed Server directly open to internet
–No clustering / failover options (other than VM restart)
–Can’t easily add OTD later without re-provisioning
–Not really an ‘Oracle intended’ design for production
10
SOA
WebLogic
DBCS

TOPOLOGIES
• So… we introduced OTD
–OTD is only internet-facing component
• Hardened, good track record/few security patches
• Option for WAF features etc
–Option to add second OTD node
• Note: actually independent – different to on-prem config.
–Allows future scale-out and rolling patching without
changing any end points
• Scale out could even be done online
11
SOA
WebLogic
DBCS
Oracle
Traffic
Director

TOPOLOGIES – ACROSS ENVIRONMENTS
• Hard to justify OTD cost
on Dev but we want all
envs to behave the same
• Wire Dev SOACS MS into
Test OTD - unsupported
but quite practical
12
SOA
WebLogic
DBCS
Oracle
Traffic
Director
PROD
SOA
WebLogic
DBCS
Oracle
Traffic
Director
TEST
SOA
WebLogic
DBCS
DEV

STORAGE TIPS
• Both DBCS and SOACS need Oracle Cloud Storage for backups
– Note: SOACS runs on “full PaaS” JCS – no virtual image option
• Provisioning containers – how to subdivide & manage storage?
– We provision per instance – exactly same name for simplicity
– Drop storage with instance (if you don’t want it make sure you don’t pay for old backups)
• Initial Sizing
– Size bigger rather than smaller
– Can extend in DBCS later though but you do need a restart of the database

OPC USER TIPS
1. ONLY use lower case email addresses. If mixed case:
– Some displays, e.g. user listing, show it as lower case
– Some interfaces are happy with lower case, (probably authentication) insist on it mixed
2. Use a non-email name for the primary provisioning account
– All objects are created with this in their identifier, e.g.
• Better to have to reference /Compute_exampleco/exampleco/…
• Than /Compute_exampleco/jane.smith@example.com/…
• (especially with multiple admins, plus admins leave)
3. Limit the use of the provisioning/owner account to the provisioning tools
– All admins should have their own logins
14

TLS (SSL)
• SOA CS (JCS) – WebLogic and OTD has SSL configured but only with demo certificates
=> not suitable for production usage, especially OTD since that’s public
• Inputs to configuring SSL:
– Signed certificate including any intermediates: either by public CA (simplest) or internal CA
– Private key: keep it safe, but keep it outside SOACS for simplicity (to save re-signing… at least for now)
– Any additional certificates to be trusted: typically the internal CA
• We currently switch from OPSS Key Store Service to Identity and Trust JKS on disk
– In domain Custom Identiy & Cutom Trust | JKS | <DOMAIN_HOME>/security/trust-v5.jks
– setUserEnv.sh  -Djavax.net.ssl.trustStore=${DOMAIN_HOME}/security/trust-v5.jks
– (we may consider OPSS later since java can reference KSS too)
15
eProseed Accelerator for Oracle Cloud
configures your SSL for OTD HTTPS listener,
Admin Server & Node Manager listening port

SSL TRUST TIP
16
DO NOT INSTALL CERTS FROM 3rd PARTY SERVICE PROVIDERS
… YOU ARE PROBABLY MISSING SOMETHING!
Otherwise you will build a brittle
integration that could break outside
of your control
Beware blogs and even MOS notes
or SRs advising otherwise!!!
Cloud providers typically use wildcard certificates.
Wildcards are not enabled out of the box
See next slide…

WEBLOGIC CONFIGURATION FOR WILDCARD SSL
• When checking SSL certificate, WebLogic looks at the CN
– CN = login.eproseed.com <= this works fine by default
– CN = *.eproseed.com <= this is a wildcard certificate
• By default, even in 12.2.x WebLogic will not trust wildcards or certificate SANs
– Use this weblogic.security.utils.SSLWLSWildcardHostnameVerifier in domain Custom Hostname
Verifier for Admin Server and all Managed Servers.
– Don’t ask me why this is not default by now!
• This hostname verifier has been available since 10.3.6 (probably backports for earlier,
otherwise roll your own)
• Same is true for Subject Alterative Names (SANs) which is what we’re supposed to use
instead of wildcards

AUTOMATED PROVISIONING – REST API
• All operations (& more) that can be done through console can also be done through
REST APIs
• Can call REST API via most modern tools, e.g. curl, Postman in Firefox etc
• Authentication
– Most APIs use header tokens
– Compute Service APIs need you to call authentication first & get a token
18

REST API
19

EXAMPLE PAYLOAD
{
"serviceName": "soacs-unit-test-1",
"level": "PAAS",
"topology": "soa",
"trial": false,
"subscriptionType": "MONTHLY",
"description": "soacs-unit-test-1",
"provisionOTD": true,
"cloudStorageContainer":"Storage-orclnnsoa/soabackup",
"cloudStorageUser":"soacs.Storageadmin",
"cloudStoragePassword":"welcome1",
"parameters": [
<see next slide>
]
}

EXAMPLE PAYLOAD (CONTD.)
{
"version": "12.1.3",
"edition": "SUITE",
"managedServerCount": "1",
"templates": "full",
"adminUserName": "weblogic",
"adminPassword": "welcome1",
"connectString": "example.com:1535:orcl12c",
"dbaName": "sys",
"dbaPassword": "fmwpwd1",
"shape": "oc3m",
"VMsPublicKey": "ssh-rsa ...",
"type": "weblogic"
},{
"listenerPortsEnabled": true,
"loadBalancingPolicy": "LEAST_CONNECTION_COUNT",
"otdAdminUserName": "otdadmin",
"otdAdminPassword": "welcome1",
"shape": "oc3",
"type": "OTD"
}

IaaS VM
EPROSEED ACCELERATOR FOR ORACLE CLOUD
22
SOACS
Oracle Cloud Service Managers
JCS
DBCS
Storage
Network
Compute
Identity
eProseed
Accelerator
for Oracle Cloud
 Creation
Virtual
Machine
Virtual
Machine
Virtual
Machine
Config. Report


AUTOMATION & PUTTING IT ALL TOGETHER
23
Env master
JSON
Blueprint
JSON
Oracle Cloud
config
JSON
Cookbook
(Recipes)
Payload
JSON
Env type
Env no
Blueprint type
DBCS override
Instances
Runlist
Sizing
etc
OPC DC
API URLs
Customer Details
JSON
Domain, Auth, DC
REST
calls

TAILORING CLOUD SERVICES
• Additional SSH keys
• User accounts (Linux & WebLogic)
• Firewall / VPN
• (Backup schedule)
25

User and Key Management
# useradd ukoug
# mkdir /home/ukoug/.ssh
The next step we copy and paste in the public key we have generated for
the new user:
# echo "<key here ssh-rsa…" > /home/ukoug/.ssh/authorized_keys
To check
# cat /home/ukoug/.ssh/authorized_keys

SSH TUNNELLING IS 1ST STEP BUT SEE…
Tomorrow, 11:35, Hall 6B

FAILING BACKUPS
1. Domain locks
2. Admin user password change
34

1. FAILING BACKUPS IF ADMIN HAS LOCK
ON WEBLOGIC CONFIGURATION
• Backup tool takes a domain lock at start, releases at end
– Primarily (presumably) to stop you changing domain during backup
– Also if an administrator has started a session there could be changes in
configuration that have not yet been activated
• If an Admin has the lock then the backup tool can’t get it
A better approach might be just to force discard of changes &
release of lock? Debatable.
35

Backups – Domain Configuration Lock

2. FAILING BACKUPS AFTER PASSWORD CHANGES
• OPC Passwords (i.e. those in an identity domain) expire monthly
• You can change the password when logging into the cloud console
• But…
39
DON’T IGNORE
THESE EMAILS!

Backups – Password Changed
• Same change in DBCS

Backups – Password Changed
• Update the Wallet
- sudo –s
- /var/opt/oracle/bkup_api update_wallet --password=new-password
• Validate in oss.cfg

EM CC MONITORING
44 DB CS instance
SOA CS instance
44
OTD
SOA
Oracle
Financials
Cloud (ERP)
DB EE
CIMA Data Centre
SQL Server
Paypal
etc
Siebel
EM 13c
Cloud Control
Hybrid Cloud
Gateway
EM
Agent
EM
Agent
EM
Agent
• Prod
• Test
• Dev

MONITORING & NOTIFICATIONS
• Target types
– SOA
– WLS
– OTD
– Database
– Hosts
– Exalytics
– TimesTen
45

ALERTING
46

REPORTING
• Daily Report about integrations
• Scheduled emails to IT Team/Managers
• Automatically raises incidents (check?)
47

EXPERIENCE & LEARNING POINTS
• Create a Hybrid Cloud Agent EM user and private key (ssh-keygen -b 2048 -t rsa)
• Add the EM Server (IP Hostname) to host
• Bug 23013302 in EM OPC Agent - 'fproxy forwarder' process is utilizing 95 to 100% CPU
• According to Oracle certifications SOACS 12.1.3 is not supported with EM 13.1
• Error Hospital Fix – Copy Jar files manually
• Weblogic Domain Refresh
• OTD SNMP Port for availability status?/ Firewall
• Downtime (Oracle Patching and Changes)
48
Now certified
Fixed in 13.2

© CIMA 50
Benefits
• Provisioning – 50% Faster
• Enhanced monitoring – 20% Reduction in Man-hours
• 50% Reduction in TX completion time
• Security
• Backups
• Scale Up – Scale Down
• Reduce Head Count ??

謝謝！Спасибо!
Dziękujemy!
‫لك‬ ‫شكرا‬!
Thank you!
Kiran Tailor
kiran.tailor@cimaglobal.com

HOW TO CONTACT US
53
@simon_haslam@kirantailoruk

Experiences of SOACS

More Related Content

What's hot (20)

Viewers also liked (13)

Similar to Experiences of SOACS (20)

More from Simon Haslam (17)

Recently uploaded (20)

Experiences of SOACS