BPEL PM 11g performance tuning - 1

Contents
LOAD & PERFORMANCE TESTING ..................................................................................................... 16
1 STRESS, LOAD, SOAK, SPIKE TESTS LOAD PROFILES ................................................................................. 17
2 LNP TESTING LIFE CYCLE................................................................................................................... 18
3 EXAMPLES OF LNP TESTING OBJECTIVES ............................................................................................... 19
3.1 RESPONSE TIME...................................................................................................................................... 19
3.2 RELIABILITY............................................................................................................................................ 19
3.3 CONFIGURATION SIZING ........................................................................................................................... 19
3.4 CAPACITY PLANNING ............................................................................................................................... 19
3.5 REGRESSION .......................................................................................................................................... 20
3.6 BOTTLENECK IDENTIFICATION ................................................................................................................... 20
3.7 SCALABILITY ........................................................................................................................................... 20
BPEL PM: INTRODUCTION ............................................................... ERROR! BOOKMARK NOT DEFINED.
1 BPEL PM PROCESS TYPES ....................................................................... ERROR! BOOKMARK NOT DEFINED.
1.1 BY INTERFACE ........................................................................................... ERROR! BOOKMARK NOT DEFINED.
1.2 BY DURABILITY .......................................................................................... ERROR! BOOKMARK NOT DEFINED.
2 TERMINOLOGY ...................................................................................... ERROR! BOOKMARK NOT DEFINED.
3 DIFFERENCE BETWEEN ASYNC & SYNC BPEL .............................................. ERROR! BOOKMARK NOT DEFINED.
4 SOA 11G CONFIGURATION FILES AND LOG FILES LOCATION ............................. ERROR! BOOKMARK NOT DEFINED.
5 DEFAULT DATA SOURCES ......................................................................... ERROR! BOOKMARK NOT DEFINED.
6 PERFORMANCE OBJECTIVE ....................................................................... ERROR! BOOKMARK NOT DEFINED.
7 WHAT TO TUNE .................................................................................... ERROR! BOOKMARK NOT DEFINED.
8 SYSTEM SETUP ...................................................................................... ERROR! BOOKMARK NOT DEFINED.
9 IMPORTANT POINTS ............................................................................... ERROR! BOOKMARK NOT DEFINED.
10 REFERENCE......................................................................................... ERROR! BOOKMARK NOT DEFINED.
JVM TUNING ................................................................................... ERROR! BOOKMARK NOT DEFINED.
1 PAUSE TIMES AND THROUGHPUT .............................................................. ERROR! BOOKMARK NOT DEFINED.
1.1 GARBAGE COLLECTION VS. THROUGHPUT ...................................................... ERROR! BOOKMARK NOT DEFINED.
1.2 DEFRAGMENTATION PAUSES AND THROUGHPUT ............................................ ERROR! BOOKMARK NOT DEFINED.
2 PERFORMANCE AND MEMORY FOOTPRINT .................................................. ERROR! BOOKMARK NOT DEFINED.
3 JVM CHOICE ........................................................................................ ERROR! BOOKMARK NOT DEFINED.
4 JROCKIT’S TUNING ................................................................................. ERROR! BOOKMARK NOT DEFINED.
4.1 BOARDING INTO SHIP – STEP 1 .................................................................... ERROR! BOOKMARK NOT DEFINED.
4.1.1 Heap Size ............................................................................................ Error! Bookmark not defined.
4.1.2 Garbage Collection............................................................................. Error! Bookmark not defined.
4.1.3 Nursery Size ...................................................................................... Error! Bookmark not defined.

2

4.1.4 Pause Target ...................................................................................... Error! Bookmark not defined.
4.2 LEAVING THE SHORE – STEP 2 ...................................................................... ERROR! BOOKMARK NOT DEFINED.
4.2.1 Lazy Unlocking ................................................................................... Error! Bookmark not defined.
4.2.2 Call Profiling ....................................................................................... Error! Bookmark not defined.
4.2.3 Large Pages ........................................................................................ Error! Bookmark not defined.
4.3 IN THE HIGH SEAS – STEP 3 ......................................................................... ERROR! BOOKMARK NOT DEFINED.
4.3.1 Compaction ........................................................................................ Error! Bookmark not defined.
4.3.2 Thread Local Area size ....................................................................... Error! Bookmark not defined.
5 HOTSPOT’S TUNING ............................................................................... ERROR! BOOKMARK NOT DEFINED.
5.1 GARBAGE COLLECTION STRATEGIES .............................................................. ERROR! BOOKMARK NOT DEFINED.
5.1.1 Serial GC Strategy .............................................................................. Error! Bookmark not defined.
5.1.2 Parallel GC Strategy ........................................................................... Error! Bookmark not defined.
5.1.3 Parallel GC strategy ............................................................................ Error! Bookmark not defined.
6 WHERE TO SPECIFY JVM AND ITS PARAMETERS ............................................ ERROR! BOOKMARK NOT DEFINED.
7 OUTOFMEMORYERROR .......................................................................... ERROR! BOOKMARK NOT DEFINED.
7.1 JAVA.LANG.OUTOFMEMORYERROR: JAVA HEAP SPACE ................................... ERROR! BOOKMARK NOT DEFINED.
7.2 JAVA.LANG.OUTOFMEMORYERROR: PERMGEN SPACE .................................... ERROR! BOOKMARK NOT DEFINED.
7.3 JAVA.LANG.OUTOFMEMORYERROR: UNABLE TO CREATE NEW NATIVE THREAD.... ERROR! BOOKMARK NOT DEFINED.
7.4 JAVA.LANG.OUTOFMEMORYERROR: REQUESTED XXX BYTES FOR CHUNKPOOL::ALLOCATE. OUT OF SWAP SPACE
ERROR! BOOKMARK NOT DEFINED.
7.5 ANALYSIS ................................................................................................. ERROR! BOOKMARK NOT DEFINED.
8 REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED.
WEBLOGIC SERVER TUNING ............................................................ ERROR! BOOKMARK NOT DEFINED.
1 THREAD MANAGEMENT/WORK MANAGER ................................................. ERROR! BOOKMARK NOT DEFINED.
1.1 REQUEST CLASSES ..................................................................................... ERROR! BOOKMARK NOT DEFINED.
1.2 CONSTRAINTS ........................................................................................... ERROR! BOOKMARK NOT DEFINED.
1.3 WORK MANGER TYPES............................................................................... ERROR! BOOKMARK NOT DEFINED.
1.3.1 Default Thread Pool ........................................................................... Error! Bookmark not defined.
1.3.2 SOAWorkManager ............................................................................. Error! Bookmark not defined.
2 NETWORK I/O ...................................................................................... ERROR! BOOKMARK NOT DEFINED.
2.1 MUXERS .................................................................................................. ERROR! BOOKMARK NOT DEFINED.
2.2 CHUNK PARAMETERS ................................................................................. ERROR! BOOKMARK NOT DEFINED.
2.3 CONNECTION BACKLOG BUFFERING .............................................................. ERROR! BOOKMARK NOT DEFINED.
2.4 CACHED CONNECTIONS .............................................................................. ERROR! BOOKMARK NOT DEFINED.
3 TUNING DATA SOURCE CONNECTION POOLS ................................................ ERROR! BOOKMARK NOT DEFINED.
3.1 STATEMENT CACHE .................................................................................... ERROR! BOOKMARK NOT DEFINED.
3.1.1 Usage Restrictions for Statement Cache ........................................... Error! Bookmark not defined.
3.2 CONNECTION TESTING OPTIONS FOR A DATA SOURCE ..................................... ERROR! BOOKMARK NOT DEFINED.
3.2.1 Automatic Testing .............................................................................. Error! Bookmark not defined.
3.3 ENABLING CONNECTION CREATION RETRIES................................................... ERROR! BOOKMARK NOT DEFINED.
3.4 ENABLING CONNECTION REQUESTS TO WAIT FOR A CONNECTION ..................... ERROR! BOOKMARK NOT DEFINED.

3

3.5 XA TRANSACTION TIMEOUT ........................................................................ ERROR! BOOKMARK NOT DEFINED.
3.6 AUTOMATICALLY RECOVERING LEAKED CONNECTIONS ..................................... ERROR! BOOKMARK NOT DEFINED.
3.7 LIMITING STATEMENT PROCESSING TIME ...................................................... ERROR! BOOKMARK NOT DEFINED.
3.8 PINNED TO THREAD PROPERTY .................................................................... ERROR! BOOKMARK NOT DEFINED.
3.9 GRIDLINK ................................................................................................. ERROR! BOOKMARK NOT DEFINED.
3.10 SINGLE CLIENT ACCESS NAME.................................................................... ERROR! BOOKMARK NOT DEFINED.
3.11 LOGGING LAST RESOURCE ......................................................................... ERROR! BOOKMARK NOT DEFINED.
4 DATABASE MODE .................................................................................. ERROR! BOOKMARK NOT DEFINED.
BPEL ENGINE TUNING ..................................................................... ERROR! BOOKMARK NOT DEFINED.
1 REAL TIME VIEWING OF PERFORMANCE METRICS .......................................... ERROR! BOOKMARK NOT DEFINED.
1.1 FUSION MIDDDLEWARE CONTROL ................................................................ ERROR! BOOKMARK NOT DEFINED.
1.2 DYNAMIC MEMORY SERVICE ....................................................................... ERROR! BOOKMARK NOT DEFINED.
2 BPEL ENGINE THREADING MODEL............................................................. ERROR! BOOKMARK NOT DEFINED.
2.1 DISPATCHER MAXIMUM REQUEST DEPTH...................................................... ERROR! BOOKMARK NOT DEFINED.
2.2 INVOKEMESSAGES IN CACHE ....................................................................... ERROR! BOOKMARK NOT DEFINED.
3 TRANSACTION TIMEOUT .......................................................................... ERROR! BOOKMARK NOT DEFINED.
3.1 SYNCMAXWAITTIME ................................................................................. ERROR! BOOKMARK NOT DEFINED.
3.2 BPEL EJBS ............................................................................................... ERROR! BOOKMARK NOT DEFINED.
3.3 GLOBAL TRANSACTION TIMEOUT AT WEBLOGIC DOMAIN LEVEL ........................ ERROR! BOOKMARK NOT DEFINED.
3.4 WEBSERVICE READ AND CONNECTION TIMEOUT ............................................. ERROR! BOOKMARK NOT DEFINED.
3.5 RELATIONSHIP AMONG TIMEOUT PARAMETERS .............................................. ERROR! BOOKMARK NOT DEFINED.
3.6 WLS TIMEOUT PARAMETERS ....................................................................... ERROR! BOOKMARK NOT DEFINED.
3.6.1 Complete Message Timeout .............................................................. Error! Bookmark not defined.
3.6.2 Idle Connection Timeout: .................................................................. Error! Bookmark not defined.
3.6.3 Tunneling Client Timeout................................................................... Error! Bookmark not defined.
3.6.4 Post Timeout ...................................................................................... Error! Bookmark not defined.
3.6.5 Duration ............................................................................................. Error! Bookmark not defined.
3.6.6 Tunneling Client Timeout................................................................... Error! Bookmark not defined.
4 BPEL PM AUDIT AND LOGGING ................................................................ ERROR! BOOKMARK NOT DEFINED.
4.1 AUDIT TUNING .......................................................................................... ERROR! BOOKMARK NOT DEFINED.
4.1.1 SOA Infrastructure level ..................................................................... Error! Bookmark not defined.
4.1.2 BPEL Engine Level .............................................................................. Error! Bookmark not defined.
4.1.3 AuditStorePolicy ................................................................................ Error! Bookmark not defined.
4.1.4 AuditFlushByteThreshold................................................................... Error! Bookmark not defined.
4.1.5 AuditFlushEventThreshold ................................................................. Error! Bookmark not defined.
4.1.6 AsyncAuditPersisterThreads .............................................................. Error! Bookmark not defined.
4.1.7 AsyncAuditQueueSize ........................................................................ Error! Bookmark not defined.
4.2 LOGGING TUNING...................................................................................... ERROR! BOOKMARK NOT DEFINED.
5 BPEL ENGINE DATABASE SCHEMA ............................................................. ERROR! BOOKMARK NOT DEFINED.
5.1 TWO SCHEMAS ......................................................................................... ERROR! BOOKMARK NOT DEFINED.

4

5.2 PURGE ..................................................................................................... ERROR! BOOKMARK NOT DEFINED.
5.3 PARTITIONING........................................................................................... ERROR! BOOKMARK NOT DEFINED.
5.3.1 Partial Partitioning ............................................................................. Error! Bookmark not defined.
5.3.2 Complete Partitioning ........................................................................ Error! Bookmark not defined.
6 MDS SCHEMA TUNING ........................................................................... ERROR! BOOKMARK NOT DEFINED.
6.1 PURGING ................................................................................................. ERROR! BOOKMARK NOT DEFINED.
6.2 STATISTIC COLLECTION ............................................................................... ERROR! BOOKMARK NOT DEFINED.
6.3 OPTIMIZING PATH_FULLNAME TABLE ........................................................... ERROR! BOOKMARK NOT DEFINED.
7 TECHNOLOGY ADAPTERS ......................................................................... ERROR! BOOKMARK NOT DEFINED.
7.1 FILE AND FTP ADAPTER .............................................................................. ERROR! BOOKMARK NOT DEFINED.
7.1.1 Threads .............................................................................................. Error! Bookmark not defined.
7.1.2 Inbound Throttling ............................................................................. Error! Bookmark not defined.
7.1.3 Outbound Throttling .......................................................................... Error! Bookmark not defined.
7.1.4 Outbound Performance Best Practices.............................................. Error! Bookmark not defined.
7.1.5 Tuning number of files to be processed at a time ............................. Error! Bookmark not defined.
7.2 DATABASE ADAPTER .................................................................................. ERROR! BOOKMARK NOT DEFINED.
7.2.1 Basic considerations .......................................................................... Error! Bookmark not defined.
7.2.2 Existence Checking............................................................................. Error! Bookmark not defined.
7.2.3 Throttling ........................................................................................... Error! Bookmark not defined.
7.2.4 Adapter Threads ................................................................................ Error! Bookmark not defined.
7.3 JMS ADAPTER .......................................................................................... ERROR! BOOKMARK NOT DEFINED.
7.3.1 Receive Threads ................................................................................. Error! Bookmark not defined.
7.4 AQ ADAPTER ............................................................................................ ERROR! BOOKMARK NOT DEFINED.
7.4.1 Dequeue Threads ............................................................................... Error! Bookmark not defined.
7.4.2 Dequeue Threads and Number of database connection................... Error! Bookmark not defined.
7.5 MQ ADAPTER ........................................................................................... ERROR! BOOKMARK NOT DEFINED.
7.5.1 Dequeue Threads ............................................................................... Error! Bookmark not defined.
7.6 SOCKET ADAPTER ...................................................................................... ERROR! BOOKMARK NOT DEFINED.
8 BPEL PM – MISCELLANEOUS PROPERTIES ................................................... ERROR! BOOKMARK NOT DEFINED.
8.1 INSTANCE KEY BLOCK SIZE........................................................................... ERROR! BOOKMARK NOT DEFINED.
8.2 AUTOMATIC RECOVERY ATTEMPTS ............................................................... ERROR! BOOKMARK NOT DEFINED.
8.3 DISABLE BPEL MONITORS AND SENSORS ...................................................... ERROR! BOOKMARK NOT DEFINED.
9 BEST PRACTICES: BPEL PROCESS/COMPOSITE/COMPONENT............................. ERROR! BOOKMARK NOT DEFINED.
9.1 LARGE PAYLOADS ...................................................................................... ERROR! BOOKMARK NOT DEFINED.
9.2 PERSIST ONLY THE FAULTED BPEL INSTANCES................................................. ERROR! BOOKMARK NOT DEFINED.
9.3 DO NOT VALIDATE INCOMING XML .............................................................. ERROR! BOOKMARK NOT DEFINED.
9.4 IDEMPOTENENCY ....................................................................................... ERROR! BOOKMARK NOT DEFINED.
9.5 NONBLOCKINGINVOKE................................................................................ ERROR! BOOKMARK NOT DEFINED.
10 DATABASE TUNING .............................................................................. ERROR! BOOKMARK NOT DEFINED.
10.1 INITIALIZATION PARAMETERS ..................................................................... ERROR! BOOKMARK NOT DEFINED.
10.2 HUGE PAGES .......................................................................................... ERROR! BOOKMARK NOT DEFINED.
10.3 DB CONTENTION..................................................................................... ERROR! BOOKMARK NOT DEFINED.
10.3.1 Wait events ...................................................................................... Error! Bookmark not defined.

5

10.4 SECUREFILES........................................................................................... ERROR! BOOKMARK NOT DEFINED.
10.4.1 Enable SecureFiles for SOAINFRA schema ....................................... Error! Bookmark not defined.
10.4.2 Enabling Encryption ......................................................................... Error! Bookmark not defined.
10.5 SQL PLAN .............................................................................................. ERROR! BOOKMARK NOT DEFINED.
11 REFERENCE......................................................................................... ERROR! BOOKMARK NOT DEFINED.
EM FUSION MIDDLEWARE CONTROL AND WLS ADMIN CONSOLE..... ERROR! BOOKMARK NOT DEFINED.
1
2
3
4
5
6
7
8
9
10
11
12

STUCK THREAD MAX TIME....................................................................... ERROR! BOOKMARK NOT DEFINED.
STUCK THREAD TIMER INTERVAL ............................................................... ERROR! BOOKMARK NOT DEFINED.
TUNE DUMP CONFIGURATION ................................................................... ERROR! BOOKMARK NOT DEFINED.
DATA DISPLAY OPTION FOR EM CONSOLE ................................................... ERROR! BOOKMARK NOT DEFINED.
CACHE TIMEOUT FOR DISCOVERY ............................................................... ERROR! BOOKMARK NOT DEFINED.
PERMSIZE OF ADMIN SERVER ................................................................... ERROR! BOOKMARK NOT DEFINED.
DMS’S FREQUENCY................................................................................ ERROR! BOOKMARK NOT DEFINED.
UN-TARGET DMS FROM SERVERS ............................................................. ERROR! BOOKMARK NOT DEFINED.
BPEL RECOVERY CONSOLE OPTION ............................................................. ERROR! BOOKMARK NOT DEFINED.
INDEX CREATION.................................................................................. ERROR! BOOKMARK NOT DEFINED.
DB MANUAL STATS COLLECTION ............................................................. ERROR! BOOKMARK NOT DEFINED.
REFERENCE......................................................................................... ERROR! BOOKMARK NOT DEFINED.

DATA COLLECTION .......................................................................... ERROR! BOOKMARK NOT DEFINED.
1 BEFORE LNP TEST .................................................................................. ERROR! BOOKMARK NOT DEFINED.
1.1 INFRASTRUCTURE ...................................................................................... ERROR! BOOKMARK NOT DEFINED.
1.2 JVM-HOTSPOT-<<MANAGEDSERVER>> AND JVM-JROCKIT-<<MANAGEDSERVER>> ....... ERROR! BOOKMARK NOT
DEFINED.
1.3 WLS-THREADING-<<MANAGEDSERVER>> ................................................... ERROR! BOOKMARK NOT DEFINED.
1.4 WLS-NETWORKIO-<<MANAGEDSERVER>> .................................................. ERROR! BOOKMARK NOT DEFINED.
1.5 WLS-DATASOURCE-<<MANAGEDSERVER>>................................................. ERROR! BOOKMARK NOT DEFINED.
1.6 BPEL PM-<<MANAGEDSERVER>> .............................................................. ERROR! BOOKMARK NOT DEFINED.
1.7 TECHNOLOGY ADAPTERS............................................................................. ERROR! BOOKMARK NOT DEFINED.
1.8 BPEL PM – COMPOSITE ............................................................................ ERROR! BOOKMARK NOT DEFINED.
1.9 DATABASE ................................................................................................ ERROR! BOOKMARK NOT DEFINED.
2 DURING LNP TEST ................................................................................. ERROR! BOOKMARK NOT DEFINED.
2.1 CPU AND RAM USAGE AND MEMORY AVAILABILITY ........................................ ERROR! BOOKMARK NOT DEFINED.
2.2 JVM’S HEAP USAGE, GARBAGE COLLECTION AND ANY MEMORY LEAK IF ANY ........ ERROR! BOOKMARK NOT DEFINED.
2.3 DATABASE CONNECTIONS AVAILABILITY AND USAGE ........................................ ERROR! BOOKMARK NOT DEFINED.
2.4 AVAILABILITY OF PERIPHERAL RESOURCES LIKE JMS QUEUES, AQ AND THEIR USAGE PATTERN .... ERROR! BOOKMARK
NOT DEFINED.
3 AFTER LNP TEST .................................................................................... ERROR! BOOKMARK NOT DEFINED.
3.1 GET FULL DUMP OF FEW IMPORTANT TABLES.................................................. ERROR! BOOKMARK NOT DEFINED.
3.2 EXECUTE SQL QUERIES AS AND WHEN REQUIRED ............................................ ERROR! BOOKMARK NOT DEFINED.

6

3.3 ANALYSIS OF AWR REPORT ........................................................................ ERROR! BOOKMARK NOT DEFINED.
3.4 ANALYSIS OF SOA SERVER MEMORY USAGE ................................................... ERROR! BOOKMARK NOT DEFINED.
LNP TEST CASE DESIGN AND LOAD TEST SELECTION ......................... ERROR! BOOKMARK NOT DEFINED.
1 LNP TEST CASE DESIGN ........................................................................... ERROR! BOOKMARK NOT DEFINED.
2 DESIGN OF EXPERIMENT .......................................................................... ERROR! BOOKMARK NOT DEFINED.
APPENDIX A: WEBLOGIC SERVER OVERVIEW ................................... ERROR! BOOKMARK NOT DEFINED.
1
2
3
4
5
6
7
8
9

WLS CONFIGURATION ............................................................................ ERROR! BOOKMARK NOT DEFINED.
DOMAIN ............................................................................................. ERROR! BOOKMARK NOT DEFINED.
SERVER ............................................................................................... ERROR! BOOKMARK NOT DEFINED.
ADMINISTRATION SERVER ....................................................................... ERROR! BOOKMARK NOT DEFINED.
MANAGED SERVER ................................................................................ ERROR! BOOKMARK NOT DEFINED.
ADMINISTRATION SERVER TO MANAGED SERVER INTERACTION ........................ ERROR! BOOKMARK NOT DEFINED.
CLUSTER .............................................................................................. ERROR! BOOKMARK NOT DEFINED.
NODE MANAGER................................................................................... ERROR! BOOKMARK NOT DEFINED.
MACHINE ............................................................................................ ERROR! BOOKMARK NOT DEFINED.

APPENDIX B: AUDITING IN BPEL PM ................................................ ERROR! BOOKMARK NOT DEFINED.
1 AUDIT LEVELS ....................................................................................... ERROR! BOOKMARK NOT DEFINED.
2 ORDER OF PRECEDENCE FOR AUDIT LEVEL SETTINGS ........................................ ERROR! BOOKMARK NOT DEFINED.
APPENDIX C: ANTI PATTERNS .......................................................... ERROR! BOOKMARK NOT DEFINED.
1
2
3
4
5
6
7
8
9
10

SYNCHRONOUS – ASYNCHRONOUS ............................................................ ERROR! BOOKMARK NOT DEFINED.
OVER USE OF ASYNCHRONOUS PROCESSES ................................................... ERROR! BOOKMARK NOT DEFINED.
OVER USE OF DURABLE PROCESSES ............................................................. ERROR! BOOKMARK NOT DEFINED.
NO FAULT HANDLING ............................................................................. ERROR! BOOKMARK NOT DEFINED.
SYNCHRONOUS FAULT HANDLING .............................................................. ERROR! BOOKMARK NOT DEFINED.
TO MANY RETRIES .................................................................................. ERROR! BOOKMARK NOT DEFINED.
CHATTING BPEL PROCESS (CALL BACK) ....................................................... ERROR! BOOKMARK NOT DEFINED.
OVER USE OF FLOWN ............................................................................. ERROR! BOOKMARK NOT DEFINED.
LOOPS AND MORE LOOPS......................................................................... ERROR! BOOKMARK NOT DEFINED.
SYNCHRONOUS AND ASYNCHRONOUS PROCESSES ON SAME MANAGED SERVER/CLUSTER ... ERROR! BOOKMARK NOT
DEFINED.
11 DURABLE AND TRANSIENT PROCESSES ON SAME MANAGED SERVER/CLUSTER...... ERROR! BOOKMARK NOT DEFINED.
12 STICKY LOAD BALANCER ......................................................................... ERROR! BOOKMARK NOT DEFINED.

7

13 NOT KEEPING ASPECT RATIO ................................................................... ERROR! BOOKMARK NOT DEFINED.
APPENDIX D: SQL QUERIES .............................................................. ERROR! BOOKMARK NOT DEFINED.
1 EM CONSOLE SQL QUERIES ..................................................................... ERROR! BOOKMARK NOT DEFINED.
1.1 RECOVERY CONSOLE QUERIES ...................................................................... ERROR! BOOKMARK NOT DEFINED.
1.2 RECENT FAULT AND REJECTED MESSAGES QUERY ............................................. ERROR! BOOKMARK NOT DEFINED.
1.3 RECENT COMPOSITE INSTANCE QUERY........................................................... ERROR! BOOKMARK NOT DEFINED.
1.4 INSTANCE TAB PAGE QUERY ......................................................................... ERROR! BOOKMARK NOT DEFINED.
1.5 INSTANCE TAB PAGE SEARCH QUERY BASED ON NAME VS TITLE QUERY ................ ERROR! BOOKMARK NOT DEFINED.
1.6 FAULT AND REJECTED MESSAGE TAB PAGE QUERIES ......................................... ERROR! BOOKMARK NOT DEFINED.
1.6.1 Parent query ...................................................................................... Error! Bookmark not defined.
1.6.2 Child query ......................................................................................... Error! Bookmark not defined.
2 MISCELLANEOUS ................................................................................... ERROR! BOOKMARK NOT DEFINED.
2.1 STORED PROCEDURE TO CONVERT BLOB IN STRING .......................................... ERROR! BOOKMARK NOT DEFINED.
2.2 QUERY TO FIND PERCENTAGE OF FREE SPACE .................................................. ERROR! BOOKMARK NOT DEFINED.
2.3 QUERY TO FIND THE WAIT EVENTS FOR LGWR USING ITS SID ........................... ERROR! BOOKMARK NOT DEFINED.
2.4 QUERY TO MONITOR ‘REDO BUFFER ALLOCATION RETRIES’................................ ERROR! BOOKMARK NOT DEFINED.
2.5 SQL STATEMENT TO RECLAIM SPACE AFTER PURGING....................................... ERROR! BOOKMARK NOT DEFINED.
2.6 QUERY TO FIND OUT TOTAL SESSIONS ON A DATABASE ..................................... ERROR! BOOKMARK NOT DEFINED.
2.7 QUERY TO FIND OUT UTILIZATION OF PROCESSES AND SESSIONS IN A DATABASE ... ERROR! BOOKMARK NOT DEFINED.
2.8 FIND OUT THE PROCESS INSTANCE FROM A CONVERSATION ID WHEN THERE IS NO INSTANCE NUMBER SHOWING IN
THE LOG FILE (BPEL INSTANCE ID FOR A TIMES OUT ITEM) ....................................... ERROR! BOOKMARK NOT DEFINED.
2.9 QUERY TO GET AUDIT DETAILS FROM AUDIT_DETAILS TABLE .............................. ERROR! BOOKMARK NOT DEFINED.
2.10 QUERY TO GET AUDIT DETAILS FROM AUDIT_TRAIL TABLE ............................... ERROR! BOOKMARK NOT DEFINED.
2.11 QUERY TO GET XML MESSAGE WITH THE GIVEN INSTANCE ID ......................... ERROR! BOOKMARK NOT DEFINED.
2.12 QUERY TO GET XML MESSAGE WITH A GIVEN INSTANCE NAME........................ ERROR! BOOKMARK NOT DEFINED.
2.13 QUERY TO GET PAYLOAD SIZE OF MESSAGE ................................................... ERROR! BOOKMARK NOT DEFINED.
2.14 QUERY TO GET EXECUTION TIME OF BPEL INSTANCES .................................... ERROR! BOOKMARK NOT DEFINED.
2.15 QUERY TO GET THE EXECUTION TIME OF BPEL INSTANCES AND TO FIND THE PARENT THAT HAS INITIALIZED THE
COMPOSITE ....................................................................................................... ERROR! BOOKMARK NOT DEFINED.
2.16 QUERY TO IDENTIFY ALL THE FAULTS FOR THE MESSAGES THAT WERE SITTING IN BPEL ENGINE LEVEL RECOVERY AS
UNDELIVERED INVOKES ........................................................................................ ERROR! BOOKMARK NOT DEFINED.
APPENDIX E: BIG OR LARGE OR HUGE PAGES ................................... ERROR! BOOKMARK NOT DEFINED.
1
2
3
4

LINUX ................................................................................................. ERROR! BOOKMARK NOT DEFINED.
WINDOWS ........................................................................................... ERROR! BOOKMARK NOT DEFINED.
SOLARIS .............................................................................................. ERROR! BOOKMARK NOT DEFINED.
REFERENCE: ......................................................................................... ERROR! BOOKMARK NOT DEFINED.

APPENDIX F: ORA-01438: VALUE LARGER THAN SPECIFIED PRECISION ALLOWED .... ERROR! BOOKMARK
NOT DEFINED.

8

5
6
7
8

WHAT IS THE ERROR IN LOGS? .................................................................. ERROR! BOOKMARK NOT DEFINED.
EFFECTS............................................................................................... ERROR! BOOKMARK NOT DEFINED.
CAUSE ................................................................................................ ERROR! BOOKMARK NOT DEFINED.
SOLUTION ............................................................................................ ERROR! BOOKMARK NOT DEFINED.

APPENDIX G: LOBS IN THE SOAINFRA SCHEMA ................................ ERROR! BOOKMARK NOT DEFINED.
APPENDIX H: AWR, ADDM, & ASH REPORTS .................................... ERROR! BOOKMARK NOT DEFINED.
1 AWR REPORT ...................................................................................... ERROR! BOOKMARK NOT DEFINED.
2 ADDM REPORT .................................................................................... ERROR! BOOKMARK NOT DEFINED.
3 ASH REPORT ........................................................................................ ERROR! BOOKMARK NOT DEFINED.
4 AWR REPORT ANALYSIS ......................................................................... ERROR! BOOKMARK NOT DEFINED.
4.1 SQL STATEMENTS ORDERED BY ELAPSED TIME ............................................... ERROR! BOOKMARK NOT DEFINED.
4.2 SQL STATEMENTS ORDERED BY CPU TIME..................................................... ERROR! BOOKMARK NOT DEFINED.
4.3 SQL STATEMENTS ORDERED BY GETS ............................................................ ERROR! BOOKMARK NOT DEFINED.
4.4 SQL STATEMENTS ORDERED BY READS .......................................................... ERROR! BOOKMARK NOT DEFINED.
4.5 SQL STATEMENTS ORDERED BY EXECUTIONS .................................................. ERROR! BOOKMARK NOT DEFINED.
4.6 SQL STATEMENTS ORDERED BY PARSE CALLS ................................................. ERROR! BOOKMARK NOT DEFINED.
APPENDIX I: MONITORING SCRIPTS ................................................. ERROR! BOOKMARK NOT DEFINED.
1 DATABASE MONITORING ......................................................................... ERROR! BOOKMARK NOT DEFINED.
2 JMS MONITORING ................................................................................ ERROR! BOOKMARK NOT DEFINED.
3 AQ MONITORING .................................................................................. ERROR! BOOKMARK NOT DEFINED.
APPENDIX J: HOW TO MONITOR SOA SERVER MEMORY USAGE ....... ERROR! BOOKMARK NOT DEFINED.
1
2
3
4

SETUP: JCONSOLE OR VISUALVM (INSTALLED LOCALLY)................................... ERROR! BOOKMARK NOT DEFINED.
SETUP: JVISUALVM (INSTALLED AT REMOTE MACHINE) ................................... ERROR! BOOKMARK NOT DEFINED.
SETUP: JROCKIT MISSION CONTROL (INSTALLED AT REMOTE MACHINE) .............. ERROR! BOOKMARK NOT DEFINED.
REFERENCE .......................................................................................... ERROR! BOOKMARK NOT DEFINED.

APPENDIX K: HEAP DUMP FILES ANALYSIS: JROCKIT AND HOTSPOT JVMS ....... ERROR! BOOKMARK NOT
DEFINED.
1 EXAMPLE ANALYSIS OF A HEAP DUMP FILE USING ECLIPSE MEMORY ANALYZER ... ERROR! BOOKMARK NOT DEFINED.
APPENDIX L: CAPACITY PLANNING................................................... ERROR! BOOKMARK NOT DEFINED.

9

1 CAPACITY PLANNING FOR BPEL PM .......................................................... ERROR! BOOKMARK NOT DEFINED.
1.1 DETERMINING PERFORMANCE GOALS AND OBJECTIVES – CURRENT & FUTURE.... ERROR! BOOKMARK NOT DEFINED.
1.2 MEASURING PERFORMANCE METRICS .......................................................... ERROR! BOOKMARK NOT DEFINED.
1.3 IDENTIFYING BOTTLENECKS ......................................................................... ERROR! BOOKMARK NOT DEFINED.
1.4 IMPLEMENTING A CAPACITY MANAGEMENT PLAN .......................................... ERROR! BOOKMARK NOT DEFINED.

Exhibits
Exhibit 1: Stress, Load, Soak, Spike Tests Load Profiles ................................................................ 18
Exhibit 2: LnP Testing Life Cycle and SDLC .................................................................................... 19
Exhibit 3: BPEL Processes ................................................................. Error! Bookmark not defined.
Exhibit 4: JVM Tuning Tradeoffs ...................................................... Error! Bookmark not defined.
Exhibit 5: JVM's Garbage Collection and Throughput ..................... Error! Bookmark not defined.
Exhibit 6: jRockit Attributes ............................................................. Error! Bookmark not defined.
Exhibit 7: HostSpot Attributes ......................................................... Error! Bookmark not defined.
Exhibit 8: WLS internal applications at startup - 1 ......................... Error! Bookmark not defined.
Exhibit 9: WLS internal applications at startup - 2 ......................... Error! Bookmark not defined.
Exhibit 10: WLS internal applications at startup - 3 ....................... Error! Bookmark not defined.
Exhibit 11: WLS internal applications at startup - 4 ........................ Error! Bookmark not defined.
Exhibit 14: SOAWorkManager - 1 .................................................... Error! Bookmark not defined.
Exhibit 17: WLS Muxers ................................................................... Error! Bookmark not defined.
Exhibit 18: FMC Login....................................................................... Error! Bookmark not defined.
Exhibit 19: FMC Farm Summary....................................................... Error! Bookmark not defined.
Exhibit 20: FMC Performance Summary - 1 ..................................... Error! Bookmark not defined.
Exhibit 23: Dynamic Memory Service .............................................. Error! Bookmark not defined.
Exhibit 24: BPEL Engine Threading Model ....................................... Error! Bookmark not defined.
Exhibit 25: Dispatcher Maximum Request Depth - 1....................... Error! Bookmark not defined.
Exhibit 26: Dispatcher Maximum Request Depth - 2....................... Error! Bookmark not defined.
Exhibit 27: InvokeMessages In Cache .............................................. Error! Bookmark not defined.
Exhibit 28: syncMaxWaitTime - 1 .................................................... Error! Bookmark not defined.
Exhibit 31: syncMaxWaitTime – 4 .................................................... Error! Bookmark not defined.
Exhibit 32: BPEL EJBs - 1 ................................................................... Error! Bookmark not defined.
10

Exhibit 35: Global transaction timeout at WLS Domain Level - 1 .... Error! Bookmark not defined.
Exhibit 36: Global transaction timeout at WLS Domain Level - 2 .... Error! Bookmark not defined.
Exhibit 37: WLS Timeout parameters - 1 ......................................... Error! Bookmark not defined.
Exhibit 40: Tunneling Client Timeout - 1.......................................... Error! Bookmark not defined.
Exhibit 43: SOA Infrastructure level audit tuning - 1 ....................... Error! Bookmark not defined.
Exhibit 44: SOA Infrastructure level audit tuning - 2 ....................... Error! Bookmark not defined.
Exhibit 45: AuditConfig - 1 ............................................................... Error! Bookmark not defined.
Exhibit 50: Payload Validation - 1 .................................................... Error! Bookmark not defined.
Exhibit 51: Payload Validation - 2 .................................................... Error! Bookmark not defined.
Exhibit 52: AuditStorePolicy - 1 ....................................................... Error! Bookmark not defined.
Exhibit 53: AuditStorePolicy - 2 ....................................................... Error! Bookmark not defined.
Exhibit 54: Logging Tuning - 1 .......................................................... Error! Bookmark not defined.
Exhibit 58: MDS Purging - 1 ............................................................. Error! Bookmark not defined.
Exhibit 59: MDS Purging - 2 ............................................................. Error! Bookmark not defined.
Exhibit 60: Stuck Thread Timer Interval - 1...................................... Error! Bookmark not defined.
Exhibit 61: : Stuck Thread Timer Interval - 2.................................... Error! Bookmark not defined.
Exhibit 62: : Stuck Thread Timer Interval - 3.................................... Error! Bookmark not defined.
Exhibit 63: Data Display Option for EM Console - 1 ........................ Error! Bookmark not defined.
Exhibit 64: : Data Display Option for EM Console - 2 ...................... Error! Bookmark not defined.
Exhibit 65: : Data Display Option for EM Console - 3 ...................... Error! Bookmark not defined.
Exhibit 66: Un-Target DMS from servers - 1 .................................... Error! Bookmark not defined.
Exhibit 68: : Un-Target DMS from servers - 3 .................................. Error! Bookmark not defined.
Exhibit 71: BPEL recovery console option - 1................................... Error! Bookmark not defined.
Exhibit 72: BPEL recovery console option - 2................................... Error! Bookmark not defined.
11

Exhibit 73: vmstat ............................................................................ Error! Bookmark not defined.
Exhibit 74: free ................................................................................. Error! Bookmark not defined.
Exhibit 75: pmap .............................................................................. Error! Bookmark not defined.
Exhibit 76: top .................................................................................. Error! Bookmark not defined.
Exhibit 77: sar-B ............................................................................... Error! Bookmark not defined.
Exhibit 78: meminfo ......................................................................... Error! Bookmark not defined.
Exhibit 79: mpstat ............................................................................ Error! Bookmark not defined.
Exhibit 80: mpstat 2 4 ...................................................................... Error! Bookmark not defined.
Exhibit 81: df-m ................................................................................ Error! Bookmark not defined.
Exhibit 82: WebLogic Homes ........................................................... Error! Bookmark not defined.
Exhibit 83: Synchronous – Asynchronous - 1 ................................... Error! Bookmark not defined.
Exhibit 84: Synchronous – Asynchronous - 2 ................................... Error! Bookmark not defined.
Exhibit 85: Database Monitoring ..................................................... Error! Bookmark not defined.
Exhibit 86: JMS Monitoring .............................................................. Error! Bookmark not defined.
Exhibit 87: AQ Monitoring ............................................................... Error! Bookmark not defined.
Exhibit 88: Setup jConsole or visualVM (installed locally) - 1 .......... Error! Bookmark not defined.
Exhibit 92: Heap Dump Files analysis - 1 ......................................... Error! Bookmark not defined.

12

Snippets
Snippet 1: Sample Work Manager - 1 .............................................. Error! Bookmark not defined.
Snippet 2: Sample Work Manager - 2 .............................................. Error! Bookmark not defined.
Snippet 3: Sample config.xml........................................................... Error! Bookmark not defined.
Snippet 4: Sample startWeblogic.sh ................................................ Error! Bookmark not defined.
Snippet 5: Using setNull In a Prepared Statement .......................... Error! Bookmark not defined.
Snippet 6: Sample thread waiting for DB Call to finish.................... Error! Bookmark not defined.
Snippet 7: composite.xml ................................................................ Error! Bookmark not defined.
Snippet 8: Drop Histrogram ............................................................. Error! Bookmark not defined.
Snippet 9: Set Table Preferences ..................................................... Error! Bookmark not defined.
Snippet 10: Single Threaded Model - jca file ................................... Error! Bookmark not defined.
Snippet 11: Partitioned Threaded Model - jca file........................... Error! Bookmark not defined.
Snippet 12: Tuning number of files to be processed at a time ....... Error! Bookmark not defined.
Snippet 13: JMS Adapter - Receive Thread - composite.xml ........... Error! Bookmark not defined.
Snippet 14: AQ Adapter - Dequeue Threads - composite.xml......... Error! Bookmark not defined.
Snippet 15: common properties schema - 1.................................... Error! Bookmark not defined.
Snippet 16: common properties schema - 2.................................... Error! Bookmark not defined.
Snippet 17: bpel.xml ........................................................................ Error! Bookmark not defined.
Snippet 18: composite.xml .............................................................. Error! Bookmark not defined.
Snippet 19: SecureFiles .................................................................... Error! Bookmark not defined.
Snippet 20: BasicFiles to SecureFiles ............................................... Error! Bookmark not defined.

13

Tables
Table 1: AuditLevel........................................................................... Error! Bookmark not defined.
Table 2: BPEL Engine - AuditLevel .................................................... Error! Bookmark not defined.
Table 3: AuditStorePolicy ................................................................. Error! Bookmark not defined.
Table 4: SOA Schemas ...................................................................... Error! Bookmark not defined.
Table 5: SOAINFRA Tables ................................................................ Error! Bookmark not defined.
Table 6: MDS Schema tables............................................................ Error! Bookmark not defined.
Table 7: CUBE_INSTANCE................................................................. Error! Bookmark not defined.
Table 8: CUBE_SCOPE ...................................................................... Error! Bookmark not defined.
Table 9: COMPOSITE_INSTANCE ...................................................... Error! Bookmark not defined.
Table 10: COMPOSITE_INSTANCE_FAULT ....................................... Error! Bookmark not defined.
Table 11: AUDIT_TRAIL .................................................................... Error! Bookmark not defined.
Table 12: AUDIT_DETAILS ................................................................ Error! Bookmark not defined.
Table 13: DLV_MESSAGE ................................................................. Error! Bookmark not defined.
Table 14: DLV_SUBSCRIPTION ......................................................... Error! Bookmark not defined.
Table 15: DOCUMENT_CI_REF ......................................................... Error! Bookmark not defined.
Table 16: DOCUMENT_DLV_MSG_REF ............................................ Error! Bookmark not defined.
Table 17: WORK_ITEM ..................................................................... Error! Bookmark not defined.
Table 18: XML_DOCUMENT ............................................................. Error! Bookmark not defined.
Table 19: XML_DOCUMENT_REF ..................................................... Error! Bookmark not defined.
Table 20: HEADERS_PROPERTIES ..................................................... Error! Bookmark not defined.
Table 21: CLUSTER_MASTER ............................................................ Error! Bookmark not defined.
Table 22: CLUSTER_NODE ................................................................ Error! Bookmark not defined.
Table 23: Partial Partitioning ........................................................... Error! Bookmark not defined.
Table 24: Complete Partitioning ...................................................... Error! Bookmark not defined.
Table 25: MDS_NAMESAPCES.......................................................... Error! Bookmark not defined.
Table 26: MDS_PARTITIONS ............................................................ Error! Bookmark not defined.
Table 27: MDS_PATHS ..................................................................... Error! Bookmark not defined.
Table 28: File & FTP Adapter - Inbound Throttling .......................... Error! Bookmark not defined.
Table 29: : File & FTP Adapter - Outbound Throttling ..................... Error! Bookmark not defined.
Table 30: : File & FTP Adapter - Outbound Best Practices............... Error! Bookmark not defined.
Table 31: completionPersistPolicy property .................................... Error! Bookmark not defined.
Table 32: validateSchema value ...................................................... Error! Bookmark not defined.
Table 33: Idempotency value........................................................... Error! Bookmark not defined.
Table 34: nonBlockingInvoke value ................................................ Error! Bookmark not defined.
Table 35: Database initialization parameters .................................. Error! Bookmark not defined.
Table 36: Contention tables ............................................................. Error! Bookmark not defined.
Table 37: Tables under potential of index contention .................... Error! Bookmark not defined.
14

Load & Performance Testing
Tuning is never a one-size-fits-all proposition or a one-off configuration change. Rather, it is an
iterative process of monitoring and tuning.
The goal of Load and Performance (LnP) testing is to appraise any user (human and machine)
experience in realistic scenarios on target application. It will also facilitate resource (computers
and associate infrastructure) capacity planning and assessment.
LnP testing has several flavors - Load, Stress, Peak Load, Capacity testing and all are based
on business requirements. LnP testing requires the simulation of real life scenarios running
against target application. Scenario designing for LnP testing is a major task.
For LnP testing of BPEL PM at least two scenarios must be considered:
•
•

Standalone BPEL PM (not connected to service clients and providers)
Fully connected BPEL PM (connected to service clients and providers)

While reporting LnP results one should publish BPEL PM results and End to End connected
system.
Why is application so slow?
• Highlights performance bottlenecks in hardware, network and software
• Reduce hardware , network and software costs by predicting application scalability and
capacity
Does system comply with business requirement?
• Helps to establish realistic service level agreements (SLA)
• Helps in setting up expectations from system
• Verifies that system meet specified performance requirements and avoid issue in production
What will happen as business grow?
• Allows simulating various type of load to analyze system reaction (What-if conditions)
• Helps in anticipating changes and evolution of business needs
The thing is, it has to be realistic
Load profile is pattern of load exerted on application under LnP testing. Choosing load profile
decides nature of test. For Stress test, load profile should be of moderate load for extended
period while for peak load test requires incremental load till application breaks.
If LnP testing is not performed well – incorrect load, duration, and analysis, one can end up with
a performance false negative or a performance false positive. A false negative is breaking of
the application under certain conditions which are not realistic (like loading an application
beyond business and performance requirements).
16

On the other hand, if load is not sufficient and one reports application is working file under load
but in reality application is never loaded sufficiently. This is false positive.
The cost of both false negative and false positive is huge. While designing load scenarios and
analysis, one need to be very careful to discount false negatives and false positives.

1 Stress, Load, Soak, Spike Tests Load Profiles
•

Stress Test is conducted to find limits of the application – performance and break point. In
this test load on the application is increased in linear fashion over time.

•

Peak Load Test is conducted when business requirements are very well understood. In this
case load is ramped up and plateaued for agreed upon level (generally at peak of busiest
business activities). At this level load is maintained for agreed upon duration and then ramped
down.
The agreed upon height of plateau and duration of plateau is an equilibrium which signifies
that application must perform satisfactorily under these conditions which are above normal
but below designed.

•

Soak Test is conducted to find out memory leak or similar issues which may arise when
application is loaded with more than normal load and for extended duration.

•

Spike test is conducted to find out applications’ behavior in case of sudden spike in load. In
this case load is for very short duration but beyond maximum design capacity. In spike test
ripple effects of spike are analyzed.

17

Exhibit 1: Stress, Load, Soak, Spike Tests Load Profiles

2 LnP Testing Life Cycle
LnP testing life cycle is not confined to SDLC (Software Development Life Cycle). LnP testing
efforts seeps into maintenance phase of application as well because of ever changing business
and technical environments. With time business requirements change – change in business
results in increased/decreased load and change in load pattern. With introduction of new
applications on same hardware and network also changes technical landscape.
Holistically LnP Testing Life Cycle (LnP TLC) may be depicted as:

18

Exhibit 2: LnP Testing Life Cycle and SDLC

3 Examples of LnP Testing Objectives
3.1 Response time
•
•

How long does it take to complete a BPEL process/composite under given load patterns?
How long does it take to complete a business process under given load patterns?

3.2 Reliability
•
•

How stable is the BPEL process/composite and/or BPEL engine under given load
patterns?
How long BPEL process/composite and/or BPEL engine can sustain under a given load
pattern?

3.3 Configuration sizing
•
•

Which configuration provides the best performance level under given load pattern?
What are optimized configurations under various given load patterns?

3.4 Capacity planning
•
•

At what point does performance degradation occur under given load pattern?
What is the performance degradation patterns under given load patterns?

19

3.5 Regression
•
•
•
•

Does a new version of the composite/process adversely affect response time?
Does a new version of the composite/process adversely affect throughput?
What is the effect of addition of newly added composite/process on response time of
already deployed composites/processes under given load pattern?
What is the effect of addition of newly added composite/process on throughput of
already deployed composites/processes under given load pattern?

3.6 Bottleneck identification
•
•

What is the cause of the degraded response time?
What is the cause of the degraded throughput?

3.7 Scalability
•
•
•

How far the BPEL process/composite and/or BPEL engine can grow to handle future load
volumes?
What type of scalability is possible – vertical or horizontal or both?
How many BPEL process/composite can be deployed on a given BPEL engine under
given/expected load conditions?

20

BPEL PM 11g performance tuning - 1

More Related Content

What's hot (18)

Similar to BPEL PM 11g performance tuning - 1 (20)

More from tusjain (8)

Recently uploaded (20)

BPEL PM 11g performance tuning - 1