![Legal Disclaimer
• © IBM Corporation 2014. All Rights Reserved.
• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained
in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are
subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing
contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and
conditions of the applicable license agreement governing the use of IBM software.
• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or
capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to
future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by
you will result in any specific sales, revenue growth or other results.
• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will
experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs
and performance characteristics may vary by customer.
• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM
Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server).
Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your
presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in
your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International
Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.
• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.
• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:
Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and
other countries.
• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:
UNIX is a registered trademark of The Open Group in the United States and other countries.
• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of
others.
• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta
Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration
purposes only.
32](https://image.slidesharecdn.com/enablebusinesscontinuityandhighavailabilitythroughactive-activetechnology-140616004701-phpapp01/85/Enable-business-continuity-and-high-availability-through-active-active-technology-33-320.jpg)
Enable business continuity and high availability through active active technology
- 1. © 2014 IBM Corporation Enable Business Continuity and High Availability through Active-Active Technology David Huang, CTO of China Everbright Bank Credit Card Qian Li Jin, jinql@cn.ibm.com, Senior manager of WebSphere Messaging
- 2. Please Note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 1
- 3. Agenda Active/Active Solution Overview • Business Continuity and High Availability Requirement • Active-Active Solution and Business Value • Active-Active Common Architecture WebSphere Messaging Technology in Active/Active Solution • Messaging Key Functions • Flexible Application Design • High Performance with Persistence China Ever Bright Bank (CEB) Active/Active Solution • CEB Credit Card System Background • Active/Active Architecture • Active/Active Operation and Site Switch • Best Practice 2
- 4. High Availability in Enterprise Enterprise Business Requires High Availability Standby Active-Active 1. Disaster Recovery • Restore the business after a disaster 2. High-Availability • Meet Service Availability objectives e.g., 99.9% availability or no more than 8 hours of down-time a year for maintenance and failures 3. Continuous Availability • No downtime ever (planned or not) 3
- 5. Technologies for Business Continuity Requirements: Disaster Recovery High Availability Continuous Availability Replicated Objects: Technologies: •Disk Copy •DB Recovery •Transaction Replay Requirements and trade-offs to consider in selecting technologies: • What needs to be recovered?: Application Data vs. DBMS vs. Entire Systems • How long does it take? Recovery Time Objective (RTO): One hours or more vs. few seconds • How much data could you lose? Recovery Point Objective (RPO): No data loss vs. seconds of data • Distance required between sites?: 10 meters vs. 100 kilometers • Hardware Utilization: Standby vs. Active • Impact on applications: Direct overhead (synchronous technologies) vs. No impact (asynchronous technologies) • CPU Overhead: Negligible (hardware e.g., PPRC) vs. Proportional to the workload (transaction replay technology) 4
- 6. Active/Active Sites Concept Two or more sites, separated by unlimited distances, running the same applications and having the same data to provide cross-site workload balancing and Continuous Availability / Disaster Recovery Customer data at geographically dispersed sites kept in sync via synchronization GDPS/PPRC GDPS/XRC or GDPS/GM Active/Active Failover model Failover model Near CA model Recovery time = 2 minutes Recovery time < 1 hour Recovery time < 1 minute Distance < 20 KM Unlimited distance Unlimited distance CD1SOURCE CD1TABLE CD1SOURCE CD1TABLE CD1SOURCE CD1TABLE CD1SOURCE CD1TABLE CD1SOURCECD1SOURCE CD1TABLECD1CD1TABLE CD1SOURCE CD1TABLE CD1SOURCE CD1TABLE CD1SOURCECD1SOURCE CD1TABLECD1CD1TABLE 5
- 7. Active-Active Business Objective Reduce planned/unplanned outage for the critical applications Both site A and B run workload Achieve extremely short data recovery time (mins) and zero data loss during disaster recovery. Achieve Continuous Availability of the key applications which support the critical business services through flexible switching over and back between site A & B “Near real time business analysis” Efficient utilization of IT resources of Site B 6
- 8. Active-Active Common Model based on Messaging Business App Business Data Sync App Messaging Sync App Messaging Business App Business Data Workload Distributor Sites at a distance •Cross Site Workload Distribution •Data synchronization •Reply on high performance, reliable messaging transmission •Flexible application design •Automation & Management 7
- 9. Agenda Active/Active Solution Overview • Business Continuity and High Availability Requirement • Active-Active Solution and Business Value • Active-Active Common Architecture WebSphere Messaging Technology in Active/Active Solution • Messaging Key Functions • Flexible Application Design • High Performance with Persistence China Ever Bright Bank (CEB) Active/Active Solution • CEB Credit Card System Background • Active/Active Architecture • Active/Active Operation and Site Switch • Best Practice 8
- 10. Messaging Technology in Active-Active Data Synchronization is the key component in Active-Active • Capture transaction change in real-time • Publish the change in high performance with low latency Messaging based implementation is proven to be the simplest way among kinds of methods of data transmission A high performance, reliable messaging product is needed for the following requirements: • Simplifies application development • Ease of use • Assured message delivery • High Performance and Scalability • Easy of Management 9
- 11. WebSphere MQ High Light for Active-Active Reliability • Assured message delivery • Performance Ubiquitous • Breadth of support for platforms, programming languages and API Loose-coupling • Location transparency • Time independence • Data transparency (with WebSphere Message Broker) • Platform independence Scalability • Incremental growth Rapid development • Reduce Complexity • Ease of use Q Manager Q Manager Message Queue Application ZApplication A Channels MQ Version Release – V5.3 Distributed: EOS – V6 GA: 2005/06 • (EOS 2012/09) – V7.0.0 GA: 2008/06 – V7.0.1 GA: 2009/09 – V7.1.0 GA: 2011/11 (include MQ for z/OS) – V7.5 GA: 2012/06 (distributed only) 10
- 12. Queue Manager Synchronization Application Design - Persistence All messages are important in banking system. Money is included. Persistent or non-persistent message need to be justified • Persistent messages will be recovered, non-persistent not • Persistent messages reduce the cost of developing the application • Persistent messages increase the cost of processing, need logging Only use a persistent message if require MQPUT Persistent Message MQPUT Non-persistent Message Application Program Logging 11
- 13. Synchronization Application Design - Syncpoint Transactions are vital to provide reliable messaging. Do you need it? • Set of work needs to either all be performed, or all not performed The Transactional behaviour should be considered in synchronization application • The change in source site been published to MQ once and only once • The message in target site been processed and removed from MQ May use some other ways to keep data consistency Actually cheaper to process in syncpoint for persistent messages MQPUT within syncpoint MQPUT within syncpoint MQPUT within syncpoint MQCMIT 1 2 3 Messages are only now available 12
- 14. Synchronization Application Design - Message Size The size of messages can be a contentious issue WebSphere MQ supports a large variety of message size Queue managers are optimised for 4KB - 100KB message size • Very large messages are inefficient • Very small message are inefficient – The queue manager has to perform the same work for a 1-byte message as for a 4KB message Application could leverage this by • Packaging small transactions into big message • Split big transaction into small messages 13
- 15. Performance Tuning Considerations Synchronize only the changed data, thus reduce the data volume Introduce more parallelism • Multiple synchronization channels for different type of workload • More threads in sync application for parallel processing • Multiple MQ channels to leverage single channel busy problem Invest to use MQ new feature • Bigger buffer pools above the bar • Sequential pre-fetch • Page set read/write performance enhancement • Channel performance improvement 14
- 16. MQ Buffer pools read ahead enhancement Symptom: When the number of messages overruns the buffer pool allocated for the queue, messages are spilled to disk and must then be retrieved from disk. The read ahead enhancement enables message pre-fetch from disk storage and improves MQGET performance. Available in PM PM63802/UK79853 in 2012 and PM81785/ UK91439 in 2013. Internal testing shows ~50% improvement with read ahead enabled (msglen=6KB). Enable this feature if MQ buffer pool may overrun. 15
- 17. Agenda Active/Active Solution Overview • Business Continuity and High Availability Requirement • Active-Active Solution and Business Value • Active-Active Common Architecture WebSphere Messaging Technology in Active/Active Solution • Messaging Key Functions • Flexible Application Design • High Performance with Persistence China Ever Bright Bank (CEB) Active/Active Solution • CEB Credit Card System Background • Active/Active Architecture • Active/Active Operation and Site Switch • Best Practice 16
- 18. Profile of China Everbright Bank (CEB) Established in August 1992 Registered capital RMB28.21689 billion By the end of 2008 • Total assets reached RMB843 billion • Total loans of RMB468.9 billion. • The operating profit and net profit are RMB7.96 billion and RMB7.36 billion respectively. Now • Covering 45 major cities of 23 provinces, municipalities and autonomous regions • 30 branches and 426 banking outlets and one representative office in HK • The Bank now has about 17, 000 employees 17
- 19. Active-Active Adaptability in Small/Medium-sized Banks China banks have setup storage based DR solution, but the business recovery time is too long Sysplex solution is too expensive, and input-output ratio is not high Need to consider application based solution, and mix with the storage based solution Active-Active is the target model of modern data center Active-Active is on its way, need a lot of innovation and there’s no copy model Not only for mainframe, but heterogeneous and periphery distributed platform also need to be active-active 18
- 20. Background of CEB Credit Card System CEB credit card system on mainframe gone live in Oct 2011. This system is based on the VisionPlus (V+) solution by First Data (Secondary) V+ Mainframe Batch Processing (Main) V+ Mainframe Batch Processing DRNET Headquarter Gateway Finance Processing in BJ Finance Processing in SH OLTP Processing OLTP Processing File Transfer VISA/MC/JCB . Non-Finance Processing OLTP Batch Terminal Anti-fraud Reporting Global Mirror files Debt-collection 19
- 21. Business Requirement of Active-Active Improve the capacity and availability of the whole credit card system. More comprehensive and more efficient services by payment systems of the banks. More flexibility accesses, more comprehensive functions of liquidity risk management, extension of the scope of system monitoring Refinement of backup infrastructure 20
- 22. 2. File Transfer The target Active-Active System Structure Both the main system and the secondary system are active Real data synchronization for OLTP transactions The main system and the secondary system backup each other Workload can be taken over in case of planned or unplanned failure 21 (Secondary) V+ Mainframe Batch Processing (Main) V+ Mainframe Batch Processing DRNET Headquarter Gateway Finance Processing in BJ Finance Processing in SH OLTP Processing OLTP Processing File Transfer VISA/MC/JCB . Non-Finance Processing OLTP Batch Terminal Anti-fraud Reporting 3. Global Mirror files Debt-collection Workload Split by Card BIN, and send to BJ and SH 1.OLTP Transaction (MQ)
- 23. CEB Active-Active Deployment Model 22 Continuous Availability – Active-Active Headquarter Gateway (Route by BIN) Encryption Encryption Front-end App System Core Data Core Data Sync Sync Beijing Shanghai Front-end App System(Main) Business Continuous Availability Achieve Business Continuous Availability by front end and mainframe active-active Reliable Services Synchronize application data based on MQ reliable messaging, keep data consistency in real time Data Backup Backup key business data through MQ series Data interchange in real time The data centers could be located in long distance
- 24. CEB Active-Active Logical model for OLTP Self implemented replication service based on WebSphere MQ for z/OS Beijing Site Shanghai Site MQ queue manager 1 send VSAM AOR Transaction Publisher VSAM Transaction Replay retrieve MQ queue manager 2 AOR Transaction Publisher Transaction Replay retrieve send Credit Card System Credit Card System Workload Distributor 23
- 25. CEB Active-Active Data Flow ① Workload balancer switch workload to BJ or SH mainframe according to card type ② Card application update local data ③ Card application write transaction input ISO8583 data to MQ ④ MQ transfer ISO8583 data to peer site ⑤ User application read data from MQ ⑥ User application do identical update in remote site’s date sets. AOR MQ AOR MQ Wrokload balancer ① ② ② ③ ③ ④ ④ ⑤ ⑤ ⑥ ⑥ BJ MF SH MF 24
- 26. Planned Site Switch Over Procedure Stop workload routing to BJ site Waiting for SH site duplex as BJ site data Workload re-rout to SH site Reverse GM from site B to site A 25
- 27. Unplanned Site Switch Over Procedure Stop workload routing to BJ site Workload re-rout to SH site Reverse GM from site B to site A 26
- 28. Characteristics of CEB’s Active-Active implementation For business which has less complex master data with less dependent database tables. For example, Credit Card business. The synchronization applications need to be developed according to your business and technical requirements, rather than an out-of-box product. 27
- 29. References WebSphere MQ V7.1 Information Center InfoSphere Data Replication for DB2 for z/OS and WebSphere Message Queue for z/OS: Performance Lessons, REDP-4947-00 GDPS Family An Introduction to Concepts and Facilities, SG24-6374-08 28
- 30. Questions?
- 31. We Value Your Feedback Don’t forget to submit your Impact session and speaker feedback! Your feedback is very important to us – we use it to continually improve the conference. Use the Conference Mobile App or the online Agenda Builder to quickly submit your survey • Navigate to “Surveys” to see a view of surveys for sessions you’ve attended 30
- 32. Thank You
- 33. Legal Disclaimer • © IBM Corporation 2014. All Rights Reserved. • The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. • References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. • If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete: Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. • If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete: All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. • Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. • If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete: Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. • If you reference Java™ in the text, please mark the first use and include the following; otherwise delete: Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. • If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete: Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. • If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete: Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. • If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete: UNIX is a registered trademark of The Open Group in the United States and other countries. • If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete: Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. • If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only. 32