IBM MQ - Monitoring and Managing Hybrid Messaging Environments
1 like1,492 views
The document discusses IBM's strategies for monitoring and managing hybrid messaging environments, focusing on the capabilities of various IBM MQ solutions, including on-prem and cloud options such as the IBM Message Hub. It outlines the tools available for administrators, including command-line interfaces and graphical user interfaces, as well as the support for future product features and functionalities. Additionally, it covers the importance of monitoring utilization and error detection in virtualized environments, highlighting the effectiveness of integration with third-party tools for log analysis and visualization.
IBM MQ - Monitoring and Managing Hybrid Messaging Environments
- 1. HHM-3537: Monitoring and Managing Hybrid Messaging Environments Mark Taylor IBM Hursley - UK
- 2. Please Note: 1 • IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. • Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. • The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. • The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. • Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
- 3. Digital IT Enterprise IT MQ cloud optionsIBM Message Hub Cloud On-Prem Cloud On-Prem IBM MQ Appliance Message Hub Based on Apache Kafka Message Hub Based on Apache Kafka Message Hub Based on Apache Kafka Public (GA: Dec 2015) Dedicated (Target: 1Q 2016) Local (Coming soon!) IBM MQ Light (API) IBM MQ IBM Messaging has Solutions to Suit All Needs
- 5. Queue Manager MQ Administration ini files OS commandsMQSC commands PCF commands Events (async) Status (polled) OS configuration These are all documented and supported interfaces – it has encouraged a management ecosystem over many years Log files
- 6. IBM MQ - MQSC • Command line interface • V8 enhanced runmqsc – Make it world-executable – Enable direct client-connection • MQSC intended for human consumption – Parsable by eye, less easy in programs – For example, DESCR('This is 'a' description with quote & paren(' ) – No guaranteed ordering in runmqsc, two-column output • Despite awkwardness, basis for many script-based admin tools – echo "DISPLAY Q(X) IPPROCS" | runmqsc QM1 • Same commands – different front-end (CSQUTIL) – for z/OS Old Example: AIX smit panels
- 7. IBM MQ - PCF • A "self-describing" MQ message used for administrative operations • Your programs can send commands and get responses using PCF – Equivalent to "DISPLAY QSTATUS" or "ALTER CHANNEL" • MQ emits events in PCF format – "Queue is getting full" • PCF intended for programs – usually C or Java – Can tell exactly what the parameter is for, its length and value – But cannot easily be scripted • Approximately one-one mapping between MQSC commands and PCF • Remember that PCF invented before formats like JSON or XML – And there are many MQ apps that are built on PCF
- 8. An event message **** Message length - 300 of 300 bytes *** 00000000: 0000 0007 0000 0024 0000 0003 0000 0063 '.......$.......c' 00000010: 0000 0001 0000 0001 0000 0000 0000 096C '...............l' 00000020: 0000 0002 0000 0014 0000 0010 0000 1F41 '...............A' 00000030: 0000 0004 0000 0004 0000 0020 0000 0BE5 '........... ...å' 00000040: 0000 0333 0000 000C 6D65 7461 796C 6F72 '...3....metaylor' 00000050: 2020 2020 0000 0003 0000 0010 0000 03F3 ' ...........ó' 00000060: 0000 0001 0000 0004 0000 0044 0000 0BE7 '...........D...ç' 00000070: 0000 0333 0000 0030 5638 3030 335F 4120 '...3...0V8003_A ' 00000080: 2020 2020 2020 2020 2020 2020 2020 2020 ' ' 00000090: 2020 2020 2020 2020 2020 2020 2020 2020 ' ' 000000A0: 2020 2020 2020 2020 0000 0003 0000 0010 ' ........' 000000B0: 0000 03FD 0000 005A 0000 0014 0000 0010 '...ý...Z........' 000000C0: 0000 1F42 0000 0004 0000 0004 0000 0018 '...B............' 000000D0: 0000 0BFB 0000 0000 0000 0001 5800 0000 '...û........X...' 000000E0: 0000 0003 0000 0010 0000 03F8 0000 0001 '...........ø....' 000000F0: 0000 0006 0000 0024 0000 0BF9 0000 0000 '.......$...ù....' 00000100: 0000 0001 0000 0008 6D65 7461 796C 6F72 '........metaylor' 00000110: 0000 0000 0000 0005 0000 0018 0000 045C '...............' 00000120: 0000 0002 0000 000B 0000 0009 '............ '
- 9. **** Message length - 300 of 300 bytes *** 00000000: 0000 0007 0000 0024 0000 0003 0000 0063 '.......$.......c' 00000010: 0000 0001 0000 0001 0000 0000 0000 096C '...............l' 00000020: 0000 0002 0000 0014 0000 0010 0000 1F41 '...............A' 00000030: 0000 0004 0000 0004 0000 0020 0000 0BE5 '........... ...å' 00000040: 0000 0333 0000 000C 6D65 7461 796C 6F72 '...3....metaylor' 00000050: 2020 2020 0000 0003 0000 0010 0000 03F3 ' ...........ó' 00000060: 0000 0001 0000 0004 0000 0044 0000 0BE7 '...........D...ç' 00000070: 0000 0333 0000 0030 5638 3030 335F 4120 '...3...0V8003_A ' 00000080: 2020 2020 2020 2020 2020 2020 2020 2020 ' ' 00000090: 2020 2020 2020 2020 2020 2020 2020 2020 ' ' 000000A0: 2020 2020 2020 2020 0000 0003 0000 0010 ' ........' 000000B0: 0000 03FD 0000 005A 0000 0014 0000 0010 '...ý...Z........' 000000C0: 0000 1F42 0000 0004 0000 0004 0000 0018 '...B............' 000000D0: 0000 0BFB 0000 0000 0000 0001 5800 0000 '...û........X...' 000000E0: 0000 0003 0000 0010 0000 03F8 0000 0001 '...........ø....' 000000F0: 0000 0006 0000 0024 0000 0BF9 0000 0000 '.......$...ù....' 00000100: 0000 0001 0000 0008 6D65 7461 796C 6F72 '........metaylor' 00000110: 0000 0000 0000 0005 0000 0018 0000 045C '...............' 00000120: 0000 0002 0000 000B 0000 0009 '............ ' An event message TYPE (cfst) LEN (24) LEN (1)CCSID (0)PARM (MQCA…) DATA
- 10. An event message decoded Event Type : Command Event Reason : Command MQSC Event created : 2015/06/03 13:28:20.51 GMT Correlation ID : 414D512056383030335F412020202020556F00F120001E05 COMMAND CONTEXT Event User Id : metaylor Event Origin : Console Event Queue Mgr : V8003_A Command : Set Auth Rec COMMAND DATA Auth Profile Name : X Object Type : Queue Principal Entity Names: metaylor Auth Add Auths : Output : Input
- 12. MQ Explorer • Provides an Eclipse front end that allows management of multiple local and remote queue managers Linux QM1 QM2 PCFz/OS MQ21 MQ23
- 13. MQ Explorer – not just MQ resources
- 14. MQ Explorer – Third-party extensions
- 19. MQ Appliance Console • Ships as part of the MQ Appliance • Provides a subset of MQ Explorer functionality • Also provides management of MQ Appliance specific capability such as HA/DR configuration MQ Appliance QM1 QM2 Liberty Appliance Console (in web browser) PCF REST
- 20. Statement of direction • IBM intends to deliver support for the new IBM MQ Console in IBM MQ – IBM intends to add support to enable the IBM MQ Console to configure and monitor IBM MQ as well as the IBM MQ Appliance. IBM intends to allow MQ administrators to make more effective use of the IBM MQ Console in environments in which both IBM MQ and the IBM MQ Appliance are installed • http://ibm.biz/MQConsoleSOD
- 21. Third-party solutions • Many vendor products – this screenshot from ITCAM/Omegamon
- 22. Using the building blocks in a cloud environment
- 23. Background • The MQ Light Bluemix service is provided by queue managers • Each queue manager runs in a VM in SoftLayer and is created and destroyed based on demand • The hosting company (IBM) needs a way to monitor and manage those queue managers – To understand utilization – To detect errors
- 24. Monitoring utilization QM1 QM2 PCF Metrics Generator VM QM1 QM2 PCF Metrics Generator GrafanaCarbon Whisper QM1 QM2 PCF Metrics Generator VM QM1 QM2 PCF Metrics Generator Graphite Visualization: Provides a dashboard to generate charts from the metrics Database: A time series database that stores the metrics Listener: Listens for metrics and stores them in the database Metrics gatherer: Custom Java code that queries queue manager status using PCF and generates the metrics in Graphite format QM1 QM2 PCF Metrics Generator VM QM1 QM2 PCF Metrics Gatherer
- 25. Design • Builds on existing monitoring infrastructure • Uses common tools to parse and represent data • Flexible generation of charts • Only the data we need is captured, reducing system load
- 26. Example output: entire service
- 27. Example output: single queue manager
- 28. Example output: smaller time interval
- 29. Detecting errors • Need to be able to visualize error logs from all queue managers across all VMs • Spikes in log entries indicates potential problem • Need to be able to filter and search data too
- 30. QM1 QM2 PCF Metrics Generator VM QM1 QM2 Logstash forwarder Error logs Error logs QM1 QM2 PCF Metrics Generator VM QM1 QM2 Logstash forwarder Error logs Error logs Detecting errors Kibana Logstash Elastic- search ELK Stack Interface for searching and visualising logs Stores and indexes the logs Collates incoming logs and stores themScrapes log updates and sends them to Logstash QM1 QM2 PCF Metrics Generator VM QM1 QM2 Logstash forwarder Error logs Error logs
- 31. Example output: single VM
- 32. Example output: single message view
- 34. Summary
- 35. Summary • Fundamental interfaces in MQ give the control and data • Have been used to build a variety of scale of solution • Can drive common modern toolkits for integration with other products
- 36. Questions?
- 37. Notices and Disclaimers Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law
- 38. Notices and Disclaimers Con’t. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
- 39. Thank You Your Feedback is Important! Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your smartphone, laptop or conference kiosk.