SlideShare a Scribd company logo

223: Modernization and Migrating from the ESB to Containers

Download as PPTX, PDF
T
Trevor Dolby

Migrating to ACE v12 and modernising to containers was the topic of the TechCon 2021 virtual experience. It discussed migrating existing ACE/IIB/WMB deployments and assets to ACE V12/11 using the mqsiextractcomponents command. This allows existing BAR files to run unchanged on new integration nodes and independent integration servers alongside existing deployments, enabling staged migration. It also covered modernizing integration by moving to containers and taking advantage of new features in ACE like the development experience and serverless capabilities.

Software
1 of 63
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
TechCon 2021 1 Virtual Experience TechCon 2021 Migrating to ACE v12 and modernising to containers August 25, 2021 David Coles App Connect Enterprise Senior Developer Trevor Dolby App Connect Enterprise Architect Application Integration | IBM Automation
TechCon 2021 2 Virtual Experience IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. Please Note
TechCon 2021 3 Virtual Experience •IBM, and the IBM logo are trademarks of International Business Machines Corporation, registered in many jurisdictions. •Other company, product and service names may be trademarks, registered marks or service marks of their respective owners. A current list of IBM trademarks is available on the web at "Copyright and trademark information" ibm.com/legal/copytrade.shtml Trademark Acknowledgements
TechCon 2021 4 Virtual Experience • Migrating to ACEv12 • Introduction + fundamentals • Migration concepts • Migration specifics • V11 -> V12 • V10 and earlier -> V11 or V12 • Any version -> V12 • Extract migration approaches • Modernising to containers • Evolution to agile integration • Containers versus containerization • ACE Development Experience • ACE & Transformation Advisor • Serverless ACE
TechCon 2021 5 Virtual Experience
TechCon 2021 6 Virtual Experience • Customers have existing ACE/IIB/WMB deployments and assets and want to bring these forward to ACE V12/11 • Existing BAR files should run unchanged • Subject to as few exceptions as possible • New V12/V11 integration nodes and independent integration servers can run alongside existing integration nodes and brokers • Supports staged migration • Tools should be available to help migrate configurations from earlier releases • Previously to V11 this was mqsimigratecomponents • There should be help to migrate to new topologies and platforms
TechCon 2021 7 Virtual Experience • V12 is a major release, hence… • Migration is needed for source code and integration nodes • Depending on source version and target topology some migration of people and processes needed to • V11 was a major re-engineering and so migration from pre-V11 versions represents a big change • If coming from v9 and earlier admin interfaces, admin security and interaction with MQ have changed significantly • For those coming from v10 the view will be more similar • For those coming from v11 the view will be almost identical • From V11 new topology options with independent integration servers
TechCon 2021 8 Virtual Experience Process Supervision Admin Web UI Internal Cfg Store Configurable Service Physical/ Virtual Machine HTTP Listener BAR Flow Node Process Supervision Admin Web UI Physical/ Virtual Machine Node Flow Policy Public Cfg Store BAR Flow Policy Public Cfg Store Flow Container BAR Policy Flow Policy ACE V11 and V12 HTTP Listener Admin Web UI UNZIP and GO ! IIB v10
TechCon 2021 9 Virtual Experience MBX Operator system Scripts Java Web QA or PROD system Integration node Default QM Integration node Default QM src code Other enterprise apps SAP CICS MQ Developer system Integration node Default QM Toolkit MBX gone Fork source code Default QMs needed? Admin via http not MQ Connect remote MQ Supported versions? Hardware platform + OS V10+ V10+ V10+ V10+
TechCon 2021 10 Virtual Experience • Ensure you’re at the latest V12 fix pack level • https://www.ibm.com/support/pages/recommended-fixes-ibm-app-connect-enterprise-ibm- integration-bus-and-websphere-message-broker • Review the Knowledge Center for known changes in behaviour to be aware of • https://www.ibm.com/docs/en/app-connect/12.0?topic=migration-behavioral-changes-in- version-120 • Review the changes of default behaviour in fixpacks tech doc • http://www-01.ibm.com/support/docview.wss?rs=849&uid=swg27049142 • Review the release notes for V12 • https://www.ibm.com/support/pages/node/6457259 • Check out any APARs you have in previous versions to ensure they’re in V12 • Consider installing V12 in a test environment to have a play • Check out the tutorials
TechCon 2021 11 Virtual Experience
TechCon 2021 12 Virtual Experience • Create new node or independent integration server • Configure in the appropriate manner • Watch out for the ‘unknown’ changes which were made to the previous install • Deploy bar files • Run! • Staged ‘safe’ approach • Old systems remain working until new systems are proven
TechCon 2021 13 Virtual Experience • Stop the node at the previous version • Run mqsimigratecomponents at the new version • Translates all deployed resources and configuration to that required by the new version • Start the node at the new version • Big-Bang approach • What happens if things do not work as expected? • mqsimigratecomponents had an ‘undo’ option, but does not remove the risk of code defects • System is down during the migration • Sometimes the only approach if the original source files/configuration scripts are not available • Used to be the only option when there was a 1 broker – 1 queue manager relationship where the QM name could not be changed. • Not an issues after V10 • mqsimigratecomponents not available in V11 or V12 • Can achieve mostly the equivalent results with extract migration
TechCon 2021 14 Virtual Experience • New command and approach to migration • Extract configuration and resources from existing node (backup) • Create independent integration server work directory • Create integration node • Cross platform • Allows for migration from retired platforms • Repeatable • Can roughly achieve in-place migration with the addition of a delete and extract to the same integration node name • Works with backups from V7 and later • V12 officially supports V10 and V11 backups • V11 officially supports V9 and V10 backups • Older backups supported in a limited manner • Try it, it will probably work and let us know if it does not • Can also be used to migrate to different topologies, platforms and clone integration nodes
TechCon 2021 15 Virtual Experience 1. Run mqsibackupbroker against an existing node 2. (optional) Transfer backup zip to new machine 3. Run mqsiextractcomponents against the backup specifying a name of the target integration node to extract to • New integration node is created • Can optionally delete an existing node with the same name as part of the extract • Matching integration server created for each server of the original node • Slightly different behaviours dependent on the source version of the backup • V10- • Deployed resources and containers are written to the run directory of each of the new servers • Each server gets a default application based on the name of the server if required • Configurable services are converted to policy and added to the node level ‘DefaultPolicies’ policy project and so are available to all servers • Registry settings are written to node.conf.yaml • Resource manager settings from each server are written to the new server’s server.conf.yaml • V11+ • Node and server conf.yaml files and run directory contents are copied across with minimal changes • Just checked for removed options and policy types • Credentials are copied across • Webadmin users and setdbparms credentials are copied across 4. Run mqsistart against the newly created integration node 5. Flows continue to work as before • V10- References to configurable services are returned the new policies
TechCon 2021 16 Virtual Experience Resource managers server.conf.yaml Policies Configurable services ISR policies Registry DSN data DSN data Deployed resources Deployed (sub)flows Run directory Node level default policy project node.conf.yaml Node configuration Web users Web users
TechCon 2021 17 Virtual Experience server.conf.yaml DSN data Run directory node.conf.yaml Web users server.conf.yaml DSN data Run directory node.conf.yaml Web users Credentials Credentials Credentials Credentials
TechCon 2021 18 Virtual Experience 1. Run mqsibackupbroker against an existing node 2. (optional) Transfer backup zip to new machine 3. Run mqsiextractcomponents against the backup specifying a new work-directory for an independent integration server + server name to extract • New work-directory structure is created • Slightly different behaviours dependent on the source version of the backup • V10- • Deployed resources and containers are written to the run directory • Each server gets a default application based on the name of the server if required • Configurable services are converted to policy and deployed • Registry and resource manager settings are written to server.conf.yaml • V11+ • Applicable node.conf.yaml properties added to server conf.yaml files and copied across • Run directory contents are copied across with minimal changes • Conf.yaml and run directory contents checked for removed options and policy types • Node level policy project copied into run directory • Server Credentials are copied across • Setdbparms credentials are copied across 4. Start an independent integration server pointing at the new work-directory 5. Flows continue to work as before • V10- References to configurable services are returned the new policies
TechCon 2021 19 Virtual Experience Resource managers server.conf.yaml Policies Configurable services ISR policies Registry DSN data DSN data Deployed resources Deployed (sub)flows Run directory Policy project Node configuration Web users
TechCon 2021 20 Virtual Experience server.conf.yaml Run directory DSN data Credentials server.conf.yaml DSN data Run directory node.conf.yaml Web users Credentials Credentials Node level default policy project
TechCon 2021 21 Virtual Experience
TechCon 2021 22 Virtual Experience • Migrating from V11 -> V12 should be no more troublesome than applying a fixpack in terms of configuration and behavioural changes • No massive re-engineering or changes to administration practices in V12 • Flows, deployed artifacts, config should all work • The major differentiator from a fixpack upgrade is the need to create new integration nodes at V12 • Possibly using mqsiextractcomponents to aid that creation • This is required because V12 commands and tooling only operates against V12 nodes and V11 against V11
TechCon 2021 23 Virtual Experience • HTTP(S)Connector policies removed • Deprecated in 11.0.0.5 • Use the yaml resource-manager settings • See later slides on any version -> V12 gotchas • JSON validation • Local webuser admin password
TechCon 2021 24 Virtual Experience • It is possible to run a V12 independent server against a V11 created work-directory • But why would you? • Independent servers and their configuration are designed to be short lived and run from containers • Ideally you would swap V11 for V12 in your pipeline and build and configure new work-directories using V12 and spin up new containers • To run against a V11 created work-directory you will need to handle the migration steps which mqsiextractcomponents takes care of when migrating an integration node • HTTP(S)Connector policies will fail to load so the configuration will have to be changed manually • The webuser password hash algorithm property will need to configured, or users will need to reset their passwords
TechCon 2021 25 Virtual Experience
TechCon 2021 26 Virtual Experience • V11 was a massive re-engineering release that requires significant reorganization of deployed resources and configuration during migration using mqsiextractcomponents • Some old nodes and resources have been removed requiring customer action to rework flows • Some administration commands have been removed or significantly changed requiring customer action to rework administration practices
TechCon 2021 27 Virtual Experience • Removed nodes • SCA / DecisionServices (replaced with ODMRules) / PHP / pre-v8 mapping nodes • Configurable services -> Policies • Used defined configurable services -> User defined policies • Java lookup code will need to be changed • Massive Integration API changes • Watch our if using IAPI from Java Compute nodes • Rest APIv1 -> Rest APIv2 • IBX -> Webui • Some commands removed + some changed in behaviour • Platform coverage • mqsiextractcomponents can help • Post deploy overrides not extracted • Stats, monitoring, flow start/stop state • Deploy info not extracted
TechCon 2021 28 Virtual Experience
TechCon 2021 29 Virtual Experience • Before V12 the JSON parser did not support validation and so it used to silently ignore if it was called with validation enabled • From V12 the parser is validation aware so it checks to see if validation is enabled, and if it is, it expects to be supplied with the name of a valid JSON schema file • If not configured appropriately then an exception is thrown • Users may hit unexpected problems on migration with ‘valid’ flows and so there are resource manager settings to disable the schema lookup exceptions JSON: #disableSchemaLookupExceptionWhen: '' # Comma-separated list of events that will not throw exceptions when validation is # requested of the JSON parser and a JSON schema cannot be loaded. # When an event in the list occurs, no exception will be thrown and instead the JSON # parser will continue to parse but without validation. # Valid events are 'notSpecified', 'notFound'. If unset will default to '' which means # that no exceptions are disabled. # 'notSpecified' disables the BIP5736 exception that is thrown if a JSON schema name is # not supplied. # 'notFound' disables the range of exceptions (BIP1318, BIP1312, BIP5737, BIP5738, # BIP5739) that are thrown if a JSON schema name is supplied, but cannot be # resolved. # This option is available to allow behaviour from previous releases to be maintained.
TechCon 2021 30 Virtual Experience • The default hashing algorithm for local webuser passwords has changed in V12 from SHA-1 to using PBKDF2 with SHA-512 • On migrate the web users are copied across as-is and so the passwords remain hashed using their original algorithm • A special algorithm property is also set in the overrides conf.yaml to indicate that the original hashing algorithm is in use • mqsiextractcomponents also issues a warning about this setting being added • It is ok to run with this setting, but we recommend that it is removed and users then reset their passwords to take advantage of the new, stronger, hashing algorithm RestAdminListener: webUserPasswordHashAlgorithm: SHA-1
TechCon 2021 31 Virtual Experience
TechCon 2021 32 Virtual Experience • mqsiextractcomponents is not a complete like for like replacement of mqsimigratecomponents in- place approach, but the same result can be achieved, particularly when using a V11 backup • You can backup and delete the existing node, then use mqsiextractcomponents to create a new node with the same name • If using a V11 backup then your new V12 integration node should now function and be configured as your V11 one was • However if using a V10 or earlier backup some configuration is deliberately not extracted (statistics, monitoring, flow/app start/stop state) and will need to be configured in V11 style • For V10 or earlier backups you will also still need to update any flows that use the removed nodes, Integration API, or access user-defined configurable services/policies
TechCon 2021 33 Virtual Experience • Side by side is the favoured approach • Use mqsiextractcomponents to extract the intent of your configuration and get it into V11/12 formats • Take the generated .policyxml files and copy them into a policy project in your toolkit ready for deployment to your new node/server • Take the generated .conf.yaml files and check them into source control and use them as-is to configure your new node/server • Or use the properties in them to help configure a new node/server • With the extracted .dfmxml and .dictionary files you get some form of ‘source’ for artifacts which you had previously deployed and then lost the original source message flow/set files • After extracting, you could delete all of the deployed artifacts from the run directories so you’re left with a configured node • Then re-deploy everything just to make sure you can • Extract a whole node to get the configuration, move out all of the directories (integration servers) in the servers directory, then move each server back in one by one to migrate a server at a time • To pickup any changes since the original backup, just redo the backup and run extract again targeting a temporary node, then just copy the newly extracted files or settings into your original node/server V10- V10- V10- V11+
TechCon 2021 34 Virtual Experience • As V12 mqsiextractcomponents supports extracting from a V12 backup you could use the command to help start a topology migration • Backup your V12 integration node • Run mqsiextractcomponents and target an independent integration server work directory and extract one of the integration servers from the integration node • You will then have a configured independent integration server work directory which can be used as the basis for moving the server to a container topology
TechCon 2021 35 Virtual Experience • As V12 mqsiextractcomponents supports extracting from a V12 backup you could use the command to clone an existing integration node • Backup your V12 integration node • Run mqsiextractcomponents and target a new integration node • This could be on the same machine but have a different name • Or be on a different machine, and maybe even different architecture • For example, a Windows to zLinux move • The new integration node should be configured identically to the original • If cloning to the same machine you will need to take note of the flows deployed and any requirements for things to be isolated or configuration which may need to be updated to avoid clashes • For example, admin or http ports, default queue manager names, aggregation node queue manager settings
TechCon 2021 36 Virtual Experience • You could also use mqsiextractcomponents to help you recover/extend/duplicate your integration nodes at previous versions • The sparse .conf.yaml files show you what properties you need to configure to replicate most of the old configuration
TechCon 2021 37 Virtual Experience
TechCon 2021 38 Virtual Experience People & Process Architecture Technology How can we improve development agility in order to accelerate innovation? Development Agility How can we improve build independence and increase production velocity? Deployment Agility How can we improve our ability to deliver dynamic scalability and inherent resilience? Operational Agility
TechCon 2021 39 Virtual Experience APIM APIM API Management APIM API Management APIM Gateway Integration Integration Int. Engagement applications Systems of record API Management Socialization/monetization Re-platforming Application autonomy API Management Centralized ESB Fine-grained integration deployment Decentralized integration ownership Socialized APIs Webinars http://ibm.biz/agile-integration-webcasts eBooklet http://ibm.biz/agile-integration-ebook IBM Redbook http://ibm.biz/agile-integration-redbook APIM APIM
TechCon 2021 40 Virtual Experience First split by business domains and functional areas to ensure high-level autonomy integration artefact integration runtime Domain 1 Domain 2 Domain 3 before after All Domains
TechCon 2021 41 Virtual Experience Next, consider non-functionals such as a) which need a separate pipeline (for agility), b) which need independent scalability, c) which have unique resilience requirements 41 before after Domain 1 Domain 1 Shared lifecycle (e.g. shared data model) Stabile requirements (e.g. integrations that haven’t needed changes in years) Specific resilience (e.g. require high replication factor to achieve availability) Inter-related availability (e.g. reliant on the availability of one another) Technical dependencies (e.g. all require local MQ server due to 2PC) Elastic scalability (e.g. occasional high spikes in usage) http://ibm.biz/aia-granularity integration artefact integration runtime
TechCon 2021 42 Virtual Experience
TechCon 2021 43 Virtual Experience • ACE does not store persistent data, and is best seen as application layer • MQSI (long ago) would have fit on the other side • Code as well as configuration • BAR files are not the same as MQSC files Application layer (transient) Java Node.js ACE Python Persistence layer Database MQ State storage and messaging
TechCon 2021 44 Virtual Experience Asset repo BAR1 BAR2 BAR3 ACE DFE 2 ACE DFE 1 ACE Integration Node App2 App3 App1 Source repo App1 App2 App3 BAR build Deploy BARs and manage integration servers Manage integration infrastructure • Many existing installations use integration nodes managed by dedicated administrators • Until ACE v11 this was the only option, and might still be appropriate • For example, if an organisational strategy for containers is still in progress
TechCon 2021 45 Virtual Experience Container infrastructure • Drivers ＋ Use containers as lightweight VMs to eliminate on-prem hardware ＋ Keeps admin model the same for ACE ＋ Might save on hardware ACE DFE 2 ACE DFE 1 ACE Integration Node App2 App3 App1 Container Manage container infrastructure Manage integration infrastructure • Challenges － Not what is usually meant by “containerization” － Does not provide any modernization benefits － Not recommended
TechCon 2021 46 Virtual Experience • Drivers ＋ Preserving admin model reduces cost of transition in the short term ＋ Lighter containers than option 1 • Challenges － Still managed as mini-nodes, limits modernization gains － Still need patching and potential maintenance window － Containers may need persistent storage, complicating container management Container infrastructure ACE server1 ACE server2 App2 App3 App1 Container Container Manage container infrastructure Manage integration infrastructure
TechCon 2021 47 Virtual Experience • Drivers ＋ Industry-standard tools in the pipeline and image build, simplifying management and operations ＋ + Container-based operation allows for rolling updates and zero downtime • Challenges － Significant change in admin model, requiring greater container literacy on the part of integration staff. － Often depends on company-wide cloud and container strategy (if one exists) － Requires placement of correct combination of apps in servers Source repo App1 App2 App3 Image build and deploy Manage container infrastructure ACE server1 ACE server2 App2 App3 App1 Container Container Container infrastructure
TechCon 2021 48 Virtual Experience • Drivers ＋ Industry-standard tools in the pipeline and image build, simplifying management and operations ＋ Lighter containers ＋ Each application in a separate container to scale better and allow independent management ＋ Often regarded as the “microservice” approach; might go as far as to put each flow in a container • Challenges － Significant change in admin model, requiring greater container literacy on the part of integration staff － Often depends on company-wide cloud and container strategy (if one exists) － Overhead in memory and management/licensing/cost － Not always needed; only in very-high-traffic scenarios would this be essential Source repo App1 App2 App3 Image build and deploy ACE server1 App1 Container ACE server2 App2 Container ACE server3 App3 Container Manage container infrastructure Container infrastructure
TechCon 2021 49 Virtual Experience
TechCon 2021 50 Virtual Experience Server Test Server Test Server Test • Components built in containers or VMs (platform- and technology-dependent) • Main server build integrates the runtime components; uses large-scale container testing for validation • Combined with the toolkit for more container testing, and then feeds into ACEcc and ACEoC as well as software and fixpack builds • Enables agility in implementing features, fixing defects, adding platform variants, etc Message catalogs Container infrastructure Common Java Web UI . . . Toolkit build and test Server Build Server Test Combined Build Server Test Server Test Server Test Container infrastructure Combined Test ACEcc Image ACE on Cloud ACE software Windows as well as Linux
TechCon 2021 51 Virtual Experience
52 Assess and analyze applications, integrations and supporting infrastructure and services Discovery Inventory and dependencies Assessment Classify simple / moderate / complex based on target compatibility Code Assistance Potential issues, severity details, possible solutions and estimated resolution effort Included and deployed on IBM Cloud Paks TA modernizes WebSphere, WebLogic, MQ and ACE/IIB deployments Provides recommendations and automation for App Modernization Migration Automation Generate build and deploy artifacts to help on-ramp to selected target platform and runtime Holistic Selectively modernize all the components of your Business Application as a unit http://ibm.biz/cloudta
TechCon 2021 53 Virtual Experience • Text goes here lorem ipsum dolor sit amet, consectetur adipiscing elit. Complexity Java EE IBM MQ IIB No code changes needed DNS reconfiguration required Admin change is required Code changes required Cluster reconfiguration, changing custom logic (e.g. Exits) Development change is required Incompatible technologies; external dependencies Client Authentication reconfiguration Difficult development task or alternate technology is required Simple Moderate Complex Severity IIB No immediate action is required, but you may wish to be aware. Immediate action is probably required or advised before you proceed. You cannot proceed without taking remedial action. Error Warning Info
TechCon 2021 54 Virtual Experience
TechCon 2021 55 Virtual Experience
TechCon 2021 56 Virtual Experience https://www.ibm.com/support/knowledgecenter/en/SSTTDS_11.0.0/com.ibm.etools.mft.doc/bh23394_.html
TechCon 2021 57 Virtual Experience
TechCon 2021 58 Virtual Experience • “Serverless” has been used to describe quite a lot of different approaches and technologies • Absence of an application server rather than a physical machine • ACE CI/CD pipeline can deploy as a serverless application as well as deploying to containers and integration nodes • Goal: avoid running an integration server all the time • Licensing cost is lower • Resource consumption lower overall • Two forms: container-based scaling and Function-as-a-Service • Knative, KEDA, etc in first category • Amazon Lambda and IBM Cloud Functions in the second Cloud Function ACE integration node running on-prem
TechCon 2021 59 Virtual Experience • Knative acts as a front-end for ACE • Receives HTTP traffic and starts/stops containers as needed • Will scale to zero after inactivity timeout • ace-demo-pipeline has knative option • KEDA handles scaling only • ACE message flows get messages directly from the MQ queue manager (or other messaging source such as Kafka) • Polling-based approach to scaling, where the MQ queue depth determines how many containers will be started • Will scale to zero after inactivity timeout • Both require a complete ACE container with applications and credentials already set up Containers ACE app Containers ACE app MQ Queue Manager Application traffic (MQ) Application traffic (HTTP) Knative infra- structure KEDA infrastructure MQ messages picked up by ACE message flow Queue depth polling Container scaling Container scaling HTTP data
TechCon 2021 60 Virtual Experience • Running ACE flows as one-shot containers • Zero overhead when not actively working • Container infrastructure abstracted: could be using any technology that runs containers • Cloud Foundry, Kubernets, Docker Swarm, etc • More setup/teardown, and connections would have to be made every time • Startup latency a concern in many cases • Require more reengineering of applications and deployment pipelines in many cases • FaaS best suited to short-lived applications that do not block on network interactions • Matches traditional out-and-back ACE message flow architecture in use since MQSI v2 • Much higher latency than the traditional server • Works best with occasionally-used flows that can accept high latency, or in situations where container management needs to be abstracted out One-Shot Container ACE app Application traffic (MQ) Application traffic (HTTP) Cloud Function infra- structure Application can connect to database One-Shot Container ACE Request Cloud Function infra- structure Kafka Service One-Shot Container ACE Response Cloud Function infra- structure DB
TechCon 2021 61 Virtual Experience • Function-as-a-Service has a higher latency but no idle time • Knative and KEDA use the containers for longer after they have started • This example shows a short-lived flow, which might not be ideal for FaaS; longer-running flows would amortize the startup cost over a greater period and be more efficient Startup Flow run Time 30 seconds 60 seconds 90 seconds 120 seconds Startup Flow run1 Flow run2 Flow run3 Idle FaaS Knative or KEDA Container scaling Container start One request sent to the server Container shutdown Multiple requests sent to the server
TechCon 2021 62 Virtual Experience Questions?
TechCon 2021 63 Virtual Experience

More Related Content

PPTX
Eucalyptus, Nimbus & OpenNebula
Amar Myana
 
PDF
IBM INTEGRATION BUS (IIB V10)—DATA ROUTING AND TRANSFORMATION
Kellton Tech Solutions Ltd
 
PPTX
414: Build an agile CI/CD Pipeline for application integration
Trevor Dolby
 
PPT
cloud computing.ppt
MunmunSaha7
 
PPTX
IBM Websphere introduction and installation for beginners
Shubham Gupta
 
PPTX
Cloud Computing Design Considerations
Mike Kavis
 
PPTX
Introduction to cloud computing
Jithin Parakka
 
PPTX
Websphere Application Server V8.5
IBM WebSphereIndia
 
Eucalyptus, Nimbus & OpenNebula
Amar Myana
 
IBM INTEGRATION BUS (IIB V10)—DATA ROUTING AND TRANSFORMATION
Kellton Tech Solutions Ltd
 
414: Build an agile CI/CD Pipeline for application integration
Trevor Dolby
 
cloud computing.ppt
MunmunSaha7
 
IBM Websphere introduction and installation for beginners
Shubham Gupta
 
Cloud Computing Design Considerations
Mike Kavis
 
Introduction to cloud computing
Jithin Parakka
 
Websphere Application Server V8.5
IBM WebSphereIndia
 

What's hot (20)

PPT
Legal And Regulatory Issues Cloud Computing...V2.0
David Spinks
 
PPTX
Virtualization Vs. Containers
actualtechmedia
 
PPT
WebSphere application server 8.5.5 - quick overview
Chris Sparshott
 
PPTX
virtualization and hypervisors
Gaurav Suri
 
PDF
Cloud Computing and Service oriented Architecture
Ravindra Dastikop
 
DOC
IT Redhat Linux Resume
RAJESH KUMAR
 
PPTX
Cloud computing hybrid architecture
Abhijeet Singh
 
PPTX
IBM MQ Whats new - including 9.3 and 9.3.1
Robert Parker
 
PPTX
Mule with rabbitmq
Anirban Sen Chowdhary
 
PPTX
Common Standards in Cloud Computing
mrzahidfaiz.blogspot.com
 
PPTX
Publish subscribe model overview
Ishraq Al Fataftah
 
PPTX
Cloud Computing PPT.pptx
HetKhandol
 
PPTX
IBM MQ Overview (IBM Message Queue)
Juarez Junior
 
PDF
Kafka with IBM Event Streams - Technical Presentation
Winton Winton
 
PPTX
2 vm provisioning
ROSHNI PRADHAN
 
PDF
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
David Ware
 
PDF
Kernel Recipes 2017 - HDMI CEC: Status Report - Hans Verkuil
Anne Nicolas
 
PPTX
Cloud computing using Eucalyptus
Abhishek Dey
 
PDF
DataPower API Gateway Performance Benchmarks
IBM DataPower Gateway
 
PPT
8. Wireless (1).ppt
SAMBIT34
 
Legal And Regulatory Issues Cloud Computing...V2.0
David Spinks
 
Virtualization Vs. Containers
actualtechmedia
 
WebSphere application server 8.5.5 - quick overview
Chris Sparshott
 
virtualization and hypervisors
Gaurav Suri
 
Cloud Computing and Service oriented Architecture
Ravindra Dastikop
 
IT Redhat Linux Resume
RAJESH KUMAR
 
Cloud computing hybrid architecture
Abhijeet Singh
 
IBM MQ Whats new - including 9.3 and 9.3.1
Robert Parker
 
Mule with rabbitmq
Anirban Sen Chowdhary
 
Common Standards in Cloud Computing
mrzahidfaiz.blogspot.com
 
Publish subscribe model overview
Ishraq Al Fataftah
 
Cloud Computing PPT.pptx
HetKhandol
 
IBM MQ Overview (IBM Message Queue)
Juarez Junior
 
Kafka with IBM Event Streams - Technical Presentation
Winton Winton
 
2 vm provisioning
ROSHNI PRADHAN
 
IBM MQ: An Introduction to Using and Developing with MQ Publish/Subscribe
David Ware
 
Kernel Recipes 2017 - HDMI CEC: Status Report - Hans Verkuil
Anne Nicolas
 
Cloud computing using Eucalyptus
Abhishek Dey
 
DataPower API Gateway Performance Benchmarks
IBM DataPower Gateway
 
8. Wireless (1).ppt
SAMBIT34
 
Ad

Similar to 223: Modernization and Migrating from the ESB to Containers (20)

PDF
SHARE2016: DevOps - IIB Administration for Continuous Delivery and DevOps
Rob Convery
 
PPT
SHARE 2014 Pittsburgh, Managing multi version applications in cics
nick_garrod
 
ODP
Building highly available architectures with WAS and MQ
Matthew White
 
PDF
ACU-1445 - Bringing workloads into production on PureApplication System
Hendrik van Run
 
PDF
Robust integration with tivoli directory integrator 7.0 redp4672
Banking at Ho Chi Minh city
 
PDF
Robust integration with tivoli directory integrator 7.0 redp4672
Banking at Ho Chi Minh city
 
PDF
Whats new in IIB v9 + Open Beta v10 GSE
Dominic Storey
 
PDF
Continuous Application Delivery to WebSphere - Featuring IBM UrbanCode
IBM UrbanCode Products
 
PDF
IBM Middleware Upgrade webinar by CROZ June 2020
Miroslav Resetar
 
PDF
Moving to IBM PureFlex System x86-to-x86 Migration
IBM India Smarter Computing
 
PPTX
Share multi versioning scenarios
nick_garrod
 
PDF
What’s new in IBM BPM 8.5.7 CF2016.06 - CF2017.03
Dennis Parrott
 
PDF
Tivoli and web sphere application server on z os sg247062
Banking at Ho Chi Minh city
 
PDF
How to Containerize WebSphere Application Server Traditional, and Why You Mig...
David Currie
 
PDF
IBM Workload Deployer
IBM India Smarter Computing
 
PPTX
WhatsNewIBMIntegrationBus10FP4
bthomps1979
 
PDF
M10: How to implement mq in a containerized architecture ITC 2019
Robert Parker
 
PPTX
Deep Dive into FME Server 2013
Safe Software
 
PDF
Was migration benefits, planning, best practices
sflynn073
 
PDF
Managing multi-version applications in cics
Matthew Webster
 
SHARE2016: DevOps - IIB Administration for Continuous Delivery and DevOps
Rob Convery
 
SHARE 2014 Pittsburgh, Managing multi version applications in cics
nick_garrod
 
Building highly available architectures with WAS and MQ
Matthew White
 
ACU-1445 - Bringing workloads into production on PureApplication System
Hendrik van Run
 
Robust integration with tivoli directory integrator 7.0 redp4672
Banking at Ho Chi Minh city
 
Robust integration with tivoli directory integrator 7.0 redp4672
Banking at Ho Chi Minh city
 
Whats new in IIB v9 + Open Beta v10 GSE
Dominic Storey
 
Continuous Application Delivery to WebSphere - Featuring IBM UrbanCode
IBM UrbanCode Products
 
IBM Middleware Upgrade webinar by CROZ June 2020
Miroslav Resetar
 
Moving to IBM PureFlex System x86-to-x86 Migration
IBM India Smarter Computing
 
Share multi versioning scenarios
nick_garrod
 
What’s new in IBM BPM 8.5.7 CF2016.06 - CF2017.03
Dennis Parrott
 
Tivoli and web sphere application server on z os sg247062
Banking at Ho Chi Minh city
 
How to Containerize WebSphere Application Server Traditional, and Why You Mig...
David Currie
 
IBM Workload Deployer
IBM India Smarter Computing
 
WhatsNewIBMIntegrationBus10FP4
bthomps1979
 
M10: How to implement mq in a containerized architecture ITC 2019
Robert Parker
 
Deep Dive into FME Server 2013
Safe Software
 
Was migration benefits, planning, best practices
sflynn073
 
Managing multi-version applications in cics
Matthew Webster
 
Ad

Recently uploaded (20)

PPTX
Materi_Pemrograman_Komputer-Looping.pptx
RanuFajar1
 
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
PDF
top salesforce developer skills in 2025.pdf
CloudMetic
 
PDF
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
ghrom2211g
 
PPTX
ManageIQ - Sprint 268 Review - Slide Deck
ManageIQ
 
PDF
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
PPTX
Essential Infomation Tech presentation.pptx
pradeepjava2016
 
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 
PDF
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
PPTX
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
AnujKumar530915
 
PDF
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PPTX
Transform Your Business with a Software ERP System
Canada Software Company
 
PPTX
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
PDF
medical staffing services at VALiNTRY
Tiyan Grayson
 
PPTX
Online Work Permit System for Fast Permit Processing
SHEQ Network Limited
 
PDF
Design an Analysis of Algorithms II-SECS-1021-03
DilanEscobar1
 
Materi_Pemrograman_Komputer-Looping.pptx
RanuFajar1
 
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
top salesforce developer skills in 2025.pdf
CloudMetic
 
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
ghrom2211g
 
ManageIQ - Sprint 268 Review - Slide Deck
ManageIQ
 
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
Essential Infomation Tech presentation.pptx
pradeepjava2016
 
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
AnujKumar530915
 
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
Transform Your Business with a Software ERP System
Canada Software Company
 
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
medical staffing services at VALiNTRY
Tiyan Grayson
 
Online Work Permit System for Fast Permit Processing
SHEQ Network Limited
 
Design an Analysis of Algorithms II-SECS-1021-03
DilanEscobar1
 

223: Modernization and Migrating from the ESB to Containers

  • 1. TechCon 2021 1 Virtual Experience TechCon 2021 Migrating to ACE v12 and modernising to containers August 25, 2021 David Coles App Connect Enterprise Senior Developer Trevor Dolby App Connect Enterprise Architect Application Integration | IBM Automation
  • 2. TechCon 2021 2 Virtual Experience IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. Please Note
  • 3. TechCon 2021 3 Virtual Experience •IBM, and the IBM logo are trademarks of International Business Machines Corporation, registered in many jurisdictions. •Other company, product and service names may be trademarks, registered marks or service marks of their respective owners. A current list of IBM trademarks is available on the web at "Copyright and trademark information" ibm.com/legal/copytrade.shtml Trademark Acknowledgements
  • 4. TechCon 2021 4 Virtual Experience • Migrating to ACEv12 • Introduction + fundamentals • Migration concepts • Migration specifics • V11 -> V12 • V10 and earlier -> V11 or V12 • Any version -> V12 • Extract migration approaches • Modernising to containers • Evolution to agile integration • Containers versus containerization • ACE Development Experience • ACE & Transformation Advisor • Serverless ACE
  • 6. TechCon 2021 6 Virtual Experience • Customers have existing ACE/IIB/WMB deployments and assets and want to bring these forward to ACE V12/11 • Existing BAR files should run unchanged • Subject to as few exceptions as possible • New V12/V11 integration nodes and independent integration servers can run alongside existing integration nodes and brokers • Supports staged migration • Tools should be available to help migrate configurations from earlier releases • Previously to V11 this was mqsimigratecomponents • There should be help to migrate to new topologies and platforms
  • 7. TechCon 2021 7 Virtual Experience • V12 is a major release, hence… • Migration is needed for source code and integration nodes • Depending on source version and target topology some migration of people and processes needed to • V11 was a major re-engineering and so migration from pre-V11 versions represents a big change • If coming from v9 and earlier admin interfaces, admin security and interaction with MQ have changed significantly • For those coming from v10 the view will be more similar • For those coming from v11 the view will be almost identical • From V11 new topology options with independent integration servers
  • 8. TechCon 2021 8 Virtual Experience Process Supervision Admin Web UI Internal Cfg Store Configurable Service Physical/ Virtual Machine HTTP Listener BAR Flow Node Process Supervision Admin Web UI Physical/ Virtual Machine Node Flow Policy Public Cfg Store BAR Flow Policy Public Cfg Store Flow Container BAR Policy Flow Policy ACE V11 and V12 HTTP Listener Admin Web UI UNZIP and GO ! IIB v10
  • 9. TechCon 2021 9 Virtual Experience MBX Operator system Scripts Java Web QA or PROD system Integration node Default QM Integration node Default QM src code Other enterprise apps SAP CICS MQ Developer system Integration node Default QM Toolkit MBX gone Fork source code Default QMs needed? Admin via http not MQ Connect remote MQ Supported versions? Hardware platform + OS V10+ V10+ V10+ V10+
  • 10. TechCon 2021 10 Virtual Experience • Ensure you’re at the latest V12 fix pack level • https://www.ibm.com/support/pages/recommended-fixes-ibm-app-connect-enterprise-ibm- integration-bus-and-websphere-message-broker • Review the Knowledge Center for known changes in behaviour to be aware of • https://www.ibm.com/docs/en/app-connect/12.0?topic=migration-behavioral-changes-in- version-120 • Review the changes of default behaviour in fixpacks tech doc • http://www-01.ibm.com/support/docview.wss?rs=849&uid=swg27049142 • Review the release notes for V12 • https://www.ibm.com/support/pages/node/6457259 • Check out any APARs you have in previous versions to ensure they’re in V12 • Consider installing V12 in a test environment to have a play • Check out the tutorials
  • 12. TechCon 2021 12 Virtual Experience • Create new node or independent integration server • Configure in the appropriate manner • Watch out for the ‘unknown’ changes which were made to the previous install • Deploy bar files • Run! • Staged ‘safe’ approach • Old systems remain working until new systems are proven
  • 13. TechCon 2021 13 Virtual Experience • Stop the node at the previous version • Run mqsimigratecomponents at the new version • Translates all deployed resources and configuration to that required by the new version • Start the node at the new version • Big-Bang approach • What happens if things do not work as expected? • mqsimigratecomponents had an ‘undo’ option, but does not remove the risk of code defects • System is down during the migration • Sometimes the only approach if the original source files/configuration scripts are not available • Used to be the only option when there was a 1 broker – 1 queue manager relationship where the QM name could not be changed. • Not an issues after V10 • mqsimigratecomponents not available in V11 or V12 • Can achieve mostly the equivalent results with extract migration
  • 14. TechCon 2021 14 Virtual Experience • New command and approach to migration • Extract configuration and resources from existing node (backup) • Create independent integration server work directory • Create integration node • Cross platform • Allows for migration from retired platforms • Repeatable • Can roughly achieve in-place migration with the addition of a delete and extract to the same integration node name • Works with backups from V7 and later • V12 officially supports V10 and V11 backups • V11 officially supports V9 and V10 backups • Older backups supported in a limited manner • Try it, it will probably work and let us know if it does not • Can also be used to migrate to different topologies, platforms and clone integration nodes
  • 15. TechCon 2021 15 Virtual Experience 1. Run mqsibackupbroker against an existing node 2. (optional) Transfer backup zip to new machine 3. Run mqsiextractcomponents against the backup specifying a name of the target integration node to extract to • New integration node is created • Can optionally delete an existing node with the same name as part of the extract • Matching integration server created for each server of the original node • Slightly different behaviours dependent on the source version of the backup • V10- • Deployed resources and containers are written to the run directory of each of the new servers • Each server gets a default application based on the name of the server if required • Configurable services are converted to policy and added to the node level ‘DefaultPolicies’ policy project and so are available to all servers • Registry settings are written to node.conf.yaml • Resource manager settings from each server are written to the new server’s server.conf.yaml • V11+ • Node and server conf.yaml files and run directory contents are copied across with minimal changes • Just checked for removed options and policy types • Credentials are copied across • Webadmin users and setdbparms credentials are copied across 4. Run mqsistart against the newly created integration node 5. Flows continue to work as before • V10- References to configurable services are returned the new policies
  • 16. TechCon 2021 16 Virtual Experience Resource managers server.conf.yaml Policies Configurable services ISR policies Registry DSN data DSN data Deployed resources Deployed (sub)flows Run directory Node level default policy project node.conf.yaml Node configuration Web users Web users
  • 17. TechCon 2021 17 Virtual Experience server.conf.yaml DSN data Run directory node.conf.yaml Web users server.conf.yaml DSN data Run directory node.conf.yaml Web users Credentials Credentials Credentials Credentials
  • 18. TechCon 2021 18 Virtual Experience 1. Run mqsibackupbroker against an existing node 2. (optional) Transfer backup zip to new machine 3. Run mqsiextractcomponents against the backup specifying a new work-directory for an independent integration server + server name to extract • New work-directory structure is created • Slightly different behaviours dependent on the source version of the backup • V10- • Deployed resources and containers are written to the run directory • Each server gets a default application based on the name of the server if required • Configurable services are converted to policy and deployed • Registry and resource manager settings are written to server.conf.yaml • V11+ • Applicable node.conf.yaml properties added to server conf.yaml files and copied across • Run directory contents are copied across with minimal changes • Conf.yaml and run directory contents checked for removed options and policy types • Node level policy project copied into run directory • Server Credentials are copied across • Setdbparms credentials are copied across 4. Start an independent integration server pointing at the new work-directory 5. Flows continue to work as before • V10- References to configurable services are returned the new policies
  • 19. TechCon 2021 19 Virtual Experience Resource managers server.conf.yaml Policies Configurable services ISR policies Registry DSN data DSN data Deployed resources Deployed (sub)flows Run directory Policy project Node configuration Web users
  • 20. TechCon 2021 20 Virtual Experience server.conf.yaml Run directory DSN data Credentials server.conf.yaml DSN data Run directory node.conf.yaml Web users Credentials Credentials Node level default policy project
  • 22. TechCon 2021 22 Virtual Experience • Migrating from V11 -> V12 should be no more troublesome than applying a fixpack in terms of configuration and behavioural changes • No massive re-engineering or changes to administration practices in V12 • Flows, deployed artifacts, config should all work • The major differentiator from a fixpack upgrade is the need to create new integration nodes at V12 • Possibly using mqsiextractcomponents to aid that creation • This is required because V12 commands and tooling only operates against V12 nodes and V11 against V11
  • 23. TechCon 2021 23 Virtual Experience • HTTP(S)Connector policies removed • Deprecated in 11.0.0.5 • Use the yaml resource-manager settings • See later slides on any version -> V12 gotchas • JSON validation • Local webuser admin password
  • 24. TechCon 2021 24 Virtual Experience • It is possible to run a V12 independent server against a V11 created work-directory • But why would you? • Independent servers and their configuration are designed to be short lived and run from containers • Ideally you would swap V11 for V12 in your pipeline and build and configure new work-directories using V12 and spin up new containers • To run against a V11 created work-directory you will need to handle the migration steps which mqsiextractcomponents takes care of when migrating an integration node • HTTP(S)Connector policies will fail to load so the configuration will have to be changed manually • The webuser password hash algorithm property will need to configured, or users will need to reset their passwords
  • 26. TechCon 2021 26 Virtual Experience • V11 was a massive re-engineering release that requires significant reorganization of deployed resources and configuration during migration using mqsiextractcomponents • Some old nodes and resources have been removed requiring customer action to rework flows • Some administration commands have been removed or significantly changed requiring customer action to rework administration practices
  • 27. TechCon 2021 27 Virtual Experience • Removed nodes • SCA / DecisionServices (replaced with ODMRules) / PHP / pre-v8 mapping nodes • Configurable services -> Policies • Used defined configurable services -> User defined policies • Java lookup code will need to be changed • Massive Integration API changes • Watch our if using IAPI from Java Compute nodes • Rest APIv1 -> Rest APIv2 • IBX -> Webui • Some commands removed + some changed in behaviour • Platform coverage • mqsiextractcomponents can help • Post deploy overrides not extracted • Stats, monitoring, flow start/stop state • Deploy info not extracted
  • 29. TechCon 2021 29 Virtual Experience • Before V12 the JSON parser did not support validation and so it used to silently ignore if it was called with validation enabled • From V12 the parser is validation aware so it checks to see if validation is enabled, and if it is, it expects to be supplied with the name of a valid JSON schema file • If not configured appropriately then an exception is thrown • Users may hit unexpected problems on migration with ‘valid’ flows and so there are resource manager settings to disable the schema lookup exceptions JSON: #disableSchemaLookupExceptionWhen: '' # Comma-separated list of events that will not throw exceptions when validation is # requested of the JSON parser and a JSON schema cannot be loaded. # When an event in the list occurs, no exception will be thrown and instead the JSON # parser will continue to parse but without validation. # Valid events are 'notSpecified', 'notFound'. If unset will default to '' which means # that no exceptions are disabled. # 'notSpecified' disables the BIP5736 exception that is thrown if a JSON schema name is # not supplied. # 'notFound' disables the range of exceptions (BIP1318, BIP1312, BIP5737, BIP5738, # BIP5739) that are thrown if a JSON schema name is supplied, but cannot be # resolved. # This option is available to allow behaviour from previous releases to be maintained.
  • 30. TechCon 2021 30 Virtual Experience • The default hashing algorithm for local webuser passwords has changed in V12 from SHA-1 to using PBKDF2 with SHA-512 • On migrate the web users are copied across as-is and so the passwords remain hashed using their original algorithm • A special algorithm property is also set in the overrides conf.yaml to indicate that the original hashing algorithm is in use • mqsiextractcomponents also issues a warning about this setting being added • It is ok to run with this setting, but we recommend that it is removed and users then reset their passwords to take advantage of the new, stronger, hashing algorithm RestAdminListener: webUserPasswordHashAlgorithm: SHA-1
  • 32. TechCon 2021 32 Virtual Experience • mqsiextractcomponents is not a complete like for like replacement of mqsimigratecomponents in- place approach, but the same result can be achieved, particularly when using a V11 backup • You can backup and delete the existing node, then use mqsiextractcomponents to create a new node with the same name • If using a V11 backup then your new V12 integration node should now function and be configured as your V11 one was • However if using a V10 or earlier backup some configuration is deliberately not extracted (statistics, monitoring, flow/app start/stop state) and will need to be configured in V11 style • For V10 or earlier backups you will also still need to update any flows that use the removed nodes, Integration API, or access user-defined configurable services/policies
  • 33. TechCon 2021 33 Virtual Experience • Side by side is the favoured approach • Use mqsiextractcomponents to extract the intent of your configuration and get it into V11/12 formats • Take the generated .policyxml files and copy them into a policy project in your toolkit ready for deployment to your new node/server • Take the generated .conf.yaml files and check them into source control and use them as-is to configure your new node/server • Or use the properties in them to help configure a new node/server • With the extracted .dfmxml and .dictionary files you get some form of ‘source’ for artifacts which you had previously deployed and then lost the original source message flow/set files • After extracting, you could delete all of the deployed artifacts from the run directories so you’re left with a configured node • Then re-deploy everything just to make sure you can • Extract a whole node to get the configuration, move out all of the directories (integration servers) in the servers directory, then move each server back in one by one to migrate a server at a time • To pickup any changes since the original backup, just redo the backup and run extract again targeting a temporary node, then just copy the newly extracted files or settings into your original node/server V10- V10- V10- V11+
  • 34. TechCon 2021 34 Virtual Experience • As V12 mqsiextractcomponents supports extracting from a V12 backup you could use the command to help start a topology migration • Backup your V12 integration node • Run mqsiextractcomponents and target an independent integration server work directory and extract one of the integration servers from the integration node • You will then have a configured independent integration server work directory which can be used as the basis for moving the server to a container topology
  • 35. TechCon 2021 35 Virtual Experience • As V12 mqsiextractcomponents supports extracting from a V12 backup you could use the command to clone an existing integration node • Backup your V12 integration node • Run mqsiextractcomponents and target a new integration node • This could be on the same machine but have a different name • Or be on a different machine, and maybe even different architecture • For example, a Windows to zLinux move • The new integration node should be configured identically to the original • If cloning to the same machine you will need to take note of the flows deployed and any requirements for things to be isolated or configuration which may need to be updated to avoid clashes • For example, admin or http ports, default queue manager names, aggregation node queue manager settings
  • 36. TechCon 2021 36 Virtual Experience • You could also use mqsiextractcomponents to help you recover/extend/duplicate your integration nodes at previous versions • The sparse .conf.yaml files show you what properties you need to configure to replicate most of the old configuration
  • 38. TechCon 2021 38 Virtual Experience People & Process Architecture Technology How can we improve development agility in order to accelerate innovation? Development Agility How can we improve build independence and increase production velocity? Deployment Agility How can we improve our ability to deliver dynamic scalability and inherent resilience? Operational Agility
  • 39. TechCon 2021 39 Virtual Experience APIM APIM API Management APIM API Management APIM Gateway Integration Integration Int. Engagement applications Systems of record API Management Socialization/monetization Re-platforming Application autonomy API Management Centralized ESB Fine-grained integration deployment Decentralized integration ownership Socialized APIs Webinars http://ibm.biz/agile-integration-webcasts eBooklet http://ibm.biz/agile-integration-ebook IBM Redbook http://ibm.biz/agile-integration-redbook APIM APIM
  • 40. TechCon 2021 40 Virtual Experience First split by business domains and functional areas to ensure high-level autonomy integration artefact integration runtime Domain 1 Domain 2 Domain 3 before after All Domains
  • 41. TechCon 2021 41 Virtual Experience Next, consider non-functionals such as a) which need a separate pipeline (for agility), b) which need independent scalability, c) which have unique resilience requirements 41 before after Domain 1 Domain 1 Shared lifecycle (e.g. shared data model) Stabile requirements (e.g. integrations that haven’t needed changes in years) Specific resilience (e.g. require high replication factor to achieve availability) Inter-related availability (e.g. reliant on the availability of one another) Technical dependencies (e.g. all require local MQ server due to 2PC) Elastic scalability (e.g. occasional high spikes in usage) http://ibm.biz/aia-granularity integration artefact integration runtime
  • 43. TechCon 2021 43 Virtual Experience • ACE does not store persistent data, and is best seen as application layer • MQSI (long ago) would have fit on the other side • Code as well as configuration • BAR files are not the same as MQSC files Application layer (transient) Java Node.js ACE Python Persistence layer Database MQ State storage and messaging
  • 44. TechCon 2021 44 Virtual Experience Asset repo BAR1 BAR2 BAR3 ACE DFE 2 ACE DFE 1 ACE Integration Node App2 App3 App1 Source repo App1 App2 App3 BAR build Deploy BARs and manage integration servers Manage integration infrastructure • Many existing installations use integration nodes managed by dedicated administrators • Until ACE v11 this was the only option, and might still be appropriate • For example, if an organisational strategy for containers is still in progress
  • 45. TechCon 2021 45 Virtual Experience Container infrastructure • Drivers ＋ Use containers as lightweight VMs to eliminate on-prem hardware ＋ Keeps admin model the same for ACE ＋ Might save on hardware ACE DFE 2 ACE DFE 1 ACE Integration Node App2 App3 App1 Container Manage container infrastructure Manage integration infrastructure • Challenges － Not what is usually meant by “containerization” － Does not provide any modernization benefits － Not recommended
  • 46. TechCon 2021 46 Virtual Experience • Drivers ＋ Preserving admin model reduces cost of transition in the short term ＋ Lighter containers than option 1 • Challenges － Still managed as mini-nodes, limits modernization gains － Still need patching and potential maintenance window － Containers may need persistent storage, complicating container management Container infrastructure ACE server1 ACE server2 App2 App3 App1 Container Container Manage container infrastructure Manage integration infrastructure
  • 47. TechCon 2021 47 Virtual Experience • Drivers ＋ Industry-standard tools in the pipeline and image build, simplifying management and operations ＋ + Container-based operation allows for rolling updates and zero downtime • Challenges － Significant change in admin model, requiring greater container literacy on the part of integration staff. － Often depends on company-wide cloud and container strategy (if one exists) － Requires placement of correct combination of apps in servers Source repo App1 App2 App3 Image build and deploy Manage container infrastructure ACE server1 ACE server2 App2 App3 App1 Container Container Container infrastructure
  • 48. TechCon 2021 48 Virtual Experience • Drivers ＋ Industry-standard tools in the pipeline and image build, simplifying management and operations ＋ Lighter containers ＋ Each application in a separate container to scale better and allow independent management ＋ Often regarded as the “microservice” approach; might go as far as to put each flow in a container • Challenges － Significant change in admin model, requiring greater container literacy on the part of integration staff － Often depends on company-wide cloud and container strategy (if one exists) － Overhead in memory and management/licensing/cost － Not always needed; only in very-high-traffic scenarios would this be essential Source repo App1 App2 App3 Image build and deploy ACE server1 App1 Container ACE server2 App2 Container ACE server3 App3 Container Manage container infrastructure Container infrastructure
  • 50. TechCon 2021 50 Virtual Experience Server Test Server Test Server Test • Components built in containers or VMs (platform- and technology-dependent) • Main server build integrates the runtime components; uses large-scale container testing for validation • Combined with the toolkit for more container testing, and then feeds into ACEcc and ACEoC as well as software and fixpack builds • Enables agility in implementing features, fixing defects, adding platform variants, etc Message catalogs Container infrastructure Common Java Web UI . . . Toolkit build and test Server Build Server Test Combined Build Server Test Server Test Server Test Container infrastructure Combined Test ACEcc Image ACE on Cloud ACE software Windows as well as Linux
  • 52. 52 Assess and analyze applications, integrations and supporting infrastructure and services Discovery Inventory and dependencies Assessment Classify simple / moderate / complex based on target compatibility Code Assistance Potential issues, severity details, possible solutions and estimated resolution effort Included and deployed on IBM Cloud Paks TA modernizes WebSphere, WebLogic, MQ and ACE/IIB deployments Provides recommendations and automation for App Modernization Migration Automation Generate build and deploy artifacts to help on-ramp to selected target platform and runtime Holistic Selectively modernize all the components of your Business Application as a unit http://ibm.biz/cloudta
  • 53. TechCon 2021 53 Virtual Experience • Text goes here lorem ipsum dolor sit amet, consectetur adipiscing elit. Complexity Java EE IBM MQ IIB No code changes needed DNS reconfiguration required Admin change is required Code changes required Cluster reconfiguration, changing custom logic (e.g. Exits) Development change is required Incompatible technologies; external dependencies Client Authentication reconfiguration Difficult development task or alternate technology is required Simple Moderate Complex Severity IIB No immediate action is required, but you may wish to be aware. Immediate action is probably required or advised before you proceed. You cannot proceed without taking remedial action. Error Warning Info
  • 56. TechCon 2021 56 Virtual Experience https://www.ibm.com/support/knowledgecenter/en/SSTTDS_11.0.0/com.ibm.etools.mft.doc/bh23394_.html
  • 58. TechCon 2021 58 Virtual Experience • “Serverless” has been used to describe quite a lot of different approaches and technologies • Absence of an application server rather than a physical machine • ACE CI/CD pipeline can deploy as a serverless application as well as deploying to containers and integration nodes • Goal: avoid running an integration server all the time • Licensing cost is lower • Resource consumption lower overall • Two forms: container-based scaling and Function-as-a-Service • Knative, KEDA, etc in first category • Amazon Lambda and IBM Cloud Functions in the second Cloud Function ACE integration node running on-prem
  • 59. TechCon 2021 59 Virtual Experience • Knative acts as a front-end for ACE • Receives HTTP traffic and starts/stops containers as needed • Will scale to zero after inactivity timeout • ace-demo-pipeline has knative option • KEDA handles scaling only • ACE message flows get messages directly from the MQ queue manager (or other messaging source such as Kafka) • Polling-based approach to scaling, where the MQ queue depth determines how many containers will be started • Will scale to zero after inactivity timeout • Both require a complete ACE container with applications and credentials already set up Containers ACE app Containers ACE app MQ Queue Manager Application traffic (MQ) Application traffic (HTTP) Knative infra- structure KEDA infrastructure MQ messages picked up by ACE message flow Queue depth polling Container scaling Container scaling HTTP data
  • 60. TechCon 2021 60 Virtual Experience • Running ACE flows as one-shot containers • Zero overhead when not actively working • Container infrastructure abstracted: could be using any technology that runs containers • Cloud Foundry, Kubernets, Docker Swarm, etc • More setup/teardown, and connections would have to be made every time • Startup latency a concern in many cases • Require more reengineering of applications and deployment pipelines in many cases • FaaS best suited to short-lived applications that do not block on network interactions • Matches traditional out-and-back ACE message flow architecture in use since MQSI v2 • Much higher latency than the traditional server • Works best with occasionally-used flows that can accept high latency, or in situations where container management needs to be abstracted out One-Shot Container ACE app Application traffic (MQ) Application traffic (HTTP) Cloud Function infra- structure Application can connect to database One-Shot Container ACE Request Cloud Function infra- structure Kafka Service One-Shot Container ACE Response Cloud Function infra- structure DB
  • 61. TechCon 2021 61 Virtual Experience • Function-as-a-Service has a higher latency but no idle time • Knative and KEDA use the containers for longer after they have started • This example shows a short-lived flow, which might not be ideal for FaaS; longer-running flows would amortize the startup cost over a greater period and be more efficient Startup Flow run Time 30 seconds 60 seconds 90 seconds 120 seconds Startup Flow run1 Flow run2 Flow run3 Idle FaaS Knative or KEDA Container scaling Container start One request sent to the server Container shutdown Multiple requests sent to the server
  • 62. TechCon 2021 62 Virtual Experience Questions?

Editor's Notes

  • #10: 9
  • #44: This slide exists due to some ongoing confusion around where ACE fits in: for historical reasons, apparently, there is still the idea that ACE is similar to MQ!
  • #49: People challange
  • #51: People challange
  • #53: IBM Cloud Transformation Advisor helps you access, analyze and modernize middleware based apps into IBM Cloud(s). It categorizes Java EE apps and MQ queue managers as simple, medium and complex based on migration complexity and provides guidance for modernization. The ACE implementation of the Transformation Advisor framework Available through the ACE console since 11.0.0.7 Built on https://github.com/IBM/transformation-advisor-sdk since 11.0.0.8 Detects 35 different potential issues and concerns (Rules) when migrating to a docker based cloud architecture https://www.ibm.com/support/knowledgecenter/SSTTDS_11.0.0/com.ibm.etools.mft.doc/bh23394_.html Works on backups and bar files Formally supported on V9+, may also work on backups from V7 & V8.