Session 3 _exposing_mainframe_applications_services_v4

Editor's Notes

• #3: Exposing mainframe applications and services to mobile Exposing mainframe applications and data to mobile devices makes business sense, but how can you ensure that the critical connection of z Systems to mobile is as straightforward as possible? This session focuses on technologies that simplify the integration of z/OS applications such as CICS, IMS and WAS on z/OS with mobile applications. We examine the WebSphere Liberty z/OS Connect feature, which provides a simple, secure and standards-based interface to applications that must access z/OS assets. Additional technologies such as MQ, API Management and DataPower, which can further enhance integration, management and security also are covered.
• #6: The "API economy” has changed how developers think about building apps, and how organizations deploy software in the cloud. Everywhere you look on the Web these days, it seems as if somebody’s launching new APIs. There are over 10,000 API documented on programmable web today… and is significantly increasing as we speak. Several years ago Jeff Bezos declared that all Amazon IT assets were to be exposed as APIs. Out of that “simple” declaration came an IT (and cultural) architecture that led to the stunning growth of Amazon Web Services, which is thought to be a billion-dollar business unit after only a few short years of growth. Perhaps even more impressive is the fact that, today, Salesforce.com generates more than half of its revenue through its APIs, not its user interfaces. Twitter is said to process 13 billion transactions a day through its APIs. Google is around 5 billion transactions a day. (Hotel map) New industries defined – Airbnb, Uber, etc
• #7: Let’s look at the Cars.com mobile app as an example. Cars.com is a consumer of APIs, and has assembled several 3rd party apis together to form the base function of their cars app. For example, they use a Map Provider API to provide a “store locator function” APIs from a car dealer data aggregator for availability of certain makes and models. They use APIs from a Bank to offer Loan calculators and origination Auto insurance from an Insurance companies APIs And can imagine them using the new Xtify API from IBM to provide notifications, that a car they were looking for was now found.
• #8: As the definition explains, the Web Economy is where companies [providers], expose their internal digital assets or service in the form of Web APIs to third party [consumers] with the goal of unlocking additional business value through the creation of new assets. <MAXX> Emphasize definition… Also… Re-sub-title – The API Economy lifecycle
• #9: Businesses in various industries have developed “core competencies” over time, with supporting software functions and matching “brick and mortar” capabilities, such as banking, auto-dealership, healthcare providers, etc. These capabilities can be leveraged by exposing these functions as APIs. Many of these software functions reside in z Systems There are three key drivers for exposing these assets as APIs Reaching new customers and markets through partner applications and solutions that leverage these business capabilities Improving end-user experience with Mobile apps and/or with new system of engagement applications, possibly running in a cloud environment that access systems of record, and Improving consumability of existing services by making it easy to look up via catalog, and easy to set up through self-service registration, and improving governance of access to existing services by managing entitlement and gaining insight from access history
• #10: Publishing an API for external consumption is not the same as Listing in the internal asset repository for development and maintenance, or Listing in runtime SOA Service Registry for use in Enterprise Service Bus (ESB) for routing, mediation and policy enforcement A business web site (as in this example, by Sabre) lists all the APIs offered by this business to its partners, external (and also internal) application developers. An API description includes not just the description of the business functions, interface for invoking the API, and detailed description of various information field, example on its use. The APIs are categorized and listed using associated categories for ease of look up. Additional details include various entitlement policies and offering levels.
• #11: SOA efforts have been driven by developers’ productivity gain and enabling reuse of functions, i.e., focused on development of services. API Management, on the other hand, is driven by consumption of these services, i.e., improving consumability of these services for both external and internal developers of applications accessing these APIs, while also retaining control by the providers of these APIs. Improving consumability includes listing APIs in a browsable/searchable catalog and making it easy to register applications with the right entitlement level Retaining control includes not only enforcing entitlements and managing workloads, but also providing insight based on access history and accountability for chargeback. CAMS – Cloud, Analytics, Mobile and Social – are the enablers for new applications and services. This includes: Cloud – new applications, including by partners, accessing existing assets can be cloud based Analytics – can be used both to improve system of interaction addressing customer experience as well as to gain business insight Mobile – new forms of applications can be used in improving customer experience, and/or to deliver new services to end-consumers directly Social – helps to understand better consumers and deliver services in new ways
• #12: Introducing IBM API Management: capabilities include Exposing business services securely as APIs to developer communities, and analyzing API usage Providing self-service API portals to external/internal app developers Managing & monitoring the entire API platform
• #14: Key steps in Managing APIs Create, assemble and define an API Secure & scale the API Share with developers Manage growth & analyze results
• #15: End-to-end Architecture and IT roles in exposing, and consuming z assets as APIs Three-tier Architecture: Development of enterprise services from existing z assets, making it easy to invoke these applications z application environments (CICS, IMS, WAS) provides tools and runtimes to develop and invoke Web and REST based services from applications based on COBOL and PL/I IT role (Bob) is knowledgeable about this SW stack Development and management of APIs exposing existing enterprise services Addressing consumption and governance of APIs Create an API , discover a suitable existing service, and assemble this API from this service Development of Mobile, Cloud or Third-party applications consuming APIs Browse catalog and identify APIs to consume Register an application that will consume a specific API at a specified entitlement level Use of developer portal to test APIs
• #16: APIM capabilities for discovering z assets: Get a list of metadata maintained for various types of z based services (e.g., CICS or IMS, Web or REST services) Discover a service based on keywords and attributes from the deployed services in z System Import definition and configurations details of a service for creation and assembly of an API Matching z capabilities: Support for Web Services in CICS and IMS z/OS Connect for supporting REST services for all z Applications Discovery protocol support for both Web and REST services
• #17: IBM API Mgmt provides the management platform, while IBM DataPower provides the API Gateway to enforce API security and control . IBM APIM sits on a server as a virtual appliance, while DataPower can be a virtual appliance or a physical appliance
• #19: We start with a discussion that sets the context for the details that follow. At the very heart of this discussion is the idea of devices and systems using RESTful calls and JSON data to access the mainframe and the applications and data that resides there. REST and JSON are becoming more and more common in the world of mobile and cloud system access to systems of record. REST and JSON are briefly described here. A more detailed description is offered a bit later in this presentation. For REST and JSON to be effective, something has to be in place to recognize and handle the REST call and the JSON data. Key Point: z/OS Connect is designed to be a REST/JSON interface to a z/OS LPAR. z/OS Connect is software function written by IBM that runs inside an instance of WebSphere Liberty Profile z/OS, and uses existing connector technology to get to the backend systems. There is no charge for z/OS Connect; it is supplied as part of license entitlement to WAS z/OS, CICS or IMS.
• #20: This chart is a follow-on to the point made at the end of the previous chart. This chart explains the business value of z/OS Connect. The bullet points on the right side of the chart summarize the value proposition of z/OS Connect. At the end of the day it’s about a manageable and simplified interface to the z/OS LPAR for mobile and cloud devices that use REST and JSON.
• #21: This chart is intended to show the various solutions that IBM currently has to access System z Systems of Record. Note that several of these solutions will be moving towards have affinity with z/OS Connect and are moving towards integration. The intent if for z/OS Connect to become established as the reference architecture for exposing z/OS through RESTful services.
• #22: As mentioned earlier, z/OS Connect is a no-charge software function that is part of your license entitlement for WAS z/OS, CICS or IMS. Each provides a delivery mechanism – Liberty and z/OS Connect will be supplied by WAS z/OS, CICS and IMS. Note: the Liberty and z/OS Connect for CICS will run inside a CICS region. It is intended to be used only inside CICS regions and not outside. The Liberty and z/OS Connect supplied by WAS z/OS and IMS look very similar on the surface, but for IMS the access mechanism is a JCA resource adapter and not the local adapters. To use the JCA resource adapter with IMS, a running copy of IMS Connect is required. For the purposes of this presentation the focus will be on WAS z/OS. That implies a Liberty Profile server instance with z/OS Connect enabled and configured. The implementation supplied with IMS is very similar, with the difference being the connectivity to IMS supplied as a JCA resource adapter that will talk to a running instance of IMS Connect. The CICS implementation is somewhat different in that the Liberty instance will run inside the CICS region, and the backend connectivity is provided through the JCICS interface of the CICS region. We mention these delivery mechanisms because we have seen some confusion created by this. The different delivery approaches tends to obscure the story of z/OS Connect and the value it brings. We thought it best to bring this point out early to clear any misunderstandings that may be present. This will allow us to focus back on z/OS Connect and what it provides. IMS specifics The IMS mobile solution is a no-charge offering that provides a RESTful endpoint to access IMS transactions. The solution comprises the service provider for z/OS Connect and a Windows-driven, Eclipse-based tooling platform for service creation and administration. Obtaining the service provider: For customers who already have WAS for z/OS Liberty Profile, the IMS service provider is downloadable as 'IMS Mobile Feature Pack' from within the Liberty repository as described here: https://developer.ibm.com/wasdev/downloads/#asset/features-com.ibm.ims.imsmobile-1.0 For customers who do not have WAS Liberty Profile, the IMS Mobile Feature Pack service provider is available as a component of IMS Enterprise Suite for z/OS Version 3.1.1, PID 5655-TDA, orderable from Shop zSeries. WAS LP is bundled with IMS ES for z/OS 3.1.1. Obtaining the IMS mobile tooling platform: Service creation and administration is handled through IMS Explorer for Development, downloadable from http://www.ibm.com/software/data/ims/downloads.html.
• #23: This chart provides a little background on what “REST” is. As the chart shows, REST is built on the notion of using HTTP verbs (GET, PUT, POST, DELETE) along with the URI (the portion of the URL that comes after the host and port) to indicate the function to perform. The example above is suggesting a function that gets some information on a customer number. For this to work, both the client and the server much understand what functions are available and what URI formats are accepted. To assist in this, z/OS Connect has a discovery function that allows developers to understand what services are available through a z/OS Connect instance, and what the details of those services are. RESTful services are growing in popularity because it’s relatively simple to implement.
• #24: JSON is a way of representing data in a name/value pair manner. The chart above shows a simple example of JSON for a customer data record – name, age, and address. When a client (smart phone, tablet, browser, etc.) sends in a REST call with data, the JSON associated with the request (or response) is serialized and appended to the end of the URI. The data flows back and forth between client and server in this fashion. That means the “state” of the interaction is maintained with the flows going back and forth. That means there’s no affinity requirements – that is, there’s no requirement that a client talk to the same server instance each time. A cluster of server instances can exist and, provided they can access the same data and provide the same functions, it doesn’t matter if the client goes to server A, B or C. The lack of affinity routing requirements means the requests are stateless (state is carried with the request; not maintained outside the request). Stateless requests are very easy to manage in a network topology; stateful requests require stateful routing or state replication. REST and JSON avoids the need to worry about stateful routing. Anticipated Question Answered: yes, z/OS Connect instances can be clustered, either on the same LPAR or across LPARs. The important thing is that all the z/OS Connect instances in a cluster operate identically, and have access to the same backend systems. The way to make multiple instances of z/OS Connect operate identically is to make sure they run with the same configuration information. On z/OS with Parallel Sysplex the ability to insure shared data in the background is quite achievable. That talk about state and affinity routing was a bit of a detour from the main point about JSON as a means of representing data. JSON tends to be favored over XML because it is a bit lighter weight. Both XML and JSON need to be parsed (that is, the data extracted so it can be understood and worked on). Parsing JSON is a bit easier than parsing XML.
• #25: One of the concerns we’ve heard expressed is this – “Does z/OS Connect expose all my backend CICS programs to Internet?” And the answer to that is an emphatic “No!” Key Point: z/OS Connect will only expose those services you code in the Liberty Profile configuration file. z/OS Connect does not scan your z/OS LPAR looking for programs to expose. z/OS Connect is only as smart as the configuration file information it operates from. You control what programs and applications are exposed as services through z/OS Connect. That’s what this chart is saying … z/OS Connect operates from the information provided in the Liberty Profile configuration file, which is the server.xml file. You control what goes into that file. Only those backend programs you wish to expose as services will be discoverable and usable through z/OS Connect.
• #26: We mentioned earlier how there is a “discovery function” provided with z/OS Connect. This provides developers a means of determining what services an instance of z/OS Connect exposes, as well as details about a given service. Sending in a URI with just /services will return a JSON list of services and their service URLs. A developer would use this to get an idea what services are configured in the server.xml for that instance of z/OS Connect. If a service is of interest, they can send in a second URL, this one asking for details on the specific service. That will return a JSON object with details about the service. It is also possible to query for and receive the JSON schema for the service … both the request JSON schema as well as the response JSON schema. With that, a developer can develop the client-side code to invoke and make use of a service exposed through z/OS Connect.
• #27: We mentioned earlier how there is a “discovery function” provided with z/OS Connect. This provides developers a means of determining what services an instance of z/OS Connect exposes, as well as details about a given service. Sending in a URI with just /services will return a JSON list of services and their service URLs. A developer would use this to get an idea what services are configured in the server.xml for that instance of z/OS Connect. If a service is of interest, they can send in a second URL, this one asking for details on the specific service. That will return a JSON object with details about the service. It is also possible to query for and receive the JSON schema for the service … both the request JSON schema as well as the response JSON schema. With that, a developer can develop the client-side code to invoke and make use of a service exposed through z/OS Connect.
• #28: Here we’ll introduce some terminology that’s used with z/OS connect – a “Service Provider.” A service provider is simply the functional implementation that gets the request to a backend system like CICS or IMS. In the previous charts we showed the local adapters used to access a single CICS region. That was an example of a single service provider. But you are not limited to a single service provider; you may configure multiple service providers if you wish. For example, if you wished to expose programs in CICS and IMS, you would configure two service providers – one for accessing CICS (using the local adapters), and one for accessing IMS (which is implemented with a JCA resource adapter that communicates with a running instance of IMS Connect to access IMS). If you had two CICS regions you wished to communicate with, you’d configure two service providers in the instance of z/OS Connect – one to access CICS region A and another to access CICS region B. Note: the service provider interface is documented so that you may write your own service provider if you wish. You are not limited to just those supplied with z/OS Connect.
• #29: The earlier chart showed a bit of XML that pointed to a data transform definition. Here we’re showing you what that data transformation XML might look like. On the surface this looks complicated but in reality it’s not – it’s really just telling z/OS Connect where to find the “bind” files and the JSON schema. Bind files provide the data field conversion information. They are produced by running a supplied utility (based on the WSBIND utility supplied by CICS) to transform program data definitions structures (for example, COPYBOOK for COBOL) into a file that allows z/OS Connect to understand the JSON-to-COPYBOOK conversion pattern. The XML above is illustrating how you tell z/OS Connect where the bind files are located, the suffix used for those bind files, as well as the location and suffix used for the JSON schema files. With that information available, z/OS Connect can read in the bind files and perform JSON conversion. The converted data – now in something other than JSON – can be passed back to the backend program for processing. The backend program doesn’t need to know anything about REST or JSON … z/OS Connect is doing all that work in its role as a service interface to backend systems.
• #30: This is emphasizing the point made on the earlier chart – the bind files are created by using a program originally developed for the CICS web services function. This utility reads in COPYBOOK (or other data structures) and produces the bind file and JSON schema. The gray box on the chart is simply showing the JCL that might be used to run this utility program. Note: you may either use the DFHLS2JS program supplied with the CICS Feature Pack, or use the utility supplied with z/OS Connect.
• #31: Another functional element of z/OS Connect is something called an “Interceptor,” which is a callout point that may be invoked for each request and invoked again for each response. It’s optional – you aren’t required to use the interceptor function – but if you wish to have callout processing performed for requests and responses you have this capability present. Two provided interceptors are available – one is for security, which will work with SAF to perform authorization (EJBROLE) checks; and another for auditing, which will work with SMF to cut activity records. You can configure global interceptors that apply to all defined services, or you may define interceptors that apply to just those you wish them to apply to. For global interceptors you can ‘opt out’ of those global interceptors for specific services if you wish. Note: like Service Providers, this Interceptor interface is documented and you may write your own interceptor code if you wish.
• #32: When the backend system is CICS and the local adapter support is used, it implies installing into the CICS region the Optimized Adapter support. This is not difficult to do; but it is necessary so the CICS region has the ability to communicate using the optimized local adapters. The picture is showing the basics of the optimized local adapter function in the CICS region: The Task Related User Exit (TRUE) provides the low-level function support for the optimized adapters. This TRUE must be active in the CICS region to communicate across the optimized local adapters. The BBO$ link server is a long running task that handles the calls from z/OS Connect prior to invoking the target program. This is what makes the optimized local adapter calls transparent to the target program – BBO$ shields the target from the local adapter specifics. BBO$ starts an instance of BBO# for each call. The reason this is separate from BBO$ is because the optimized local adapters supports the assertion of identity. When the thread ID from Liberty is asserted over, then each instance of BBO# is run under the identity of the asserted ID. When thread ID from Liberty is not asserted, then BBO# is run under the same ID as BBO$ and the instance is re-used. The final link in this chain is an EXEC CICS LINK call to the target program from the BBO# invocation task. To the target program this looks like any other CICS program calling it. It has no knowledge of the optimized local adapters being part of the picture. The WP101490 Techdoc at ibm.com/support/techdocs provides more information on WebSphere Optimized Local Adapters.
• #33: Another use-case for z/OS Connect and the optimized local adapters is for exposing “batch as a service.” This is possible because the local adapter APIs support a “listen mode” API that will put a batch program into a wait state until someone calls. That “someone” can be z/OS Connect calling to the batch. To avoid updating all your batch programs, you can write a batch program that uses the local adapter APIs to “host a service” (go into a wait state), then have that program do a dynamic or static link to another program. That way the linked program doesn’t have to have any knowledge of the local adapter APIs. Upon return the first batch program would loop back and call the “host a service” API again. It would keep doing that for as long as you wished z/OS Connect to have a backend batch program exposing batch as a service.
• #34: z/OS Connect may work with IMS as well, though in the case of IMS it will use a JCA resource adapter to connect to IMS, not the local adapters. The JCA resource adapter will make a network call to a running copy of IMS Connect, which will then turn and use OTMA to get into the IMS region. Everything else about z/OS Connect is the same as we discussed earlier. The only difference is the “service provider” being a JCA resource adapter. (And in truth the optimized local adapters also use a JCA resource adapter, but a different one. The difference is really the implementation below the JCA API layer – for the optimized local adapters code is invoked that does a cross-memory call; for IMS code is called that makes a network call to IMS Connect.)
• #36: Now let’s take a peek inside z/OS Connect to get a feel for how it works.
• #37: If we wanted to quantify those benefits we do have a view….this is from an external analyst – studying… And the more use messaging is put to…the greater the benefits
• #38: So what does MQ offer? – messaging middleware – an interface layer helping to isolate applications connecting across different platforms from the mainframe to the mobile…
• #39: As we just covered MQ provides guaranteed, assured messaging that provides transactional integrity with the speed and security that any enterprise will need for mission critical applications. E.g., transactions at a bank, money movement in finance sector, retailer processing of payment card or purchase details, border security and immigration processing, moving data through factories and in and out of ERP systems MQ enables you to connect applications and services together with valuable qualities-of-service. Applications can exchange information without tying themselves up – just like email where people communication a-synchronously. MQ has been doing all this for years – we have 10000+ customers who have built their businesses around it – and rely on it not just every day – but every second. It helps their businesses be more flexible, more reliable and more secure. Deployments on platforms like System z with its coupling facility helps customers to be robust and recover from failures without interrupting their processing – and we have hundreds if not thousands of partners and skilled practioners to support these deployments – as well as big teams in IBM enhancing the product and delivering major new updates – such as the new IBM MQ V8.
• #40: In april 2014 IBM announced our latest major new version of MQ – V8 to be available on all major platforms this quarter. As we have covered – by using messaging solutions like MQ, businesses can keep their applications simple, while delivering data from pretty much any source to any other endpoint, helping the business to streamline, cutting costs and time, and meeting the changing needs of the business. So although thousands of customers are already current with MQ – what can they expect in the new V8 release? Well MQ is broadly common across bother distributed and z/OS platforms but there are some distinctions – on distributed there are updates focused on helping customers to scale their workload deployments – supporting larger workloads, more messages, more queues, and doing that without increasing the tasks on the administration and operation team. Security is of course high on everyone’s agenda and there have been big changes there too – both to enhance security and to ensure it is easier to use and manage. And there are some new functions as well – in support of new standards like JMS2.0 – plus exciting news for customers on IBM I – with MQ AMS available on that platform for the first time. For z/OS it gets the benefit of a lot of the same updates but also some specific new features to really exploit the platform capabilities – helping deployments to scale better, perform better, and with easier administration, boosting reliability and robustness. And of course with z being critical to so many businesses, security again is featured with stronger SHA-2 encryption. Plus as a bonus customers on V8, as well as V7 can now connect MQ clients on other platforms directly to the z/OS MQ without the additional CAF charge – this will help customers choose how to connect and exploit the data movement and business needs far more flexibly.
• #46: By consuming APIs to assemble their App, Cars.com… Can focus on their core competence and value add, rather then reinventing common capabilties Sees Quicker time to market, decreases cost and speeds up their cars.com app development and delivery Drives innovation, from others, into their app (like, perhaps the notifications from Xtify) Perhaps the insurance company offers value to their affiliates (which cars.com is now part of) by marketing the cars.com app to their insurance customers
• #48: Expands into new customer bases and niches that the bank would normally not reach on their own Expands their brand and brand loyalty – you think about google when you see their maps appear in apps, perhaps you will think of “the bank” when you see their signature mortgage calculator And now you have third parties (not necessarily external) innovating with your content in ways you haven’t imagined and you are now monetizing those sources – across a wide array of app, running on a wide array of platforms
• #49: The primary configuration file for Liberty Profile is server.xml. It is in this configuration file that updates are made to enable the z/OS Connect function and to configure the behavior of z/OS Connect. The chart above is showing an example of a “HelloWorld” URL coming into z/OS Connect and the configuration that would be in place to support responding to the request: The <featureManager> section of server.xml is used to tell the instance of Liberty what functional features you wish to be loaded. z/OS Connect is loaded with the zosConnect-1.0 feature. z/OS Connect uses the cross-memory local adapters function to access backend services like CICS and Batch, and to enable that function requires the zosLocalAdapters-1.0 feature. The next section is what tells z/OS Connect about the HelloWorld service. The serviceName= value is what is matched to the URI. The serviceRef= value is what points to the section of XML that defines how the backend system will be accessed to deliver the service. That section defines the local adapter “register name” and “service name” to be used when the HelloWorld service is requested by a client. The register name is the cross-memory connection to, in this example, a CICS region. The service name is the program back in CICS that will be called. Without getting too deep into configuration XML, the main point here is that the configuration spells out what services are exposed, and how the backend system is accessed. In this “HelloWorld” example we illustrated the URL, the service definition and the backend local adapter definitions. Those aren’t the only configuration updates needed, but that’s a good start to show the essential elements.
• #50: The previous chart showed one configured service. This chart is illustrating the point that you may have multiple configured services. Here we’re showing three configured services – Create, Update and Delete contact information. Three service definitions are configured in the server.xml … one for each service to be exposed. In addition, the example above is showing that all three are going to make use of the same serviceRef=, which is the local adapter connector to the backend CICS. That means Create, Update and Delete are all hosted in the same CICS region. (You are not limited to a single CICS region; you can configure services in multiple CICS regions if you wish.) Further, this example is showing dataxform=, which is a pointer to XML (not shown on this chart, but we’ll show it on an upcoming chart) that defines how the JSON data will be converted to the format needed by CICS and the backend program. This illustrates another important function of z/OS Connect – the ability to transform data back and forth.
• #51: This chart illustrates the SMF record information to be written for each request. This will produce an SMF 120.11 record, which is different from the traditional WebSphere Application Server z/OS 120.9 record.