IBM Connectivity and Integration

IBM Connectivity and Integration
Lee Gavin
European WebSphere Technical Sales
gavinlee@uk.ibm.com

© 2013 IBM Corporation

The World As We Knew It
WAS, CICS, IMS
Application platforms
provide containers for
application and
buisness logics

Messaging backbone in
the data center

MQ

2


The Changing World: Towards SOA
WAS, CICS, IMS

Integration Bus

application and
buisness logics

Enterprise Service Bus integrates
apps/data/services and partners,
appliance provides integration
gateway to heritage services

the data center

MQ

3


The Changing World: Emergence of Appliances
WAS, CICS, IMS

Integration Bus

application and
buisness logics

WXS, DataPower XC10
Cache grids improve
scale and performance
of applications and
services

Secure
appliances
enable
controlled
access to
Enterprise
resources

the data center

MQ

DataPower XG45
4


The Changing World: Budget Versus Architecture
WAS, CICS, IMS
application and
buisness logics

Cast Iron, DataPower XH40
Connectivity to
applications in the
public cloud enables
Enterprises to leverage
a new cloud economy

Integration Bus
WXS, DataPower XC10
Cache grids improve
of applications and
services

Public
Cloud

SaaS

Secure
appliances
enable
controlled
access to
Enterprise
resources

the data center

MQ

DataPower XG45
5


The Changing World: Budget Versus Architecture
Enterprises looking to
achieve “more with less”
by better managing IT
resources as collectives

IWD,
PureApp

Cast Iron, DP XH40
Connectivity to
applications in the
a new cloud economy

Public
Cloud

WAS, CICS, IMS

Integration Bus

application and
buisness logics

WXS, DataPower XC10
Cache grids improve
of applications and
services

Private
Cloud

Secure
appliances
enable
controlled
access to
Enterprise
resources

the data center

MQ

DataPower XG45
6


The Changing World: And Then There Was Mobile

IWD,
PureApp

Cast Iron, DP XH40
Connectivity to
applications in the
a new cloud economy

Public
Cloud

WAS, CICS, IMS

Integration Bus

application and
buisness logics

WXS, DataPower XC10
Cache grids improve
of applications and
services

Private
Cloud

Internet

Mobile
Sensors

Scale and ubiquity of
mobile and sensor-rich
environments has
changed requirements of
the enterprise
IBM Mobile Foundation
(Worklight)
7

Secure
appliances
enable
controlled
access to
Enterprise
resources

the data center has
extended to external
clients connected via
the Internet
Internet
MQ, MessageSight

DataPower XG45

The Changing World: (Social) Evolution of “Partner”

IWD,
PureApp

Cast Iron, DP XH40
Connectivity to
applications in the
a new cloud economy

Public
Cloud

WAS, CICS, IMS

Integration Bus, DataPower XI52

application and
buisness logics

WXS, DataPower XC10
Cache grids improve
of applications and
services

Private
Cloud

Internet

Mobile

Partner community
Sensors

environments has
the enterprise
(Worklight)
8

Secure
appliances
enable
controlled
access to
Enterprise
resources
DataPower XG45

the data center has
the Internet
Internet
MQ

B2B partners opened
channels and
collaboration while a
new genre “App
Developer partner” is
emerging
Sterling Commerce
DataPower XB62
API Management

Great…but what about SOA?
A Service

Service Orientation

A repeatable
business task –
e.g., check customer
credit; open new
account

A way of thinking about
your business through
linked services and the
outcomes that they
bring

Service Oriented
Architecture (SOA)
An business-centric architectural
approach based on service
oriented principles
9

SOA mediates between consumers and providers (ESB pattern)

APIs

Cloud
Services

Partners

Suppliers

Apps

Patterns

2005: Connecting and mediating in an IT
transactional context
2010: Connecting and mediating e2e processes
10

Customers

2015: Connecting and mediating people,
devices, Cloud, ….

Developers

The World As We Knew It
WAS, CICS, IMS
application and
buisness logics

the data center

MQ

Why is Messaging Important?
Synchronous

Asynchronous

Improvements in network reliability haven’t eliminated failures
Failure handling still more effective and simpler to implement by separating it out from
business and application logic
–

Connectivity interruptions handled seamlessly without needing expert application integration skills

Consistent interfaces, no matter the system, or the application programming language
– Removes complexity from application, reducing time and cost for development and
maintenance

Moves any type of data and any type of system, device or environment
Can provide variety of qualities of service
–
–
12

Persistence, non-persistence
Point-to-point, publish/subscribe, multicast

–

Move from batch and offline processing, to real-time event driven architectures

Build a universal messaging backbone

Managed File Transfers
Improve reliability and security
leveraging the internal
messaging backbone

Enhanced end-to-end
security
Securing your business data
and helping meet industry
regulations

Application Connectivity
Moving data between applications,
systems and services using MQI, JMS,
REST, HTTP, Web Services, etc.
Connect remote devices,
mobile enterprise and
sensors
Harness the power of data
outside your internal
network

Ultra low latency messaging
Highest message throughput and
lowest message latency over any
transport

Use WebSphere MQ to flexibly integrate the pieces of your business
13

1
4

IBM WebSphere MQ 7.5 highlights

– Enhanced utilization of messaging through
use of Managed File Transfer solutions
over the messaging backbone

WMQ
V7.5
WMQ
Telemetry

– Access to end-to-end message encryption
capability with a single install

WMQ AMS

– Increased entitlement to integrated
Telemetry capabilities reducing TCO of
Mobile applications and smarter planet
technology

WMQ MFT
Agent + Service

– Client applications gain access to
transactional integrity and updates across
all environments
WebSphere MQ becomes a single offering with a
single install on distributed platforms, with the existing
WMQ FTE and WMQ AMS code integrated as a part of
the WMQ Server
14 14

A fully integrated messaging
solution with secure, reliable and
scalable data movement for today’s
development needs

Message Level Security
“Valuable” messages
– In flight on the network
– At rest, on disk
– Monitoring and control messages

Large networks, difficult to prove security of messages
– Injection
– Modification
– Unauthorized viewing

Data subject to standards compliance (PCI, HIPAA, etc)
– Credit card data protected by PCI
– Confidential & personal data (government / healthcare etc)
– Data at rest, administrative privileges, etc

Message Level Security - Requirements

Assurance that messages have not been altered in transit
– When issuing payment information messages, ensure the payment amount does not change before
reaching the receiver

Assurance that messages originated from the expected source
– When processing control messages, validate the sender

Assurance that messages can only be viewed by intended recipient(s)
– When sending confidential information

WebSphere MQ AMS – Signed Message Format (Integrity
Policy)

Original MQ Message

AMS Signed Message

Message Properties

Message Properties
PDMQ Header
PKCS #7 Envelope

Message Data
Message Data
Signature

WebSphere MQ AMS – Encrypted Message Format (Privacy
Policy)

Original MQ Message

Message Properties

AMS Encrypted Message

Message Properties
PDMQ Header
PKCS #7 Envelope

Message Data

Key encrypted with certificate
Data encrypted with key

Message Data
Signature

How to secure an existing MQ application – No protection

Alice
Sending
App

ORDERS

STOCK

Bob
Receiving
App

How to secure an existing MQ application SPLCAP(ENABLED)

Alice
Sending
App

ORDERS

STOCK

1.Install WebSphere MQ 7.5 AMS Component on server

Bob
Receiving
App

How to secure an existing MQ application – Assign Certificates

Alice
Sending
App

ORDERS

STOCK

Bob
Receiving
App

Keystore

Keystore

Alice Private
Alice Public

Bob Private
Bob Public

2.Create certificates (public / private key pairs)

How to secure an existing MQ application – Exchange Public
Key

Alice
Sending
App

ORDERS

STOCK

Bob
Receiving
App

Keystore

Keystore

Alice Private
Alice Public

Bob Private
Bob Public

Bob Public

Alice Public

3.Exchange public keys

How to secure an existing MQ application – Set security policy
Policy
ORDERS
Privacy
Recipient : Bob
Signer : Alice

Alice
Sending
App

ORDERS

STOCK

Bob
Receiving
App

Keystore

Keystore

Alice Private
Alice Public

Bob Private
Bob Public

Bob Public

Alice Public

4.Define security policy for the queue

How to secure an existing MQ application – Privacy & Integrity
Policy
ORDERS
Privacy
Recipient : Bob
Signer : Alice

Alice
Sending
App

Keystore
Alice Private
Alice Public
Bob Public

ORDERS

STOCK

Bob
Receiving
App

Charlie
Keystore

Send/Rcv
App

Bob Private
Bob Public
Alice Public

4.Define security policy for the queue
5.Messages can only be viewed by Bob, Bob will only accept messages from Alice

WebSphere MQ Advanced Message Security
Provides additional security to that provided by base MQ
Non-invasive
– No code changes or re-linking of applications

End-to-end security, message level protection
– A security policy defines what protection should be applied to messages

Asymmetric cryptography used to protect each message
– Integrity Policies prove message origin, content not changed
– Privacy policies as per integrity plus each message encrypted with unique key

AMS Policies

–Integrity (End-to-end digital signing of messages)
–Privacy (End-to-end message content encryption)

How do most organizations move files today?
Most organizations rely on a mix of home-grown code,
several legacy products and different technologies …
and even people!
FTP
– Typically File Transfer Protocol (FTP) is combined with writing and maintaining homegrown
code to address its limitations

Why is FTP use so widespread?
– FTP is widely available – Lowest common denominator
– Promises a quick fix – repent at leisure
– Simple concepts – low technical skills needed to get started
– FTP products seem “free”, simple, intuitive and ubiquitous

Legacy File Transfer products
– A combination of products often used to provide silo solutions
– Often based on proprietary versions of FTP protocol
– Can’t transport other forms of data besides files
– Usually well integrated with B2B but rarely able to work with the rest of the IT infrastructure –
especially with SOA

People
– From IT Staff to Business staff and even Security Personnel
– Using a combination of email, fax, phone, mail, memory keys…

Shortcomings of Basic FTP
Limited
Reliability
Unreliable delivery – Lacking
checkpoint restart – Files can
be lost
Transfers can terminate
without notification or any
record – corrupt or partial
files can be accidentally used
File data can be unusable
after transfer – lack of
Character Set conversion

Limited
Flexibility
Changes to file transfers often require updates to many
ftp scripts that are typically scattered across machines
and require platform-specific skills to alter
All resources usually have to be available concurrently
Often only one ftp transfer can run at a time
Typically transfers cannot be prioritized

Limited
Security
Often usernames and
passwords are sent with file –
as plain text!
Privacy, authentication and
encryption often not be
available
Non-repudiation often lacking

Limited visibility
and traceability
Transfers cannot be monitored and managed
centrally or remotely
Logging capabilities may be limited and may only
record transfers between directly connected systems
Cannot track the entire journey of files – not just from
one machine to the next but from the start of its
journey to its final destination

A consolidated transport for both files and messages
• Traditional approaches to file transfer
result in parallel infrastructures
‒ One for files – typically built on FTP
‒ One for application messaging – based on
WebSphere MQ, or similar

• High degree of duplication in creating
and maintaining the two
infrastructures

File Transfers

Application
Messaging

• MQ MFT reuses the MQ network for
managed file transfer and yields
‒ Operational savings and simplification
‒ Reduced administration effort
‒ Reduced skills requirements and
maintenance

Consolidated Transport
for Messages & Files

Handling both files and messages across MQ MFT
One file to one message

MQ
MFT
One file to a group of messages

MQ
MFT

• One file becomes one message

• File can be split based on:
‒ Size
‒ Binary delimiter
‒ Regular expression

One message to one file

MQ
MFT

• One message becomes one file

A group of messages (or all messages on the queue) to one file • Optionally, a delimiter can be

MQ
MFT

inserted between each
message used to compose the
file

Staged migration to messaging
Pain-point:
– Hard to migrate to an event driven architecture as lots of applications
communicate by transferring files

Managed File Transfer Helps:
– Deliver files as message payloads and vice versa
– Monitor queues and transfer message payloads to files

WebSphere
MQ Managed
File Transfer


Monitoring queues for the arrival of messages
The WMQ MFT agent can
monitor queues for the arrival of
messages, then perform an
action, such as transferring the
payload from the messages as a
file (as per the previous slide)

Conditions that can be monitored
for:
– Queue not empty
– Complete group of messages

Remember we said MFT
can monitor for files arriving…
Existing
Application

WMQ
MFT
Agent

Well, it can also monitor for
messages arriving on a queue…
Existing
Application

WMQ
MFT
Agent


Example usage of monitoring + program execution
3. MFT transports file
to destination

Existing
Application

1. Application writes
file to file
system

WMQ
MFT
Agent

2. Agent monitors file
system, spots arrival
of file and based on
rules, transfers the file

5. MFT can also start another
application to process the file

WMQ
MFT
Agent

p
* ta

*

Existing
Application

4. At destination MQ MFT
writes file to file system


End-to-end encryption with WebSphere MQ Advanced
(Advanced Message Security)

svrconn WebSphere
MQ MFT channel
MQ

Agent

Queue
Manager

svrconn WebSphere
MQ MFT channel
MQ

Agent

sndr/rcvr
channels

Queue
Manager

sndr/rcvr
channels

WebSphere
MQ
MQ MFT
Queue
Agent
Manager

WebSphere
MQ
MQ MFT
Queue
Agent
Manager

• MQ MFT already supports
transport level encryption using
SSL
• Data is encrypted before it is
sent over a channel and
decrypted when it is received

• The MFT and Advanced
Security of MQ Advanced
allows file data to be encrypted
at the source system and only
decrypted when it reaches the
destination system
‒ This helps reduce encryption costs
‒ Data is secure even when at rest
on a queue


IBM Integration Bus and MFT

Integration Bus
Agent

Message Flow
C:D
Node

WebSphere MQ
Agent

FTEInput

FTEOutput

Message Flow

Agent
File and
Message data

External C:D network
C:D
Node
C:D
Node
C:D
Node

Internal C:D network

• WebSphere Message Broker provides support for integration with IBM’s MFT suite to
transform, enrich file and route data
‒ E.g. Transforming and loading file data into a SAP system

• Broker adds intelligence to MFT networks

Protocol Bridging Agents
Support for transferring files located on FTP and SFTP servers
– The source or destination for a transfer can be an FTP or an SFTP server
Enables incremental modernization of FTP-based home-grown solutions
– Provides auditability of transfers across FTP/SFTP to central audit log
– Ensures reliability of transfers across FTP/SFTP with checkpoint restart
Fully integrated into graphical, command line and XML scripting interfaces
– Just looks like another MFT agent…
Files exchanged between MFT and FTP/SFTP

Agent

Agent

Audit
information

FTP/
SFTP

Agent

WebSphere MQ

FTP/
SFTP
Client

Protocol
Bridge
Agent

FTP/
SFTP
Server

FTP/
SFTP
Client
FTP/
SFTP
Client

The Changing World: Towards SOA
WAS, CICS, IMS

Integration Bus

application and
buisness logics


the data center

MQ

Introducing IBM Integration Bus
IBM’s Strategic Integration Technology
– Single engineered product for .NET, Java and fully heterogeneous integration scenarios
– DataPower continues to evolve as IBM’s integration gateway
Integration
Gateway

Edge

A Natural Evolution for WebSphere Message Broker users
– Significant innovation and evolution of WMB technology base
– New features for Policy-based WLM, BPM integration, Business rules and .NET
Designed to incorporate WebSphere Enterprise Service Bus use cases
– Capabilities of WESB are folded in to IBM Integration Bus over time
– Conversion tools for initial use cases built in to IIB from day one
– WESB technology remains in market, supported. Migrate to Integration Bus when ready

A Broad Range of Supported Platforms and Environments
Broad range of operating system and hardware platforms supported
– AIX, Windows, z/OS, HP-UX, Linux on xSeries, pSeries, zSeries, Solaris (x86-64 & SPARC), Ubuntu
– Optimized 64-bit support on all platforms; 32-bit option available for Windows and x/Linux
– New support for Windows 8 and Windows Server 2012; .NET CLR V4.5 included on Windows
– Express, Standard and Advanced editions make IIB applicable for all solutions and budgets
• All new V9 features available in all editions unless otherwise stated
Virtual images for efficient utilization & simple provisioning
– Extensive support for virtualized environments, e.g. VMWare, AIX Hypervisor… any!
– IBM Workload Deployer for x/Linux & AIX
– Support for Pure on POWER hardware to complement xLinux
– SmartCloud and IBM Workload Deployer images for simplified solution provisioning
Includes access to full range of industry standard databases and ERP systems
– DB2, Oracle, Sybase, SQL Server, Informix, solidDB
– Open Driver Manager support enables new ODBC databases to be accessed
– JDBC Type 4 for popular databases
– SAP, Siebel, Peoplesoft, JDEdwards at no additional cost
Technology components and pre-requisites
– Java 7 on all platforms
– MQ 7.5 prerequisite (7.1 on z/OS)
Detailed System Requirements
– Will be posted on www.ibm.com/integration-bus

Traditional
OS

Public Cloud

Private
Cloud
IBM Workload
Deployer
IBM Pure


Migration from WebSphere Message Broker V6.1, V7 and V8
Migration from WMB V6.1, V7 and V8
– All development assets (e.g. message flows, ESQL, DFDL, Java, Maps and XSLT) import directly
• Right-click convert action for pre-V8 maps; some manual tasks may be required
– Migrate brokers using a single command, or create new brokers for phased migration
• No broker redeployment necessary when using built-in migrate command
• All existing BAR files can be deployed to IB V9 brokers without change
Migration commands for in-place migration
– Includes migration of configuration data including broker databases, queues and registry
– Forwards and backwards migration of existing components, in situ
• mqsimigratecomponents command (includes –t option for rollback to V7 and V8)
Flexible co-existence options remove the need for additional hardware when migrating
– IB V9 co-exists on the same OS with all previous MB versions
– MQ V7.5 required for all IB V9 brokers
• MQ V7.5 supported with all V6.1, V7 and V8 brokers for the purposes of V9 migration

1

Install IB V9

2

Stop broker

Install IB V9

2

Create new broker

3

Run migrate command

4

Start broker

4

Stop old broker

or

1

3 Deploy existing assets


Conversion from WebSphere Enterprise Service Bus
Built-in conversion tools for WESB source assets
– Initial emphasis on web services use cases (e.g. StockQuote)
– Advanced use cases over time; convert when appropriate for your installation
– Open framework for user and partner extensions
Simple workflow creates IB resources
1. Export WESB PI from IID
2. Import mediations into Eclipse Toolkit
3. Right-click “convert” task to start conversion
4. Follow guided editor to generate resources
5. Task List will identify remaining manual steps
6. Iterate as necessary

Open Conversion Framework
– Extensibility means more WESB primitives and resource types can be converted over time
• No minimum version requirement of WESB source
• Builds directly into WESB conversion editor
– Design allows for future assisted resource creation from non-Integration Bus sources, e.g.
– eGate Java collaborations and Event Type Definition, exploiting existing JAXB support
– ICS collaborations, including ASBO and GBO model, exploiting new GDM pattern enablement

Web Visualisation and Analytics
A comprehensive tool for web management
– Manage all integration resources from zero-footprint client
– Analyze integration performance in real-time
– Supported on a variety of browsers: IE10, Firefox, Safari…
– Complements MQ Explorer and WAS Admin consoles
Managing Integration Resources
– View top-level integration node properties
– Add/remove/change integration servers
– Start/Stop integration data flows
– Role based access to control usage
– Advanced options include data replay, policy & monitoring
– Exploits underlying public REST/JSON API
Integration Performance Analysis
– Operational experience; no developer intervention required
• New and existing flows can exploit without change
– Many metrics of integration flow available in real-time
• CPU & I/O time shown by default in integration analyzer
• Other metrics include thread, data sizes, errors…
– Flexible display includes data tables and flow profile
• Drill down to understand detailed behaviour
– Exploits underlying MQTT web sockets technology
• Asynchronous notification at low CPU cost


Controlling Integrations with Policy
Integration Workload Management
– Provide intelligent mechanisms to control processing speed
– Most common scenario is to reduce back-end server load
– Design allows more policy-based processing over time
– Can be applied to new or existing integration data flows
Policy defines threshold limits and relevant actions
– Set thresholds for integration data flow throughput
– Specify actions at threshold, for example:
• NOTIFY: Higher (or lower) than threshold generates publication
• DELAY: Excessive workload will have latency added to shape throughput
• REDIRECT: Send excess to input node’s failure terminal or backout
Web Console used to manage WLM policy
– Sophisticated behaviour controllable by broker WLM policy
– Workload can be managed across classes of message flows (e.g. batch vs. online)
– Policies stored in local registry, and dynamically configurable
– Developer can also specify limits as integration data flow properties
200

200

200

180

180

180

160

160

160

140

140

140

120

120

120

100

100

100

80

80

80

60

60

60

40

40

40

20

20

20

0

0

0

200
180
160
140
120
100
80
60
40
20
0


Managing Unresponsive Integration Flows
Target unresponsive flows through policy to improve overall system reliability
– Additional WLM option aimed at unresponsive integration flows
– An integration flow can become unresponsive for multiple reasons
• e.g. Waiting for external system, infinite loop, deadlock, malformed XML
Waiting
for
response

Flexible configuration, actions and reporting options
– Specify threshold at which flows are considered unresponsive, e.g. 30 seconds for processing
• Configured via WLM policy, or directly on the flow in the BAR file
– Define action to trigger when flow considered unresponsive
• Administrative notification through a new “timeout exceeded” event message
– If flow eventually continues through to completion, a second event is published
• Restart the integration server (execution group) on which the unresponsive flow is running
– New command option to forcibly stop integrations manually: mqsistopmsgflow –f


Synergy with ODM (Rules)
Provide business insight during integration data flows
– e.g. intelligent decision making; score then action in-flight request based on a business rule
– User creates (e.g.) if-then-else rules using tool of choice (Excel, Word, Eclipse…)
– The bus acts on these rules in flow, e.g. for business level routing
New Decision Service node
– Identifies inputs to business rules from in-flight data
• e.g. details of book order from request
• e.g. the item price from key fields…
– Invokes built-in rule engine to perform business logic
• Open interfaces for 3rd party and user engines
– Captures rules output for downstream processing
• Business objects mapped back to in-flight data
Create rules directly inside Integration Bus toolkit
– Significant rules authoring facility built-in
– Automatic package & deploy with integration assets
– Dynamically reconfigure business rule using configurable service policy
– Optionally refer to business rules on external ODM decision server
– Exploit separate full ODM Decision Center for BRMS scenarios
Embedded rules engine for high performance
– Rule is executed in the same OS process as integration data flow
• Succeeds IAM9 Support Pac
– Rule update notification ensures consistent rule execution
– Optional governance of rules through remote ODM Decision Center

Synergy with BPM (Express and Standard)
IB provides powerful connectivity layer for BPM workflows
– Allows BPM developer to exploit rich integration features
• E.g. .NET, Healthcare Pack, TCP/IP, GDM, DFDL…
– No changes required to existing BPM programming model
• Helps maintain separation of concerns between roles
• Process designer works with integration developer
– Complements SCA nodes for BPM Advanced (WPS)
Start with business process definition
– Process Center snapshots provides integration handover
• Snapshot can include multiple service definitions
• Captured as .twx file
– Integration developer imports snapshot from BPM
• Provides implementation of selected definitions
• Built-in integration tools simplify this activity (see below)
– Process designer re-imports updated snapshot from IB
• Completes business process definition
• Calls integration service in BPM system activity
New BPM pattern simplifies creation of integration solution
– Start from Pattern Explorer, or right-click on existing service
• Import .twx file to create skeleton integration flow
– Customize created integration flow with IB capability…
• All other IB features available
– Deploy integration and pass back concrete references to BPM e.g. server IP address, etc.

Comprehensive .NET Support
New patterns and samples for MS Dynamics CRM and MSMQ
– SAP CRM pattern for customer account synchronization
• Map account operations between BAPI & CRM Entities
– Advanced CRM pattern enables dynamic graphical mapping
– New customizable sample for 2-way MSMQ and MQ exchange
New and enhanced nodes for .NET programmers
– .NET Input node allows developers to initiate integration logic from any .NET system
• e.g. receive request from Dynamics CRM, AX, periodically read EXCEL file…
• Highly customizable polling and trigger mechanisms
– CLR V4.5 runtime embedded within the integration server provides .NET technology foundation
• Languages include C#, VB .NET (COM), JScript & F#, with full range of .NET data types
• Also includes app domains for isolation
• Exploited by .NET Compute node and .NET Input node
– Further extensions include Visual Studio 2012, Windows 8/Server 2012 and Azure Cloud compatibility
Developer Customizations
– Cloned .NET nodes
• Easy to understand,
consume and reuse
– Custom user properties
• Expose key properties
• e.g. CRM IP address
– Simple cloned node capture
• User-defined icons
• .NET Toolkit drawer

Integration Services for SOA
Integration Services are well-defined containers of integration logic
–Integration Services are created for and reside inside the bus
–Interface is expressed via WSDL with a port type
• Interface and structure are both required

–e.g. Request, response & fault handlers per operation
• Default binding is created out of the box

Integration Services are defined through standard resources
–WSDL (port type) defines service interface
–Service interface defines one or more operations
–Service Descriptor (XML) ties service interface with implementation
–Each operation is implemented as an integration subflow
• Supporting resources also associated (e.g. Maps, XSDs)
• Resources optionally reside in libraries

–Main entry point is implemented as an integration flow
Simple lifecycle for services creation and management
–Simple creation of new integration services
•
•
•
•

Creating a new “Service” container
Import WSDL or create from scratch
Implement services
Specify binding before or during deployment

–Deployment as per standard integration applications
–Unit Test and Team options available


MQ Service Discovery
Service definitions allow you to make best use of available resources
– Facilitates sharing of service information between users and systems
– Allows users to understand interfaces (e.g. CustomerAddress.Update operation)
– Provides a connector with which to exchange technical configuration (e.g. hostname)
– Provides attachment points for associated policies (e.g. authorization)
New framework enables discovery, cataloguing and re-use of services
– Discovery connectors translate the service provider description to a common model
– Interrogate IT systems for definition of technical assets – objects, functions and interaction points
– User selects and refines definition of technical assets
– Service definitions created and associated with technical assets
– Discovered service definitions stored in embedded registry
– Use catalogued services to configure integration solutions
Initial implementation discovers and catalogs MQ service definitions
1.Discover queues from referenced queue manager endpoint
2.IB develops MQ service definition and stores in registry
3.Use service definitions to configure MQ connectivity
–Sets required MQ headers and queue references
New and existing nodes will be updated over time
– Completely aligned with runtime connector framework
•Simple protocol points appropriate for style of interaction
•Allows for simple development of custom connectors

Database Service Discovery and Data Analysis
Customers

Accounts

Products

Partners

DBMS represents system of record for key business entities
– Customers, accounts, partners, products… all stored in databases
– Integration Bus tools discover and represent these key data
– Integration services extends access to end-user applications
New integration tools discover key database assets
1.Connect to DBMS
2.Discover source artefacts (tables, views, etc.)
3.Map CRUD operations to service interface
4.Save in canonical WSDL document
•Custom bindings for SQL access
5.Re-use database WSDL in multiple scenarios
Many uses for database service definition
– WSDL contains both logical and physical database information
– Drag and drop WSDL to automatically create SQL access methods
– Create new integration service to exploit customized database access
•End-user application consumes as regular (e.g.) web service
Customize integration services with data analysis
– Tools for solving the problem of XML document understanding
•XML message formats can be structurally diverse
•Often useful to semantically interpret related elements, e.g. healthcare CDA exchange format
– New Data Analysis Perspective provides a collection of useful data views
•Model data based on input element XML; understand and visualise related elements
•Generate resources (subflows, maps) that allow transformation between modelled elements

Easy Data Modelling with DFDL
Simple & powerful open standard for data modelling
– For use in IBM and non-IBM products
• e.g. Integration Bus, Rational Performance Test
Server, Rational Test Virtualization Server,
Rational Test Workbench, Rational Developer for
System z…
• May also be used in standalone applications
– DFDL web community now active
• Public GitHub repositories for DFDL models
– Including HL7, ISO8583 and TLOG
• Commercial and scientific formats
• Collaborative development of models
Support more features of the DFDL specification
– User-defined variables in DFDL expressions
– TLOG packed numeric fields
– Delimited binary data
– Fields lengths given by regular expressions
Improved performance
– Continued improvement when parsing & writing
– Improved deployment times
Usability Improvements to the DFDL editor
– Copy and paste of schema objects
– More refactoring operations
51


Graphical Transformations

IBM Graphical Data Mapper
InfoSphere MDM

IBM Graphical Data Mapper (GDM)
DataPower
– Visually map and transform source to target data
IBM Integration Designer
– GDM designed for whole IBM product set, e.g.
• Integration Bus V9, WebSphere Message Broker v8, DataPower
• InfoSphere Master Data Management v10, Integration Designer v7.5/v8
• Rational Application Developer for WebSphere Software v8.5
• Rational Software Architect v8.5, RSA for WebSphere Software v8.5
• Other products yet to announce
– Rich feature set and simplicity make this a good default transformation choice

RAD for WebSphere
IBM Integration Bus
Rational Software Architect

Directly access stored procedures from within a map
– Complements existing database select, insert, update, delete
– Incorporate user-defined database functions into your graphical transforms
– All standard broker databases supported, e.g. Oracle, DB2, SQLServer…
Maps available to user patterns
– Graphical creation of flows which require transformation logic
• e.g. new input or output messages
– Invocation of mapper when pattern instances are generated
– User guidance through HTML pattern help and task list
– Patterns to demonstrate include CRM account mapping
Migration of pre-V8 maps to IBM GDM
– Most sophisticated maps can now be converted in a single step
– Editor provides enhanced feedback about conversion to assist user understanding

Natural Integration with WAS Admin Console
IIB is a compelling choice for WebSphere Application Server
–WAS provides efficient application development and delivery
–New tools to simplify learning curve for WAS users
–Addresses administrator requirements
–Supported on WAS V7 and V8.x
WAS Admin Console Broker Plug-in
–Varied set of MB administration tasks available
•
•
•
•

Connect to multiple local or remote brokers
View available execution groups and their current status
View services, applications, libraries, message flows
View Message Broker console help topics

–Uses standard features for ease of configuration
• Role-based access to prevent unauthorized administration
• REST APIs for local and remote management

–Use WAS Admin Console for WAS centric administrator
• Complements IIB Web UI for IIB-centric administrator

–Design allows for future modification of integration resources, start, stop etc.
Fully compatible with WAS ND
–Integration feature operates at cell level
–The configuration data is stored by the Deployment manager
–All Application Servers have access to cell level configuration data


WAS Service Mapping for Application Developers
Service Mapping
Pipeline
APP

APP

Provides basic mapping, routing and versioning capabilities to applications
– Scenarios include service versioning and meet-in-the-middle interfaces
– Uses existing developer skills and tools, e.g. Java and RAD
– Service Mapping pipeline invoked directly from application
– Delivered as part of WebSphere Application Server
– Service Mapper can generate events for Integration Bus
• Enables audit, replay and out-of-band processing

App Server
Events
Integration Bus

Developer experience enables mapping between Application Server services
– Line-of-business application developer calls service from within application logic
– New WAS mapping service intercepts, maps and reroutes WS calls accordingly
– Uses existing application developer tools
• Develop service maps directly within RAD using standard GDM technology
• Configure and enable in WAS Admin Console
– Map between services without leaving WAS runtime environments for high efficiency
• Included in WAS v8.5.5 – no additional install or license required
IIB consumes events from service mapping for “system of awareness”
– Service map invocations can emit business events
• Just like IB integration flows, can be captured, recorded and replayed
– Built on GDM technology with interface mapping
• Allows for future hosting of same service maps inside Integration Bus


Easily Integrate with Appliance-based Messaging
Introducing IBM MessageSight
– An appliance-based messaging server built on special purpose hardware
– Supports very large numbers of connected clients and devices, and high volumes of messages
– Secures the edge of the enterprise and enables use cases like mobile and telemetry
Two new patterns for integrating IBM MessageSight with backend systems
– Covers common use cases for bi-directional connectivity
– Use of JMS enables standards-based appliance connectivity that is also extensible to other providers
– Pattern design allows for future selection of high performance, standard MQTT as protocol

MessageSight

MessageSight
Many
connected
devices

IB

Back-end
system

Filter

IB

Back-end
system

Notification

Many
connected
devices

1) Event Filter Pattern
– Messaging appliance routes inbound events into the broker via JMS
– The broker narrows down events using decision service and inserts the subset into backend systems
2) Event Notification Pattern
– The broker detects an event from a backend system (e.g. message queue, database trigger)
– Broker fans out event via JMS to the appliance to interested connected clients

Integration Bus and Mobile Integration

Mobile enable any enterprise service in 2 clicks !
Build robust solutions with integrated caching and security
Push data to mobile users from enterprise applications

Mobile App
Platform

Create end-to-end mobile solutions for Microsoft .NET

Four new IIB development patterns
− Fully integrated in IIB toolkit
− Generate Worklight adapter, test application and supporting
WMB flows
1. Simple Service to mobile – WSDL based
2. Resource access from mobile - Controlled access to
enterprise data as a resource: methods for read create,
update and delete
3. Microsoft .Net service pattern
4. Queue based Push Notification
Patterns exploits value add IIB capability including policy decision
point for access authorization and WebSphere eXtreme Scale for
performance


MyVar = Cache.Value;

Global Cache Enhancements

Broker1

Broker2

Cache.Value = 42;

External Cache

IB contains a built-in facility to share data between multiple brokers
–Improve mediation response times and dramatically reduce application load
–Typical scenarios include multi-broker request-reply and multi-broker aggregation
–Uses WebSphere Extreme Scale coherent cache technology
Support for external software and hardware caches
–Access separate eXtreme Scale and DataPower XC10 appliances from within the broker
–Allows broker to interact with enterprise caching solution without embedding additional libraries
–Cache access, activity log, resource statistics etc. just like embedded cache
–Operationally configured using dynamic configurable service
–New EG options to specify SSL connections to external WXS grids
•Uses existing MB SSL infrastructure to configure certificates

Cache Expiry options
–New getGlobalMap() variant to set the time to live for data in the embedded global cache.
•MbGlobalMap evictMap = MbGlobalMap.getGlobalMap("…", new MbGlobalMapSessionPolicy(30));
•evictMap.put("key", "val");

–Specify a value in seconds. The default value is 0, which means data never gets automatically removed.
Programming and operational enhancements
–Insert and lookup map data using a wider range of Java object types for simplified programming logic
–Support for highly available multi-instance configurations


Accelerating Integration with Built-in Patterns
Best practice, reusable solutions to common integration problems
• Can be quickly configured and deployed, and extended to create new patterns
• Patterns for SAP, mobile, BPM, Message-based integration and more!
Service
Enablement

Service
Virtualization

Application
Integration

Encapsulating functionality and
presenting it through a serviceoriented interface

Loose coupling and mediation
between services

Enabling rapid integration of
applications across the enterprise
e.g. SAP

Message-based
Integration

File
Processing

Custom
Patterns

Providing routing, transformation
and logging services for
messages

Managed runtime environment for
processing files such as shredding
into individual transaction records

Create your own patterns to reuse across your organization

The Changing World: And Then There Was Mobile

IWD,
PureApp

Cast Iron, DP XH40
Connectivity to
applications in the
a new cloud economy

Public
Cloud

WAS, CICS, IMS

Integration Bus

application and
buisness logics

WXS, DataPower XC10
Cache grids improve
of applications and
services

Private
Cloud

Internet

Mobile
Sensors

environments has
the enterprise
(Worklight)

Secure
appliances
enable
controlled
access to
Enterprise
resources
DataPower XG45

the data center has
the Internet
Internet
MQ, MessageSight

Mobile … a new frontier in business growth
Mobile B2C
• Increase customer
satisfaction by enabling
banking, insurance, and
trading anywhere,
anytime
• Reach customers in new
ways through mobile
applications, SMS, email

Mobile B2E & B2B
• Enable field employees for
increased productivity
• Greater efficiency and
accuracy in supply chain
operations
• Exchange business
information with partners
securely

(Mobile) M2M
• Enable the exchange of
data and events between
businesses and machines
• Internet of Things - sensor
events feeding information
and driving a smarter
planet

Why messaging for mobile and machine-to-machine?

HTTP revolutionized how we consume data
– Simple request/response model
– Available on any tablet, laptop, phone, PC
– Not designed for wireless
– Slow and unreliable on mobile networks

Mobile and M2M applications have additional challenges
– Requires a real-time, event-driven model
– Publishing information one-to-many
– Listening for events as they happen
– Sending small packets of data in huge volumes
– Reliably pushing data over unreliable networks

Use case – Connected car

predicts part
failure

vibration
detected, details
published

Find
my car

schedules
appointment
with car owner

Unlock
my car

Connected car


MQTT – the protocol for mobile and M2M messaging
MQTT is a messaging protocol ideally suited to mobile and M2M environments
It has an open specification (http://mqtt.org)
– There are over 40 different client implementations
– Standardization with OASIS is in progress
High-quality, open-source implementations of clients
– Hosted at the Eclipse Paho project
– Build the clients yourself or use free ones from IBM or others
MQTT is very lean and fast
– Efficient format with minimal overhead
– Client implementations are small and can run on small devices
Communication using messaging is much more flexible than request/response
– Bi-directional, asynchronous “push” communication
– Publish/subscribe decouples the senders of information from the receivers


Introducing IBM MessageSight

IBM MessageSight is a messaging appliance
Allows your business to scale to the demands of the mobile and M2M use cases
Easily and securely extends your existing messaging infrastructure across the
Internet
Enables real-time, streaming analytics and event processing


IBM MessageSight feature summary

• Appliance form-factor simplifies
physical installation
• Up and running in under 30
minutes
• Task-oriented UI guides the
administrator through the first steps
• Simple and scalable administration
using policies
• Fast time-to-value

Secure
and Reliable
DMZ Ready
Appliance

• Hardened appliance form-factor suitable for DMZ use
• Secure firmware
• No user-visible, general-purpose OS
• SSL/TLS support for MQTT, JMS and MQ connections
• Reliability
• Supports both nonpersistent and persistent messaging
• Hardware-assisted persistent store
• High availability using pairs of appliances
• Synchronous data replication from primary persistent store
to standby

Secure
and Reliable
DMZ Ready
Appliance

Optimized
for Wireless
Clients
M2M and
Mobile

• Efficient MQTT protocol ideal for mobile and M2M
deployments
• Large range of clients supported including JavaScript using
WebSockets
• Apps can be HTML5 web apps, native or hybrid
• Integrates easily with IBM Worklight


Secure
and Reliable
DMZ Ready
Appliance

Optimized
for Wireless
Clients
M2M and
Mobile

Developer
Friendly
Simple
Programming
Model

• Simple yet powerful APIs consistent across a wide choice of
languages
• Simple programming model (connect, publish, subscribe)
• Active development community on IBM developerWorks
http://www.ibm.com/developerworks/connect/IBMmessaging


Secure
and Reliable
DMZ Ready
Appliance

Optimized
for Wireless
Clients
M2M and
Mobile

Developer
Friendly

Easy to
Integrate

Simple
Programming
Model

Open
Standards

• Compatible with a wide variety of environments
• JMS for Java Standard Edition environments
• WebSockets for Rich Internet Applications
• MQTT protocol with many open-source clients
• Built-in connectivity with WebSphere MQ
• One appliance can connect to multiple queue managers
• Destination mappings to/from MQ queues/topics
• Integrates with IBM Integration Bus (Message Broker)
• New mobile patterns integrate using JMS nodes


Secure
and Reliable
DMZ Ready
Appliance

Optimized
for Wireless
Clients
M2M and
Mobile

Developer
Friendly

Easy to
Integrate

Massive
Scale

Simple
Programming
Model

Open
Standards

Orders of
Magnitude
Improvements

• A million concurrently connected devices
• 13 million nonpersistent messages per second
• Over 400K persistent messages per second
• Predictable low latency
• Carefully optimised design


MessageSight solutions for Automotive
What are they interested in doing today?
Offering new innovative services to the drivers, dealers, partners, etc
Find my car / unlock my car
Predictive analysis for parts failure
Impose limits on policy and driver
(who can go where at what speed..)
Unattended car rental
eCall 2015 (EU obligatory emergency call capability in vehicle)
Automated driving system

What problems are they facing?
Connect to millions of vehicles gathering vast amounts of telematic data in real time, analyze the data, and determine
message response back to a single or multiple vehicles or other interested parties
Ability to store messages for vehicles that temporarily lose communication
Integrate with existing data sources in enterprise systems such as vehicle service history records
Integrate with partners

What is needed?
Efficient and reliable two way communication
Access security to ensure information is only sent between authorised vehicles and trusted sources
Open protocols (WebSockets, JMS, MQTT) and MQ Connectivity
for easy integration to both the internet and the enterprise


Messaging Clients for Mobile Apps
Available from new IBM Messaging Community on developerWorks
Clients for Mobile and M2M Messaging
Including Simple Javascript Messaging API

* = beta only
Apple iOS
Messaging for
native iOS apps
Android
Messaging
for native
Android apps

Worklight
developers
Developer
experience
augmented with
mobile
messaging

JavaScript
Messaging for
Hybrid apps
JavaScript*
Messaging for
Web apps

Service

APP

APP

DB

DB

APP Service

DB

APP

Internet

MQTT
•Reliable messaging
protocol
WebSphere MQ or Appliance
•Access to enterprise
•Universal Enterprise Messaging
•Reliable delivery
•Provide access to enterprise apps
•Conserve device
and data already connected by MQ
power
•Pub/sub model to enable dynamic
•Reduce network traffic
distribution of notifications


The Changing World: (Social) Evolution of “Partner”

IWD,
PureApp

Cast Iron, DP XH40
Connectivity to
applications in the
a new cloud economy

Public
Cloud

WAS, CICS, IMS

Integration Bus, DataPower XI52

application and
buisness logics

WXS, DataPower XC10
Cache grids improve
of applications and
services

Private
Cloud

Internet

Mobile

Partner community
Sensors

environments has
the enterprise
(Worklight)

Secure
appliances
enable
controlled
access to
Enterprise
resources
DataPower XG45

the data center has
the Internet
Internet
MQ

B2B partners opened
channels and
collaboration while a
new genre “App
Developer partner” is
emerging
Sterling Commerce
DataPower XB62
API Management

The Basics of a web API

What is a Web API?
A web API is a public persona for an enterprise; exposing defined assets,
data or services for public consumption
A web API is simple for app developers to use, access and understand
A web API can be easily invoked via a browser, mobile device, etc.

What Value Does a Web API Provide?
Extends an enterprise and opens new markets by allowing external app
developers to easily leverage, publicize and/or aggregate a company’s
assets for broad-based consumption

What “assets, data or services”
are exposed via an Web API?:
Product catalogs
Phone listings
Insurance cases
Order status
Bank loan rates

External
App Developer

Apps, APIs and API Mgmt…

Benefits
New business opportunities
• New markets
• Increase customers
• Enhance branding
• Competitive advantage

Consumers

Challenges

Extend development team
•Increase innovation
•Increase scale

Developer

Business strategy
Infrastructure
• Security
• Creation
• Scalability

Partner/supplier
alignment

Business
Owner

IT

Operational control
• Publish
• Analyze
• Monitor

Easily Define APIs

• Define the API you wish to expose
• Then configure the API by proxying
an existing REST API or assemble
a new API
• Provide examples of the request
and response messages, headers
and parameters

Assemble New APIs Through Configuration
• Connect to one or more
datasource
‒
‒
‒
‒
‒
‒
‒

DB2
MySQL
SQL Server
Oracle
Salesforce.com
SOAP to REST
HTTP

• Drag and connect linking
the request and response
messages
• Transform the message
elements with a click

The Changing World: Emergence of Appliances
WAS, CICS, IMS

Integration Bus

application and
buisness logics

WXS, DataPower XC10
Cache grids improve
of applications and
services

Secure
appliances
enable
controlled
access to
Enterprise
resources
DataPower XG45

the data center

MQ

DataPower 6.0

Secure. Integrate. Optimize.
Secure integration

Pattern-based configuration

Securely integrate API, Web & Mobile
workloads, in addition to SOA & B2B

Create & deploy common configuration
patterns for reduced time to value,
improved productivity & quality

Mobile-ready security gateway

System z integration

Secure & optimize delivery of Mobile
applications & integrate with
IBM Worklight

Easily consume external web services
from IMS & expose IMS data as a
service

Faster consistent response time

Deployment flexibility

Reduce load on back-end systems and
optimize delivery through local & external
caching and intelligent load distribution

Use physical or virtual appliance with
seamless configuration migration

DataPower in a Nutshell
Enable additional use-cases with a single, policy-driven converged gateway
Focus so far
Web Service
Gateway

•
•

•

•

DMZ-ready
Security
gateway (AAA,
XML threat)
Service level
management
and monitoring
Intelligent load
distribution &
dynamic routing

Expanded focus

Integration
Gateway

•

•

•

“Any-to-Any”
conversion at
wire-speed
Bridges multiple
transport
protocols
Mainframe
integration &
enablement

B2B Gateway

•
•

•

•

DMZ-ready
B2B Messaging
(AS1/AS2/AS3/
ebMS)
Trading Partner
Profile
Management
B2B
Transaction
Viewer

Web
Application
Gateway
•
•

•
•
•

DMZ-ready
First-class
integration with
WAS
Cache response
content
Web application
security
Traffic mgmt

On-premise
API
Management
•
•
•
•

DMZ-ready
Web API
security
Monitor API use
Enforce API
consumption
policies

Form factors
Physical Appliance for hardware performance & security

Virtual Appliance for deployment flexibility

Mobile
Application
Gateway
•
•

•

•

DMZ ready
Mobile
application
security
Support
Worklight
mobile platform
Monitor and
control mobile
app access

DataPower Classic Use Cases
• Monitoring and control
– Example: centralized ingress management for all Web Services using ITCAM SOA

• Deep-content routing and data aggregation
– Example: XPath (content) routing on Web Service parameters

• Functional acceleration
– Example: XSLT, WS Security

• Application-layer security and threat protection
– Example: XML Denial-of-Service protection, WS Security, AAA
In-the-clear
SOAP/HTTP

• Protocol and message bridging
– Example: Convert to WS to legacy Cobol/MQ

SOAP

In-the-clear
SOAP/HTTP

Encrypted and
Signed SOAP/HTTP

SOAP

SOAP

Clients

Cobol/
MQ
Appl

Malicious
SOAP/HTTP

Cobol/MQ

Service
Providers

Use Case: Security Gateway (Consumer or Employee)
AAA authenticates
user and ensures
they are authorized
to the resource
beings accessed

SLM shapes and
throttles traffic based
on seggregated
consumer types

DMZ

Trusted
Zone

Internet
XG45

Usually (REST)
JSON or XML over
HTTP from web or
mobile application
XML and JSON wellformedness, schema
valication and DoS
protection

Request are
transformed to
internal schema and
routed appropriately

Responses are cached
to improve response
time

Use Case: Security Gateway (B2B)
AAA authenticates
user and ensures
they are authorized
to the resource
beings accessed

SLM shapes and
on seggregated
consumer types

DMZ

Trusted
Zone

Internet
XG45

Usually Web Service
(SOAP) or AS*
(XB62) protocol

XML wellformedness, schema
valication and DoS
protection

Request are
transformed to
internal schema and

to improve response
time

Use Case: Security Gateway (Inter-LoB)
AAA authenticates
user and ensures
they are authorized
to the resource
beings accessed

SLM shapes and
on seggregated
consumer types

Security
Zone
Boundary

Trusted
Zone B

Trusted
Zone A
XG45

Usually Web Service
(SOAP) or AS*
(XB62) protocol

XML wellformedness, schema
valication and DoS
protection

Request are
transformed to
internal schema and

to improve response
time

Use Case: Integration Gateway (Edge of Service)
Trusted Zone
AAA authenticates
user and ensures
they are authorized
to the resource
beings accessed

SLM throttles all
inbound traffic to
perform overload
protection of backend
resource

XI52

Usually Web Service
(SOAP) or REST

Requests are
transfomed to local
format (eg. Cobol)

Request are
transmitted over local
transport and
intelligently
distributed (e.g. MQ)
to improve response
time

IBM Messaging Focus Areas

Deliver Messaging Backbone for Enterprise
Focus on traditional MQ values, rock-solid enterprise-class
service, ease-of-operation, breadth of platform coverage,
availability, z/OS exploitation

Capture Big Data from Mobile and Internet of Things
Focus on Internet-scale events, m2m device enablement,
zero-admin, security and privacy, feed into real-time
analytics, location-based notifications

Enable Developers to build more scalable, responsive
applications
Focus on new app dev use cases, breadth of languages, ease-ofdeployment, lightweight services, integration with developer
frameworks

IBM Connectivity and Integration

More Related Content

What's hot (19)

Similar to IBM Connectivity and Integration (20)

More from IBM Sverige (20)

Recently uploaded (20)

IBM Connectivity and Integration