Microservices - Death of the Enterprise Service Bus (ESB)? (Update 2016)

Microservices –
Death of the Enterprise Service Bus?
(Update 2016)
Kai Wähner
Technical Lead
kwaehner@tibco.com
Xing / LinkedIn
@KaiWaehner
www.kai-waehner.de

Key Messages
–  Microservices = SOA done right!
–  Integration is key for success – the product name does not matter!
–  Real time event correlation is the game changer!
“Microservices = Death of the ESB?” by Kai Wähner © Copyright 2000-2016 TIBCO Software Inc.

Agenda
–  Digitalization
–  Enterprise Service Bus
–  Microservices
–  Architecture and Requirements
–  Live Demo
–  Challenges

Everything Generates More Data
© Copyright 2000-2016 TIBCO Software Inc.

New Applications, New Requirements
•  Purposeful: Users are
looking for tools not toolboxes.
•  Adaptable: Similar application
services can be consumed via
a variety of channels, in a
variety of contexts.
•  Sustainable: Applications
services need to support user
experience by combining
performance and flexibility.

“Applications created today using the
good-old traditional architecture will
be a business-constraining legacy
before they are completed.”
Yefim Natis, Software Defined Applications
Webinar, May 8th 2015
Too Slow, Too Complex

Integration is key for success!
Integration will get even more important
in the future than it is today!
The number of different data sources and technologies increases
even more than in the past
CRM, ERP, Host, B2B, etc. will not disappear
DWH, Hadoop cluster, event / streaming server, In-Memory
DB – all of them have to communicate
Cloud, Mobile, APIs, Big Data, Internet of Things are no
option, but our future!
EVERYTHING HAS TO BE INTEGRATED!

Evolution of Integration
Microservices
Web APIs
Real-3me
Web
Oriented
Architecture
Enabling Technologies
In-Memory, Mul3core
REST, JSON
Demand Drivers
Mobile, Cloud
Fast Data, IoT
Services
Web Services
Real-3me
Service
Oriented
Architecture
XML, SOAP, WS-*
Process Modeling
Demand Drivers
E-Commerce
BPM
Interfaces
Adapters
Real-3me
Enterprise
Applica3on
Integra3on
Client-Server
Messaging Middleware
Demand Drivers
ERP
Analy3cs
Records
Batch Jobs
Non-real3me
Mainframe
ETL, Databases
Data
Integra3on
Demand Drivers
Data Processing
MIS
Are we
there yet?
Level-Up by
u2lizing the
lessons,
assets and
prac2ces of
the previous
Level
Accelera2ng Produc2vity & Agility

Acting in Real Time gets more and more important!
Time
Business
Value
Business Event
Data Ready for Analysis
Analysis Completed
Decision Made
$$$$
$$$
$$
$ Action Taken
Event Processing
speeds action and
increases business
value by seizing
opportunities while
they matter

12
Branding of Integration Software
•  Keywords PAST:
Application Integration, EAI, Broker, Application, Integration, Enterprise, Hub
and Spoke, Backbone, Scalability, Platform, Batch
•  Keywords PRESENT:
Service Integration, Bus, SOA, Service, ESB, Flexibility, Distribution,
Events, EDA, Real Time, Event Correlation, Open, Standards, Extensibility
•  Keywords FUTURE:
Integration of Everything, Cloud, IoT, Gateway, Microservice, API, Public
Independence, Continuous Delivery, Self-Service, Prediction, In-Memory

13
TIBCO

oﬀers middleware for mission-cri3cal real 3me

Integra3on and Event Processing

for 20+ Years…

14
TIBCO Website (Year 2000)

15

16

17

18
Same story for

IBM, Oracle, SoSware AG, …

19
TIBCO BusinessWorks

(which is TIBCO’s integra3on ﬂagship product)

was never branded ESB


20
Enterprise Service Bus (ESB)
An enterprise service bus (ESB) is a so#ware
architecture model used for designing and
implemen3ng communica3on between mutually
interac3ng soSware applica3ons in a service-
oriented architecture (SOA). Its primary use is in
enterprise applica3on integra3on (EAI) of
heterogeneous and complex landscapes.

h@p://en.wikipedia.org/wiki/Enterprise_service_bus (Wikipedia, 2016)

21
Central ESB
hp://stage.reﬂectsoSware.com/SOA/Enterprise%20Integra3on%20EAI%20vs.%20SOA%20vs.%20ESB.pdf

22
Distributed ESB
hp://stage.reﬂectsoSware.com/SOA/Enterprise%20Integra3on%20EAI%20vs.%20SOA%20vs.%20ESB.pdf

23
Is the ESB dead? #NoESB
Gartner
hps://www.gartner.com/user/registra3on/webinar?
resId=2855231&commId=128383&channelId=5500&srcId=null
Akana (former SOA SoSware)
hps://blog.soa.com/noesb/

24
Definition of the Term “ESB” in 2016
•  Do not care about branding
•  What is a “modern” ESB?
–  flexible, distributed, scalable infrastructure
–  build, deploy and monitor any kind of (micro)services in an agile, efficient way with open standards
–  Development and deployment can be done on-premise, in the cloud, or in a hybrid approach
–  Be aware of re-branded central EAI brokers with old code base and new name. Watch out for API-only
platforms, which re-implement ESB features.
•  What to use an ESB for?
–  Integration, orchestration, routing, (some kinds of) event processing / correlation / business activity
monitoring
–  API and REST are great. However, have you ever used a mature and powerful SAP connector? Or
what about Internet of Things – it needs messaging (WebSockets, MQTT, AMQP, …)?
–  You can also build business applications via (micro)services, which implement your requirements and
solve your business problems
–  Deploy these services independently from each other with a standardized interface to a scalable
runtime platform – automatically
–  The services are decoupled and scale linearly across commodity hardware
–  Think of an ESB as a “service delivery platform”, not just an integration platform

•  Services implementing a limited set of functions
•  Services developed, deployed and scaled independently
Microservices

Shorter time to results
•  Scale development and reuse of services
•  Use the right technology for the job
Increased flexibility
•  Change / improve any Microservice without
major disruption on apps or other services
Benefits of Microservices

28
Mhhh….
Sounds like SOA!

Microservices clearly specify important differences to SOA
(as we see SOA implemented in most enterprises today):
•  No commitment to a unique technology
•  Greater flexibility of architecture
•  Services managed as products, with their own lifecycle
•  Industrialized deployment
•  Dumb routes and smart endpoints instead of a heavyweight ESB
Sounds like SOA?
Integra3on s3ll needed somewhere!

Requirements for a Microservices Architecture
①  Service Contracts
②  Exposing new and existing Services
③  Discovery of Services
④  Coordination Across Services
⑤  Managing Complex Deployments and their Scalability
⑥  Visibility and Correlation across Services

Requirements for Microservices Architecture
#1: Services Contract
32

33
Service Contracts
Service provider express the purpose of the
Microservice, and its requirements

Other developers can easily access this
informa3on
Service contracts, and the ability for
developers to discover them, serve that
purpose.

•  Examples: Java Interface, JMS, SOAP, REST, …
•  In Practice today:
–  SOAP: Internal, standards-based, XML Schema, easy mappings and
transformations, performance no issue (anymore)
–  REST (i.e. RESTful HTTP without HATEOAS): External, XML or JSON,
Good architecture for mobile devices (simplicity, separation of concerns,
no state, uniform interface)
–  Messaging (e.g. WebSockets, MQTT): Good for thousands of devices
and millions of messages (Internet of Things!)
•  De facto standard for Microservices as of today: REST
•  Internet of Things will move Messaging forward!
34
Technologies for (Micro)Service Contracts

#2: Exposing new and existing Microservices

36
Services come in various forms
Integration
Service
Monolith application SOA
Integration
Service
Service ServiceService Service
API Gateway
SaaS Service

Smart endpoints and dumb pipes
“When building communica3on structures between different processes, we've seen many products and
approaches that stress pukng significant smarts into the communica3on mechanism itself. A good
example of this is the Enterprise Service Bus (ESB), where ESB products o#en include sophis6cated
facili6es for message rou6ng, choreography, transforma6on, and applying business rules.

The Microservice community favours an alterna6ve approach: smart endpoints and dumb pipes.
Applica6ons built from Microservices aim to be as decoupled and as cohesive as possible - they own their
own domain logic and act more as filters in the classical Unix sense - receiving a request, applying logic as
appropriate and producing a response. These are choreographed using simple RESTish protocols rather
than complex protocols such as WS-Choreography or BPEL or orchestra6on by a central tool.

The two protocols used most commonly are HTTP request-response with resource API's and lightweight
messaging. The best expression of the first is

Be of the web, not behind the web

-- Ian Robinson”
hp://mar3nfowler.com/ar3cles/microservices.html#SmartEndpointsAndDumbPipes

Smart endpoints and dumb pipes
“When building communica3on structures between different processes, we've seen many products and approaches
that stress pukng significant smarts into the communica3on mechanism itself. A good example of this is the
Enterprise Service Bus (ESB), where ESB products o#en include sophis6cated facili6es for message rou6ng,
choreography, transforma6on, and applying business rules.

The Microservice community favours an alterna6ve approach: smart endpoints and dumb pipes. Applica6ons
built from Microservices aim to be as decoupled and as cohesive as possible - they own their own domain logic
and act more as filters in the classical Unix sense - receiving a request, applying logic as appropriate and producing a
response. These are choreographed using simple RESTish protocols rather than complex protocols such as WS-
Choreography or BPEL or orchestra3on by a central tool.

The two protocols used most commonly are HTTP request-response with resource API's and lightweight
messaging. The best expression of the first is

Be of the web, not behind the web

-- Ian Robinson”
Agreed!
However, be aware that you have to do “ESB tasks” (integra3on, rou3ng, transforma3on, etc.) in the service then!

à  Op3on A: Build (REST) Microservices with plain source code or a framework
à  Op3on B: Use a tool to generate that code for (REST) Microservices, and choose the technologies you need
ESB - as defined in this talk – has nothing to do with WS-*, BPEL, or other specific technologies!

Integration as foundation of Microservices
§  Access any data to use in
Microservices
§  Expose standard transport from
Microservices
§  Assemble new Microservices

Leverage any technology to create Microservices
Abstract complex APIs using:
•  Standard connectors
–  File, JDBC, SOAP, REST, JMS,
etc.
•  Application connectors
–  SaaS (SFDC, Marketo), SAP, Big
Data, Mobile, legacy applications,
etc.
•  Plugin development kit
•  Programming languages
–  Java, Scala, Ruby, etc.
§  Onboard new technologies
§  New channels
§  New data sources

Create REST APIs service quickly
§  Top-down or bottom-up modeling
§  Automatic docs and testing web UI

#3: Discovery of Services
42

The new “Open API” Economy
Your
Enterprise
Closed APIs
EDI Web
B2B SOA FTP
Pre-deﬁned integra3on points
Limited, trusted partners
Strictly constrained interac3ons
Enterprise friendly
Open APIs
Opportunis3c access points
Many partners, untrusted
Encourage new ideas
Developer friendly
Your
Enterprise
Partners
Innovators
Known
Par3es
Consumers
Unknown
Par3es
Suppliers
Coope33on
Employees
API API
API
API
API
API
API
API
API API
API
API
API
API
API
API
API
API
API
API

Real World Use Cases for Open API
•  Paypal (eCommerce consumer) è Pay everything with the
same online payment service in a secure, but also very easy
way
•  Amazon Web Services (IT infrastructure) è Use Amazon‘s
gigantic data center in a ﬂexible, elastic, but also very cheap
way for your changing computation demands
•  Domino‘s Pizza (mobile enablement) è Order your next pizza
from your smartphone app (includes choosing menu, using
coupons, doing payment - via Paypal API for instance)

45
API Management at Paypal
hp://www.3bco.com/blog/2016/03/23/crea3ng-business-value-by-example-open-api-and-api-management-at-paypal/

End-to-End API Management
Create APIs
Technical Orientation
•  Create an API From Any Data
•  Compose API Integration Flows
•  Support for REST or SOAP
•  On-Prem, Cloud, or Hybrid
Productize APIs
Business Orientation
•  Portal for the API consumer
•  Transform APIs Into Products
•  Developer & Partner Mgt
•  Publishing and Versioning
•  Private Branding Supported
•  SLAs and Service Plans
Distribute APIs
Operations Orientation
•  Access Control/Security
•  Route/Transform/Throttle
•  SLA Enforcement
•  Operational Analytics
API Management

•  Developer self-service
•  API Lifecycle
•  API Mone3za3on
API Portal
•  Security & Access Control
•  Event Based Policy Mgt.
•  Federated Internet Scale
API Gateway
•  Repor3ng / Visualiza3on
•  SLA’s & KPI’s
•  Full Audi3ng
API Analy3cs
API Management
Cloud Based
Or
On-Premise

#4: Coordination across services
48

49
Coordination across services via NEW services ?!
Smart service, dumb pipe (no ESB in the middle)…

How to coordinate?

Coordination across Services
•  Apps / business services
are composed from
Microservices
•  Some Microservices can
be composed to
accelerate developments §  Graphical design and debug
§  Stateful or stateless
§  Service or event driven

#5: Managing complex deployments and
their scalability

52
Continuous Integration and Continuous Delivery
hp://en.wikipedia.org/wiki/Con3nuous_delivery
BENEFITS

•  Accelerated Time to Market
•  Building the Right Product
•  Improved Produc3vity and Eﬃciency
•  Reliable Releases
•  Improved Product Quality
•  Improved Customer Sa3sfac3on

COMBINED WITH “CLOUD”

•  Private / Public / Hybrid PaaS
•  Flexible Infrastructure
•  Elas3city


53
Frameworks and Products for Continuous Delivery and DevOps
•  Build Management
–  Ant, Maven, Gradle, …
•  Continuous Integration
–  Jenkins, Bamboo, …
•  Continuous Delivery
–  Chef, Puppet, Salt, …
•  Deployment (Elastic VMs / Cloud / Containers)
–  Amazon Web Services, Microsoft Azure
–  Openstack, CloudFoundry
–  Virtual Machines, Docker, Spring Boot

54
Continuous Delivery at Netﬂix
hp://www.oraclejavamagazine-digital.com/javamagazine/july_august_2016
“In today’s market,
companies need to
innovate con3nuously”

“Velocity becomes a key
requirement in soSware
engineering organiza3ons”

“Canary tes3ng rolls
out new features to a
small set of end users via
immutable deployments”

#6: Visibility and Correlation across Services

Opera3onal Intelligence Pla{orm
ü  Centralize and Store of Record
ü  Search, Auto-id, OOTB Parsing, Correla3on
ü  Forensics and Alerts
ü  Reports

Operational Intelligence PlatformNorthbound
Integra3on
-  Turn-key solu3on
-  All Microservices
-  No coding required Sensors
Applica3on Logs
Transac3ons
Monitoring
Conﬁgura3on
Messaging
Exis3ng Machine Data Sources
Southbound
Integra3on
Streaming Analy3cs
-  Filter forward to
upstream apps
-  Out of box analysis
tools
-  Visual Explora3on
Data Discovery
WEB UI WS API
Analysis
Tools

57
Alternatives for Log Management
Open Source Closed Source
SaaS
On Premise
(no complete list)

Real Time Correlation of Microservice Events
•  Events are analyzed and processed in
real-time as they arrive.
•  Decisions are timely, contextual, and
based on fresh data.
•  Decision latency is eliminated, resulting
in:
ü  Superior Customer Experience
ü  Operational Excellence
ü  Instant Awareness and Timely Decisions
Act &
Monitor
Analyze
Store

Data Monitoring
•  Motor temperature
•  Motor vibra3on
•  Current
•  Intake pressure
•  Intake temperature
Ø  Flow
Electrical power cable
Pump
Intake
Protector
ESP motor
Pump monitoring unit
Pump Components
Predic3ve Sensor Analy3cs

Voltage
Temperature
Vibration
Device
history
Temporal analytic: “If vibration spike is followed by temp spike then
voltage spike [within 12 minutes] then flag high severity alert.”
Event Processing (Correlation of Microservice Events)
… saving millions of dollars with predic6ve fault management!

61
Alternatives for Streaming Analytics
OPEN SOURCE CLOSED SOURCE
PRODUCT
FRAMEWORK
(no complete list!)
Azure MicrosoS
Stream Analy3cs

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes
only and its contents are subject to change without notice. © Copyright 2000-2014 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.

© Copyright 2000-2014 TIBCO Software Inc. 64
Microservices Architecture for the Live Demo
TIBCO ESB
Gateway
App
BW CE
App
BW CE
App
Cloud Foundry
BW6
Node App Spring App
Gateway
App
iPhone
App
.com
Web App
Android
App
Android
App
EMS
MongoDB
ROUTER
CONTROLLER
HEALTH MGR
NATS
LOGGING
SVC BROKER
BW CE
App
Java App
eFTL

65
Live Demo
Microservices with Cloud Foundry, TIBCO BusinessWorks Container Edi6on,
TIBCO Mashery and Papertrail…
•  Cloud (PaaS, Cloud Foundry, Microservices)
•  IDE (REST, APIs, Integra3on)
•  Administra3on (Deployment, Versioning)
•  Opera3ons (Failover, Scalability, Log Management)
•  Log Management (Distributed Search and Correla3on)
•  Open API (Packaging, Discovery)

67
Necessary Rules and Guidelines
Stefan Tilkov, h@ps://speakerdeck.com/s2lkov/microservices-talk-berlin

68
Avoid a zoo of technologies and frameworks!
Java JavaScript
Java Process
(JAR File)
Java EE
App Server
Node.js
Server
.NET
Pla{orm
C# + F#
Scala + Groovy
Framework 1
Framework 2
Framework X
No Framework
Tool 1
Tool 2
No Tool
Tool X
Middleware
Pla{orm

69
DevOps / Continuous Delivery
Theory
Every team is responsible for development, test, deployment and operations.
Therefore, technology and tool choice do not matter.
Practice
People and intellectual property leave companies.
An enterprise strategy exists to reduce risks and costs.
The team cannot control everything (e.g. when using Open API or SaaS services).

70
Microservices is a lot of effort!
•  Significant operations overhead
•  Substantial DevOps skills required
•  Implicit interfaces
•  Duplication of effort
•  Distributed system complexity
•  Asynchronicity is difficult
•  Testability Challenges
hp://highscalability.com/blog/2016/4/8/microservices-not-a-free-lunch.html
”[…] when considering Microservice like architectures, it's really important to not be
aracted to the hype on this one as the challenges and costs are as real as the benefits."

Did you get the Key Message?

Questions?
Kai Wähner
kwaehner@tibco.com
@KaiWaehner
www.kai-waehner.de
Xing / LinkedIn à Please connect!

Microservices - Death of the Enterprise Service Bus (ESB)? (Update 2016)

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to Microservices - Death of the Enterprise Service Bus (ESB)? (Update 2016) (20)

More from Kai Wähner (20)

Recently uploaded (20)

Microservices - Death of the Enterprise Service Bus (ESB)? (Update 2016)