Complexity, Components & Clouds (Paremus)

Complexity, Components &
Clouds

Dr R Nicholson
FITE club - April 2010
www.paremus.com
FITE Club (London) www.paremus.com April 2010
Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.

FITE Club?

“Fight Club is a metaphor for the need to push through
the walls we put around ourselves and just go for it”

“...probe the frustrations of the people that live in the
system...”


What do we mean by Complexity?


Complexity

Information !"#"!$"%"&

A suitable Representation

Complexity
(m)
Volume = V
(x,x’)
(z,z’)

(y,y’)

pressure = P

Temperature = T

Information !"1(m,x,y,z,x',y',z') Information !"#$#%&

A suitable Abstraction

Accidental Complexity

• Redundant Information
• Same System may be fully described in a more succinct
manner
• Ability to easily re-factor enables Accidental Complexity
to be driven out over time


Necessary Complexity

When all accidental complexity is stripped away - we are left with
necessary complexity...
•The actual ‘structure’ of the System
•Adaptive Systems are by necessity more complex than non-
adaptive Systems

Strategies for dealing with necessary complexity?
•Abstraction
•Perhaps a change Perspective / Representation


Abstraction & Virtualisation

Virtualisation - Emulates the characteristics of an entity
Abstraction - Masks the complexity of the entity (physical or virtual), providing a simplified
description
Abstraction and Virtualisation are orthogonal & complementary concerns
• As system complexity increases - stop attempting a microscopic description use emergent
macroscopic characteristics
• Boundaries - Where/when does it makes sense to change point of view?
• Several such boundaries may exist!
• Manage via emergent macroscopic properties and operational complexity should
dramatically reduce
• Conversely virtualisation ‘strategies’ never decrease operational complexity

Source - Richard Nicholson - Paremus
https://adaptevolve.paremus.com/?p=18

Components: Why?


Components: The need to Abstract!

as change
isolate change
occurs

modules and their
dependencies

Source - Kirk Knoernschild (Burton Group)
http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/


Components: Something is Missing?

? Reuse Release Equivalence:
Unit of reuse is the unit of release!



Components: Something is Missing?

Reuse Release Equivalence:
Unit of reuse is the unit of release!



Components: Benefits

- reus
e
- re du
ce co m
- ease plexity
mainte
- incr nance
ease
exten
sibility

Reduces Complexity HOW?
• OSGi Modularity - enables rapid ongoing code
refactoring - essential for driving out
• OSGi Modularity - provides the required Source - Kirk Knoernschild (Burton Group)
abstraction, masking Necessary Complexity

Components: Benefits

- reus
- re du
e Increases architectural agility!
ce co m
- ease plexity
mainte
- incr nance
ease
exten
sibility

Reduces Complexity HOW?
• OSGi Modularity - enables rapid ongoing code
refactoring - essential for driving out
• OSGi Modularity - provides the required Source - Kirk Knoernschild (Burton Group)
abstraction, masking Necessary Complexity

Components: Modularity is not a new story

“First they ignore you, then they
ridicule you, then they ﬁght you,
then you win.”
-- Mahatma Gandhi



Components: Modularity is not a new story

“First they ignore you, then they
ridicule you, then they ﬁght you,
then you win.”
-- Mahatma Gandhi

OSGi is a disruptive technology that will transform how
enterprise Java applications are designed, developed,
and managed!


Components: Business Driver

90% of application lifetime cost concerns maintaining and
evolving the Production Systems

Modularity is a MUST


Clouds


Cloud: The bandwagon is rolling!

What are the critical
characteristics required of any
credible Cloud solution?


Cloud 1. An Elastic Resource Landscape

• All resource is transient
• Resource landscape will expand / contract over time
• Resource characteristics will change over time


Cloud 2. Simplicity through Abstraction

• Manage Populations - not Instances
• Manage Business Systems - not Components


Cloud 3. Robust & Recovery Oriented

A system accident is an "unanticipated interaction of
multiple failures" in a complex system. This complexity can
either be technological or organizational, and often has
elements of both.
(Normal Accidents - Perrow 1984)

Complex Systems tend to suffer correlated
& cascading failures!


Cloud 3. Robust & Recovery Oriented

A system accident is an "unanticipated interaction of

•
multiple failures" in a complex system. This complexity can
Agile Systems - Are Robust Systems
either be technological or organizational, and often has
•
Robust Systems - Are Agile Systems
elements of both.
• Self-Forming / Auto-Repairing (Normal Accidents - Perrow 1984)

• Optimised for MTTR not MTBF
• No single point oftend to suffer correlated
failure (SPoF)
• No single point of
Complex Systems reference (SFoR)
& cascading failures!


Cloud 4. Must understand Composite Systems

Deployment via VM images
NOT GOOD ENOUGH

Remember - Reuse Release Equivalence:
Unit of Release is the unit of Reuse!


Cloud 5. Is Multi-Tenant

Isolation / Visibility - control at all levels of structural hierarchy /
abstraction:

• Groups of Systems
• Groups of Composites within a System


Cloud 6. Is Architecturally Invisible

The Cloud runtime while providing options should not constrain or
enforce:
•Communication protocol
•Middleware Services
•Data Services (CAP)
•Caching Services (CAP)
•Or type of development framework


Cloud 7. Support Adaptive / Reactive Applications

Dynamically assemble, and if necessary re-assemble, in response to
changes in operational / business environment:
•Rapid patch / fix and roll back capabilities
•In future?
- Dynamic component re-wiring to account for co-locality
- Policy based roll-back based on System behaviour
- Self-Tuning / SLA aware Service assembly behaviours
Deployment via VM images
IS DEFINITELY NOT GOOD ENOUGH


The Paremus Service Fabric


The Paremus Service Fabric

SaaS
(1..m) “Systems” may run upon a
single Service Fabric

IaaS (1..n) “private Resource Clouds”
may contribute to a Service Fabric


Atlas Resource Management

‘Target State’ driven resource acquisition Service Fabric ! Production
"#"Cloud ! Blue$
"&"Cloud ! Red$"os%name!Darwin$$$

• Ongoing discovery
• To join a Service Fabric - Atlas agent must have
appropriate X509 certificate
• Only resource with correct characteristics are
assimilated into Service Fabric
• Resource may leave at any point in time
Create a “Red” & “Blue” Service Fabric


Service Fabric - Infrastructure Services
Registry
Provisioner Service Advertisements
System Managers

Principles:
• No ‘special’ nodes
• Node ‘Roles’ may/do change over time
C A
Techniques: B
• Dynamic Group formation & subsequent membership
• Dynamic leadership election
• Eventual Consistency across group members
Repository
Management & OSGi bundles
System Descriptions
Monitoring
Nimble Policies
WAR
EAR
General artifacts


Service Fabric - A Model-Driven Runtime
A model of the desired business service is submitted. The Service Fabric dynamically
deploys all required business and infrastructure components.

System
Running System
Description

Abstraction Corresponding
‘Form’ runtime Entity


Service Fabric - Model-Driven Management

The Service Fabric continually compares the runtime structure with its corresponding model

Target State Runtime State

=1
=3
=1

Deploy




Provision Delta

Planned Deltas
=1 e.g. Conﬁguration
M!del changes
=3 Runtime
Target State
Entity
"Structure#SLA$ Unplanned Deltas
=1 e.g. Resource
failures

Bundle
Bundle
Bundle
Bundle

Bundle
Bundle Bundle Bundle

Bundle Bundle
Bundle Bundle
Bundle

Bundle

Bundle Bundle
Bundle

Monitor Bundle




Provision Delta

Planned Deltas
M!del changes
=3 Runtime
Target State
Entity
"Structure#SLA$ Unplanned Deltas
=1 e.g. Resource
failures

Necessary Complexity
Bundle
Bundle

Bundle
Bundle

Bundle Bundle
Bundle

Bundle

masked by ‘Nimble’
Bundle Bundle
Bundle Bundle
Bundle

Bundle

Bundle Bundle
Bundle

Monitor Bundle



To change a complex runtime System simply change its model!


=1
=5
=1

Re-Conﬁgure



To change a complex runtime System simply change its model!

Provision Delta

Planned Deltas
changes
M!del
=5 Target State
Runtime
Unplanned Deltas
Entity
"Structure#SLA$
=1 e.g. Resource
failures

Monitor


Anatomy of a Business System

System Target Scaling Behaviour
SCA Structure Resource Contract
State (Replication Handlers)

= (os.name=linux) & (CPU.speed > 3 Ghz)

=
= fl(x)

+ = fm(y)
= fn(z)
+ = !(os.name=Windows)

= (cost_center=engineering)

Composites & Scale behaviour Resources
Bindings required


Dynamic Scaling in response to Load

Store

input stream a-e
CEP Processing
u-z

Store
input stream
a-b
CEP Processing
y-z


Scoping & Visibility

System "
$Group A%

System !
$Group A%
System #

Level Description

A service at composite visibility is only bound to
Composite references within the same composite

A service at system visibility is visible to any
System reference in a composite in the same system

A service at group visibility is visible to any reference
Group within composites in systems in the same group

A service at fabric visibility is visible to all references
Fabric within the fabric

A service at fabric visibility is visible to all references
Public within the fabric


Development

The Service Fabric does not mandate any particular IoC
framework. A System may consists of any mix of:

• Java POJO’s
• OSGi Declarative Services
• OSGi Blueprint
• iPojo
• Spring and Spring DM
• Google Guice / Peaberry
• Scala (or any other language that targets the JVM)


System Architecture - Communication

Service Fabric does not enforce any particular architectural
‘form’

• Messaging middleware / protocols are the concerns of each System; via
choice of bindings and infrastructure components
• SEDA, ESB, Broker, P2P patterns are all possible
• Low latency, batch grid, order matching & transactional message based
Systems may be concurrently supported on the same Fabric


System Architecture - Data / Caching

Service Fabric does not enforce an particular data model.
A true No-SQL Platform as a Service (PaaS):

• Unstructured data processing - Hadoop
• Key / Value - Voldemort
• Column - Cassandra
• Graph Database - Neo
• Relational - Derby, MySQL


A self-configuring PaaS
Service Fabric - ‘I need
to deploy a Servlet containers
for this WAR based service!’
EAR

EAR
war

Avoid middleware lock-in. Avoid bending you business systems to the dictates of monolithic middleware.
The Paremus Service Fabric is a PaaS runtime that dynamically assembles and
configures itself around the requirements of each business service it hosts

What to know more?

• Talk to Paremus.
• on OSGi? Join UK OSGi Forum - http://uk.osgiusers.org/Main/HomePage
• on OSGi? Buy the Book - http://www.manning.com/hall/


Thank You

www.paremus.com/servicefabric

www.paremus.com/nimble

http://felix.apache.org/site/apache-felix-sigil.html


Complexity, Components & Clouds (Paremus)

More Related Content

More from mfrancis (20)

Recently uploaded (20)

Complexity, Components & Clouds (Paremus)