Ready for Fast Data: How Lightbend Enables Teams To Build Real-Time, Streaming Data Applications

Ready for Fast Data
How Lightbend Enables Teams to Build Real-Time, Streaming Data
Applications

2
The next frontier of competitive
advantage
is the speedwith which you can
extract value from data.”
MIKE GUALTIERI
VP, Principal Analyst Forrester Research, Inc
“

Predictive Analytics Real-time Personalization
Real-Time Financial Processes Internet of Things
Powering Innovation
with Fast Data
Lightbend provides a
platform for building,
running and managing
large-scale, real-time,
streaming applications

Introducing Lightbend Fast Data Platform
5

Your Path To Fast Data
6
Data Needs to
go Fast
Fast App Needs
Data
Spark -
Batch to
Streaming
Micro-
services
with Akka
Streams
True Fast
Data

The Talent “Accordion”
• Data Scientist
• Data Engineer
• Create & manage data “plumbing”
• Spark / Flink / Kafka
• App Developer
• Reactive Microservice Development
• Operator / DevOps
• Size and Deploy Infrastructure
• Configure Monitoring and Management
7
• Data Scientists
• Microservice Developers
• Kafka Developers
• K8s Specialists
• Streaming Engine Developers / Data
Engineers
• IT Operators - Management and Monitoring
• Data Custodians (ie DBA)

Starting with a Framework - the Skills Matrix
9

Rankings 0 thru 5
You don't know
this item at all,
other than
perhaps having
heard it's name
in passing
No
Knowledge
0
Attended a
course or read a
book, but you
haven’t actually
tried it out
Theoretical
Knowledge
1
You have
experimented or
done a learning
project, but
haven't put it to
use in a
production
environment
Junior
2
You have
completed a
course or an
extensive
sample project,
but haven't yet
used it in
production
Apprentice
3
You have used this
technology on a
real project that
was deployed and
used in production,
but are not yet
completely
comfortable with it
Intermediate
4
Used this in
production at least
once, increasing
comfort with the
more advanced
aspects of the
technology or
technique
Intermediate
Plus
5

Rankings 6 thru 10
You have used the
technology or
technique to the
point where it's
comfortable and
well-understood,
likely with multiple
uses in production
or with real clients
Advanced
6
You are the expert
within your team
in the given topic,
others frequently
ask for guidance
on this topic
Expert
Knowledge
7
Someone the
community looks to
for answers, has
written blog posts,
presented at meetups
or conferences on
this technology or
technique
Noted
Expert
8
One of the foremost
in the business, you
are known outside
your company as a
leading expert in this
technology or
technique.
Industry
Expert
9
You are a committer
for or the originator
of this item, and/or
you have written one
or more books about
it, for instance. One of
the leading experts in
the field on this tech
Industry-Leading
Expert
10

It is not possible to have a
ranking greater than
Intermediate without having
used the skill in a real project
Experience is Key
A team that needs a skill
(importance 1, 2 or 3) must
have at least one team member
at Advanced or greater,
anything less is a high risk of
failure
When a person declares a skill
level of Intermediate or higher,
note during the evaluation the
project that provided this
experience - this serves as a
cross-check for the person to
consider their ranking more
carefully
Production Use Target is Advanced Note Experience

13
I understand the theory
I can see how this works
I can build in this technology
I can predict future failure
points
Books,tutorials, self-paced
learning
Mentor and Instructor supported exercises
and sample projects
Initial “real” project supported by experts
Production deployment
Building the Ladder

14
I understand the theory
I can see how this works
I can build in this technology
I can predict future failure
points
Books,tutorials, self-paced
learning
Mentor and Instructor supported exercises
and sample projects
Initial “real” project supported by experts
Production deployment
Building a Skills Ladder
Theoretical Knowledge (0-1)
Practice (2-3)
Implementation (4-5)
Production Expertise (6-7)

Team Sizes
PoC / Science Project
(1-4 people)
Context
Initial team, usually led by an
expert visionary.
Enablement Strategy
• Individual learning supported
by key formal courses
• Support and consulting as
required
Business Critical
Project
(5-250 people)
Context
Fully staffed team having
achieved production success,
with additional teams
onboarding over time
Enablement Strategy
• Consistent cohort-based
training
• Academy Pattern
• Layers of support
Enterprise Adoption
(More than 250 people)
Context
Corporate initiative towards
digital transformation with Fast
Data
Enablement Strategy
• Mentor-supported self-led
training programs
• Formalized Apprenticeship
• Robust support strategy
• Center of Excellence

Foundational Elements
• Lightbend Subscription (http://lightbend.com/subscription)
• Embedded Lightbend Engineer (http://lightbend.com/ELE)
• Reactive Project Blueprint
17

“PoC Project” Enablement Strategy
• Do things that “don’t scale” - it’s not worth over formalizing the learning strategy at the
outset
• Take advantage of books, online and in-person courses, etc…
• There’s a lot of advantage in trying things out and seeing how they work, but since things
operate very differently at scale than they do in the lab in a Fast Data environment, it’s
worthwhile speaking to people who have done this before
• Core goal of a PoC should be to get the small team skilled up to the point where they can
form the seed of the next set of teams
18

“Business Critical Project” Enablement Strategy
• Structure and repeatability are critical
• Create an enablement plan
• Run any scheduled events periodically on an appropriate cadence (such as in-person or
live instructor remote training)
• Ensure that participants follow the enablement strategy with real project work as soon
after the theoretical knowledge steps
• Plan for both new projects and new entrants
• Supplement new teams with appropriate levels of expert support according to the
Academy Pattern
• Build for internal sustainability
19

Sample Enablement Strategy
20
Mentor Supported Practice
Structured sample project with mentor
support
Mentors can come from Lightbend or your
organization
2
Theoretical Knowledge
Periodic live courses
Curated list of self-study assets
1
Lead Project
Share the love by working in a project as
the expert and mentor others, or spin out
a new project
4
Initial Project
Work in a project with a strongly enabled
team - either Lightbend Reactive Launch
or existing successful project
3

Reactive / Fast Data Launch
Rapidly onboard your
team. Expert review of
your system architecture
and design, with
recommendations for next
steps. Establish clear goals
and priorities to kick off
the Reactive Launch
Report and
Recommendations.
Tackle important
technology Spikes to prove
out your assumptions,
de-risk your delivery
pipeline (steel thread) and
prove key business cases.
With sustained focus on
continuous
improvement,
progressively build the
platform while
addressing capacity gaps
with one-on-one
mentorship, facilitation
and coaching on Fast
Data Platform delivery.
This touchpoint week
(scheduled up to one
month later) will ensure
that your team continues
on the right path, offering
training, one-on-one
mentorship, facilitation and
coaching as required.
Closing out the Reactive
Launch report and
recommendations.
WEEK 1
INCEPTION
WEEKS 2 - 3
STEEL THREAD
WEEKS 4-7
CONSTRUCTION
WEEK 8
Review/Recommend

Academy Pattern
22
Return to Project
New Project regains their
people ready to proceed with
the new project
Initial Project
First Project uses a Reactive
Launch methodology to
achieve success
Loaned Personnel
Other projects looking to develop talent
loan people to the initial project
Academy Pattern

“Enterprise Adoption” Enablement Strategy
• Used by major consultancies, usually to train 10’s of thousands in order to offer new
capabilities to market
• Strategies have been “battle tested” and the experience is useful for enterprises looking
for similar levels of enablement
• Scale necessitates participant-led, opt-in and highly structured programs
23
Continuous
Built in ongoing learning
programs that integrate with
career planning including
certification
Theoretical
Mentor-supported self-led online
videos, exercises, quizzes
Structured through internal
learning management system
Practical
Robust set of ongoing
projects with apprenticeship
programs, all supported by
deployable Center of
Excellence

Theoretical Knowledge Assets
• Lightbend Fast Data Platform (http://lightbend.com/fast-data-platform)
• Lightbend Reactive Architecture (http://lightbend.com/LRA)
• Fast Data Architectures for Streaming Applications (http://lightbend.com/ebooks)
• Lightbend Scala Language - Professional (http://lightbend.com/services/training)
• Lightbend Akka for Scala - Professional (http://lightbend.com/services/training)
• Lightbend Apache Spark for Scala - Professional (http://lightbend.com/services/training)
• Lightbend Akka Streams for Scala - Professional (http://lightbend.com/services/training)
• Simplifying Data Pipelines (Kafka)
(https://cognitiveclass.ai/courses/simplifyingdatapipelines/)
• Lightbend Reactive Operations - Deployment with Kubernetes (Coming soon)
24

Ready for Fast Data: How Lightbend Enables Teams To Build Real-Time, Streaming Data Applications

More Related Content

What's hot (20)

Similar to Ready for Fast Data: How Lightbend Enables Teams To Build Real-Time, Streaming Data Applications (20)

More from Lightbend (20)

Recently uploaded (20)

Ready for Fast Data: How Lightbend Enables Teams To Build Real-Time, Streaming Data Applications