MeasureWorks - Windesheim Almere - Why Performance matters?

@jeroentjepkema
Windesheim Almere
Why Performance Matters?

Performance? Why should we care?

0
25
50
75
100
1 2 3 4 5
Slow Average Fast
%ofrespondentsthatratedtheactualwebsitespeed
Design score (1=bad - 5=beautiful)

When users experienced a slow
website, 60% of them rated
design lower

(bad) Performance triggers emotions

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour/day
Performance complaints patterns per branche/day
Retail100 Travel30
Research from MeasureWork & Social and More. Twitter mentions collected for Twinkle100
webshops between July 1 2014 - July 1 2015
Average:
179 tweets p/hour

21% of social complaints are about
user experience

30 33 36 39 42 45 48 51 2 5 8 11 14 17 20 23 26
Week/Year (2014-2015)
Performance complaints pattern per branche/day
Retail100 Travel30
Normal average:
2398 tweets p/day

30 33 36 39 42 45 48 51 2 5 8 11 14 17 20 23 26
Week/Year (2014-2015)
Retail100 Travel30
Normal average:
2398 tweets p/day
Holiday average:
2897 tweets p/day

18% increase in
performance complaints
during holiday season

30 33 36 39 42 45 48 51 2 5 8 11 14 17 20 23 26
Week/Year (2014-2015)
Retail100 Travel30
Holiday average:
2897 tweets p/day
Complaint top 3:
1. Downtime (36%)
2. Mobile readiness (21%)
3. Slow (19%)
Complaint top 3:
1. Downtime (39%)
2. Slow (29%)
3. Mobile readiness (11%)

0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title
Real User Monitoring: True Conversion Rate
Data collected by MeasureWorks at various e-commerce websites using real user monitoring, reference: http://bit.ly/MW-VEUrum
0%
10%
Competitive Custom B2B

0%
10%
Optimal Conversion: 1,8s
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title

0%
10%
LD50: 4,3s
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title

0%
10%
LD50: 4,3s
Poverty Line: 7,6s
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Conversionrate(%)
Category Title

On average a fast experience
converts up to 70% higher...

...this is a familiar sight ;-(

Presented by MeasureWorks & Philips at ShoppingToday 2013, reference: http://bit.ly/MW-ST2013

“You’re more likely to miss stuff just
because it takes a long time to scroll
down the page”
~ User 56A on the MeetHue.com website

“Your getting blamed for
things that are not your fault”

the WAITING experience
Source: Jakob Nielsen

0 0,5
Visual
conﬁrmation

0 0,5 1,5
Visual
conﬁrmation
Understand
Navigation

0 0,5 1,5 3
Visual
conﬁrmation
Understand
Navigation
Relevant
Content?

0 0,5 1,5 3 10
Visual
conﬁrmation
Understand
Navigation
Relevant
Content?
Bye
Bye!

“Both ofﬂine and online consumers
associate slow performance with
poor customer service”

Click away slidePerformance tolerance?

0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8 9 10
Bouncerate per pagetype/session
Bouncerate(%)
Page load time (sec.)
Median Campaign Product search

0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8 9 10
Bouncerate(%)
Page load time (sec.)
Median Campaign Product search
Bouncerate per pagetype/session

Queuing for iPhone6 launch - London 2014
Purpose vs. Context

Online users often lack context for delays...

...and see no other option than to click away

For many, Performance is like the gym...

There’s always an excuse ;-(

Three simple performance checks

actionable,
non-technical,
End User Focused

55
go to http://webpagetest.org

56
www.example.com
select location
select browser
Set connection speed
Create video

1: Conﬁgure test
2: Key Optimizations
3: Performance metrics
Use Akamai State of the Internet quarterly reports to select right bandwith per country: http://www.akamai.com/stateoftheinternet/

58
http://www.digitalmarketinglive.nl

0
5.0
10.0
15.0
20.0
IE10 Chrome Firefox Safari iOS Android
Page load time per browser type (sec.)
Backend First Paint Page Load time
Tested with webpagetest.org (WiFi (10Mbs down/1,5Mbs up), 4G (5Mbps Down, 1Mbps Up) & 3G (2000Kbps Down, 764Kbps Up))

>

Performance inﬂuence cycle
Search/Orientation phase
Perception of experience
1
http://benchmark.measureworks.nl

Delivered
experience
Your website
3
5
2
Stimulate content/
conversion
4
Performance inﬂuence cycle
Search/Orientation phase
Perception of experience
1
http://benchmark.measureworks.nl

9.76 sec. 4.75 sec. 7.09 sec. 2.31 sec.

Or the easy way: http://benchmark.measureworks.nl

http://whatismyscreenresolution.com

Do you trust your third parties?

4.3 sec
22.1 sec
Source: webpagetest.org

Source: requestmap.webperf.tools

3 types of data:
Which pages are clicked on?
How fast is it on every device?
Which elements do they click on?

1
Insert tag (.js ﬁle)
into (mobile) web
pages
Pages are requested
from browser/device
As pages execute,
tag collects
performance metrics
After onload tag
send detailed report
for further analysis
tag.js
tag.js
tag.js
tag.js
2 3 4

Relies on navigation timing API, custom variables can be added
Turn on
“Site Speed”
in your GA
account!
http://bit.ly/1ivGWTt

Or try
Soasta “Lite”
http://bit.ly/I5ngs8

Why is (nigerian) spam so badly
written?

Aunshul Rege of Rutgers University, USA in 2009
1000 emails
1-2 responses
1 fool and their money, parted.
Bad language (0.1% conversion)
Gullible (70% conversion)
1000 emails
100 responses
1 fool and their money, parted.
Good language (10% conversion)
Not-gullible (.07% conversion)

Nigerian spammers
really understand their target
market and ‘customers’

Your Analytics measures the
movement towards these goals

Complete Web Monitoring
Web Analytics
(what did they
do on the site?)

Web Analytics
(what did they
do on the site?)
Usability
(how did they
interact with it?)

Web Analytics
(what did they
do on the site?)
Performance
(could they do
what they wanted
to?)
Usability
(how did they
interact with it?)

Web Analytics
(what did they
do on the site?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)

Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)

Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)
Social Media
(what were they
saying?)

Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do
what they wanted
to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)
Social Media
(what were they
saying?)
“Soft” data
“Hard” data

X
A simple online business model:
Marketing Conversion Optimization Revenue
New
visitors
Bounce
rate
Conversion
rate
Order
value
Growth
Loss
Time on
site
Pages per
visit
Number of
visits
Search
Tweets
Mentions
ADs seen

Checkout
Visits
Shopping cart
Payment options

Checkout
Visits
Shopping cart
Payment options
Unpaid search Social mentions Email
campaign Banner ads
Generate trafﬁc to website

Don’t trust your data, analyse it

9 followers
9 reach
4 followers
16 reach

Just imagine if data tells you how to
inﬂuence your visitors?

...inﬂuence your followers...

Conversions
Visits
Shopping cart
Payment options
Impressions
Spread
Seed
Higher likelihood
of conversion
Generate trafﬁc to website

Importance of
correlation vs. causation

1
10
100
1000
10000
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Relation between Ice cream and drownings?
Ice cream consumption Drownings

http://www.ﬂickr.com/photos/25159787@N07/3766111564

http://www.ﬂickr.com/photos/25159787@N07/3766111564
1
10
100
1000
10000
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
True causality
Ice cream consumption Drownings Temperature

A leading, causal, connection is
the analytics superpower...

That’s why we pick one at at time...

‣ A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
‣ A Dropbox user who puts at least one ﬁle in one folder on one
device (ChenLi Wang)
‣ Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh
Elman)
‣ A LinkedIn user getting to X connections in Y days (Elliot
Schmukler)
Some examples
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)

A wealth of information creates a
poverty of attention...
(Computers, Communications and the Public Interest, pages 40-41,
Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert
Simon

Focus on the desired behavior, not
just the information.
http://www.psychologytoday.com/blog/yes/200808/changing-minds-and-changing-towels
26% increase in towel re-use with an appeal to social
norms; 33% increase when tied to the speciﬁc room.

Validating experiments:
B.R.A.S.S. framework

Blink Relevance Availability Scalability Sum

5-second rule
Gut feeling
Instinct score
Don’t overthink it

5-second rule
Gut feeling
Instinct score
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?

5-second rule
Gut feeling
Instinct score
Is this Channel
relevant to your
product/service?
Does your
audience hang out
on this channel?
Can you target
them effectively?
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?

5-second rule
Gut feeling
Instinct score
Is this Channel
relevant to your
product/service?
Does your
audience hang out
on this channel?
Can you target
them effectively?
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?
How scalable is
this channel?
Can we easily
increase it?
Law of diminishing
returns

5-second rule
Gut feeling
Instinct score
Is this Channel
relevant to your
product/service?
Does your
audience hang out
on this channel?
Can you target
them effectively?
Do we have the
available
resources?
Do we have the
skills?
Do we have the
tools?
Is it expensive?
How scalable is
this channel?
Can we easily
increase it?
Law of diminishing
returns
1-5 1-5 1-5 1-5 BRASS score

The Skip: http://musicmachinery.com/2014/05/02/the-skip/

“Hit them with a beat! The decision to continue
listening to a song is often determined in the ﬁrst 10
seconds” - Dr. Dre, Source Magazine
The Skip: http://musicmachinery.com/2014/05/02/the-skip/

Current state of responsive web ;-(

WiFi 3G

6 must haves
for happy customers (again)...

http://www.nytimes.com/2012/08/19/opinion/sunday/why-waiting-in-line-is-torture.html?pagewanted=all&_r=0

http://answers.yahoo.com/question/index?qid=1005081200005

nytimes.com/2004/02/27/nyregion/27BUTT.html

Which is faster?

“The faster you can visualize your content,
the more engaged your visitors will become”

2.
Understand & Prioritize
your feature content

A: Load carousel ﬁrst B: Delayed download
20% time spent on priority content 1% time spent on priority content
http://www.nngroup.com/articles/website-response-times/

Yet, 83% of all Twinkle100 sites
loads menu before feature
content...

“The faster you can
paint your feature content,
the more engaged
your visitors will become”

“Provide your users the time to
understand navigation as they often
overlook things that change too fast and/or
often”

Homepage Search
Product
overview
Product
Detail
Basket

Homepage Search
Product
overview
Product
Detail
Basket
Understand
navigation
(1-2 sec.)
Consistent
navigation
(0,5-1 sec.)
Renew your
orientation
(1-2 sec.)
renew
your navigation
(1-2 sec.)
Consistent
navigation
(0,5-1 sec.)

Consistent, simple &
user generated navigation

Use speed to increase perceived value

Homepage Search
Product
overview
Product
Detail
Payment
Tolerated
speed

Fast response = Fast Experience?

The kayak effect: http://bit.ly/UgTneD
People prefer to wait for up
to a minute to get what they
want from an app rather
than get it instantly – if, and
it’s an important if, they
believe the app is working
for them

http://www.fastcodesign.com/1669788/the-3-white-lies-behind-instagrams-lightning-speed

http://blog.placeit.net/ux-tactics-make-slow-things-seem-faster/

Show effort,
provide speciﬁc content,
build trust

3.
Set your performance budget

Setting a performance budget:
A pre-deﬁned set of metrics
that describe normal behavior
in order to detect variances
and to be comparable in historical context

Perception
Actionable
Flow
TtFB
Page Size
Time to ﬁrst Byte 0.0 - 0.2 sec.
Speed Index 0.5 - 1 sec.
Document complete: 1-3 sec
Variation < 20% between pages in funnel
Page size < 1Mb (75 requests)
Example performance budget...

A practical (and easy DIY) example:

Service Window
Purchasing a bed,
must be completed (speed),
where every page loads under 3 sec.,
from any location in the Netherlands,
for 95% of all users,
every 5 minutes between 6am and 12pm,
using IE9 and higher,
Customer journey
Metric: Speed
Target: Sec
User scenario
User locations
Percentile
measured with Synthetic Monitoring. Monitoring type
Business
Performance

4.
How fast is your (front-end) code?

Anatomy of a
web page
Anatomy of a web page

A set of resources to be fetched from a server...
The browser renders the page...
This results in the page being displayed on your
screen
1
2
3

Introducing the waterfall chart

See also: 90 minute optimization cycle, Strangeloop, Velocity 2012

HTML

HTML
Resources

HTML
Resources
Start
Render

HTML
Resources
Start
Render
Document
Complete

Waterfall technique
a.k.a. show me where it hurts

Code Green:
Time to ﬁrst byte?

Code Orange:
Too many connections

106 Connections
(almost one per request)

Code Blue:
Too many bytes (page bloat)

Why is this important?
85-90% of all slowdowns are
caused by your front-end code...

5.
Fast enough?
Then test for peak trafﬁc

Load/Performance testing:
Measure performance under both normal and
peak volumes to discover how your application
behaves under stress

Find your ceilings in time to ﬁx them...
The End
Current state

0
250
500
750
1000
January-14 April-14 July-14 October-14 January-15 April-15 July-15 October-15
#Pageviews(x1000)
Forecast pageviews Actual pageviews
Trafﬁc realized
Trafﬁc forecast
Max infra capacity (90%)
Safety infra capacity (80%)
# pageviews at peak
hour of peak day per
week/month
Build your spreadsheet...

Your marketing campaign must never be a
suprise for your site...

6.
Don’t forget?
Optimize for Mobile

Make life easy, insert touch events...

It’s a process. Integrate the
performance budget in your
continuous delivery cycle

Analyze the performance as your
users experience it and
compare against your budget to
identify what areas need
improvement
Build faster websites
and optimize the
rendering of your
application and the
perception of speed
Automate & Update
performance testing and
integrate into your delivery
processes
Repeat
Measure Build Run

Performance is a team sport...

Developer
Makes it real,
builds &
deploys code.
Marketer
BizDev,
validates
customers
demand and
continuously
generates new
ideas
Designer
Designs
compelling
customer
experience
with
performance
in mind
Analyst
Tracks
performance
budget, drives
conversion
experiments
and keeps
you honest.

...Use performance as a business
metrics to make it stick

Performance = Reducing Complexity

Frictionless = Recognizable navigation

You owe it to your customers...

Thanks! More questions?
M: jeroen@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl
View slides:
bit.ly/MW-WHA15

MeasureWorks - Windesheim Almere - Why Performance matters?

More Related Content

What's hot (20)

Viewers also liked (18)

Similar to MeasureWorks - Windesheim Almere - Why Performance matters? (20)

More from MeasureWorks (18)

Recently uploaded (20)

MeasureWorks - Windesheim Almere - Why Performance matters?