Synthetic and rum webinar
0 likes891 views
The document discusses the differences between Real User Measurement (RUM) and Synthetic Monitoring, emphasizing their roles in web performance analysis. RUM collects metrics from actual user interactions while synthetic monitoring uses automated agents to simulate user behavior. The content highlights the benefits and challenges of both methods, advocating for their combined use to gain a comprehensive understanding of web performance.
Synthetic and rum webinar
- 1. © 2014 SOASTA. All rights reserved. October 23, 2014 ‹#› CONFIDENTIAL – Not for Distribution Synthetic and RUM A Recipe for Webperf Success
- 3. A C T I V E M O N I T O R I N G • Simulated health-checks of the system and it’s parts • Scripted journeys through an application • “Creation” of a singular or set of events and/or actions • Synthetic is a form of active monitoring AP Photo/ Eric Risberg
- 4. PA S S I V E M O N I T O R I N G • Listening to actual traffic as it moves through a system • Observing behavior of the system and/or its operator • “Measurement” of actions or events • RUM is a form of passive monitoring
- 5. Our focus is on the experience of the crowd allhumorpic.com
- 6. What is RUM? Real User Measurement (traditional definition) RUM 101 Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user. • Involves instrumentation of your website via JavaScript • Measurements are fired across the network to a collection point through a small request object (beacon) <JS> <beacon>
- 7. T H E G O O D PA R T S - R U M • Always on • Every user, every browser, every network, anywhere in the world • Ability to capture human behavior/events with performance data • Only getting better (waterfalls)
- 8. What is RUM? Real User Measurement (traditional definition) Synthetic 101 Synthetic monitoring (for purposes of this discussion) refers to the use of automated agents (bots) to measure your website from different physical locations. • A set ‘path’ or URL is defined • Tests are run either adhoc or scheduled and data is collected
- 9. T H E G O O D PA R T S - S Y N T H E T I C • Rich data collected (waterfall, videos/filmstrip, HTTP Headers) • Consistent “clean room” baseline • Nothing to install • Doesn’t require users/ability to measure pre- production, competition
- 10. C O M M O N T H I N G S W E H E A R A B O U T B O T H R U M A N D S Y N T H E T I C • “Why are these numbers so different?” • “I don’t trust your data. Your numbers are wrong.” • “How are you calculating Page Load time?” • “I can’t share two sets of numbers with the business.”
- 11. 3.23s – Median 10.45s – p95 17.26s – p98 R E A L U S E R S A R E N O T N O R M A L
- 12. OS Median 95th Percentile 98th Percentile Windows 7 2.41s 9.29s 15.89s Mac OS X/10 2.30s 8.11s 13.45s iOS7 3.27s 10.64s 15.79s Android 4 4.06s 14.30s 27.93s iOS8 3.53s 11.54s 19.72s Windows 8 2.67s 10.75s 18.74s R E A L U S E R S A R E N O T N O R M A L
- 13. 0 1 2 3 5 6 7 8 9 10 11 12 14 15 16 17 18 19 Median: 3.23s R U M S Y N T H E T I C F I R S T S Y N T H E T I C R E P E AT 3 . 2 3 4 . 1 2 . 2 W H Y A R E T H E T W O N U M B E R S S O FA R O F F ?
- 14. “ B U T, I T L O A D S S O M U C H FA S T E R F O R M E ? ? ! ! ” • 2015 Macbook Pro • Warm browser cache • FIOS vs. • x86 - Windows 7 VM • Completely cold cache/dns • Throttled bandwidth 4 seconds 20 seconds
- 15. R U M S Y N T H E T I C • Primary source of truth • User behavior/ conversion • Complete coverage • Diagnostics • Page Construction • Pre-production & Competitive benchmarking
- 16. R U M S Y N T H E T I C • Primary source of truth • User behavior/ conversion • Complete coverage • Diagnostics • Page Construction • Pre-production & Competitive benchmarking You need both
- 17. U N D E R S TA N D W H AT 1 S M E A N S T O Y O U R B U S I N E S S