Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-scale applications
0 likes217 views
The document is a presentation by Konrad Malawski on the importance of asynchronous programming for scalable applications, discussing concepts such as back-pressure, concurrency models, and performance measurement. It emphasizes the need for design principles that enhance scalability in modern software systems, especially in handling high loads and latency issues. The discussion covers various asynchronous techniques and frameworks like Akka and Reactive Streams to manage concurrent processes efficiently.
![* Both versions are used: lock-free / lockless
class CASBackedQueue[A] {
val _queue = new AtomicReference(Vector[A]())
// does not block, may spin though
@tailrec final def put(a: A): Unit = {
val queue = _queue.get
val appended = queue :+ a
if (!_queue.compareAndSet(queue, appended))
put(a)
}
}
Concurrent < lock-free < wait-free](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-81-320.jpg)
![class CASBackedQueue[A] {
val _queue = new AtomicReference(Vector[A]())
// does not block, may spin though
@tailrec final def put(a: A): Unit = {
val queue = _queue.get
val appended = queue :+ a
if (!_queue.compareAndSet(queue, appended))
put(a)
}
}
Concurrent < lock-free < wait-free](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-82-320.jpg)
![class CASBackedQueue[A] {
val _queue = new AtomicReference(Vector[A]())
// does not block, may spin though
@tailrec final def put(a: A): Unit = {
val queue = _queue.get
val appended = queue :+ a
if (!_queue.compareAndSet(queue, appended))
put(a)
}
}
Concurrent < lock-free < wait-free](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-83-320.jpg)
![Kernels and CPUs
[…] switching from user-level to kernel-level
on a (2.8 GHz) P4 is 1348 cycles.
[…] Counting actual time, the P4 takes 481ns […]
http://wiki.osdev.org/Context_Switching](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-104-320.jpg)
![Asynchronous I / O [Linux]
Linux AIO = JVM NIO](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-112-320.jpg)
![Asynchronous I / O [Linux]
NewIO… since 2004!
(No-one calls it “new” any more)
Linux AIO = JVM NIO](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-113-320.jpg)
![Asynchronous I / O [Linux]
Less time wasted waiting.
Same amount of buffer copies.](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-114-320.jpg)
![ZeroCopy = sendfile [Linux]
“Work smarter.
Not harder.”
http://fourhourworkweek.com/](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-115-320.jpg)
![ZeroCopy = sendfile [Linux]](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-116-320.jpg)
![ZeroCopy = sendfile [Linux]
Data never leaves kernel mode!](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-117-320.jpg)
![C10K – epoll [Linux]](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-127-320.jpg)
![C10K – epoll [Linux]](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-128-320.jpg)
![C10K – epoll [Linux]](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-129-320.jpg)
![C10K – epoll [Linux]](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-130-320.jpg)
![C10K – epoll [Linux]](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-131-320.jpg)
![Learn more at:
• SCKRK.com –
• KrakowScala.pl – Kraków Scala User Group
• LambdaKRK.pl – Lambda Lounge Kraków
• GeeCON.org – awesome “all around the JVM” conference,
[Kraków, Poznań, Prague, and more…!]
Software Craftsmanship Kraków
Computer Science Whitepaper Reading Club Kraków](https://image.slidesharecdn.com/konradmalawski-150602133453-lva1-app6892/85/Atmosphere-Conference-2015-Need-for-Async-In-pursuit-of-scalable-internet-scale-applications-163-320.jpg)
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-scale applications
- 1. Konrad 'ktoso' Malawski GeeCON 2014 @ Kraków, PL Konrad `@ktosopl` Malawski need for async hot pursuit for Scalable apps
- 2. Konrad `ktoso` Malawski Akka Team, Reactive Streams TCK
- 3. Konrad `@ktosopl` Malawski akka.io typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl lambdakrk.pl
- 4. Nice to meet you! Who are you guys?
- 6. High Performance Software Development For the majority of the time, high performance software development is not about compiler hacks and bit twiddling. It is about fundamental design principles that are key to doing any effective software development. Martin Thompson practicalperformanceanalyst.com/2015/02/17/getting-to-know-martin-thompson-...
- 7. Agenda • Why? • Async and Synch basics / definitions • Back-pressure applied • Async where it matters: Scheduling • How NOT to measure Latency • Concurrent < lock-free < wait-free • I/O: IO, AIO, NIO, Zero • C10K: select, poll, epoll / kqueue • Distributed Systems: Where Async is at Home • Wrapping up and Q/A
- 8. Sync / Async Basics
- 9. Sync / Async
- 10. Sync / Async
- 11. Sync / Async
- 12. Sync / Async
- 13. Sync / Async
- 14. Sync / Async
- 15. Sync / Async
- 16. Sync / Async
- 17. Sync / Async
- 18. Asynchronous systems rely on buffering.
- 19. Asynchronous systems rely on buffering. What if they overflow? How do you back-pressure?
- 20. Back-pressure? Example Without Fast Publisher Slow Subscriber
- 21. Back-pressure? (example without) Subscriber usually has some kind of buffer.
- 24. Back-pressure? (example without) What if the buffer overflows?
- 25. Back-pressure? (example without) Use bounded buffer, drop messages + require re-sending
- 26. Back-pressure? (example without) Kernel does this! Routers do this! (TCP) Use bounded buffer, drop messages + require re-sending
- 27. Back-pressure “not needed” when: speed(publisher) < speed(subscriber)
- 28. Back-pressure? Fast Subscriber, No Problem No problem!
- 29. Back-pressure? NACKing is NOT enough.
- 31. Back-pressure? Example NACKing Telling the Publisher to slow down / stop sending…
- 32. Back-pressure? Example NACKing NACK did not make it in time, because M was in-flight!
- 34. Just push – not safe when Slow Subscriber Just pull – too slow when Fast Subscriber Back-pressure? RS: Dynamic Push/Pull
- 35. Solution: Dynamic adjustment Back-pressure? RS: Dynamic Push/Pull Just push – not safe when Slow Subscriber Just pull – too slow when Fast Subscriber
- 36. Back-pressure? RS: Dynamic Push/Pull Slow Subscriber sees it’s buffer can take 3 elements. Publisher will never blow up it’s buffer.
- 37. Back-pressure? RS: Dynamic Push/Pull Fast Publisher will send at-most 3 elements. This is pull-based-backpressure.
- 38. Back-pressure? RS: Dynamic Push/Pull Fast Subscriber can issue more Request(n), before more data arrives!
- 39. Back-pressure? RS: Dynamic Push/Pull Fast Subscriber can issue more Request(n), before more data arrives. Publisher can accumulate demand.
- 40. Back-pressure? RS: Accumulate demand Publisher accumulates total demand per subscriber.
- 41. Back-pressure? RS: Accumulate demand Total demand of elements is safe to publish. Subscriber’s buffer will not overflow.
- 42. Back-pressure? Reactive Streams Mini in-line pitch Specification and TCK reactive-streams.org Implementations Akka Streams / Akka Http Slick RxJava Reactor RatPack Vert.x MongoDB driver http://www.reactive-streams.org/announce-1.0.0
- 43. Highly parallel systems Event loops Async where it matters:
- 44. Async where it matters: Scheduling
- 45. Async where it matters: Scheduling
- 46. Async where it matters: Scheduling
- 47. Async where it matters: Scheduling
- 48. Async where it matters: Scheduling
- 49. Async where it matters: Scheduling
- 50. Async where it matters: Scheduling
- 51. Async where it matters: Scheduling
- 52. Async where it matters: Scheduling
- 53. Scheduling (notice they grey sync call)
- 54. Scheduling (notice they grey sync call)
- 55. Scheduling (now with Async db call)
- 56. Scheduling (now with Async db call)
- 57. Scheduling (now with Async db call)
- 58. Scheduling (now with Async db call)
- 59. Latency
- 60. Latency Time interval between the stimulation and response.
- 61. Latency Quiz Gil Tene style, see:“How NOT to Measure Latency” Is 10s latency acceptable in your app? Is 200ms latency acceptable? How about most responses within 200ms? So mostly 20ms and some 1 minute latencies is OK? Do people die when we go above 200ms? So 90% below 200ms, 99% bellow 1s, 99.99% below 2s?
- 62. Latency in the “real world” Gil Tene style, see:“How NOT to Measure Latency” “Our response time is 200ms average, stddev is around 60ms” — a typical quote
- 63. Latency in the “real world” Gil Tene style, see:“How NOT to Measure Latency” “Our response time is 200ms average, stddev is around 60ms” — a typical quote Latency does NOT behave like normal distribution! “So yeah, our 99,99%’ is…”
- 65. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
- 66. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
- 67. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
- 68. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
- 69. Gil Tene style, see:“How NOT to Measure Latency” Hiccups
- 70. Concurrent < lock-free < wait-free
- 71. Concurrent < lock-free < wait-free
- 72. Concurrent < lock-free < wait-free
- 73. Concurrent < lock-free < wait-free Concurrent data structure
- 74. A locks; B tries lock... A writes; A unlocks; B acquires the lock... B (finally) writes & unlocks What can happen in concurrent data structures: Concurrent < lock-free < wait-free
- 75. A locks; B tries lock... A writes; A unlocks; B acquires the lock... B (finally) writes & unlocks What can happen in concurrent data structures: Moral? 1) Thread A is not very nice to B. 2) A lot of time is wasted. Concurrent < lock-free < wait-free Wastedtime
- 76. Concurrent < lock-free < wait-less Concurrency is NOT Parallelism. Rob Pike - Concurrency is NOT Parallelism (video) def offer(a: A): Boolean // returns on failure def add(a: A): Unit // throws on failure def put(a: A): Boolean // blocks until able to enqueue
- 77. Concurrent < lock-free < wait-free concurrent data structure < lock-free* data structure * lock-free a.k.a. lockless
- 78. What lock-free programming looks like:
- 79. An algorithm is lock-free if it satisfies that: When the program threads are run sufficiently long, at least one of the threads makes progress. Concurrent < lock-free < wait-free
- 80. Concurrent < lock-free < wait-free What can happen in concurrent data structures: A tries to write; B tries to write; B wins! A tries to write; C tries to write; C wins! A tries to write; D tries to write; D wins! A tries to write; B tries to write; B wins! A tries to write; E tries to write; E wins! A tries to write; F tries to write; F wins! … Moral? 1) Thread A is a complete loser. 2) Thread A eventually should be able to progress…
- 81. * Both versions are used: lock-free / lockless class CASBackedQueue[A] { val _queue = new AtomicReference(Vector[A]()) // does not block, may spin though @tailrec final def put(a: A): Unit = { val queue = _queue.get val appended = queue :+ a if (!_queue.compareAndSet(queue, appended)) put(a) } } Concurrent < lock-free < wait-free
- 82. class CASBackedQueue[A] { val _queue = new AtomicReference(Vector[A]()) // does not block, may spin though @tailrec final def put(a: A): Unit = { val queue = _queue.get val appended = queue :+ a if (!_queue.compareAndSet(queue, appended)) put(a) } } Concurrent < lock-free < wait-free
- 83. class CASBackedQueue[A] { val _queue = new AtomicReference(Vector[A]()) // does not block, may spin though @tailrec final def put(a: A): Unit = { val queue = _queue.get val appended = queue :+ a if (!_queue.compareAndSet(queue, appended)) put(a) } } Concurrent < lock-free < wait-free
- 84. Concurrent < lock-free < wait-free
- 85. Concurrent < lock-free < wait-free
- 86. Concurrent < lock-free < wait-free
- 87. Concurrent < lock-free < wait-free
- 88. Concurrent < lock-free < wait-free
- 89. Concurrent < lock-free < wait-free
- 90. Concurrent < lock-free < wait-free “concurrent” data structure < lock-free* data structure < wait-free data structure * Both versions are used: lock-free / lockless
- 91. Concurrent < lock-free < wait-free Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms Maged M. Michael Michael L. Scott An algorithm is wait-free if every operation has a bound on the number of steps the algorithm will take before the operation completes.
- 92. wait-free: j.u.c.ConcurrentLinkedQueue Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms Maged M. Michael Michael L. Scott public boolean offer(E e) { checkNotNull(e); final Node<E> newNode = new Node<E>(e); for (Node<E> t = tail, p = t;;) { Node<E> q = p.next; if (q == null) { // p is last node if (p.casNext(null, newNode)) { // Successful CAS is the linearization point // for e to become an element of this queue, // and for newNode to become "live". if (p != t) // hop two nodes at a time casTail(t, newNode); // Failure is OK. return true; } // Lost CAS race to another thread; re-read next } else if (p == q) // We have fallen off list. If tail is unchanged, it // will also be off-list, in which case we need to // jump to head, from which all live nodes are always // reachable. Else the new tail is a better bet. p = (t != (t = tail)) ? t : head; else // Check for tail updates after two hops. p = (p != t && t != (t = tail)) ? t : q; } } This is a modification of the Michael & Scott algorithm, adapted for a garbage-collected environment, with support for interior node deletion (to support remove(Object)). For explanation, read the paper.
- 93. I / O
- 94. IO / AIO
- 95. IO / AIO / NIO
- 96. IO / AIO / NIO / Zero
- 97. Synchronous I / O — Havoc Pennington (HAL, GNOME, GConf, D-BUS, now Typesafe) When I learned J2EE about 2008 with some of my desktop colleagues our reactions included something like: ”wtf is this sync IO crap, where is the main loop?!” :-)
- 98. Interruption! CPU: User Mode / Kernel Mode
- 99. Kernels and CPUs
- 100. Kernels and CPUs
- 101. Kernels and CPUs
- 102. Kernels and CPUs
- 104. Kernels and CPUs […] switching from user-level to kernel-level on a (2.8 GHz) P4 is 1348 cycles. […] Counting actual time, the P4 takes 481ns […] http://wiki.osdev.org/Context_Switching
- 105. I / O
- 106. I / O
- 107. I / O
- 108. I / O
- 109. I / O
- 110. I / O “Don’t worry. It only gets worse!”
- 111. I / O “Don’t worry. It only gets worse!” Same data in 3 buffers!4 mode switches!
- 112. Asynchronous I / O [Linux] Linux AIO = JVM NIO
- 113. Asynchronous I / O [Linux] NewIO… since 2004! (No-one calls it “new” any more) Linux AIO = JVM NIO
- 114. Asynchronous I / O [Linux] Less time wasted waiting. Same amount of buffer copies.
- 115. ZeroCopy = sendfile [Linux] “Work smarter. Not harder.” http://fourhourworkweek.com/
- 116. ZeroCopy = sendfile [Linux]
- 117. ZeroCopy = sendfile [Linux] Data never leaves kernel mode!
- 118. ZeroCopy…
- 119. C10K and beyond
- 120. C10K and beyond “10.000 concurrent connections” Not a new problem, pretty old actually: ~12 years old. http://www.kegel.com/c10k.html
- 121. C10K and beyond It’s not about performance. It’s about scalability. These are orthogonal things. Threading differences: apache httpd / nginx / akka (1 thread = 1 request) == Very heavy
- 122. select/poll
- 123. C10K – poll
- 124. C10K – poll
- 125. C10K – poll
- 126. epoll
- 127. C10K – epoll [Linux]
- 128. C10K – epoll [Linux]
- 129. C10K – epoll [Linux]
- 130. C10K – epoll [Linux]
- 131. C10K – epoll [Linux]
- 132. C10K O(n) is a no-go for epic scalability.
- 133. C10K O(n) is a no-go for epic scalability. State of Linux scheduling: O(n) O(1) CFS (O(1)/ O(log n)) And Socket selection: Select/Poll O(n) EPoll (O(1))
- 134. C10K O(n) is a no-go for epic scalability. State of Linux scheduling: O(n) O(1) CFS (O(1)) And Socket selection: Select/Poll O(n) EPoll (O(1)) O(1) IS a go for epic scalability. Moral:
- 135. Distributed Systems
- 136. Distributed Systems “… in which the failure of a computer you didn't even know existed can render your own computer unusable.” — Leslie Lamport http://research.microsoft.com/en-us/um/people/lamport/pubs/distributed-system.txt
- 137. Distributed Systems The bigger the system, the more “random” latency / failure noise. Embrace instead of hiding it.
- 139. Backup requests A technique for fighting “long tail latencies”. By issuing duplicated work, when SLA seems in danger.
- 140. Backup requests
- 141. Backup requests
- 142. Backup requests - send
- 143. Backup requests - send
- 144. Backup requests - send
- 145. Backup requests Avg Std dev 95%ile 99%ile 99.9%ile No backups 33 ms 1524 ms 24 ms 52 ms 994 ms After 10ms 14 ms 4 ms 20 ms 23 ms 50 ms After 50ms 16 ms 12 ms 57 ms 63 ms 68 ms Jeff Dean - Achieving Rapid Response Times in Large Online Services Peter Bailis - Doing Redundant Work to Speed Up Distributed Queries Akka - Krzysztof Janosz @ Akkathon, Kraków - TailChoppingRouter (docs, pr)
- 147. Combined requests A technique for avoiding duplicated work. By aggregating requests, possibly increasing latency.
- 148. Combined requests A technique for avoiding duplicated work. By aggregating requests, possibly increasing latency. “Wat?Why would I increase latency!?”
- 149. Combined requests
- 150. Combined requests
- 151. Combined requests
- 152. Combined requests
- 153. Combined requests
- 154. Combined requests
- 155. World of Tradeoffs
- 156. Combined requests with backpressure reactive-streams.org Timer can be replaced with back-pressure.
- 157. Wrapping up
- 158. Wrapping up • Someone has to bite the bullet though! • We’re all running on real hardware. • We do it so you don’t have to. Summing up the goal of this talk: • Keep your apps pure and simple • Be aware of internals • Use libraries which apply these techniques • Maybe indeed • Messaging all the way! These techniques are hard to get right, yet:
- 159. Links • akka.io • reactive-streams.org • akka-user • Gil Tene - How NOT to measure latency, 2013 • Jeff Dean @Velocity 2014 • Alan Bateman, Jeanfrancois Arcand (Sun) Async IO Tips @ JavaOne • http://linux.die.net/man/2/select • http://linux.die.net/man/2/poll • http://linux.die.net/man/4/epoll • giltene/jHiccup • Linux Journal: ZeroCopy I, Dragan Stancevis 2013 • Last slide car picture: http://actu-moteurs.com/sprint/gt-tour/jean- philippe-belloc-un-beau-challenge-avec-le-akka-asp-team/2000
- 160. Links • http://wiki.osdev.org/Context_Switching • CppCon: Herb Sutter "Lock-Free Programming (or, Juggling Razor Blades)" • http://www.infoq.com/presentations/reactive-services-scale • Gil Tene’s HdrHistogram.org • http://hdrhistogram.github.io/HdrHistogram/plotFiles.html • Rob Pike - Concurrency is NOT Parallelism (video) • Brendan Gregg - Systems Performance: Enterprise and the Cloud (book) • http://psy-lob-saw.blogspot.com/2015/02/hdrhistogram-better-latency-capture.html • Jeff Dean, Luiz Andre Barroso - The Tail at Scale (whitepaper,ACM) • http://highscalability.com/blog/2012/3/12/google-taming-the-long-latency-tail-when- more-machines-equal.html • http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system- architects/ • Marcus Lagergren - Oracle JRockit:The Definitive Guide (book) • http://mechanical-sympathy.blogspot.com/2013/08/lock-based-vs-lock-free- concurrent.html • Handling of Asynchronous Events - http://www.win.tue.nl/~aeb/linux/lk/lk-12.html • http://www.kegel.com/c10k.html
- 161. Links • www.reactivemanifesto.org/ • Seriously the only right way to micro benchmark on the JVM: • JMH openjdk.java.net/projects/code-tools/jmh/ • JMH for Scala: https://github.com/ktoso/sbt-jmh • http://www.ibm.com/developerworks/library/l-async/ • http://lse.sourceforge.net/io/aio.html • https://code.google.com/p/kernel/wiki/AIOUserGuide • ShmooCon: C10M - Defending the Internet At Scale (Robert Graham) • http://blog.erratasec.com/2013/02/scalability-its-question-that-drives-us.html#.VO6E11PF8SM •User-level threads....... with threads. - Paul Turner @ Linux Plumbers Conf 2013 •Jan Pustelnik’s talk yesterday: https://github.com/gosubpl/sortbench
- 162. Special thanks to: • Aleksey Shipilëv • Andrzej Grzesik • Gil Tene • Kirk Pepperdine • Łukasz Dubiel • Marcus Lagergren • Martin Thompson • Mateusz Dymczyk • Nitsan Wakart Thanks guys, you’re awesome. alphabetically, mostly • Peter Lawrey • Richard Warburton • Roland Kuhn • Sergey Kuksenko • Steve Poole • Viktor Klang a.k.a. √ • Antoine de Saint Exupéry :-) • and the entire AkkaTeam • the Mechanical Sympathy Mailing List
- 163. Learn more at: • SCKRK.com – • KrakowScala.pl – Kraków Scala User Group • LambdaKRK.pl – Lambda Lounge Kraków • GeeCON.org – awesome “all around the JVM” conference, [Kraków, Poznań, Prague, and more…!] Software Craftsmanship Kraków Computer Science Whitepaper Reading Club Kraków
- 164. Questions? ktoso @ typesafe.com twitter: ktosopl github: ktoso team blog: letitcrash.com home: akka.io
- 165. ©Typesafe 2015 – All Rights Reserved