Georgia Tech: Performance Engineering - Queuing Theory and Predictive Modeling
8 likes2,160 views
The document provides an overview of performance engineering, focusing on queuing theory as it pertains to enterprise computing performance. It discusses key concepts including types of queues, performance bottlenecks, job flow balance, and Little's Law, which relates to queue length and waiting times. Additionally, it introduces mathematical notations and various queuing systems while emphasizing the importance of early performance modeling and prediction tools.
![Formulae Notation A = Arrival (Production) Rate (usually noted: ) Ts = Service (Consumption) Time: Average Time it takes to service one message in the queue Tq = Average Time a message spends in the queue (I.e. drop of water in the funnel) T = Ts + Tq [Average response time] 1 = 100% Service Capacity or 1 Time Unit (1/Ts) = Service Rate (1/A) = Average Job Inter-arrival Time [average amount of time between job arrivals] Enterprise Computing Performance - Course Overview](https://image.slidesharecdn.com/week2bperfengineering2-12610803511574-phpapp02/85/Georgia-Tech-Performance-Engineering-Queuing-Theory-and-Predictive-Modeling-11-320.jpg)
Georgia Tech: Performance Engineering - Queuing Theory and Predictive Modeling
- 1. Performance Engineering Overview 2 Enterprise Computing Performance Brian Wilson CS 4803 EPR
- 2. Lecture Overview Performance Engineering Overview - Part 2… Queuing Theory Overview Early life-cycle performance modeling Simple Distributed System Model Sequence Diagrams Enterprise Computing Performance - Course Overview
- 3. Queuing Theory Simplified A brief introduction to queuing theory, as it applies to computing performance
- 4. What is Queuing Theory? a collection of mathematical models of various queuing systems that take inputs based on probability or assumption, and that provide quantitative parameters describing the system performance.
- 5. Introduction Series of mathematical formulae Calculates event probability Predicts capacity Enterprise Computing Performance - Course Overview
- 6. What’s Queuing Theory? The theoretical study of waiting lines, expressed in mathematical terms Enterprise Computing Performance - Course Overview input output queue server residence time = wait time + service time
- 7. Types of Queues Markovian Exponential distribution Deterministic Constant arrival rates General Arbitrary or random distribution of arrival rates Enterprise Computing Performance - Course Overview
- 8. Queuing Disciplines The representation of the way the queue is organized (rules of inserting and removing customers to/from the queue): 1) FIFO (First In First Out) also called FCFS (First Come First Serve) - orderly queue. 2) LIFO (Last In First Out) also called LCFS (Last Come First Serve) - stack. 3) SIRO (Serve In Random Order). (distributed/web) 4) Priority Queue, that may be viewed as a number of queues for various priorities. 5) Many other more complex queuing methods that typically change the customer’s position in the queue according to the time spent already in the queue, expected service duration, and/or priority. Typical for computer multi-access systems. Enterprise Computing Performance - Course Overview
- 9. What’s a Bottleneck? If the Production Rate, on average over time, exceeds Consumption Rate… Performance Bottleneck! What’s Job Flow Balance? T = A Enterprise Computing Performance - Course Overview
- 10. QT Assumptions For any Queuing Theory to work on paper, averages for all numbers must be assumed Cannot calc real-time (instantaneous) data-points without mechanical means Enterprise Computing Performance - Course Overview
- 11. Formulae Notation A = Arrival (Production) Rate (usually noted: ) Ts = Service (Consumption) Time: Average Time it takes to service one message in the queue Tq = Average Time a message spends in the queue (I.e. drop of water in the funnel) T = Ts + Tq [Average response time] 1 = 100% Service Capacity or 1 Time Unit (1/Ts) = Service Rate (1/A) = Average Job Inter-arrival Time [average amount of time between job arrivals] Enterprise Computing Performance - Course Overview
- 12. Kendall Notation Queuing systems are described with 3 parameters… Parameter 1 M = Markovian inter-arrival rates D = Deterministic inter-arrival rates Parameter 2 M = Markovian service rates G = General service rates D = Deterministic service times Parameter 3 Number of servers Examples: M/M/1 - D/D/2 - M/G/3 Enterprise Computing Performance - Course Overview
- 13. Example: The M/M/1 System Enterprise Computing Performance - Course Overview Job output queue Exponential server
- 14. Little’s Law 1 Sometimes called “Little’s Theorem” “ Length of a queue is the product of the message arrival rate multiplied by the time they stay in the queue. ” Notated: Q = ATq Enterprise Computing Performance - Course Overview
- 15. Little’s Law 2 Alternate definition: “ The average number of jobs waiting in the queue ( N ) is equal to the product of the average arrival rate and the average response time . ” Notated: N = AT Enterprise Computing Performance - Course Overview
- 16. Little’s “LAW” A great way to remember it: If we notate the length of time spent in the queue as L the arrival rate as A and the time spent in the queue (residence or Wating time), then we can say: L = AW Enterprise Computing Performance - Course Overview
- 17. Web Server Queuing Model Enterprise Computing Performance - Course Overview
- 18. Review Questions A system is said to be stable when? A > (1/Ts) means? Explain… If the avg inter-arrival times (1/A) are unpredictable (no correlation to a known or trend number), the arrival rate exhibits what type of distribution? What’s another name for a memoryless state? Such queuing networks are called ____ if new jobs arrive from outside the network, and may eventually depart from the network. __________ Theory describes discreet, yet rare events where arrival rates are randomly distributed, yet can be averaged over a given period of time. Enterprise Computing Performance - Course Overview
- 19. Resources A really good website for queuing tools and techniques: http://www.me.utexas.edu/~jensen/ORMM/index.html Queuing Theory Terminology: http://www.me.utexas.edu/~jensen/ORMM/models/unit/queue/subunits/terminology/index.html Enterprise Computing Performance - Course Overview
- 20. Early Life-cycle Performance Modeling A brief overview
- 21. Sequence Diagram Example Enterprise Computing Performance - Course Overview
- 22. Expanded Sequence Enterprise Computing Performance - Course Overview
- 23. Distributed System Model Enterprise Computing Performance - Course Overview
- 24. Resource Requirements Enterprise Computing Performance - Course Overview See Page 38 Add requirements (in terms of time) for resources such as CPU, Disk, NetDelay, etc for each step of each scenario.
- 25. Performance Prediction Tools Enterprise Computing Performance - Course Overview Many new UML tools for modeling Very time intensive Many assumptions Must be done before design finalization Saves time and money in the long run
Editor's Notes
- #2: Enterprise Computing Performance
- #6: Enterprise Computing Performance Explain the idea of pouring water into a bottle through a funnel… Q: How do you calculate how much water the funnel can handle, given a fixed time interval? What are the factors? (Production Rate, Consumption Rate, Funnel Size) Talk about each computer system resource (ask for examples) can be viewed autonomously; each having a queue that can get overfilled, thus degrading performance… give and ask for analogies.
- #8: Enterprise Computing Performance Explain the idea of pouring water into a bottle through a funnel… Q: How do you calculate how much water the funnel can handle, given a fixed time interval? What are the factors? (Production Rate, Consumption Rate, Funnel Size) Talk about each computer system resource (ask for examples) can be viewed autonomously; each having a queue that can get overfilled, thus degrading performance… give and ask for analogies.
- #9: Enterprise Computing Performance Explain the idea of pouring water into a bottle through a funnel… Q: How do you calculate how much water the funnel can handle, given a fixed time interval? What are the factors? (Production Rate, Consumption Rate, Funnel Size) Talk about each computer system resource (ask for examples) can be viewed autonomously; each having a queue that can get overfilled, thus degrading performance… give and ask for analogies.
- #10: Enterprise Computing Performance Job Flow Balance = The assumption that the system is fast enough to handle the arrives and thus the completion rate or throughput equals the arrive rate.
- #11: Enterprise Computing Performance Queuing theory models can only describe average behavior over time, NOT instantaneous or real-time data-points or complex performance trends, without mechanical means (simulation and analysis tools), as simple theory must then be extrapolated and applied to complex practice.
- #12: Enterprise Computing Performance Hand this out
- #15: Enterprise Computing Performance Ask for someone to give the class his/her definition, before clicking Emphasize that this will be on the final exam
- #16: Enterprise Computing Performance Little's Law (N = AT) states that the average number of jobs waiting in the queue (N) is equal to the product of the average arrival rate and the average response time. Little's Law is surprisingly general, and applies to all queuing systems that are both stable and conservative (i.e., no work is lost when switching between jobs). Little's Law is especially useful when applied to queuing networks. Typically, a single queue is insufficient for modeling a complex system such as a Web server. In many such cases a system can be modeled as a graph or network in which each queue represents one node. Such queuing networks are called open if new jobs arrive from outside the network, and may eventually depart from the network.
- #19: Enterprise Computing Performance Answers: When the arrival rate is less than the service rate (1/Ts)… A < (1/Ts) The system is stable – sooner or later ALL messages/requests will be serviced. The performance may be poor, but the queue will function. “ Memoryless” or exponential – THIS WILL BE ON THE TEST Markov
- #20: Enterprise Computing Performance Answers: When the arrival rate is less than the service rate (1/Ts)… A < (1/Ts) The system is stable – sooner or later ALL messages/requests will be serviced. The performance may be poor, but the queue will function. “ Memoryless” or exponential – THIS WILL BE ON THE TEST Markov