analyzing procedures
- 1. 7.3 Analyzing Procedures By considering the asymptotic growth of functions, rather than their actual outputs, the OO, ΩΩ, and ΘΘ operators allow us to hide constants and factors that change depending on the speed of our processor, how data is arranged in memory, and the specifics of how our interpreter is implemented. Instead, we can consider the essential properties of how the running time of the procedures increases with the size of the input. 7.3.1 Input size Procedure inputs may be many different types: Numbers, Lists of Numbers, Lists of Lists, Procedures, etc. Our goal is to characterize the input size with a single number that does not depend on the types of the input. We use the Turing machine to model a computer, so the way to measure the size of the input is the number of characters needed to write the input on the tape. The characters can be from any fixed-size alphabet, such as the ten decimal digits, or the letters of the alphabet. The number of different symbols in the tape alphabet does not matter for our analysis since we are concerned with orders of growth not absolute values. Within the OO, ΩΩ, and ΘΘ operators, a constant factor does not matter (e.g., Θ(n)≡Θ(17n+523)Θ(n)≡Θ(17n+523)).
- 2. This means is doesn't matter whether we use an alphabet with two symbols or an alphabet with 256 symbols. With two symbols the input may be 8 times as long as it is with a 256-symbol alphabet, but the constant factor does not matter inside the asymptotic operator. Thus, we measure the size of the input as the number of symbols required to write the number on a Turing Machine input tape. To figure out the input size of a given type, we need to think about how many symbols it would require to write down inputs of that type. Booleans. There are only two Boolean values: true and false. Hence, the length of a Boolean input is fixed. Numbers. Using the decimal number system (that is, 10 tape symbols), we can write a number of magnitude nn using log10nlog10n digits. Using the binary number system (that is, 2 tape symbols), we can write it using log2nlog2n bits. Within the asymptotic operators, the base of the logarithm does not matter (as long as it is a constant) since it changes the result by a constant factor. We can see this from the argument above --- changing the number of symbols in the input alphabet changes the input length by a constant factor which has no impact within the asymptotic operators.
- 3. Lists. If the input is a List, the size of the input is related to the number of elements in the list. If each element is a constant size (for example, a list of numbers where each number is between 0 and 100), the size of the input list is some constant multiple of the number of elements in the list. Hence, the size of an input that is a list of nn elements is cncn for some constant cc. Since Θ(cn)=Θ(n)Θ(cn)=Θ(n), the size of a List input is Θ(n)Θ(n) where nn is the number of elements in the List. If List elements can vary in size, then we need to account for that in the input size. For example, suppose the input is a List of Lists, where there are nn elements in each inner List, and there are nn List elements in the main List. Then, there are n2n2 total elements and the input size is in Θ(n2)Θ(n2).
- 4. 7.3.2 Running Time We want a measure of the running time of a procedure that satisfies two properties: (1) it should be robust to ephemeral properties of a particular execution or computer, and (2) it should provide insights into how long it takes evaluate the procedure on a wide range of inputs. To estimate the running time of an evaluation, we use the number of steps required to perform the evaluation. The actual number of steps depends on the details of how much work can be done on each step. For any particular processor, both the time it takes to perform a step and the amount of work that can be done in one step varies. When we analyze procedures, however, we usually don't want to deal with these details. Instead, what we care about is how the running time changes as the input size increases. This means we can count anything we want as a "step" as long as each step is the approximately same size and the time a step requires does not depend on the size of the input. The clearest and simplest definition of a step is to use one Turing Machine step. We have a precise definition of exactly what a Turing Machine can do in one step: it can read the symbol in the current square, write a symbol into that square, transition its internal state number, and move one square to the left or right.
- 5. Counting Turing Machine steps is very precise, but difficult because we do not usually start with a Turing Machine description of a procedure and creating one is tedious. Time makes more converts than reason. -- Thomas Paine Instead, we usually reason directly from a Scheme procedure (or any precise description of a procedure) using larger steps. As long as we can claim that whatever we consider a step could be simulated using a constant number of steps on a Turing Machine, our larger steps will produce the same answer within the asymptotic operators. One possibility is to count the number of times an evaluation rule is used in an evaluation of an application of the procedure. The amount of work in each evaluation rule may vary slightly (for example, the evaluation rule for an if expression seems more complex than the rule for a primitive) but does not depend on the input size. Hence, it is reasonable to assume all the evaluation rules to take constant time. This does not include any additional evaluation rules that are needed to apply one rule. For example, the evaluation rule for application expressions includes evaluating every subexpression. Evaluating an application constitutes one work unit for the application rule itself, plus all the work required to evaluate the subexpressions. In cases where the bigger steps are unclear, we can always return to our precise definition of a step as one step of a Turing Machine.
- 6. 7.3.3 Worst Case Input A procedure may have different running times for inputs of the same size. For example, consider this procedure that takes a List as input and outputs the first positive number in the list: (if (> (car p) 0) (car p) (list-first-pos (cdr p))))) If the first element in the input list is positive, evaluating the application of list- first- pos requires very little work. It is not necessary to consider any other elements in the list if the first element is positive. On the other hand, if none of the elements are positive, the procedure needs to test each element in the list until it reaches the end of the list (where the base case reports an error). In our analyses we usually consider the worst case input. For a given size, the worst case input is the input for which evaluating the procedure takes the most work. By focusing on the worst case input, we know the maximum running time for the procedure. Without knowing something about the possible inputs to the procedure, it is safest to be pessimistic about the input and not assume any properties that are not known (such as that the first number in the list is positive for the first-pos example). (define (list-first-pos p) (if (null? p) (error "No positive element found")
- 7. In some cases, we also consider the average case input. Since most procedures can take infinitely many inputs, this requires understanding the distribution of possible inputs to determine an "average" input. This is often necessary when we are analyzing the running time of a procedure that uses another helper procedure. If we use the worst-case running time for the helper procedure, we will grossly overestimate the running time of the main procedure. Instead, since we know how the main procedure uses the helper procedure, we can more precisely estimate the actual running time by considering the actual inputs. We see an example of this in the analysis of how the + procedure is used by list-length in Section 7.4.2.