![Average-Case Time Complexity Analysis
Basic operation: the comparison of an item in the array
with searchkey.
Input size: n, the number of items in the array.
We first analyze the case in which it is known that
searchkey is in the array, where the items in the array
are all distinct, and where we have no reason to believe
that searchkey is more likely to be in one array slot than
it is to be in another. Based on this information, for 1 ≤ k
≤ n, the probability that searchkey is in the kth array slot
is 1/n. If searchkey is in the kth array slot, the number of
times the basic operation is done to locate searchkey
(and therefore, to exit the loop) is k. This means that the
average time complexity is given by
21
Best-Case Time Complexity Analysis
Basic operation: the comparison of an
item in the array with searchkey.
Input size: n, the number of items in the
array.
Because n ≥ 1, there must be at least one
pass through the loop. If searchkey =
array[0], there will be one pass through the
loop regardless of the size of n. Therefore,
B(n) = 1.
22
11](https://image.slidesharecdn.com/lecture6-datastructurealgorithms-121202123602-phpapp02/85/Lecture6-data-structure-algorithms-11-320.jpg)
Lecture6 data structure(algorithms)
- 1. Lecture 6 Algorithms and Data Structures (CS221) Abdisalam Issa-Salwe Taibah University College of Computer Science & Engineering Computer Science Department Outline • Computer Problem-Solving • Efficiency of Algorithms • Properties of algorithms • Developing an Algorithm • Complexity Analysis • Big-O Notation • Pseudocode • Flowcharts 2 1
- 2. Algorithms An algorithm is a finite set of precise instructions for performing a computation or for solving a problem. A set of instructions for solving a problem or subproblem in a finite amount of time using a finite amount of data The instructions must be unambiguous 3 Algorithm (cont…) Algorithm is a systematic method of instructing an agent how to accomplish a task Usually expressed in a step-wise sequential form May involve alternation, iteration or recursion Detail and language may depend on the agent 4 2
- 3. Algorithm (cont…) It is a set of instructions of how to carry out a process. A list of the steps that must be followed to complete the process and to reach the solution An algorithm can be represented diagrammatically in the form of a flow chart. 5 Divide and Conquer Problem solving: The act of finding a solution to a perplexing, distressing, vexing, or unsettled question Break up a large problem into smaller units that we can handle Applies the concept of abstraction The divide-and-conquer approach can be applied over and over again until each subtask is manageable 6 3
- 4. Computer Problem-Solving 7 Problem-Solving Phases 8 4
- 5. Task 1: Making A Cup of Tea! The following list 1) Take tea bag out of cup represents the steps Boil the water needed in order to make 2) Put tea bag in cup a cup of tea. Put the list in 3) Add milk? the right order and 4) Pour boiling water in cup produce an algorithm in 5) Fill kettle with water POWERPOINT showing 6) Stir the processes and 7) Ready decisions you need to 8) Add sugar ? make. 9) Fetch cup Answers follow 9 START The answers… Fill kettle 1. Fill kettle Boil the water 2. Boil the water 3. Fetch cup Fetch cup 4. Put tea bag in cup Put tea bag in cup 5. Pour boiling water in cup Pour boiling water in cup 6. Take tea bag out of cup Take tea bag out of cup 7. Add sugar ? 8. Stir Add sugar ? N 9. Add milk? 10. Stir Stir in sugar 11. Ready Add milk? N Now add the shapes… Pour in milk Ready 10 5
- 6. Description of an Algorithm • Numbered steps with indentation commonly used • Different descriptions possible for the same algorithm • Difference between the algorithm description (design) and the completion of the task (implementation) • More than one algorithm possible for the same task 11 Efficiency of Algorithms • What resources are required to accomplish the task • How one algorithm compares with other algorithms 12 6
- 7. Efficiency and Complexity • Efficiency • How much time or space is required • Measured in terms of common basic operations • Complexity • How efficiency varies with the size of the task • Expressed in terms of standard functions of n • e.g.: O(n), O(n2), O(log n), O(n log n) 13 Properties of algorithms Input from a specified set, Output from a specified set (solution), Definiteness of every step in the computation, Correctness of output for every possible input, Finiteness of the number of calculation steps, Effectiveness of each calculation step and Generality for a class of problems. 14 7
- 8. Developing an Algorithm The plan must be suitable in a suitable form Two methodologies that currently used Top-down design Top-down design Object-oriented design 15 Top-Down Design Breaking the problem into a set of subproblems called modules Creating a hierarchical structure of problems and subproblems (modules) 16 8
- 9. Testing the Algorithm Desk checking Working through a design at a desk with a pencil and paper Walk-through Manual simulation of the design by the team members, taking sample data values and simulating the design using the sample data Inspection One person (not the designer) reads the design (handed out in advance) line by line while the others point out errors 17 Complexity Analysis Number of CPU cycles it takes to run an algorithm depends on the computer on which the algorithm is run. Count of how many instructions are executed depends on the programming language used to implement the algorithm and the way the programmer writes the program. We want a measure of algorithm efficiency that is independent of the computer, the programming language, the programmer, and all the complex details of the algorithm such as incrementing of loop indices, setting of pointers, etc. In general, the running time of an algorithm increases with the size of the input, and the total running time is roughly proportional to how many times some basic operation (such as a comparison instruction) is done. 18 9
- 10. Complexity Analysis (cont…) We therefore analyze the algorithm’s efficiency by determining the number of times some basic operation is done as a function of the input size. This is called a time complexity analysis of an algorithm. The basic operation may be a single instruction or a group of instructions; in some cases, we may want to consider more than one basic operation. The input size may be easy to determine – such as the size of an array for Sequential or Binary Search -- or it may be more difficult. In some algorithms, the basic operation is always done the same number of times for every instance of size N. When this is the case, the every-case time complexity of the algorithm, T(n), is defined as the number of times the algorithm does the basic operation for an instance of size n. 19 Worst-Case Time Complexity Analysis Basic operation: the comparison of an item in the array with searchkey. Input size: n, the number of items in the array. The basic operation is done at most n times, which is the case if searchkey is the last item in the array or if searchkey is not in the array. Therefore, W(n) = n. 20 10
- 11. Average-Case Time Complexity Analysis Basic operation: the comparison of an item in the array with searchkey. Input size: n, the number of items in the array. We first analyze the case in which it is known that searchkey is in the array, where the items in the array are all distinct, and where we have no reason to believe that searchkey is more likely to be in one array slot than it is to be in another. Based on this information, for 1 ≤ k ≤ n, the probability that searchkey is in the kth array slot is 1/n. If searchkey is in the kth array slot, the number of times the basic operation is done to locate searchkey (and therefore, to exit the loop) is k. This means that the average time complexity is given by 21 Best-Case Time Complexity Analysis Basic operation: the comparison of an item in the array with searchkey. Input size: n, the number of items in the array. Because n ≥ 1, there must be at least one pass through the loop. If searchkey = array[0], there will be one pass through the loop regardless of the size of n. Therefore, B(n) = 1. 22 11
- 12. Performance Analysis Determining an estimate of the time and memory requirement of the algorithm. Time estimation is called time complexity analysis Memory size estimation is called space complexity analysis. Because memory is cheap and abundant, we rarely do space complexity analysis Since time is “expensive” , analysis now defaults to time complexity analysis 23 Big-O Notation The most common method and notation for discussing the execution time of algorithms is "Big O”. For the alphabetized dictionary the algorithm requires O(log N) steps. For the unsorted list the algorithm requires O(N) steps. Big O is the asymptotic execution time of the algorithm. 24 12
- 13. Big-O Notation (In Practice) When computing the complexity, f(n) is the actual time formula g(n) is the simplified version of f Since f(n) stands often for time, we use T(n) instead of f(n) In practice, the simplification of T(n) occurs while it is being computed by the designer 25 Big-O Notation (Common Complexities) T(n) = O(1) // constant time T(n) = O(log n) // logarithmic T(n) = O(n) // linear T(n) = O(n2) //quadratic T(n) = O(n3) //cubic T(n) = O(nc), c≥ 1 // polynomial T(n) = O(logc n), c≥ 1 // polylogarithmic T(n) = O(nlog n) 26 13
- 14. Other Algorithmic Analysis Tools Big Omega T(N) is Ω( F(N) ) if there are positive constants c and N0 such that T(N) > cF( N )) when N > N0 Big O is similar to less than or equal, an upper bound. Big Omega is similar to greater than or equal, a lower bound. Big Theta T(N) is θ( F(N) ) if and only if T(N) is O( F(N) )and T( N ) is Ω( F(N) ). Big Theta is similar to equals. 27 Applying algorithm When applying the theory of algorithm analysis, one must sometimes be aware of the time that it takes to execute the basic operation, the overhead instructions, and the control instructions on the actual computer on which the algorithm is implemented. “Overhead instructions” includes things such as initialization instructions before a loop; the number of times these instructions execute does not increase with input size. “Control instructions” means instructions such as incrementing an index to control a loop; the number of times these instructions execute increases with input size. The basic operation, overhead instructions, and control instructions are all properties of an algorithm and the implementation of the algorithm; they are not properties of a problem. 28 14
- 15. Applying algorithm (cont…) Algorithms with time complexities such as n and 100n are called linear-time algorithms, because their time complexities are linear in the input size n. Algorithms with time complexities such as n2 and 0.01n2 are called quadratic-time algorithms, because their time complexities are quadratic in the input size n. A fundamental principle: any linear-time algorithm is eventually more efficient than any quadratic-time algorithm. Algorithms can be grouped into orders according to their eventual behavior. 29 Pseudocode Uses a mixture of English and formatting to make the steps in the solution explicit 30 15
- 16. Pseudocode Pseudocode is a kind of structured English for describing algorithms. It allows the designer to focus on the logic of the algorithm without being distracted by details of language syntax. At the same time, the pseudocode needs to be complete. It describe the entire logic of the algorithm so that implementation becomes a rote mechanical task of translating line by line into source code. 31 Pseudocode (cont…) The vocabulary used in the pseudocode should be the vocabulary of the problem domain, not of the implementation domain. The pseudocode is a narrative for someone who knows the requirements (problem domain) and is trying to learn how the solution is organized 32 16
- 17. Constructs for flow of control (cont…) 1. SEQUENCE: A linear progression where one task is performed sequentially after another. Sequential control is indicated by writing one action after another, each action on a line by itself, and all actions aligned with the same indent. The actions are performed in the sequence (top to bottom) that they are written. 2. WHILE: A loop (repetition) with a simple conditional test at its beginning. The WHILE construct is used to specify a loop with a test at the top. The beginning and ending of the loop are indicated by two keywords WHILE and ENDWHILE. 33 Constructs for flow of control (cont…) 3. IF-THEN-ELSE: A decision (selection) in which a choice is made between two alternative courses of action. Binary choice on a given Boolean condition is indicated by the use of four keywords: IF, THEN, ELSE, and ENDIF. 4. REPEAT-UNTIL: A loop with a simple conditional test at the bottom. This loop is similar to the WHILE loop except that the test is performed at the bottom of the loop instead of at the top. Two keywords, REPEAT and UNTIL are used. 34 17
- 18. Constructs for flow of control (cont…) 5. CASE: A CASE construct indicates a multiway branch based on conditions that are mutually exclusive. Four keywords, CASE, OF, OTHERS, and ENDCASE, and conditions are used to indicate the various alternatives. FOR: This loop is a specialized construct for iterating a specific number of times, often called a "counting" loop. Two keywords, FOR and ENDFOR are used. 35 Constructs for flow of control (cont…) NESTED CONSTRUCTS: The constructs can be embedded within each other, and this is made clear by use of indenting. Nested constructs should be clearly indented from their surrounding constructs. INVOKING SUBPROCEDURES: Using the CALL keyword. 36 18
- 19. Flowcharts (cont…) Step-form and pseudocode program designs are both text-based, the statements are written. Flow charts are a graphical method of designing programs and once the rules are learned are very easy to draw. 37 Flowcharts A well-drawn flow chart is also very easy to read since it basically uses just two symbols, two decision constructs. and two iteration constructs: the sequence symbol, the decision symbol, the decision construct if ... then the decision construct if ... then ... else the repetition construct - repeat the repetition construct - while 38 19
- 20. Flowcharts (cont…) 39 Flowcharts (cont…) Every flow chart has a START symbol and a STOP symbol The flow of sequence is generally from the top of the page to the bottom of the page. This can vary with loops which need to flow back to an entry point. Use arrow-heads on connectors where flow direction may not be obvious. There is only one flow chart per page A page should have a page number and a title A flow chart on one page should not break and jump to another page A flow chart should have no more than around 15 symbols (not including START and STOP) 40 20
- 21. Flowcharts (cont…) The major symbols are the DECISION (also known as selection) and the SEQUENCE (or process) symbols. The START and STOP symbols are called the terminals. The SUBPROCESS symbol is a variation on the sequence symbol. Connectors are drawn between the symbols There must be at least one other sequence symbol to represent input/output processes 41 Flowcharts (cont…) Processes can have only one entry point and one exit point. Decisions have only one entry point, one TRUE exit point and one FALSE exit point. Repeat loop: repeat loop has the process preceding the decision; a repeat loop will always execute the process part at least once. While loop is the reverse of the repeat loop; the decision comes first, followed by the process. IF ... THEN construct is also known as the NULL ELSE, meaning that there is no ELSE part. IF ... THEN ... ELSE ... construct has a process at each branch of the decision symbol. 42 21
- 22. Topic Questions What is Algorithms and how it relates to computerized programming? Can you a problem with two solutions? which one of the solution you think it is more efficient? What is the difference between time complexity and space complexity? What is flowchart and how can be used to solve a problem? 43 References Michael Main, Data Structures and Other Objects Using Java (Third Edition) Abdisalam Issa-Salwe, Taibah University, Madinah, Saudi Arabia. 44 22