Preparation Data Structures 01 introduction

1. Data Structures Introduction Andres Mendez-Vazquez May 16, 2015 1 / 50

2. Images/ITESM.p Outline 1 What the Course is About? 2 What is a Good Algorithm? Sorting A Naive Algorithm Counting Steps A more realistic step count Deﬁnition of Big O Meaning for insertion sort 3 Examples of Complexities Problems Calculating Complexities 2 / 50

3. Images/ITESM.p Introduction Data Manipulation Data structures is concerned with the representation and manipulation of data. Further All programs manipulate data. Thus!!! So, all programs represent data in some way. Data manipulation requires a data structure and an algorithm!!! 3 / 50

7. Images/ITESM.p Examples Data Bases Using B-trees for fast access to the records!!! 4 / 50

8. Images/ITESM.p Example Storing Sparse Matrices 5x5 Matrix Numeric Elements Empty Elements Sparse Matrix Why? Reduce the amount memory used for storage!!! 5 / 50

9. Images/ITESM.p Example Storing Sparse Matrices 5x5 Matrix Numeric Elements Empty Elements Sparse Matrix Why? Reduce the amount memory used for storage!!! 5 / 50

10. Images/ITESM.p Example Shortest Path in a Map 6 / 50

11. Images/ITESM.p What is the course about? First Understanding Data Structure and Algorithms needed to develop programs for data manipulation. Why? The study of Data Structures and Algorithms is fundamental to Computer Science. 7 / 50

12. Images/ITESM.p What is the course about? First Understanding Data Structure and Algorithms needed to develop programs for data manipulation. Why? The study of Data Structures and Algorithms is fundamental to Computer Science. 7 / 50

14. Images/ITESM.p In sorting, we want to have the following Rearrange a sequence of numbers 9,3,4,2,1 Into an increasing sequence 1, 2, 3, 4, 9 Or Into a decreasing sequence 9, 4, 3, 2, 1 9 / 50

17. Images/ITESM.p Notation For now, we will assume that the data It is an array A. Meaning item 9 3 4 2 1 index 0 1 2 3 4 Thus If we need to reference elements on the array through an index, we use the following notation A [index] (1) 10 / 50

21. Images/ITESM.p Naive Algorithm What if we do the following 1 Pick an element in the array A. 2 Put that element in the correct position. Thus We will concentrate ﬁrst in inserting the element in the correct position For this imagine the following We already have an array A = [1|3|6|4| − |−] where “-” represents null elements. 12 / 50

25. Images/ITESM.p Thus Where do we start comparing to insert? Ideas? Maybe Let us do it with A = [1|3|6|4| − |−] 13 / 50

26. Images/ITESM.p Thus Where do we start comparing to insert? Ideas? Maybe From the right!!! After all we have space Let us do it with A = [1|3|6|4| − |−] 13 / 50

27. Images/ITESM.p Thus Where do we start comparing to insert? Ideas? Maybe From the right!!! After all we have space Let us do it with A = [1|3|6|4| − |−] 13 / 50

28. Images/ITESM.p What about the code? What do we use for going through a sequence? Ideas? In addition, we need to make space For the new item. Code 14 / 50

29. Images/ITESM.p What about the code? What do we use for going through a sequence? Ideas? In addition, we need to make space For the new item. Code 14 / 50

30. Images/ITESM.p What about the code? What do we use for going through a sequence? Ideas? In addition, we need to make space For the new item. Code p r i v a t e i n t [ ] i n s e r t ( i n t [ ] A, i n t t ){ // Extra V a r i a b l e s i n t j ; // I n s e r t i n t o A [ 0 : i −1] f o r ( j = A. lenght −1; j >=0 && t<A[ j ] ; j −−){ // s h i f t to the r i g h t A[ j +1]=A[ j ] ; } A[ j +1]=t ; r e t u r n A; } 14 / 50

31. Images/ITESM.p Ok, We have that...Now what? How do we use this piece of code for the insertion sort? Ideas? Problem with the code!! You require that the array where you insert to be sorted!!! Look at this!!! At the Board... Thus How do we solve this problem? 15 / 50

34. Images/ITESM.p Then, Insertion Sort works like this First Start with a sequence of size 1. After all an array with one element is sorted!!! Second Repeatedly insert remaining elements Example Sort 7, 3, 5, 6, 1 16 / 50

37. Images/ITESM.p Final Code Something Notable // Sort A assume i s f u l l p u b l i c i n t [ ] I n s e r t i o n S o r t ( i n t [ ] A){ // Extra Space i n t B [ ] = new i n t [A. l e n g t h ] ; // I n i t Array B B[0]=A [ 0 ] ; i n t s i z e = 1; i n t j , t ; f o r ( i n t i = 1; i < A. l e n g t h ; i ++){ t = A[ i ] ; f o r ( j = s i z e −1; j >=0 && t<B[ j ] ; j −−) { // s h i f t to the r i g h t B[ j +1]=B[ j ] ; } B[ j +1]=t ; s i z e ++; } r e t u r n B; } 17 / 50

39. Images/ITESM.p Question: Which Complexity You Have with Insertion Sort? After all you have!!! Time Complexity!!! For this, we have Time Complexity Techniques: Count a particular operation Count number of steps Asymptotic complexity 19 / 50

44. Images/ITESM.p Basically, we will use the second one First Look at this part of the code f o r ( j = s i z e −1; j >=0 && t<B[j] ; j −−) B[ j +1]=B[ j ] ; Then How many comparisons are made? 20 / 50

45. Images/ITESM.p Basically, we will use the second one First Look at this part of the code f o r ( j = s i z e −1; j >=0 && t<B[j] ; j −−) B[ j +1]=B[ j ] ; Then How many comparisons are made? 20 / 50

46. Images/ITESM.p The Possible Cases Worst-case count Maximum count Best-case count Minimum count In addition, we have Average Count 21 / 50

49. Images/ITESM.p Example: Worst-Case For the Code Code, Look at the red compares f o r ( j = s i z e −1; j >=0 && t<B[j] ; j −−) B[ j +1]=B[ j ] ; Case I If B = [1, 2, 3, 4] and t = 0 =⇒ 4 compares. What about the general case If B = [1, 2, 3, ..., i] and t = 0 =⇒ i compares. 22 / 50

52. Images/ITESM.p Worst-Case For An Almost Final Version Simpliﬁed Version Remember s i z e i s the s i z e of B f o r ( i = 2; i < A. l e n g t h ; i++ ) f o r ( j = s i z e −1; j >=0 && t<B[j] ; j −−) B[ j +1]=B[ j ] ; Worst Case for the Outer Loop 1 Outer loop takes n steps What about the inner loop? Answer 23 / 50

53. Images/ITESM.p Worst-Case For An Almost Final Version Simpliﬁed Version Remember s i z e i s the s i z e of B f o r ( i = 2; i < A. l e n g t h ; i++ ) f o r ( j = s i z e −1; j >=0 && t<B[j] ; j −−) B[ j +1]=B[ j ] ; Worst Case for the Outer Loop 1 Outer loop takes n steps What about the inner loop? Answer 23 / 50

54. Images/ITESM.p Worst-Case For An Almost Final Version Simpliﬁed Version Remember s i z e i s the s i z e of B f o r ( i = 2; i < A. l e n g t h ; i++ ) f o r ( j = s i z e −1; j >=0 && t<B[j] ; j −−) B[ j +1]=B[ j ] ; Worst Case for the Outer Loop 1 Outer loop takes n steps What about the inner loop? Answer Number of steps in the inner loop = size+1 23 / 50

55. Images/ITESM.p What is the range of j? Very Simple For the ﬁrst step j = 1 − 1 =⇒Number of Steps = 2 For the last step j = n − 1 − 1 =⇒Number of Steps = n Total Compares 2 + 3 + 4 + ... + n = n (n + 1) 2 − 1 (2) 24 / 50

56. Images/ITESM.p What is the range of j? Very Simple For the ﬁrst step j = 1 − 1 =⇒Number of Steps = 2 For the last step j = n − 1 − 1 =⇒Number of Steps = n Total Compares 2 + 3 + 4 + ... + n = n (n + 1) 2 − 1 (2) 24 / 50

58. Images/ITESM.p A more realistic step count Counting when A.length = n // Sort A assume i s f u l l p u b l i c i n t [ ] I n s e r t i o n S o r t ( i n t [ ] A){ Step // I n i t i a l V a r i a b l e s 0 i n t B [ ] = new i n t [A. l e n g t h ] ; 1 i n t s i z e = 1; 1 i n t i , j , t ; 1 // I n i t i a l i z e the Array B 0 B[0]=A [ 0 ] ; 1 f o r ( i = 1; i < A. l e n g t h ; i ++){ n t = A[ i ] ; n−1 f o r ( j=s i z e −1; j>=0&&t<B[ j ] ; j −−) i +1 { // s h i f t to the r i g h t 0 B[ j +1]=B[ j ] ; } i B[ j +1]=t ; n−1 s i z e ++; n−1 } r e t u r n B; 1 } 26 / 50

59. Images/ITESM.p The Result Step count for body of for loop is 6 + 3 (n − 1) + n + n−1 i=1 (i + 1) + n−1 j=1 (i) (3) The summation They have the quadratic terms n2. Complexity Insertion sort complexity is O n2 27 / 50

63. Images/ITESM.p Deﬁnition of Big O Deﬁnition For a given function g(n) O(g(n)) ={f (n)| There exists c > 0 and n0 > 0 s.t. 0 ≤ f (n) ≤ cg(n) ∀n ≥ n0} Graphically 29 / 50

64. Images/ITESM.p Deﬁnition of Big O Deﬁnition For a given function g(n) O(g(n)) ={f (n)| There exists c > 0 and n0 > 0 s.t. 0 ≤ f (n) ≤ cg(n) ∀n ≥ n0} Graphically 29 / 50

65. Images/ITESM.p What? The term n0 It tells you when for the following n s you will have that f (n) ≤ cg(n) What about the so called c It can be seen a way to “increase” or “decrease” the height of the function!!! 30 / 50

69. Images/ITESM.p What does this means for insertion sort? We have 6 + 3 (n − 1) + n + n−1 i=1 (i + 1) + n−1 j=1 (i) = ... 3 + 4n + n (n − 1) 2 + n − 1 + n (n − 1) 2 = ... 2 + 5n + n(n − 1) = ... n2 + 4n + 2 ≤ n2 + 4n2 + 2n2 Thus n2 + 4n + 2 ≤ 7n2 (4) With Tinsertion(n) = n2 + 4n + 2 describing the number of steps for insertion when we have n numbers. 32 / 50

70. Images/ITESM.p What does this means for insertion sort? We have 6 + 3 (n − 1) + n + n−1 i=1 (i + 1) + n−1 j=1 (i) = ... 3 + 4n + n (n − 1) 2 + n − 1 + n (n − 1) 2 = ... 2 + 5n + n(n − 1) = ... n2 + 4n + 2 ≤ n2 + 4n2 + 2n2 Thus n2 + 4n + 2 ≤ 7n2 (4) With Tinsertion(n) = n2 + 4n + 2 describing the number of steps for insertion when we have n numbers. 32 / 50

71. Images/ITESM.p Actually For n0 = 2 22 + 4 × 2 + 2 = 14 < 7 × 22 = 28 (5) Graphically 33 / 50

72. Images/ITESM.p Actually For n0 = 2 22 + 4 × 2 + 2 = 14 < 7 × 22 = 28 (5) Graphically 33 / 50

73. Images/ITESM.p Meaning First Time or number of operations does not exceed cn2 for a constant c on any input of size n (n suitably large). Questions Is O(n2) too much time? Is the algorithm practical? For this imagine, we have a machine able to make 109instructions/seconds 34 / 50

77. Images/ITESM.p Then We have the following n n n log n n2 n3 n4 1000 1 micros 10 micros 1 milis 1 second 17 minutes 10,000 10 micros 130 micros 100 milis 17 minutes 116 days 106 1 milis 20 milis 17 minutes 32 years 3 × 107 years It is much worse 35 / 50

78. Images/ITESM.p Then We have the following n n n log n n2 n3 n4 1000 1 micros 10 micros 1 milis 1 second 17 minutes 10,000 10 micros 130 micros 100 milis 17 minutes 116 days 106 1 milis 20 milis 17 minutes 32 years 3 × 107 years It is much worse n n10 2n 1000 3.2 × 1013 years 3.2 × 10283 years 10,000 ??? ??? 106 ????? ????? The Reign of the Non Polynomial Algorithms 35 / 50

79. Images/ITESM.p Basically These complexities allow us to compare algorithms You can compare 2 algorithms having diﬀerent asymptotic complexity For example two types of sorting algorithms with O(n) and O(n2) complexities. However This notation does not account for constant factors. For example For many cases of n, 1010n is much worse than n2. 36 / 50

83. Images/ITESM.p A Huge Problem for calculating complexities The memory hierachy: Slower Memories as further you go from the CORE Main Memory L3 Cache L2 Cache L1i Cache L1d Cache CPU Core BUS Note: I will recommend to read “What Every Programmer Should Know About Memory” by Ulrich Drepper Red Hat, Inc. 38 / 50

84. Images/ITESM.p Examples B-Trees are generalized Binary Search Trees With element in the nodes sorted in increasing order 39 / 50

85. Images/ITESM.p Example We can improve B-Tree policies to ﬁnd indexes inside nodes by Binary Search Linear Search 40 / 50

86. Images/ITESM.p Example We can improve B-Tree policies to ﬁnd indexes inside nodes by Binary Search Linear Search 40 / 50

87. Images/ITESM.p Here, a reasonable assumption!!! Classic Analysis Binary Search Faster than Linear Search!!! Node Structure a(i) ≤ a(i+1) ∀i However, Core i7 has prefetching Thus, for a certain size of a node, linear searching will be faster than binary search due to prefetching hardware!!! 41 / 50

90. Images/ITESM.p Main Problem Main problem Our analysis does not account for this diﬀerence in memory access times. Thus What do we need? What? A way to measure the time in our Java programs. 42 / 50

93. Images/ITESM.p What do we need? Data 1 Worst-case data 2 Best-case data 3 Average-case data In addition A way to measure the time in a machine and language Problem We require certain degree of accuracy that we do not have 43 / 50

96. Images/ITESM.p What do we have? Java Instruction System.currentTimeMillis() It returns the current time in milliseconds. The granularity of the value depends on the underlying operating system and may be larger. Yes!!! The accuracy is not great for it!!! 44 / 50

100. Images/ITESM.p Possible Solution What to do? Ideas? What about this logic? 1 Assume that you have a 100 millisecond accuracy in a Linux System. 2 Assume that you want to have a measurement error of 10%. 3 What do we do? 45 / 50

104. Images/ITESM.p Simple Idea for Accuracy If we want to have a 10% error We need to measure 100% of the total time. In the case of 100 milliseconds We need to measure operations for ≥ 1000 milliseconds!!! Or trueElapsedTime = ﬁnishTime - startTime +/- Error 46 / 50

107. Images/ITESM.p First Solution Repeat method until you accumulate enough time For this, if you have 100 milliseconds for the code to be measured, then you need a Time≥1000 milliseconds to obtain an accuracy of 10%. First we device the timing code 47 / 50

108. Images/ITESM.p First Solution Repeat method until you accumulate enough time For this, if you have 100 milliseconds for the code to be measured, then you need a Time≥1000 milliseconds to obtain an accuracy of 10%. First we device the timing code // g i v e s time i n m i l l i s e c o n d s s i n c e 1/1/1970 GMT long startTime = System . c u r r e n t T i m e M i l l i s ( ) ; // code to be timed comes here long elapsedTime = System . c u r r e n t T i m e M i l l i s ()− startTime ; 47 / 50

109. Images/ITESM.p What is the problem with this Solution? PROBLEM!!! long startTime = System . c u r r e n t T i m e M i l l i s ( ) ; long counter ; // Put code to i n i t i a l i z e a [ ] here // Go f o r randomized a r r a y do{ counter++; SomeClass . I n s e r t i o n S o r t (A ) ; } w h i l e ( System . c u r r e n t T i m e M i l l i s ()− startTime < 1000) long elapsedTime = System . c u r r e n t T i m e M i l l i s ()− startTime ; f l o a t timeForMethod = (( f l o a t ) elapsedTime )/ counter ; 48 / 50

110. Images/ITESM.p Fix Solution long startTime = System . c u r r e n t T i m e M i l l i s ( ) ; long counter ; do{ counter++; // Move the code to i n i t i a l i z e a [ ] here // Go f o r randomized a r r a y SomeClass . I n s e r t i o n S o r t (A ) ; } w h i l e ( System . c u r r e n t T i m e M i l l i s ()− startTime < 1000) long elapsedTime = System . c u r r e n t T i m e M i l l i s ()− startTime ; f l o a t timeForMethod = (( f l o a t ) elapsedTime )/ counter ; 49 / 50

111. Images/ITESM.p Another Way to Measure Time In Unix/Linux Systems, we have the following command ~$ time MyProgram 50 / 50

Preparation Data Structures 01 introduction

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Preparation Data Structures 01 introduction (20)

More from Andres Mendez-Vazquez (20)

Recently uploaded (20)

Preparation Data Structures 01 introduction