![Analysis big-Oh
Running time
PROGRAM OPERATIONS STEPS
int sum = 0; 1 assignment 1
for (i=0;i<128;i=++) i =1, 2, 3, 4 … 128 128
for (j = 128; j>0; j=j/2) j = 128,64,32,16,8,4,2,1 log2128+1
sum = sum + a[i][j]; 1 addition, 1 assignment 2
1+128*(log2128+1)*2
n could be the size of the stack, queue, list or the dimension of a matrix, etc.
In general we have an arbitrary number “n” instead of 128, in that case:
1+n*(log2n+1)*2 can we simplify the expression?
1+2n+2n*log2n YES!, By using big-Oh notation
we can specify the asymptotic
complexity of the algorithm
4](https://image.slidesharecdn.com/basic-data-structure-part-i-unsplitted-121106082817-phpapp02/85/unsplitted-slideshare-4-320.jpg)
![Arrays
• Contiguous blocks of memory of a data type
o Any position can be randomly access
• Example
o Int[] integer_array = new int[1024];
int size
0 1023
Java takes care of the
memory management
o ObjectY[] object_y_array = new ObjectY[512]; for you.
1,934,218
ObjectY size ObjectZ
ObjectX 0 511 0
… …
512 ObjectsY
fixed boundary 12](https://image.slidesharecdn.com/basic-data-structure-part-i-unsplitted-121106082817-phpapp02/85/unsplitted-slideshare-12-320.jpg)
unsplitted slideshare
- 1. Data structures & algorithms Basics: Part I By Daniel Gomez-Prado Sept 2012 Disclaimer: This tutorial may contain errors, use it at your own discretion. The slides were prepared for a class review on basic data structures at University of Massachusetts, Amherst. http://www.dgomezpr.com
- 2. Outline • Analysis of complexity o Q1 Fall 2011 problems • Classes, objects and containers • Array • Stack • Queue • (Single) Linked and Double Linked List • Iterators • Linear and Binary search • Merge sort • Quick sort • Q2 – Q6, Fall 2011 problems 2
- 3. Analysis big-Oh Random access memory Your program Memory Import java.util.* class review { Static public main() { // this is a review // for ECE242 exam } } • Assumptions: o Unlimited memory We have what we need: 1 Gb or 1,000 Tb No hierarchical memory, o All memory accesses takes 1 unit time Cache, L1, L2, hard drive 3
- 4. Analysis big-Oh Running time PROGRAM OPERATIONS STEPS int sum = 0; 1 assignment 1 for (i=0;i<128;i=++) i =1, 2, 3, 4 … 128 128 for (j = 128; j>0; j=j/2) j = 128,64,32,16,8,4,2,1 log2128+1 sum = sum + a[i][j]; 1 addition, 1 assignment 2 1+128*(log2128+1)*2 n could be the size of the stack, queue, list or the dimension of a matrix, etc. In general we have an arbitrary number “n” instead of 128, in that case: 1+n*(log2n+1)*2 can we simplify the expression? 1+2n+2n*log2n YES!, By using big-Oh notation we can specify the asymptotic complexity of the algorithm 4
- 5. Analysis big-Oh Definition • Given functions f(n) and g(n): o f(n) is said to be O(g(n)) o if and only if • there are (exist) 2 positive constants, C>0 and N>0 o such that • f(n) ≤ Cg(n) for every n>N 5
- 6. Analysis big-Oh Example of definition usage keywords f(n) is given g(n) is given Relationship between C & n O(n*log2n) is true for C=3 and n≥32 6
- 7. Analysis big-Oh Example 1 State the asymptotic complexity of: big-Oh i. Print out middle element of an array of size n arrays allow access to any position randomly recall Your program Memory Solution is: O(1) o All memory accesses takes 1 unit time 7
- 8. Analysis big-Oh Example 1 ii. Print out the middle element of a linked list of size n recall a linked list head tail next next next next n/2 object object object memory object locations f(n) = n/2 Solution is: the asymptotic complexity is O(n) 8
- 9. Analysis big-Oh Example 1 iii. Print out the odd elements of an array of size n f(n) = n/2 Solution: the asymptotic complexity is O(n) iv. Pop 10 elements from a stack that is implemented with an array. Assume that the stacks contains n elements and n > 10. When in doubt, ASK! is n = 11 or is n > 1000 ? f(n) = 10 Solution: the asymptotic complexity is O(1) 9
- 10. Classes and objects • The goal of a “class” (in object-oriented language) o Encapsulate state and behavior • A class is a blueprint that has o a constructor to initialize its data members o a destructor to tear down the object o A coherent interface to interact with the object (public methods) o Private methods unreachable from the outside o The possibility to extend and inherit members from other classes • An object is an instant of a class • What are the benefits: o Through inheritance, extensions, packages allows to structure a program o Exposes behavior that could be reused o Alleviates the problem of understanding somebody else code 10
- 11. ADT (Abstract Data Type) • ADTs are containers • ADTs are primarily concern in: o Aggregation of data o Access of data o Efficiency of memory is used o Efficiency of the container access 11
- 12. Arrays • Contiguous blocks of memory of a data type o Any position can be randomly access • Example o Int[] integer_array = new int[1024]; int size 0 1023 Java takes care of the memory management o ObjectY[] object_y_array = new ObjectY[512]; for you. 1,934,218 ObjectY size ObjectZ ObjectX 0 511 0 … … 512 ObjectsY fixed boundary 12
- 13. Stacks • Enforce a LIFO behavior (last in, first out) o It is based on an array o It overrides the random access of an array by a LIFO access 1023 1023 1023 isEmpty isEmpty isEmpty true false false peek push pop pop status peek index push 0 0 push 0 13
- 14. Stacks … ObjectZ • Enforce a LIFO behavior (last in, first out) o It is based on an array o It overrides the random access of an array by a LIFO access 0 1024 1023 Recall what a class encapsulates isEmpty o Status & false o Behavior pop Does it mean we are always safe o index = -1, stack is empty, good o index = 1024, peek index o refuse to push objects o overflow, runtime exception 0 14 -1
- 15. Queues • Enforce a FIFO behavior (first in, first out) o It is based on an array o It overrides the random access of an array by a FIFO access 1023 1023 1023 isEmpty isEmpty isEmpty true false false status peek enqueue peek index1 index2 dequeue enqueue 0 0 enqueue 0 dequeue 15
- 16. Queues • Enforce a FIFO behavior (first in, first out) o It is based on an array o It overrides the random access of an array by a FIFO access 1023 Recall what a class encapsulates isEmpty o Status & false status o Behavior peek index1 Does it mean we are always safe > o index1 = index2, stack is empty, good index2 o index1 or index2 = 1024, dequeue o rewind to 0 o test condition o Increment using mod 1024 o What if index2 > index1 0 dequeue 16
- 17. Queues • Enforce a FIFO behavior (first in, first out) o It is based on an array o It overrides the random access of an array by a FIFO access 1023 1023 status index2 status peek index1 > > index2 peek index1 dequeue 0 0 dequeue 17
- 18. Is everything an Array? can we use something else? beginning end • Recall an array o Contiguous memory ObjectX 0 ObjectY N-1 0 ObjectZ … … o Fixed bound size fixed boundary o Random access How do we know in this container? • Let’s use another construct o the beginning, the end o which element is next o Non contiguous memory o Unlimited size ObjectX ObjectY ObjectZ … o Sequential access only … head next edges prev next Node head object 18
- 19. Linked List • Use the prior construct (node and edge) head next next next next … object object object push next pop peek object 19
- 20. Double linked List • Use the prior construct (node and edge) head prev next prev next prev next … prev next object object object 20
- 21. Quick Questions Can we do: • a linked list or double linked list from an array o Yes • a queue with nodes and edges o Why not. • a stack with nodes and edges o Sure 21
- 22. Iterators encapsulate container traversals • we have two implementations of a stack 1023 peek push pop pop prev next prev next head prev next prev 4 next peek index 1 2 3 4 prev 3 next prev 2 next 0 push prev 1 next 22
- 23. Iterators encapsulate container traversals • we have two implementations of a stack Traverse container behavior 1023 according to container rules state update next update prev prev next increment by 1 head decrement by 1 prev next prev 5 next peek index 1 2 3 4 5 prev 4 next prev 3 next prev 2 next 0 push prev 1 next 23
- 24. Searching Linear vs Binary • If you make no assumptions o iterate (traverse) all elements to find an existing element o iterate (traverse) all elements to realize you don’t have an element Looking for u worst case all elements a x z b n m l j i b c u are visited. O(n) • If you assume the container is already order (according to certain rule) o Iterate back/forth skipping some elements to speed up the process worst case there are log(n)+1 a b b c i j l n m u x z element visited. O(log(n)) Looking for u 24
- 25. So binary search is faster but the assumption is… • The container is already order, so o how do we sort a container o how do insert elements in a sorted container o How do we remove elements in a sorted container • What is more expensive (big-Oh) o A linear search o Order a container and then a binary search o Maintain a container sorted and then a binary search 25
- 26. Sorting a container Merge sort • Use the divide and conquer approach o Divide the problem into 2 subsets (unless you have a base case) o Recursively solve each subset o Conquer: Solve the sub-problem and merge them into a solution 85 24 63 45 19 37 91 56 DIVIDE Wait, what? 24 45 85 24 63 45 85 19 37 91 56 CONQUER RECUR 85 85 24 24 63 63 45 45 take the min 85 85 24 24 45 24 63 63 45 26
- 27. Sorting a container Merge sort • What is the complexity of merge sort o Divide the problem into 2 subsets (2 times half the problem) o Recursively solve each subset (keep subdividing the problem) o Conquer: Solve the sub-problem and merge (the merge running time) f(n) 85 24 63 45 19 37 91 56 2f(n/2) 85 24 63 45 19 37 91 56 4f(n/4) 85 24 63 45 log2(n) 24 85 x elements O(x+y) 24 45 45 63 y elements 27
- 28. Sorting a container Merge sort O(nlogn) 28
- 29. Sorting a container Merge sort • Drawback of merge sort algorithm o The merge is not in place take the min Additional memory 85 24 45 63 • The merge could be modified to be in place, but the overhead will slow down the running time. 29
- 30. Sorting a container Quick sort • Use the divide and conquer approach o Divide the problem into 3 subsets (unless you have a base case) • A (random) pivot x • A subset with numbers lower than x • A subset with numbers greater than x o Recursively solve each subset o Conquer: Solve the sub-problem and merge them into a solution 85 24 63 19 37 91 56 45 24 37 19 85 63 91 56 In place sorting, 24 37 it does not require additional memory 30
- 31. Sorting a container Quick sort • Complexity o Quick sort (with random pivot) is O(nlogn) • Drawback o No replicated element allowed in the container * The pivot is randomly chosen, * if an element occurs twice, in different divisions * then the merging mechanism won’t work 31
- 32. Arrays Example 2 Accept 2 integer Arrays: A and B. And find the number of common elements in both assuming no duplicates in each array. o Brute force A, n elements O(nm) B, m elements o Merge-sort modified C, n+m elements Instead of merging compare and increment count when equal O( (n+m)log(n+m) ) 32
- 33. Stacks and Queues Example 3 a) Write a reverseQueue method using only stacks and queues in in out in a b c d e f g h a b c d e f g h h g f e d c b a O(n) out out Queue Stack Queue FIFO LIFO FIFO 33
- 34. Stacks and Queues Example 4 b) Write a method cutQueue that adds an element to the head of the queue using only stacks and queues N O(n) in in out in out in h g f e d c b a N a b c d e f g h N a b c d e f g h N a b c d e f g h out out Queue FIFO Stack Stack Queue LIFO LIFO FIFO 34
- 35. List Example 5 Write a method is_Sorted_Ascedent to check if a single linked list is sorted in non-decreasing order head next next next … next Java pseudo code while ( node.next ) { if (node.next.key < node.key) return false; node = node.next; } return true; 35
- 36. List Example 6 Write a method compress to remove duplicated elements in a single linked list head next next next … next Java pseudo code while ( node.next ) { if ( node.key == node.next.key) ) { node.next = node.next.next; } else { node = node.next; } } 36