![Machine-independent time: An exampleMachine-independent time: An example
A pseudocode for insertion sort ( INSERTION SORT ).
INSERTION-SORT(A)
1 for j ← 2 to length [A]
2 do key ← A[ j]
3 ∇ Insert A[j] into the sortted sequence A[1,..., j-1].
4 i ← j – 1
5 while i > 0 and A[i] > key
6 do A[i+1] ← A[i]
7 i ← i – 1
8 A[i +1] ← key](https://image.slidesharecdn.com/algorithm-180405121038/85/Algorithm-23-320.jpg)
![Analysis of INSERTION-SORT(contd.)Analysis of INSERTION-SORT(contd.)
1]1[8
)1(17
)1(][]1[6
][05
114
10]11[sequence
sortedtheinto][Insert3
1][2
][21
timescostSORT(A)-INSERTION
8
27
26
25
4
2
1
−←+
−−←
−←+
>>
−−←
−−⋅⋅
∇
−←
←
∑
∑
∑
=
=
=
nckeyiA
tcii
tciAiA
tckeyiAandi
ncji
njA
jA
ncjAkey
ncAlengthj
n
j j
n
j j
n
j j
do
while
do
tofor](https://image.slidesharecdn.com/algorithm-180405121038/85/Algorithm-24-320.jpg)
Algorithm
- 2. IntroductionIntroduction • The methods of algorithm design form one of the core practical technologies of computer science. • The main aim of this lecture is to familiarize the student with the framework we shall use through the course about the design and analysis of algorithms. • We start with a discussion of the algorithms needed to solve computational problems. The problem of sorting is used as a running example. • We introduce a pseudocode to show how we shall specify the algorithms.
- 3. AlgorithmsAlgorithms • The word algorithm comes from the name of a Persian mathematician Abu Ja’far Mohammed ibn-i Musa al Khowarizmi. • In computer science, this word refers to a special method useable by a computer for solution of a problem. The statement of the problem specifies in general terms the desired input/output relationship. • For example, sorting a given sequence of numbers into nondecreasing order provides fertile ground for introducing many standard design techniques and analysis tools.
- 4. The problem of sortingThe problem of sorting
- 6. Example of Insertion SortExample of Insertion Sort
- 7. Example of Insertion SortExample of Insertion Sort
- 8. Example of Insertion SortExample of Insertion Sort
- 9. Example of Insertion SortExample of Insertion Sort
- 10. Example of Insertion SortExample of Insertion Sort
- 11. Example of Insertion SortExample of Insertion Sort
- 12. Example of Insertion SortExample of Insertion Sort
- 13. Example of Insertion SortExample of Insertion Sort
- 14. Example of Insertion SortExample of Insertion Sort
- 15. Example of Insertion SortExample of Insertion Sort
- 16. Example of Insertion SortExample of Insertion Sort
- 17. Analysis of algorithmsAnalysis of algorithms The theoretical study of computer-program performance and resource usage. What’s more important than performance? • modularity • correctness • maintainability • functionality • robustness • user-friendliness • programmer time • simplicity • extensibility • reliability
- 18. Analysis of algorithmsAnalysis of algorithms Why study algorithms and performance? • Algorithms help us to understand scalability. • Performance often draws the line between what is feasible and what is impossible. • Algorithmic mathematics provides a language for talking about program behavior. • The lessons of program performance generalize to other computing resources. • Speed is fun!
- 19. Running TimeRunning Time • The running time depends on the input: an already sorted sequence is easier to sort. • Parameterize the running time by the size of the input, since short sequences are easier to sort than long ones. • Generally, we seek upper bounds on the running time, because everybody likes a guarantee.
- 20. Kinds of analysesKinds of analyses Worst-case: (usually) • T(n) = maximum time of algorithm on any input of size n. Average-case: (sometimes) • T(n) = expected time of algorithm over all inputs of size n. • Need assumption of statistical distribution of inputs. Best-case: • Cheat with a slow algorithm that works fast on some input.
- 21. Machine-Machine-IIndependent timendependent time The RAM Model Machine independent algorithm design depends on a hypothetical computer called Random Acces Machine (RAM). Assumptions: • Each simple operation such as +, -, if ...etc takes exactly one time step. • Loops and subroutines are not considered simple operations. • Each memory acces takes exactly one time step.
- 22. Machine-independent timeMachine-independent time What is insertion sort’s worst-case time? • It depends on the speed of our computer, • relative speed (on the same machine), • absolute speed (on different machines). BIG IDEA: • Ignore machine-dependent constants. • Look at growth of “Asymptotic Analysis” ∞→nnT as)(
- 23. Machine-independent time: An exampleMachine-independent time: An example A pseudocode for insertion sort ( INSERTION SORT ). INSERTION-SORT(A) 1 for j ← 2 to length [A] 2 do key ← A[ j] 3 ∇ Insert A[j] into the sortted sequence A[1,..., j-1]. 4 i ← j – 1 5 while i > 0 and A[i] > key 6 do A[i+1] ← A[i] 7 i ← i – 1 8 A[i +1] ← key
- 24. Analysis of INSERTION-SORT(contd.)Analysis of INSERTION-SORT(contd.) 1]1[8 )1(17 )1(][]1[6 ][05 114 10]11[sequence sortedtheinto][Insert3 1][2 ][21 timescostSORT(A)-INSERTION 8 27 26 25 4 2 1 −←+ −−← −←+ >> −−← −−⋅⋅ ∇ −← ← ∑ ∑ ∑ = = = nckeyiA tcii tciAiA tckeyiAandi ncji njA jA ncjAkey ncAlengthj n j j n j j n j j do while do tofor
- 25. Analysis of INSERTION-SORT(contd.)Analysis of INSERTION-SORT(contd.) )1()1()1()( 2 6 2 5421 −++−+−+= ∑∑ == n j j n j j tctcncnccnT ).1()1( 8 2 7 −+−+ ∑ = nctc n j j The total running time is
- 26. Analysis of INSERTION-SORT(contd.)Analysis of INSERTION-SORT(contd.) The best case: The array is already sorted. (tj =1 for j=2,3, ...,n) )1()1()1()1()( 85421 −+−+−+−+= ncncncncncnT ).()( 854285421 ccccnccccc +++−++++=
- 27. Analysis of INSERTION-SORT(contd.)Analysis of INSERTION-SORT(contd.) •The worst case: The array is reverse sorted (tj =j for j=2,3, ...,n). )12/)1(()1()( 521 −++−+= nncncncnT )1()2/)1(()2/)1(( 876 −+−+−+ ncnncnnc ncccccccnccc )2/2/2/()2/2/2/( 8765421 2 765 +−−++++++= 2 )1( 1 + =∑ = nn j n j cbnannT ++= 2 )(
- 28. Growth of FunctionsGrowth of Functions Although we can sometimes determine the exact running time of an algorithm, the extra precision is not usually worth the effort of computing it. For large inputs, the multiplicative constants and lower order terms of an exact running time are dominated by the effects of the input size itself.
- 29. Asymptotic NotationAsymptotic Notation The notation we use to describe the asymptotic running time of an algorithm are defined in terms of functions whose domains are the set of natural numbers { }...,2,1,0=N
- 30. O-notationO-notation • For a given function , we denote by the set of functions • We use O-notation to give an asymptotic upper bound of a function, to within a constant factor. • means that there existes some constant c s.t. is always for large enough n. )(ng ))(( ngO ≥≤≤ = 0 0 allfor)()(0 s.t.andconstantspositiveexistthere:)( ))(( nnncgnf ncnf ngO ))(()( ngOnf = )(ncg≤)(nf
- 31. ΩΩ--OmegaOmega notationnotation • For a given function , we denote by the set of functions • We use Ω-notation to give an asymptotic lower bound on a function, to within a constant factor. • means that there exists some constant c s.t. is always for large enough n. )(ng ))(( ngΩ ≥≤≤ =Ω 0 0 allfor)()(0 s.t.andconstantspositiveexistthere:)( ))(( nnnfncg ncnf ng ))(()( ngnf Ω= )(nf )(ncg≥
- 32. --ThetaTheta notationnotation • For a given function , we denote by the set of functions • A function belongs to the set if there exist positive constants and such that it can be “sand- wiched” between and or sufficienly large n. • means that there exists some constant c1 and c2 s.t. for large enough n. )(ng ))(( ngΘ ≥≤≤≤ =Θ 021 021 allfor)()()(c0 s.t.and,,constantspositiveexistthere:)( ))(( nnngcnfng nccnf ng )(nf ))(( ngΘ 1c 2c )(1 ngc )(2 ngc Θ ))(()( ngnf Θ= )()()( 21 ngcnfngc ≤≤
- 33. Asymptotic notationAsymptotic notation Graphic examples of and .ΩΘ ,, O
- 34. 2 2 22 1 3 2 1 ncnnnc ≤−≤ 21 3 2 1 c n c ≤−≤ Example 1.Example 1. Show that We must find c1 and c2 such that Dividing bothsides by n2 yields For )(3 2 1 )( 22 nnnnf Θ=−= )(3 2 1 ,7 22 0 nnnn Θ=−≥
- 35. TheoremTheorem • For any two functions and , we have if and only if )(ng ))(()( ngnf Θ= )(nf )).(()(and))(()( ngnfngOnf Ω==
- 36. Because : )2(5223 nnn Ω=+− Example 2.Example 2. )2(5223)( nnnnf Θ=+−= )2(5223 nOnn =+−
- 37. Example 3.Example 3. 610033,3forsince)(61003 2222 +−>==+− nnncnOnn
- 46. Standard notations and common functionsStandard notations and common functions • Floors and ceilings 11 +<≤≤<− xxxxx
- 47. Standard notations and common functionsStandard notations and common functions • Logarithms: )lg(lglglg )(loglog logln loglg 2 nn nn nn nn kk e = = = =
- 48. Standard notations and common functionsStandard notations and common functions • Logarithms: For all real a>0, b>0, c>0, and n b a a ana baab ba c c b b n b ccc ab log log log loglog loglog)(log log = = += =
- 49. Standard notations and common functionsStandard notations and common functions • Logarithms: b a ca aa a b ac bb bb log 1 log log)/1(log loglog = = −=
- 50. Standard notations and common functionsStandard notations and common functions • Factorials For the Stirling approximation: Θ+ π= ne n nn n 1 12! 0≥n )lg()!lg( )2(! )(! nnn n non n n Θ= ω= =