Quicksort in practice___________________

Quicksort and Randomized
Algorithms

Quicksort
• Proposed by C.A.R. Hoare in 1962.
• Divide-and-conquer algorithm.
• Sorts “in place” (like insertion sort, but not like merge
sort).
• Very practical (with tuning).
Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson
L1.2
September 21, 2005

Divide and conquer
Copyright © 2001-5 by Erik D.
Demaine and Charles E. Leiserson
L1.3
Quicksort an n-element array:
1. Divide: Partition the array into two subarrays around a
pivot x such that elements in lower subarray ≤ x ≤
elements in upper subarray.
≤ x x ≥ x
September 21, 2005
2. Conquer: Recursively sort the two subarrays.
3. Combine: Trivial.
Key: Linear-time partitioning subroutine.

Running time
= O(n) for n
elements.
Partitioning subroutine
L1.4
PARTITION(A, p, q) ⊳A[ p . .q]
x ← A[p] ⊳pivot = A[p]
i ← p
for j ← p + 1 to q
do if A[ j] ≤x
then i ← i + 1
exchange A[i] ↔ A[j]
exchange A[p] ↔A[i]
return i
≥ x ?
September 21, 2005
p i j q
Invariant: x ≤ x

Example of partitioning
L1.5
6 10 13 5 8 3 2 11
i j
September 21, 2005
https://visualgo.net/en/sorting

L1.6
6 10 13 5 8 3 2 11
i j
September 21, 2005

L1.7
6 10 13 5 8 3 2 11
i j
September 21, 2005

L1.8
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
i j
September 21, 2005

L1.9
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
i j
September 21, 2005

L1.10
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
i j
September 21, 2005

L1.11
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
i j
September 21, 2005

L1.12
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
i j
September 21, 2005

L1.13
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
i j
September 21, 2005

L1.14
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
i j
September 21, 2005

L1.15
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
i j
September 21, 2005

L1.16
6 10 13 5 8 3 2 11
6 5 13 10 8 3 2 11
6 5 3 10 8 13 2 11
6 5 3 2 8 13 10 11
2 5 3 6 8 13 10 11
i
September 21, 2005

Pseudocode for quicksort
L1.17
September 21, 2005
QUICKSORT(A, p, r)
if p < r
then q ← PARTITION(A, p, r)
QUICKSORT(A, p, q–1)
QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)

Is Quicksort correct?
Assuming Partition is correct
Proof by induction
⚫ Base case: Quicksort works on a list of 1 element
⚫ Inductive case:
⚫ Assume Quicksort sorts arrays for arrays of smaller < n elements, show that it
works to sort n elements
⚫ If partition works correctly then we have:
⚫ and, by our inductive assumption, we have:
A
pivot
≤ pivot > pivot
sorted sorted

Analysis of quicksort
• Assume all input elements are distinct.
• In practice, there are better partitioning algorithms for
when duplicate input elements may exist.
• Let T(n) = worst-case running time on an array of n
elements.
L1.19
September 21, 2005

Worst-case of quicksort
L1.20
September 21, 2005
= Θ(n2)
• Input sorted or reverse sorted.
• Partition around min or max element.
• One side of partition always has no elements.
T(n) =T(0) +T(n −1) +Θ(n)
= Θ(1)+T(n −1) +Θ(n)
=T(n −1) + Θ(n)
(arithmetic series)

Best-case analysis
L1.21
September 21, 2005
If we’re lucky, PARTITION splits the arrayevenly:
T(n) = 2T(n/2) + Θ(n)
= Θ(n lg n) (same as merge sort)
What if the split is always
1
10
:
9
10
?
T(n) = T(
1
10
n)+ T(
9
10
n)+ Θ(n)
What is the solution to this recurrence?

September 21, 2005
Analysis of “almost-best” case
Copyright © 2001-5 by Erik D. Demaine and Charles
E. Leiserson
L4.28
T(n)

Analysis of “almost-best” case
L1.23
cn
T( n)
10
1
T( n)
10
9
September 21, 2005

Analysis of “almost-best”
case
cn
10
1 cn
10
9 cn
100
1
T( n)T( n)
100
9
T(100
9
n)T( n)
100
81
September 21, 2005 Copyright © 2001-5 by Erik D. Demaine and Charles E. Leiserson L1.24

case
cn
10
1 cn
10
9 cn
1 cn 9 cn
100 100 100
9 cn 100
81 cn
Θ(1)
Θ(1)
10/9
log n
cn
cn
cn
…
O(n) leaves

log10
case
cn
10
1 cn
10
9 cn
n 1 cn 9 cn
100 100 100
9 cn 100
81 cn
10/9
log n
cn
cn
cn
…
Θ(1)
cnlog10n ≤ T(n) ≤ cnlog10/9n + Ο(n)
O(n) leaves
Θ(1)
Θ(n lgn)
Lucky!

More intuition
L1.27
Suppose we alternate lucky, unlucky,
lucky, unlucky, lucky, ….
L(n) = 2U(n/2) + Θ(n)
U(n) = L(n – 1) + Θ(n)
lucky
unlucky
Solving:
L(n) = 2(L(n/2 – 1) + Θ(n/2)) + Θ(n)
= 2L(n/2 – 1) + Θ(n)
= Θ(n lg n)
How can we make sure we are usually lucky?
Lucky!
September 21, 2005

Randomized quicksort
IDEA: Partition around a random element.
• Running time is independent of the input order.
• No assumptions need to be made about the input
distribution.
• No specific input elicits the worst-case behavior.
• The worst case is determined only by the output of a
random-number generator.
L1.28
September 21, 2005

Randomized quicksort
analysis
L1.29
Let T(n) = the random variable for the running time of
randomized quicksort on an input of size n, assuming
random numbers are independent.
For k = 0, 1, …, n–1, define the indicator
random variable
k
X =
1 if PARTITION generates a k : n–k–1 split,
0 otherwise.
E[Xk] = Pr{Xk = 1} = 1/n, since all splits are equally
likely, assuming elements are distinct.
September 21, 2005

Analysis (continued)
L1.30
T(n) =
T(0) + T(n–1) + Θ(n) if 0 : n–1split,
T(1) + T(n–2) + Θ(n) if 1 : n–2split,
…
T(n–1) + T(0) + Θ(n) if n–1 : 0split,
n−1
September 21, 2005
= ∑Xk (T(k) +T(n −k −1) +Θ(n))
k=0

Calculating expectation
L1.31
September 21, 2005
Take expectations of both sides.
𝐸 𝑇(𝑛) = 𝐸 ෍
𝑘=0
𝑛−1
𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)

L1.32
September 21, 2005
Linearity of expectation.
𝐸 𝑇(𝑛) = 𝐸 ෍
𝑘=0
𝑛−1
𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
= ෍
𝑘=0
𝑛−1
𝐸 𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)

L1.33
September 21, 2005
Independence of Xk from other random choices.
𝐸 𝑇(𝑛) = 𝐸 ෍
𝑘=0
𝑛−1
𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
= ෍
𝑘=0
𝑛−1
𝐸 𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
= ෍
𝑘=0
𝑛−1
𝐸 𝑋𝑘 𝐸 (𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)

L1.34
September 21, 2005
Linearity of expectation; E[Xk] = 1/n.
𝐸 𝑇(𝑛) = 𝐸 ෍
𝑘=0
𝑛−1
𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
= ෍
𝑘=0
𝑛−1
𝐸 𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
= ෍
𝑘=0
𝑛−1
𝐸 𝑋𝑘 𝐸 (𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
=
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 𝑇 𝑘 +
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 𝑇 𝑛 − 𝑘 − 1 +
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 Θ(𝑛)

L1.35
September 21, 2005
Summations have
identical terms.
𝐸 𝑇(𝑛) = 𝐸 ෍
𝑘=0
𝑛−1
𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
= ෍
𝑘=0
𝑛−1
𝐸 𝑋𝑘(𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
= ෍
𝑘=0
𝑛−1
𝐸 𝑋𝑘 𝐸 (𝑇 𝑘 + 𝑇 𝑛 − 𝑘 − 1 + Θ(𝑛)
=
2
𝑛
෍
𝑘=0
𝑛−1
𝐸 𝑇 𝑘 +
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 Θ(𝑛)
=
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 𝑇 𝑘 +
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 𝑇 𝑛 − 𝑘 − 1 +
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 Θ(𝑛)

Hairy recurrence
L1.36
September 21, 2005
Use fact: 8
2 2
1 1
2
n−1
∑
k=2
n lg n − n
k lg k≤ (exercise).
(The k = 0, 1 terms can be absorbed in the Θ(n).)
Prove: E[T(n)] ≤ an lgn for constant a > 0.
• Choose a large enough so that anlg n dominates E[T(n)] for
sufficiently large n ≥ 2.
𝐸 𝑇(𝑛) =
2
𝑛
෍
𝑘=2
𝑛−1
𝐸 𝑇 𝑘 +
1
𝑛
෍
𝑘=0
𝑛−1
𝐸 Θ(𝑛)

Substitution method
L1.37
September 21, 2005
Substitute inductive hypothesis.
𝐸 𝑇(𝑛) ≤
2
𝑛
෍
𝑘=2
𝑛−1
𝑎∙𝑘∙log 𝑘 +Θ(𝑛)

Substitution method
L1.38
September 21, 2005
Use fact.
𝐸 𝑇(𝑛) ≤
2
𝑛
෍
𝑘=2
𝑛−1
≤
2𝑎
𝑛
1
2
𝑛2
log𝑛−
1
8
𝑛2
+Θ(𝑛)

Substitution method
L1.39
September 21, 2005
Express as desired – residual.
𝐸 𝑇(𝑛) ≤
2
𝑛
෍
𝑘=2
𝑛−1
≤
2𝑎
𝑛
1
2
𝑛2
log𝑛−
1
8
𝑛2
+Θ(𝑛)
≤𝑎𝑛log𝑛−
𝑎𝑛
4
+Θ(𝑛)

Substitution method
L1.40
September 21, 2005
if a is chosen large enough so that an/4 dominates
the Θ(n).
𝐸 𝑇(𝑛) ≤
2
𝑛
෍
𝑘=2
𝑛−1
≤
2𝑎
𝑛
1
2
𝑛2 log𝑛−
1
8
𝑛2 +Θ(𝑛)
≤𝑎𝑛log𝑛−
𝑎𝑛
4
+Θ(𝑛)
≤𝑎𝑛log𝑛

Quicksort in practice
• Quicksort is a great general-purpose sorting
algorithm.
• Quicksort is typically over twice as fast as merge sort.
• Quicksort can benefit substantially from code tuning.
• Quicksort behaves well even with caching and
virtual memory.
L1.41
September 21, 2005

Quicksort in practice___________________

More Related Content

Similar to Quicksort in practice___________________ (20)

Recently uploaded (20)

Quicksort in practice___________________