Lecture3a sorting

Lecture 3 Part 1
Sorting
23/10/2018 Sorting Lecture 3 1

Content Lecture 3 Sorting
 Introduction
 Characteristics of Sorting Algorithm
 Indirect Sort
 Distribution Sort
 Stable Sorting
 Sorting Algorithm
 Selection Sort
 Insertion Sort
 Quick Sort
 Bucket Sort or Bin Sort
 Radix Sort
 Merge Sort
 Bubble Sort
 Comparing the Algorithm

Sorting
What is sorting?

Sorting
Introduction
 Sorting is the rearranging of a given set of objects in a
specific order
 The purpose is often to simplify a search on the set later
 Sorting is done for example in telephone books, data
warehouses, libraries, databases, etc.
 The structure of the data dramatically influences the
sorting algorithm and its performance

Sorting
 There exists a great diversity of sorting algorithm
 To choose a proper algorithm it is necessary to understand
the significant of performance
 Sorting algorithm are classified in two categories:
 sorting of arrays
 sorting of files
 They are also called internal and external sorting because
 arrays are stored in the internal store of a computer
(internal sorting)
 files are stored on external devices like disks or folder
(external sorting)

Sorting
Definition
If we have a number n of items a0, a1, …, an-1 a sorting algorithm
gains in permuting these items into an array ak0,ak1…ak(n-1) so that
for a given order function f:
f(ako) ≤ f(ak1) ≤ … ≤ f(ak(n-1))
The value of the ordering function is called the key of the item.
A sorting method is called stable (see Stable Sorting) if the
relative order of items with equal keys remains unchanged by the
sorting process.

Sorting Algorithm
 A sorting problem must not be numerical but it must be
distinguishable (e.g. by colors, by size)
 The compiler automatically knows how to treat numerical values
 For others however it must be possible for a sorting algorithm to
decide if an element of the set is smaller than another element
 For example you can make a numbered list for colors
 For characters you can use the ASCII table to decide how the
sorting is done
 All the algorithms now discussed work with numerical
representations where a smaller-, bigger-, same-relation is
defined

Sorting Algorithm
 The steps in every sorting algorithm can be simplify by
 Selecting and inserting
 Interchanging
 Spreading and collection
 Distributing
 There are three different characteristics for sorting
algorithm
 Indirect Sorting
 Distribution Sorting
 Stable Sorting

Indirect Sorting
 If you have an array with large elements sorting might be
expensive
 Therefore you use for sorting a list with references to the
original array instead of sorting the array itself
 The original array data are compared but the element in
the reference list is swapped
 Afterwards the original array is still untouched
 But the array with references helps to rearrange the data in
the record
 This is called Indirect Sorting

Indirect Sorting
Definition
A an array
n the number of elements in the array
P another array/list is defined by P[i]=i for all i = 1, 2 … n
Task is to modify P so that
A[P[1]] ≤ A[P[2]] ≤ … ≤ A[P[n]]
Therefore instead of changing the array A we change the list
P

Indirect Sorting
Example
 Sorted after the Reg.-No.
 Use the Id column to indicate every line
 The original table is not changed
Id Reg. No. Last Name First Name
1 102 Smith John
2 99 Black Rose
3 376 Miller Fred
4 22 Baker Joseph
5 86 White Agnes
Id Reg. No.
1 102
2 99
3 376
4 22
5 86
Id Reg. No.
4 22
5 86
2 99
1 102
3 376
sort

Distribution Sorting
 A sorting algorithm is called a Distribution Sorting
Algorithm if the data is distributed from its input to a
multiple temporary structure
 This structure is used to collect the elements and place
them on the output
Examples
 Merge Sort
 Radix Sort
Input data
Output data
Temporary
Structure

Stable Sorting
 For some data you wish to sort them by more than one
criterion
 For example in a list of addresses you sort first by the
surnames and than by the first names
 A sorting algorithm is called stable when one sort does not
destroy the result of the previous sort
 That means if the elements of the original array with the
same value (key) appear in the output array in the same
order as they did in the original array
 Therefore a stable sort preserves the order of equal keys

Stable Sorting
Example
List of pairs to order: {(3, A),(1, C),(2, B),(3, D),(1, B),(2, A),(3, C)}
Two possibilities to sort them after the first key:
{(1, B), (1, C), (2, A), (2, B), (3, A), (3, C), (3, D)}
order changed/not stable
{(1, C), (1, B), (2, B), (2, A), (3, A), (3, D), (3, C)}
order maintained/stable
Sorting the names Victoria, Brenda, Angela by length than:
Brenda, Angela, Victoria would be stable
Angela, Brenda, Victoria would be not stable

Sorting Algorithm
 We going to look at some important sorting algorithm
 There are far more!
 The most sorting algorithm are comparison sorts meaning
that they compare the keys
 The other basic operation is to swap

Selection Sort
 A Selection Sort compares the data to decide if they have
to be swapped or not
 It starts with the first element
 It compares this element with the next elements and so on
 If the value of a next element is smaller than it swap the
two elements
 If no smaller value can be found the algorithm continues
with the next element
 Therefore almost every value is compared with the other
values (time costly)
 If there are only two elements in the record this algorithm
is faster than all others

Selection Sort
Example
21 36 3 25
3 36 7 25
3 7 21 25
3 7 21 25
3 7 21 25 36

Selection Sort
Example
53 2 12 8 64 16 15

Selection Sort
Solution
53 2 12 8 64 16 15
2 53 12 8 64 16 15
2 12 53 8 64 16 15
2 8 53 12 64 16 15
2 8 12 53 64 16 15
2 8 12 16 64 53 15
2 8 12 15 64 53 16
2 8 12 15 53 64 16
2 8 12 15 16 64 53
2 8 12 15 16 53 64

Selection Sort
Example
53 2 12 8 64 16 15
2 53 12 8 64 16 15
2 8 12 53 64 16 15
2 8 12 53 64 16 15
2 8 12 15 64 16 53
2 8 12 15 16 64 53
2 8 12 15 16 53 64
A variant of the Selection
Sort tries to find the
minimum of the remaining
data and interchanges the
two data elements

Insertion Sort
 Two groups of data - sorted and unsorted
 Every repetition of the Insertion Sort removes an element
of the original input data (unsorted)
 This element is put in the correct position of the already
sorted part of the original data (sorted)
 The repetition takes place until no element remains
 Recommended for data that is nearly sorted otherwise very
time costly

Insertion Sort
Be s the element to be sorted:
sorted part unsorted data
≤ s > s s
≤ s > ss

Insertion Sort
Example
2 5 4 0 3 7 1 6
2 5 4 0 3 7 1 6
2 5 4 0 3 7 1 6
2 4 5 0 3 7 1 6
0 2 4 5 3 7 1 6
0 2 3 4 5 7 1 6
0 2 3 4 5 7 1 6
0 1 2 3 4 5 7 6
0 1 2 3 4 5 6 7

Insertion Sort
Example
53 2 12 8 64 16 15

Insertion Sort
Solution
53 2 12 8 64 16 15
53 2 12 8 64 16 15
2 53 12 8 64 16 15
2 12 53 8 64 16 15
2 8 12 53 64 16 15
2 8 12 53 64 16 15
2 8 12 16 53 64 15
2 8 12 15 16 53 64

Quick Sort
 Quick Sort algorithm uses a so called pivot element
 The pivot element is selected in such a way that around
half of the values are smaller and half of the values are
bigger in the input data
 The data are separated accordingly into a sub part and high
part
 The method is repeated recursively with each part
 Equal elements can be put in one of both parts
 If a part has no element or one element it is defined as
sorted

Quick Sort
pivot< pivot > pivot
pivot<pivot’ > pivot’’pivot’ >pivot’ <pivot’’ pivot’’
• The pivot element is chosen randomly for example the
middle index of the data or the median of the first, middle
and last element
• Fasted sort algorithm in practice
• But a simple Quick Sort algorithm performs very badly on
already sorted array of data

Quick Sort
Example
(3 2 6 1 8 4 9 7 5)
Pivot = 5 (smallest 1, biggest 9  9 + 1 = 10/2 = 5)
(3 2 1 4)(5)(6 8 9 7)
(2 1)(3)(4)(5)(6)(7)(8 9)
(1)(2)(3)(4)(5)(6)(7)(8)(9)

Quick Sort
Example
(53 2 12 8 64 16 15)

Quick Sort
Solution
(53 2 12 8 64 16 15)
median: 2 + 64 = 66/2 = 33  15 or 16
( 2 12 8 15)(16)(53 64)
( 2)( 8)(12 15)(16)(53)(64)
( 2)( 8)(12)(15)(16)(53)(64)

Bucket Sort or Bin Sort
 Bucket Sort partitioning an array into a number of buckets
 Each of these buckets is sorted individually
 This can be done by using another sorting algorithm or again the
bucket sort
 A bucket sort is a distribution sort
 These are the steps to be performed:
 Set up an array of empty buckets
 Put every item of the original array in its bucket
 Sort each of the buckets that is not empty
 Put all the elements now sorted back to the original array

Example
33, 41, 22, 8, 4, 12, 19, 37, 45, 7, 17, 26, 29, 34
Bucket 0-9 Bucket 10-19 Bucket 20-29 Bucket 30-39 Bucket 40-49
4
7
8
12
17
19
22
26
29
33
34
37
41
45
41
45
33
37
34
22
26
29
12
19
17
8
4
7
Recollection:
4 7 8 12 17 19 22 26 29 33 34 37 41 45
Sorting each bucket:

Example
3 34 16 7 13 22 4 15 27

Solution
3 34 16 7 13 22 4 15 27
Bucket 0-9 Bucket 10-19 Bucket 20-29 Bucket 30-39
3
4
7
13
15
16
22
27
34
34
22
27
16
13
15
3
7
4
Recollection:
3 4 7 13 15 16 22 27 34
Sorting each bucket:

Variants of Buckets Sort:
 Generic bucket sort: operates on a list of n numeric inputs
between 0 and a max value; divided the value range into n
buckets with size Max/n
 Postman’s sort: operates on hierarchical structure elements;
used by letter-sorting machines; Mail is sorted first between
nation/international, then state, province, district, city,
streets/routes, etc. keys not sorted against each other.
 Shuffle sort: operates by removing the first 1/8 of the elements
n, sorts them recursively and puts them in an array. It creates n/8
buckets to which the remaining 7/8 elements are distributed.
Each bucket is sorted and concatenated into a sorted array

Radix Sort
 A very old sorting algorithm (invented 1887 by Herman
Hollerith) is the Radix Sort
 The Radix sort was used to sort cards
 In general it sorts integers but it is not limited to it
 The algorithm distributes items to a bucket according to
the item’s value beginning with the least significant digit
 After each round the items are recollected from the buckets
 The process is repeated with the next most significant digit
 This is called Least Significant Digit Radix Sort (LSD)
 A variant is the Most Significant Digit (MSD) starting
with the most significant digit

Radix Sort
Example
Input keys: 34, 12, 42, 32, 44, 41, 34, 11, 32, 23
4 buckets, because there are 4 different digits 1, 2, 3, 4
Sorting by the least significant digit:
1. Bucket: 41 11
2. Bucket: 12 42 32 32
3. Bucket: 23
4. Bucket: 34, 44, 34
Recollecting: 41 11 12 42 32 32 23 34 44 34

Radix Sort
The recollected data are now sorted by the next most
significant digit (here the highest digit):
41 11 12 42 32 32 23 34 44 34
1. Bucket: 11 12
2. Bucket: 23
3. Bucket: 32 32 34 34
4. Bucket: 41 42 44
Recollecting: 11 12 23 32 32 34 34 41 42 44

Radix Sort
Example
3 34 16 7 13 22 4 15 27

Radix Sort
Solution
03 34 16 07 13 22 04 15 27
1. Bucket: 22
2. Bucket: 03 13
3. Bucket: 34 04
4. Bucket: 15
5. Bucket: 16
6. Bucket: 07 27
Recollecting: 22 03 13 34 04 15 16 07 27

Radix Sort
22 03 13 34 04 15 16 07 27
1. Bucket: 03 04 07
2. Bucket: 13 15 16
3. Bucket: 22 27
4. Bucket: 34
Recollecting: 03 04 07 13 15 16 22 27 34

Merge Sort
 Merge Sort is a comparison-based sorting algorithm
 Most of the used implementation produces a stable sort
 The algorithm works as follows:
 The unsorted input list is divided in two sub-lists of about
half the size of the original data
 Each sub-list is sorted recursively by using again a merge sort
 Afterwards the two sub-list are merged into one sorted list
 The basic ideas behind the Merge Sort:
 a smaller list takes less runtime than a bigger
 fewer steps are necessary to construct a sorted list from two
sorted lists than from an unsorted list

Merge Sort
Example
(37 26 42 1 7 70 12 56)
Dividing into 2 parts:
(37 26 42 1)( 7 70 12 56)
Dividing each part again into 2 parts:
(37 26)(42 1)( 7 70)(12 56)
Sorting of each part:
(26 37)( 1 42)( 7 70)(12 56)
Sorting & merging back to previous size (two parts):
( 1 26 37 42)( 7 12 56 70)
Sorting & merging back to the original size(now
sorted):
( 1 7 12 26 37 42 56 70)

Merge Sort
Example
3 34 16 7 13 22 4 15 27

Merge Sort
Solution
3 34 16 7 13 22 4 15 27
dividing: ( 3 34 16 7)(13 22 4 15 27)
dividing: ( 3 34)(16 7)(13 22)( 4 15 27)
sorting: ( 3 34)( 7 16)(13 22)( 4 15 27)
sorting
&merging: ( 3 7 16 34)( 4 13 15 22 27)
sorting
&merging: ( 3 4 7 13 15 16 22 27 34)

Bubble Sort
23/10/2018
 The Bubble Sort is a simple sorting algorithm
 It works by repeatedly stepping through a list of data
 It compares each pair of elements and swaps them if they
are in wrong order
 This is done until no swap is needed any more
 The name comes from the way smaller elements bubble to
the top of the list
Sorting Lecture 3 46

Bubble Sort
 The performance depends strongly on the position of the
elements
 If the smaller elements are stored at the end of the list the
sort is extremely slow
 If the larger elements are at the beginning this cause no
problem
 They are therefore called turtles (small elements at the
end of the list) and rabbits (larger elements at the
beginning of the list)

Bubble Sort
Example
 First round: 7 2 6 3 9  2 7 6 3 9 
2 6 7 3 9  2 6 3 7 9 
2 6 3 7 9
 Second round: 2 6 3 7 9  2 6 3 7 9 
2 3 6 7 9  2 3 6 7 9 
2 3 6 7 9
 Third round: 2 3 6 7 9  2 3 6 7 9 
2 3 6 7 9  2 3 6 7 9 
2 3 6 7 9
No swaps!

Bubble Sort
Example
3 34 16 7 13 22 4 15 27

Bubble Sort
First round: 3 34 16 7 13 22 4 
3 34 16 7 13 22 4 
3 16 34 7 13 22 4 
3 16 7 34 13 22 4 
3 16 7 13 34 22 4 
3 16 7 13 22 34 4 
3 16 7 13 22 4 34
Second round: 3 16 7 13 22 4 34 
3 16 7 13 22 4 34 
3 7 16 13 22 4 34 
3 7 13 16 22 4 34 
3 7 13 16 22 4 34 
3 7 13 16 4 22 34 
3 7 13 16 4 22 34

Bubble Sort
Third round: 3 7 13 16 4 22 34 
3 7 13 16 4 22 34 
3 7 13 16 4 22 34 
3 7 13 16 4 22 34 
3 7 13 4 16 22 34 
3 7 13 4 16 22 34 
3 7 13 4 16 22 34
Forth round: 3 7 13 4 16 22 34 
3 7 13 4 16 22 34 
3 7 13 4 16 22 34 
3 7 4 13 16 22 34 
3 7 4 13 16 22 34 
3 7 4 13 16 22 34 
3 7 4 13 16 22 34

Bubble Sort
Fifth round: 3 7 4 13 16 22 34 
3 7 4 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34
Sixth round: 3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34 
3 4 7 13 16 22 34
No swaps!

Bubble Sort
Variants
Variant of Bubble Sort Description
Od-even sort Parallel version of bubble sort for
message passing systems
Cocktail sort Parallel version
Right-to-left Instead of starting from the left side
you start from the right side

Comparing the algorithms
 To evaluate the different sorting algorithm you can
consider the following factors
 Run-Time: the cost/period of time for each performed
operation (comparing, swapping, distributing, etc.)
 Memory: the memory you need for the sorting
 Can either be
 None or constant
 Linear
 Exponential (worst case)
 The goal is of course to use less memory and run-time
 In the most cases you look at the run-time behaviour

 The following table compares all comparison sorts (Insertion,
Binary tree sort (later), Selection, Bubble, Merge, Quick)
 n is the number of records
 Best describes the best possible performance if the data are
favourable distributed
 Average describes the average run time performance
 Worst describes the behaviour of the algorithm if the data are
badly distributed
 Method is the used operation of the algorithm
 For Average and Worst is assumed that all comparisons, swaps
and other necessary operations can proceed in constant time
(O(1))

Name Best Average Worst Method
Insertion sort O(n) O(n2) O(n2) Insertion
Heap sort (later) O(n) O(nlog(n)) O(nlog(n)) Insertion
Selection Sort O(n2) O(n2) O(n2) Selection
Bubble Sort O(n) O(n2) O(n2) Exchanging
Merge Sort O(nlogn) O(nlogn) O(nlog(n)) Merging
Quick Sort O(nlog(n)) O(nlog(n)) O(n2) Partitioning

 This table compares all sorting algorithm which are not
comparison sorts
 n the number of items, k the size of the key/value, d the
digit implementation size
Name Average Worst
Bucket Sort O(n+k) O(n2*k)
LSD Radix Sort O(n*(k/d)) O(n*(k/d))

Developing an algorithm for the
selection sort
Proceeding
 Comparing the start value with all other values to find a
minimum
 If a minimum exists swap the two elements
 Given an array of elements: list[]
 n number of elements in the list
0 1 2 3 4 5 … n-3 n-2 n-1
list[0] list[1] list[2] list[3] list[4] list[5] … list[n-3] list[n-2] list[n-1]

selection sort
 We compare every element starting with the first element
  for loop
for (int i = 0; i < n-1; i++)
 Index i goes from 0 to n-2 because last index n-1 is already
the maximum
 Now we compare list[i] with all other elements
 Meaning we have to compare it with all elements where the
index is greater than i and smaller than n
  second for loop
for (int j = i + 1; j < n; j++)

selection sort
 To find the maximum we compare list[i] with list[j]
 If list[i] >= list[j]  a new minimum is found
 Set min: int min = i; before second for loop
 In second for loop;
íf (a[min] >= a[j])
min = j;
 Finally swap the elements:
int tmp = list[i];
list[i] = list[min];
list[min] = tmp;

selection sort
Complete code
public static void sort(int[] list, int n) {
for (int i = 0; i < n-1; i++) {
int min = i;
for (int j = i+1; j < n; j++) {
if (list[min] >= list[j])
min = j;
}
int tmp = list[i];
list[i] = list[min];
list[min] = tmp;
}
}

Any
questions?

Lecture3a sorting

More Related Content

Similar to Lecture3a sorting (20)

More from mbadhi barnabas (9)

Recently uploaded (20)

Lecture3a sorting