Machine Learning All topics Covers In This Single Slides

Introduction History
The Word: Algorithm
Abu Abdallah Muhammad ibn
Musa al-Khwarizmi
Earlier transliterated as
Algoritmi or Algaurizin
A Persian mathematician,
astronomer and geographer
Born: 780 AD, Khwarezm
Died: 850 AD, Baghdad, Iraq
Nationality: Iranian
Books: The Compendious Book
on Calculation by Completion
and Balancing
Dr. Hemanta Kumar Kalita Algorithm Analysis and Design January 9, 2025 1 / 22

Introduction History
Algorithm by Euclid
Euclid created this algorithm
This algorithm calculates the
greatest common divisor, a
divide the number a by b, the
remainder is r
replace a by b
replace b by r
continue until a can’t be more
divided

Introduction Algorithm Design Paradigms
Algorithm Design Paradigms
Brute Force
Greedy Algorithms
Divide and Conquer
Decrease and Conquer
Transform and Conquer
Dynamic Programming
Backtracking
Genetic Algorithms

Brute Force
Based on the problem’s statement and definitions of the concepts
involved. Examples:
Sequential search
Exhaustive search: TSP, knapsack
Simple sorts: selection sort, bubble sort
Computing n!

Greedy Algorithms
Take what you can get now strategy
Work in phases
⇒ In each phase the currently best decision is made
Examples:
Dijkstra’s algorithm–shortest path is weighted graphs
Prim’s algorithm, Kruskal’s algorithm–minimal spanning tree in
weighted graphs
Coin exchange problem
Huffman Trees

Divide and Conquer
In general, problems that can be defined recursively
Reduce the problem to smaller problems (by a factor of at least
2) solved recursively and then combine the solutions
Examples:
Binary Search
Mergesort
Quick sort
Tree traversal

Decrease and Conquer
Reduce the problem to smaller problems solved recursively and
then combine the solutions
Examples:
Insertion sort
Topological sorting
Binary Tree traversals: inorder, preorder and postorder (recursion)
Computing the length of the longest path in a binary tree (recursion)
Computing Fibonacci numbers (recursion)
Reversing a queue (recursion)

Transform and Conquer
The problem is modified to be more amenable to solution. In the
second stage the problem is solved.
Problem Simplification. For example, presorting. Finding the two
closest numbers in an array of numbers.
Brute force solution: ⃝(n2)
Transform and conquer with presorting:⃝(nlogn)
Change in the representation. Example: AVL trees guarantee
⃝(nlogn) search time.
Problem reduction. Example: least common multiple
lcm(m, n) =
m ∗ n
gcd(m, n)

Dynamic Programming
Bottom-Up Technique in which the smallest sub-instances are
explicitly solved first and the results of these used to construct
solutions to progressively larger sub-instances.
Example:Fibonacci numbers computed by iteration.

Backtracking
Generate-and-Test methods based on exhaustive search in mul-
tiple choice problems.Typically used with depth-first state space
search problems.
Example: puzzles
State Space Search
initial state
goal state(s)
a set of intermediate states
a set of operators that transform one state into another. Each
operator has preconditions and postconditions.
a cost function – evaluates the cost of the operations (optional)
a utility function – evaluates how close is a given state to the goal
state (optional)

Genetic Algorithms
Search for good solutions among possible solutions
The best possible solution may be missed
A solution is coded by a string, also called chromosome. The words
string and chromosome are used interchangeably
A string’s fitness is a measure of how good a solution it codes.
Fitness is calculated by a fitness function
Selection: The procedure to choose parents
Crossover is the procedure by which two chromosomes mate to create
a new offspring chromosome
Mutation: with a certain probability flip a bit in the offspring

Introduction Concept of Algorithmic Efficiency
Algorithmic Efficiency
Analogous to Engg Productivity

Properties of an algorithm which relate to the amount of resources
used by the algorithm

Resources–Time and Space(memory)

Measurement of resource usage :
– Expressed as a function of the size of the input n
– Time: how long does the algorithm take to complete
– Space: how much working memory (typically RAM) is needed by
the algorithm for its code and data

Measurement of resource usage :
– Expressed as a function of the size of the input n
– Time: how long does the algorithm take to complete
– Space: how much working memory (typically RAM) is needed by
the algorithm for its code and data
Some other measurements :
– Direct/Indirect power consumption
– Transmission size
– External space
– Response time
– Total cost of ownership

Run Time Analysis of Algorithms
Is accurate measurement of run time possible?

Suppose, you are asked to design and analyze an algorithm. What would
you do?

you do?
1 Design the algorithm

you do?
2 Analyze time taken on RAM model

you do?
3 Details of analysis is not applicable to other computer

you do?
Suppose, RAM Time is 100n3 + 15n2 + 20

you do?
For other computer, you would say time taken is Cubic.

you do?
However, you would say cubic even for 5n3 + 20n + 39.

you do?
However, you would say cubic even for 5n3 + 20n + 39.
You are saying approximate time taken by an algorithm in other
computer.

Introduction Asymptotic Notation
Asymptotic–the Word
What is an asymptote ?

– In analytic geometry, an asymptote of a curve
is a line such that the distance between the
curve and the line approaches zero as they tend
to infinity

to infinity
– a straight line that is closely approached by a
plane curve so that the perpendicular distance
between them decreases to zero as the distance
from the origin increases to infinity

to infinity
– straight line continually approaching but never
meeting a curve

to infinity
– straight line continually approaching but never
meeting a curve
– History: Comes from Greek word asumptotos,
which means not falling together

So, what is asymptotic?

relating to or of the nature of an asymptote;

For example, an asymptotic function

For example, an asymptotic function
Asymptotic word is related to function

Classes of Function
What is Asymptotic Notation?

Classes of Function
Asymptotic Notation Formal way or notation to speak about functions
and classify them.

Classes of Function
and classify them.
Asymptotic Analysis Classifying functions

Classes of Function
and classify them.
Two features:

Classes of Function
and classify them.
Two features:
1 Functions 100n3 + 15n2 + 20 and 5n3 + 20n + 39 should go to same
class. Here, ignore constant multipliers.

Classes of Function
and classify them.
Two features:
2 Give more importance to behavior as n → ∞.

Classes of Function
and classify them.
Two features:
2 Give more importance to behavior as n → ∞.
Θ, Ω and ⃝ notations: Definitions and examples to follow.

Θ Notation

Θ Notation
Assume, two functions f and g, so that

Θ Notation
both are non-negative, and

Θ Notation
both has non-negative arguments

Θ Notation
Why non-negative ?

Θ Notation
Why non-negative ?
Because, time and input are non-negative

Θ Notation
Why non-negative ?
Θ(g) :
{f |f is a non-negative function s.t. ∃
constants c1, c2 and n0

Θ Notation
Why non-negative ?
Θ(g) :
constants c1, c2 and n0 s.t.
c1g(n) ≤ f (n) ≤ c2g(n)}

Θ Notation
Why non-negative ?
Θ(g) :
constants c1, c2 and n0 s.t.
c1g(n) ≤ f (n) ≤ c2g(n)} for n ≥ n0

Ω Notation

Ω Notation
Ω(g) :
constants c and n0

Ω Notation
Ω(g) :
constants c and n0 s.t. 0 ≤ cg(n) ≤ f (n)}

Ω Notation
Ω(g) :
for n ≥ n0

Ω Notation
Ω(g) :
for n ≥ n0
Asymptotic lower bound

Ω Notation
Ω(g) :
for n ≥ n0
Used to say an Algorithm shall take at least xyz time

Ω Notation
Ω(g) :
for n ≥ n0
Used to say an Algorithm shall take at least xyz time
Running time is Ω(f (n)) ⇒ Best case is Ω(f (n))

⃝ Notation

⃝ Notation
⃝(g) :
constants c and n0

⃝ Notation
⃝(g) :
constants c and n0 s.t. 0 ≤ f (n) ≤ cg(n)}

⃝ Notation
⃝(g) :
for n ≥ n0

⃝ Notation
⃝(g) :
for n ≥ n0
Asymptotic upper bound

⃝ Notation
⃝(g) :
for n ≥ n0
Used to say an Algorithm shall take at most xyz time

⃝ Notation
⃝(g) :
for n ≥ n0
Used to say an Algorithm shall take at most xyz time
Running time is ⃝(f (n)) ⇒ Worst case is ⃝(f (n))

Θ, Ω and ⃝ Notation

f (n) = Θ(g(n)) should be read as f (n) belongs to class Θ(g(n))

or, f (n) is in class Θ(g(n))

or, f (n) is Θ(g(n))

or, f (n) is Θ(g(n))
Equality here is analogous to, for example, Rose is Red.

Examples
Q. Show that 1
2n2 − 3n = θ(n2)

Examples
Q. Show that 1
2n2 − 3n = θ(n2)
We need to determine c1, c2 and n0 so that it satisfies the condition

Examples
Q. Show that 1
2n2 − 3n = θ(n2)
We need to determine c1, c2 and n0 so that it satisfies the condition :
c1n2 ≤ 1
2n2 − 3n ≤ c2n2

Examples
Q. Show that 1
2n2 − 3n = θ(n2)
c1n2 ≤ 1
2n2 − 3n ≤ c2n2
for all n ≥ n0

Examples
Q. Show that 1
2n2 − 3n = θ(n2)
c1n2 ≤ 1
2n2 − 3n ≤ c2n2
for all n ≥ n0
Dividing by n2 yields
c1 ≤
1
2
−
3
n
≤ c2

Examples
Q. Show that 1
2n2 − 3n = θ(n2)
c1n2 ≤ 1
2n2 − 3n ≤ c2n2
for all n ≥ n0
c1 ≤
1
2
−
3
n
≤ c2
The right-hand inequality can be hold for any value of n ≥ 1 by choosing
c2 ≥ 1
2.

Examples
Q. Show that 1
2n2 − 3n = θ(n2)
c1n2 ≤ 1
2n2 − 3n ≤ c2n2
for all n ≥ n0
c1 ≤
1
2
−
3
n
≤ c2
c2 ≥ 1
2.
The left-hand inequality can be hold for any value of n ≥ 7 by choosing
c1 ≤ 1
14

Examples
Q. Show that 1
2n2 − 3n = θ(n2)
c1n2 ≤ 1
2n2 − 3n ≤ c2n2
for all n ≥ n0
c1 ≤
1
2
−
3
n
≤ c2
c2 ≥ 1
2.
The left-hand inequality can be hold for any value of n ≥ 7 by choosing
c1 ≤ 1
14
Thus, by choosing c1 = 1
14, c2 = 1
2 and n0 = 7, we can prove that
1
2n2 − 3n = θ(n2).

xyz
abc

Machine Learning All topics Covers In This Single Slides

More Related Content

Similar to Machine Learning All topics Covers In This Single Slides (20)

Recently uploaded (20)

Machine Learning All topics Covers In This Single Slides