2. Optimization
Problem:
Optimization Problem
Problem with an objective function to either:
• Maximize some profit
• Minimize some cost
Optimization problems appear in so many application
● Maximize the number of jobs using a resource [Activity-Selection Problem ]
● Collect the maximum value of goods that fit in a given bucket [knapsack Problem]
4. Dynamic
Programming
Dynamic Programming is a general algorithm design technique for solving problems
defined by recurrences with overlapping subproblems.
Invented by American mathematician Richard Bellman in the 1950s to solve optimization
problems and later assimilated by CS.
"Programming" here means
"planning”.
5. Feature of Dynamic
Programming
DP is used to solve problems with the following characteristics:
● Simple subproblems -
We should be able to break the original problem to smaller subproblems that have the same structure
● Optimal substructure of the problems -
The optimal solution to the problem contains within optimal solutions to its subproblems.
● Overlapping sub-problems -
There exist some places where we solve the same subproblem more than once.
7. Top-down approach
The top-down approach follows the memorization technique, while bottom-up approach follows the
tabulation method. Here, memorization is equal to the sum of recursion and caching.
Recursion means calling the function itself, while caching means storing the intermediate results.
8. Disadvantages:-
● It uses the recursion technique that occupies more memory in the call stack. Sometimes when
the recursion is too deep, the stack overflow condition will occur.
● It occupies more memory that degrades the overall performance.
Advantages:-
● It is very easy to understand and implement.
● It solves the subproblems only when it is required.
● It is easy to debug.
Top-down approach
10. Bottom-Up
approach
The bottom-up approach is also one of the techniques which can be used to implement the dynamic
programming. It uses the tabulation technique to implement the dynamic programming approach. It solves
the same kind of problems but it removes the recursion. If we remove the recursion, there is no stack
overflow issue and no overhead of the recursive functions. In this tabulation technique, we solve the
problems and store the results in a matrix.
Key Points :
● We solve all the smaller sub-problems that will be needed to solve the larger sub-problems then move
to the larger problems using smaller sub-problems.
● We use for loop to iterate over the sub-problems.
● The bottom-up approach is also known as the tabulation or table filling method.
11. Advantages:
-
● It helps to improve employee collaboration.
● Occupies less memory and won’t cause stack overflow.
● Conflicts and disagreements will be solved
constructively.
Disadvantages:-
● Time consuming.
● Comparatively hard to debug.
Bottom-Up approach
13. Greedy
Algorithm
● The Greedy method is the simplest and straightforward approach. It is not an algorithm, but it is a
technique.
● The main function of this approach is that the decision is taken on the basis of the currently
available information.
● Whatever the current information is present, the decision is made without worrying about the effect
of the
current decision in future.
14. Characteristics of Greedy
method
The following are the characteristics of a greedy method:
● To construct the solution in an optimal way, this algorithm creates two sets where one set
contains all the chosen items, and another set contains the rejected items.
● A Greedy algorithm makes good local choices in the hope that the solution should be either
feasible or optimal.
15. Applications of Greedy
Algorithm
● It is used in finding the shortest path.
● It is used to find the minimum spanning tree using the prim's algorithm or the Kruskal's
algorithm.
● It is used in a job sequencing with a deadline.
● This algorithm is also used to solve the fractional knapsack problem.
17. Components of Greedy
Algorithm
The components that can be used in the greedy algorithm are:
● Candidate set: A solution that is created from the set is known as a candidate set.
● Selection function: This function is used to choose the candidate or subset which can be added
in the solution.
● Feasibility function: A function that is used to determine whether the candidate or subset can be
used to contribute to the solution or not.
● Objective function: A function is used to assign the value to the solution or the partial solution.
● Solution function: This function is used to intimate whether the complete function has been reached
or not.
18. Disadvantages of using Greedy
algorithm
Greedy algorithm makes decisions based on the information available at each phase without
considering the broader problem. So, there might be a possibility that the greedy solution does
not give the best solution for every problem.
It follows the local optimum choice at each stage with a intend of finding the global optimum. Let's
understand through an example.
19. Feature Greedy method Dynamic programming
Feasibility
In a greedy Algorithm, we make whatever choice
seems best at the moment in the hope that it will lead
to global optimal solution.
In Dynamic Programming we make decision at each
step considering current problem and solution to
previously solved sub problem to calculate optimal
solution .
Recursion
A greedy method follows the problem solving
heuristic of making the locally optimal choice at each
stage.
A Dynamic programming is an algorithmic technique
which is usually based on a recurrent formula that
uses some previously calculated states.
Memoization
It is more efficient in terms of memory as it never look
back or revise previous choices
It requires Dynamic Programming table for
Memoization and it increases it’s memory complexity.
Time
complexity
Faster. For example, Dijkstra’s shortest path algorithm
takes O(ELogV + VLogV) time.
Slower. For example, Bellman Ford algorithm takes
O(VE) time.
Fashion
The greedy method computes its solution by making
its choices in a serial forward fashion, never looking
back or revising previous choices.
Dynamic programming computes its solution bottom
up or top down by synthesizing them from smaller
optimal sub solutions.
20. Conclusio
n
If an optimization problem has an optimal substructure, it may be solved using Greedy or Dynamic
Programming. Now you need to look further for some other properties →
1. If Greedy Choice Property holds for the problem, use the Greedy Approach. It will return the
correct answer faster than DP.
2. If Greedy Choice Property doesn’t hold and there are overlapping subproblems, use DP to find the
correct answer
![Optimization
Problem:
Optimization Problem
Problem with an objective function to either:
• Maximize some profit
• Minimize some cost
Optimization problems appear in so many application
● Maximize the number of jobs using a resource [Activity-Selection Problem ]
● Collect the maximum value of goods that fit in a given bucket [knapsack Problem]](https://image.slidesharecdn.com/daappt-250406123318-f64b6e9d/85/esign-and-Analysis-of-Algorithms-Presentation-pptx-2-320.jpg)
