AI3391 ARTIFICIAL INTELLIGENCE UNIT II notes.pdf

AI3391
ARTIFICAL INTELLIGENE
B.TECH
II YEAR – III SEM (R18)
(2023-2024)
Prepared
By
Asst.Prof.M.Gokilavani
Department of Artificial Intelligence and Data Science

DEPARTMENT OF ARTIFICIAL INTLLIGENCE AND DATA SCIENCE

UNIT II
Problem solving: Heuristic search strategies, Heuristic functions. Local search and optimization problems,
local search in continuous space, search with non-deterministic actions, search in partial observations
environment, online search agents and unknown environment.
1. INFORMED SEARCH ALGORITHM:
• Informed search algorithm contains an array of knowledge such as how far we are
from the goal, path cost, how to reach to goal node, etc. This knowledge helps agents
to explore less to the search space and find more efficiently the goal node.
Example Tree: node with information (weight)
Heuristics function:
• The informed search algorithm is more useful for large search space. Informed search
algorithm uses the idea of heuristic, so it is also called Heuristic search.
• Heuristics function: Heuristic is a function which is used in Informed Search, and it
finds the most promising path.
• It takes the current state of the agent as its input and produces the estimation of how
close agent is from the goal.
• The heuristic method, however, might not always give the best solution, but it
guaranteed to find a good solution in reasonable time. Heuristic function estimates
how close a state is to the goal.
• It is represented by h (n), and it calculates the cost of an optimal path between the pair
of states. The value of the heuristic function is always positive.

Where,
h (n) <= h*(n)
Here,
H (n) is heuristic cost,
h*(n) is the estimated cost.
Hence heuristic cost should be less than or equal to the estimated cost.
Pure Heuristic Search:
 Pure heuristic search is the simplest form of heuristic search algorithms. It expands
nodes based on their heuristic value h (n).
 It maintains two lists, OPEN and CLOSED list.
 In the CLOSED list, it places those nodes which have already expanded and in the
OPEN list, it places nodes which have yet not been expanded.
 On each iteration, each node n with the lowest heuristic value is expanded and
generates all its successors and n is placed to the closed list. The algorithm continues
unit a goal state is found.
In the informed search we will discuss two main algorithms which are given below:
o Best First Search Algorithm(Greedy search)
o A* Search Algorithm
2. BEST FIRST SEARCH ALGORITHM (GREEDY SEARCH):
• BFS uses the concept of a Priority queue and heuristic search. To search the graph
space, the BFS method uses two lists for tracking the traversal.
• An ‘Open’ list that keeps track of the current ‘immediate’ nodes available for
traversal and a ‘CLOSED’ list that keeps track of the nodes already traversed.
• In the best first search algorithm, we expand the node which is closest to the goal
node and the closest cost is estimated by heuristic function,
• Where,
F (n) = g (n)
g (n) path distance
Algorithm:
1. Create 2 empty lists: OPEN and CLOSED
2. Start from the initial node (say N) and put it in the ‘ordered’ OPEN list
3. Repeat the next steps until the GOAL node is reached
 If the OPEN list is empty, then EXIT the loop returning ‘False’

 Select the first/top node (say N) in the OPEN list and move it to the
CLOSED list. Also, capture the information of the parent node
 If N is a GOAL node, then move the node to the closed list and exit the loop
returning ‘True’. The solution can be found by backtracking the path
 If N is not the GOAL node, expand node N to generate the ‘immediate’ next
nodes linked to node N and add all those to the OPEN list
 Reorder the nodes in the OPEN list in ascending order according to an
evaluation function f (n).
Example:1
Solutions:
Expand the nodes of S and put in the CLOSED list
• Initialization: Open [A, B], Closed [S]
• Iteration 1: Open [A], Closed [S, B]
• Iteration 2: Open [E, F, A], Closed [S, B]
: Open [E, A], Closed [S, B, F]
• Iteration 3: Open [I, G, E, A], Closed [S, B, F]
: Open [I, E, A], Closed [S, B, F, G]
Hence the final solution path will be:
S----> B----->F----> G

Advantages:
– Best first search can switch between BFS and DFS by gaining the advantages
of both the algorithms.
– This algorithm is more efficient than BFS and DFS algorithms.
Disadvantages:
– It can behave as an unguided depth-first search in the worst case scenario.
– It can get stuck in a loop as DFS.
– This algorithm is not optimal.
3. A* SEARCHING ALGORITHM:
• A* Algorithm is one of the best and popular techniques used for path finding and
graph traversals.
• A lot of games and web-based maps use this algorithm for finding the shortest path
efficiently.
• It is essentially a best first search algorithm.
• This is informed search technique also called as HEURISTIC search. This algorithm
Works using heuristic value.
Working of A* Search algorithm:
A* Algorithm works as-
• It maintains a tree of paths originating at the start node.
• It extends those paths one edge at a time.
• It continues until its termination criterion is satisfied.
• A* Algorithm extends the path that minimizes the following function-
• Evaluation function

F (n) = g(n) + h(n)
Here,
• ‘n’ is the last node on the path
• g(n) is the cost of the path from start node to node ‘n’
• h(n) is a heuristic function that estimates cost of the cheapest path from
node ‘n’ to the goal node
Algorithm:
• The implementation of A* Algorithm involves maintaining two lists- OPEN and
CLOSED.
• OPEN contains those nodes that have been evaluated by the heuristic function but
have not been expanded into successors yet.
• CLOSED contains those nodes that have already been visited.
The algorithm is as follows-
Step-01:
• Define a list OPEN.
• Initially, OPEN consists solely of a single node, the start node S.
Step-02: If the list is empty, return failure and exit.
Step-03: Remove node n with the smallest value of f(n) from OPEN and move it to
list CLOSED. If node n is a goal state, return success and exit.
Step-04: Expand node n.
Step-05: If any successor to n is the goal node, return success and the solution by
tracing the path from goal node to S. Otherwise, go to Step-06.
Step-06: For each successor node, apply the evaluation function f to the node. If the
node has not been in either list, add it to OPEN.
Step-07: Go back to Step-02.

Example: 1
Consider the following graph,
• The numbers written on edges represent the distance between the nodes.
• The numbers written on nodes represent the heuristic value.
• Find the most cost-effective path to reach from start state A to final state J using A*
Algorithm.
Step-01:
• We start with node A.
• Node B and Node F can be reached from node A.
A* Algorithm calculates f (B) and f (F).
• f(B) = 6 + 8 = 14
• f(F) = 3 + 6 = 9
Since f (F) < f (B), so it decides to go to node F.
Path- A → F
Step-02:
• Node G and Node H can be reached from node F.
A* Algorithm calculates f (G) and f (H).
• f(G) = (3+1) + 5 = 9
• f(H) = (3+7) + 3 = 13
Since f (G) < f (H), so it decides to go to node G.

Path- A → F → G
Step-03:
• Node I can be reached from node G.
• A* Algorithm calculates f (I).
f (I) = (3+1+3) + 1 = 8
• It decides to go to node I.
Path- A → F → G → I
Step-04:
• Node E, Node H and Node J can be reached from node I.
• A* Algorithm calculates f (E), f (H) and f (J).
• f(E) = (3+1+3+5) + 3 = 15
• f(H) = (3+1+3+2) + 3 = 12
• f(J) = (3+1+3+3) + 0 = 10
• Since f (J) is least, so it decides to go to node J.
Path- A → F → G → I → J
• This is the required shortest path from node A to node J.
Solution:

Example: 4: 8 Puzzle problem using A* searching algorithm
Given an initial state of an 8-puzzle problem and final state to be reached-
Find the most cost-effective path to reach the final state from initial state using A*
Algorithm.
Consider,
G (n) = Depth of node
H (n) = Number of misplaced tiles.
Solution:
 A* Algorithm maintains a tree of paths originating at the initial state.
 It extends those paths one edge at a time.
 It continues until final state is reached.

Advantages of A* Searching
• A* Algorithm is one of the best path finding algorithms.
• It is Complete & Optimal
• Used to solve complex problems.
Disadvantages of A* searching
• Requires more memory
4. BEYOND CLASSICAL SEARCH:
• We have seen methods that systematically explore the search space, possibly using
principled pruning (e.g., A*)
What if we have much larger search spaces?
• Search spaces for some real-world problems may be much larger e.g., 1030,000 states
as in certain reasoning and planning tasks.
• Some of these problems can be solved by Iterative Improvement Methods.
Local search algorithm and optimization problem:
• In many optimization problems the goal state itself is the solution.
• The state space is a set of complete configurations.
• Search is about finding the optimal configuration (as in TSP) or just a feasible
configuration (as in scheduling problems).
• In such cases, one can use iterative improvement, or local search, methods.
• An evaluation, or objective, function h must be available that measures the quality of
each state.
• Main Idea: Start with a random initial configuration and make small, local changes
to it that improve its quality.
Hill Climbing Algorithm:
• In Hill-Climbing technique, starting at the base of a hill, we walk upwards until we
reach the top of the hill.
• In other words, we start with initial state and we keep improving the solution until
it’s optimal.
• It's a variation of a generate-and-test algorithm which discards all states which do not
look promising or seem unlikely to lead us to the goal state.
• To take such decisions, it uses heuristics (an evaluation function) which indicates
how close the current state is to the goal state.
Hill-Climbing = generate-and-test + heuristics
Feature of hill climbing Algorithm:
Following are some main features of Hill Climbing Algorithm:

• Generate and Test variant: Hill Climbing is the variant of Generate and Test
method. The Generate and Test method produce feedback which helps to decide
which direction to move in the search space.
• Greedy approach: Hill-climbing algorithm search moves in the direction which
optimizes the cost.
• No backtracking: It does not backtrack the search space, as it does not remember
the previous states.
Algorithm for Simple Hill Climbing:
o Step 1: Evaluate the initial state, if it is goal state then return success and Stop.
o Step 2: Loop Until a solution is found or there is no new operator left to apply.
o Step 3: Select and apply an operator to the current state.
o Step 4: Check new state:
a. If it is goal state, then return success and quit.
b. Else if it is better than the current state then assign new state as a current state.
c. Else if not better than the current state, then return to step2.
o Step 5: Exit.
Example:
• Key point while solving any hill-climbing problem is to choose an
appropriate heuristic function.
• Let's define such function h:
• h (x) = +1 for all the blocks in the support structure if the block is correctly
positioned otherwise -1 for all the blocks in the support structure.

Solution:
State-space Diagram for Hill Climbing:

 The state-space landscape is a graphical representation of the hill-climbing algorithm
which is showing a graph between various states of algorithm and Objective
function/Cost.
 On Y-axis we have taken the function which can be an objective function or cost
function, and state-space on the x-axis.
 If the function on Y-axis is cost then, the goal of search is to find the global minimum
and local minimum.
 If the function of Y-axis is Objective function, then the goal of the search is to find
the global maximum and local maximum.
• Local Maximum: Local maximum is a state which is better than its neighbor states, but
there is also another state which is higher than it.
• Global Maximum: Global maximum is the best possible state of state space landscape. It
has the highest value of objective function.
• Current state: It is a state in a landscape diagram where an agent is currently present.
• Flat local maximum: It is a flat space in the landscape where all the neighbor states of
current states have the same value.
• Shoulder: It is a plateau region which has an uphill edge.
5. TYPES OF HILL CLIMBING ALGORITHM:
o Simple hill Climbing
o Steepest-Ascent hill-climbing
o Stochastic hill Climbing
1. Simple Hill Climbing:
 Simple hill climbing is the simplest way to implement a hill climbing algorithm.
 It only evaluates the neighbor node state at a time and selects the first one which
optimizes current cost and set it as a current state.
 It only checks it's one successor state, and if it finds better than the current state, then
move else be in the same state.
 This algorithm has the following features:
o Less time consuming
o Less optimal solution and the solution is not guaranteed
Algorithm for Simple Hill Climbing:
o Step 1: Evaluate the initial state, if it is goal state then return success and Stop.
o Step 2: Loop Until a solution is found or there is no new operator left to apply.
o Step 3: Select and apply an operator to the current state.
o Step 4: Check new state:
a. If it is goal state, then return success and quit.

b. Else if it is better than the current state then assign new state as a current state.
c. Else if not better than the current state, then return to step2.
o Step 5: Exit.
2. Steepest-Ascent hill climbing:
 The steepest-Ascent algorithm is a variation of simple hill climbing algorithm.
 This algorithm examines all the neighboring nodes of the current state and selects
one neighbor node which is closest to the goal state.
 This algorithm consumes more time as it searches for multiple neighbors
Algorithm for Steepest-Ascent hill climbing:
o Step 1: Evaluate the initial state, if it is goal state then return success and stop, else
make current state as initial state.
o Step 2: Loop until a solution is found or the current state does not change.
a. Let SUCC be a state such that any successor of the current state will be better
than it.
b. For each operator that applies to the current state:
a. Apply the new operator and generate a new state.
b. Evaluate the new state.
c. If it is goal state, then return it and quit, else compare it to the SUCC.
d. If it is better than SUCC, then set new state as SUCC.
e. If the SUCC is better than the current state, then set current state to
SUCC.
o Step 5: Exit.
3. Stochastic hill climbing:
 Stochastic hill climbing does not examine for all its neighbor before moving.
 Rather, this search algorithm selects one neighbor node at random and decides
whether to choose it as a current state or examine another state.
6. SIMULATED ANNEALING:
 A hill-climbing algorithm which never makes a move towards a lower value
guaranteed to be incomplete because it can get stuck on a local maximum.

 And if algorithm applies a random walk, by moving a successor, then it may complete
but not efficient.
 Simulated Annealing is an algorithm which yields both efficiency and
completeness.
 In mechanical term Annealing is a process of hardening a metal or glass to a high
temperature then cooling gradually, so this allows the metal to reach a low-energy
crystalline state.
 The same process is used in simulated annealing in which the algorithm picks a
random move, instead of picking the best move.
 If the random move improves the state, then it follows the same path. Otherwise, the
algorithm follows the path which has a probability of less than 1 or it moves downhill
and chooses another path.
Genetic Algorithm:

7. LOCAL SEARCH IN CONTINUOUS SPACE:
 The distinction between discrete and continuous environments pointing out that most
real-world environments are continuous.
 A discrete variable or categorical variable is a type of statistical variable that can
assume only fixed number of distinct values.
 Continuous variable, as the name suggest is a random variable that assumes all the
possible values in a continuum.
 Which leads to a solution state required to reach the goal node.
 But beyond these “classical search algorithms," we have some “local search
algorithms” where the path cost does not matters, and only focus on solution-
state needed to reach the goal node.
o Example: Greedy BFS* Algorithm.
 A local search algorithm completes its task by traversing on a single current node rather
than multiple paths and following the neighbors of that node generally.
o Example: Hill climbing and simulated annealing can handle continuous
state and action spaces, because they have infinite branching factors.
Solution for Continuous Space:
 One way to avoid continuous problems is simply to discretize the neighborhood of
each state.
 Many methods attempt to use the gradient of the landscape to find a maximum. The
gradient of the objective function is a vector ∇f that gives the magnitude and direction
of the steepest slope.
Local search in continuous space:

Does the local search algorithm work for a pure optimized problem?
• Yes, the local search algorithm works for pure optimized problems.
• A pure optimization problem is one where all the nodes can give a solution. But the
target is to find the best state out of all according to the objective function.
• Unfortunately, the pure optimization problem fails to find high-quality solutions to reach
the goal state from the current state.
• Note: An objective function is a function whose value is either minimized or maximized
in different contexts of the optimization problems.
• In the case of search algorithms, an objective function can be the path cost for reaching
the goal node, etc.
Working of a Local search algorithm:

Problems in Hill Climbing Algorithm:
1. Local Maximum: A local maximum is a peak state in the landscape which is better than
each of its neighboring states, but there is another state also present which is higher than the
local maximum.
Solution: Backtracking technique can be a solution of the local maximum in state space
landscape. Create a list of the promising path so that the algorithm can backtrack the search
space and explore other paths as well.
2. Plateau: A plateau is the flat area of the search space in which all the neighbor states of
the current state contains the same value, because of this algorithm does not find any best
direction to move. A hill-climbing search might be lost in the plateau area.
Solution: The solution for the plateau is to take big steps or very little steps while searching,
to solve the problem. Randomly select a state which is far away from the current state so it
is possible that the algorithm could find non-plateau region.
3. Ridges: A ridge is a special form of the local maximum. It has an area which is higher
than its surrounding areas, but itself has a slope, and cannot be reached in a single move.

Solution: With the use of bidirectional search, or by moving in different directions, we can
improve this problem.
Conclusion:
• Local search often works well on large problems
– optimality
– Always has some answer available (best found so far)
8. SEARCHING WITH NON-DETERMINISTIC ACTIONS:
 In an environment, the agent can calculate exactly which state results from any sequence
of actions and always knows which state it is in.
 Searching with non-deterministic Actions
 Searching with partial observations
 When the environment is nondeterministic, percepts tell the agent which of the possible
outcomes of its actions has actually occurred.
 In a partially observable environment, every percept helps narrow down the set of
possible states the agent might be in, thus making it easier for the agent to achieve its
goals.
Example: Vacuum world, v2.0
• In the erratic vacuum world, the Suck action works as follows:
• When applied to a dirty square the action cleans the square and sometimes
cleans up dirt in an adjacent square, too.
• When applied to a clean square the action sometimes deposits dirt on the carpet.
• Solutions for nondeterministic problems can contain nested if–then–else statements; this
means that they are trees rather than sequences.
The eight possible states of the vacuum world; states 7 and 8 are goal states.
• Suck(p1, dirty)= (p1,clean) and sometimes (p2, clean)
• Suck(p1, clean)= sometimes (p1,dirty)
Solution: contingency plan
• [Suck, if State = 5 then [Right, Suck] else [ ]].
• nested if–then–else statements

AND–OR search trees:
• Non-deterministic action= there may be several possible outcomes
• Search space is an AND-OR tree
• Alternating OR and AND layers
• Find solution= search this tree using same methods.
• Solution in a non-deterministic search space
• Not simple action sequence
• Solution= subtree within search tree with:
• Goal node at each leaf (plan covers all contingencies)
• One action at each OR node
• A branch at AND nodes, representing all possible outcomes
• Execution of a solution = essentially
• The first two levels of the search tree for the erratic vacuum world.
• State nodes are OR nodes where some action must be chosen.
• At the AND nodes, shown as circles, every outcome must be handled, as indicated by
the arc linking the outgoing branches.
• The solution found is shown in bold lines.

i. Non-deterministic search trees:
 Start state = 1
 One solution:
o Suck,
o if(state=5) then [right, suck]
ii. Non-determinism: Actions that fail (Try, try again):
• Action failure is often a non-deterministic outcome
• Creates a cycle in the search tree.
• If no successful solution (plan) without a cycle:
• May return a solution that contains a cycle
• Represents retrying the action
• Infinite loop in plan execution?

• Depends on environment
• Action guaranteed to succeed eventually?
• In practice: can limit loops
• Plan no longer complete (could fail)
• Part of the search graph for the slippery vacuum world, where we have shown (some) cycles
explicitly.
• All solutions for this problem are cyclic plans because there is no way to move reliably.
9. SEARCHING WITH PARTIAL OBSERVATIONS:
 In a partially observable environment, every percept helps narrow down the set of possible
states the agent might be in, thus making it easier for the agent to achieve its goals.
 The key concept required for solving partially observable problems is the belief state.
o Belief state -representing the agent’s current belief about the possible physical
states.
 Searching with no observations
 Searching with observations

Conformant (sensorless) search: Example space:
• Belief state space for the super simple vacuum world
Observations:
– Only 12 reachable states. Versus 2^8= 256 possible belief states
– State space still gets huge very fast! à seldom feasible in practice
– We need sensors! à Reduce state space greatly!
i. Searching with no observations:
(a) Predicting the next belief state for the sensorless vacuum world with a deterministic
action, Right.
(b) Prediction for the same belief state and action in the slippery version of the sensorless
vacuum world.

ii. Searching with observations:
(a) In the deterministic world, Right is applied in the initial belief state, resulting in a new
belief state with two possible physical states; [B, Dirty] and [B, Clean].
(b) In the slippery world, Right is applied in the initial belief state, giving a new belief state
with four physical states; [A, Dirty], [B, Dirty], and [B, Clean].

10.ONLINE SEARCH AGENTS AND UNKNOWN ENVIRONMENTS:
 An online search problem must be solved by an agent executing actions, rather than by
pure computation.
 We assume a deterministic and fully observable environment but we stipulate that the
agent knows only the following:
o ACTIONS(s), which returns a list of actions allowed in state’s;
o The step-cost function c(s, a, s’)—note that this cannot be used until the agent
knows that s’ is the outcome; and
o GOAL-TEST(s).
Considered “offline” search problem
 Works “offline” à searches to compute a whole plan...before ever acting
 Even with percepts à gets HUGE fast in real world
 Lots of possible actions, lots of possible percepts...plus non-det.
Online search
 Idea: Search as you go. Interleave search + action
 Problem : actual percepts prune huge subtree of search space @ each move
 Condition: plan ahead less à don’t foresee problems

 Best case = wasted effort. Reverse actions and re-plan
 Worst case: not reversible actions. Stuck!
Online search only possible method in some worlds
 Agent doesn’t know what states exist (exploration problem)
 Agent doesn’t know what effect actions have (discovery learning)
 Possibly: do online search for a while
o until learn enough to do more predictive search
Example:
The nature of active online search:
Executing online search = algorithm for planning/acting
 Very different than offline search algorithm!
 Offline: search virtually for a plan in constructed search space...
 Can use any search algorithm, e.g., A* with strong h(n)
 A* can expand any node it wants on the frontier (jump around)
 Online agent: Agent literally is in some place!
 Agent is at one node (state) on frontier of search tree
 Can’t just jump around to other states...must plan from current state.

 (Modified) Depth first algorithms are ideal candidates!
 Heuristic functions remain critical!
 H (n) tells depth first which of the successors to explore!
 Admissibility remains relevant too: want to explore likely optimal paths first
 Real agent = real results. At some point I find the goal
 Can compare actual path cost to that predicted at each state by H(n)
 Competitive Ratio: Actual path cost/predicted cost. Lower is better.
 Could also be basis for developing (learning!) improved H (n) over time.
Online local search for agents:
• Hill-climbing is already an online search algorithm but stops at local optimum. How
about randomization?
• Cannot do random restart (you can’t teleport a robot)
• How about just a random walk instead of hill-climbing?
• Can be very bad (two ways back for every way forward above)
• Let’s augment HC with memory
• Learning real-time A* (LRTA*)
• Updates cost estimates, g(s), for the state it leaves
• Likes unexplored states
• f(s) = h(s) not g(s) + h(s) for unexplored states

LRTA* Example:
• We are in shaded state
LRTA* algorithm:

AI3391 ARTIFICIAL INTELLIGENCE UNIT II notes.pdf

More Related Content

What's hot (20)

Similar to AI3391 ARTIFICIAL INTELLIGENCE UNIT II notes.pdf (20)

More from Guru Nanak Technical Institutions (20)

Recently uploaded (20)

AI3391 ARTIFICIAL INTELLIGENCE UNIT II notes.pdf