Artificial Intelligence
Download as PPTX, PDF2 likes1,194 views
The document discusses artificial intelligence (AI), defining it as a field focused on creating intelligent machines capable of data analysis and responses, while outlining various search algorithms such as depth-first search, breadth-first search, and A* algorithm. It explains techniques for problem representation, including state spaces and graphs, and introduces key concepts like forward and backward chaining in inference logic. Additionally, the document touches on simulated annealing as a method for solving optimization problems through random variations and temperature-based probability acceptance.
![ A technique to find a good solution to an optimization problem by trying random variations of the current
solution. A worse variation is accepted as the new solution with a probability that decreases as the
computation proceeds. The slower the cooling schedule, or rate of decrease, the more likely the
algorithm is to find an optimal or near-optimal solution.
Simulated annealing is a generalization of a Monte Carlo method for examining the equations of state
and frozen states of n-body systems [Metropolis et al. 1953]. The concept is based on the manner in
which liquids freeze or metals recrystalize in the process of annealing. In an annealing process a melt,
initially at high temperature and disordered, is slowly cooled so that the system at any time is
approximately in thermodynamic equilibrium. As cooling proceeds, the system becomes more ordered
and approaches a "frozen" ground state at T=0. Hence the process can be thought of as an adiabatic
approach to the lowest energy state. If the initial temperature of the system is too low or cooling is done
insufficiently slowly the system may become quenched forming defects or freezing out in metastable
states (ie. trapped in a local minimum energy state).
The original Metropolis scheme was that an initial state of a thermodynamic system was chosen at
energy E and temperature T, holding T constant the initial configuration is perturbed and the change in
energy dE is computed. If the change in energy is negative the new configuration is accepted. If the
change in energy is positive it is accepted with a probability given by the Boltzmann factor exp -(dE/T).
This processes is then repeated sufficient times to give good sampling statistics for the current
temperature, and then the temperature is decremented and the entire process repeated until a frozen
state is achieved at T=0.
By analogy the generalization of this Monte Carlo approach to combinatorial problems is straight
forward [Kirkpatrick et al. 1983, Cerny 1985]. The current state of the thermodynamic system is
analogous to the current solution to the combinatorial problem, the energy equation for the
thermodynamic system is analogous to at the objective function, and ground state is analogous to the
global minimum. The major difficulty (art) in implementation of the algorithm is that there is no obvious
analogy for the temperature T with respect to a free parameter in the combinatorial problem.
Furthermore, avoidance of entrainment in local minima (quenching) is dependent on the "annealing
schedule", the choice of initial temperature, how many iterations are performed at each temperature,
and how much the temperature is decremented at each step as cooling proceeds.
Simulated annealing has been used in various combinatorial optimization problems and has been
particularly successful in circuit design problems
Define simulated annealing?](https://image.slidesharecdn.com/presentation1-180325181223/85/Artificial-Intelligence-19-320.jpg)
Artificial Intelligence
- 1. Artificial Intelligence (AI Study Material) by Jay Nagar Email: nagarjay007@gmail.com Website: https://jaynagarblog.wordpress.com/ Call: 9601957620
- 2. What is artificial intelligence? AI refers to only a computer that is able to "seem" intelligent by analyzing data and producing a response. The science and engineering of making intelligent machines. This concept was founded on the belief that human intelligence "can be so precisely described that a machine can be made to simulate it." The central goals of AI research include reasoning, knowledge, planning, learning, natural language processing (communication), perception and the ability to move and manipulate objects. The AI field is interdisciplinary, including computer science, mathematics, psychology, linguistics, philosophy and neuroscience, as well as other specialized fields such as
- 3. State Space Representation and Search In this section we examine the concept of a state space and the different searches that can be used to explore the search space in order to find a solution. Before an AI problem can be solved it must be represented as a state space. The state space is then searched to find a solution to the problem. A state space essentially consists of a set of nodes representing each state of the problem, arcs between nodes representing the legal moves from one state to another, an initial state and a goal state. Each state space takes the form of a tree or a graph. Factors that determine which search algorithm or technique will be used include the type of the problem and the how the problem can be represented. Search techniques that will be examined in the course include: • Depth First Search • Depth First Search with Iterative Deepening • Breadth First Search • Best First Search • Hill Climbing • Branch and Bound Techniques • A* Algorithm
- 4. Depth First Search Depth First Search One of the searches that are commonly used to search a state space is the depth first search. The depth first search searches to the lowest depth or ply of the tree. Consider the tree in Figure. In this case the tree has been generated prior to the search Each node is not visited more than once. A depth first search of the this tree produces: A, B, E, K, S, L ,T, F, M, C, G, N, H, O, P, U, D, I, Q, J, R. Although in this example the tree was generated first and then a search of the tree was conducted. However, often there is not enough space to generate the entire tree representing state space and then search it. The algorithm presented below conducts a depth first search and
- 5. Depth First Search with Iterative Deepening Algorithm Depth First Search with Iterative Deepening Iterative deepening involves performing the depth first search on a number of different iterations with different depth bounds or cutoffs. On each iteration the cutoff is increased by one. There iterative process terminates when a solution path is found. Iterative deepening is more advantageous in cases where the state space representation has an infinite depth rather than a constant depth. The state space is searched at a deeper level on each iteration. DFID is guaranteed to find the shortest path. The algorithm does retain information between iterations and thus prior iterations are “wasted”. Depth First Iterative Deepening Algorithm procedure DFID (intial_state, goal_states) begin search_depth=1 while (solution path is not found) begin dfs(initial_state, goals_states) with a depth bound of search_depth increment search_depth by 1 endwhile end
- 6. Breadth First Search Breadth First Search The breadth first search visits nodes in a left to right manner at each level of the tree. Each node is not visited more than once. A breadth first search of the tree in Figure 5.1. produces A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U. Although in this example the tree was generated first and then a search of the tree was conducted. However, often there is not enough space to generate the entire tree representing state space and then search it. The
- 7. Bayesian network Inferring unobserved variables. Because a Bayesian network is a complete model for the variables and their relationships, it can be used to answer probabilistic queries about them. ... This process of computing the posterior distribution of variables given evidence is called probabilistic inference. Bayesian probability is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation representing a state of knowledge or as quantification of a
- 8. Graphs v/s Trees If states in the solution space can be revisited more than once a directed graph is used to represent the solution space. In a graph more than one move sequence can be used to get from one state to another. Moves in a graph can be undone. In a graph there is more than one path to a goal whereas in a tree a path to a goal is more clearly distinguishable. A goal state may need to appear more than once in a tree. Search algorithms for graphs have to cater for possible loops and cycles in the graph. Trees may be more “efficient” for representing such problems as loops and cycles do not have to be catered for. The entire tree or graph will not be generated. Figure illustrates the both a graph a representation of the same the states.
- 10. These characteristics are often used to classify problems in AI: Decomposable to smaller or easier problems Solution steps can be ignored or undone Predictable problem universe Good solutions are obvious Uses internally consistent knowledge base Requires lots of knowledge or uses knowledge to constrain solutions Requires periodic interaction between human and computer
- 11. This is a variety of depth-first (generate - and - test) search. A feedback is used here to decide on the direction of motion in the search space. In the depth-first search, the test function will merely accept or reject a solution. But in hill climbing the test function is provided with a heuristic function which provides an estimate of how close a given state is to goal state. The hill climbing test procedure is as follows : 1. General he first proposed solution as done in depth-first procedure. See if it is a solution. If so quit , else continue. 2. From this solution generate new set of solutions use , some application rules 3. For each element of this set (i) Apply test function. It is a solution quit. (ii) Else see whether it is closer to the goal state than the solution already generated. If yes, remember it else discard it. 4. Take the best element so far generated and use it as the next proposed solution. This step corresponds to move through the problem space in the direction Towards the goal state. 5. Go back to step 2. Sometimes this procedure may lead to a position, which is not a solution, but from which there is no move that improves things. This will happen if we have reached one of the following three states. (a) A "local maximum " which is a state better than all its neighbors , but is not better than some other states farther away. Local maxim sometimes occur with in sight of a solution. In such cases they are called " Foothills". (b) A "plateau'' which is a flat area of the search space, in which neighboring states have the same value. On a plateau, it is not possible to determine the best direction in which to move by making local comparisons. (c) A "ridge" which is an area in the search that is higher than the surrounding areas, but can not be searched in a simple move. To overcome theses problems we can: (a) Back track to some earlier nodes and try a different direction. This is a good way of dealing with local maxim. (b) Make a big jump an some direction to a new area in the search. This can be done by applying two more rules of the same rule several times, before testing. This is a good strategy is dealing with plate and ridges. Hill climbing becomes inefficient in large problem spaces, and when combinatorial explosion occurs. But it is a useful when combined with other methods. Hill Climbing Procedure:
- 12. The AO* ALGORITHM The problem reduction algorithm we just described is a simplification of an algorithm described in Martelli and Montanari, Martelli and Montanari and Nilson. Nilsson calls it the AO* algorithm , the name we assume. 1. Place the start node s on open. 2. Using the search tree constructed thus far, compute the most promising solution tree T 3. Select a node n that is both on open and a part of T. Remove n from open and place it on closed. 4. If n is a terminal goal node, label n as solved. If the solution of n results in any of n’s ancestors being solved, label all the ancestors as solved. If the start node s is solved, exit with success where T is the solution tree. Remove from open all nodes with a solved ancestor. 5. If n is not a solvable node (operators cannot be applied), label n as unsolvable. If the start node is labeled as unsolvable, exit with failure. If any of n’s ancestors become unsolvable because n is, label them unsolvable as well. Remove from open all nodes with unsolvable ancestors. 6. Otherwise, expand node n generating all of its successors. For each such successor node that contains more than one sub problem, generate their successors to give individual sub problems. Attach to each newly generated node a back pointer to its predecessor. Compute the cost estimate h* for each newly generated node and place all such nodes that do not yet have descendents on open. Next, recomputed the values of h* at n and each ancestor of n. 7. Return to step 2.
- 13. A* Algorithm2. A* algorithm: The best first search algorithm that was just presented is a simplification an algorithm called A* algorithm which was first presented by HART. Algorithm: Step 1: put the initial node on a list start Step 2: if (start is empty) or (start = goal) terminate search. Step 3: remove the first node from the start call this node a Step 4: if (a= goal) terminate search with success. Step 5: else if node has successors generate all of them estimate the fitness number of the successors by totaling the evaluation function value and the cost function value and sort the fitness number. Step 6: name the new list as start 1. Step 7: replace start with start 1. Step 8: go to step 2.
- 14. A Heuristic technique helps in solving problems, even though there is no guarantee that it will never lead in the wrong direction. There are heuristics of every general applicability as well as domain specific. The strategies are general purpose heuristics. In order to use them in a specific domain they are coupler with some domain specific heuristics. There are two major ways in which domain - specific, heuristic information can be incorporated into rule-based search procedure. - In the rules themselves - As a heuristic function that evaluates individual problem states and determines how desired they are. A heuristic function is a function that maps from problem state description to measures desirability, usually represented as number weights. The value of a heuristic function at a given node in the search process gives a good estimate of that node being on the desired path to solution. Well designed heuristic functions can provides a fairly good estimate of whether a path is good or not. ( " The sum of the distances traveled so far" is a simple heuristic function in the traveling salesman problem) . the purpose of a heuristic function is to guide the search process in the most profitable directions, by suggesting which path to follow first when more than one path is available. However in many problems, the cost of computing the value of a heuristic function would be more than the effort saved in the search process. Hence generally there is a trade-off between the cost of evaluating a heuristic function and the HEURISTIC FUNCTIONS:
- 15. Tower of Honai Problem.?
- 17. Generate and Test Procedure This is the simplest search strategy. It consists of the following steps; 1. Generating a possible solution for some problems; this means generating a particular point in the problem space. For others it may be generating a path from a start state. 2. Test to see if this is actually a solution by comparing the chosen point at the end point of the chosen path to the set of acceptable goal states. 3. If a solution has been found, quit otherwise return to step 1. The generate - and - Test algorithm is a depth first search procedure because complete possible solutions are generated before test. This can be implemented states are likely to appear often in a tree; it can be implemented on a search graph rather than a tree.
- 18. What is the difference between forward and backward chaining in artificial intelligence? An AI cannot give proofs somehow thinking and assuming meanings of statements. So to get the proofs there are set of rules that are fixed for inference logic and within that fixed set of rules we have forward and backward chaining. Forward chaining. It is also known as data driven inference technique. Forward chaining matches the set of conditions and infer results from these conditions. Basically, forward chaining starts from a new data and aims for any conclusion. It is bottom up reasoning. It is a breadth first search. It continues until no more rules can be applied or some cycle limit is met. For example: If it is cold then I will wear a sweater. Here “it is cold is the data” and “I will wear a sweater”is a decision. It was already known that it is cold that is why it was decided to wear a sweater, This process is forward chaining. It is mostly used in commercial applications i.e event driven systems are common example of forward chaining. It can create an infinite number of possible conclusions. Backward chaining. It is also called as goal driven inference technique. It is a backward search from goal to the conditions used to get the goal. Basically it starts from possible conclusion or goal and aims for necessary data. It is top down reasoning. It is a depth first search. It process operations in a backward direction from end to start, it will stop when the matching initial condition is met. For example: If it is cold then I will wear a sweater. Here we have our possible conclusion “I will wear a
- 19. A technique to find a good solution to an optimization problem by trying random variations of the current solution. A worse variation is accepted as the new solution with a probability that decreases as the computation proceeds. The slower the cooling schedule, or rate of decrease, the more likely the algorithm is to find an optimal or near-optimal solution. Simulated annealing is a generalization of a Monte Carlo method for examining the equations of state and frozen states of n-body systems [Metropolis et al. 1953]. The concept is based on the manner in which liquids freeze or metals recrystalize in the process of annealing. In an annealing process a melt, initially at high temperature and disordered, is slowly cooled so that the system at any time is approximately in thermodynamic equilibrium. As cooling proceeds, the system becomes more ordered and approaches a "frozen" ground state at T=0. Hence the process can be thought of as an adiabatic approach to the lowest energy state. If the initial temperature of the system is too low or cooling is done insufficiently slowly the system may become quenched forming defects or freezing out in metastable states (ie. trapped in a local minimum energy state). The original Metropolis scheme was that an initial state of a thermodynamic system was chosen at energy E and temperature T, holding T constant the initial configuration is perturbed and the change in energy dE is computed. If the change in energy is negative the new configuration is accepted. If the change in energy is positive it is accepted with a probability given by the Boltzmann factor exp -(dE/T). This processes is then repeated sufficient times to give good sampling statistics for the current temperature, and then the temperature is decremented and the entire process repeated until a frozen state is achieved at T=0. By analogy the generalization of this Monte Carlo approach to combinatorial problems is straight forward [Kirkpatrick et al. 1983, Cerny 1985]. The current state of the thermodynamic system is analogous to the current solution to the combinatorial problem, the energy equation for the thermodynamic system is analogous to at the objective function, and ground state is analogous to the global minimum. The major difficulty (art) in implementation of the algorithm is that there is no obvious analogy for the temperature T with respect to a free parameter in the combinatorial problem. Furthermore, avoidance of entrainment in local minima (quenching) is dependent on the "annealing schedule", the choice of initial temperature, how many iterations are performed at each temperature, and how much the temperature is decremented at each step as cooling proceeds. Simulated annealing has been used in various combinatorial optimization problems and has been particularly successful in circuit design problems Define simulated annealing?