2. Things to cover in this lecture
• Agents and environments
• Rationality
• PEAS (Performance measure, Environment,
Actuators, Sensors)
• Environment types
• Agent types
3. Agents
• An agent is anything that can be viewed as perceiving
its environment through sensors and acting upon
that environment through actuators
• Human agent:
– sensors :eyes, ears, and other organs;
– Actuators: hands, legs, mouth, and other body
• Robotic agent: cameras and infrared range finders for
sensors; various motors for actuators
4. Agents and environments
• The agent function maps from percept histories to actions:
• [f: P* A]
• The agent program runs on the physical architecture to produce f
• agent = architecture + program
6. A vacuum-cleaner agent
Percept Sequence Action
[A, Clean] Right
[A, Dirty] Pick_Dirt
[B, Clean] Left
[B, Dirty] Pick_Dirt
[A, Clean], [A, Clean] Right
[A, Clean], [A, Dirty] Pick_Dirt
…
7. Rational agents
• An agent should strive to "do the right thing", based on
what it can perceive and the actions it can perform. The
right action is the one that will cause the agent to be most
successful
• Performance measure: An objective criterion for success of
an agent's behavior
• E.g., performance measure of a vacuum-cleaner agent
could be amount of dirt cleaned up, amount of time taken,
amount of electricity consumed, amount of noise
generated, etc.
8. Rational agents
• Rationality depends on four things at any given
point in time
– PEAS: Performance measure, Environment, Actuators,
Sensors
• Rational Agent: For each possible percept
sequence, a rational agent should select an action
that is expected to maximize its performance
measure, given the evidence provided by the
percept sequence and whatever built-in knowledge
the agent has.
9. Rational agents traits
• Rationality is distinct from omniscience (all-knowing
with infinite knowledge, knowing the actual outcome
of an action)
• Agents can perform actions in order to modify future
percepts so as to obtain useful information
(information gathering, exploration)
• An agent is autonomous if its behavior is determined
by its own experience (with ability to learn and adapt)
10. PEAS
• PEAS: Performance measure, Environment,
Actuators, Sensors.
• Must first specify the PEAS setting for intelligent
agent design, collectively called the task
environment
• Consider, e.g., the task of designing an automated
taxi driver:
• Performance measure
• Environment
• Actuators
• Sensors
11. PEAS
• Must first specify the setting for intelligent agent
design.
• Consider, e.g., the task of designing an automated
taxi driver:
• Performance measure: Safe, fast, legal, comfortable trip,
maximize profits
– Environment: Roads, other traffic, pedestrians, customers
– Actuators: Steering wheel, accelerator, brake, signal, horn
– Sensors: Cameras, sonar, speedometer, GPS, odometer,
engine sensors, keyboard
13. PEAS
• Agent: Part-picking robot
• Performance measure: Percentage of parts in
correct bins
• Environment: Conveyor belt with parts, bins
• Actuators: Jointed arm and hand
• Sensors: Camera, joint angle sensors
14. PEAS
• Agent: Interactive English tutor
• Performance measure: Maximize student's
score on test
• Environment: Set of students
• Actuators: Screen display (exercises,
suggestions, corrections)
• Sensors: Keyboard
15. Environment Types
• Fully Observable (vs. Partially Observable): An agent's
sensors give it access to the complete state of the
environment at each point in time
• In FO, the agent can be confident that it requires
nothing more unobservable to decide on the optimal
action
• Partially Observable: environment with noise ,
inaccurate sensor data or missing the sensor data ( due
to some fault etc.)
• unobservable environment- agent with no sensor at all.
– Vacuum cleaner: failure of local dirt sensor.
16. Environment Types
• Single Agent (vs. Multi-Agent): An agent
operating by itself in an environment
– In the multi-agent case, the performance
measure of one agent depends on the
performance measures of the other agent(s)
– Competitive multi-agent: Chess Playing
– Collaborative multi-agent: Robo Soccer.
– A quite complicated field which is currently the
focus of much research.
16
17. Environment Types
• Deterministic (vs. Stochastic): The next state of the
environment is completely determined by the
current state and the action executed by the agent
– Stochastic (Non-Deterministic): There can be more than
one next state, for a given state-action combination
– Vacuum cleaner example. Random dirt
• Consider a Multi-agent environment
– If the environment is deterministic except for the actions
of other agents, then the environment is strategic
– Strategy Games
– Chess example
17
18. Environment Types
• Episodic (vs. sequential): The agent's experience
is divided into atomic ”episodes”
– Each episode consists of the agent perceiving and
then performing a single action, and the choice of
action in each episode depends only on the episode
itself, e.g., a robot whose job is to detect faulty parts
on a line in some factory
– In a sequential setting, the next episode depends on
the previous one(s), e.g., learning which chess move
to execute at each sequential step, in order to win the
game at the end
– Also called a sequential decision process.
18
19. Environment Types
• Static (vs. Dynamic): The environment is unchanged
while an agent is deliberating which action to execute
– Much simpler to deal with
– For the dynamic case, the agent needs to keep track of the changes
– The environment is semi-dynamic if the environment itself does not
change with the passage of time, but the agent's performance score
does, e.g., chess with clock.
– Puzzle vs chess with clock vs taxi
• Discrete (vs. Continuous): A limited number of
distinct, clearly defined percepts and actions.
– Puzzle vs taxi
20. Environment types
Chess with Chess without Taxi driving
a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No
• The environment type largely determines the agent design
21. Agent types
• Four basic types in order of increasing
generality:
– Simple Reflex agents
– Model-based Reflex agents
– Goal-based agents
– Utility-based agents
• And Finally: Learning agents
23. Simple Reflex Agents
• Automated Taxi:
– Agent observes rain falling on the
windshield: Agent powers on the
viper
– Agent observes a red signal; Agent
breaks the taxi until it stops.
25. Model-based Reflex Agents
• Automated Taxi:
– Changing Lane
– Need to know where other cars are if the current view provide
partial information
• Robo-Soccer Example:
– Imagine a robotic goalkeeper
– It can build a model of the dynamics of the game that is played
on the field, e.g., when the ball is kicked in its direction, the ball
will be nearer to it in the next time step
– If this robot is not able to acquire its state at some time step,
then using the model, it knows that the ball has come nearer
– It also know what consequences a dive will have
– So, it can time its dive early and hence, save the goal.
27. Goal-based Agents
• Automated Taxi:
– Consider the agent at a crossing, where it can
turn right, left, or go straight
– Using the model, the Agent can understand the
consequences of turning left, right or going
straight ahead
– All 3 might seem the optimal actions to take
– However, the agent needs to select one of these
actions in order to reach the destination of the
passenger.
27
29. Utility-based Agent
• Automated Taxi:
– Consider the agent at a crossing, where it can turn
right, left, or go straight
– The agent will calculate the utility of each such
action
– It will select the action which maximizes the utility
function, i.e., in most cases, the expected profit
that the agent can expect to receive in the long
run (when the passenger reaches the destination)
– E.g., going straight could have highest utility.
29
31. Learning Agents
• Programming agents by hand can be very tedious. “Some
more expeditious method seem desirable” (Alan Turing, 1950).
• Four conceptual components:
– Learning element: responsible for making improvements
– Performance element: responsible for selecting external actions. It is what we
considered as an agent so far.
– Critic: How well is the agent is doing w.r.t. a fixed
performance standard.
– Problem generator: allows the agent to explore.
Editor's Notes
#1:Most generic definition, suits scientific development, rationality is well defined mathematical concept
(don’t mimic birds to fly, understand aerodynamics to fly)
![Agents and environments
• The agent function maps from percept histories to actions:
• [f: P* A]
• The agent program runs on the physical architecture to produce f
• agent = architecture + program](https://image.slidesharecdn.com/lec-2-rationalagents-250807055828-8d8af230/85/rational-agent-it-is-a-lecture-for-ai-course-4-320.jpg)
![Vacuum-cleaner world
• Percepts: location and contents, e.g., [A, Dirty]
• Actions: Left, Right, Suck, NoOp](https://image.slidesharecdn.com/lec-2-rationalagents-250807055828-8d8af230/85/rational-agent-it-is-a-lecture-for-ai-course-5-320.jpg)
![A vacuum-cleaner agent
Percept Sequence Action
[A, Clean] Right
[A, Dirty] Pick_Dirt
[B, Clean] Left
[B, Dirty] Pick_Dirt
[A, Clean], [A, Clean] Right
[A, Clean], [A, Dirty] Pick_Dirt
…](https://image.slidesharecdn.com/lec-2-rationalagents-250807055828-8d8af230/85/rational-agent-it-is-a-lecture-for-ai-course-6-320.jpg)
