Agents1

Artiﬁcial Intelligence
Intelligent Agents

Agents and environments
• Agent: An agent is anything that can be viewed as:
– perceiving its environment through sensors and
– acting upon that environment through actuators.

• An agent program runs in cycles of: (1)perceive, (2)think,
and (3)act.

• An agent program runs in cycles of: (1)perceive, (2)think,
and (3)act.
• Agent = Architecture + Program

• Human agent:
– Sensors: eyes, ears, and other organs.
– Actuators: hands, legs, mouth, and other body parts.
• Robotic agent:
– Sensors: Cameras and infrared range ﬁnders.
– Actuators: Various motors.

• Human agent:
– Sensors: eyes, ears, and other organs.
– Actuators: hands, legs, mouth, and other body parts.
• Robotic agent:
– Sensors: Cameras and infrared range ﬁnders.
– Actuators: Various motors.
• Agents everywhere!
– Thermostat
– Cell phone
– Vacuum cleaner
– Robot
– Alexa Echo
– Self-driving car
– Human
– etc.

Vacuum cleaner
A B
• Percepts: location and contents e.g., [A, Dirty]
• Actions: Left, Right, Suck, NoOp
• Agent function: mapping from percepts to actions.

Vacuum cleaner
A B
• Percepts: location and contents e.g., [A, Dirty]
• Agent function: mapping from percepts to actions.
Percept Action
[A, clean] Right
[A, dirty] Suck
[B, clean] Left
[B, dirty] Suck

Well-behaved agents
Rational Agent:
“For each possible percept sequence, a rational agent should select
an action that is expected to maximize its performance measure,
given the evidence provided by the percept sequence and whatever
built-in knowledge the agent has.”

Rationality
• Rationality is relative to a performance measure.
• Judge rationality based on:
– The performance measure that deﬁnes the criterion of suc-
cess.
– The agent prior knowledge of the environment.
– The possible actions that the agent can perform.
– The agent’s percept sequence to date.

PEAS
• When we deﬁne a rational agent, we group these properties
under PEAS, the problem speciﬁcation for the task environ-
ment.
• The rational agent we want to design for this task environment
is the solution.
• PEAS stands for:
– Performance
– Environment
– Actuators
– Sensors

PEAS
What is PEAS for a self-driving car?
• Performance:
• Environment:
• Actuators:
• Sensors:

PEAS
• Performance: Safety, time, legal drive, comfort.
• Environment:
• Actuators:
• Sensors:

PEAS
• Environment: Roads, other cars, pedestrians, road signs.
• Actuators:
• Sensors:

PEAS
• Actuators: Steering, accelerator, brake, signal, horn.
• Sensors:

PEAS
• Actuators: Steering, accelerator, brake, signal, horn.
• Sensors: Camera, sonar, GPS, Speedometer, odometer, ac-
celerometer, engine sensors, keyboard.

PEAS
How about a vacuum cleaner?
iRobot Roomba series

PEAS
• Performance: cleanness, eﬃciency: distance traveled to clean,
battery life, security.
• Environment:
• Actuators:
• Sensors:

PEAS
• Environment: room, table, wood ﬂoor, carpet, diﬀerent ob-
stacles.
• Actuators:
• Sensors:

PEAS
stacles.
• Actuators: wheels, diﬀerent brushes, vacuum extractor.
• Sensors:

PEAS
stacles.
• Actuators: wheels, diﬀerent brushes, vacuum extractor.
• Sensors: camera, dirt detection sensor, cliﬀ sensor, bump sen-
sors, infrared wall sensors.

Environment types
• Fully observable (vs. partially observable): An agent’s
sensors give it access to the complete state of the environment
at each point in time.
• Deterministic (vs. stochastic): The next state of the en-
vironment is completely determined by the current state and
the action executed by the agent. (If the environment is de-
terministic except for the actions of other agents, then the
environment is strategic)
• Episodic (vs. sequential): The agent’s experience is divided
into atomic ”episodes” (each episode consists of the agent
perceiving and then performing a single action), and the choice
of action in each episode depends only on the episode itself.

Environment types
• Static (vs. dynamic): The environment is unchanged while
an agent is deliberating. (The environment is semi-dynamic
if the environment itself does not change with the passage of
time but the agent’s performance score does.)
• Discrete (vs. continuous): A limited number of distinct,
clearly deﬁned percepts and actions. E.g., checkers is an ex-
ample of a discrete environment, while self-driving car evolves
in a continuous one.
• Single agent (vs. multi-agent): An agent operating by itself
in an environment.
• Known (vs. Unknown): The designer of the agent may or
may not have knowledge about the environment makeup. If
the environment is unknown the agent will need to know how
it works in order to decide.

Agent types
• Four basic types in order of increasing generality:
– Simple reﬂex agents
– Model-based reﬂex agents
– Goal-based agents
– Utility-based agents
• All of which can be generalized into learning agents that can
improve their performance and generate better actions.

Simple reﬂex agents
• Simple reﬂex agents select an action based on the current state
only ignoring the percept history.
• Simple but limited.
• Can only work if the environment is fully observable, that is
the correct action is based on the current percept only.
Agent
Environment
Sensors
What action I
should do nowCondition-action rules
Actuators
What the world
is like now

Vacuum (reﬂex) agent
A B
• Let’s write the algorithm for the Vacuum cleaner...
• Percepts: location and content (location sensor, dirt sensor).
Percept Action
[A, clean] Right
[A, dirty] Suck
[B, clean] Left
[B, dirty] Suck

Vacuum (reﬂex) agent
A B
• Let’s write the algorithm for the Vacuum cleaner...
• Percepts: location and content (location sensor, dirt sensor).
Percept Action
[A, clean] Right
[A, dirty] Suck
[B, clean] Left
[B, dirty] Suck
What if the vacuum agent is deprived from its location sen-
sor?

Model-based reﬂex agents
• Handle partial observability by keeping track of the part of the
world it can’t see now.
• Internal state depending on the percept history (best guess).
• Model of the world based on (1) how the world evolves in-
dependently from the agent, and (2) how the agent actions
aﬀects the world.
Agent
Environment
Sensors
State
How the world evolves
What my actions do
Condition-action rules
Actuators
What the world
is like now
What action I
should do now

Goal-based agents
• Knowing the current state of the environment is not enough.
The agent needs some goal information.
• Agent program combines the goal information with the envi-
ronment model to choose the actions that achieve that goal.
• Consider the future with “What will happen if I do A?”
• Flexible as knowledge supporting the decisions is explicitly rep-
resented and can be modiﬁed.
Agent
Environment
Sensors
What action I
should do now
State
What my actions do
Actuators
What the world
is like now
What it will be like
if I do action A
Goals

Utility-based agents
• Sometimes achieving the desired goal is not enough. We may
look for quicker, safer, cheaper trip to reach a destination.
• Agent happiness should be taken into consideration. We call
it utility.
• A utility function is the agent’s performance measure
• Because of the uncertainty in the world, a utility agent choses
the action that maximizes the expected utility.
Agent
Environment
Sensors
How happy I will be
in such a state
State
What my actions do
Utility
Actuators
What action I
should do now
What it will be like
if I do action A
What the world
is like now

Learning agents
• Programming agents by hand can be very tedious. “Some
more expeditious method seem desirable” Alan Turing, 1950.
• Four conceptual components:
– Learning element: responsible for making improvements
– Performance element: responsible for selecting external ac-
tions. It is what we considered as agent so far.
– Critic: How well is the agent is doing w.r.t. a ﬁxed perfor-
mance standard.
– Problem generator: allows the agent to explore.
Performance standard
Agent
Environment
Sensors
Performance
element
changes
knowledge
learning
goals
Problem
generator
feedback
Learning
element
Critic
Actuators

Agent’s organization
a) Atomic Representation: Each state of the world is a black-
box that has no internal structure. E.g., ﬁnding a driving route,
each state is a city. AI algorithms: search, games, Markov decision
processes, hidden Markov models, etc.

b) Factored Representation: Each state has some attribute-
value properties. E.g., GPS location, amount of gas in the tank.
AI algorithms: constraint satisfaction, and Bayesian networks.

c) Structured Representation: Relationships between the ob-
jects of a state can be explicitly expressed. AI algorithms: ﬁrst or-
der logic, knowledge-based learning, natural language understand-
ing.

Intelligent agents
• The concept of intelligent agent is central in AI.
• AI aims to design intelligent agents that are useful, reactive,
autonomous and even social and pro-active.
• An agent perceives its environment through percept and acts
through actuators.
• A performance measure evaluates the behavior of the agent.
• An agent that acts to maximize its expected performance mea-
sure is called a rational agent.
• PEAS: A task environment speciﬁcation that includes Perfor-
mance measure, Environment, Actuators and Sensors.
Agent = Architecture + Program

Intelligent agents
• Four types of agents: Reﬂex agents, model-based agents, goal-
based agents, and utility-based agents.
• Agents can improve their performance through learning.
• This is a high-level present of agent programs.
• States representations: atomic, factored, structured. Increas-
ing expressiveness power.

Intelligent agents
• Four types of agents: Reﬂex agents, model-based agents, goal-
based agents, and utility-based agents.
• Agents can improve their performance through learning.
• This is a high-level present of agent programs.
• States representations: atomic, factored, structured. Increas-
ing expressiveness power.
Credit: Courtesy Percy Liang

Credit
• Artiﬁcial Intelligence, A Modern Approach. Stuart Russell and
Peter Norvig. Third Edition. Pearson Education.
http://aima.cs.berkeley.edu/

Agents1

More Related Content

What's hot (20)

Similar to Agents1 (20)

More from Amar Jukuntla (18)

Recently uploaded (20)

Agents1