Artificial Agents
39
2. Agent chooses an action to perform, a ∈ A(s t ).
3. As a consequence of its action, the environment changes its state, s(t+1),
and receives a reward or payoff, r t .
4. Based on inputs, the agent chooses a set of actions to help maximize
reward obtained.
Summarizing above, formally a multi-agent system should consist of the
following, in addition to Equation 2.1:
• A set of environment states, s ∈ S,
• A set of actions for agent, a ∈ A,
• A set of allowable actions at state s t for agent A(s t ) ⊆ A,
• The action chosen by agent at time t, a t ∈ A(s t ),
• A set of scalar rewards r t received by agent at time t dependent on how
it performed at time t − 1.
2.6 Objects or Agents?
Code can be objects or agents. The differences are summarized,
• An object is a term that accommodates object-oriented programming
principles, which allows objects to relate to other objects, through inheritance and attributes. Agents, however, are complete code pieces that
hold all data properties within itself.
• An object allows data size to be reduced by inheriting functions and
attributes from parent classes. An agent has a bigger size for an individual, as they contain data and functions with their memory. As agents
are isolated and work independently, this is a great advantage in parallel computing, when more than thousands of agents are deployed over
processors and minimum communication across processors is preferred.
If there is too much communication across processors, this increases
computational overhead of messages, introducing latency. All communicating agents can be placed on the same processor to reduce overhead.
These are load-balancing issues in parallel computing.
• Agents allow experimenting with bounded information principles.
• Agents can use machine learning techniques by learning. Multi-agent
learning can be both cooperative and competitive learning.
39
2. Agent chooses an action to perform, a ∈ A(s t ).
3. As a consequence of its action, the environment changes its state, s(t+1),
and receives a reward or payoff, r t .
4. Based on inputs, the agent chooses a set of actions to help maximize
reward obtained.
Summarizing above, formally a multi-agent system should consist of the
following, in addition to Equation 2.1:
• A set of environment states, s ∈ S,
• A set of actions for agent, a ∈ A,
• A set of allowable actions at state s t for agent A(s t ) ⊆ A,
• The action chosen by agent at time t, a t ∈ A(s t ),
• A set of scalar rewards r t received by agent at time t dependent on how
it performed at time t − 1.
2.6 Objects or Agents?
Code can be objects or agents. The differences are summarized,
• An object is a term that accommodates object-oriented programming
principles, which allows objects to relate to other objects, through inheritance and attributes. Agents, however, are complete code pieces that
hold all data properties within itself.
• An object allows data size to be reduced by inheriting functions and
attributes from parent classes. An agent has a bigger size for an individual, as they contain data and functions with their memory. As agents
are isolated and work independently, this is a great advantage in parallel computing, when more than thousands of agents are deployed over
processors and minimum communication across processors is preferred.
If there is too much communication across processors, this increases
computational overhead of messages, introducing latency. All communicating agents can be placed on the same processor to reduce overhead.
These are load-balancing issues in parallel computing.
• Agents allow experimenting with bounded information principles.
• Agents can use machine learning techniques by learning. Multi-agent
learning can be both cooperative and competitive learning.