Part B | 22.1
502 Part B Autonomous Ocean Vehicles, Subsystems and Control
An important output of a mobile vehicle controller
is path planning [22.20, 21], especially for distributed
sensors in a surveillance mission [22.22]. Path planning can be myopic (with receding horizon) or nonmyopic. The planning horizon in an energetic ocean
environment is highly variable, and targets are often
unpredictable. Executing initially good plans without
adapting to new conditions is risky. Thus, extrema in
the cost function solution surface, which is a formulation of the risk, will vary with time in the absence of
adaptive pathways.
22.1.2 Multiagent Approach
In a surveillance scenario, targets and target trackers
can all be viewed as agents. Some agents may communicate and cooperate, some may be passive, some
reactive. The agent paradigm is helpful because it
explicitly encapsulates features that in the CE representation are implicit. In a cooperative system, each agent
seeks to maximize a joint global reward function.
Cooperative target tracking can be formally described as a decentralized partially observable Markov
decision process (DEC-POMDP). Since standard procedures for solving POMDPs are computationally intractable for scenarios of large dimensions, approximations play a major role (as in the CE approach), and the
art is in adapting the way approximations are made to
the structure of the interactions [22.24–28]. The local
policy for agent I is a mapping from the local history
of observations made by agent I to an action. To proTimeline or comments
Target team description
Tracking team description
Decision process by tracking team, reactive
Target team has a state,
tracking team has a state
Nature makes a decision
on how to change
the measurements
Team makes their decision
on their own state
and on the state of the
opponent and on their
next movement
Nature makes a decision
on how to influence the
movement of the target
team and the tracking team
Tracking team has a new
state
t t
t+1 t+1
Fig. 22.1 An influence diagram (a special type of PGM) contains nodes that represent chance variables (circles), decisions
(rectangles), and utility values (the utility values are similar to
the attributes discussed in the CE representation) (diamonds) (after [22.23])
gram the autonomy in the cooperative target-tracking
team members, this local policy has to be found. This is
equal to finding the control design in the CE approach.
The tracking of reactive targets is the field of multiagent partially observable stochastic games (POSG).
POSG formulations can be found in [22.29]. Formulated for noncooperative multiagent systems, they allow
each agent to have a unique reward function. Further
analysis of Bayesian games can be found in [22.30].
22.1.3 Cellular Automata Approach
A cellular automaton consists of a regular grid of cells,
each in one of a finite number of states. The grid can
be in any finite number of dimensions. For each cell,
a set of cells called neighborhood is defined relative to
the specific cell. An initial state is selected by assigning a state for each cell. A new generation is created
by some fixed rule which can include stochastic and
asynchronous elements. When analyzing systems for
cooperative target tracking by cellular automata (CA),
simplifications are made on the internal activities, and
the focus is set on the interactions in the neighborhood of the cell. Often these interactions generate group
behavior that allows each member to be maximally independent of the target.
Linking the CA approach to the MAS approach,
each tracking agent has interactions with the other
agents and the target. For many agents, such interactions result in a complex system. Self-organization methods like random graphs and finite state machines are used for discrete event control in distributed
surveillance [22.31, 32]. A general description of such
complex, nonlinear systems gives a useful insight, in
emergent or swarming behaviors of the entire system [22.33]. Starting with many CAs, which are all
independent of each other, but depend on the target,
they trade their independence among each other, to become more independent of the target. In CA language,
this is a formulation of the main idea behind the trading
methodology discussed in the introduction. In a swarm
of fish, for example, the individuals have a strong dependency to their nearest neighbors, but in the swarm
the individuals are well protected against enemies.
22.1.4 Probabilistic Graphical Models
Approach
The power of this approach is best described by a citation [22.34]:
Probabilistic graphical models (PGM) encode our
domain knowledge declaratively, using a representation that has its own inherent semantics. Thus the
502 Part B Autonomous Ocean Vehicles, Subsystems and Control
An important output of a mobile vehicle controller
is path planning [22.20, 21], especially for distributed
sensors in a surveillance mission [22.22]. Path planning can be myopic (with receding horizon) or nonmyopic. The planning horizon in an energetic ocean
environment is highly variable, and targets are often
unpredictable. Executing initially good plans without
adapting to new conditions is risky. Thus, extrema in
the cost function solution surface, which is a formulation of the risk, will vary with time in the absence of
adaptive pathways.
22.1.2 Multiagent Approach
In a surveillance scenario, targets and target trackers
can all be viewed as agents. Some agents may communicate and cooperate, some may be passive, some
reactive. The agent paradigm is helpful because it
explicitly encapsulates features that in the CE representation are implicit. In a cooperative system, each agent
seeks to maximize a joint global reward function.
Cooperative target tracking can be formally described as a decentralized partially observable Markov
decision process (DEC-POMDP). Since standard procedures for solving POMDPs are computationally intractable for scenarios of large dimensions, approximations play a major role (as in the CE approach), and the
art is in adapting the way approximations are made to
the structure of the interactions [22.24–28]. The local
policy for agent I is a mapping from the local history
of observations made by agent I to an action. To proTimeline or comments
Target team description
Tracking team description
Decision process by tracking team, reactive
Target team has a state,
tracking team has a state
Nature makes a decision
on how to change
the measurements
Team makes their decision
on their own state
and on the state of the
opponent and on their
next movement
Nature makes a decision
on how to influence the
movement of the target
team and the tracking team
Tracking team has a new
state
t t
t+1 t+1
Fig. 22.1 An influence diagram (a special type of PGM) contains nodes that represent chance variables (circles), decisions
(rectangles), and utility values (the utility values are similar to
the attributes discussed in the CE representation) (diamonds) (after [22.23])
gram the autonomy in the cooperative target-tracking
team members, this local policy has to be found. This is
equal to finding the control design in the CE approach.
The tracking of reactive targets is the field of multiagent partially observable stochastic games (POSG).
POSG formulations can be found in [22.29]. Formulated for noncooperative multiagent systems, they allow
each agent to have a unique reward function. Further
analysis of Bayesian games can be found in [22.30].
22.1.3 Cellular Automata Approach
A cellular automaton consists of a regular grid of cells,
each in one of a finite number of states. The grid can
be in any finite number of dimensions. For each cell,
a set of cells called neighborhood is defined relative to
the specific cell. An initial state is selected by assigning a state for each cell. A new generation is created
by some fixed rule which can include stochastic and
asynchronous elements. When analyzing systems for
cooperative target tracking by cellular automata (CA),
simplifications are made on the internal activities, and
the focus is set on the interactions in the neighborhood of the cell. Often these interactions generate group
behavior that allows each member to be maximally independent of the target.
Linking the CA approach to the MAS approach,
each tracking agent has interactions with the other
agents and the target. For many agents, such interactions result in a complex system. Self-organization methods like random graphs and finite state machines are used for discrete event control in distributed
surveillance [22.31, 32]. A general description of such
complex, nonlinear systems gives a useful insight, in
emergent or swarming behaviors of the entire system [22.33]. Starting with many CAs, which are all
independent of each other, but depend on the target,
they trade their independence among each other, to become more independent of the target. In CA language,
this is a formulation of the main idea behind the trading
methodology discussed in the introduction. In a swarm
of fish, for example, the individuals have a strong dependency to their nearest neighbors, but in the swarm
the individuals are well protected against enemies.
22.1.4 Probabilistic Graphical Models
Approach
The power of this approach is best described by a citation [22.34]:
Probabilistic graphical models (PGM) encode our
domain knowledge declaratively, using a representation that has its own inherent semantics. Thus the