Part B | 22.2
504 Part B Autonomous Ocean Vehicles, Subsystems and Control
To adapt to inherent uncertainties, various learning
methods can be employed. Machine learning with reinforcement is a powerful tool; however, its extension
to multiagent and large dimensions is not trivial. The
mathematics for reinforcement learning are similar to
the CE approach. Learning is also equivalent to the setting of specific parameters in a POMDP. Learning for
CAs can be executed via genetic algorithms. Learning
in Influence Diagrams is formulated as an optimization
and/or fixed point problem (i. e., solvable by iterative
application of an optimization operation).
In general, learning works better if the parameter space is smaller and if the free parameters can
be treated independently in the optimization process.
An open research question is whether best preplanned
collaboration exists. From a coordination expert system point of view it could be a decision algorithm on
what to do under specific situations. This implies that
for a large number of specific situations, a large number of coordination settings have to be stored, which
becomes impractical in realistic scenarios. A possible solution is the creation of a team template library
as an algorithmic version of the trading methodology mentioned in the Introduction. The algorithmic
approach first identifies critical parameters for a specific situation. Optimization with respect to coordination is then performed at this identified criticality
ridge. Since situations are constantly changing, both
a similarity measure for situations (this situation is
similar to the one we had yesterday where we successfully applied the following coordination scheme)
and a fast identification method for critical parameters are needed. For the construction of such a fast
identification method, the heuristic of blind separation techniques could be applied: a critical parameter changes the system behavior independently of the
value settings of other parameters. Information science offers the tools to design, store and implement
situational and process-specific details in ontologies.
Ontologies represent all bottom-up (the devil is in
the detail) information. The team template library is
where all detailed specific bottom-up information is
linked to the criteria of applicable coordination schemes
(which can also be described as algorithms representing the top-down approach to cooperative tasks).
Sections 22.3–22.5 further elaborate on the trading
methodology that incorporates both heuristic and expert knowledge to generate and evaluate coordination
schemes.
Coordination for cooperative target tracking is
a tool to allow cooperation. Coordination methods differ in the amount of data transfer they need, the speed
of convergence, and the sensitivity to error.
Coordination schemes applied in cooperative systems (multirobot, human societies, business processes,
and information science) are:
Divide and conquer: no interaction necessary after
planning phase.
Decentralization due to a limited sensor range
[22.44].
Decentralization due to limited control access.
Auction: bidding for a task. An auction is a process
of buying and selling services by offering them up
for bid, taking bids, and then selling the item to the
highest bidder. The auction procedure has to be set
up before the execution of the auction.
Voting: recommendations based on ratings or
rankings collected from many sources. Voting is
a method for a group such as a meeting or an electorate to make a decision or express an opinion. The
majority wins in a typical voting procedure.
Consensus: recommendations based on ratings or
rankings collected from many sources, Consensus
decision making is a group decision-making process
that seeks the consent of all participants. Consensus
may be defined as an acceptable resolution, one that
can be supported by all, even if not the favorite of
each individual [22.45, 46].
Game theoretic: an equilibrium state in which each
player has achieved the maximum that can be
achieved (i. e., any further move will decrease the
payoff).
Bio-inspired: similar to ants forming teams for
transporting heavy pieces of food, bio-inspired cooperative target tracking can be interpreted to be
a trial-and-error approach on the information gathering task of finding and holding track of a target.
Adaptive hierarchy: explicitly allowing changes of
hierarchical structures at run time leads to coordination schemes adaptive to specific situations the team
is facing in cooperative target tracking, for example,
the surveillance units can reason about a reorganization of the team structure in the case of a sudden
loss of target contact [22.47].
Synchronized covertness: the design idea is to exploit the covertness of receivers, but at the same
time allows for coordinated activities among them.
The covertness of receivers is important when dealing with stealth targets which can avoid detection
only in a limited number of geometric setups. If
the stealth targets have no knowledge about the
receivers positions, they cannot adjust their path
planning to exploit their stealth design in these geometric setups. The mixed architecture which results
from the design idea of synchronized covertness
504 Part B Autonomous Ocean Vehicles, Subsystems and Control
To adapt to inherent uncertainties, various learning
methods can be employed. Machine learning with reinforcement is a powerful tool; however, its extension
to multiagent and large dimensions is not trivial. The
mathematics for reinforcement learning are similar to
the CE approach. Learning is also equivalent to the setting of specific parameters in a POMDP. Learning for
CAs can be executed via genetic algorithms. Learning
in Influence Diagrams is formulated as an optimization
and/or fixed point problem (i. e., solvable by iterative
application of an optimization operation).
In general, learning works better if the parameter space is smaller and if the free parameters can
be treated independently in the optimization process.
An open research question is whether best preplanned
collaboration exists. From a coordination expert system point of view it could be a decision algorithm on
what to do under specific situations. This implies that
for a large number of specific situations, a large number of coordination settings have to be stored, which
becomes impractical in realistic scenarios. A possible solution is the creation of a team template library
as an algorithmic version of the trading methodology mentioned in the Introduction. The algorithmic
approach first identifies critical parameters for a specific situation. Optimization with respect to coordination is then performed at this identified criticality
ridge. Since situations are constantly changing, both
a similarity measure for situations (this situation is
similar to the one we had yesterday where we successfully applied the following coordination scheme)
and a fast identification method for critical parameters are needed. For the construction of such a fast
identification method, the heuristic of blind separation techniques could be applied: a critical parameter changes the system behavior independently of the
value settings of other parameters. Information science offers the tools to design, store and implement
situational and process-specific details in ontologies.
Ontologies represent all bottom-up (the devil is in
the detail) information. The team template library is
where all detailed specific bottom-up information is
linked to the criteria of applicable coordination schemes
(which can also be described as algorithms representing the top-down approach to cooperative tasks).
Sections 22.3–22.5 further elaborate on the trading
methodology that incorporates both heuristic and expert knowledge to generate and evaluate coordination
schemes.
Coordination for cooperative target tracking is
a tool to allow cooperation. Coordination methods differ in the amount of data transfer they need, the speed
of convergence, and the sensitivity to error.
Coordination schemes applied in cooperative systems (multirobot, human societies, business processes,
and information science) are:
Divide and conquer: no interaction necessary after
planning phase.
Decentralization due to a limited sensor range
[22.44].
Decentralization due to limited control access.
Auction: bidding for a task. An auction is a process
of buying and selling services by offering them up
for bid, taking bids, and then selling the item to the
highest bidder. The auction procedure has to be set
up before the execution of the auction.
Voting: recommendations based on ratings or
rankings collected from many sources. Voting is
a method for a group such as a meeting or an electorate to make a decision or express an opinion. The
majority wins in a typical voting procedure.
Consensus: recommendations based on ratings or
rankings collected from many sources, Consensus
decision making is a group decision-making process
that seeks the consent of all participants. Consensus
may be defined as an acceptable resolution, one that
can be supported by all, even if not the favorite of
each individual [22.45, 46].
Game theoretic: an equilibrium state in which each
player has achieved the maximum that can be
achieved (i. e., any further move will decrease the
payoff).
Bio-inspired: similar to ants forming teams for
transporting heavy pieces of food, bio-inspired cooperative target tracking can be interpreted to be
a trial-and-error approach on the information gathering task of finding and holding track of a target.
Adaptive hierarchy: explicitly allowing changes of
hierarchical structures at run time leads to coordination schemes adaptive to specific situations the team
is facing in cooperative target tracking, for example,
the surveillance units can reason about a reorganization of the team structure in the case of a sudden
loss of target contact [22.47].
Synchronized covertness: the design idea is to exploit the covertness of receivers, but at the same
time allows for coordinated activities among them.
The covertness of receivers is important when dealing with stealth targets which can avoid detection
only in a limited number of geometric setups. If
the stealth targets have no knowledge about the
receivers positions, they cannot adjust their path
planning to exploit their stealth design in these geometric setups. The mixed architecture which results
from the design idea of synchronized covertness