172 Environments of intelligence
which they obtain. What is he or she looking at? What object is he or she following? What is relevant to him or her in the present context? Tracking the domains
of information relevant to the human counterpart will require convergence with
natural informational domains.
To cite issues that arose in trials of another gesture-based interface, albeit one
more modest than Sixth Sense and concerned with gestural control functions for
utility vehicles, it was observed (as reported in FAR-a-2015) first, that the available vocabulary of unequivocally discernible gestures is very limited with respect
to the set of commands required for effective and comprehensive control of the
vehicle. Second, that natural gestures inadvertently produced by the operator
which are not intended as commands are easily mistaken as commands by the
system – a problem referred to as the “Midas Touch” problem: just as the mythical
king’s wish that everything he touches turns to gold coming true turned against
him once he reached for food and drink, one would wish to be able discern what
shall be a command and what a natural gesture when interacting with a gesturebased interface. Disambiguation, then, will require invocation commands or similar indicators of the proper context of a gesture to be available to the system, so as
to provide the domain of command gestures with explicit and reliably trackable
boundaries.
To sum up the importance of informational convergence in AR systems: such
convergence may but need not obtain with respect to affairs in a shared environment. Instead of convergence with natural information for perception, more general isomorphisms might be the relation of choice, depending on application and
usage involved. However, what has to converge in either case is the information
required for successful interaction between AR system and user, and hence the
cues by which objects and activities are recognised. If convergence is achieved
on either or on both levels, human beings and AR systems will share a focus
on certain objects of reference or towards certain conditions in their common
surroundings, thereby implying some kind of symmetrical relationship: both are
situated in a shared environment of interaction, where the success of both human
beings’ actions and the systems’ behaviours depends on shared attention to relevant conditions in that environment, and on the stability of the conditions under
which they interact. If successful, the goal states of the behaviours towards an
object, condition or activity may indeed be identical, in terms of a certain task
being fulfilled cooperatively.
However, this observation does not imply that the same conditions will necessarily be relevant in the same way to both sides. The particular purposes of
the human and artificial participants in the situation may remain at variance, and
may have to be so for the sake of accomplishing a given task. To the extent that
virtual objects are integrated in real environments in AR, the function of AR is to
simulate, in a different medium, some of the key properties of the environment in
relation to human perception. In contrast to attempts at imitating human-like features for the artificial counterparts in an interaction, as in Turing’s imitation game,
conversational agents or social robotics, the imitation involved in AR concerns
features of the environment as they present themselves to the perceiving human
which they obtain. What is he or she looking at? What object is he or she following? What is relevant to him or her in the present context? Tracking the domains
of information relevant to the human counterpart will require convergence with
natural informational domains.
To cite issues that arose in trials of another gesture-based interface, albeit one
more modest than Sixth Sense and concerned with gestural control functions for
utility vehicles, it was observed (as reported in FAR-a-2015) first, that the available vocabulary of unequivocally discernible gestures is very limited with respect
to the set of commands required for effective and comprehensive control of the
vehicle. Second, that natural gestures inadvertently produced by the operator
which are not intended as commands are easily mistaken as commands by the
system – a problem referred to as the “Midas Touch” problem: just as the mythical
king’s wish that everything he touches turns to gold coming true turned against
him once he reached for food and drink, one would wish to be able discern what
shall be a command and what a natural gesture when interacting with a gesturebased interface. Disambiguation, then, will require invocation commands or similar indicators of the proper context of a gesture to be available to the system, so as
to provide the domain of command gestures with explicit and reliably trackable
boundaries.
To sum up the importance of informational convergence in AR systems: such
convergence may but need not obtain with respect to affairs in a shared environment. Instead of convergence with natural information for perception, more general isomorphisms might be the relation of choice, depending on application and
usage involved. However, what has to converge in either case is the information
required for successful interaction between AR system and user, and hence the
cues by which objects and activities are recognised. If convergence is achieved
on either or on both levels, human beings and AR systems will share a focus
on certain objects of reference or towards certain conditions in their common
surroundings, thereby implying some kind of symmetrical relationship: both are
situated in a shared environment of interaction, where the success of both human
beings’ actions and the systems’ behaviours depends on shared attention to relevant conditions in that environment, and on the stability of the conditions under
which they interact. If successful, the goal states of the behaviours towards an
object, condition or activity may indeed be identical, in terms of a certain task
being fulfilled cooperatively.
However, this observation does not imply that the same conditions will necessarily be relevant in the same way to both sides. The particular purposes of
the human and artificial participants in the situation may remain at variance, and
may have to be so for the sake of accomplishing a given task. To the extent that
virtual objects are integrated in real environments in AR, the function of AR is to
simulate, in a different medium, some of the key properties of the environment in
relation to human perception. In contrast to attempts at imitating human-like features for the artificial counterparts in an interaction, as in Turing’s imitation game,
conversational agents or social robotics, the imitation involved in AR concerns
features of the environment as they present themselves to the perceiving human