The intelligence of environments 171
the book and projects meta-information (such as ratings, reviews, additional
information on the content) directly onto the cover or any open page. Moreover,
a wall may become a telephone dial, a display for travel information or a game
board, depending on the activity intended by the user. Beyond tangible practical
applications of this kind, the design leitmotif of Sixth Sense is that it shall “give
us seamless access and easy access to meta-information or information that may
exist somewhere and that may be relevant to help us make the right decision about
whatever it is that we are coming across” (Maes and Mistry 2009, TED Talk).
An interface with the capacities of Sixth Sense projects information and interactive elements into the environment that may, but neither need nor always do,
concern concrete world affairs within that environment. Hence, there will be situations where there is no information on objects or situations within that concrete
environment that would be projected into it. The information provided by Sixth
Sense may concern any other, related or unrelated world affair, such as the status
of one’s flight being projected onto any given wall or the back of the front seat
of one’s taxi to the airport. It may also concern fictional objects if gaming is
involved. Hence, the virtual nature of some object is indifferent to the convergence/isomorphism distinction.
At the same instance, and independently of the issue of the reference of the
information involved, a smooth integration of what is displayed with perceptual
uptake of natural information in a given environment has to be accomplished, in
order for the user not to become unduly distracted or irritated by the display. An
AR system of the kind and sophistication of either Sixth Sense or the Driver Focus
System is supposed to track both environmental variables and as users’ perspectives and behaviours with respect to those variables, and integrate them. Tracking
of either sort has become the predominant topic in AR research, and is acknowledged as one of its main design challenges (Zhou et al. 2008). Inasmuch as people
normally rely on a number of different, not exclusively visual cues when tracking
an object in their environment, a richer realistic impression of virtual objects in
AR will be provided by hybrid tracking across different sensory modalities (Feiner
2002, 53). In practice, however, these additional modalities are typically confined
to auditory and tactile feedback, at the expense both of the other senses and of genuine provision of information by non-visual means (FAR-a-2015, LFE-a-2015).
For systems that, like Sixth Sense but unlike the Driver Focus System, project virtual objects into the environment as related to the user, the movements
of a user’s body, head and eyes, and thus his or her position and movement relative to a virtual object, will be factored into the equation by which virtual object
is aligned with the physical objects surrounding it. The object’s changing shape
when viewed from different angles has to be accounted for as well as the effects
of changing lighting conditions or partial occlusion by physical objects. Thus,
besides an adequate design of digital objects, a correct and real-time calculation
of how to integrate an object into the users’ stream of perception is part of the
practical requirements for those AR systems that use the user’s surroundings as
the interface. Even for the more modest varieties of AR, the users’ perspective
and behaviours will have to be reliably tracked, in conjunction with the context in
the book and projects meta-information (such as ratings, reviews, additional
information on the content) directly onto the cover or any open page. Moreover,
a wall may become a telephone dial, a display for travel information or a game
board, depending on the activity intended by the user. Beyond tangible practical
applications of this kind, the design leitmotif of Sixth Sense is that it shall “give
us seamless access and easy access to meta-information or information that may
exist somewhere and that may be relevant to help us make the right decision about
whatever it is that we are coming across” (Maes and Mistry 2009, TED Talk).
An interface with the capacities of Sixth Sense projects information and interactive elements into the environment that may, but neither need nor always do,
concern concrete world affairs within that environment. Hence, there will be situations where there is no information on objects or situations within that concrete
environment that would be projected into it. The information provided by Sixth
Sense may concern any other, related or unrelated world affair, such as the status
of one’s flight being projected onto any given wall or the back of the front seat
of one’s taxi to the airport. It may also concern fictional objects if gaming is
involved. Hence, the virtual nature of some object is indifferent to the convergence/isomorphism distinction.
At the same instance, and independently of the issue of the reference of the
information involved, a smooth integration of what is displayed with perceptual
uptake of natural information in a given environment has to be accomplished, in
order for the user not to become unduly distracted or irritated by the display. An
AR system of the kind and sophistication of either Sixth Sense or the Driver Focus
System is supposed to track both environmental variables and as users’ perspectives and behaviours with respect to those variables, and integrate them. Tracking
of either sort has become the predominant topic in AR research, and is acknowledged as one of its main design challenges (Zhou et al. 2008). Inasmuch as people
normally rely on a number of different, not exclusively visual cues when tracking
an object in their environment, a richer realistic impression of virtual objects in
AR will be provided by hybrid tracking across different sensory modalities (Feiner
2002, 53). In practice, however, these additional modalities are typically confined
to auditory and tactile feedback, at the expense both of the other senses and of genuine provision of information by non-visual means (FAR-a-2015, LFE-a-2015).
For systems that, like Sixth Sense but unlike the Driver Focus System, project virtual objects into the environment as related to the user, the movements
of a user’s body, head and eyes, and thus his or her position and movement relative to a virtual object, will be factored into the equation by which virtual object
is aligned with the physical objects surrounding it. The object’s changing shape
when viewed from different angles has to be accounted for as well as the effects
of changing lighting conditions or partial occlusion by physical objects. Thus,
besides an adequate design of digital objects, a correct and real-time calculation
of how to integrate an object into the users’ stream of perception is part of the
practical requirements for those AR systems that use the user’s surroundings as
the interface. Even for the more modest varieties of AR, the users’ perspective
and behaviours will have to be reliably tracked, in conjunction with the context in