170 Environments of intelligence
A failure of informational convergence, so conceived, between what a driver
could perceive and what an assistance system is capable of registering appears to
be responsible for the first-ever fatal accident of a semi-autonomous vehicle on
a Florida highway in May 2016. According to the accident investigation report,
the car was moderately speeding at the time of the accident while “the driver
was operating the car using the advanced driver assistance features Traffic-Aware
Cruise Control and Autosteer lane keeping assistance” (National Transportation Safety Board 2016). Notably, the car manufacturer refers to these features
as an “autopilot” function (Tesla Motors 2016). The systems involved, as several reports indicate, failed to distinguish between the side of a plain-coloured
white semi-tractor trailer that was crossing the highway at right angle and the
bright sky in its background (Knight 2016a; Reuters 2016; Tesla Motors 2016).
It appears that the driver on the highway neither tried to intervene when the other
vehicle infringed his right of way nor paid any attention to the traffic situation
in the first place (some witnesses suggest that he was watching a film instead,
see Reuters 2016). Issues with distinguishing bright, plain-coloured objects from
the bright sky were known to the manufacturer, and detecting obstacles crossing
the path of the vehicle at right angle was not part of the sensor’s specifications
(Knight 2016b). At the same instance, the car, being a semi-autonomous vehicle
without a genuine autopilot function, included a feature to monitor the driver’s
hands remaining at the steering wheel. This single-cue driver monitoring feature,
which was not coupled with the traffic monitoring and steering system, was supposed both to issue warnings to the driver and to slow down the vehicle if his or
her hands remained off the wheel for a certain period (Tesla Motors 2016), but
appears to have been ineffective for some reason. Hence, the information available to the traffic monitoring system did not match the visual information that the
driver would have had available if he had kept the traffic situation in view, and
no information was exchanged between traffic and driver monitoring systems that
could have corrected for that situation. All of these sets of information would have
had to be integrated for convergence to obtain.
The second, historically slightly older example is the mobile gesture-based
interface Sixth Sense, developed at the MIT Media Lab (Mistry et al. 2009) where,
however, it did not progress beyond an advanced prototype status. The basic concept of that interface is to provide information and afford activities relevant to the
context of whatever may be the user’s current purposes independently of a fixed
physical interface. The system consists of a small camera and a small projector
that, sufficiently miniaturised, could be worn in front of the torso like a pendant,
and that are networked with a mobile device operating in the background. When
looking forward, camera and projector are usually equally oriented in viewing
direction. With small, maximally intuitively designed gestures, objects or people
can be recognised, relevant information can be retrieved from the Internet or interaction possibilities can be invoked without further user input; drawings can be
created, photos can be taken and virtual objects projected onto a suitable surface
can be manipulated. For example it should be possible for the user to draw a real,
physical book from the shelf and flip through it, while an application recognises
A failure of informational convergence, so conceived, between what a driver
could perceive and what an assistance system is capable of registering appears to
be responsible for the first-ever fatal accident of a semi-autonomous vehicle on
a Florida highway in May 2016. According to the accident investigation report,
the car was moderately speeding at the time of the accident while “the driver
was operating the car using the advanced driver assistance features Traffic-Aware
Cruise Control and Autosteer lane keeping assistance” (National Transportation Safety Board 2016). Notably, the car manufacturer refers to these features
as an “autopilot” function (Tesla Motors 2016). The systems involved, as several reports indicate, failed to distinguish between the side of a plain-coloured
white semi-tractor trailer that was crossing the highway at right angle and the
bright sky in its background (Knight 2016a; Reuters 2016; Tesla Motors 2016).
It appears that the driver on the highway neither tried to intervene when the other
vehicle infringed his right of way nor paid any attention to the traffic situation
in the first place (some witnesses suggest that he was watching a film instead,
see Reuters 2016). Issues with distinguishing bright, plain-coloured objects from
the bright sky were known to the manufacturer, and detecting obstacles crossing
the path of the vehicle at right angle was not part of the sensor’s specifications
(Knight 2016b). At the same instance, the car, being a semi-autonomous vehicle
without a genuine autopilot function, included a feature to monitor the driver’s
hands remaining at the steering wheel. This single-cue driver monitoring feature,
which was not coupled with the traffic monitoring and steering system, was supposed both to issue warnings to the driver and to slow down the vehicle if his or
her hands remained off the wheel for a certain period (Tesla Motors 2016), but
appears to have been ineffective for some reason. Hence, the information available to the traffic monitoring system did not match the visual information that the
driver would have had available if he had kept the traffic situation in view, and
no information was exchanged between traffic and driver monitoring systems that
could have corrected for that situation. All of these sets of information would have
had to be integrated for convergence to obtain.
The second, historically slightly older example is the mobile gesture-based
interface Sixth Sense, developed at the MIT Media Lab (Mistry et al. 2009) where,
however, it did not progress beyond an advanced prototype status. The basic concept of that interface is to provide information and afford activities relevant to the
context of whatever may be the user’s current purposes independently of a fixed
physical interface. The system consists of a small camera and a small projector
that, sufficiently miniaturised, could be worn in front of the torso like a pendant,
and that are networked with a mobile device operating in the background. When
looking forward, camera and projector are usually equally oriented in viewing
direction. With small, maximally intuitively designed gestures, objects or people
can be recognised, relevant information can be retrieved from the Internet or interaction possibilities can be invoked without further user input; drawings can be
created, photos can be taken and virtual objects projected onto a suitable surface
can be manipulated. For example it should be possible for the user to draw a real,
physical book from the shelf and flip through it, while an application recognises