190
lization,” etc. from this map. When the user wants to generate maps from more detailed
viewpoints, he/she can specify kinds of concepts and relationships to follow. When we
want to know what countermeasures are appropriate for the focused problem, we can
obtain another map using the system by selecting the problem as the starting point for
an ontology exploration. This map suggests the utility of the system for facilitating
policy making processes by stimulating policy makers with such maps demonstrating
possible relations between problems and possible countermeasures against them.
The goal of the second step is consensus making with the help of the system
through discussion among stakeholders with the maps they generated. The system
integrates all the maps generated by them to enhance differences and commonalities
among those maps which facilitate mutual understanding among participants. The
integrated map thus helps them reach a consensus. Furthermore, the system is
equipped with a touch table display which is shared by all the stakeholders as shown
in Fig. 12.3. They stand around the table to observe and manipulate the integrated
map through the user-friendly touch interface during the discussion.
12.4 Usability and Effectiveness of the System
12.4.1 Evaluation Experiment by Domain Experts
To assess the effectiveness of the mapping tool, the authors asked four domain
experts to use the tool and evaluate its practical performance (Fig. 12.5). After basic
instruction regarding its use, they created 13 conceptual maps (3 or 4 maps per
expert) within an hour in accordance with their specific interests. Then they chose
61 conceptual paths (linkages between concepts in a map) from the 13 maps; they
explored and evaluated the paths with a four-level scale (4, very important or interesting; 3, important or interesting; 2, relevant, but neither important nor interesting;
1, wrong path). As a result, 30 paths (49%) were graded as level 4, 22 paths (36%)
as level 3, 8 paths (13%) as level 2, and 1 path (2%) as level 1; thus 85% of the
selected paths were evaluated as level 3 or level 4. Although one should not exaggerate the tool’s performance based on an experiment with such few samples, the
experimental result suggests its practical applicability and effectiveness to some
extent and provides useful feedback for its improvement (Kozaki et al. 2011).
12.4.2 An Experiment of Consensus Making by Role-Play
Discussion
12.4.2.1 Overview of the Experiment
The goal of this experiment is to explore the feasibility of system. In the experiment
we assigned a couple of subjects roles of stakeholders related to biofuel production
and policy making for it and ask them to discuss the related topics by role-playing
K. Kozaki et al.
lization,” etc. from this map. When the user wants to generate maps from more detailed
viewpoints, he/she can specify kinds of concepts and relationships to follow. When we
want to know what countermeasures are appropriate for the focused problem, we can
obtain another map using the system by selecting the problem as the starting point for
an ontology exploration. This map suggests the utility of the system for facilitating
policy making processes by stimulating policy makers with such maps demonstrating
possible relations between problems and possible countermeasures against them.
The goal of the second step is consensus making with the help of the system
through discussion among stakeholders with the maps they generated. The system
integrates all the maps generated by them to enhance differences and commonalities
among those maps which facilitate mutual understanding among participants. The
integrated map thus helps them reach a consensus. Furthermore, the system is
equipped with a touch table display which is shared by all the stakeholders as shown
in Fig. 12.3. They stand around the table to observe and manipulate the integrated
map through the user-friendly touch interface during the discussion.
12.4 Usability and Effectiveness of the System
12.4.1 Evaluation Experiment by Domain Experts
To assess the effectiveness of the mapping tool, the authors asked four domain
experts to use the tool and evaluate its practical performance (Fig. 12.5). After basic
instruction regarding its use, they created 13 conceptual maps (3 or 4 maps per
expert) within an hour in accordance with their specific interests. Then they chose
61 conceptual paths (linkages between concepts in a map) from the 13 maps; they
explored and evaluated the paths with a four-level scale (4, very important or interesting; 3, important or interesting; 2, relevant, but neither important nor interesting;
1, wrong path). As a result, 30 paths (49%) were graded as level 4, 22 paths (36%)
as level 3, 8 paths (13%) as level 2, and 1 path (2%) as level 1; thus 85% of the
selected paths were evaluated as level 3 or level 4. Although one should not exaggerate the tool’s performance based on an experiment with such few samples, the
experimental result suggests its practical applicability and effectiveness to some
extent and provides useful feedback for its improvement (Kozaki et al. 2011).
12.4.2 An Experiment of Consensus Making by Role-Play
Discussion
12.4.2.1 Overview of the Experiment
The goal of this experiment is to explore the feasibility of system. In the experiment
we assigned a couple of subjects roles of stakeholders related to biofuel production
and policy making for it and ask them to discuss the related topics by role-playing
K. Kozaki et al.