about implementation of the pedagogical principles and course design, usefulness of
the support materials, and attainment of the course objectives, (b) applied the same
evaluation framework, and (c) used the same instruments for data collection and
analysis. The course evaluation was guided by two main questions:
A. To what extent were the four pedagogical principles implemented as intended?
B. To what extent did the ICT in IBSE course achieve its four objectives?
Question B involves the evaluation of the effects of the course on participants,
which resulted from actual implementation of the pedagogical principles, the course
design, and the support materials. The evaluation of this actual implementation was
guided by Question A and based on a comparison between (a) the intended course
programme and (b) the actual activities of participants during the course. To evaluate
attainment of the course objectives, we first operationalised performance levels for
each objective. The definition of these levels was based on theoretical considerations
and aligned with time-constraint conditions of the course. After that, we collected
data, compared the data-analysis outcomes with the pre-defined levels of the course
objectives, and concluded which level(s) of each objective the participants achieved.
This evaluation framework includes instruments for data collection and analysis
(i.e. pre-course, post-course, and follow-up questionnaires; observations and video
recording of live sessions and classroom tryouts; participants’ ICT in IBSE lesson
plans and self-evaluation reports of the classroom tryouts; computer performance
test for each tool; the inquiry-analysis inventory; and the communication records).
With these instruments, data were collected from a variety of sources and by
different data collectors (i.e. the researcher, the course instructor, course participants). Accordingly, we could record both intended and possibly unintended outcomes as the course was implemented. Most outcomes were evaluated by more than
one instrument thus allowing for data triangulation.
In the Dutch case study, we further operationalised the pedagogical principles in
the initial scenario of the ICT in IBSE course. With “scenario”, we mean the
Fig. 12.10 Design and research process including an explorative phase and a cyclic-research phase
with three case studies in the Netherlands (NL), Slovakia (SK) and Vietnam (VN)
12 Stem, Inquiry Practices and Technology in Physics Education
151
the support materials, and attainment of the course objectives, (b) applied the same
evaluation framework, and (c) used the same instruments for data collection and
analysis. The course evaluation was guided by two main questions:
A. To what extent were the four pedagogical principles implemented as intended?
B. To what extent did the ICT in IBSE course achieve its four objectives?
Question B involves the evaluation of the effects of the course on participants,
which resulted from actual implementation of the pedagogical principles, the course
design, and the support materials. The evaluation of this actual implementation was
guided by Question A and based on a comparison between (a) the intended course
programme and (b) the actual activities of participants during the course. To evaluate
attainment of the course objectives, we first operationalised performance levels for
each objective. The definition of these levels was based on theoretical considerations
and aligned with time-constraint conditions of the course. After that, we collected
data, compared the data-analysis outcomes with the pre-defined levels of the course
objectives, and concluded which level(s) of each objective the participants achieved.
This evaluation framework includes instruments for data collection and analysis
(i.e. pre-course, post-course, and follow-up questionnaires; observations and video
recording of live sessions and classroom tryouts; participants’ ICT in IBSE lesson
plans and self-evaluation reports of the classroom tryouts; computer performance
test for each tool; the inquiry-analysis inventory; and the communication records).
With these instruments, data were collected from a variety of sources and by
different data collectors (i.e. the researcher, the course instructor, course participants). Accordingly, we could record both intended and possibly unintended outcomes as the course was implemented. Most outcomes were evaluated by more than
one instrument thus allowing for data triangulation.
In the Dutch case study, we further operationalised the pedagogical principles in
the initial scenario of the ICT in IBSE course. With “scenario”, we mean the
Fig. 12.10 Design and research process including an explorative phase and a cyclic-research phase
with three case studies in the Netherlands (NL), Slovakia (SK) and Vietnam (VN)
12 Stem, Inquiry Practices and Technology in Physics Education
151