conducted on a small scale to pre-test the instrument in the
preparation of the main study. Many serious errors can be
avoided by taking the time to make an adequate pre-test.
Dillman’s (2000) suggestion was used as guidelines for the
pilot study. First, the survey questionnaire was reviewed by
the experts, researchers and knowledgeable colleagues to
ensure its relevancy, completeness, and appropriateness. The
construct validity was achieved by a discussion of survey
instrument with research experts to check if the questions are
appropriate for measuring the variables. The suggestions
from the supervisor and the research experts were taken into
consideration. The content validity was checked by a discussion on questions with a non-native instructor. An English instructor was requested to check the questionnaire for
typographical errors. Lastly, a pilot study was conducted to
get reliable and robust data that could lead the research
towards reliable results.
6.6 Instrument Reliability
Evaluating the reliability of any survey instrument is an
essential part of the analysis. Saunders et al. (2009, 2016)
stated that reliability means that if the research is conducted
over different periods, the results should be consistent. The
reliability is “the extent to which your data collection techniques or analyzing procedures will yield consistent findings” (Saunders et al. 2009, p. 156). The Cronbach alpha
reliability coefficient is used for measuring the reliability of
the instrument (Im, Il Hong, Seongtae Kang, 2011). In this
research, Cronbach’s alpha is one of the main elements to
evaluate the overall reliability of each variable, and the
instrument as well. The value of Cronbach’s alpha ranges
from 0 to 1. According to George and Mallery (2003), as a
rule of thumb, Alpha values should be more than 0.7.
A value of Cronbach’s alpha more than 0.7 is taken as a
sufficient and reliable value (Schutte, Toppinnen, Kalimo &
Schaufeli, 2000). Although the scales and items for the
constructs of UTAUT2 were adapted from Venkatesh et al.
(2003, 2012) that have already been tested and validated, it
would be more appropriate to re-assess the reliability and
validity of the instrument again because of its novel context
(Nistor, Lerche, Weinberger, Ceobanu & Heymann, 2014).
6.7 Instrument Validity
According to Smith, Passmore and Faught (2009), validity is
about not only the instrument itself but also about the outcome. Srivastava (2010) described that content validity,
criterion validity, and construct validity are three types of
empirical research validity. To achieve the validity, the six
criteria that must be satisfied are highlighted by Messick
(1995) are that: (a) content is representative and relevant,
(b) content is substantive and is supported by a proper theory, (c) the instrument structure must be the representative of
the given construct, (d) outcomes should have generalizability strength (e) the outcome obtained should have the
ability to be related to other external variables, and (f) the
consequences and implications of the yielded outcome
should have a significant and consequential impact.
Although the questions adopted from Venkatesh et al. (2003,
2012) were validated by previous researchers, new items of
the constructs must also be content-validated. For this
research, to ensure that the instrument is constructed properly, instructors from various institutions such as KFUPM,
DU, DCC, and other research experts having a thorough
knowledge of technology adoption models were consulted to
assess the content validity of the instrument.
6.8 Confidentiality and Ethics
After getting ethical approval from the researcher’s institution,
the survey link was sent to the Director of Information and
Technology Centre (ITC) to distribute the survey questionnaire
to all instructors. The higher management of other universities
was also contacted to send the survey link to their instructors for
filling the survey. In the survey, the instructors were given a
briefing regarding the objectives and use of the research findings. The consent letter of voluntary participation was to assure
that there would not be any physical or psychological harm to
the participants and personal information would be kept confidential by the researcher. Ethical principles regarding Saudi
society and the Islamic religion were also considered.
6.9 Summary
In this study, the Saunders et al. (2016)’s layered onion model
was used to choose the most appropriate research method to
answer the research questions and to meet the objectives of this
study. In this chapter, the philosophy of research, methodology,
and strategy to be used were discussed. Data collection, analysis procedures, and their justification are also discussed in this
chapter. The strategies to improve the validity and to enhance
the response rate were also discussed. This chapter also discussed how to enhance the reliability and validity of the study
so that any possible bias could be minimized.
References
Baker, T. L. (1994). Doing social research (2nd ed). McGraw-Hill Inc.
Bell, J. (1993). Doing your research project: A guide to first-time
researchers in education and social science. Milton Keynes: Open
56
6 How Did We Find Evidence for LMS Adoption?
preparation of the main study. Many serious errors can be
avoided by taking the time to make an adequate pre-test.
Dillman’s (2000) suggestion was used as guidelines for the
pilot study. First, the survey questionnaire was reviewed by
the experts, researchers and knowledgeable colleagues to
ensure its relevancy, completeness, and appropriateness. The
construct validity was achieved by a discussion of survey
instrument with research experts to check if the questions are
appropriate for measuring the variables. The suggestions
from the supervisor and the research experts were taken into
consideration. The content validity was checked by a discussion on questions with a non-native instructor. An English instructor was requested to check the questionnaire for
typographical errors. Lastly, a pilot study was conducted to
get reliable and robust data that could lead the research
towards reliable results.
6.6 Instrument Reliability
Evaluating the reliability of any survey instrument is an
essential part of the analysis. Saunders et al. (2009, 2016)
stated that reliability means that if the research is conducted
over different periods, the results should be consistent. The
reliability is “the extent to which your data collection techniques or analyzing procedures will yield consistent findings” (Saunders et al. 2009, p. 156). The Cronbach alpha
reliability coefficient is used for measuring the reliability of
the instrument (Im, Il Hong, Seongtae Kang, 2011). In this
research, Cronbach’s alpha is one of the main elements to
evaluate the overall reliability of each variable, and the
instrument as well. The value of Cronbach’s alpha ranges
from 0 to 1. According to George and Mallery (2003), as a
rule of thumb, Alpha values should be more than 0.7.
A value of Cronbach’s alpha more than 0.7 is taken as a
sufficient and reliable value (Schutte, Toppinnen, Kalimo &
Schaufeli, 2000). Although the scales and items for the
constructs of UTAUT2 were adapted from Venkatesh et al.
(2003, 2012) that have already been tested and validated, it
would be more appropriate to re-assess the reliability and
validity of the instrument again because of its novel context
(Nistor, Lerche, Weinberger, Ceobanu & Heymann, 2014).
6.7 Instrument Validity
According to Smith, Passmore and Faught (2009), validity is
about not only the instrument itself but also about the outcome. Srivastava (2010) described that content validity,
criterion validity, and construct validity are three types of
empirical research validity. To achieve the validity, the six
criteria that must be satisfied are highlighted by Messick
(1995) are that: (a) content is representative and relevant,
(b) content is substantive and is supported by a proper theory, (c) the instrument structure must be the representative of
the given construct, (d) outcomes should have generalizability strength (e) the outcome obtained should have the
ability to be related to other external variables, and (f) the
consequences and implications of the yielded outcome
should have a significant and consequential impact.
Although the questions adopted from Venkatesh et al. (2003,
2012) were validated by previous researchers, new items of
the constructs must also be content-validated. For this
research, to ensure that the instrument is constructed properly, instructors from various institutions such as KFUPM,
DU, DCC, and other research experts having a thorough
knowledge of technology adoption models were consulted to
assess the content validity of the instrument.
6.8 Confidentiality and Ethics
After getting ethical approval from the researcher’s institution,
the survey link was sent to the Director of Information and
Technology Centre (ITC) to distribute the survey questionnaire
to all instructors. The higher management of other universities
was also contacted to send the survey link to their instructors for
filling the survey. In the survey, the instructors were given a
briefing regarding the objectives and use of the research findings. The consent letter of voluntary participation was to assure
that there would not be any physical or psychological harm to
the participants and personal information would be kept confidential by the researcher. Ethical principles regarding Saudi
society and the Islamic religion were also considered.
6.9 Summary
In this study, the Saunders et al. (2016)’s layered onion model
was used to choose the most appropriate research method to
answer the research questions and to meet the objectives of this
study. In this chapter, the philosophy of research, methodology,
and strategy to be used were discussed. Data collection, analysis procedures, and their justification are also discussed in this
chapter. The strategies to improve the validity and to enhance
the response rate were also discussed. This chapter also discussed how to enhance the reliability and validity of the study
so that any possible bias could be minimized.
References
Baker, T. L. (1994). Doing social research (2nd ed). McGraw-Hill Inc.
Bell, J. (1993). Doing your research project: A guide to first-time
researchers in education and social science. Milton Keynes: Open
56
6 How Did We Find Evidence for LMS Adoption?