Content validity and reliability of questionnaires: trends, prospects and innovation in the digital research epoch

Abstract

This paper exploits a systematic literature review (SLR) technique to capture the trends in content validity and reliability of questionnaires published from 2018-2024 and indexed mainly on Scopus. It was confirmed that Content validity is one of the widely used and known measure to ensure the accuracy of questionnaires before it is administered to the target participants of study. This is commonly done by subjecting the questionnaire itself to face validation with at least 5 or more experts in the field of research. English language experts and statisticians are the usual validators of questionnaires looking into the accuracy of statements, scales, and format. Reliability aspect of questionnaire, nonetheless, is of equal value as that of the validity aspect. Questionnaires will not be able to capture what it intends to obtain if they are devoid of the reliability value as well as Content validity index (CVI).  The SLR allows the author to arrive at a conclusion that validity and reliability of questionnaires are vital aspects of research pursuit in the digital era as substantiated by the researchers cited. It is a serious and mandatory step in conducting philosophical inquiries to address prevailing phenomena in education, social and health sciences. A valid questionnaire is one which passed the CVI categorized into four (7 – not valid; 8 – fairly valid; 9 – moderately valid; 10 – highly valid). Conversely, a reliable questionnaire must at least reach the moderate reliability index more delineated as 0.00 - 0.69 poor reliability; 0.70 - 0.79 fair reliability; 0.80 – 0.89 good reliability; and, 0.90 – 0.99 excellent reliability.

Reference

Abbas, M., Jam, F.A., & Khan, T.I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, 21(10). https://doi.org/10.1186/s41239-024-00444-7

Acharya, B., Kharel, K. R., Upadhyaya, Y. M., & Kharel, S. (2024). A critical appraisal of education policy practices ensuring free and compulsory school education: Insights from Nepal. International Journal of Education and Practice, 12(2), 146–162. https://doi.org/10.18488/61.v12i2.3629

Abdolmaleki, N., & Saeedi, Z. (2024). Brain-based CALL in flipped higher education GE courses held through LMS: Boosting vocabulary learning and reading comprehension. International Journal of Educational Technology in Higher Education, 21(11). https://doi.org/10.1186/s41239-024-00442-9

Aiken, L. R. (1980). Content Validity and Reliability of Single Items or Questionnaires. Educational and Psychological Measurement, 40(4), 955-959. https://doi.org/10.1177/001316448004000419

Aldawsary, W., & Alzboon, A. (2024). The Role of Secondary School in Confronting Cyberterrorism in Saudi Arabia. Pegem Journal of Education and Instruction, 14(2), 27–36. https://doi.org/10.47750/pegegog.14.02.04

Astawa, N. L. P. N. S. P., Santosa, M. H., Artini, L. P., & Nitiasih, P. K. (2024). Sustainable Development Goals in EFL Students’ Learning: A Systematic Review. Pegem Journal of Education and Instruction, 14(2), 1–11. https://doi.org/10.47750/pegegog.14.02.01

Awan, W. A., Ameen, K., & Soroya, S. (2022). Content validity and reliability of an instrument to investigate serendipitous retrieval of information: Information Encountering and its Subsequent Keeping and Sharing Behaviour Scale (IEKSBS). Journal of Information Science, 0(0). https://doi.org/10.1177/01655515221128756

Azraii, A.B., Ramli, A.S., Ismail, Z. (2021). Validity and reliability of an adapted questionnaire measuring knowledge, awareness and practice regarding familial hypercholesterolaemia among primary care physicians in Malaysia. BMC Cardiovascular Disorders, 21, 39 (2021). https://doi.org/10.1186/s12872-020-01845-y

Basantes-Andrade, A., López-Gutiérrez, J. C., Grijalva, M., & Ricardo Y. (2023). Validity and reliability of the questionnaire of academic knowledge of teachers of basic general education. F1000Research, 12, 642. https://doi.org/10.12688/f1000research.134261.1

Bolarinwa, O. A. (2015). Principles and Methods of Validity and Reliability Testing of Questionnaires Used in Social and Health Science Researches. Nigerian Postgraduate Medical Journal 22(4),195-201. https://doi.org/10.4103/1117-1936.173959

Çevik, A., & Uygun , T. (2024). Examining the effects of creative drama onteacher efficacy belief, critical thinking disposition, emphatic tendency and reflective thinking tendency: . Pegem Journal of Education and Instruction, 14(2), 67–77. https://doi.org/10.47750/pegegog.14.02.08

Cobos Alvarado, F., Peñaherrera León, M., & Ortiz Colon, A. (2016). Validation of a questionnaire on research-based learning with engineering students. Journal of Technology and Science Education, 6(3), 219-233. https://doi.org/10.3926/jotse.227

Da Cunha de Sá-Caputo, D., Sonza, A., Bachur, J. A., & Bernardo-Filho, M. (2020). Development, validation and reliabilty of a questionnaire to evaluate the changes on the level of physical exercises and in daily life habits due to COVID-19 pandemic social distancing. Acta bio-medica: Atenei Parmensis, 91(3), e2020004. https://doi.org/10.23750/abm.v91i3.9888

Dönmez, E., Dolu, İ., & Yılmaz, S. (2022). Validity and Reliability of the Questionnaire to Measure the Patient Satisfaction With Nursing Care Quality-Turkish Version: The patient satisfaction with nursing care quality. Asia Pacific Journal of Health Management, 17(2). https://doi.org/10.24083/apjhm.v17i2.1519

Ilangovan, P., Rajamohan, P., Sudhashini. N., Thangiah, M., Sesaiah, S., & Perumal, G. (2023). Impact of Logistics AI System on Manufacturing Companies in Malaysia. Hong Kong Journal of Social Sciences, 62 Autumn/Winter. https://doi.org/10.55463/hkjss.issn.1021-3619.62.2

Jalalian, F., Hosseininejad, S. M., Montazer, S. H., & Hesamzadeh, A. (2023). Emergency Department Nurses’ Clinical Competence and Its Related Factors: A cross-sectional study. Asia Pacific Journal of Health Management, 18(3). https://doi.org/10.24083/apjhm.v18i3.2195

König, C.M., Karrenbauer, C., & Breitner, M.H. (2024). Development guidelines for individual digital study assistants in higher education. International Journal of Educational Technology in Higher Education,  21, 9. https://doi.org/10.1186/s41239-024-00439-4

Liu, H., Váchová, L., & Plevová, I. (2024). Effect of mobile phone dependence on various aspects of academic achievement: evidence from Chinese and Czech university students. Pegem Journal of Education and Instruction, 14(2), 20–26. https://doi.org/10.47750/pegegog.14.02.03

Mohammadi, M., Larijani, B., Tabatabaei, S. M., Nedjat, S., Yunesian, M., & Nayeri, F. S. (2019). A study of the validity and reliability of the questionnaire entitled "physicians' approach to and disclosure of medical errors and the related ethical issues". Journal of medical ethics and history of medicine, 12, 2. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6642466/

Panda, B. & Leepsa, N. (2018). Institutional Ownership Activism, Market Performance, and Financial Crisis: Evidence from an Emerging Market. Asia-Pacific Social Science Review, 17(3). https://ejournals.ph/article.php?id=11989

Pattier, D. (2024). Communication and Emerging Teaching Models: A Study on Youtubers Teachers. Multidisciplinary Journal of Educational Research, 14(1), 1–16. https://doi.org/10.17583/remie.9887

Pīgozne, T., Medveckis, A., & Pelnena, I. (2024). Students’ Digital Learning Resources for Transversal Skills Improvement and Virtues Inculcation. Pegem Journal of Education and Instruction, 14(2), 12–19. https://doi.org/10.47750/pegegog.14.02.02

Rafhi, E., Stupans, I., Park, J. S., & Wang, K. (2023). The Influence of Social and Individual Factors on Medicine Use in Older Adults. Asia Pacific Journal of Health Management, 18(3). https://doi.org/10.24083/apjhm.v18i3.2691

Sirivunnabood, P. (2018). Understanding Political Party Finance in Thailand: New Regulations but Old Practices. Asia-Pacific Social Science Review, 17(3). https://ejournals.ph/article.php?id=11985

Spathopoulou, F., & Pitychoutis, K. M. (2024). Teachers’ attitudes on gamification: The Greek EFL context . International Journal of Education and Practice, 12(2), 163–176. https://doi.org/10.18488/61.v12i2.3630

Thabo, A. A. (2023). The Political System in the Republic of South Sudan: Understanding Conceptual Theory of Sudan People Liberation Movement Governance since 2005. IJRDO - Journal of Social Science and Humanities Research, 9(9), 1-12. https://doi.org/10.53555/sshr.v9i9.5943

Torkian, S., Shahesmaeili, A., Malekmohammadi, N., & Khosravi, V. (2020). Content Validity and Test-Retest Reliability of a Questionnaire to Measure Virtual Social Network Addiction Among Students. International Journal of High Risk Behaviors and Addiction, 9(1):e92353. https://doi.org/10.5812/ijhrba.92353.

Yazdi-Feyzabadi, V., Nakhaee, N., Mehrolhassani, M.H. (2021). Development and validation of a questionnaire to determine medical orders non-adherence: a sequential exploratory mixed-method study. BMC Health Services Research, 21, 136. https://doi.org/10.1186/s12913-021-06147-3

Zhang, Y., & Aryadoust, V. (2022). A Systematic Review of the Validity of Questionnaires in Second Language Research. Education Sciences, 12, 723. https://doi.org/10.3390/educsci12100723