Using artificial intelligence to create biology multiple choice questions for higher education

Nanda Eska Anugrah Nasution

doi:10.29333/agrenvedu/13071

Full Text (PDF)

Nanda Eska Anugrah Nasution ¹ ^*

More Detail

¹ UIN Kiai Haji Achmad Siddiq Jember, Jawa Timur, INDONESIA^* Corresponding Author

Abstract

This study aims to determine the validity, reliability, level of difficulty, and discrimination power of an artificial intelligence (AI)-generated collection of biology questions for higher education. Students’ responses to AI-generated questions are also presented in this study. A sample of 272 students was selected using a random sampling technique to answer a series of multiple-choice questions and complete a questionnaire. Based on the research findings, 20 of the 21 questions generated by ChatGPT AI are valid. Cronbach’s alpha coefficient was determined to be 0.65 (fairly reliable) for the twenty valid questions. Based on student responses to questions generated by ChatGPT’s AI, it was determined that 79% of students indicated that the AI-generated questions were relevant to the class subject. 72% of students reported that the clarity of AI-generated questions was acceptable. 73% of students reported that the accuracy of AI-generated questions was good.

Keywords

ChatGPT
multiple choice questions
artificial intelligence
validity
reliability

License

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Article Type: Research Article

Agricultural and Environmental Education, Volume 2, Issue 1, June 2023, Article No: em002

https://doi.org/10.29333/agrenvedu/13071

Publication date: 14 Mar 2023

Article Views: 1988

Article Downloads: 2845

Open Access References How to cite this article

References

Arulogun, O. T., Akande, O. N. Akindele, A. T., & Badmus, T. A. (2020). Survey dataset on open and distance learning students’ intention to use social media and emerging technologies for online facilitation. Data in Brief, 31, 105929. https://doi.org/10.1016/j.dib.2020.105929
ChatGPT. (2023). ChatGPT. https://chat.openai.com/chat
Chauhan, P. R., Ratrhod, S. P., Chauhan, B. R., Chauhan, G. R., Adhvaryu, A., & Chauhan, A. P. (2013). Study of difficulty level and discriminating index of stem type multiple choice questions of anatomy in Rajkot. Biomirror, 4(6), 1-4.
Cho, K., Schunn, C. D., & Wilson, R. W. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. Journal of Educational Psychology, 98(4), 891-901. https://doi.org/10.1037/0022-0663.98.4.891
Considine, J., Botti, M., & Thomas, S. (2005). Design, format, validity and reliability of multiple choice questions for use in nursing research and education. Collegian, 12(1), 19-24. https://doi.org/10.1016/S1322-7696(08)60478-3
de Barros Ahrens, R., da Silva Lirani, L., & de Francisco, A. C. (2020). Construct validity and reliability of the work environment assessment instrument WE-10. International Journal of Environmental Research and Public Health, 17(20), 7364. https://doi.org/10.3390/ijerph17207364
Friatma, A., & Anhar, A. (2019). Analysis of validity, reliability, discrimination, difficulty and distraction effectiveness in learning assessment. Journal of Physics: Conference Series, 1387, 012063. https://doi.org/10.1088/1742-6596/1387/1/012063
George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference 11.0 update. Allyn & Bacon.
Ghazali, N. H. M. (2016). A reliability and validity of an instrument to evaluate the school-based assessment system: A pilot study. International Journal of Evaluation and Research in Education, 5(2), 148-157. http://doi.org/10.11591/ijere.v5i2.4533
Haladyna, T. M. (1999). Developing and validating multiple-choice test items. Lawrence Erlbaum.
Harahap, F., Nasution, N. E. A., & Manurung, B. (2019). The effect of blended learning on student’s learning achievement and science process skills in plant tissue culture course. International Journal of Instruction, 12(1), 521-538. https://doi.org/10.29333/iji.2019.12134a
Harahap, M. P., & Nasution, N. E. A. (2022). Validity of computer based learning media to improve junior high school students’ learning outcomes on ecosystem topics. META: Journal of Science and Technological Education, 1(1), 31-45.
Hosseini, M., Rasmussen, L. M., & Resnik, D. B. (2023). Using AI to write scholarly publications. Accountability in Research, 1-9. https://doi.org/10.1080/08989621.2023.2168535
Jackson, S. L. (2003). Research methods and statistics: A critical thinking approach. Thomson Wadsworth.
Kimberlin, C. L., & Winterstein, A. G. (2008). Validity and reliability of measurement instruments used in research. American Journal of Health-System Pharmacy, 65(23), 2276-2284. https://doi.org/10.2146/ajhp070364
McCowan, R. J., & McCowan, S. C. (1999). Item analysis for criterion–Referenced tests. Center for Development of Human Services. https://files.eric.ed.gov/fulltext/ED501716.pdf
Mehta, G., & Mokhasi, V. (2014). Item analysis of multiple choice questions–An assessment of the assessment tool. International Journal of Health Sciences and Research, 4(7), 197-202. https://doi.org/10.1016/j.mjafi.2020.11.007
Mohajan, A. K. (2017). Two criteria for good measurements in research: Validity and reliability. Annals of Spiru Haret University, 17(3), 58-82. https://doi.org/10.26458/1746
Morgan, P. J., Cleave‐Hogg, D., DeSousa, S., & Tarshis, J. (2004). High‐fidelity patient simulation: Validation of performance checklists. BJA: British Journal of Anaesthesia, 92(3), 388-392. https://doi.org/10.1093/bja/aeh081
Muijs, D. (2011). Doing quantitative research in education with SPSS. SAGE. https://doi.org/10.4135/9781849203241
Mutmainah, I., & Isdiati, A. (2022). Validity and reliability test of a written English test online-based using Google Form. INTERACTION: Jurnal Pendidikan Bahasa [INTERACTION: Journal of Language Education], 9(1), 89-100.
Rao, C., Kishan, P. H. L., Sajitha, K., Permi, H., & Shetty, J. (2016). Item analysis of multiple choice questions: Assessing an assessment tool in medical students. International Journal of Education and Psychological Research, 2, 201-214. https://doi.org/10.4103/2395-2296.189670
Rii, K. B., Choi, L. K., Shino, Y., Kenta, H., & Adianita, I. R. (2020). Application of iLearning Education in Learning Methods for Entrepreneurship and Elementary School Student Innovation. Aptisi Transactions on Technopreneurship, 2(2), 131-142. https://doi.org/10.34306/att.v2i2.90
Salwa, A. (2012). The validity, reliability, level of difficulty and appropriateness of curriculum of the English test [PhD thesis, Diponegoro University].
Setiawaty, R., Sulistyorini, T. B., Margono, & Rahmawat, L. E. (2017). Validity test and reliability of Indonesian language multiple choice in final term examination. In Proceedings of the 1^st International Seminar on Language, Literature and Education (pp. 43-50). https://doi.org/10.18502/kss.v3i9.2609
Suruchi, S., & Rana, S. S. (2014). Test item analysis and relationship between difficulty level and discrimination index of test items in an achievement test in biology. Paripex-Indian Journal of Research, 3(6), 56-58. https://doi.org/10.15373/22501991/JUNE2014/18
Susnjak, T. (2022). ChatGPT: The end of online exam integrity? arXiv. https://doi.org/10.48550/arXiv.2212.09292
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48, 1273-1296. https://doi.org/10.1007/s11165-016-9602-2
Thorndike, R. M., Cunningham, G. K., Thorndike, R. L., & Hagen, E. P. (1991). Measurement and evaluation in psychology and education. MacMillan.
van Griethuijsen, R. A. L. F., van Eijck, M. W., Haste, H., den Brok, P. J., Skinner, N. C., Mansour, N.,Gencer, A. S., & BouJaoude, S. (2014). Global patterns in students’ views of science and interest in science. Research in Science Education, 45(4), 581-603. http://doi.org/10.1007/s11165-014-9438-6
Wongpakaran, T., & Wongpakaran, N. (2012). A comparison of reliability and construct validity between the original and revised versions of the Rosenberg self-esteem scale. Psychiatry Investigation, 9(1), 54-58. https://doi.org/10.4306/pi.2012.9.1.54

How to cite this article

APA

Nasution, N. E. A. (2023). Using artificial intelligence to create biology multiple choice questions for higher education. Agricultural and Environmental Education, 2(1), em002. https://doi.org/10.29333/agrenvedu/13071

Vancouver

Nasution NEA. Using artificial intelligence to create biology multiple choice questions for higher education. Agricultural and Environmental Education. 2023;2(1):em002. https://doi.org/10.29333/agrenvedu/13071

AMA

Nasution NEA. Using artificial intelligence to create biology multiple choice questions for higher education. Agricultural and Environmental Education. 2023;2(1), em002. https://doi.org/10.29333/agrenvedu/13071

Chicago

Nasution, Nanda Eska Anugrah. "Using artificial intelligence to create biology multiple choice questions for higher education". Agricultural and Environmental Education 2023 2 no. 1 (2023): em002. https://doi.org/10.29333/agrenvedu/13071

Harvard

Nasution, N. E. A. (2023). Using artificial intelligence to create biology multiple choice questions for higher education. Agricultural and Environmental Education, 2(1), em002. https://doi.org/10.29333/agrenvedu/13071

MLA

Nasution, Nanda Eska Anugrah "Using artificial intelligence to create biology multiple choice questions for higher education". Agricultural and Environmental Education, vol. 2, no. 1, 2023, em002. https://doi.org/10.29333/agrenvedu/13071