Applying the Rasch Model to Assess Retention and Transfer Test Instruments in Science Education on Additive and Addictive Substances
Abstract
Purpose of the study: This study aims to evaluate the quality of items in retention and transfer tests related to additive and addictive substances. Using Rasch modeling, the study seeks to enhance the management of learning evaluations and improve our understanding of student abilities and question quality.
Methodology: The research utilizes the Rasch Model to analyze retention and transfer test instruments on science topics involving additives and addictive substances. Conducted with Winstep software, the analysis focuses on the performance of 92 purposively sampled 8th-grade students during their first semester of junior high school. The study examines retention and transfer abilities, comprehensively evaluating the test items.
Main Findings: The Winstep program analysis reveals that, according to the Rasch model, the average ± MNSQ Outfit values for both items and persons are 0.92. The Outfit ZSTD values for items and persons are -0.12 and -0.01, respectively. The instrument's reliability, measured by Cronbach's alpha, is 0.60, indicating moderate reliability. The research findings demonstrate that each item in the instrument is valid and reasonably reliable, with all 20 items deemed suitable for assessing student performance in retention and transfer tests.
Novelty/Originality of this study: This study offers a detailed examination of retention and transfer test instruments' quality using the Rasch Model, providing valuable insights for enhancing the accuracy and reliability of these assessment tools. The research significantly improves educational assessments in science education, particularly in evaluating students' understanding of additives and addictive substances.
References
U. Usmeldi, R. Amini, “The effect of integrated learning model to the students competency on the natural science,” J Phys Conf Ser, 2019, doi: 10.1088/1742-6596/1157/2/022022.
J. Suroso, I, Indrawati, S. Sutarto, I. Mudakir, “Profile of high school students science literacy in east java,” J Phys Conf Ser, 2021, doi: 10.1088/1742-6596/1832/1/012040.
A. K. Dewi, “Improving Students Learning Outcomes Through Mind Map in Human Reproductive System Topic in Natural Science Learning,” Int J Educ Vocat Stud, vol. 1, pp. 702-706, 2019, doi: 10.29103/ijevs.v1i7.1675.
S. Maulana, A. Rusilowati, S. E. Nugroho, E. Susilaningsih, “Implementasi Rasch Model dalam Pengembangan Instrumen Tes Diagnostik,” Pros Semin Nas Pascasarj, vol. 6, pp. 748-756, 2023.
Md. Ghazali, “A Reliability and Validity of an Instrument to Evaluate the School-Based Assessment System: A Pilot Study,” Int J Eval Res Educ, vol. 5, pp. 148, 2016, doi: 10.11591/ijere.v5i2.4533.
A. Maulana, “Analysis of Validity, Reliability and Feasibility of Student Confidence Assessment Instruments,” Schola, vol. 1, pp. 1-12, 2023.
E. Romiyati, A. Ardi Rahman, and E. Budiyono, “Development of Mathematical Student Worksheets Based on Scientific Approaches and PQ4R Learning Strategies on Associated Materials”, Jor. Eva. Edu, vol. 4, no. 1, pp. 17-20, 2023, doi: 10.37251/jee.v4i1.296.
A. Doelvia, V. T. T. Hien, and S. Rathee, “Assessment: The Effectiveness of Video Media Through the Tiktok Application on Teenagers’ Knowledge About Clean and Healthy Living Behavior at Junior High School Level”, Jor. Eva. Edu, vol. 4, no. 4, pp. 168-174, 2023, doi: 10.37251/jee.v4i4.948.
Z. Zainal, “Pengukuran, Assessment dan Evaluasi dalam Pembelajaran Matematika,” J Pendidik Mat, vol. 3, pp. 8-26, 2020.
V. P. Sabandar, and H. B. Santoso, “Evaluasi Aplikasi Media Pembelajaran Statistika Dasar Menggunakan Metode Usability Testing,” Teknika, vol. 7, pp. 50-59, 2018, doi: 10.34148/teknika.v7i1.81.
M. Ibnu, B. Indriyani, H. Inayatullah, and Y. Guntara, “Aplikasi rasch model: Pengembangan instrumen tes untuk mengukur miskonsepsi mahasiswa,” Pros Semin Nas Pendidik FKIP, vol. 2, pp, 205-210, 2019.
S. E. Mokshein, H. Ishak, H. Ahmad, “The use of rasch measurement model in English testing,” Cakrawala Pendidik, vol. 38, pp. 16-32, 2019, doi: 10.21831/cp.v38i1.22750.
N. Ngadi, “Analisis model rasch untuk mengukur kompetensi pengetahuan siswa smkn 1 Kalianget pada mata pelajaran perawatan sistem kelistrikan sepeda motor,” J Pendidik Vokasi Otomotif, vol. 6, pp. 1-19, 2023.
G. Hamdu, F. N. Fuadi, A. Yulianto, Y. S. Akhirani, “Items Quality Analysis Using Rasch Model To Measure Elementary School Students’ Critical Thinking Skill On Stem Learning,” JPI (Jurnal Pendidik Indones), vol. 9, no. 1, 2020, doi:.org/10.23887/jpi-undiksha.v9i1.20884.
T. Rachman, D. B. Napitupulu, “Rasch model for validation a user acceptance instrument for evaluating e-learning system,” CommIT (Communication Inf Technol J), vol. 11, no. 1, 2017, doi: 10.21512/commit.v11i1.2042.
Y. E. Suryani, “Aplikasi rasch model dalam mengevaluasi intelligenz structure test (IST),” Psikohumaniora J Penelit Psikol, vol. 3, no. 1, pp. 73, 2018, doi: 10.21580/pjpp.v3i1.2052.
W. Widhiarso, “Penerapan model rasch untuk mengevaluasi Tes UKKS dan UKPS,” Tenaga Kependidikan, vol. 1, pp. 50-61, 2016.
D. Pratama, I. Husnayaini, “Applying rasch model to measure students` reading comprehension, “JISAE J Indones Student Assess Eval, vol. 6, no. 2, pp. 203-209, 2020, doi: 10.21009/jisae.v6i2.14920.
A. Fauziana, D. Wulansari, “Analisis kualitas butir soal ulangan harian di sekolah dasar dengan model rasch,” Ibriez J Kependidikan Dasar Islam Berbas Sains, vol. 6, no. 1, pp. 10-19, 2021, doi: 10.21154//ibriez.v6i1.112.
H. Retnawati, Teori Respon Butir dan Penerapanya, Yogyakarta Nuha Med, 2014.
R. Tate, “Test dimensionality,” In Large-scale assessment programs for all students (pp. 155-180). Routledge, 2012.
F. Bond, “Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.),” Lawrence Erlbaum Associates Publishers, 2007.
J. M. Linacre, “Understanding rasch measurement: Optimizing rating scale category effectiveness,” J Appl Meas, vol. 3, pp. 85-106, 2002.
W. J. Boone, R. J. Staver, and S. M. Yale, Rasch analysis in the human sciences. London: Springer; 2014.
S. Sudaryono, “Sensitivity of differential item functioning (dif) Detection Method,” J Eval Pendidik, vol. 3, pp. 82-94, 2012.
A. Darmana, A. Sutiani, H. A. Nasution, I. Ismanisa, N. Nurhaswinda, “Analysis of rasch model for the validation of chemistry national exam instruments,” J Pendidik Sains Indones, vol. 9, no. 3, pp. 329-345, 2021, doi: 10.24815/jpsi.v9i3.19618.
L. Tesio, A. Caronni, D. Kumbhare, S. Scarano, “Interpreting results from Rasch analysis 1. The “most likely” measures coming from the model,” Disabil Rehabil, vol. 46, pp. 591-603, 2024 doi: 10.1080/09638288.2023.2169771.
H. Hambleton, Fundamentals of item response theory, Sage Publications, Inc. 1991.
U. D. Purnamasari, B. Kartowagiran, “Application rasch model using R program in analyze the characteristics of chemical items,” J Inov Pendidik IPA, vol. 5, no. 2, pp. 147-157, 2019, doi: 10.21831/jipi.v5i2.24235.
B. Sumintono, and W. Widhiarso, Aplikasi Pemodelan Rasch pada Assessment Pendidikan. Cimahi: Trim Komunikata Publishing House. 2015.
Copyright (c) 2024 Anisa Fitria, Jodion Siburian, Ilham Falani, Damris Muhammad
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and acknowledge that the Integrated Science Education Journal is the first publisher licensed under a Creative Commons Attribution 4.0 International License.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges and earlier and greater citation of published work.
