Evaluating Risks in an Outcome-Based Education Assessment Information System: A Qualitative Case Study in Higher Education
DOI:
https://doi.org/10.37251/ijoer.v7i3.2705Keywords:
Higher Education, Outcome-Based Education, PMBOK, Project Risk Management, Risk AnalysisAbstract
Purpose of the study: This study aims to identify, categorize, and prioritize risks in an Outcome-Based Education (OBE)-based assessment information system related to learning outcome evaluation, Program Learning Outcomes–Course Learning Outcomes assessment processes, and institutional quality assurance implementation in higher education.
Methodology: This study used a qualitative case study approach with the Project Management Body of Knowledge® Guide Sixth Edition Project Risk Management framework, Risk Breakdown Structure (RBS), and qualitative probability–impact matrix. Data collection methods included document analysis, direct observation, semi-structured interviews, prototype system review, and Program Learning Outcomes–Course Learning Outcomes mapping validation.
Main Findings: Eighteen risks were identified across technical, data-related, operational, human resource, curriculum-related, and infrastructure categories. High-priority risks involved inconsistencies in Program Learning Outcomes–Course Learning Outcomes mapping and assessment weighting structures. Extreme risks included system integration limitations, developer dependency, and infrastructure readiness issues affecting learning outcome evaluation and assessment consistency.
Novelty/Originality of this study: This study presents a structured qualitative risk analysis framework for Outcome-Based Education (OBE)-based assessment information systems by integrating educational evaluation perspectives with project risk management approaches. The study highlights how organizational, operational, and data-related risks influence learning outcome assessment validity and institutional quality assurance processes.
References
Pemerintah Republik Indonesia, Peraturan Presiden Republik Indonesia Nomor 8 Tahun 2012 tentang Kerangka Kualifikasi Nasional Indonesia. Indonesia, 2012.
Kementerian Pendidikan dan Kebudayaan, Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia Nomor 3 Tahun 2020 tentang Standar Nasional Pendidikan Tinggi, no. 47. 2020.
Pemerintah Republik Indonesia, Peraturan Menteri Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia Nomor 53 Tahun 2023 tentang Penjaminan Mutu Pendidikan Tinggi. 2023, pp. 1–45.
R. M. Harden, “Outcome-Based Education: the future is today,” Med. Teach., vol. 29, no. 7, pp. 625–629, 2007, doi: 10.1080/01421590701729930.
R. Mufanti, D. Carter, and N. England, “Social Sciences & Humanities Open Outcomes-based education in Indonesian higher education : Reporting on the understanding , challenges , and support available to teachers,” Soc. Sci. Humanit. Open, vol. 9, no. February, p. 100873, 2024, doi: 10.1016/j.ssaho.2024.100873.
V. Keo, C. Hak, B. Lan, R. Khong, S. Neang, and S. Hoeurn, “Implementing Outcome-Based Education in Higher Education : Challenges and Benefits,” Indones. J. Educ. Psychol. Sci., vol. 3, no. 5, pp. 511–530, 2025, doi: 10.59890/ijeps.v3i5.138.
M. Akbar and D. Irianto, “Usulan Sistem Penilaian Capaian Pembelajaran Mata Kuliah (CPMK) dalam Menunjang Penilaian Capaian Pembelajaran Lulusan (CPL) Program Studi,” in Prosiding Seminar Nasional Teknik Industri (SENASTI) 2023, SENASTI, 2023, pp. 974–984.
B. Kushari and L. Septiadi, “A Learning Outcome Assessment Information System to Facilitate Outcome-Based Education (OBE) Implementation,” J. Pendidik. Teknol. dan Kejuru., vol. 28, no. 2, pp. 238–250, 2022, doi: 10.21831/jptk.v28i1.42339.
J. Tai, R. Ajjawi, D. Boud, P. Dawson, and E. Panadero, “Developing evaluative judgement : enabling students to make decisions about the quality of work,” pp. 467–481, 2018.
D. Ifenthaler, J. Yin, and K. Yau, “Utilising learning analytics to support study success in higher education : a systematic review,” Educ. Technol. Res. Dev., vol. 68, no. 4, pp. 1961–1990, 2020, doi: 10.1007/s11423-020-09788-z.
O. Viberg, M. Hatakka, O. Bälter, and A. Mavroudi, “Computers in Human Behavior The current landscape of learning analytics in higher education,” vol. 89, no. July, pp. 98–110, 2018.
K. Schildkamp and K. Schildkamp, “improvement : Research insights and gaps Research insights and gaps,” Educ. Res., vol. 61, no. 3, pp. 257–273, 2019, doi: 10.1080/00131881.2019.1625716.
Y. S. Mian, F. Khalid, A. Wong, C. Qun, and S. S. Ismail, “Learning Analytics in Education , Advantages and Issues : A Systematic Literature Review,” pp. 2913–2920, 2022, doi: 10.4236/ce.2022.139183.
P. Sotiriadou et al., “Studies in Higher Education The role of authentic assessment to preserve academic integrity and promote skill development and employability,” vol. 5079, 2020, doi: 10.1080/03075079.2019.1582015.
M. T. Kane, “Validating the interpretations and uses of test scores.,” J. Educ. Meas., vol. 50, no. 1, pp. 1–73, 2013, doi: 10.1111/jedm.12000.
U. De Cádiz, G. Rodríguez-gómez, L. Sánchez-calleja, A. González-elorza, and M. S. I, “The challenge to design and assess learning outcomes in higher education El reto del diseño de los resultados de aprendizaje y,” pp. 1–12, 2025.
B. Kholifah, I. Thoib, N. Sururi, and D. S. Nugraha, “Efektivitas Sistem Informasi Akademik dalam Meningkatkan Komunikasi dan Informasi Akademik di Institut Teknologi Mojosari,” J. Komun. Islam, vol. 5, no. 1, pp. 89–100, 2024.
S. D. Pohan, I. Ali, S. A. Widiana, and Y. W. Mandaya, “Evaluation of Success Factors for Academic Information System Applications in Higher Education Using the Hot-Fit Model,” J. Adv. Inf. Syst. Technol., vol. 7, no. 1, pp. 15–30, 2025.
A. Alyasin, R. Nasser, M. El Hajj, and H. Harb, “Assessing Learning Outcomes in Higher Education : From Practice to Systematization,” TEM J., vol. 12, no. 3, pp. 1593–1604, 2023, doi: 10.18421/TEM123.
O. Zlatkin-troitschanskaia, M. Toepper, H. Anand, C. Lautenbach, and C. Kuhn, Assessment of Learning Outcomes in Higher Education. Springer, 2018. doi: 10.1007/978-3-319-74338-7.
T. Y. Khaw and A. P. Teoh, “Risk management in higher education research : a systematic literature review,” Qual. Assur. Educ., vol. 31, no. 2, pp. 296–312, 2022, doi: 10.1108/QAE-04-2022-0097.
M. Jesry et al., “Exploring the value of a risk-management quality-assurance model to support delivery of quality higher education in the conflict-affected northwest of Syria,” Int. J. Educ. Res. Open, vol. 3, no. February, p. 100134, 2022, doi: 10.1016/j.ijedro.2022.100134.
M. Rosiawan and J. D. Trisnawati, “Implementation of Risk Management for Outcome-Based Learning,” J. Eduscience, vol. 10, no. 2, pp. 497–508, 2023.
A. Taman, L. D. Prasojo, L. Yuliana, and N. A. Purwanto, “Identifying Risks in Educational Institutions Management : A Case in a Legal Entity Higher Education Institution,” J. Prima Edukasia, vol. 12, no. 1, pp. 120–128, 2024, doi: 10.21831/jpe.v12i1.68277.
Project Management Institute, A Guide to the Project Management Body of Knowledge (PMBOK Guide), Sixth Edit. Newtown Square, PA: Project Management Institute, 2017.
H. Syahputra, W. Abdillah, S. Widodo, and S. Anwar, “Manajemen Risiko Proyek Berdasarkan Panduan Body of Knowledge Manajemen Proyek 2017,” J. Bisnis dan Manaj., vol. 2, no. 1, pp. 1–13, 2022.
R. Pramana and R. D. P. Aritonang, “Use of risk management register using qualitative risk analysis method for land drilling works in rokan work area,” Int. J. Environ. Sustain. Soc. Sci., vol. 6, no. 1, pp. 50–88, 2025.
O. Bazaluk et al., “Improving the risk management process in quality management systems of higher education,” Sci. Rep., vol. 14, no. 3977, pp. 1–19, 2024, doi: 10.1038/s41598-024-53455-9.
Munjin and D. Kurniadin, “The Implementation of Risk Management (A Case Study at State Islamic University of Walisongo Semarang , Indonesia),” Int. J. Multidiscip. Res. Anal., vol. 08, no. 11, pp. 6323–6336, 2025, doi: 10.47191/ijmra/v8-i11-50.
Handriadi, H. N. O. Kustiyanto, and S. Chai, “Evaluation of Risk Management Implementation in Educational Institutions,” Int. J. Educ. Narrat., vol. 3, no. 1, pp. 50–58, 2025.
Mónica Hernández-Campos, A. Gonzalez-Torres, and F. J. García-Peñalvo, “Learning Outcomes Evaluation Through Learning Analytics Systems in Higher Education : A Systematic Literature Review,” SAGE Open, vol. 15, no. September, pp. 1–17, 2025, doi: 10.1177/21582440251347374.
F. Jose, “Learning Outcomes Evaluation Through Learning Analytics Systems in Higher Education” no. September, pp. 1–17, 2025
R. K. Yin, Case Study Research and Applications: Design and Methods, 6th ed. SAGE Publications, 2018.
J. W. Creswell, Research Design: Qualitative, Quantitative, and Mixed Method Approaches, 4th ed. SAGE Publications.
B. Barghi and S. S. Sikari, “Qualitative and quantitative project risk assessment using a hybrid PMBOK model developed under uncertainty conditions,” Heliyon, vol. 6, no. 1, 2020, doi: 10.1016/j.heliyon.2019.e03097.
V. Braun, V. Clarke, V. Braun, and V. Clarke, “Using thematic analysis in psychology Using thematic analysis in psychology,” Qual. Res. Psychol., vol. 0887, no. 2006, 2008, doi: 10.1191/1478088706qp063oa.
V. Braun, V. Clarke, V. Braun, and V. Clarke, “To saturate or not to saturate ? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales useful concept for thematic analysis and sample-size rationales,” Qual. Res. Sport. Exerc. Heal., vol. 13, no. 2, pp. 201–216, 2021, doi: 10.1080/2159676X.2019.1704846.
J. C. R. Moreira, “Risk Management in Education: A Systematic Literature Review of the Last Five Years,” RGSA – Rev. Gestão Soc. e Ambient., vol. 19, no. 1, pp. 1–15, 2025, doi: 10.24857/rgsa.v19n1-127.
K. Oliveira, M. Méxas, M. Meiriño, and G. Drumond, “Critical success factors associated with the implementation of enterprise risk management,” J. Risk Res., vol. 9877, pp. 1–16, 2019, doi: 10.1080/13669877.2018.1437061.
T. Fawns, D. Boud, and P. Dawson, “Assessment & Evaluation in Higher Education Identifying what our students have learned : a framework for practical assessment validation,” Assess. Eval. High. Educ., vol. 0, no. 0, pp. 1–17, 2026, doi: 10.1080/02602938.2026.2620053.
M. Schurr, M. De Tuya, and K. Noll, “Risk – Informed Decision Making in Information System Implementation Projects : Using Qualitative Assessment and Evaluation of Stakeholders’ Perceptions of Risk,” in Proceedings of the 50th Hawaii International Conference on System Sciences, 2017, pp. 6120–6129.
M. P. Narasimhan, “A Conceptual Overview of Risk Management in Education Sector Projects,” Holist. Res. Perspect., vol. 5, pp. 138–145, 2020, doi: 10.47059/CIIR/BP20002/13.
S. Pastore, “Teacher assessment literacy : a systematic review,” 2023.
M. M. De Carvalho and R. R. Junior, “Impact of risk management on project performance : the importance of soft skills,” Int. J. Prod. Res., vol. 7543, pp. 1–20, 2015, doi: 10.1080/00207543.2014.919423.
Y. Xu and G. T. L. Brown, “Teacher assessment literacy in practice : A reconceptualization,” Teach. Teach. Educ., vol. 58, p. 2016, 2016, doi: 10.1016/j.tate.2016.05.010.
B. Stensaker, G. T. Bilbow, L. Breslow, and R. Van Der Vaart, Strengthening Teaching and Learning in Research Universities. Springer Nature. doi: 10.1007/978-3-319-56499-9.
R. I. Asiyai, “Best practices for quality assurance in higher education : implications for educational administration,” Int. J. Leadersh. Educ., vol. 25, no. 5, pp. 843–854, 2022, doi: 10.1080/13603124.2019.1710569.
T. A. Carbone and D. D. Tippett, “Project Risk Management Using the Project Risk FMEA,” Eng. Manag. J., vol. 16, no. 4, pp. 28–35, 2004, doi: 10.1080/10429247.2004.11415263.
A. Nurrifqhi and J. Sembiring, “Risk Assessment Model Development in GX PMBOK Using Checklist Scenario Analysis and Simple Additive Weighting,” 2023 Int. Semin. Intell. Technol. Its Appl., pp. 660–665, 2023, doi: 10.1109/ISITIA59021.2023.10221069.
K. A. O. Al-husseini and A. H. Obaid, “Interaction between project tasks and risk management tasks in software development,” Period. Eng. Nat. Sci., vol. 8, no. 4, pp. 2300–2308, 2020.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Olivia Wardhani, Pringgo Widyo Laksono
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and acknowledge that the Indonesian Journal of Education Research (IJoER) is the first publisher licensed under a Creative Commons Attribution 4.0 International License.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges and earlier and greater citation of published work.
.png)
.png)
