SINTA

2.5

Impact

Google Scholar

10

h-index

Indonesian Journal of Education Research (IJoER)

Open Access Journal

Evaluating Risks in an Outcome-Based Education Assessment Information System: A Qualitative Case Study in Higher Education

Olivia Wardhani ORCID
Universitas Tidar ROR
Indonesia
olivia@untidar.ac.id
Pringgo Widyo Laksono
Sebelas Maret University ROR
Indonesia
Share

  • Purpose of the study: This study aims to identify, categorize, and prioritize risks in an Outcome-Based Education (OBE)-based assessment information system related to learning outcome evaluation, Program Learning Outcomes–Course Learning Outcomes assessment processes, and institutional quality assurance implementation in higher education.

    Methodology: This study used a qualitative case study approach with the Project Management Body of Knowledge® Guide Sixth Edition Project Risk Management framework, Risk Breakdown Structure (RBS), and qualitative probability–impact matrix. Data collection methods included document analysis, direct observation, semi-structured interviews, prototype system review, and Program Learning Outcomes–Course Learning Outcomes mapping validation.

    Main Findings: Eighteen risks were identified across technical, data-related, operational, human resource, curriculum-related, and infrastructure categories. High-priority risks involved inconsistencies in Program Learning Outcomes–Course Learning Outcomes mapping and assessment weighting structures. Extreme risks included system integration limitations, developer dependency, and infrastructure readiness issues affecting learning outcome evaluation and assessment consistency.

    Novelty/Originality of this study: This study presents a structured qualitative risk analysis framework for Outcome-Based Education (OBE)-based assessment information systems by integrating educational evaluation perspectives with project risk management approaches. The study highlights how organizational, operational, and data-related risks influence learning outcome assessment validity and institutional quality assurance processes.

  • How to cite

    [1]
    O. Wardhani and P. W. Laksono, “Evaluating Risks in an Outcome-Based Education Assessment Information System: A Qualitative Case Study in Higher Education”, Ind. Jou. Edu. Rsc, vol. 7, no. 3, pp. 270–283, Jun. 2026, doi: 10.37251/ijoer.v7i3.2705.

    More Citation Formats

  • 41
    Abstract views
    18
    Downloads

    Metrics — Badges

    1. Pemerintah Republik Indonesia, Peraturan Presiden Republik Indonesia Nomor 8 Tahun 2012 tentang Kerangka Kualifikasi Nasional Indonesia. Indonesia, 2012.
    2. Kementerian Pendidikan dan Kebudayaan, Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia Nomor 3 Tahun 2020 tentang Standar Nasional Pendidikan Tinggi, no. 47. 2020.
    3. Pemerintah Republik Indonesia, Peraturan Menteri Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia Nomor 53 Tahun 2023 tentang Penjaminan Mutu Pendidikan Tinggi. 2023, pp. 1–45.
    4. R. M. Harden, “Outcome-Based Education: the future is today,” Med. Teach., vol. 29, no. 7, pp. 625–629, 2007, doi: 10.1080/01421590701729930. DOI: https://doi.org/10.1080/01421590701729930
    5. R. Mufanti, D. Carter, and N. England, “Social Sciences & Humanities Open Outcomes-based education in Indonesian higher education : Reporting on the understanding , challenges , and support available to teachers,” Soc. Sci. Humanit. Open, vol. 9, no. February, p. 100873, 2024, doi: 10.1016/j.ssaho.2024.100873. DOI: https://doi.org/10.1016/j.ssaho.2024.100873
    6. V. Keo, C. Hak, B. Lan, R. Khong, S. Neang, and S. Hoeurn, “Implementing Outcome-Based Education in Higher Education : Challenges and Benefits,” Indones. J. Educ. Psychol. Sci., vol. 3, no. 5, pp. 511–530, 2025, doi: 10.59890/ijeps.v3i5.138. DOI: https://doi.org/10.59890/ijeps.v3i5.138
    7. M. Akbar and D. Irianto, “Usulan Sistem Penilaian Capaian Pembelajaran Mata Kuliah (CPMK) dalam Menunjang Penilaian Capaian Pembelajaran Lulusan (CPL) Program Studi,” in Prosiding Seminar Nasional Teknik Industri (SENASTI) 2023, SENASTI, 2023, pp. 974–984.
    8. B. Kushari and L. Septiadi, “A Learning Outcome Assessment Information System to Facilitate Outcome-Based Education (OBE) Implementation,” J. Pendidik. Teknol. dan Kejuru., vol. 28, no. 2, pp. 238–250, 2022, doi: 10.21831/jptk.v28i1.42339. DOI: https://doi.org/10.21831/jptk.v28i2.42339
    9. J. Tai, R. Ajjawi, D. Boud, P. Dawson, and E. Panadero, “Developing evaluative judgement : enabling students to make decisions about the quality of work,” pp. 467–481, 2018. DOI: https://doi.org/10.1007/s10734-017-0220-3
    10. D. Ifenthaler, J. Yin, and K. Yau, “Utilising learning analytics to support study success in higher education : a systematic review,” Educ. Technol. Res. Dev., vol. 68, no. 4, pp. 1961–1990, 2020, doi: 10.1007/s11423-020-09788-z. DOI: https://doi.org/10.1007/s11423-020-09788-z
    11. O. Viberg, M. Hatakka, O. Bälter, and A. Mavroudi, “Computers in Human Behavior The current landscape of learning analytics in higher education,” vol. 89, no. July, pp. 98–110, 2018. DOI: https://doi.org/10.1016/j.chb.2018.07.027
    12. K. Schildkamp and K. Schildkamp, “improvement : Research insights and gaps Research insights and gaps,” Educ. Res., vol. 61, no. 3, pp. 257–273, 2019, doi: 10.1080/00131881.2019.1625716. DOI: https://doi.org/10.1080/00131881.2019.1625716
    13. Y. S. Mian, F. Khalid, A. Wong, C. Qun, and S. S. Ismail, “Learning Analytics in Education , Advantages and Issues : A Systematic Literature Review,” pp. 2913–2920, 2022, doi: 10.4236/ce.2022.139183. DOI: https://doi.org/10.4236/ce.2022.139183
    14. P. Sotiriadou et al., “Studies in Higher Education The role of authentic assessment to preserve academic integrity and promote skill development and employability,” vol. 5079, 2020, doi: 10.1080/03075079.2019.1582015. DOI: https://doi.org/10.1080/03075079.2019.1582015
    15. M. T. Kane, “Validating the interpretations and uses of test scores.,” J. Educ. Meas., vol. 50, no. 1, pp. 1–73, 2013, doi: 10.1111/jedm.12000. DOI: https://doi.org/10.1111/jedm.12000
    16. U. De Cádiz, G. Rodríguez-gómez, L. Sánchez-calleja, A. González-elorza, and M. S. I, “The challenge to design and assess learning outcomes in higher education El reto del diseño de los resultados de aprendizaje y,” pp. 1–12, 2025.
    17. B. Kholifah, I. Thoib, N. Sururi, and D. S. Nugraha, “Efektivitas Sistem Informasi Akademik dalam Meningkatkan Komunikasi dan Informasi Akademik di Institut Teknologi Mojosari,” J. Komun. Islam, vol. 5, no. 1, pp. 89–100, 2024. DOI: https://doi.org/10.53429/j-kis.v5i1.953
    18. S. D. Pohan, I. Ali, S. A. Widiana, and Y. W. Mandaya, “Evaluation of Success Factors for Academic Information System Applications in Higher Education Using the Hot-Fit Model,” J. Adv. Inf. Syst. Technol., vol. 7, no. 1, pp. 15–30, 2025. DOI: https://doi.org/10.15294/jaist.v7i1.20847
    19. A. Alyasin, R. Nasser, M. El Hajj, and H. Harb, “Assessing Learning Outcomes in Higher Education : From Practice to Systematization,” TEM J., vol. 12, no. 3, pp. 1593–1604, 2023, doi: 10.18421/TEM123. DOI: https://doi.org/10.18421/TEM123-41
    20. O. Zlatkin-troitschanskaia, M. Toepper, H. Anand, C. Lautenbach, and C. Kuhn, Assessment of Learning Outcomes in Higher Education. Springer, 2018. doi: 10.1007/978-3-319-74338-7. DOI: https://doi.org/10.1007/978-3-319-74338-7
    21. T. Y. Khaw and A. P. Teoh, “Risk management in higher education research : a systematic literature review,” Qual. Assur. Educ., vol. 31, no. 2, pp. 296–312, 2022, doi: 10.1108/QAE-04-2022-0097. DOI: https://doi.org/10.1108/QAE-04-2022-0097
    22. M. Jesry et al., “Exploring the value of a risk-management quality-assurance model to support delivery of quality higher education in the conflict-affected northwest of Syria,” Int. J. Educ. Res. Open, vol. 3, no. February, p. 100134, 2022, doi: 10.1016/j.ijedro.2022.100134. DOI: https://doi.org/10.1016/j.ijedro.2022.100134
    23. M. Rosiawan and J. D. Trisnawati, “Implementation of Risk Management for Outcome-Based Learning,” J. Eduscience, vol. 10, no. 2, pp. 497–508, 2023. DOI: https://doi.org/10.36987/jes.v10i2.4589
    24. A. Taman, L. D. Prasojo, L. Yuliana, and N. A. Purwanto, “Identifying Risks in Educational Institutions Management : A Case in a Legal Entity Higher Education Institution,” J. Prima Edukasia, vol. 12, no. 1, pp. 120–128, 2024, doi: 10.21831/jpe.v12i1.68277. DOI: https://doi.org/10.21831/jpe.v12i1.68277
    25. Project Management Institute, A Guide to the Project Management Body of Knowledge (PMBOK Guide), Sixth Edit. Newtown Square, PA: Project Management Institute, 2017.
    26. H. Syahputra, W. Abdillah, S. Widodo, and S. Anwar, “Manajemen Risiko Proyek Berdasarkan Panduan Body of Knowledge Manajemen Proyek 2017,” J. Bisnis dan Manaj., vol. 2, no. 1, pp. 1–13, 2022. DOI: https://doi.org/10.32509/jmb.v2i1.1987
    27. R. Pramana and R. D. P. Aritonang, “Use of risk management register using qualitative risk analysis method for land drilling works in rokan work area,” Int. J. Environ. Sustain. Soc. Sci., vol. 6, no. 1, pp. 50–88, 2025. DOI: https://doi.org/10.38142/ijesss.v6i1.1281
    28. O. Bazaluk et al., “Improving the risk management process in quality management systems of higher education,” Sci. Rep., vol. 14, no. 3977, pp. 1–19, 2024, doi: 10.1038/s41598-024-53455-9. DOI: https://doi.org/10.1038/s41598-024-53455-9
    29. Munjin and D. Kurniadin, “The Implementation of Risk Management (A Case Study at State Islamic University of Walisongo Semarang , Indonesia),” Int. J. Multidiscip. Res. Anal., vol. 08, no. 11, pp. 6323–6336, 2025, doi: 10.47191/ijmra/v8-i11-50. DOI: https://doi.org/10.47191/ijmra/v8-i11-50
    30. Handriadi, H. N. O. Kustiyanto, and S. Chai, “Evaluation of Risk Management Implementation in Educational Institutions,” Int. J. Educ. Narrat., vol. 3, no. 1, pp. 50–58, 2025. DOI: https://doi.org/10.70177/ijen.v3i1.2156
    31. Mónica Hernández-Campos, A. Gonzalez-Torres, and F. J. García-Peñalvo, “Learning Outcomes Evaluation Through Learning Analytics Systems in Higher Education : A Systematic Literature Review,” SAGE Open, vol. 15, no. September, pp. 1–17, 2025, doi: 10.1177/21582440251347374. DOI: https://doi.org/10.1177/21582440251347374
    32. F. Jose, “Learning Outcomes Evaluation Through Learning Analytics Systems in Higher Education” no. September, pp. 1–17, 2025
    33. R. K. Yin, Case Study Research and Applications: Design and Methods, 6th ed. SAGE Publications, 2018.
    34. J. W. Creswell, Research Design: Qualitative, Quantitative, and Mixed Method Approaches, 4th ed. SAGE Publications.
    35. B. Barghi and S. S. Sikari, “Qualitative and quantitative project risk assessment using a hybrid PMBOK model developed under uncertainty conditions,” Heliyon, vol. 6, no. 1, 2020, doi: 10.1016/j.heliyon.2019.e03097. DOI: https://doi.org/10.1016/j.heliyon.2019.e03097
    36. V. Braun, V. Clarke, V. Braun, and V. Clarke, “Using thematic analysis in psychology Using thematic analysis in psychology,” Qual. Res. Psychol., vol. 0887, no. 2006, 2008, doi: 10.1191/1478088706qp063oa. DOI: https://doi.org/10.1191/1478088706qp063oa
    37. V. Braun, V. Clarke, V. Braun, and V. Clarke, “To saturate or not to saturate ? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales useful concept for thematic analysis and sample-size rationales,” Qual. Res. Sport. Exerc. Heal., vol. 13, no. 2, pp. 201–216, 2021, doi: 10.1080/2159676X.2019.1704846. DOI: https://doi.org/10.1080/2159676X.2019.1704846
    38. J. C. R. Moreira, “Risk Management in Education: A Systematic Literature Review of the Last Five Years,” RGSA – Rev. Gestão Soc. e Ambient., vol. 19, no. 1, pp. 1–15, 2025, doi: 10.24857/rgsa.v19n1-127. DOI: https://doi.org/10.24857/rgsa.v19n1-127
    39. K. Oliveira, M. Méxas, M. Meiriño, and G. Drumond, “Critical success factors associated with the implementation of enterprise risk management,” J. Risk Res., vol. 9877, pp. 1–16, 2019, doi: 10.1080/13669877.2018.1437061. DOI: https://doi.org/10.1080/13669877.2018.1437061
    40. T. Fawns, D. Boud, and P. Dawson, “Assessment & Evaluation in Higher Education Identifying what our students have learned : a framework for practical assessment validation,” Assess. Eval. High. Educ., vol. 0, no. 0, pp. 1–17, 2026, doi: 10.1080/02602938.2026.2620053. DOI: https://doi.org/10.1080/02602938.2026.2620053
    41. M. Schurr, M. De Tuya, and K. Noll, “Risk – Informed Decision Making in Information System Implementation Projects : Using Qualitative Assessment and Evaluation of Stakeholders’ Perceptions of Risk,” in Proceedings of the 50th Hawaii International Conference on System Sciences, 2017, pp. 6120–6129. DOI: https://doi.org/10.24251/HICSS.2017.739
    42. M. P. Narasimhan, “A Conceptual Overview of Risk Management in Education Sector Projects,” Holist. Res. Perspect., vol. 5, pp. 138–145, 2020, doi: 10.47059/CIIR/BP20002/13. DOI: https://doi.org/10.47059/CIIR/BP20002/13
    43. S. Pastore, “Teacher assessment literacy : a systematic review,” 2023. DOI: https://doi.org/10.3389/feduc.2023.1217167
    44. M. M. De Carvalho and R. R. Junior, “Impact of risk management on project performance : the importance of soft skills,” Int. J. Prod. Res., vol. 7543, pp. 1–20, 2015, doi: 10.1080/00207543.2014.919423. DOI: https://doi.org/10.1080/00207543.2014.919423
    45. Y. Xu and G. T. L. Brown, “Teacher assessment literacy in practice : A reconceptualization,” Teach. Teach. Educ., vol. 58, p. 2016, 2016, doi: 10.1016/j.tate.2016.05.010. DOI: https://doi.org/10.1016/j.tate.2016.05.010
    46. B. Stensaker, G. T. Bilbow, L. Breslow, and R. Van Der Vaart, Strengthening Teaching and Learning in Research Universities. Springer Nature. doi: 10.1007/978-3-319-56499-9. DOI: https://doi.org/10.1007/978-3-319-56499-9
    47. R. I. Asiyai, “Best practices for quality assurance in higher education : implications for educational administration,” Int. J. Leadersh. Educ., vol. 25, no. 5, pp. 843–854, 2022, doi: 10.1080/13603124.2019.1710569. DOI: https://doi.org/10.1080/13603124.2019.1710569
    48. T. A. Carbone and D. D. Tippett, “Project Risk Management Using the Project Risk FMEA,” Eng. Manag. J., vol. 16, no. 4, pp. 28–35, 2004, doi: 10.1080/10429247.2004.11415263. DOI: https://doi.org/10.1080/10429247.2004.11415263
    49. A. Nurrifqhi and J. Sembiring, “Risk Assessment Model Development in GX PMBOK Using Checklist Scenario Analysis and Simple Additive Weighting,” 2023 Int. Semin. Intell. Technol. Its Appl., pp. 660–665, 2023, doi: 10.1109/ISITIA59021.2023.10221069. DOI: https://doi.org/10.1109/ISITIA59021.2023.10221069
    50. K. A. O. Al-husseini and A. H. Obaid, “Interaction between project tasks and risk management tasks in software development,” Period. Eng. Nat. Sci., vol. 8, no. 4, pp. 2300–2308, 2020.