Designing Formative Adaptive Assessment for Engineering Education: Integrating Computerized Adaptive Testing and Competency-Based Diagnostic Modelling

Mohamed El Msayer; Bouchra Bouihi; Abdelmajid Bousselham; Essaadia Aoula; Adel Deraoui

doi:10.3991/ijep.v16i1.60479

Authors

Mohamed El Msayer Laboratory 2IACS ENSET Mohammedia, Hassan II University of Casablanca, Casablanca, Morocco https://orcid.org/0009-0000-5643-7437
Bouchra Bouihi Laboratory 2IACS ENSET Mohammedia, Hassan II University of Casablanca, Casablanca, Morocco https://orcid.org/0000-0002-1652-8470
Abdelmajid Bousselham Laboratory 2IACS ENSET Mohammedia, Hassan II University of Casablanca, Casablanca, Morocco
Essaadia Aoula Laboratory 2IACS ENSET Mohammedia, Hassan II University of Casablanca, Casablanca, Morocco
Adel Deraoui Regional Center for Education and Training, Casablanca, Morocco

DOI:

https://doi.org/10.3991/ijep.v16i1.60479

Keywords:

Engineering education, Formative adaptive assessment, Computerized adaptive testing, Item Response Theory, Curriculum Alignment, Educational Measurement

Abstract

Assessing competencies in engineering education increasingly requires digital assessment approaches that support learning regulation, instructional decision-making, and educational quality, rather than focusing solely on measurement efficiency. Computerized adaptive testing (CAT), grounded in Item Response Theory (IRT), provides a robust methodological foundation for personalized assessment. However, its pedagogical effectiveness in formative contexts depends critically on curriculum alignment, diagnostic capacity, and adaptive control strategies. This study proposes and evaluates a formative adaptive assessment framework for engineering education that integrates an IRT-based CAT engine with a Bayesian network– based diagnostic component. The framework is designed to support competency-oriented feedback, learning monitoring, and instructional interpretation within a curriculum-aligned assessment structure. Assessment relies on dichotomous multiple-choice items explicitly aligned with engineering learning outcomes, while item selection dynamically adapts to learners’ evolving proficiency estimates. In parallel, probabilistic diagnostic modelling prioritizes under-assessed competencies throughout the adaptive process. Item calibration was conducted using empirical data collected from 612 university students in computer science, and system performance was examined through a simulation-based evaluation involving 500 simulated learners. Results demonstrate high estimation accuracy (r = 0.912) and satisfactory reliability for formative use across most learner profiles. Reduced precision at the extremes of the proficiency continuum and imbalances in item exposure were also observed, highlighting structural limitations primarily related to item bank coverage and curriculum representation rather than to the adaptive algorithms themselves. Overall, the proposed framework positions adaptive assessment as a pedagogically grounded tool for formative learning support, instructional decision-making, and quality assurance in engineering education.

Author Biography

Bouchra Bouihi, Laboratory 2IACS ENSET Mohammedia, Hassan II University of Casablanca, Casablanca, Morocco

Bouchra Bouihi received the degree in computer science engineering from the National School of Applied Sciences, in 2014, and the Ph.D. degree in computer science from the Faculty of Science and Technology, Hassan 1st University, Settat, Morocco, in 2019. She is currently an Affiliate Professor with the Department of Mathematics and Computer Science and a Research Member of the 2IACS Laboratory, ENSET Mohammedia, University Hassan II , Casablanca. Her research interests include artificial intelligence, with specific interests in machine learning, deep learning, and ontology engineering. Her work delves into solving real-world problems including the education field by using AI models.

References

[1] Y. Laaziz, G. Chemsi, et M. Radid, « The Influence of E-Assessment on Students’ Cognitive Engagement in Higher Education », International Journal of Engineering Pedagogy (iJEP), vol. 14, no 4, p. 54‑67, mai 2024, doi: 10.3991/ijep.v14i4.48293.

[2] K. Smahi, O. Labouidya, et K. E. Khadiri, « Enhancing Online Assessment Quality in Higher Education: The Design of Moodle Plug-in for Personalised Exam Revision (PER) », International Journal of Engineering Pedagogy (iJEP), vol. 15, no 5, p. 127‑140, juill. 2025, doi: 10.3991/ijep.v15i5.55055.

[3] M. E. Msayer, E.-S. Aoula, et B. Bouihi, « Artificial intelligence in computerized adaptive testing to assess the cognitive performance of students: A Systematic Review », in 2024 International Conference on Intelligent Systems and Computer Vision (ISCV), mai 2024, p. 1‑8. doi: 10.1109/ISCV60512.2024.10620092.

[4] K. Sodoké, G. Raîche, et R. Nkambou, « La plateforme d’évaluation adaptative des apprentissages: PersonFit ».

[5] Q. Liu et al., « Survey of Computerized Adaptive Testing: A Machine Learning Perspective », 5 avril 2024, arXiv: arXiv:2404.00712. doi: 10.48550/arXiv.2404.00712.

[6] P. T. D. Thuy, N. T. Van, N. H. Trang, N. M. Thuong, V. T. T. Hien, et V. Q. Chung, « Designing an AI-Supported Formative Assessment Model for Pre-Service Mathematics Teacher Self-Study in Vietnam », International Journal of Interactive Mobile Technologies (iJIM), vol. 19, no 22, p. 50‑68, nov. 2025, doi: 10.3991/ijim.v19i22.57723.

[7] D. Spenassato, A. C. Bornia, et R. Tezza, « Computerized Adaptive Testing: A Review of Research and Technical Characteristics », IEEE Latin America Transactions, vol. 13, no 12, p. 3890‑3898, déc. 2015, doi: 10.1109/TLA.2015.7404924.

[8] E. Tsutsumi, R. Kinoshita, et M. Ueno, « Deep Item Response Theory as a Novel Test Theory Based on Deep Learning », Electronics, vol. 10, no 9, Art. no 9, janv. 2021, doi: 10.3390/electronics10091020.

[9] M. Kaplan, J. de la Torre, et J. R. Barrada, « New Item Selection Methods for Cognitive Diagnosis Computerized Adaptive Testing », Appl Psychol Meas, vol. 39, no 3, p. 167‑188, mai 2015, doi: 10.1177/0146621614554650.

[10] Y. Choi et C. McClenen, « Development of Adaptive Formative Assessment System Using Computerized Adaptive Testing and Dynamic Bayesian Networks », Applied Sciences, vol. 10, no 22, Art. no 22, janv. 2020, doi: 10.3390/app10228196.

[11] L. Burigana, « Bayesian networks and knowledge structures in cognitive assessment: Remarks on basic comparable aspects », Journal of Mathematical Psychology, vol. 123, p. 102875, déc. 2024, doi: 10.1016/j.jmp.2024.102875.

[12] N. Thompson et D. Weiss, « A Framework for the Development of Computerized Adaptive Tests », Practical Assessment, Research, and Evaluation, vol. 16, no 1, nov. 2019, doi: https://doi.org/10.7275/wqzt-9427.

[13] M. von Davier, « Automated Item Generation with Recurrent Neural Networks », Psychometrika, vol. 83, no 4, p. 847‑857, déc. 2018, doi: 10.1007/s11336-018-9608-y.

[14] J. B. Bjorner, C.-H. Chang, D. Thissen, et B. B. Reeve, « Developing tailored instruments: item banking and computerized adaptive assessment », Qual Life Res, vol. 16 Suppl 1, p. 95‑108, 2007, doi: 10.1007/s11136-007-9168-6.

[15] J. Sharpnack, P. Mulcaire, K. Bicknell, G. LaFlair, et K. Yancey, « AutoIRT: Calibrating Item Response Theory Models with Automated Machine Learning », 13 septembre 2024, arXiv: arXiv:2409.08823. doi: 10.48550/arXiv.2409.08823.

[16] S. E. Stemler et A. Naples, « Rasch Measurement v. Item Response Theory: Knowing When to Cross the Line », Practical Assessment, Research, and Evaluation, vol. 26, no 1, Art. no 1, mai 2021, doi: 10.7275/v2gd-4441.

[17] R. Ostini et M. Nering, Polytomous Item Response Theory Models. 2455 Teller Road, Thousand Oaks California 91320 United States of America: SAGE Publications, Inc., 2006. doi: 10.4135/9781412985413.

[18] Y. Zhuang, Q. Liu, Z. Huang, Z. Li, S. Shen, et H. Ma, « Fully Adaptive Framework: Neural Computerized Adaptive Testing for Online Education », Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no 4, Art. no 4, juin 2022, doi: 10.1609/aaai.v36i4.20399.

[19] E. V. Chumakova, T. A. Chernova, Y. A. Belyaeva, D. G. Korneev, et M. S. Gasparian, « Use of Neural Networks in the Adaptive Testing System », IJACSA, vol. 13, no 5, 2022, doi: 10.14569/IJACSA.2022.0130504.

[20] T. Song et al., « Novel Application of Deep Learning for Adaptive Testing Based on Long Short-Term Memory », in 2019 IEEE 37th VLSI Test Symposium (VTS), avr. 2019, p. 1‑6. doi: 10.1109/VTS.2019.8758628.

[21] M. Phankokkruad et K. Woraratpanya, « An Automated Decision System for Computer Adaptive Testing Using Genetic Algorithms », in 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, août 2008, p. 655‑660. doi: 10.1109/SNPD.2008.118.

[22] M. Ueno et P. Songmuang, « Computerized Adaptive Testing Based on Decision Tree », 2010 10th IEEE International Conference on Advanced Learning Technologies, p. 191‑193, juill. 2010, doi: 10.1109/ICALT.2010.58.

[23] M. Imran, N. Almusharraf, M. S. Abdellatif, et M. Y. Abbasova, « Artificial Intelligence in Higher Education: Enhancing Learning Systems and Transforming Educational Paradigms », International Journal of Interactive Mobile Technologies (iJIM), vol. 18, no 18, p. 34‑48, sept. 2024, doi: 10.3991/ijim.v18i18.49143.

[24] Y. Choi et C. McClenen, « Development of Adaptive Formative Assessment System Using Computerized Adaptive Testing and Dynamic Bayesian Networks », Applied Sciences, vol. 10, no 22, Art. no 22, janv. 2020, doi: 10.3390/app10228196.

[25] V. M. G. Jatobá, J. S. Farias, V. Freire, A. S. Ruela, et K. V. Delgado, « ALICAT: a customized approach to item selection process in computerized adaptive testing », Journal of the Brazilian Computer Society, vol. 26, no 1, p. 4, mai 2020, doi: 10.1186/s13173-020-00098-z.

[26] J. R. Barrada, J. Olea, V. Ponsoda, et F. J. Abad, « Incorporating randomness in the Fisher information for improving item-exposure control in CATs », Br J Math Stat Psychol, vol. 61, no Pt 2, p. 493‑513, nov. 2008, doi: 10.1348/000711007X230937.

[27] J. Wang, Y. Xie, Z. Feng, et J. Li, « Psychometric properties of computerized adaptive testing for chronic obstructive pulmonary disease patient-reported outcome measurement », Health Qual Life Outcomes, vol. 22, p. 73, sept. 2024, doi: 10.1186/s12955-024-02291-6.

[28] R. K. Hambleton et H. Swaminathan, Item Response Theory: Principles and Applications. Springer Science & Business Media, 2013.

[29] D. Кazimova, G. Tazhigulova, G. Shraimanova, A. Zatyneyko, et A. Sharzadin, « Transforming University Education with AI: A Systematic Review of Technologies, Applications, and Implications », International Journal of Engineering Pedagogy (iJEP), vol. 15, no 1, p. 4‑24, janv. 2025, doi: 10.3991/ijep.v15i1.50773.

[30] A. Klein, J. Müller, N. Link, et R. Schaffer, « Assessment and Level Modelling in Fundamentals of Electrical Engineering », International Journal of Engineering Pedagogy (iJEP), vol. 14, no 6, p. 37‑53, août 2024, doi: 10.3991/ijep.v14i6.47865.

[31] M. L. Stocking et C. Lewis, « Methods of Controlling the Exposure of Items in CAT », in Computerized Adaptive Testing: Theory and Practice, W. J. van der Linden et G. A. W. Glas, Éd., Dordrecht: Springer Netherlands, 2000, p. 163‑182. doi: 10.1007/0-306-47531-6_9.

[32] Y. Song, J. Du, et Q. Zheng, « Automatic item generation for educational assessments: a systematic literature review », Interactive Learning Environments, vol. 33, no 9, p. 5386‑5405, oct. 2025, doi: 10.1080/10494820.2025.2482588.

[33] M. Bellaj, A. B. Dahmane, S. Boudra, et M. L. Sefian, « Big Data Analytics in Higher Education: A New Adaptive Learning Analytics Model Integrating Traditional Approaches », International Journal of Interactive Mobile Technologies (iJIM), vol. 18, no 06, p. 24‑39, mars 2024, doi: 10.3991/ijim.v18i06.46289.

[34] I. Azaiz, O. Deckarm, et S. Strickroth, « AI-Enhanced Auto-Correction of Programming Exercises: How Effective is GPT-3.5? », International Journal of Engineering Pedagogy (iJEP), vol. 13, no 8, p. 67‑83, déc. 2023, doi: 10.3991/ijep.v13i8.45621.

[35] S. Lim et S. W. Choi, « Item exposure and utilization control methods for optimal test assembly », Behaviormetrika, vol. 51, no 1, p. 125‑156, janv. 2024, doi: 10.1007/s41237-023-00214-1.