Optimising Dyslexia Intervention Leveraging a Mobile Adaptive Multi-Sensory AI Model with Real-Time Speech Analytic

Resmi Darni; Yulyanti Harisman; Lili Dasa Putri; Efi Fitriana; Rezki Ashriyana Sulistiobudhi; Andhika Herayono

doi:10.3991/ijim.v20i09.61571

Authors

Resmi Darni Universitas Negeri Padang, Padang, Indonesia
Yulyanti Harisman Universitas Negeri Padang, Padang, Indonesia https://orcid.org/0000-0002-5219-8851
Lili Dasa Putri Universitas Negeri Padang, Padang, Indonesia
Efi Fitriana Universitas Padjadjaran, Bandung, Indonesia
Rezki Ashriyana Sulistiobudhi Universitas Padjadjaran, Bandung, Indonesia
Andhika Herayono Universitas Negeri Padang, Padang, Indonesia

DOI:

https://doi.org/10.3991/ijim.v20i09.61571

Keywords:

Phonological Dyslexia, Adaptive AI, Speech Analytics, Mobile Learning, Cognitive Rehabilitation

Abstract

Phonological dyslexia is a neurodevelopmental learning disorder characterised by persistent difficulties in phoneme–grapheme mapping, phonological decoding, and pronunciation accuracy. Conventional dyslexia interventions are often non-adaptive, therapist-dependent, and limited in scalability, reducing their effectiveness in inclusive education contexts. This study aims to develop and evaluate a mobile adaptive multi-sensory artificial intelligence (AI) model integrated with real-time speech analytics to optimise phonological dyslexia intervention. The study employed a research and development (R&D) approach using the ADDIE framework, combined with a quasi-experimental pre- and post-test control group design. Participants were elementary school learners diagnosed with phonological dyslexia. The proposed system integrates multi-sensory phonological training, adaptive difficulty adjustment, and real-time speech analytics to provide immediate feedback and personalised learning pathways. Data were collected through expert validation, usability questionnaires, phonological reading tests, and speech analytics metrics, including phoneme error rate (PER) and word error rate (WER).

References

[1] Suharno, N. A. Pambudi, and B. Harjanto, “Vocational education in Indonesia: History, development, opportunities, and challenges,” Child. Youth Serv. Rev., vol. 115, no. May, p. 105092, 2020, doi: 10.1016/j.childyouth.2020.105092.

[2] R. Darni, Y. Harisman, D. Sukma, E. Fitriana, and R. A. Sulistiobudi, “Integration of a Mobile-Based Smart Measurement System to Assess the Level of Work Readiness of Vocational Students in Higher Education,” International Journal of Interactive Mobile Technologies (iJIM), vol. 18, no. 17, pp. 61–74, 2024. doi: 10.3991/ijim.v18i17.50679.

[3] C. Dyah, S. Indrawati, P. Ninghardjanti, C. Huda, and A. Dirgatama, “The effect of practicum learning based audiovisual on students ’ learning outcomes in Indonesian vocational secondary school,” vol. 11, no. 1, pp. 403–408, 2022, doi: 10.11591/ijere.v11i1.21762.

[4] D. Amini, M. H. Anhari, and A. Ghasemzadeh, “Modeling the relationship between metacognitive strategy awareness, self-regulation and reading proficiency of Iranian EFL learners,” Cogent Educ., vol. 7, no. 1, 2020, doi: 10.1080/2331186X.2020.1787018.

[5] T. Tang, A. M. Abuhmaid, M. Olaimat, D. M. Oudat, M. Aldhaeebi, and E. Bamanger, “Efficiency of flipped classroom with online-based teaching under COVID-19,” Interact. Learn. Environ., vol. 0, no. 0, pp. 1–12, 2020, doi: 10.1080/10494820.2020.1817761.

[6] S. Wang, R. H. Wardi, and R. Ghazali, “Influencing factors on undergraduate engagement with Chinese visual arts in the intangible cultural heritage : A structural equation model approach,” Alexandria Eng. J., vol. 123, no. March, pp. 332–340, 2025, doi: 10.1016/j.aej.2025.03.060.

[7] M. Muafi, W. Syafri, H. Prabowo, and S. A. Nur, “Digital Entrepreneurship in Indonesia: A Human Capital Perspective,” J. Asian Financ. Econ. Bus., vol. 8, no. 3, pp. 351–359, 2021, doi: 10.13106/jafeb.2021.vol8.no3.0351.

[8] S. Nelson, R. Darni, F. Haris, I. Ilham, J. Ndayisenga, S. Septri, Y. Ockta, D. N. Sari, and R. Festiawan, “The effectiveness of learning media based on digital augmented reality (AR) technology on the learning outcomes of martial arts,” Retos, no. 63, pp. 878–885, Feb. 2025. doi: 10.47197/retos.v63.108948.

[9] S. Seo, H. Park, and C. Koo, “Impact of interactive learning elements on personal learning performance in immersive virtual reality for construction safety training,” Expert Syst. Appl., vol. 251, no. April, p. 124099, 2024, doi: 10.1016/j.eswa.2024.124099.

[10] B. Sonnleitner and Y. R. Sagaert, “Computers and Education : Art ficial Intelligence Evaluation of early student performance prediction given concept drift,” Comput. Educ. Artif. Intell., vol. 8, no. July 2024, p. 100369, 2025, doi: 10.1016/j.caeai.2025.100369.

[11] R. Darni, L. Mursyida, and A. D. Samala, “Career Exploration System (C-EXSYS) in Era Society 5.0 Based on Expert System,” J. Teknol. Inf. dan Pendidik., vol. 14, no. 2, pp. 131–143, 2021, doi: 10.24036/tip.v14i2.491.

[12] T. Supriyadi and N. Rahminawati, “Higher-Order Thinking Skills in Primary School : Teachers ’ Perceptions of Islamic Education,” vol. 9, no. 1, pp. 56–76, 2022.

[13] M. Anwar et al., “Blended Learning Based Project In Electronics Engineering Education Courses: A Learning Innovation after the Covid-19 Pandemic,” Int. J. Interact. Mob. Technol., vol. 16, no. 14, pp. 107–122, 2022, doi: 10.3991/ijim.v16i14.33307.

[14] J. Liu, Y. Zheng, L. Zhou, F. Jin, and H. Chen, “Engineering Applications of Artificial Intelligence A novel probabilistic linguistic decision-making method with consistency improvement algorithm and DEA cross-efficiency,” Eng. Appl. Artif. Intell., vol. 99, no. 111, p. 104108, 2021, doi: 10.1016/j.engappai.2020.104108.

[15] N. Alobidi, R. Alnanih, and H. Bakhsh, “ScienceDirect Virtual Virtual Reality-Based Reality-Based Interventions Interventions for for Improving Improving Learning Learning Outcomes in Children with ADHD Outcomes in Children with ADHD,” Procedia Comput. Sci., vol. 241, no. 2019, pp. 179–186, 2024, doi: 10.1016/j.procs.2024.08.025.

[16] R. Darni, Y. Harisman, and I. N. A. F. Setiawan, “The Implementation and Empirical Analysis of Adaptive Virtual Mentor: Mobile Technology Empowers Introverts’ Business Communication Skills,” International Journal of Interactive Mobile Technologies (iJIM), vol. 19, no. 8, pp. 159–173, 2025. doi: 10.3991/ijim.v19i08.53887..

[17] M. Husen and R. Aditama, “Online Career Position Dictionary as Media to Improve Junior High School Students’ Career Exploration,” vol. 18, no. 02, pp. 133–145, 2020, [Online]. Available: http://jurnal.uns.ac.id/Teknodika

[18] H. Nofrianto, J. Jama, A. Indra, B. Rahim, S. Wardi, and U. Verawardina, “Validity of Cooperative-Discovery Learning Model to Improve Competencies of Engineering Students,” vol. 11, no. 12, pp. 1134–1138, 2020.

[19] D. Fitria et al., “The Effect of Using Web-Based Interactive Learning Media for Vocational High School Students to Understanding of Looping : Qualitative Approach,” vol. 5, no. March, 2022, doi: 10.17509/jsl.v5i1.35534.

[20] A. P. Pribadi, Y. Mukasyafah, and R. Rahman, “Heliyon Analysis of the effectiveness and user experience of employing virtual reality to enhance the efficacy of occupational safety and health learning for electrical workers and graduate students,” Heliyon, vol. 10, no. 15, p. e34918, 2024, doi: 10.1016/j.heliyon.2024.e34918.

[21] A. Kompaniets and H. Chemerys, “Using 3D modelling in design training simulator with,” pp. 213–223, 2019.

[22] S. J. Seage and M. Türegün, “The effects of blended learning on STEM achievement of elementary school students,” Int. J. Res. Educ. Sci., vol. 6, no. 1, pp. 133–140, 2020, doi: 10.46328/ijres.v6i1.728.

[23] X. Song, Y. Cong, Y. Song, Y. Chen, and P. Liang, “A bearing fault diagnosis model based on CNN with wide convolution kernels,” J. Ambient Intell. Humaniz. Comput., vol. 13, no. 8, pp. 4041–4056, 2022, doi: 10.1007/s12652-021-03177-x.

[24] R. Darni, Y. Harisman, D. Sukma, E. Fitriana, R. A. Sulistiobudi, and A. Herayono, “Work Readiness in Vocational Education: Perception and Correlation Between Measurement Level and System Integration,” TEM Journal, vol. 14, no. 2, pp. 1388–1397, May 2025. doi: 10.18421/TEM142-39.

[25] R. Ramadhani, N. S. Bina, S. F. Sihotang, S. D. Narpila, and M. R. Mazaly, “Students’ critical mathematical thinking abilities through flip-problem based learning model based on LMS-google classroom,” J. Phys. Conf. Ser., vol. 1657, no. 1, 2020, doi: 10.1088/1742-6596/1657/1/012025.

[26] K. Domains, C. Scale, T. K. Domains, C. Scale, K. Domains, and C. Scale, “Translation and Validation of the Kaufman Domains of Creativity Scale on a Croatian Sample of Early Childhood and Preschool Education Students Prenos in potrditev Kaufmanove lestvice ustvarjalnih področij na hrvaškem vzorcu učencev v zgodnjem otroštvu in predšolskih otrok,” vol. 11, pp. 163–179, 2021, doi: 10.26529/cepsj.708.