An AI-Powered Interactive Mobile Application for Multilingual Conference Accessibility: Design, Deployment, and Field Evaluation

Ameen Shaheen; Wael Alzyadat; Aysh Alhroob

doi:10.3991/ijim.v20i18.62706

Authors

Ameen Shaheen Al-Zaytoonah University of Jordan, Amman, Jordan https://orcid.org/0009-0003-3120-1892
Wael Alzyadat Al-Zaytoonah University of Jordan, Amman, Jordan
Aysh Alhroob Al-Zaytoonah University of Jordan, Amman, Jordan https://orcid.org/0000-0002-4653-7386

DOI:

https://doi.org/10.3991/ijim.v20i18.62706

Keywords:

Artificial Intelligence, Interactive Mobile Application, Conference Accessibility, Real-Time Speech Translation, Multilingual Communication

Abstract

Global academic conferences often face accessibility challenges for multilingual participants, especially during live sessions where real-time comprehension is important. This study presents the design, deployment, and evaluation of an AI-powered multilingual mobile application developed to support conference accessibility. The application integrates real-time speech translation, agenda management, venue navigation, and cultural information in a single mobile platform. It was deployed during the International Conference on Information Technology (ICIT 2025) and evaluated with 200 participants across four supported languages: Arabic, English, French, and Spanish. Using a design-based research (DBR) approach, the evaluation combined objective performance metrics with user-reported usability measures. The system achieved 92.5% translation accuracy, a word error rate (WER) of 7.2%, and system uptime of 99.1%. The usability results showed high participant satisfaction, with a mean score of 4.6 out of 5 and strong internal reliability (Cronbach’s α = 0.91). Open-ended feedback indicated that participants valued translation usefulness, ease of use, and the integration of conference services, while also suggesting additional language support and offline functionality. The findings suggest that integrating AI-based multilingual translation with mobile conference services can help reduce language barriers and improve participant experience in international academic events.

References

[1] T. Rogers and P. Wynn-Moylan, Conferences and Conventions. Routledge, 2022. doi: https://doi.org/10.4324/9781003298953.

[2] A. U. Patel, Q. Gu, R. Esper, D. Maeser, and N. Maeser, “The Crucial Role of Interdisciplinary Conferences in Advancing Explainable AI in Healthcare,” BioMedInformatics, vol. 4, no. 2, pp. 1363–1383, Jun. 2024, doi: https://doi.org/10.3390/biomedinformatics4020075

[3] Y. Xia, S.-Y. Shin, and J.-C. Kim, “Cross-Cultural Intelligent Language Learning System (CILS): Leveraging AI to Facilitate Language Learning Strategies in Cross-Cultural Communication,” Applied Sciences, vol. 14, no. 13, p. 5651, Jan. 2024, doi: https://doi.org/10.3390/app14135651.

[4] L. Li, G. Wang, and G. Freeman, “‘It’s about Research. It’s Not about Language’: Understanding and Designing for Mitigating Non-Native English-Speaking Presenters’ Challenges in Live Q&A Sessions at Academic Conferences,” Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pp. 1–15, Apr. 2025, doi: https://doi.org/10.1145/3706598.3713124.

[5] R. E. Prasetya, “Exploring English language instruction through synchronous virtual meetings in Indonesian EFL context: Pedagogical approaches, opportunities and challenges, and multimedia integration,” Journal on English as a Foreign Language, vol. 14, no. 1, pp. 179–208, Mar. 2024, doi: https://doi.org/10.23971/jefl.v14i1.7532.

[6] J. Bernatavičienė, “15th Conference on Data Analysis Methods for Software Systems. November 28–30, 2024,” Vilnius University Proceedings, vol. 52, pp. 1–136, Nov. 2024, doi: https://doi.org/10.15388/damss.15.2024.

[7] Michael Robert Haupt, M. Chiu, J. Chang, Z. Li, R. Cuomo, and T. K. Mackey, “Detecting nuance in conspiracy discourse: Advancing methods in infodemiology and communication science with machine learning and qualitative content coding,” PLOS ONE, vol. 18, no. 12, pp. e0295414–e0295414, Dec. 2023, doi: https://doi.org/10.1371/journal.pone.0295414.

[8] N. Arman, F. Khamayseh, and E. Awad, “A Semi-Automated Approach for Classifying Non Functional Arabic User Requirements using NLP Tools,” International Journal of Advances in Soft Computing and its Applications, vol. 17, no. 1, Mar. 2025, doi: https://doi.org/10.15849/ijasca.250330.15.

[9] M. Lafi, B. Hawashin, and S. AlZu’ bi, “Eliciting Requirements from Stakeholders’ Responses Using Natural Language Processing,” Computer Modeling in Engineering & Sciences, vol. 127, no. 1, pp. 99–116, 2021, doi: https://doi.org/10.32604/cmes.2021.013026.

[10] Y. A. Mohamed, A. Khanan, M. Bashir, A. H. H. M. Mohamed, M. A. E. Adiel and M. A. Elsadig, "The Impact of Artificial Intelligence on Language Translation: A Review," in IEEE Access, vol. 12, pp. 25553-25579, 2024, doi: https://doi.org/10.1109/ACCESS.2024.3366802

[11] A. Genovese et al., “Artificial intelligence in clinical settings: a systematic review of its role in language translation and interpretation,” Annals of Translational Medicine, vol. 12, no. 6, pp. 117–117, Dec. 2024, doi: https://doi.org/10.21037/atm-24-162.

[12] S. Jaballi, M. J. Hazar, S. Zrigui, A. Mahjoubi, H. Nicolas, and M. Zrigui, “Decoding Multilingual Topic Dynamics and Trend Identification through ARIMA Time Series Analysis on Social Networks: A Novel Data Translation Framework Enhanced by LDA/HDP Models,” Journal of Electrical and Computer Engineering, vol. 2024, no. 1, Jan. 2024, doi: https://doi.org/10.1155/2024/6669491.

[13] M. Amini, L. Ravindran, and K.-F. Lee, “Implications of using AI in Translation Studies: Trends, Challenges, and Future Direction,” Asian Journal of Research in Education and Social Sciences, vol. 6, no. 1, Apr. 2024, doi: https://doi.org/10.55057/ajress.2024.6.1.67.

[14] D. R. Seals, “The Complete Guide to Hosting a Guest Speaker,” Advances in Physiology Education, vol. 48, no. 2, Jan. 2024, doi: https://doi.org/10.1152/advan.00187.2023.

[15] F. Federici, “Translating hazards: multilingual concerns in risk and emergency communication,” Translator, vol. 28, no. 4, pp. 375–398, Oct. 2022, doi: https://doi.org/10.1080/13556509.2023.2203998.

[16] O. Oliinyk, N. Bevz, I. Shuliakov, and I. Derevianko, “The Impact of Multilingual Education On Developing Communication Skills for University Students Within Foreign Language Learning,” Taduction et Langues, vol. 23, no. 1, pp. 125–142, Jul. 2024, doi: https://doi.org/10.52919/translang.v23i1.973.

[17] S. G. Yaseen, Q. H. A. Albadrany, D. Dajani, and M. M. Al-Afaishat, “Artificial Intelligence Regulation: A Bibliometric Analysis,” International Journal of Advances in Soft Computing and its Applications, vol. 17, no. 1, Mar. 2025, doi: https://doi.org/10.15849/ijasca.250330.12.

[18] R. O. Ogundokun, J. B. Awotunde, S. Misra, T. Segun-Owolabi, E. A. Adeniyi, and V. Jaglan, “An android based language translator application,” Journal of Physics: Conference Series, vol. 1767, no. 1, p. 012032, Feb. 2021, doi: https://doi.org/10.1088/1742-6596/1767/1/012032.

[19] D. Hu and J. Li, “Contrastive Learning for Robust Neural Machine Translation with ASR Errors,” Lecture notes in computer science, vol. 13551, pp. 81–91, Jan. 2022, doi: https://doi.org/10.1007/978-3-031-17120-8_7.

[20] S. Kafle and M. Huenerfauth, “Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing,” Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, Oct. 2017, doi: https://doi.org/10.1145/3132525.3132542.

[21] A. Herrmann-Werner, T. Loda, S. Zipfel, M. Holderried, F. Holderried, and R. Erschens, “Evaluation of a Language Translation App in an Undergraduate Medical Communication Course: Proof-of-Concept and Usability Study,” JMIR mHealth and uHealth, vol. 9, no. 12, p. e31559, Dec. 2021, doi: https://doi.org/10.2196/31559.

[22] K. Shehadeh, N. Arman, and F. Khamayseh, “Classification of Arabic User Requirements: A Semi-Automated Approach using NLP Tools,” International Journal of Advances in Soft Computing and its Applications, vol. 16, no. 3, Nov. 2024, doi: https://doi.org/10.15849/ijasca.241130.01.

[23] L. Zhao et al., “Adaptive Policy with Wait-k Model for Simultaneous Translation,” Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 4816–4832, 2023, doi: https://doi.org/10.18653/v1/2023.emnlp-main.293.

[24] L. Zhang, K. Fan, J. Bu, and Z. Huang, “Training Simultaneous Speech Translation with Robust and Random Wait-k-Tokens Strategy,” Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7814–7831, Jan. 2023, doi: https://doi.org/10.18653/v1/2023.emnlp-main.484.

[25] M. Li, W. Liu, and C. Chen, “Adaptive Financial Literacy Enhancement through Cloud-Based AI Content Delivery: Effectiveness and Engagement Metrics”, Annal. App. Sci, vol. 5, no. 1, May 2024.

[26] M. Gupta, M. Dutta, and M. C. Kumar, “Direct Speech-to-Speech Neural Machine Translation: A Survey,” arXiv (Cornell University), Nov. 2024, doi: https://doi.org/10.48550/arxiv.2411.14453.

[27] L. Wang et al., “Benchmarking and Improving Long-Text Translation with Large Language Models,” Acl Anthology, pp. 7175–7187, Jan. 2024, doi: https://doi.org/10.18653/v1/2024.findings-acl.428.

[28] S. Katkov, A. Liotta, and A. Vietti, “ASR Systems Under Acoustic Challenges: A Multilingual Study,” Lecture notes in computer science, vol. 15450, pp. 200–213, Dec. 2024, doi: https://doi.org/10.1007/978-3-031-80607-0_16.

[29] K. Mølnvik, “Automatic Speech Recognition (ASR) Performance in Noisy Conditions; A Study on Norwegian Police Interviews,” Handle.net, 2024, doi: no.ntnu:inspera:187430191:46846561.

[30] J. R. Batista, Learn OpenAI Whisper: Transform your understanding of GenAI through robust and accurate speech processing solutions. Packt Publishing Ltd, 2024.

[31] Z. Ma et al., “Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 23, pp. 24840–24848, Apr. 2025, doi: https://doi.org/10.1609/aaai.v39i23.34666.

[32] S. Malakul and I. Park, “The effects of using an auto-subtitle system in educational videos to facilitate learning for secondary school students: learning comprehension, cognitive load, and satisfaction,” Smart Learning Environments, vol. 10, no. 1, Jan. 2023, doi: https://doi.org/10.1186/s40561-023-00224-2.

[33] N. Giovanni and W. Nurhayat, “The Effectiveness Of Automatic Speech Recognition In The Capcut Application For Developing Inclusive Learning Media,” JKTP: Jurnal Kajian Teknologi Pendidikan, vol. 8, no. 2, pp. 157–169, May 2025, doi: https://doi.org/10.17977/um038v8i22025p157.

[34] E. J. McDonnell, P. Liu, S. M. Goodman, R. Kushalnagar, J. E. Froehlich, and L. Findlater, “Social, Environmental, and Technical: Factors at Play in the Current Use and Future Design of Small-Group Captioning,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW2, pp. 1–25, Oct. 2021, doi: https://doi.org/10.1145/3479578.

[35] J. Cao, K.-Y. Lam, L.-H. Lee, X. Liu, P. Hui, and X. Su, “Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence,” ACM Computing Surveys, vol. 55, no. 9, Aug. 2022, doi: https://doi.org/10.1145/3557999.

[36] L.-H. Lee and Z. Lin, “Danger, Nuisance, Disregard: Analyzing User-Generated Videos for Augmented Reality Gameplay on Hand-held Devices,” Proceedings of the ACM on Human-Computer Interaction, vol. 8, no. CHI PLAY, pp. 1–33, Oct. 2024, doi: https://doi.org/10.1145/3677063.

[37] H. Stefanidi, M. Tatzgern, and A. Meschtscherjakov, “Augmented Reality on the Move: A Systematic Literature Review for Vulnerable Road Users,” Proceedings of the ACM on Human-Computer Interaction, vol. 8, no. MHCI, pp. 1–30, Sep. 2024, doi: https://doi.org/10.1145/3676490.

[38] R. Ivanov and V. Velkova, “Analyzing Visitor Behavior to Enhance Personalized Experiences in Smart Museums: A Systematic Literature Review,” Computers, vol. 14, no. 5, pp. 191–191, May 2025, doi: https://doi.org/10.3390/computers14050191.

[39] Google LLC, “Google Translate,” [Online]. Available: https://translate.google.com. Accessed: Jan. 2026.

[40] iTranslate GmbH, “iTranslate Voice & Text Translation App,” [Online]. Available: https://itranslate.com. Accessed: Jan. 2026.

[41] L. Tinoca, J. Piedade, S. Santos, A. Pedro, and S. Gomes, “Design-Based Research in the Educational Field: A Systematic Literature Review,” Education Sciences, vol. 12, no. 6, p. 410, Jun. 2022, doi: https://doi.org/10.3390/educsci12060410

[42] B. Langenberg, M. Janczyk, V. Koob, R. Kliegl, and A. Mayer, “A tutorial on using the paired t test for power calculations in repeated measures ANOVA with interactions,” Behavior Research Methods, vol. 55, no. 5, pp. 2467–2484, Aug. 2022, doi: https://doi.org/10.3758/s13428-022-01902-8

[43] NIST, “The NIST Scoring Toolkit (SCTK),” National Institute of Standards and Technology, [Online]. Available: https://www.nist.gov/itl/iad/mig/tools. Accessed: Jan. 2026.

[44] C. P. Malkewitz, P. Schwall, C. Meesters, and J. Hardt, “Estimating reliability: A comparison of Cronbach’s α, McDonald’s ωt and the greatest lower bound,” Social Sciences & Humanities Open, vol. 7, no. 1, p. 100368, Jan. 2023, doi: https://doi.org/10.1016/j.ssaho.2022.100368.

[45] I. Kusmaryono, D. Wijayanti, and H. R. Maharani, “Number of Response Options, Reliability, Validity, and Potential Bias in the Use of the Likert Scale Education and Social Science Research: a Literature Review,” International Journal of Educational Methodology, vol. 8, no. 4, pp. 625–637, Nov. 2022, doi: https://doi.org/10.12973/ijem.8.4.625.

[46] D. Lakens, “Concerns About Replicability Across Two Crises in Social Psychology,” International Review of Social Psychology, vol. 38, no. 1, 2025, doi: https://doi.org/10.5334/irsp.1036

[47] J. Elliott, “The Craft of Using NVivo12 to Analyze Open-Ended Questions: An Approach to Mixed Methods Analysis,” The Qualitative Report, vol. 27, no. 6, 2022, doi: https://doi.org/10.46743/2160-3715/2022.5460.