Developing a Chatbot for Soft Skills Education: Comparing ReLU-LSTM and GloAT-Transformer Models

Chaimaa Bouafoud; Abdellah Madani; Khalid Zine-Dine

doi:10.3991/ijim.v20i17.62615

Authors

Chaimaa Bouafoud LAROSERI Laboratory, Chouaïb Doukkali University, El Jadida, Morocco https://orcid.org/0009-0003-0686-4838
Abdellah Madani LAROSERI Laboratory, Chouaïb Doukkali University, El Jadida, Morocco
Khalid Zine-Dine Faculty of Sciences Rabat (FSR), Mohammed V University, Rabat, Morocco https://orcid.org/0000-0003-0395-0805

DOI:

https://doi.org/10.3991/ijim.v20i17.62615

Keywords:

Artificial Intelligence, Deep Learning, NLP, Chatbots, Education, Soft Skills, Classification Intent, Neural Network, LSTM, Transformers

Abstract

Recently, soft skills, like communication, teamwork, and problem-solving, have become more important in higher education. In Moroccan universities, fostering these skills is essential to better prepare students for the demands of the job market and modern work environments. However, traditional educational methods often lack interactive and personalized approaches to develop these competencies. This led us to develop a conversational AI system to support soft skills learning through intent classification using Natural Language Processing (NLP) and Neural Networks. Two deep learning models were developed and evaluated; a ReLU-Enhanced LSTM, optimized for stable sequential processing, and a GloAT-Transformer model leveraging global self-attention to capture contextual relationships.

Author Biographies

Chaimaa Bouafoud, LAROSERI Laboratory, Chouaïb Doukkali University, El Jadida, Morocco

Chaimaa BOUAFOUD is a doctoral researcher and teaching assistant (monitrice) at the Faculty of Sciences, Chouaïb Doukkali University, El Jadida, Morocco. She is actively engaged in research at the intersection of artificial intelligence, educational technologies, and data science. Her work focuses on developing intelligent systems and adaptive learning models that enhance student engagement and performance in higher education. She has contributed to several research projects and academic events, including organizing activities for new university students. Her current research explores the integration of AI-driven adaptive learning platforms, such as chatbots and recommendation systems, within the Moroccan educational context. She can be contacted at email: bouafoud.chaimaa@ucd.ac.ma.

Abdellah Madani, LAROSERI Laboratory, Chouaïb Doukkali University, El Jadida, Morocco

Pr. Abdellah Madani is a lecturer and researcher at the Department of Computer Science, Chouaïb Doukkali University, El-Jadida, Morocco. He serves as the Director and an active member of the LAROSERI research laboratory, where he contributes to scientific leadership, research development, and collaborative academic initiatives. His research interests include artificial intelligence, machine learning, natural language processing, and cybersecurity, with a particular focus on intelligent systems and real-world applications. He has authored numerous scientific publications in national and international venues and actively participates in academic committees and scientific events. Dr. Madani collaborates with research teams from various institutions, supervises postgraduate students, and plays an important role in advancing research and innovation activities within his university. He can be contacted at email: madani.a@ucd.ac.ma.

Khalid Zine-Dine, Faculty of Sciences Rabat (FSR), Mohammed V University, Rabat, Morocco

Pr. Khalid ZINE-DINE obtained his Ph.D. in 2000 from Mohammed V University. He has over 24 years of combined academic and professional experience in the field of computer systems/networks and cybersecurity. Prior to his academic tenure, he served for four years as a Network and System Security Project Manager in the banking sector, where he led major initiatives to secure critical information systems and IT infrastructure. Currently, Prof. ZINE-DINE serves as a Full Professor at the Faculty of Sciences – Mohammed V University in Rabat. His research interests cover a broad spectrum, including System and network security, Network architectures and communication protocols, Wireless ad hoc and sensor networks, Cloud computing, Smart grids and AI applied to cybersecurity. Prof. ZINE-DINE is actively involved in Ph.D. supervision, mentoring several doctoral candidates, and contributes to both national and international research projects. He also serves regularly as a co-organizer and co-chair of scientific conferences and workshops, playing a key role in advancing and disseminating research in his fields of expertise. He can be contacted at email: k.zinedine@um5r.ac.ma.

References

[1] Mónika-Anetta Alt, Ibolya Vizeli, and Zsuzsa Săplăcan. “Banking with a chatbot–A study on technology acceptance”. In: Studia Universitatis Babes Bolyai Oeconomica 66.1 (2021), pp. 13–35.

[2] Ryan Jackson. “Understanding (and using) ChatGPT in banking”. In: American Bankers Association. ABA Banking Journal 115.3 (2023), pp. 16 17.

[3] Michael R King and ChatGPT. “A conversation on artificial intelligence, chatbots, and plagiarism in higher education”. In: Cellular and molecular bioengineering 16.1 (2023), pp. 1–2.

[4] Seonghun Kim et al. “Why and what to teach: AI curriculum for elementary school”. In: proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. 17. 2021, pp. 15569–15576.

[5] Alpay Sabuncuoglu. “Designing one year curriculum to teach artificial intelligence for middle school”. In: Proceedings of the 2020 ACM conference on innovation and technology in computer science education. 2020, pp. 96–102.

[6] Tingchen Fu et al. “Learning towards conversational AI: A survey”. In: AI Open 3 (2022), pp. 14 28.

[7] Chongyang Tao et al. “A Survey on Response Selection for Retrieval-based Dialogues.” In: IJCAI. 2021, pp. 4619–4626.

[8] Anh D Tran, Jason I Pallant, and Lester W Johnson. “Exploring the impact of chatbots on consumer sentiment and expectations in retail”. In: Journal of Retailing and Consumer Services 63 (2021), p. 102718.

[9] Andrej Miklosik, Nina Evans, and Athar Mahmood Ahmed Qureshi. “The use of chatbots in digital business transformation: A systematic literature review”. In: IEEE Access 9 (2021), pp. 106530 106539.

[10] Chinedu Wilfred Okonkwo and Abejide Ade Ibijola. “Chatbots applications in education: A systematic review”. In: Computers and Education: Artificial Intelligence 2 (2021), p. 100033.

[11] Emmanuel Mogaji et al. “Emerging-market consumers’ interactions with banking chatbots”. In: Telematics and Informatics 65 (2021), p.

[12] Soufyane Ayanouz, Boudhir Anouar Abdelhakim, and Mohammed Benhmed. “A smart chatbot architecture based NLP and machine learning for health care assistance”. In: Proceedings of the 3rd international conference on networking, information systems & security. 2020, pp. 1–6.

[13] Eleni Adamopoulou and Lefteris Moussiades. “Chatbots: History, technology, and applications”. In: Machine Learning with Applications 2 (2020), p. 100006.

[14] Zhenhui Peng and Xiaojuan Ma. “A survey on construction and enhancement methods in service chatbots design”. In: CCF Transactions on Pervasive Computing and Interaction 1 (2019), pp. 204 223.

[15] Rui Yan, Yiping Song, and Hua Wu. “Learning to respond with deep neural networks for retrieval-based human-computer conversation system”. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 2016, pp. 55–64.

[16] Xiangyang Zhou et al. “Multi-turn response selection for chatbots with deep attention matching network”. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018, pp. 1118–1127.

[17] Chang Shu et al. “Open Domain Response Generation Guided by Retrieved Conversations”. In: IEEE Access (2022).

[18] Lantao Yu et al. “Seqgan: Sequence generative adversarial nets with policy gradient”. In: Proceedings of the AAAI conference on artificial intelligence. Vol. 31. 1. 2017.

[19] Yi-Lin Tuan and Hung-Yi Lee. “Improving conditional sequence generative adversarial networks by stepwise evaluation”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 27.4 (2019), pp. 788–798.

[20] Nura Esfandiari, Kourosh Kiani, and Razieh Rastgoo. “A conditional generative chatbot using transformer model”. In: arXiv preprint arXiv:2306.02074 (2023).

[21] Quan Thanh Tho. “N/A Modern Approaches in Natural Language Processing”. In: VNU Journal of Science: Computer Science and Communication Engineering 39.1 (2022).

[22] Ehsan Fathi and Babak Maleki Shoja. “Deep neural networks for natural language processing”. In: Handbook of statistics. Vol. 38. Elsevier, 2018, pp. 229–316.

[23] Zhouhan Lin. “Deep neural networks for natural language processing and its acceleration”. In: (2020).

[24] S Hochreiter. “Long Short-term Memory”. In: Neural Computation MIT-Press (1997).

[25] Murtaza Roondiwala, Harshal Patel, Shraddha Varma, et al. “Predicting stock prices using LSTM”. In: International Journal of Science and Research (IJSR) 6.4 (2017), pp. 1754–1756.

[26] Jian Cao, Zhi Li, and Jian Li. “Financial time series forecasting model based on CEEMDAN and LSTM”. In: Physica A: Statistical mechanics and its applications 519 (2019), pp. 127–139.

[27] Wei Bao, Jun Yue, and Yulei Rao. “A deep learning framework for financial time series using stacked autoencoders and long-short term memory”. In: PloS one 12.7 (2017), e0180944.

[28] Thomas Fischer and Christopher Krauss. “Deep learning with long short-term memory networks for financial market predictions”. In: European journal of operational research 270.2 (2018), pp. 654–669.

[29] Sima Siami-Namini, Neda Tavakoli, and Akbar Siami Namin. “A comparative analysis of forecasting financial time series using arima, lstm, and bilstm”. In: arXiv preprint arXiv:1911.09512 (2019).

[30] Eliyahu Kiperwasser and Yoav Goldberg. “Simple and accurate dependency parsing using bidirectional LSTM feature representations”. In: Trans actions of the Association for Computational Linguistics 4 (2016), pp. 313–327.

[31] Ashish Vaswani et al. “Attention is all you need”. In: Advances in neural information processing systems 30 (2017).

[32] Alex Wang et al. “GLUE: A multi-task benchmark and analysis platform for natural language understanding”. In: arXiv preprint arXiv:1804.07461 (2018).

[33] Tom Brown et al. “Language Models are Few-Shot Learners”. In: Advances in Neural Information Processing Systems. Ed. by H. Larochelle et al. Vol. 33. Curran Associates, Inc., 2020, pp. 1877–1901.

[34] Victor Sanh et al. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. 2020.

[35] Ahriz, S., Gharbaoui, H., Benmoussa, N., Chahid, A., & Mansouri, K. (2024). Enhancing information technology governance in universities: A smart chatbot system based on information technology infrastructure library. Engineering, Technology & Applied Science Research, 14(6), 17876-17882.

[36] Darmawan, I., Elmunsyah, H., & Prasetya, D. D. (2025). ALBERTIR: A BERT-Based Pretraining for Indonesian Religious Texts Using Qur'an and Hadith Translations. Engineering, Technology & Applied Science Research, 15(5), 28307-28312.

[37] Raza, A., Latif, M., Farooq, M. U., Baig, M. A., & Akhtar, M. A. (2023). Enabling context-based AI in chatbots for conveying personalized interdisciplinary knowledge to users. Engineering, Technology & Applied Science Research, 13(6), 12231-12236.

[38] Ministère de l'Enseignement Supérieur, de la Recherche Scientifique et de l'Innovation. "PACTE ESRI 2030: Plan National d'Accélération de la Transformation de l'Écosystème de l'Enseignement Supérieur, de la Recherche Scientifique et de l'Innovation." (2023).