Design and Evaluation of an Augmented Reality and Generative AI-Driven English Speaking Training System - Wu, Jingfang; Shen, Yueying (International Journal of Interactive Mobile Technologies (iJIM), 2025)
This paper addresses the demand for English speaking training that integrates augmented reality (AR) with generative artificial intelligence (AI). We design and implement a system that supports highly real-time and robust speech interaction. At its core, the system employs mobile-oriented streaming speech recognition, enhanced with a Conformer encoder using causal convolution, a hybrid Connectionist Temporal Classification (CTC)/Attention decoding mechanism, dynamic block-wise streaming training, dynamic signal-to-noise ratio (SNR) data augmentation, and a multi-speaker speech filtering model. These techniques effectively address challenges in AR environments, such as complex noise, heterogeneous devices, and multi-speaker interference, thereby significantly improving recognition accuracy and response speed. Building on this foundation, we develop an English speaking training application that integrates AR scenarios with a generative AI dialogue engine, offering an immersive and adaptive practice environment. Experimental results demonstrate that the system substantially reduces word error rates in noisy and multi-speaker conditions, achieves latency suitable for real-time interaction, and provides a positive user experience, validating both its technical effectiveness and application feasibility.