to Teaching in a Mobile… Teachers’ Views on the Use of Chatbots to Support English Language Teaching in a Mobile Environment

— The development in machine learning has allowed chatbots to be widely applied into educational settings. However, limited study has investigated teacher’s views on its usage for teaching and learning. This paper reports an exploratory study on English as a Second Language (ESL) teachers’ views with regards to the use of chatbots for their teaching and learning delivery in a mobile environment. Using survey research design, views from 142 ESL teachers were gathered using questionnaires, which consist of Likert-scale items and open-ended questions. The teachers were sampled using purposive sampling method. The items and questions were developed based on the principles of the Community of Inquiry (CoI) framework, which focuses on social, cognitive and teaching presence. Data from the Likert-scaled items were analyzed using descriptive sta-tistics while open-ended questions were coded thematically. The findings showed that teachers perceived the use of chatbots in giving feedback to their students as very helpful though some of them needed extra training on how to use them. They also thought chatbots can simulate an interaction cycle for students to practice the target language. In addition, the teachers believed chatbots augmented a greater level of social presence, which eventually creates an environment for their students to be active. All in all, the findings provided valuable insights on the proper integration of chatbots in teaching and learning while gauging essential affordances and constraints of its use from ESL teachers’ perspective.


Introduction
The proliferation of mobile technologies for teaching and learning, especially in English as a Second Language (ESL), has permitted teachers to support their classroom instructions in many ways. Its application can be identified across all levels of education from kindergarten until higher education [1]. Previous studies in the area of mobile-assisted second language learning [2,3,4,5] have shown how mobile tools can increase opportunities for authentic use of language (e.g., interaction with native speakers, social networking), provide engaging activities that enhance students' understanding (e.g., game-based tools) and allow seamless access to useful materials and resources (e.g., open educational resources). Thus, with appropriate pedagogy, mobile tools are able to extend the learning of English language beyond the class time and not confined to textbooks. As stipulated by Chinnery [6], "the effective use of any tool in language learning requires the thoughtful application of second language pedagogy" (p. 9). Therefore, any excitement in promoting mobile applications for second language learning in terms of its affordances or advantages should be viewed from the pedagogical viewpoint. In addition, Papadakis [7] highlighted the challenge in selecting educational applications is reducing the focus on entertainment value and prioritise educational impact on learners' development. In light of this, teachers' views on how a specific application is used is pivotal as they are at the forefront of decision making and designing learning tasks.
One area in mobile learning that has grown rapidly is the use of conversational agents or chatbots thanks to the advancement in artificial intelligence. A chatbot is a computer programme that converses with people on a certain subject or area in a natural, conversational manner utilising text or speech [8]. Chatbots have been used for a variety of reasons in many areas, including marketing, customer service, technical assistance, education, and training [9]. The origin of chatbots can be traced as far back as the late 1960's when ELIZA was introduced -a simple bot that gives responses based on keywords inputted by users [10]. Now, it is more human-like and accepts more than just text inputs. From virtual assistants (e.g., Siri, Google Assistant) to text-based chatbots that can be installed in chat apps (e.g., Telegram, Discord, Facebook Messenger). Many studies related to chatbots were mainly in terms of its development algorithms [11] or for general educational use [12]. Studies on the use of chatbots in English language learning, however, have focused on three major areas.
The first area is in terms of the general provisions on how chatbots can be integrated in language teaching and learning. Dokukina and Gumanova [13] highlighted the role of chatbots in creating natural language interaction, which allows learners to use the target language in simulated contexts. Wu and Yan [14], on the other hand, proposed several deep-learning mechanisms that could enhance the interactivity of chatbots in order to make them closer to features of human interactions. Despite the initial challenges in integrating chatbots for language learning, Fryer et al. [15] showed optimism that chatbots will begin to dominate language learning especially in informal settings. They argued that the development in machine learning has allowed chatbots to function more effectively for educational purposes. Their empirical study has also indicated language learners' preference to practise language with chatbots as the absence of a human instructor reduced their fear of making mistakes. Echoing the same view, Alm and Nkomo [16] reported that language learners were more willing to engage in conversation with chatbots though they showed frustrations when the conversations did not match their learning goals. However, it is worthy to note that their study was focusing on learners' own initiative to learn the target language through chatbots without teacher's interventions.
The second area of concern is the linguistic accuracy of chatbots. Coniam [17] evaluated the performance of five chatbots for English language learning. Three of the five chatbots had favourable acceptability figures in the range of 90% in terms of grammar but all five struggled in the semantic or pragmatic part in which many meaningless responses were generated. The study reveals the shortcomings in the current implementation of chatbots that are still highly dependent on keyword detection with minimal efforts to consolidate changes contextualized meaning. Das and Kumar [18] also stipulated that it is useful to ensure the accuracy of chatbots in providing information for better user experience. Although chatbots may not be able to converse with human-like competence, its accuracy and consistency in providing word-level drills and practice is undisputed. Vanjani et al. [19], for example, demonstrated how chatbots could be used to learn phrases and vocabulary through a multilingual chatbot that is capable of realtime translation.
The third area, which has been studied is about the effectiveness or usability issues of chatbots. Kim [20] conducted an experimental study on two groups of Korean EFL learners and found that both groups (with and without chatbot intervention) showed significant improvement in listening and reading skills. However, the group with chatbots showed more improvements in the post-listening test. They demonstrated the potential use of Elbot in enhancing learners' engagement on the use of English. Patrovic and Jovanoic [21] shared their review of four chatbots and explained the promising usage of chatbots for personalized language learning. A review by Smutny and Schreiberova [22], on the other hand, outlined the affordances and constraints of chatbots, focusing more on what each bot is able to provide but not specific to English language teaching.
In essence, the review of previous studies shows that very few studies have been done to gather the end-users' opinions of chatbots. In the context of language learning, the key end-users are teachers and students. While there are studies that examined students' perception [23,24], those that focused on teachers remain scarce. Since teachers are the catalyst of any pedagogical intervention, it is pivotal to understand their views on tools that they can use in teaching and learning. The research done by Chocarro et al. [25] focused on the teachers' attitudes towards chatbots from the Technology Acceptance Model (TAM) point of view but were on chatbots not specific to language learning. Nevertheless, they provided key insights on the potential use of chatbots in educational settings as most studies were on the role of chatbots in customer relations and online business platforms.
In addition, the review reveals that it is rather apparent that ESL teachers' adoption of chatbots for English language teaching has not been examined thoroughly. Existing studies were mostly on general usability issues with little emphasis on uncovering ESL teachers' views on its use for English language teaching. It is also important to consolidate teachers' view on the use of chatbots through a theoretical framework related to online learning environment such as Community of Inquiry by Garrison et al. [26] in order to assess its potential as a teaching tool. By using the appropriate framework, future integration of chatbots in English language teaching and learning can be done more systematically. Hence, this study aims to answer the following research questions (RQ): 1. What are the ESL teachers' views on the use of chatbots in English language teaching through a mobile environment? 2. To what extent can chatbots support the Community of Inquiry among the learners?
The following section of this paper describes the theoretical underpinnings that guide this study, particularly with regards to why chatbots are used to support English language teaching in a mobile environment. This is then followed by the explanation of methods used. Next, the results and discussion are provided in accordance with the research questions. This paper ends with the conclusion section that summarises the key results while highlighting their implications and suggestions for future research.

Theoretical underpinning
To guide the investigation on ESL teachers' views on the use of chatbots for language learning, the Community of Inquiry (CoI) framework as proposed by Garrison et al. [26] was used. Being a framework inspired by constructivism, CoI focuses on learning that spawns from experience that is contextualised and socially situated [27]. Furthermore, CoI was formulated with a specific focus on computer-mediate communication (CMC), which is the relevant to this exploratory study as chatbots were used to assist teachers in CMC. Previous studies on CoI have shown its usefulness in assisting researchers to identify affordances and constrains in various CMC platforms such as discussion forums and chat channels [28,29]. Qin et al. [30] also demonstrated how a chatbot was developed using CoI and showed that features within the chatbot can address the presences that CoI aims to create in a learning environment. Nevertheless, their study was on students' perception and regarding a general topic, which is not related to language learning. Thus, there is a gap in further investigating how a mobile learning environment created based on chatbots can be helpful in creating all the presences within the CoI framework. Specifically, the emphasis in this study is placed on how chatbots (regarded as a virtual human-like assistant) can simulate a learning environment that fosters social, cognitive and teaching presences. As explained in [26], social presence covers the affective part of the learning environment as to whether teacher and students are able to build a trusting environment that fosters interpersonal relationships. Cognitive presence, on the other hand, refers to the ability of the learners to construct and confirming meaning by constant reflection or engaging in peer-to-peer discourse. In other words, cognitive presence is related to cognitive engagement in which students are being guided into deep learning rather than merely accepting what is conveyed. Lastly, teaching presence refers to the facilitation, direction of cognitive and social processes that lead to intended learning outcomes.
This framework is chosen due to its relevance in online learning, particularly in asynchronous contexts. Since chatbots, in this study, were used as supplementary activities or tasks in a mobile learning environment, there is a tendency for the chatbots to be used in asynchronous mode, much like a discussion forum. Although chatbots can offer spontaneous feedback, learners may take time to respond since the responses are textbased, are not being put under pressure to respond immediately. Therefore, it gives a practical sense to investigate the chatbot usage among ESL teachers using this CoI framework.

Methods
A survey research design was used as the purpose of this exploratory study is to identify the views of ESL teachers on the use of chatbots to support their teaching. Online questionnaires with 5-point Likert-scale items (1 being strongly disagree, 5 being strongly agree) and open-ended questions were constructed based on the selected CoI framework. The questionnaire contains three main parts.
The first part is on the participants' general info such as gender and teaching experience. The second part contains 10 Likert-scale items and 1 multiple-choice question. Ten Likert-scale items for the questionnaire were derived based on three main constructs namely language learning, feedback to learners and general usability. The items for language learning (n=4) aim to identify teachers' views on the potential use of chatbots in assisting their students to learn the target language through the simulated social, cognitive and teaching presences as indicated in CoI. The items for feedback to learners (n=3) are to check teacher's opinions on the chatbots' ability to provide real-time feedback especially during their absence. Feedback is seen as crucial in creating an environment that supports cognitive and teaching presences as it can facilitate discourse and assist learners to move from triggering events to at least exploration stage. The usability items (n-3) are to find out teachers' views on the convenience of using the chatbots. The last question in the second part is teachers' overall view on the affordance of chatbots in addressing the three presences in CoI. The third part of the questionnaire contains open-ended questions on suggestions for the integration of chatbots in English language teaching based on the CoI presences as well.

Sample
Through purposive sampling, invitations to ESL secondary school teachers in Malaysia were sent out via email and chat groups. 154 ESL teachers signed up for the study but only 142 of them managed to provide complete responses to the survey, giving a response rate of about 92%. The gender distribution is almost equal. The criteria set for the sampling ensured that they are all secondary school teachers with at least three years of teaching experience and are familiar with the use of mobile tools for learning. The primary reason for these criteria is to make sure the teachers were comfortable in using technology and had sufficient experience in lesson design. Also, in the context of the study, the use of mobile tools at the secondary school level is permitted and this makes it easier to manage than primary school students due to students' level of maturity. Table 1 shows the demographic information of the selected sample.

Data collection procedures
Prior to data collection, the teachers who signed up for the study were specifically briefed on the three core components of CoI so that they were aware of the key elements in each presence. The briefing was done via video-conference platform (Google Meet) in several sessions according to the teachers' choice of slots. Table 2 shows examples of chatbot prompts for each corresponding component of CoI, which were explained to teachers. This briefing is important in ensuring the data gathered from the questionnaire are valid and reliable since all teachers must be able to differentiate the CoI presences. In collecting the data, teachers were required to use chatbots as part of their mobile learning activities for about two months. Prior to designing the mobile learning tasks, the teachers were also required to use the chatbots in order to identify the features available. Some example tasks were also given to teachers as guidance. For example, students can be told to complete a portfolio of weekly logs with the chatbots on several topics. The two chatbots selected for this study are shown in Fig 2. These chatbots can be integrated into Telegram and Facebook Messenger and are related to English language learning. As the teachers have been using Telegram to manage their classroom, it makes it more convenient for teachers to directly introduce them to the students without the need for extra installation. After two months, the teachers were required to complete the survey questionnaire, which was administered online for a period of one week.

Data analysis procedures
The gathered data from the Likert-scale items were analysed using descriptive statistics in which mean and standard deviation values were computed and tabulated. Since this was an exploratory study, only ten key items were used and analysed. As for the item on teachers' overall view of CoI presences, it was quantified using frequency count and percentages as teachers can select more than one presence. For the open-ended responses on suggestions for integration and improvements, the gathered data were analysed using thematic analysis where each response was read and coded according to major themes.

Results and discussion
This section presents the results obtained from the study based on the research questions. The results are also discussed in relation to pertinent literature.
1. What are the ESL teachers' views on the use of chatbots in English language teaching through a mobile environment? 2. To what extent can chatbots support the Community of Inquiry among the learners? Table 3 shows the overall means score of items concerning ESL teachers' views on the use of chatbots (RQ1) after integrating them for about two months. The ESL teachers were positive about the use of chatbots in their teaching but remained reserved on chatbots' accuracy as shown in the lower mean score for grammar. On the whole, the items were rated highly with strong agreement on the usefulness of chatbots in simulating authentic language use (mean=4.13) and model of good use of words and phrases (mean=4.03) as well as grammar (mean=3.67). The simulation of authentic language use is largely due to the prevalent social presence when chatbots are used. Since chatbots are "human-like", the conversations are often filled with social cues and emotions (e.g., the use of emoji). This type of environment encourages learners to be more comfortable and more willing to communicate as indicated in the study by De Cicco et al. [31].

Teachers' views on chatbot integration
It is also important to note that despite chatbots being able to respond immediately (mean=3.92), the teachers found that some of the corrective feedbacks were not meaningful. This is reflected in their low mean score (mean=2.92) for the items on correcting students' mistakes. Despite this problem, the teachers seem to agree that the chatbots could build cognitive presence as students were able to learn some good usage of English on their own and perform necessary evaluation on the responses provided by the chatbots (exploration and integration in cognitive presence). This finding reflects the concerns raised by Coniam [17] and Das and Kumar [18] in which the accuracy of the chatbots could be the factor that hinder its continuous use in teaching and learning. However, as stipulated by Qin et al. [30], this problem can be overcome when teachers learn to design their own chatbots and use accurate language input to guide learners.
In terms of usability, the teachers also believed that the chatbots are easy to use (mean=4.34) and they had no difficulty introducing them to the students during their teaching and learning activities. One reason for this could be the selected chatbots in this study were capable of being embedded within existing chat applications (i.e., Telegram and FB Messenger) and do not require complex installation. These findings echoed what was found in [13] as teachers explore the use of chatbots to assist them in providing more opportunities for language use though improvements are necessary to increase the chatbots' accuracy.

Teachers' views on the role of chatbots in supporting presences in CoI
The teachers were also asked to indicate their views on the three presences (RQ2) explained to them, as to whether chatbots can simulate a learning environment that contains those presences. Table 4 shows their overall view. Clearly, social presence is regarded to be dominant while cognitive presence is lacking since the chatbots were not intelligent enough to prompt the students to engage in deeper reflection. As found in the study by Huang et al. [32], social presence was highly rated among their participants as the chatbots were able to interact with them without fail though there were instances when they did not understand what the bots were saying. It was still capable of creating a virtual setting for interpersonal interactions that allowed learners to practise using the language.
It is interesting to note that teachers accepted the role of chatbots in creating teaching presence particularly in situations where they could not make themselves available. For example, when students are at home, it would hard for teachers to monitor them progressively especially when the students would like to ask questions pertaining to word usage. It can be assumed that chatbots can act as an assistant to teachers in dealing with trivial questions. The teachers can focus more on important corrective feedback to the learners. As reviewed by Smutny and Schreiberova [22], most chatbots are still not able to engage in meaningful conversation beyond the keywords set for each bot. Therefore, teacher intervention is still needed in ensuring the learners are getting accurate feedback.

Teachers' suggestions for improvement
Additionally, to supplement the results for RQ2, the teachers were asked to provide suggestions on the kind of improvements that can be done on the chatbots in accordance to the three presences in CoI. Three recurring themes emerged as shown in Table 5.

Improvements in terms of artificial intelligence
The chatbots need to improve its responses. It seems not so intelligent.
Cognitive I find the chatbots rather mechanistic after a few rounds of usage. My students complained the same. Need to improve this to make social presence seems more natural.

Accuracy of the corrective feedback
It is very useful since students can learn the language rather authentically, but the accuracy can be improved.

Teaching & Cognitive
My students love it since they don't have to ask me regularly. But some responses are not accurate. I think maybe need to check on this.

Cognitive Collaboration
It would be good if the chatbots can also support group chats. This would be useful for discussion-like tasks.

Social
My students wanted to work together, so I had to design a task where they get to discuss but respond to the chatbots together. Can add this feature.

Social Lesson Design
Since the tasks were mainly on students talking to chatbots, I think if we can design other types of activities would be interesting

Teaching
Cognitive wise, it reaches maybe up to Comprehension in Bloom's Taxonomy or exploration stage in the framework you mentioned. But I like how chatbots become like a social partner encouraging my students to use the language more. Pretty brilliant.

Cognitive & Social
In general, the teachers believed that the responses provided by the chatbots could be made more intelligent so that it covers a more natural discourse that can be used to encourage continuous conversations. The issue of accuracy in guiding the learners is also prevalent in which the bots are designed to deal with keyword-based responses [30]. Some of the responses were not meaningful and caused confusion among the learners. This seems to affect the teaching and cognitive presences. There are also suggestions for chatbots to include a collaborative feature whereby group conversations can be made as this feature is capable of alleviating social presence. The feature would also be helpful in the ESL context since many task-based activities require group discussions. Moreover, the teachers provided suggestions in terms of lesson design. They felt that chatbots can be better integrated into English language teaching if a variety of tasks could be designed. They value the social presence that chatbots create but believe that it would be beneficial in simulating higher cognitive presence. This suggestion is in line with the study done by Go and Sundar [33] whereby human-like chatbots would increase the level of interactivity. Nevertheless, the overall feedback from the openended questions still indicates a strong acceptance among the teachers with regards to chatbots being their assistant in guiding the learners. Since one of the challenges in ESL context is to encourage learners to use the target language, chatbots are seen as a good tool to support teachers. Learners can engage in conversations that highlight how specific words or phrases are used in a friendlier mode.

Conclusion
The exploratory study has provided insights into ESL teachers' views related to the use of chatbots for English language teaching in a mobile learning environment. Similar to studies in mobile learning [34,35,36,40], the findings from this study have shown ESL teachers' willingness to adopt mobile tools to enhance the learning experience. There is a huge potential of using chatbots to encourage ESL students to actively use the target language as ESL learners tend to lack the chance to use them at home [37]. The findings serve as a foundation on how to integrate the use of chatbots in English language teaching by taking into consideration the views of ESL teachers. They have shown interest in continuing the use of chatbots in designing their teaching and learning activities despite the problem with the accuracy in the corrective feedback provided by the chatbots.
This study has contributed to the related body of research pertaining to the Community of Inquiry framework by examining its potential uses in chatbot interactions within the context of English language teaching in a mobile environment. As reported in previous studies [28,29], the CoI framework is commonly applied for discussion forums or asynchronous CMC but in this study, it was used to investigate the conversations between human and non-human virtual agents. From the methodological perspective, this study has shown that conversations that take place via chatbots can be examined by the CoI framework in order to uncover the levels of social, cognitive and teaching presence. The results have indicated how chatbots could play a crucial role in increasing social and teaching presences. In relation to teaching presence, the chatbots have served as a facilitator or teaching assistance to guide the students. It creates an environment where students feel that they are being scaffolded in learning the target language. With regards to social presence, chatbots are capable of simulating human-like social interactions, which make the students feel at ease during the learning process. However, it is important to note that although there are constraints in encouraging its use to simulate cognitive presence, the teachers still view chatbots to be useful in motivating the students to be more actively involved in knowledge exploration and integration. For example, whenever they encounter feedbacks which are inaccurate, they would cross check and reflect on those errors. Such reflective activity is an example of chatbot's capability in creating cognitive presence [26]. Interestingly, ESL teachers regard chatbots' social presence can build students' confidence in practising the correct usage of the English language though the cognitive presence is minimal. As shown in this study, ELS teachers have mentioned how students were more proactive and confident in using English in the conversations with chatbots. Thomas [38] reported a similar trend in his study on EFL learners in which there was a weak positive relationship between critical thinking and achievement emotions. As such, despite the lack of cognitive presence, social presence can be useful in motivating learners to perform better in a specific academic task.
In the aspect of practical implications, chatbot developers could take note of the areas for improvements as suggested by the teachers. One suggestion that was repetitive is in enhancing chatbots' ability to provide meaningful feedback with minimal intervention from the teachers. Since chatbots are playing the role of a facilitator, the accuracy of their feedback would avoid students from learning the wrong input. In addition, teachers and instructional designers could also identify elements within the CoI framework that should be given more emphasis when integrating chatbots into their ESL lesson designs. As emphasised by Mahzan et al. [39], understanding ways to accommodate cognitive and emotional needs of ESL learners is useful in boosting learning engagement.
Although this study is exploratory in nature, it has uncovered some areas for further investigation. Firstly, the present study did not measure students' performance in language gain or production as the aim was to investigate ESL teachers' views on the use of chatbots in supporting their lessons. Thus, it would be beneficial for future research to measure ESL students' performance and correlate the data with the teachers' use of chatbots in the language learning activities by establishing the connection with the CoI framework. It would also be beneficial to include variables such as teachers' and students' technological readiness in the experiments. Secondly, this study is limited to only two text-based chatbots as they are more convenient to be integrated into the existing chat apps that the participants were using. It has confined the scope of language learning mainly to reading and writing skills. Thus, future research can include voicebased chatbots for comparative analysis as the advanced chatbots could engage in audio conversations that could be used for students to practise listening and speaking skills. A comparative study between text-based and voice-based chatbots can potentially yield interesting results in informing their effectiveness for language learning especially in ESL contexts. With these suggestions, it is hoped that future studies within the same scope can further inform the use of chatbots for language learning purposes.