A Scenario of the Formative E-assessment Based on the ARCS Model: What Is the Impact on Student Motivation in Educational Context?

halimalajane@gmail.com Abstract — The purpose of this research is to measure the motivation of nursing students, option: multi-skilled nurse (n=58), following their participation in an e-learning activity in the form of formative quizzes. The pedagogical scenario was based on Keller's ARCS motivation model. Data were collected through the IMMS questionnaire adopted from the ARCS model. The results show that the students were motivated by the e-learning activity (mean score = 4.22±, 51091), and that the motivation variables studied correlated positively with the overall motivation score (p-value < .001). Through this research, it was found that an instructional scenario based on the ARCS model had a positive impact on students' motivation for e-learning, thus encouraging allied health educators to design educational tools that can motivate nursing


Introduction
One of the fundamental activities that teachers use when planning courses is defining forms of evaluation, which includes formative assessment [1,2]. Through formative assessment, teachers review learners' understanding and obtain data on learning achieved for formative feedback [3,4]. In recent years, various studies have prompted changes in the way formative assessment is designed in the classroom to include digital assessment [5,6,7]. Moreover, technology-enabled formative assessment has demonstrated its significant pedagogical potential, particularly in terms of performance, commitment, and motivation to learn [8,9,10,11].
Although the literature indicates that the use of digital technology generates levels of student motivation [12,13,14], several questions remain about the nature of the implementation of distance learning materials and regarding how teachers can design tools to interest students in the learning process and maintain their motivation [15,16]. These topics are critical, as motivation remains one of the key factors that determine student learning outcomes [17].

ARCS model of motivational design (Keller)
There are many theories of motivation for learning, and each theory provides teachers with the basis for building student-accepted teaching tools [18]. One such theory is the ARCS model of John Keller's motivational design theories [19]. This theory presents four steps to promote student motivation in distance learning activities: attention, relevance, confidence and satisfaction.
The bases and principles of the ARCS model [20] can be summarized as follows: ─ Attention: in this stage, it is necessary to arouse the learners' interest and stimulate their curiosity through a variety of methods when presenting the teaching material to encourage their active participation. This might include activities such as roleplaying, quizzes, and other methods. Visual stimuli and humor can also be used to capture attention during training. ─ Relevance: in this stage, the aim is to design educational content that meets the students' learning needs. The teacher should show them how useful the material is and demonstrate how it will be used to develop their knowledge and skills. ─ Confidence: in this stage, students must feel able to succeed in their training and understand their chances of success. Thus, learners must be aware of performance requirements and evaluation criteria. ─ Satisfaction: in this stage, the learner is motivated to learn by appreciating their results. Learning should be made rewarding by providing feedback and reinforcement.
According to this model (figure 1), learners need a balance between all four principles (attention, relevance, confidence, and satisfaction) to promote and maintain their motivation to complete training or e-learning.
The objective of this work is, therefore, to study the motivational reactions of nursing students to a distance formative assessment activity based on the ARCS model, as part of the teaching of a surgical nursing course. More specifically, two research questions were asked: ─ What are the motivational responses of nursing students to the online educational material? ─ Which aspect(s) of the ARCS model most influence student motivation? The purpose of this study was to evaluate students' motivation after their participation in an e-learning activity. The learning activity used in this study was a series of formative quizzes built on a learning platform. The chosen setting was in a theoretical course entitled "Surgical Nursing." The design of the online quizzes met the principles of the ARCS model of motivation and respected a set of essential points presented in the table 1.  • Feedback provided to give students the confidence to solve the online quizzes

Satisfaction
• Towards the end of each quiz, the student obtains his or her score • Ability to view both incorrect and correct answers • A message is provided to the students according to the score obtained, e.g., Bravo, good work

The participants
The participants in this study were multi-skilled nursing students in their second year of initial training, enrolled in the professional license cycle (n = 58) at the Higher Institute of Nursing and Health Techniques of Casablanca. In 2020, these students participated in an e-learning activity.

Measuring instrument
To meet our research objective, we used Keller's IMMS questionnaire [21]. This questionnaire was developed as part of the ARCS model to measure learners' levels of motivation [19] regarding distance learning activities. The questionnaire consists of 36 questions with 5-point Likert scale items, and each group of questions is linked to a factor in the ARCS model. Here is the guide to the IMMS questionnaire (table 2).

Data analysis
The data collected were analyzed using SPSS version 20 software. The internal consistency of the questionnaire was measured by calculating the Cronbach's α. The results are presented as a mean and standard deviation to assess student motivation levels for each dimension of the questionnaire. A Pearson test was conducted to identify correlations between motivational factors and the overall motivation score. We also determined the correlation between the four factors.
An ANOVA test was carried out to discern whether there were differences in motivation levels between students based on their demographic characteristics.

Internal consistency of the IMMS questionnaire
The internal consistency of the questionnaire was calculated by measuring the Cronbach's Alpha, found equal to 0.93. Therefore, we determined that the questionnaire has a good internal consistency by comparing it with Keller's original questionnaire, which had a Cronbach's Alpha of 0.96.

Motivation element Questions
Attention (12 questions Table 3. The internal consistency of the questionnaire

Motivation scores/ levels of motivation
According to Table 4, the minimum level of motivation was 3.11 and the maximum level of motivation was 4.92, with an average score of 4.22, which is considered very positive. In addition, the data in Table 5 show that, among the 30 students who responded to the questionnaire, 21 (70%) recorded a high level of motivation (between 4 and 5) as a result of their participation in the online learning activity, and 9 (30%) recorded an average level of motivation (between 3 and 4). This suggests that the students were motivated by the numerical formative assessment.  According to the ARC model of motivation, attention is the first element that students must have during a learning activity involving technology. In this research, the "Attention" variable had a average score of 4.48 with a minimum score of 4.20 (items 2 and 7) and a maximum score of 4.73 (item 8), showing that the Online Quizzes were positively engaging for the students and were catchy rather than boring.
The 2nd element of the model is the relevance of the tool and the exercises proposed. The average score obtained was 3.93, with a minimum score of 3.8 (item 21) and a maximum score of 4.27 (item 14). This means that students believed the online quizzes to be relevant. In the Confidence variable, the average score obtained was 3.93 (minimum score = 3.8 and a maximum score = 4.27) showing that the quizzes were not difficult, which gave the students confidence that they could solve them.
In the last element of the model, satisfaction occupies an important place. In our study, the students were satisfied with the Online Quizzes and would like them to be included in other courses (average score = 4.36; minimum score = 4.27; maximum score= 4.53).

Score Ranges
Motivation category Frequency N = 30 Percentage

Correlation between motivation and ARCS variables
To determine which of the four factors in the ARCS model (attention, confidence, relevance, and satisfaction) most influenced nursing students' motivation towards the online educational activity, we analyzed the data with a Pearson's correlation test, which revealed a correlation between the four factors and motivation (p-value < .001).
To assess whether there were differences in motivation levels among participants based on their characteristics (age, gender, baccalaureate track, and previous year's grade), an ANOVA test was performed. The results of this test did not signify a significant difference between all factors.
In addition to the data presented, an examination of the correlation between the four motivation variables indicated that they correlate with each other quite significantly (pvalue < .001).

Discussion
Measuring motivation is essential to evaluate the effectiveness of an instructional scenario. The goal of this research was to study nursing students' motivation towards an online learning activity (formative quizzes) based on the ARCS model of motivation. According to the model, the scores obtained through the distributed questionnaire demonstrate positive motivation following the students' participation in a distance learning activity. The average motivation score was 4.22 and 70% of the participants recorded high motivation (between 4 and 5). All the variables studied correlated positively with the total motivation score.
Our results corroborate those of other similar studies; particularly research conducted using the ARCS model of motivation. For example, David et al. (2009) [22] reported that pedagogical changes based on the ARCS model can enhance motivation and improve learning and that trainers can apply the model effectively. Similarly, another study conducted by Huang (2016) [23] to measure participant motivation in the mass implementation of open online courses (MOOCs) using the IMMS questionnaire found that the motivation level of most participants was positive and that they were satisfied with the pedagogical materials provided in the MOOCs. However, learners also anticipated improvements in some areas.
More recently, Kimberly (2019) [24] concluded that learning and assessment strategies based on the ARCS model can be effectively used to improve and maintain participant motivation in a pharmacy elective. In summary, other studies in the literature have highlighted the value of applying the ARCS model based on the four dimensions of motivation (attention, confidence, relevance, and satisfaction) in the design and development of educational tools. This provides a method for teachers, particularly in the nursing sciences, to adopt effective teaching and learning strategies regarding engagement and motivation.

Conclusion
Based on the data from this study, although conducted with a small sample (N=58), a pedagogical scenario based on the ARCS model can stimulate students' motivation to follow an e-learning activity.
In fact, the overall results showed a satisfactory motivation score of n = 4.22, and the students enjoyed this learning activity enough to want to repeat it in other courses. This finding provides nursing educators with avenues for developing educational materials that motivate learning and breaks with traditional models of formative evaluation that have historically been practiced in paper-and-pencil format.