Exploring Factors and Indicators for Measuring Students’ Performance in Moodle Learning Environment

—One of the most important pillars of smart cities is the smart learning environment. This environment should be well prepared and managed to improve the instruction process for instructors from one side and the learning process for students from the other side. This paper presents the student’s En-gagement, Behavior and Personality (EBP) predictive model. This model uses Moodle log data to investigate the influence and the effect of the students’ EBP factors on their performance. For this purpose, this paper uses the data log files of the "Search Strategies on the Internet" online course in Fall 2019 at Sultan Qaboos University (SQU) extracted from Moodle database. The intention of conducting this kind of experiments is of three-facets: 1. to assist in gaining a holistic understanding of online learning environments by focusing on student EBP and performance within the course activities, 2. to explore whether the student’s EBP can be considered as indicators for predicting student’s performance in online courses, and 3. to support instructors with insights to develop better learning strategies and tailor instructions for personal learning of individual students. Moreover, this paper takes a step forward in identifying effective methods to measure student’s EBP during the learning process. This may contribute to proposing a framework for the smart learning behavior environment that would guide the instructors to observe students’ performance in a more creative way. All the 38 students who participated in this experiment had compatible statistics and results as the relationship between their Engagement, Behavior, Personality was symmetric with their Performance. This relationship was presented using a group of condition rules (If-then). The extracted rules gave us a straightforward and visual picture of the relationship between the factors mentioned in this paper


Introduction
Successful learning involves motivation for the students to achieve the learning objectives they want [1]. Not all students, however, may establish a successful direction that is useful for learning on their own [2]. In the meantime, if a specific student does not like or feel inspired while learning anything, the learning outcomes may not be similar to the desired fulfillment. Through the provision of information on engagement, behaviors of learning, personality and performance of every student would assist the instructors in adjusting instruction techniques and taking any necessary precautions to enhance learning environments. Essentially, making learning smart for the student is the primary objective of smart cities. Numerous researchers have conducted their research in the education and computer science fields. There is no doubt that students are considered as a significant factor in all processes of learning. However, even with many advantages of using Moodle, there are still some factors that need to be weighed in order to ensure their effective implementation [3]. For online and smart learning, there is an increase in the use of the Moodle platform. In order to test the course material, students interact with the Moodle and thus, students generate a large amount of data through their interaction with Moodle [4]. In the current Moodle environment, no enough attention given to the student's engagement, behavior and personality, together with their performance. Some studies were discussed one aspect of these factors, such as student behavior and disregard other aspects and vice versa (see section 3).
This paper presents part of an ongoing research based on examining the engagement, behavior and personality of students in an online learning environment. The paper looks at the potential relationship between the students' EBP against the course performance of the student. This paper is structured as follows: Section 2 gives an overview background about the main concepts used in this paper. Section 3 presents the literature review. Section 4 sets the research question addressed in this paper. A method followed in this paper is presented in Section 5. Section 6 provides the analysis of data with results discussion. Section 7 provides a conclusion and future direction.

Background
Most of the styles are intuitive. However, we invite you to read carefully the brief description below.

Moodle
The acronym "Modular Object-Oriented Dynamic Learning Environment" stands for Moodle. Moodle is a popular example of an open-source LMS framework on which more than 50,000 university members can rely [5]. Moodle is a system that arranges the content as units that relate to the courses and as parts comprising the tasks and services of the course materials. Sultan Qaboos University (SQU) uses Moodle in the teaching and learning process besides face to face learning. Both students and instructors can access Moodle using their login credentials (SQU username and password).

Student's behavior
The behavior of students performs a significant task in learning. The two-way communication between students and the learning environment are learning behavior. The purpose of these behaviors is to make the desired improvements in what students know and what they can do [8].

Student's personality
Personality refers to human variations in characteristic thinking, feeling and action patterns [9]. Personality measures student satisfaction and academic performance [10,11].

Student's performance
Student's performance is the assessment of student's achievement across different academic subjects. Typically, instructors evaluate the achievement of students using the outcomes of courses, graduation rates and results from exams [12]. The performance of students also determines whether an issue occurs. If students do not advance at an appropriate pace through the course, then there is an issue [13].

Literature Work
To address the importance of the factors used in this paper, this section considers the student's EBP and student's performance separately.

Student's engagement
Student's participation in academic activities is often categorized into "academic engagement" conduct directly related to the learning process, such as time spent on assignment or participation in organized learning activities and the extent to which students are in communication with teachers or peers [14]. Engagement is vital in online courses for student learning and satisfaction. To clarify numerous things, the term student engagement has been used. These may be objects of concern, time on task experiments that examine the quality of commitment and ability to engage in learning tasks [15]. Guo and colleagues researched students' engagement when students watched recordings. The feedback elements of this study were based on the time spent watching the video and the number of times the student responded to assessments [16]. Martin and Bolliger explored the effect of engagement methods by students of inequalities in age, ethnicity and years of the online learning experience. The study results have implications for online learners, instructional designers and administrators who want to increase participation in online courses [17].

Student's behavior
By evaluating student's preferences by using the E-learning framework developed using Moodle, Ahmad and colleagues suggested mapping the development of student's characteristics into the Integrated Felder Silverman (IFS) learning style model. Significant characteristics have been established related to the Felder Silverman learning dimension: active/reflective, sensor/intuitive, visual/verbal and sequential/global. They noticed that the preferences of students were consistent with the characteristics of the learning styles described in the Felder Silverman model [18]. Abdullah suggested an approach, based on their learning style, to dynamically distinguish students. With 35 students for the Data Structures online course generated using Moodle, the method had experimented. By extracting student behavior data from the Moodle log, the learning style for each student was specified according to the Felder and Silverman model. At the end of the course, the behavior-based learning style was also compared to the quiz results [19].

Student's personality
Personality impacts students' behavior in various ways, such as their relationships with peers, instructor interactions and their inspiration, academic performance and learning. The personality of students influences their academic motivation, academic performance, their interaction with other colleagues and teachers, as well as their future behavior in society [20]. In the context of distance and online education, the connection between personality and performance had been examined. Although findings had been mixed, they showed a significant personality-performance relationship [21]. In 2019, the effect of five personality traits (extraversion, agreeability, conscientiousness, neuroticism, and intellect/imagination) on the experience of online learning by students was examined by Bhagat and colleagues. A total of 208 Taiwanese students engaged in an online survey. The findings provided a proof that online classes for students with common personality characteristics had different interests and experiences. The authors recommended additional studies on the possible effect of different personality attributes on learning and success in online environments [22].

Student's performance
In 23 online courses at two community colleges, Jaggars built an online course quality rubric spanning four fields, investigating the relationship between each quality area and student end-of-semester results. The findings revealed that the quality of human engagement within a course contributed to student grades favorably and sub-stantially [23]. An empirical evaluation of a self-assessment methodology on 272 students across nine courses using a logistic regression method showed that student grades at the end of the semester could be estimated by students' own self-reports of their learning styles at the beginning of the course [24].

Research Question
The aim of this current research is to broaden our understanding of student engagement, behavior and personality in online courses by using intense longitudinal courses to capture students' day-to-day experiences as they navigate an online learning course over a semester via extracted course logs from Moodle as a Learning Management System platform. This approach helps us to examine how much of the engagement, behavior and personality of students could be viewed as student performance indicators or factors. Specifically, this paper addresses the following question: Can Student's Engagement, Behavior and Personality be considered as indicators for Student's Performance?

Context
The aim of the predictive model of the student's Engagement, Behavior and Personality (EBP) is to get an idea of how the student's engagement, behavior and personality could influence the student's performance in online courses. Also, to predict all the possibilities in which the performance of students could vary depending on their EBP [25] as shows in Fig.2. It would be possible to enhance instruction techniques through tracking the student's EBP and performance and be able to take any required precautions to improve learning environments [26].

Course information
"Search Strategies on the Internet Course," was given in Fall 2019 by the Department of Information Studies, College of Arts and Social Sciences at SQU. Thirtyeight undergraduate students enrolled in it. Every part of this course was given electronically, including recorded lectures, assignments, weekly discussions, sample quizzes, mid-term exams and final exams. The course aimed to enhance students' skills in searching for information sources using different tools such as search engines, subject directories, library catalogs, and online databases. The teaching model of most SQU courses focused on both online and face-to-face approaches in the learning process. It seems that the level of involvement of individual students in online classes was difficult for teachers to assess since students were not physically present [27]. Moodle has built-in features related to this configuration that could generate several types of reports to track student's activity. Such as action logs, exam logs, log files and etc. [28]. The current research started to put the case study into experimentation and analyzed the log file of the selected course which was fetched from the Moodle database. The authors dealt with the large amount of data coming from the log file obtained from Moodle. However, they only focused on four factors: Student's Engagement, Behavior, Personality and Performance. The main aim of the current research is to help instructors forecast the performance of their students before the end of the semester and then evaluate the data in a smart learning environment [29]. Interpretation of The Results and Discussion

Prepare performance factor
The performance factor is represented by the total mark of the (Assignment 1, Mid Exam, Report, Weekly Discussions, Assignment 2 and Final Exam), the full mark of all assessments course is out of 100. The student name was replaced by the student ID for student's privacy reasons. Therefore, the student ID and his/ her mark were preserved, and the other details were removed. The file of all marks is shown in Table  2. Dividing the numerical data into categories in this work is based on the percentiles approach. It is based on dividing the data into unequal intervals, but each interval points to a specific category. "The percentage of scores that fall below a specific value is indicated by percentiles. They tell you where, compared to other ratings, a score stands.
Percentiles are a fantastic tool to use when you need to grasp a value's relative standing" [30]. (1) Percentiles are not as highly affected by the distribution extreme values as the mean value [31]; (2) do not rely on a particular probability density function to be chosen compared to the arithmetic mean needed for normally distributed results [32]. In educational and psychological testing and other study environments, percentiles have multiple uses and are typically more informative than raw scores as performance level measures. Almost all standardized test manuals use percentile tables or similar metrics that allow for raw scores to be interpreted [33].
Here, the marks of students were converted to three categories (High, Average, and Low) by dividing the period, as follow: The marks of students using categories are shown in Table 3.

Prepare engagement factor
The engagement of student is represented by the number of actions that are conducted by him/her. This means, in the full log file the actions for each student in column "Event Name" will be counted. All other columns are not required. Table 4 shows part of the required fields of "full log file" which are: Student ID and the Event name. The others are removed (Full data of this file is available upon request). Table 4. Part of the required fields of "full log" file (The data Available upon Request)

Student ID Event name 1000
The user's score report is reviewed 1000 The course has been reviewed 1000 The course module has been reviewed Counting the number of events name: In order to count the number of event names which are related to each student, the authors used MS Access by import the file of a full log after removing the unrequired fields. Then counted the number of events names for each student without duplications. Convert the counts into category: In order to represent all data with the same range as Performance (0 to 100), the values of Engagement were transformed to be up to 100 as follow: • The maximum value of Engagement is 1174.
• The 1174 can be transformed to 100 by dividing it by 11.74.
• So, all the values of Engagement can be transformed to the range of 100 by dividing them by 11.74. Table 6 shows the Engagement factor after transforming it into range 0 to 100. In addition, these values are represented by three categories (High, Average, Low) by dividing the period as follow: • Low: 0.00 -35 • Average: 35.1 -75 • High: 75.1 -100.0

Prepare behavior factor
The behavior of a student is represented by the percentage of access elements for each one. The "Element" column is the required filed from the "Full log" file. The total unique accessed Elements is 17 without duplication. The value of behavior will be calculated by dividing the accessed elements for each student by the total accessed elements, which is 17. Table 7 shows the accessed elements for each student.

Count the number of accessed elements:
To count the number of accessed elements, the MS Access was used to count the number of events as in Table 7. Table 8 shows the number of accessed elements for each student. Calculate the Value of Behavior: The value of behavior is calculated by Equation 1: The behavior of Student i= As an example, for Student with ID "1000", behavior value= 66 67 × 100 = 64.7 Table 9 shows the values of behavior for each student. Converting the behavior values to categories 0.01: The behavior of students was converted to three categories (High, Average, Low) as the following: The behavior of students using categories is shown in Table 10.

Prepare personality factor
The Personality factor is a value of the number of accessed elements by a student without duplication. This value is calculated by tracking the interaction of students with all elements, as shown in Table 12. In Table 11, 1 point to that the student interacts with this element while 0 points to that the student does not interact with this element.  To transform the values of Personality to be in range of 0 to 100, the value of Personality is multiplied by 7.69, that the maximum current value of personality is 13, 100\13 equals 7.69. Then Personality of students was converted to three categories (High, Average, Low) as follow: The Personality of students using categories is shown in Table 12.

The Relationship between performance and EBP factors
The details of the three factors and performance for the 38 students are illustrated in Table 13.  Fig.3 illustrates the overall distribution of data for EBP factors and Performance. By tracking the details of students, and observation of Fig.3, most of students' performance is affected by other factors, as, mostly the performance is "High" or "Average" when at least one of the categories of EBP factors is "High" or "Average". The clear relationship between EBP and Performance was also detected using the decision tree as follows. As it is shown in Fig.3 and Table XIII, mostly the factors are played with others, when the category of Engagement is "High" the Personality is "High" and the others are the same except the category of Performance can be "Average". And it is clear when all the categories are "Low" the Performance is "Low". All the 38 students have compatible statistics and results as the relationship between their Engagement, Behavior, Personality and Performance is symmetric.
These results can be represented using a group of condition rules (If-then). The condition rules give us a clear and visual view of the relationships among the variables. The rules are not 100% true for all cases in the dataset. These are the best rules that can be extracted. These rules as follow: Overall, these rules prove that there is a clear relationship between the three factors (EBP) and the Performance which is affected by all of them. It is obvious that the effectiveness of the three factors are nested. They collaborate to influence student's performance.

Conclusion
Building a smart learning environment is the basis for reforming instruction and learning practices. Since higher education institutions have students with different needs, they need specific smart learning environments, which is customized and personalized with the learning materials to meet the needs of students. To get the most out of information technologies, educational institutions change their teaching methods [34]. As well as to be able to use emerging technologies in the teaching and learning process, instructors must implement technology, closely monitor it, and demonstrate a constructive attitude toward it [35]. More in more, instructors might encouraged their students to communicate with the course or engage in the learning process [36].
Good analysis techniques can help researchers to analyze students' logfile in educational platforms in a smart manner. In this paper, the full log of the "Internet Search Strategies "course was analyzed to examine the relationship between the three indicators: Student's EBP to predict student's performance. The results along with the rules proved the existence of a strong relationship between the student's Engagement, Behavior and Personality and the performance of the student.
The future direction is to propose a prototype of the Student Tracking Performance Tool for Sultan Qaboos University instructors to promote student's EBP in the learning environment. Of course, this tool could then be generalized and used in any educational environment.