Extending the Technology Acceptance Model to Understand Students’ use of Learning Management Systems in Saudi Higher Education

— Although learning management systems (LMS) have been widely adopted by higher educational institutions in many countries, they are considered an emerging technology in Saudi Arabia. Furthermore, research has demonstrated that the students’ use of them is not always satisfactory. This quantitative study investigated the factors that affect the students use of LMS in higher education by extending the technology acceptance model (TAM) and adapting eight external variables. Based on the probability multi-stage cluster sampling technique, online surveys were sent by email to 2000 students registered in three public universities in Saudi Arabia. 851 responses were submitted by participants, and 833 responses were used for data analysis. Using Partial Least Squares Structural Equations Modeling (PLS-SEM), the results revealed that perceived ease of use is affected by six factors (content quality, system navigation, ease of access, system interactivity, instructional assessment and system learnability). The findings confirmed that perceived usefulness has five determinants (content quality, learning support, system interactivity, instructional assessment and perceived ease of use). This research is relevant to researchers, decision makers and e-learning systems designers working to enhance students’ use of e-learning systems in higher education, in particular where there is not yet widespread adoption.


Introduction
The development of information technologies has led to the acquisition of new applications in the field of education [1]. E-learning, as a result of this advancement, refers to an educational style that utilizes computer technologies to convey education to learners [2]. LMS are still the most popular medium for conducting e-learning in many higher educational institutions [3,4]. Dahlstrom et al. [5] found that 99% of educational institutions, 85% of teachers and 83% of students had adopted LMS in the USA.
Regardless of this reach, the effectiveness of LMS places reliance on the students' use [6,7], and their benefits are minimized if students do not use them [8,9]. Research has demonstrated that the use of LMS is still not within expectations [8,10]. Studies [11,12,13,4] found that not all functions of LMS are used, and students primarily use LMS for storing and downloading materials. A recent study [14] examined factors affecting the students' use of Blackboard at King Abdulaziz university in Saudi Arabia and revealed that the mean score of students' actual use was the lowest among the examined variables. Similarly, another study [15] evaluated the students' perceived usability of LMS at Jeddah Community College in Saudi Arabia and found that most students (56%) use LMS either rarely or occasionally. This indicates that issues remain discouraging the use of LMS and suggests investigating factors that encourage effective utilization.
Addressing the gap, this study aims to empirically investigate the factors that influence the students' use of LMS in higher educational institutions. The authors proposed a theoretical framework by extending the TAM model and adapting eight external variables, namely content quality, learning support, visual design, system navigation, ease of access, system interactivity, instructional assessment and system learnability. Therefore, this paper will validate the proposed theoretical framework with students from Saudi higher education.
This paper is organized as follows. The TAM model is briefly described. Section 3 introduces the proposed theoretical framework. It is followed by a section on the research methodology. In section 5, the model testing is described in detail. Finally, the discussion and conclusion sections are presented.

Technology Acceptance Model
The TAM model was developed by Davis et al. [16] with the aim of producing a model for computer technology acceptance based on the theory of reasoned action [17]. The TAM model assumes that when someone is introduced to a new technology, his or her decision to use it will be influenced by a number of factors. Primarily, TAM is composed of four constructs: perceived ease of use (PEOU), perceived usefulness (PU), behavioral intention (BI) and actual use (AU). PEOU refers to the extent to which an individual believes that using the technology under investigation would be free of cognitive effort, and PU can be defined as the degree to which an individual believes that using the technology under investigation would improve his or her performance [16]. AU is directly influenced by BI, that is affected by both PEOU and PU. PEOU affects PU directly, and both PEOU and PU are influenced by external variables.
Theoretical Framework and Research Hypotheses:The research model was mainly developed based on the TAM model [18] and eight usability attributes, namely content quality, learning support, visual design, system navigation, ease of access, system interactivity, instructional assessment and system learnability. These usability attributes were adopted from the work done by Zaharias and Poylymenakou [19], as they were carefully selected based on a profound review of many studies in the domain of usability, e-learning and educational technologies. The proposed variables are presented in the next sub-sections.

Content quality
The content quality (CQ) refers to the accuracy of used terms, sufficiency of materials to support the course objectives and relevance of information [20]. The content of e-learning systems should be organized in an appropriate sequence and provide adequate resources [19]. E-learning systems with high-quality content can maximize the chance of system acceptance and vice versa [21]. DeLone and McLean [22] highlighted the significance of information quality in their information systems success model and postulated the influence of information quality on users' satisfaction and users' intention. Therefore, the following hypotheses were proposed.
• H1: CQ has a direct positive influence on students' PEOU of LMS.
• H2: CQ has a direct positive influence on students' PU of LMS.

Learning support
Learning support (LS) refers to the ability of e-learning systems to provide users with tools and features needed to support learning activities [19]. Further, those elearning systems should support students in terms of help documents. In [23], it was found that students were unable to achieve difficult learning tasks using e-learning systems without help. Those help documents of e-learning systems should be written in a clear language for students [23], rich with the information that students need [24] and available for students whenever necessary [25]. Therefore, the authors proposed following hypotheses.
• H3: LS has a direct positive influence on students' PEOU of LMS. • H4: LS has a direct positive influence on students' PU of LMS.

Visual design
Visual design (VD) refers to how the interface layout and menus are appropriate and attractive [26]. The user interface has become more and more complicated [27], and students usually make their judgments regarding e-learning systems based on the interface design [28]. Interfaces should be appealing to look at and should have an appropriate use of aesthetics to improve the navigation of e-learning systems [24]. Systems with good VD place important information in an area to which students will be attracted [19]. Therefore, the following hypotheses were proposed.
• H5: VD has a direct positive influence on students' PEOU of LMS. • H6: VD has a direct positive influence on students' PU of LMS.

System navigation
System navigation (SN) refers to the degree to which the organization of the system is understandable and appropriate [29]. SN is a map that connects the components of a system and is expected to enable users to move within the system in a clear and easy way. The navigation of e-learning systems should allow students to leave when they desire and easily return to the system [19]. Additionally, information should be reached easily and efficiently [29]. With a system that has good SN, users are informed where they are [30] and where they can go within the system [31]. Therefore, the following hypotheses were proposed.
• H7: SN has a direct positive influence on students' PEOU of LMS.
• H8: SN has a direct positive influence on students' PU of LMS.

Ease of access
Ease of access (EOA) refers to the degree to which users can access the system without difficulty from the login process to the course content [29]. EOA includes, but not limited to, the support of different platforms [32], smooth login, response time, quick download, appropriate use of texts [23] and freedom from technical issues [19]. The poor accessibility of LMS might cause students frustration [29] and reduce their learning because of waiting and time limitations [23]. Therefore, the following hypotheses were proposed.
• H9: EOA has a direct positive influence on students' PEOU of LMS. • H10: EOA has a direct positive influence on students' PU of LMS.

System interactivity
System interactivity (SI) represents how students are engaged with e-learning systems during their education [23]. In the view of [33], SI refers to how students learn by interacting with other students, teachers and objects in the LMS. SI includes all sorts of communications via e-learning systems during the learning experience [20]. This communication can be (1) between students and teachers, (2) between students themselves and (3) between students and LMS. It was shown that the systems' interactivity affects students' intention to use LMS [8] and their perceived learning success [34]. Therefore, the following hypotheses were proposed.
• H11: SI has a direct positive influence on students' PEOU of LMS. • H12: SI has a direct positive influence on students' PU of LMS.

Instructional assessment
Instructional assessment (IA) is a crucial element in designing e-learning systems [23] as it is a good way to assess students' learning [35]. IA can give feedback about the students' accomplishments in relation to course objectives [36], enable students to learn more by answering questions [37] and enhance students' academic achievement [38]. LMS usually provide a variety of assessment tools including surveys, quizzes and tests. These should be self-assessment tools to help students in understanding the content of courses. Therefore, the following hypotheses were proposed.
• H13: IA has a direct positive influence on students' PEOU of LMS.
• H14: IA has a direct positive influence on students' PU of LMS.

System learnability
According to Nielsen [39], system learnability (SL) refers to the degree to which users can learn how to use the system without difficulty. It is very important especially for novice users to be able to successfully interact with the system within a short time [40]. With a highly learnable system, users believe that they can start using the system with minimum training, help and orientation [41]. Systems with poor learnability can lead to more user training, technical support and maintenance cost. In an ideal world, e-learning systems should not have a learning curve, so students will learn how to use the system from the first attempt [23]. Therefore, the following hypotheses were proposed.
• H15: SL has a direct positive influence on students' PEOU of LMS.
• H16: SL has a direct positive influence on students' PU of LMS.

Perceived ease of use, perceived usefulness and behavioral intention
Perceived ease of use (PEOU), perceived usefulness (PU) and behavioral intention (BI) was proposed by several technology models, e.g. TAM [16], TAM2 [42], PEOU determinants [43] and TAM3 [44]. In our work, PEOU can be defined as the extent to which students believe that utilizing LMS would be free of effort, while PU can be defined as the degree to which students believe that utilizing LMS would improve their performance and BI refers to the students' aim or plan to use LMS [45]. Further, PEOU and PU were postulated to be an antecedent to BI in various technology models, such as TAM, TAM2, PEOU determinants and TAM3. Therefore, the following hypotheses were proposed.
• H17: PEOU has a direct positive influence on students' PU of LMS. • H18: PEOU has a direct positive influence on students' BI of LMS. • H19: PU has a direct positive influence on students' BI of LMS. • H20: BI has a direct positive influence on students' AU of LMS. Methodology

Population and sampling
The target population in this study is higher education students who are using LMS in their education at Saudi public universities. According to the Ministry of Education in Saudi Arabia [46], 26 public universities have LMS with over 1.3 million students. As the population is quite large and widely dispersed across the country, the probability multi-stage cluster sampling technique was employed as suggested by Bryman [47]. The population was divided into clusters, and each cluster represented the students at one university. Second, the clusters were grouped based on the geographical regions. This yielded three groups: Western region, Central region and Eastern region. Finally, from each geographical region, one university was selected randomly. The selected universities were: King Abdulaziz University, King Saud University and Imam Abdulrahman Bin Faisal University.

Instrumentation
Following the majority of technology acceptance studies, this study benefits from utilizing surveys for data collection [48,49,50,14,51,52,53,54]. The instrument of the study comprises three sections. The first section describes the participants' demographic information. The second section includes the eight external variables with 35 positive statements. The last section covers the TAM model constructs with 17 positive statements. The last two sections can be answered using a five-point Likert scale, where 1 indicates that students strongly disagree with the statements and 5 indicates that students strongly agree with the statements.
To ensure that the questionnaire items measure the desired content, the face validity of the developed questionnaire was tested in collaboration with six experts from different academic fields. As this study was conducted in Saudi Arabia, where Arabic is the first language and most students speak Arabic, the decision was made to translate the questionnaire from English into Arabic. Two academics who are native Arabic speakers and fluent in English used the back-translation method [55] to translate the questionnaire into the Arabic language.

Data collection
Emails were sent to 2000 students registered in different academic programs and various levels of education in the three universities. 851 responses were submitted by participants, equivalent to a response rate of 42.6%. After the preliminary examination for outliers, normality and unengaged responses, 833 responses (41.7% response rate) were used for data analysis. The demographic information of the students is summarized in Table 1. The results indicated that 193 students (22.8%) were below 20 years old, 576 students (68.1%) were between 20 and 30 years old and 77 students (9.1%) were above 30 years old.

Data analysis
For the data analysis stage, the researchers exported the obtained data into SPSS 23 to perform the preliminary examination including missing data, collinearity, outliers, normality and unengaged responses. This study benefits from using the PLS-SEM technique and SmartPLS 3 software to analyze and test the research proposed model. PLS-SEM is convenient when the primary objective of the research is to extend an existing theory or identify key drivers [56]. Therefore, data were exported from SPSS and imported into SmartPLS version 3.2.7 to perform further analysis and model testing.

4
Model Testing

Measurement model assessment
The results of the measurement model assessment are shown in Table 2. The indicators' reliability is achieved when the loading of each indicator is above 0.7 [57,58,56]. Using the PLS algorithm with 1,000 iterations, the results demonstrated that all indicators were reliable except SN05 (0.66) and AU02 (0.50). Those two indicators did not meet the recommended threshold; therefore, they were removed.  [58], providing evidence of the high reliability of the constructs.
Researchers [57,58,56] believe that convergent validity is achieved when the outer loading of each indicator is above 0.7 and average variance extracted (AVE) of each construct is 0.5 or above. Table 2 shows that AVE values exceeded 0.5 demonstrating the convergent reliability of the constructs.
One recent method for measuring the discriminant validity in PLS-SEM is the Heterotrait-Monotrait Ratio (HTMT) which was developed by Henseler et al. [59]. HTMT represents the estimate for the construct's correlation with the other constructs, that should be smaller than one [60]. A correlation closer to one shows a lack of discriminant validity. Henseler et al. [59] suggested a threshold of 0.90 when the constructs are conceptually similar and 0.85 when the constructs are conceptually different. The results of HTMT assessment in Table 2 ranged between 0.44 and 0.90, indicating the discriminant validity of the constructs.

Structural model assessment
Testing the hypotheses and path coefficients in SmartPLS 3 entails the utilization of bootstrapping, a non-parametric statistical approach that draws many sub-samples from the sample data and examines models for each sub-sample [61]. For bootstrapping options, 10,000 sub-samples, one-tailed as the hypotheses were proposed to be positive (+) and a significance level of 0.05 were employed. The results of hypothesis and direct relationship testing are presented in Table 3, showing that PEOU was affected by six independent variables, namely CQ, SN, EOA, SI, IA and SL. In terms of PU, CQ, LS, VD, SI, IA and PEOU were significant. The students' behavioral intention to use LMS was significantly influenced by PEOU and PU.  Coefficient of determination (R 2 ) refers to the effect of independent variables on the dependent variables [57]. Table 3 summarizes the result of R 2 . The findings revealed that CQ, SN, EOA, SI and SL explained 74% of the variance in PEOU, where SL explained the most compared to the other variables. Regarding PU, six independent variables, namely CQ, LS, VD, SI, IA and PEOU, accounted for 67% of the variance in PU, where PEOU contributed the most. Both PEOU and PU explained 62% of the variance in BI. According to Hair et al. [58], those R 2 estimates were substantial, which indicated the high quality of the proposed model.

Discussion
The findings indicated that 14 out of 20 path relationships in the structural model were supported. In accordance with the previous literature in Saudi e-learning [21,62,63,50,14,49], the structural model examination demonstrated the TAM model constructs (PEOU, PU, BI and AU) for an LMS in the context of higher educational institutions in Saudi Arabia.
In this study, CQ represents the extent to which students in Saudi universities believe that LMS have good content. The results revealed that CQ has a significant effect on PEOU (β = 0.055, p < 0.05), and accordingly H1 was accepted. One possible interpretation is that students prefer LMS that have easy to reach, updated, sufficient and well-organized content which, consequently, make their education process easy and effortless. Supporting this result, it was empirically found [64,65,66,67,68] that the content of e-learning systems is a determinant of students' PEOU. Moreover, the path analysis demonstrated that CQ is a significant predictor for PU (β = 0.065, p < 0.05), and therefore H2 was accepted. This result conflicts with the findings of Kang and Shin [69] who found no effect of CQ on PU in the context of virtual classes, which is not the case in this study. They attributed their result to the concern that the existence of teachers in synchronous e-learning might reduce the influence of content quality. In contrast with [69], many studies in e-learning [21,64,66,65,68] have demonstrated the effect of CQ on students' PU.
In this study, it was proposed that LS has a direct positive influence on students' PEOU and PU of LMS. The results revealed that LS does not have a significant effect on PEOU (β = 0.046, p = 0.072), and accordingly H3 was rejected. On the other hand, the path analysis demonstrated that LS is a significant predictor for PU (β = 0.158, p < 0.001), and therefore H4 was accepted. This implies that when students perceive that LMS provide good learning support, students are more likely to perceive them to be useful. More specifically, students tend to choose LMS that have appropriate and sufficient tools to support their education with proper help. This augments their perception of the usefulness of the systems.
The results of the structural model assessment unexpectedly disclosed the lack of a direct positive influence of VD on PEOU. Rejecting the hypothesis H5 contradicts elearning research [21,28,33] Nevertheless, the non-existence of VD influence on PEOU can be attributed to participants' experience with LMS and self-declared moderate and high level of computer and Internet skills. Thus, the students' wide exposure to technology might contribute to minimizing the significance of the interface visual design. In terms of PU, the examination findings revealed that PU is negatively affected by VD (β = -0.102, p < 0.01), and accordingly H6 was rejected. Reviewing the literature revealed that the relationship between VD and PU in e-learning systems is still indeterminate. For example, Khedr et al. [28] demonstrated this effect, while [21] found that VD does not influence students' PU. Further, the finding of this study could be justified as most participants expressed advanced computer and Internet skills indicating that they have computer self-efficacy, which was found to negatively affect PU in e-learning research [70]. Additionally, the interface visual design might come at the expense of the system's usefulness. In other words, the designer sometimes needs to sacrifice several functions in the system or remove some parts in order to develop a system that has a good user interface.
In this study, it was hypothesized that SN has a direct positive influence on students' PEOU and PU of LMS. The results revealed that SN has a significant effect on PEOU (β = 0.176, p < 0.001), and accordingly H7 was accepted. One possible interpretation is that students favor LMS enabling them to find information, predict links, leave and return easily which, consequently, makes the navigation between the course content effortless. Supporting this result, it was empirically found [71,72,33] that the navigation of e-learning is a substantial determinant of students' PEOU. Regarding PU, the path analysis demonstrated that SN is not a significant predictor for PU (β = -0.065, p = 0.054), and therefore H8 was rejected. This finding was unexpected as past literature in information systems [72,26] demonstrated that SN is an important predictor for PU. However, those studies were not conducted in the domain of e-learning systems. Furthermore, Jeong [71] investigated the use of an e-library in Korea and found that SN does not influence students' PU.
The findings supported the hypothesis H9, stating that EOA has a direct positive influence on students' PEOU of LMS, (β = 0.054, p < 0.05). Nevertheless, the path coefficient indicated that the relationship between EOA and PEOU was the weakest significant relationship compared to the other relationships. This is understandable as many IT infrastructure and telecommunication projects have been taking place recently in Saudi Arabia under the vision of 2030, so most students do not have problems with accessibility and Internet connection and can login to the system at anytime and anywhere. The finding is consistent with several empirical studies in e-learning [21,69,66,6,67,73]. In terms of EOA → PU, the study provided evidence that EOA does not have an influence on PU (β = -0.014, p = 0.324), and therefore H10 was rejected. The results demonstrated that the students' perception of ease of access of LMS does not play an important role on their view of the usefulness of LMS. Rejecting the hypothesis H10 is in accordance with most past literature in e-learning [69,6].
The results revealed that SI has a significant effect on PEOU (β = 0.124, p < 0.001), and accordingly H11 was accepted. A plausible interpretation that communication tools provided by LMS were easy to use, not complicated and limitation-free in terms of time and place which contributed to an increase in the students' belief about the user friendliness of the systems. Even though studies [48,74] in e-learning systems contradict this finding, both studies were conducted with only undergraduate students enrolled at a single institution (university) in Malaysia and Taiwan respectively. Nevertheless, the result of this study is compatible with most previous research in e-learning [75,51,76,77,73]. Regarding SI and PU, examining the relationships disclosed that SI positively impacts PU (β = 0.272, p < 0.001), and thus H12 was accepted. More specifically, SI → PU is the second strongest path among the external variables. This could be explained that students considered that LMS are rich with asynchronous and synchronous tools that facilitate the students' communication with each other and with teachers. Highlighting the relative advantages of LMS [64]. This finding is consistent with past literature in e-learning [78,64,75,48,51,77,33].
It was hypothesized, in the proposed model, that IA has a direct positive influence on students' PEOU (H13) and PU (H14) of LMS. The results revealed that IA significantly effects both PEOU (β = 0.059, p < 0.05) and PU (β = 0.220, p < 0.001), and accordingly the hypotheses H13 and H14 were accepted. This implies that when students are provided with good assessment tools, they are more likely to perceive LMS effortless and useful. One possible interpretation is that students prefer LMS that have easy to use self-assessment tools enabling them to understand the content of the course and measure their achievements of learning objectives. This in turn makes their education process easy and valuable.
The findings supported the hypothesis H15, stating that SL has a direct positive influence on students' PEOU of LMS, (β = 0.440, p < 0.001). Therefore, hypothesis H15 is accepted. The path coefficient indicated that the relationship between SL and PEOU was the strongest significant relationship compared to the other external variables. This effect can be explained as e-learning systems are a relatively new technology in the education of Saudi Arabia, so students require an easy to learn LMS. The finding is well aligned with several empirical studies in information systems [79,26]. In terms of SL → PU, the study provided evidence that SL does not have an influence on PU (β = 0.014, p = 0.376), and therefore H16 was rejected. The results demonstrated that an easy to learn LMS does not play an important role on the students' decision regarding the usefulness of LMS in their education. Although studies in information systems [79,26] confirmed the relationship between SL and PU, the three studies did not examine e-learning systems, did not survey students and were not conducted in Saudi Arabia.
The results demonstrated that the relationships in TAM are significant, and accordingly H17, H18, H19 and H20 were accepted. Previous studies in LMS revealed the impact of PEOU on PU [50,14,49,8,66,63]. A positive influence of PEOU on the students' intention to use e-learning systems was found in [8,66,68,80]. Further, many studies [75,50,80,81] concluded that when students perceive that LMS are useful, they usually intend to use the system. Past research [8,50,48,81] in elearning systems supported the relationship between BI and AU.

Conclusion
LMS have been implemented in almost every higher educational institution; however, studies [11,12,13,4] provided evidence of the low level of students' use of them. Hence, this cross-sectional study was conducted with the objective of investigating the factors that affect the students' use of LMS in higher education by the TAM model with eight external variables, namely content quality, learning support, visual design, system navigation, ease of access, system interactivity, instructional assessment and system learnability.
Using the PLS-SEM technique, the results confirmed that the relationships in TAM are significant in the context of LMS in Saudi Arabia. Fourteen out of twenty hypotheses were supported in this study. The findings revealed that students' perceived ease of use is positively affected by CQ, SN, EOA, SI, IA and SL, and the strongest correlation with SL. While students' perceived usefulness is positively influenced by CQ, LS, SN, IA and PEOU, and the strongest correlation with PEOU and SI. This research is relevant to researchers, decision makers and e-learning systems designers working to enhance students' use of e-learning systems in higher education.