Defining E-Learning Level of Use in Jordanian Universities Using CBAM Framework

This paper presents a continuous work related to the status and engagement with e-learning systems by faculty members in Jordanian universities. Previous research study entitled (Evaluating E-Learning System Use by CBAM-Stages of Concern Methodology in Jordanian Universities, 2015) focused on defining stages of concern using one tool that is presented by CBAM Framework. The CBAM Framework provides another tool that is used specifically for measuring level of use of any educational technology. This research focused on using CBAM framework for the first time in Jordanian Higher educational context in order to present a systematic and well-defined methodology for evaluation that will be used for defining the exact engagement with e-learning. It is believed that the results of this research can be reflected positively on universities future policies and practices for developing the use of this educational technology in Jordan Key Words—CBAM, Evaluation, E-learning, Jordanian Universities, Level of Use


Introduction
The use of e-learning technology in Jordanian universities started in the late 2003, as some universities had a vision for the benefits of adopting this educational technology. In the later years, most universities adopted this technology and many researchers found a fertile ground for conducting their studies on the benefits, obstacles and future development within the Jordanian context [1]. Less concern have been oriented towards evaluating the level of use of this educational technology in Jordanian universities, as there are still different obstacles that are hindering the adaptation towards technological practices [2]. In terms of technological challenges, Jordanian Information and Communication Infrastructure (ICT) supported by the strategies of The Ministry of Information Technology and private sector communication companies have managed to maintain a high growth rate, which reflected positively on minimizing the technological obstacles of adapting e-learning in educational Institutions [3]. However, the exact engagement and use of this tool has not been defined in earlier studies with respect to the high growth, quality of services and spread of ICT. On the other hand, many tools and strategies have been developed recently to investigate the use and effect of educational technology, and they proved to be robust and effective in outlining and defining the current engagement level. One of the most prominent framework for investigation is called Concerns-Based Adoption Model (CBAM) [4]. This framework has been used widely in educational institutions in the West, and it proved by many research studies to be capable of providing assessment towards using e-learning and defining its effect and impact on instructors practices [5]. CBAM framework provides different tools for assessment that are: Stages of Concern (SoC); Level of Use (LOU) and Innovation Configuration (IC). Each of the previous tools has its own use and application, they can be used separately to define some aspects of investigating e-learning use and adoption, or they can be used together with respect to the order starting with (Stages of concern and ending by Innovation Configuration). This research study is a continuation for a previous two studies that investigated the use of e-learning using SoC, as the previous results showed that majority of faculty members are defined in the informational stage [4]. The informational stage shows that users have general understanding and awareness for the technology used, with interest to learn more details about it. However, it is important to investigate their level of use, in order to have better understanding to the status and adaptation towards e-learning, in order to present recommendations and solutions for future development in Jordan. The next section will introduce CBAM's (LOU) and discuss the aspects related to each LOU.

The CBAM's Levels Of Use
The CBAMs -LOU concentrates on the behavioral patterns in its use. It describes the experiences of faculty members in their path to learn and engage with the tool. The LOU is perceived as the model of change in practice [6]. When it comes to performance, rather than assuming that faculty members are using or not using the technology, the use of LoU classifies five distinct levels of use (levels III-VI) and three levels of nonuse (levels 0-II) [7]. Based on CBAM research and development team, they defined eight levels of use through a multi-year development of inductive data analysis based on interviews and observations of users implementing various educational tools. Moreover, the use of LOU contains description of key decision points that occur when users shift from one level of use to another. The classification of LOU is rational and it is not mandatory that users will go through all Levels in a lock-step growing fashionǁ [7], [8]. The LOU is fixed through different dimensions of behavioral pointers related to the technology used. These dimensions are gathered into seven categories that are: (Knowledge ,Acquiring Information ,Sharing ,Assessing ,Planning ,Status Reporting ,Performing ) [8].

Mechanical Use
The users attentions is on the short-term, daily use of the innovation with little time for reproduction. The effort is primarily directed toward learning tasks compulsory to use the educational tool. Use is often fragmented and shallow Decision Point D-1 Routine form of use is recognized Level IVA

Routine
The use is stabilized. little, if any, changes are being made in ongoing use. Minimal efforts and thoughts to improve the use or its consequences.

Decision Point D-2
Users change the use of the educational tool based on formal or informal assessment to improve expected benefits. Level IVB Refinement Users differs the use of the educational tool to rise the expected benefits within the immediate scope of influence. Variations are based on knowledge of both short and long-term consequences and benefits.

Decision Point E
Users initiates changes in the use of the educational tool based on input from and in coordination with colleagues to improve expected benefits Level V Integration User is combining own effort with related activities of colleagues to achieve a collective impact within the collective scopes of influence.

Decision Point F
User begins to explore alternatives or major modifications to the educational tool presently in use.

Renewal
The user re-evaluates the superiority of use of the educational tool, seeks major adjustments of, or alternatives to, present tools to achieve increased impact, examines new developments in the field, and explores new goals for self and the larger community.
Based on the previous table, it shows that LOU provides a wide-ranging operational definition of level of use [9]. The use of LOU defines each user in one level, unlike the use of SOC that can defined users in different categories [4]. Moreover, the use of LOU valuation should always be perceived developmentally and not as summative end state.

Research Methodology
This research methodology is based on using the CBAM's LOU tool with different faculty members in Jordanian universities through a direct participation in semistructured interviews [10]. CBAM's LOU has been used in different research studies that proved reliability, internal consistency and validity by diverse samples with more than 11 educational tools [9], [11], [12]. A previous research study that included 12 public and private Jordanian Universities with the use of CBAM's SOC had a participation of 116 faculty members [4]. The same sample was approached and a number of 47 faculty members that are using e-learning system accepted to participate in this study. Another sample of 73 faculty members was approached to participate in this research study, and they were identified as faculty members that are not using elearning systems. The interviews were conducted in different settings and locations as appropriate for the participants. The Level of Use assessment results are in the purpose of defining one and only one Level of Use for each faculty member [9]. In terms of investigating the data the descriptive analysis was performed using SPSS package in order to define descriptive numeric data (frequencies) regarding the LOU for all participants.

CBAM's Level of Use Results
This section presents the results that have been collected and analyzed from faculty members that participated in the interview based on the CBAM's Level of Use. The first table presents the results of the faculty members that participated and used the tool during two full semesters, and they were 46, while the second table presents the answers from 73 faculty members that didn't have much activities using the e-learning system and activities. User is not doing 9 19%

100%
The previous results give an indication of users level in using the e-learning system, and it gives one result that is different from using CBAM's Stages of concern. Based on the level of use matrix that has been used with this research study [7] [9], the results shows that most of the users are defined as (Mechanical use) based on the behavioral indications provided by level of use methodology. The following section will present the questions and will discuss the results generated by this study.

CBAM's Level of Use-Discussion -Using the System
The LOU semi-structured interview defined eight questions that users have to answer in order to outline their category with respect to LOU defined categories. The previous table outlined the results of 46 faculty members that participated in this study. The following discussion will be related to the previous results with respect for each defined category in CBAM's LOU.

Knowledge Category:
This category presented the following question, (What do you see as the strengths and weaknesses of the e-learning system in your situation? Have you attempted to do anything about the weaknesses). The answers came as two results. The first result identified faculty members as mechanical use in this category and it scored the highest percent of (87%). On the other hand the second result came for routine uses and scored the result of (15%). The answers from the 40 faculty members were focusing on the strength of e-learning, and their answers agreed that the e-learning assisted in broadening their knowledge about using e-learning and other associated tools and services. Moreover, they agreed that they need more to learn about e-learning and associated activities. On the other hand, related to weaknesses, they agreed that the major challenge is related to time and commitment needed to prepare and perform activities with respect to their teaching responsibilities. In the same category seven faculty members out of were defined as routine users. The "routine users", showed more activity and engagement in the use of e-learning. In terms of weaknesses, they agreed that the added responsibilities are defined as the major challenge.

Acquiring Information
This category had the following question, (Are you currently looking for any information about the e-learning systems? What kind? For what purpose?) . The answers for this category came into two results. The first result was identified as mechanical use and scored the percent of (59%). On the other hand, the second result came for routine uses and scored the result of (37%) for faculty members. The users defined as mechanical agreed that they are using different sources of information related to e-learning. On the other hand, 17 faculty member were identified as routine users and they had different approach as they explored many sources and activities that could be merged with current e-learning practices.

Sharing
This category had the following question,( Do you ever talk with others about elearning? What do you tell them? ). The answers came into three results. The first result for mechanical use scored the highest percent of (43%)in this category, the second result came for routine uses scored (32%), and the last group scored the percent of (6%) for the (user is not doing). The mechanical users agreed that they have little discussions about e-learning with other faculty members. On the other hand, routine users showed more engagement in discussing the features and activities they explored from e-learning systems. Three faculty members stated that they did not have any sharing of information with anyone in regard for using e-learning.

5.4
Assessing This category had the following two questions categories that are (What do see as being the effects of the e-learning? In what way have you determined this? Are you doing any evaluation, either formally or informally, of your use of the e-learning? Have you received any feedback from the students? What have you done with the information you got?) .The second category questions were (Have you made any changes recently in how you use e-learning? What? Why? How recently? Are you considering making any changes). The results from the two questions category came as following, (2%) of faculty members were defined as orientation, as they are exploring and planned to do some assessment toward their activities. They also agreed that they are considering enhancing their practices with e-learning in the future. On the other hand, twenty-one faculty member have been defined as mechanical in this category with a percent of (45%). Those users confirmed using different activities with other faculty members to assess and provide insights on the use of different activities related to videos and presentations. In terms of doing changes in the use of e-learning, they agreed on working and performing some basic changes to the courses with respect to their current level of knowledge in this field. On the other side, more than half of faculty members (58%), confirmed that they are not performing any assessment with course material. The reason for that was related to not having enough time to be included in such activities as they are overloaded with different tasks and responsibilities. Moreover, they designated that they have not done any change on the use of e-learning, neither they are considering to do so.

Planning
This category had the following question (As you look ahead to later this year, what plans you have in relation to the use of e-learning?). Most faculty members in this category were found to be mechanical users; they had the percent of (76%). The mechanical users have confirmed having plans to continue to use e-learning and to try to master the current provided skills. On the other hand, (19%) of faculty members were defined as routine users, they confirmed that they have learned most of the activities that are related to the use of e-learning, and they working on learning more about other activities that are available.

Status Reporting
This category has the following question (Are you working with others (outside of anyone you may have worked with from the beginning) in your use of e-learning? Have you made any changes in your use of e-learning based on this coordination?). The results for this category came as (32%) of faculty members have been defined as mechanical, they confirmed having some talk and discussion among each other on the uses of e-learning. In terms of making any changes in the use of e-learning, they confirmed that their discussion have had some impact on their work, but no major change have been identified. One faculty member in this category have been identified as routine user. He confirmed having more frequent reporting activities with other faculty members, and such activities helped him in expanding his knowledge. The last defined results in this category were for users that answered they are not doing. The percent was the highest for this section with a percent of (39%). These users confirmed that they did not collaborate or had any kind of reporting activities among each other, and they mentioned that the teaching activities and little time they had for other activities was the reason for not being able of having reporting activities.

Performing
This category had the following question (Are you considering or planning to make major modifications or to replace e-learning use at this time). This question had three answers. First group of faculty members were defined as mechanical user with a percent of (50%). They all agreed that currently they are not considering performing any change to e-learning, as they need more time to learn about many features provided by this tool and the different associated activities. On the other hand, a percent of (30%) were defined as routine users, as they showed more positive attitude towards modifying and expanding their use of some activities and modules related to e-learning. Moreover, they confirmed having some kind of research on other available tools that are used for e-learning course content creation (like Video recording, Storytelling), and they mentioned that the list of activities should be updated with other available tools that are using presentation and quizzes. The rest of the faculty members with a percent of (19%), confirmed that they are not considering the change of e-learning uses, and they have not done any activities that are related to explore, modify or change the current ee-learning system uses.
From the previous answers in all categories, it has been evident that the users are interacting with the e-learning system according to their basic to intermediate knowledge they have acquired during their work in their institutions. Most uses and interaction with those activities are defined as mechanical uses, as they are having their most efforts on the short-term, day-to-day use of e-learning with little time for reflection. It has been noticed that mechanical users are primarily directing their efforts towards mastering tasks required using e-learning, and their use is generally defined as disjointed and superficial. However, a small number of users managed to be more engaged and showed more steadiness's with the use of e-learning and activities provided by the system. Those users were classified as routine users as they showed few, or simple changes in their ongoing use, and they provide minimal efforts and thoughts to improve their use of e-learning. Also in some different activities, users showed the attitude of not doing, especially in the following categories (Sharing, Assessing, Status Reporting, and Performing). It is important to understand that CBAM's level of use for most of the users have been identified as mechanical users as this methodology identifies one and only one level for users. The following section will present the answers from the users that did not interact with the system and clarify their status of the e-learning systems use.

CBAM's Level of Use -Discussion-Not using the System
The second group that participated in this study consisted of 73 faculty members that did not interact with e-learning systems. These users were grouped in a focus group sessions and had the CBAM's level of use interview to identify their status according to the CBAM's level of use matrix [7], [9]. The following results are displaying their answers to each of the questions that have been used within the CBAM level of use.

Knowledge Category:
This category had the following two questions (Have you made a decision to use elearning system in the future? If so, when? ) and (Can you describe the e-learning system for me as you see it). A percent of (10%) were identified as (None use) and they did not show any willingness for using the e-learning systems, nor they described the use and purpose of the e-learning. Their description for e-learning came short without relating any benefits of engagement with this educational technology. These users favoured the traditional approach of teaching and learning and they confirmed that the use of e-learning system would added burden to their current responsibilities, and there is no compensation or recognition for future use of this technology in their institutions. The rest of participants with a percent of (90%) were identified as orientation, and they showed positive attitude for this educational technology. They defined e-learning as a new educational approach for connecting faculty members and students, and for promoting the use of electronic services that are currently used throughout the web. Moreover, they confirmed, that this tool is effective in the educational assessment it provides different methods for providing information for students. On the other side, they agreed that using this tool would add more load and additional responsibilities.

Acquiring Information Category:
This category had the following question (Are you currently looking for any information about e-learning? What kinds? For what purposes? ). A percent of (96%) faculty members were defined as (not using). These users showed that they are not currently looking for any information related to e-learning. Moreover, they established interest that in the future, they might consider attaining more information about the benefits and use of e-learning. On the other hand, a percent of (4%) were identified as orientation. The orientation users showed more interest in obtaining more information about e-learning, and agreed that they had some discussions with other faculty members that were using e-learning system. Those discussions were mainly related to different features that are available using e-learning, and time required to do and perform such tasks.

Sharing Category:
This category had the following question (Do you ever talk with others for sharing information about the e-learning? What do you share? ). A percent of (93%) of faculty members were identified as (not using), they confirmed that they didn't share any information regarding the use or benefits of e-learning. On the other hand, a percent of (7%) were identified as orientation, as they mentioned that they have shared information with other colleagues that used the system. In terms of the shared questions, they focused on knowing the activities and their benefits, also the amount of efforts performed on such activities.

Assessing Category:
This category had the following two questions (What do you see as the strengths and weaknesses of e-learning for your situation? ), and (At this point in time, what kinds of questions are you asking about e-learning? Give examples if possible). A percent of (81%) were identified as (not using). These users did not define any strength for using e-learning and they mentioned that they preferred the traditional approach as it gives them more control over the educational process. In relation to the weaknesses, they confirmed that the use of e-learning is time consuming and will add additional burden for their current work. In relation to the second question, they agreed on the following concern (why to use this tool when many resources are available on the internet, and I can relate with my student more effectively in class). The second group of orientation had the percent of (19%) they agreed that using e-learning will create more opportunities for faculty members to be more engaged with different information that are presented in different styles and pedagogy. Moreover, they agreed that using the elearning will provide an educational environment through which the activities of both (faculty members and students) can be observed and evaluated more successfully. In relation to weaknesses, they agreed that learning and using e-learning will need more obligation, time and efforts that they can't afford currently. In regard for the questions related to the use of e-learning, they approved that they have concerns for using elearning in the future and suggested that in order for more actual use and engagement with e-learning there should be recognition for efforts or compensation.

Planning Category:
This category had the following question (What are you planning with respect to elearning? Can you tell me about any preparation or plans you have been making for the use of e-learning?). A percent of (93%) were identified as (not using) and they agreed that they are not having any plans or preparations for current use, and they are looking at this educational tool as adding more load to their current tasks. On the other hand, (7%) of faculty members were identified as orientation. They showed more positive attitude about using e-learning, but they needs better management skills for managing their time and tasks. Moreover, they agreed that they are planning use the tool in the future, especially if they can get assistant from other users that are currently using the system.

Status Reporting Category:
This category had the following question (Can you summarize where you see yourself right now in relation to the use of e-learning? (Status Reporting). A percent of (96%) were identified as (not using). They agreed that they didn't perform any activities for being engaged with e-learning system and they are not having current plans to do so in the near future. On the other hand, a percent of (4%) were identified as orientation. They that they are learning more about e-learning from other users they interact with, and they are willing to consider e-learning after they manage their time and responsibilities.
This section showed that users are having serious distresses towards using e-learning system, and the main concern is related to additional load and activities that they need to learn and perform. Moreover, they showed concerns related to identifying and compensating their efforts that is currently not considered by their institutions. Such concerns are significant to be included in any future recommendation of e-learning use, as the common practice is to resist the change. Those considerations are going to be outlined in this research study.

Conclusion
Evaluating the use of any educational tool or system is considered a demand for future development. Many evaluation studies have researched the obstacles and challenges of using and adopting e-learning, and less has been oriented towards evaluating the current engagement, challenges and obstacles facing such engagement. This research study evaluated the level of use of e-learning systems in 12 different public and private universities in Jordan. The evaluation methodology was based on using CBAM's LOU framework by conducting semi-structured interviews. The first group was identified as 46 faculty members that are using e-learning and using CBAM's LOU framework they were identified as mechanical users. The mechanical are perceived in CBAM's LOU framework as users having their attentions on shortterm, daily use of the e-learning with little time for reproduction. Their effort is mainly directed toward learning tasks compulsory to use the educational tool. Use is often fragmented and shallow. Moreover, the main challenges associated with this group were identified as such activities in interacting with e-learning are adding more load to their teaching and research responsibilities. On the other hand, a group of 73 faculty members that did not use or interact with e-learning system were evaluated using CBAM's LOU matrix. The results managed to identify majority of users as being in the orientation (knowledge) category. The orientation level shows that users are obtaining some information about e-learning and might discovered some of its value and what commitment it needs to involve. The major obstacles facing this group was related to not having enough information about this educational technology, and the need for their educational institutions to recognize the efforts and responsibilities associated with using and managing e-learning. It is important to understand that Educational institutions in Jordan are not having official evaluation for the use of e-learning, and it is evident from the result that this tool showed the competence in providing measurement for level of use in systematic way that is accepted widely in educational field and research. Thus the methodology and output is believed to help in providing better policies for enhancing the adaptation and minimize the challenges of using this educational technology.