Usability Testing of a Collaborative and Interactive University on a Mobile Device

—The use of mobile devices for delivering learning tools is an attractive concept. Termed mobile learning (m-learning), this new technology allows people to participate in learning activities without being tied to a fixed location and provides users with convenient and flexible access to learning resources anytime and anywhere. While many m-learning applications have been developed to date, most provide tools to help students’ with specific learning tasks rather than a general interface to online courses. Few support online learning communities or allow users to download multimedia learning content. These features would engage mobile users and enable them to interact with one another, thus allowing them to participate in group learning activities despite their changing location. In this article, we describe an m-learning system which we have developed which aims to incorporate these facilities. This system provides access to multimedia learning resources and supports mobile users in an interactive synchronous learning environment with their desktop peers. Details of the evaluation techniques which we utilised to appraise the system are provided and the results are presented. Feedback suggests that the features offered by our system are beneficial for collaborative m-learning.


INTRODUCTION
The emergence of mobile technologies has made it possible to deliver a diverse range of services to users through their mobile devices. Coupled with wireless networks, these devices have potential to bring powerful applications directly to the user. Their portable nature makes them convenient for many to use while on the move and therefore extends the boundaries where people can access digital data. A wide variety of services are now available for tablet computers, Personal Digital Assistants (PDAs), smartphones and mobile phones. The mobility of these devices mean that they are particularly well suited to location-based applications such as tourist guides and navigation aids. In addition, WebPages are being designed specifically for these devices so that it is now easier than ever to access facilities such as online banking and online retail while not in a fixed location.
Mobile technologies also show great potential within the area of education. Through mobile devices it is now possible to access learning services and content anytime and anywhere. The term mobile learning (m-learning) has been introduced to describe this concept and much research is ongoing examining the various ways that these devices can be used to effectively aid students in their learning. Firstly, the provision of administrative services has been explored which allows students to remain up-todate with course information. Through their mobile devices, students can access information regarding course registration, timetables, contact details of lecturers, assignment deadlines and assessment grades for example. Applications have been developed for the PDA platform which deliver this information [1], [2] while Short Message Services (SMSs) have been widely employed to reach students through their mobile phones [3], [4]. Some universities have also examined the use of SMSs to reduce dropout rates from their courses by sending messages of encouragement to students [5]. Feedback regarding these facilities has been very positive and, in particular, students found the use of SMSs convenient and liked the personal content they provided.
Much research has also examined the potential of these mobile platforms for delivering learning content. Course material can be presented in a variety of ways whether it is in the form of short revision notes or quizzes sent via SMS, more detailed Multimedia Message Service (MMS) or course notes downloaded across a wireless network. The M-learning project (www.m-learning.org) developed a wide range of learning tools for mobile phones which delivered learning content to users through interactive SMS quizzes and Java-based games [6]. These tools were tailored for use on PDAs and were primarily used to develop aspects of literacy and numeracy in young adults who were no longer taking part in formal education. Learning modules to teach additional topics such as languages, health and safety, the driver theory test and first aid were also developed. The MOBIlearn project (www.mobilearn.org) explored the possibility of delivering context-based learning content to users via mobile devices [7]. They developed a system architecture capable of delivering content to users which can be personalised according to their preferences, previous interactions with the system and their current location. Learning material is also customised for different device types and network connections. Learning scenarios were developed for use on PDAs, mobile phones and tablet PCs using this architecture. The Swedish telecommunications company, Ericsson (www.ericsson.com), are also involved in mlearning research projects. They focus on the development of course materials for display on mobile devices [8], [9]. Their work integrates technologies such as web browsing, streaming audio and video, SMS and MMS for presenting learning material to users through their PDAs, Smartphones and mobile phones. Evaluation studies for these research projects show that users responded well to these m-learning services. They showed great interest in the possibilities that mobile devices could offer for learning and found the learning tools a useful and enjoyable way to learn [10], [11].
While these research studies have made major contributions to the area of m-learning, most of the applications that have been developed for mobile devices to date have been tools which support individual learning where users 24 http://www.i-jim.org access course information or work through learning content alone. Little support for interaction among users is provided and thus collaborative work among students is limited. The benefits of supporting collaboration among students and of creating a learning community for online learners have been highlighted in many studies [12], [13], [14], [15], [16], [17]. Ultimately users feel less isolated in online learning courses when they are aware of others, and feel more comfortable in an e-learning environment when they know that help and support is available if required [15]. In addition, communication among students in an online learning community enables them to participate in group learning activities. This allows students to construct knowledge together, to further their understanding of topics through group discussion and to develop their problem solving abilities [16]. Our research examines the possibility of incorporating mobile users into an online learning community with their peers. In particular, we aim to develop an mlearning tool which allows mobile users to participate in synchronous learning activities with others. Additionally, given the popularity of online 3D environments such as Second Life (www.secondlife.com) and Active Worlds (www.activeworlds.com), together with previous research studies into the benefits that this type of environment can provide within the e-learning domain [18], [19], [20], [21], we also examine the use of 3D Graphical User Interfaces (GUIs) for presenting the learning tools to students. This type of interface supports the incorporation of multimedia features into the system design and provides a stimulating and interactive learning experience for users. This article describes mCLEV-R, the mlearning system which we have developed, and details a number of evaluation studies which we carried out to ascertain the effectiveness of the collaboration and multimedia tools provided.
II. MCLEV-R mCLEV-R is the m-learning interface which we developed as part of the Collaborative Learning Environments with Virtual Reality (CLEV-R) research project. This pro-ject examines the use of real-time communication technologies and 3D GUIs for creating interactive online learning communities. A client-server architecture which incorporates a number of different components was developed to support these technologies. This architecture is displayed in Fig.1. Clients can connect to the server side components to access various learning resources and to interact in real-time with other users. An e-learning interface was developed which allows users to connect to the system through desktop computers and laptops [22], while our m-learning interface allows users to access the learning facilities through their PDAs. The e-learning interface, shown in Fig.2, provides access to learning resources through a multi-user 3D environment [22]. Real-time text and audio communication methods are incorporated to allow users to interact with one another and participate in group activities. mCLEVR was developed to act as an accompaniment to this desktop interface, to allow mobile users to participate in learning activities. Similar to the desktop interface, mCLEV-R utilises 3D and synchronous communication technologies to present the m-learning tools to users. The mCLEV-R interface is presented to users through a series of webpages. It consists of 2 main components; the 3D environment and the communication interface. These components provide access to learning resources and allow mobile users to participate in real-time communication with other members of their course.

A. The 3D Environment
The 3D environment provided in the desktop CLEV-R interface is a large scale university setting with many rooms where lectures, group meetings and social activities can take place. Many users can be present in this environment at the same time and each user is displayed as an avatar [22]. While PDA's are currently not capable of supporting a multi-user large scale 3D environment like this, we wished to examine the use of 3D GUIs on the mobile platform. This would provide greater consistency between the desktop and mobile applications and also provide some valuable insight into the use of 3D GUIs on mobile devices. Thus the 3D environment provided in mCLEV-R is a single-user office style environment which provides access to learning resources. Users can navigate around the environment and interact with a number of 3D objects to access learning material, communication facilities and course announcements. The 3D environment provided in mCLEV-R is displayed in Fig.3. Fig.3(a) displays the download section of the 3D environment. From here, students can access course material uploaded by a tutor. They can download available files directly to their device across a wireless network and view them using an external application such as Pocket Acrobat Reader, Conduit Pocket Slides or Pocket Microsoft Word. Music and video files are also supported. The area of the 3D environment shown in Fig.3(b) provides access to other mCLEV-R features including the communication interface, course announcements and help facilities.

B. The Communication Interface
The communication interface of mCLEV-R provides access to the synchronous communication facilities. It allows mobile users to communicate in real-time with their course tutor and peers, and plays an important role in creating a sense of social presence for mobile users within a learning community. Real-time text and audio communication, which can be used for participating in both social and learning activities, are supported in mCLEV-R. These facilities are linked to the text and audio communication features provided in the desktop CLEV-R system [22], thus mobile users and those connected through the desktop interface can interact with each other. The text-chat facility allows mobile users to communicate with other users of the CLEV-R system using short text messages. As shown in Fig.4, users can send short messages to one another and the message dialogue is displayed for all users. The dropdown menu provided is used to indicate the intended receiver of the message. Users can send public messages to all other connected users by selecting 'Everyone' on this menu. Alternatively, they can select a user's name to send them a private correspondence. This list is an important feature because it ensures that all users are aware of the presence of others.
The mCLEV-R audio communication feature, displayed in Fig.5, enables mobile users to broadcast live audio streams from their microphones and to listen to broadcasts of other users. A number of broadcasting 26 http://www.i-jim.org streams are available which represent various locations in the 3D environment used for the desktop CLEV-R interface [22]. When mobile users select an area which they wish to listen or broadcast to, using the dropdown menu provided, a list of all users currently in that part of the 3D environment is displayed so that mCLEVR users are always aware of those listening to their broadcast. mCLEV-R is implemented using half-duplex data transmission. This means that only one user can broadcast to a particular stream at any one time. If a user is broadcasting to a stream, the 'Talk' button will be disabled and a 'Busy' status is displayed for all other users connected to that stream. Information regarding the current speaker is also displayed. Fig.5(a) displays the audio communication interface when the audio controls are available, while Fig.5(b) shows it in a busy state when the controls are being utilised by another user. The communication interface also provides access to other mCLEV-R components. The 'Home' and 'Help' buttons in the centre panel of this interface provide access to the 3D environment and the mCLEV-R help files respectively. The button labelled 'Hand' was added to allow users to signal to the teacher that they have aquestion. Finally, the 'Exit' button logs users out of the system. III. EVALUATION In order to determine the overall usability of mCLEVR and its effectiveness for m-learning, we carried out a number of evaluation studies. In particular 2 user evaluation studies were employed to gauge user reaction to the concept mCLEV-R presents and the functionality it provides. This section discusses the techniques used in these evaluation studies while the next section presents the results obtained.
The 2 user evaluation studies examined the usability of mCLEV-R, users' attitudes towards the interface and its effectiveness for m-learning. The International Standards Organisation (ISO) defines usability as 'The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use' [23]. mCLEV-R was evaluated in accordance with this definition. Both user evaluations required the test-subjects to participate in a number of predefined tasks using mCLEV-R and provide feedback by means of a number of different questionnaires. The user evaluations were carried out in conjunction with the evaluations of the corresponding desktop interface. Thus, test-subjects were able to communicate with both mobile and desktop users during the evaluation sessions.
4 tasks were designed to ensure users were exposed to all features of mCLEV-R in a variety of different learning scenarios.

A. Task 1 -Social Interaction
The main aim of this task was to introduce users to mCLEV-R and, in particular, to the communication tools it provides. These communication facilities form the basis of all user interaction and collaboration within mCLEV-R and so it was important that users quickly became familiar with them. The task requested users to access the communication interface through the 3D environment and connect to the audio broadcast for one of the social rooms where all desktop CLEV-R users were located. All users participated in an icebreaker game using the text and audio communication facilities to converse with one another. They also accessed the announcements page during this session and collaborated with one another to decide on the topic of a group project which they would present through mCLEV-R in Task 3 of the evaluation study.

B. Task 2 -Online Lecture
This task requested users to attend a synchronous online lecture using mCLEV-R. Firstly, users were asked to download the lecture material through the 3D environment and open the lecture slides in an appropriate external application. They then accessed the communication interface and connected to the audio broadcast for the lecture room where the tutor would present the learning material. Test-subjects had to listen to the live lecture and follow the learning material simultaneously. They could ask questions via the text or audio communication facilities as desired. The objective of this task was to evaluate mCLEV-R as a means for accessing and downloading learning content, and to explore its use for attending and taking part in synchronous online learning activities.

C. Task 3 -Collaboration
Task 3 involved users participating in a group meeting where they presented and discussed the group project chosen in Task 1. The objective was to show users the support mCLEV-R provides for group learning activities such as project meetings or discussion groups. All users connected to the audio broadcast for a specified meeting room. Each user presented their findings on their chosen topic using the audio communication facility and listened to the findings of others. A group discussion on the presented material then took place.

D. Task 4 -Free Session
This final task consisted of a free session where users could further explore the various features of mCLEV-R and interact with others as they wished. The main objectives here were to ensure that users experienced all the different features of mCLEV-R and to observe ways in which users would interact informally within the learning environment.

E. Evaluation Setup
The first user study was carried out on completion of a first working prototype of mCLEV-R. Its aim was to discover any usability issues that had not arisen during the development process. A number of usability problems were highlighted relating, in particular, to the design of the communication interface. These issues were addressed before the second evaluation was carried out. Further details on the first user study and the results obtained can be found in [24]. In this article, we focus our attention on the results of the second evaluation study. In total, 12 users participated in this user evaluation. While a greater number of users would have been desirable, it was difficult to iJIM -Volume 3, Issue 4, October 2009 obtain volunteers to participate in the user trials. The users consisted of 9 males and 3 females and included 9 university students, 2 professionals and 1 secondary school teacher. The majority of these users were university students since they are the target users of mCLEV-R. Further details regarding the user sample are shown in Table 1. All users had good to excellent computer skills. This level of computer literacy among university students is realistic nowadays as the majority use computers frequently throughout their studies. Thus, the sample set is a reasonable representation of the prospective end-users. 2 users also had previous experience of using personal organiser tools, Geographic Information Systems and web browsing on mobile devices. In order to receive feedback from users about their experience with mCLEV-R, a number of different questionnaires were used. Firstly, a task evaluation questionnaire was designed which was filled in on completion of each task. This consisted of the After Scenario Questionnaire (ASQ) and some questions designed specific to mCLEVR and the functionality it provides. The ASQ, which was designed by IBM [25], is a 3 item questionnaire used to determine user satisfaction upon completion of a task. In particular, it examines their satisfaction regarding the ease of completing the task, the time taken to complete the task and the support information available when completing the task. It is designed using a 7-point likert scale anchored at 1 by Strongly Agree and at 7 by Strongly Disagree. [25] and [26] describe a psychometric evaluation of the ASQ which shows that the 3 items can be condensed to one scale through summation, thus giving an overall satisfaction score for the task. The additional questions of the task evaluation questionnaire dealt with individual features of mCLEV-R and user feeling towards their effectiveness for carrying out the task in question.
An overall evaluation questionnaire was also distributed to users after their last session with the system. Again, this consisted of a number of different components.
The Computer System Usability Questionnaire (CSUQ), also developed by IBM [26], consists of 19 questions which assess user satisfaction with overall system usability. It can be divided into 3 subscales, namely System Usefulness (questions 1-8), Information Quality (questions 9-15) and Interface Quality (questions [16][17][18], to obtain feedback on these aspects of the system. The final question relates to overall satisfaction with the system, and so does not fall into any of these subcategories. Similar to the ASQ, this questionnaire is designed using a 7-point likert scale anchored at 1 by Strongly Agree and at 7 by Strongly Disagree. The CSUQ and its 3 subscales can also be condensed to a single scale score through summation [26]. Another component on which the overall evaluation questionnaire was based was the Microsoft Product Reaction Cards [27]. They were developed by Microsoft as a means of determining the desirability of a product or system. They consist of 118 words of which approximately 60% are positive and 40% are negative. Users are asked to select the words that they feel best describe the system or the feelings they had when using the system. This provides valuable qualitative feedback regarding users' reaction to the system in a short amount of time. Finally, some questions specific to mCLEV-R were designed to receive feedback on different aspects of the system including the communication interface, the 3D environment, users' sense of presence in an online community and general thoughts regarding mobile devices for learning. Additional user comments were also encouraged.

IV. RESULTS
This section discusses the results obtained using the evaluation questionnaires described. User reaction in relation to the effectiveness of mCLEV-R for completing each of the tasks, the overall usability of the system, their social presence with an online learning community and the notion of collaborative m-learning are presented.

A. Task Results
The average ASQ usability ratings given by participants for each of the evaluation tasks are shown in Table 3. This score can range between 1 and 7, where 1 is positive and 7 is negative. Thus we can see that all 4 tasks were rated positively by the test-subjects.
In general, the users were satisfied that the tools provided by mCLEV-R are effective for participating in each of the different learning and social scenarios. Some users did experience difficulty using the device input methods of the PDA. In particular, they found inputting text using the virtual keyboard and stylus slow and cumbersome. These users were, however, inexperienced with mobile devices prior to this evaluation and once they become familiar with this input method, their typing speed should improve significantly. Another issue raised by testsubjects was that they disliked navigating between different applications to view lecture material and access the communication facilities. However, the screen size of PDAs is a severe limitation on the amount of information that can be displayed at any one time. It is thus not possible to provide the functionality of mCLEVR without alternating between different applications. Despite these issues, all users successfully completed each task and actively participated in group activities.

B. Computer System Usability Questionnaire
The CSUQ was used to determine the overall system usability. Table 3 shows the average usability rating given by participants for the system, as well as the System Usefulness, Information Quality and Interface Quality subcategories. Once again this score ranges between 1 and 7 where 1 is positive and 7 is negative.

28
http://www.i-jim.org mCLEV-R received a positive overall usability rating. However, the high standard deviations displayed in Table  4 suggest that user opinion varied and that the averages were somewhat affected by negative feedback from some users. The participants' overall satisfaction with mCLEV-R is shown in Fig.6. 11 of the users agreed that they were satisfied with mCLEV-R giving this question a rating between 1 and 3 on the 7-point scale. Only 1 user disagreed strongly with this. This user was unfamiliar with PDAs prior to this evaluation study and strongly disliked the mobile platform. They felt that the screen size was insufficient for effectively displaying information to the user and that the device input methods were restrictive. They found mCLEVR frustrating to use and disliked alternating between different applications. They did, however, comment that an experienced PDA user might be better able to use the interface and that the functionality provided by the system would be useful when it is not possible to login on a desktop computer. The 3 subscales of the CSUQ are now further explored for this evaluation study.

C. System Usefulness
The 12 participants of this evaluation study gave mCLEV-R an average system usefulness rating of 2.78. However, average individual ratings for the 8 questions in this subcategory ranged from 1 to 5.5 as is reflected in the high standard deviation of 1.421. Some results from this section of the questionnaire are analysed in Fig.7. The majority of participants were satisfied with the ease of using mCLEV-R and 10 agreed that it was easy to learn how to use it. While 9 agreed that they could effectively complete their work using the system, only 7 felt that they could complete their work efficiently. The remaining users felt the stylus interaction method was slow, particularly for typing messages, and that switching between various applications was time consuming. This is primarily due to their inexperience with mobile devices and as they become more familiar with these devices, their interaction speeds with mCLEVR should improve. Despite this, 9 felt comfortable using the system during their mlearning sessions.

D. Information Quality
This category is concerned with the quality of user support provided with the system including error notification, user instructions and help files. The average rating given by users for this subscale was 2.65 with a standard deviation of 0.996. Error notification proved the most negative feature of the system within this category. As shown in Fig.8, 6 users rated this feature poorly, while 3 did not respond to this question. Many users did not experience any error messages and some encountered problems which they expected to be reported as errors. However, many of the problems encountered by users during the user sessions were not directly related to the system. Instead they were device related issues such as difficulties navigating between different applications on the PDAs or problems using the stylus for interaction. For example, some users accidentally tapped the stylus on hyperlinks within the 3D environment and as a result were redirected to the webpage of another mCLEV-R component. While it would be inappropriate to report an error in this case, the problem is dealt with in other ways by ensuring that all mCLEV-R webpages have adequate support for navigating between different system components. Despite this, 8 of the participants agreed that they recovered from mistakes quickly and easily.  (1) In relation to information provided with the system including on-screen messages, help files and user instructions, overall feedback was more positive (see Fig.9). 8 users felt the information was clear and 11 considered it easy to understand. The majority (9) found it effective in helping them complete their work, while 11 felt that the organisation of information on the screen was clear.

E. Interface Quality
An interface quality rating of 2.47 was determined for mCLEV-R. Fig.10 displays the feedback for this category and as shown user response was generally positive. 10 of the test-subjects considered the system interface pleasant with only 1 user stating that they disliked using it. 10 of the users agreed that the system has all the capabilities they would expect it to have. Suggestions from the remaining users include adding functionality for full-duplex audio communication rather than the 'push to talk' method currently employed, and providing support for users to share live web-cam broadcasts with one another. Both of these proposals are possible within the mCLEV-R system framework.

F. Microsoft Product Reaction Cards
Overall user reaction towards mCLEV-R was also recorded using the Microsoft Product Reaction Cards. Words selected by the users give qualitative feedback of the system, highlighting their thoughts and overall impressions. In total, 84 out of the 118 words were selected to describe mCLEV-R. Of these, 74 (88%) were positive and 10 (12%) were negative. Table 4 shows the most highly occurring words selected by users. As shown, these are all positive and were selected by 5 users or more. The negative words selected had a lower occurrence (3 users or less). The words with greatest occurrence included useful, collaborative, creative and fun which are all attributes which we aimed for in our initial system design and specification.

G. mCLEV-R Features
The communication facilities provided in mCLEV-R play an important role in creating a sense of community among connected users. Through various features of the communication interface, we aim to promote awareness of others and form an online learning community allowing students to interact and learn together. Users were asked if this component was effective in achieving these goals. The majority of the participants stated that they were aware of all other users, both those connected through mCLEV-R and those using the desktop CLEV-R interface. Despite some initial problems with the device input methods, the majority of users found the text and audio communication facilities easy to use and effective for communicating with others. When using audio communication, all users found the feature displaying the names of other users in the same location beneficial. 11 users felt part of a group when using mCLEVR (see Fig.11). In addition, 11 of the users agreed that the presence of others enhanced their learning experience and kept them engaged in the learning activities. Feedback was also gathered to determine users' reaction to using a 3D interface for accessing information and system features, and to determine if it was appealing and engaging for the user. 11 users found the 3D interface appealing and 9 found it easy to navigate within it on the PDA. All participants successfully accessed the communication facilities, announcements page and downloaded course material through the 3D interface and 9 of the users rated the intuitiveness of accessing these facilities positively. In comparison to a text-based interface, 10 found the 3D environment more engaging. While users did experience some difficulties with the system during this evaluation study, their response to the system as a whole was very positive. All test-subjects see the benefits in providing learning tools for mobile users. Further feedback regarding the use of mCLEV-R for m-learning is displayed in Fig.12. 10 agreed that the interface provides a good medium for learning and 11 feel this is especially 30 http://www.i-jim.org true when it is not possible to be present in a fixed location. 10 out of the 12 users stated that they would use mCLEV-R if it were provided with an e-learning course and that they had a high enjoyment level when using the system. Figure 12. Users' Feedback: mCLEV-R for mLearning DISCUSSION AND FUTURE WORK The emergence of mobile devices has led to an interest in the development of applications which provide various services for mobile users. M-learning is one such service and many applications which provide learning tools have been developed for a multitude of mobile platforms. Our research also examines this area. In particular, we aim to provide an m-learning tool which supports mobile users within an online learning community. This article discusses our system, mCLEV-R, which provides facilities for mobile users to access multimedia learning material and allows them to converse and learn together with their peers through a 3D interface and synchronous communication methods. Many different aspects of mCLEV-R have been validated including its overall usability, user reaction to the 3D interface, the effectiveness of the communication facilities for learning and social activities, and its value as an m-learning tool. Overall the results are very satisfactory.
In general, users enjoyed the available functionality and agreed that mCLEV-R would be a useful system for learning on the move. The synchronous communication methods created a sense of community among the participants and proved sufficient for supporting collaboration among them. Moreover, users responded well to the 3D interface and found it more engaging than a text-based alternative. Navigating between various applications and inputting text via a virtual keyboard and stylus were the main difficulties experienced. While such issues cannot be overlooked, they are more related to users' inexperience with PDAs than to the mCLEV-R application. These results suggest that the functionality provided by mCLEV-R is beneficial for m-learning. Access to learning material while on the move is in itself an extremely useful feature, however, when coupled with support for synchronous communication, the benefits increase significantly. Mlearning no longer needs to be a solitary activity. Instead users can participate in group activities and get help from others as it is required.
There are a number of ways in which the mCLEVR system can be extended to provide further support for mobile users. Since mCLEV-R was developed as a prototype to provide a proof of concept of the benefits that interactive and synchronous communication technologies pro-vide for m-learning, our research focused on these aspects of the system development and did not include other features that are present in current learning management systems. Some of these features such as asynchronous communications technologies could be incorporated into the mCLEV-R system design. While the synchronous communication methods currently supported provide many benefits, they place time constraints on user interactions as they can only communicate with others when they are simultaneously logged in to the system. Asynchronous communication methods offer greater flexibility in this respect and also their lack of immediacy allows users to contemplate their contributions to the discussion before posting messages. Thus, these communication technologies may be more suitable than synchronous methods for certain learning scenarios.
Much research is currently ongoing within other areas of e-learning and m-learning. Techniques such as personalisation of learning content, adaptive user interfaces and the provision of software agents to support students are being investigated. The incorporation of these features into the CLEV-R system could also be explored. For example, user profiles detailing student preferences could be stored to allow the mCLEV-R interface to personalise content for individual users. The integration of these facilities would further enhance the learning environment supported by the CLEV-R architecture and further advance the benefits of mobile devices for e-leaning.