The Effectiveness of Mobile Learning in the Form of Performance Support System in Higher Education

—Students in engineering education need high-quality educational materials, and what is the most important, continually up-to-dated course. The students need a possibility to self-assess their skills and knowledge as the learning steps for improvement - the courses, will provide them knowledge on the newest achievements in the field of engineering. The complexity of the knowledge and skills, necessary to perform successfully the tasks, determines the learners’ needs of “practical learning”. The view of learning process, as а performance-centered, is highly effective as a means of providing students in engineering education timely and relevant information. Performance-centered educational systems have a strong potential to help students mastering job-related skills. This paper presents m-learning as a form of performance support system for educational and training purposes in higher education. m-Learning solutions integrate mobile devices within the learning process to help students perform a task by providing information, guidance, and learning experiences when and where they are needed.


I. INTRODUCTION
A review of the literature is engineering education reveals the following four major issues in delivering effective instruction: Problems in Content Understanding, Problems in Concentration and Lack of Variety, Need for Multiple Explanations, and Lack of Assistance [1]: Performance-centered approach helps in solving all these four problems in teaching Engineering disciplines. The view of learning process, as а performance-centered is highly effective as a means of providing students in professional education timely and relevant information. As a new technology, performance-centered educational systems will move the traditional teaching systems to training closely related to the job. These systems have a strong potential to help students mastering job-related skills.
Why we need to move from performance-centered educational system to mobile performance-centered educational system?
The main reason is to contribute to the continued development of mobile learning and to address the imbalance between the availability of mobile devices and the lack of education and training provision on the sophisticated communications devices which every student and actively working person carries and uses constantly -ex-cept in education. We present mlearning as a form of performance support system for educational and training purposes. The advantages for trainees are derived from providing learners with a job aid in the context of their work:  puts training and performance support where the actual work takes place  allows new skills or knowledge to be immediately applied  enables training when it is needed  allows use of rich media when appropriate The advantages for students:  students have more flexibility and choice in where and when they learn outside of the wired (or unwired) classroom.  students use the technology in their study that would enhance their readiness for tomorrow's workplace where employers want graduates who know how to use technology for learning and working.
Given the trend to lifelong learning, many "students" are working adults with full-or part-time jobs. Mobility offers them an opportunity to maximize learning time.

II. MODEL FOR PERFORMANCE-CENTERED MOBILE LEARNING (PML)
A. The concept of PML Integrating performance-centered learning and mlearning, results in a performance-centered mobile learning (PML) approach, in which students receive performance support via a mobile device when performing jobrelated tasks [2]. Important features of m-learning are that it makes learning processes more flexible and it offers the opportunity to individualize the learning material and environment. By using a mobile device that meets the aforementioned requirements (i.e., small, easy to carry) the learner has access to information at the right time (i.e., just-in-time), for the specific learning situation at hand (i.e., just-in-case), which fits his or her need (i.e., just enough) and fits his or her learning style (i.e., just-forme). These features are in line with the aims of performance-centered learning. Performance-centered m-learning (PML) (i.e., mobile performance support systems) are similar to traditional PSS. That is, m-Learning solutions integrate mobile devices within the learning process to help students perform a task by providing information,

B. Instructional design for performance-centered mlearning
It is important that the existing course is developed according to the principles of performance support for learning [3]. First, it is important that the performance in education that is supported is related to performance of students' future working environments. During the course students should be confronted with and trained for situations they will also encounter in their future profession:  Identify the reference situation of a particular course. These are the professional settings where students are going to apply in practice what they have learned during the course.  Define a set of authentic problems and develop tasks related to a specific working environment.  Shift the instructional focus from knowledge and understanding (i.e., the lower levels of the learning taxonomy), towards solving real-world problems (i.e., the higher levels of the learning taxonomy).  Applying adequate summative performance-oriented assessment methods.
Second, to design support for students to perform these authentic problems and especially learn from performing these tasks, the following instructional guidelines can be formulated:  Sequence the learning tasks from easy to complex.  Create opportunities for deliberate practice of the tasks.  Gradually diminish the amount of support provided to perform the task (scaffolding).  Provide a variety of instructional stimuli (resources) such as background information, examples and procedures.  Allow constant access to learning resources.  Provide formative performance feedback.  Adapt the instruction to the level of knowledge and learning style of the students. This can be done by the system or the students themselves.
Third, students' learning will profit from adapting the type and amount of performance support to the students' needs and learning style. This adaptation can be done by the system or by the students themselves.
Mobile performance support systems can be implemented in education according to four possible instructional design scenarios: 1. Mobile performance support courseware -mobile learning is interpreted as mobility of the content. A course provided via a wired device is simply repackaged and transferred to a wireless mobile device 2. Industry-based mobile performance support systems -performance support system is implemented in higher education in a similar way as the support system is implemented in industry 3. Mobile social support systems -Learning as a lifelong activity implies that it may happen anywhere and anytime. The advent of technology and its impact in our daily life provides an opportunity to support learning in different situations, irrespective of time and location 4. Integrated mobile performance support learningmobile performance support is part of a blended solution of knowledge distribution rather than a primary channel for content delivery. This scenario is based on the assumption that mobile learning is effective if considered as only one of the components of the overall programme of learning interventions. Another assumption that underlies this scenario is that when applying mobile performance support systems in education, instructors should use different materials, devices and resources as building blocks to develop a learning design. For each type of information the most suitable information, delivery device and resource should be selected. This means, that only when delivering support through a mobile device has an added value for learning mobile performance support should be implemented Practical instructional design of performance-centered m-learning deals with: small screens, graphics and illustrations, problems with data input and assessments.
III. IMPLEMENTATION OF PML -REALIZATION OF SCENARIOS 4, MDIPSEIL SYSTEM For PML implementation a mobile learning (mlearning) environment was developed, called mDIPSEIL. This learning environment was based on the existing elearning environment (Fig.1), DIPSEIL -performancecentred e-learning management system http://env.dipseil.net. The instructional design of the DIPSEIL and mDIPSEIL courses was based on the principles of performance support system (PSS), including a task description, task-specific training, reference information, and expert advice.
There are two user types: Students are the learners of the educational content. They use their mobile devices to login to a simplified web-based system. The mobile device should have an HTML-supporting web browser. Before they login, they should have an account in the DIPSEIL system. The mDIPSEIL provides a simplified interface for studying the educational content. The main informational content, such as tasks list or learning materials, is displayed at the centre of the screen, wrapped according to the resolution of the mobile device. Navigation menu is found underneath the informational content and helps the student navigating through the educational content; Teachers are users, who develop, upload and organize educational materials in mDIPSEIL system. They can reorganize courses, modules and tasks developed, upload, delete objects in the system and add or remove them from their task specific objects.
The environment from students point of view, from GSM, iPhone, PDA, Smart phones -http://m.dipseil.net The student has to be registered in DIPSEIL and the modules from the courses have to be chosen, student's course building. Selecting a task and choosing "Learn Selected" will show the Task Description. The system's menu is always at the bottom of the screen. Use it to navigate through the different learning objects and to use available instruments. If there are videos or sounds, they will be presented as links, which allow you to download the multimedia content and study it. On Fig. 4 one can see examples with screens from the course "Information Technology".

A. Performance-centered m-learning environment evaluation
To evaluate the effects of a system it is important to evaluate the effects of the system on the learning processes for which it is designed [4]. We have performed pilot studies to investigate what effects mDIPSEIL had on learners' performance (i.e., an objective measure) and how learners perceived the effectiveness of the system (i.e., a subjective measure). It was hypothesized that mDIPSEIL would help learners to adapt the learning environment to their needs and get just-in-time information, which would have a positive effect on their learning outcomes. Regarding the perceptions of the users it was hypothesized that the feeling of being in control of when to learn, what to learn and to ask for support when needed, would have a positive effect on students' performance.
We give here the procedure, participants and results from one of our pilots, performed at Plovdiv University 2009/2010 study year.  Participants were 37 first-year students (21 male, 16 female; M age = 20,7; sd = 2,7) who were enrolled in the course Information Technology at the Plovdiv University in Bulgaria. Students voluntarily decided whether they wanted to use mDIPSEIL to study the material (i.e. experimental condition; n = 21) or to use DIPSEIL for this purpose (i.e., control condition; n = 16).
Learning style questionnaire. To measure students' learning style, the short 40-items version of Learning Style Questionnaire (LSQ) of Honey and Mumford (1992) was used. The questionnaire helps to indicate whether a student is an activist (i.e., prefers the challenges of new experiences and problems), a reflector (i.e., likes to first watch and think about actions and to review them afterwards), a theorists (i.e., think systematically step-by-step problems), or a pragmatist (i.e., likes to apply new learning to actual practice).
Learning outcomes. Students' learning outcomes, were measured per task and by an overall score. Students' scores (min 1, max 10 per task), for the 10 performance tasks they completed during the course were gathered. In addition, students received a final, overall score (min 2, max 6) on the course, computed by the teachers of the course. The 10 grade scores were transformed into 6 grade scores, because this is the official assessment scoring system in Bulgaria.
Reflective questionnaire. A reflective questionnaire was developed to gain more insight into the perceptions of the students on several topics. These topics were (a) the usability of mDIPSEIL (e.g., it was easy to view the graphs), (b) the efficiency of mDIPSEIL (e.g., I could learn whenever I wanted), (c) the effectiveness of mDIPSEIL on their learning (e.g., it helped me to learn better than with traditional learning), and (d) the effects on their motivation (e.g., it was fun and appealing). Furthermore the questionnaire was aimed at getting some information about the mDIPSEIL performance support system, asking how helpful they found the support. Finally students were asked for their appreciation of mDIPSEIL in terms of action.
The questionnaire's items were formulated as statements and students were asked to indicate the extent to which they agreed with the statements on a 4-point Likert scale (1 = I totally agree, 4 = I totally disagree).
Think aloud protocol. To identify possible issues or problems with mDIPSEIL, eleven students had to perform the task 'Routing', using mDIPSEIL on their mobile device. They had to access the course, find the task, try all options for performance support and send the assignment back to the teacher. While performing the task students got the instruction to describe verbally what they are doing and experiencing. Students' behaviour was recorded and their utterances were transcribed.
During the first week of the course, all students filled out the 40-items Learning Style Questionnaire. Next, during 4 weeks, students studied the course material using either mDIPSEIL (i.e., on their Smartphone or iPhone) or DIPSEIL. Students in both conditions followed the course only at their university, in a classroom in which two teachers observed and assisted the students if necessary. At the end of the course all students in the experimental condition filled out the reflective questionnaire. In addition, eleven students in the experimental condition performed a think aloud protocol.
The learning styles were measured for the experimental group only. The majority of the students (n=8) appeared to have an activist learning style. Only one student scored a theorist learning style. The other students had a pragmatist (n=5) or reflector (n=7) learning style (Fig.5).
Students' learning outcomes. Students in the experimental condition outperformed the students in the control condition in 7 of the 10 tasks, whereas the students in the control condition had a slightly higher overall score (Fig.6). All differences between the conditions were not significant. In addition, no effect of learning style was found.
Students' perceived effectiveness: questionnaire. Regarding usability, students indicated that it was easy to navigate through mDIPSEIL and to view graphics and presentations. With respect to the effectiveness students agree that mDIPSEIL helped them to learn the material better and more often. They disagree that it hampered their learning. However, also seven students indicated that mDIPSEIL did not fit their way of learning. When being asked specifically about the helpfulness of the performance support system components, students indicated that all components were helpful, except for the expert advice. With respect to efficiency, students agree that mDIPSEIL allowed them to learn whenever and wherever they wanted and when needed. Regarding motivation students even strongly agree that learning with mDIPSEIL was fun and appealing to them. Regarding the appreciation of the course in terms of actions, students would recommend others to take a course in a m-learning environment or to do so themselves. They also disagreed that the course would have no added value. However, most striking was, that students also agree that they would have preferred a traditional course for this course material. No significant differences were found regarding the responds to questionnaire of students with different learning styles.

Students' perceived effectiveness: think aloud protocol.
The think aloud protocol also provided insight into the perceived and experienced effectiveness of mDIPSEIL. The analysis of the transcriptions and the interviews with the observers of the think aloud protocol did not show any problems with using mobile devices in this course. All students were able to accomplish the task efficiently and successfully.
The data and the whole investigation could be found on http://mpss.dipseil.net. Project results section, Evaluation report from the pilots.

A. Conclusions from the performed investigations and in the light of the hypothesis
It appears that learning the course on a mobile device does not contribute more to students' learning than elearning does. However, these results also indicate that it does not impair or hamper learning. Learners performed very well. This was also confirmed by the students who indicate that the use of a mobile device did not hamper their learning. The absence of a significant contribution to learning might be explained by the fact that the implementation scenario used in this study, is one of the most straightforward scenarios. That is, the only difference between the control and experimental group is the use of a mobile device. The content is the same, only the medium used differs. A more sophisticated implementation approach in which, for example, the mobile device would provide the student with information just-in-time, justwhen-needed, while performing a learning task, would have made m-learning more beneficial for learning compared to e-learning.
Regarding the hypothesis about perceived effectiveness, the hypothesis was mainly confirmed. Students agreed that mDIPSEIL was easy to use, helped them to learn better, and whenever or wherever they wanted and its use was motivating. Also regarding the use of the components of the performance support system, students indicated that they use them frequently and that they are helpful.
Overall the students positively appreciated the mDIP-SEIL and perceived it to be effective to learning. However, some answers of the students in the questionnaire and interviews show that students are not completely convinced of the effectiveness and positive effects of mDIP-SEIL on learning.
The most important lesson learned from the studies conducted is to assure that the way m-learning is implemented and designed in a course, has an added value to the learning of students. In the current studies, m-learning is interpreted as mobility of the content, enabling learners to consult information resources, ask for advice or any other type of support whenever they want and wherever they need it. With respect to instructional design, this scenario requires simple repackaging performance support that is already available on a wired device and transferring it to a wireless mobile device. It is important that the existing course is developed according to the principles of performance support for learning.

B.
How we use these conclusions in on-going work Using our experience and investigations in implementation of performance-centered approach in mobile learning, we are going further to the development and mainstreaming Europe wide skills recognition and qualification service for m-learning job roles -in management, development and other individual management level charged with evaluating sourcing or implementing m-learning solutions. We started with m-Learning Manager job role in the context of growth of m-learning because of m-technology development and availability, and qualifications and programs paucity at managerial (as premise and stimulus to design and development levels) levels in the field.
Here we give answers to some possible questions, which could be raised: Why "m-learning"? There never was a technology as widely available to citizens as mobile telephony. This technology connects people working at different places and different education and learning paths with opportunities for expert and peer feedback and co-learning. Mobile technology offers unprecedented possibilities for combining the strengths of formal and non-formal education and professional internship. For the first time in the history of the use of technology in education and training, is a technology that will cost the learners nothing, because they own the technology to be used.
Why "m-learning job-role"? The new mobile learning arena imposes significant new design requirements of training programs -the ways they are structured and maintained. The effective m-learning imposes specific usability requirement. The assessment of the mobile learning in terms of learning outcomes is similar in all VET systems but techniques in m-learning are specific. The validation of the assessed formal and non-formal m-learning should be done in accordance with the common European principles. The quality assurance should be an integral part of the management of m-learning providing institutions.
Why "m-learning manager job-role"? Our study shows that managerial level in the field will acts as premise and stimulus to development of job-roles in the design and development levels.
Why "m-learning job-role now"? The m-learning is emerging as a new sector in education and training provision, side by side with face-to-face education, distance education and e-learning. Just as distance education was recognized and accepted as a field in the 1970s when the great European Open Universities were founded and elearning was accepted as a field in 1995 when the World Wide Web first became available to educators. We can say that we have been in the process of acceptance of mlearning from the beginning of the 21st century, along with 3G/UMTS and Smartphone.