Integration of remote and virtual laboratories in the educational process

This paper describes a new holistic approach to integrate remote and virtual laboratories in the educational process. Aim of the integration is to test students' knowledge not only on lower levels as usual multiple choice tests and the like are doing, but also on higher-order thinking skills. Based on these tests the students should get a detailed feedback about their strengths, weaknesses or mis-conceptions. Therefore Learning Analytics methods are applied to a database, storing learning activities of the students during their online work- phase in the educational process. The paper discusses the actual approach as well as the architecture of a prototype implementation.


INTRODUCTION
The assessment of students thinking skills during e-learning processes is one of the main research directions in e-learning. Goals of this research are to qualify the assessment students' knowledge at different levels of thinking skills. to give students feedback about their learning status as early as possible during the learning process to personalize the feedback as well as the learning methods considering the individual learning status. Assessment can be categorized as diagnostic, formative and summative, formal and informal, final and continuous, process and product, divergent and convergent as well as group-, peer and self-assessment [1]. In our research we concentrate on diagnostic, formative and summative assessment to support the whole learning process. While diagnostic assessment is used as pre-assessment, formative assessment takes place during the educational process and summative assess only the final status of the learning outcomes.
Aim of the assessment is to find out knowledge level that a student has reached during the educational process in relation to well-known skill taxonomies (e.g., Bloom's or Anderson and Krathwohl's taxonomy) [2].
Actual learning management systems support assessment mostly in the form of multiple choice questions and the like. These quiz types can only assess lower order thinking skill of the students (see Fig. 1). With our approach we address the higher-order thinking skills. Therefor we integrate remote and virtual experiments in the learning process. In the full paper we describe the concept and architecture of the system. The rest of the paper is organized as follows: Section II gives a short introduction to the previous and new teaching concept of the educational process. Section III describes the possibilities of our online laboratory and section IV introduces the holistic approach to support different knowledge levels as well as to accompany the whole learning process using diagnostic as well as formative and summative assessment methods.

II. TEACHING CONCEPT
The course, were the remote laboratory was applied for the first time, is called "Digital Systems Design" and is provided for Bachelor students in the fourth semester. The number of registered students for this course is about 80. The objectives of the course are getting acquainted with deeper knowledge in the field of computer engineering, the design of digital systems and hardware implementation process such as timing problems, the use of undefined values and the like. Generally, the course consists of a lecture given by one of the authors and an accompanying seminar. While during the lecture primarily theoretical basics are taught, the seminars focus on exercising the application of the theoretical foundations. The course "Technical Informatics" provides the basics in Boolean algebra and the understanding of computer internal information processing, computer architectures and organizations.
The course "Design and Validation of Complex Parallel Systems" deals with test methods for digital circuits and printed circuit boards such as design for testability, fault models, test pattern generation and boundary scan.

A. Teaching Subject
The teaching subject of the course "Digital Systems Design" is the systematic and provable development of digital control systems considering practical requirements as partial defined and non-determined functions. Therefore students are taught in suitable methods to analyze and optimize their design. In a hands-on laboratory they are confronted with a number of practical design problems that typically occur in an underspecified environment.

B. Teaching Tools
A number of highly interactive Java applets (so-called "Living Pictures") support the understanding of underlying methods, algorithms and concepts. The algorithms and methods can be explored by students own examples. Using these tools it is e.g. possible to explore the minimization of Boolean expressions. Each step of the designing process is supported by such tools. All tools are available at any time via the Internet. Actually, they are transferred to Java-Script and HTML 5 to prepare them also for a use on mobile devices like tablets and smart phones. Further details of the teaching concept can be found at [3].
We analyzed the usage of the tools and found out that some students don't use them because there are no concrete questions, tailored to their actual state of knowledge. In a totally open environment they felt lost and didn't know where and how to start. Only high motivated students with intrinsically motivation use them and thus get deeper knowledge. Personalized questions connected to these tools could help to lead the students to a broader usage of these tools. Therefor in cooperation with the Fraunhofer Institute for Digital Media Technology (IDMT) Ilmenau a supporting environment called "AskMe!" was developed. Aim of the environment "AskMe!" is to offer the students an personalized overview of their status of knowledge by recording their learning activities, faults and well done tasks. Thus they are confronted with their strengths and weaknesses. To overcome the weaknesses they get additional tasks to solve problems, based on their actual status of knowledge. For further details see [4].

III. GOLDI-ONLINE LABORATORY
For more complex design tasks the GOLDI online laboratory was developed, allowing students to design control algorithms, based on different hardware or software oriented technologies such as programming FPGAs (complex programmable devices) or microcontrollers. Fig. 2 shows a screenshot of the online laboratory. On the right hand side of the window students can edit the control algorithm. The lower part provides buttons for running the experiment as well as feedback information about correct / incorrect inputs. The left hand side of the window visualizes the running design applied to a simulation and/ or a real remote physical system.
Before starting an experiment it is possible to choose between real and virtual control devices and objects to be controlled (See Fig. 3 for a screenshot.) Therefor it is possible to test a control algorithm on a virtual device before accessing the real equipment in the online laboratory. In this virtual mode many students can make the same experiment at the same time. When using real devices in the lab, a secure one to one connection is guaranteed. The online laboratory and its features are described in detail in [5].

IV. ASSESSMENT SYSTEM
This section describes the assessment system, integrating the AskMe!-system as well as the LMS moodle and the online laboratory as interactive content object (ICO).
The assessment system extends a learning management system like "Moodle" with a component for personalized questions and tests (the "AskMe!" system), a component for data collection (the "learning record store") a component, providing a collection of interactive content objects (ICOs) as well as a component interconnecting the remote laboratory with the system. Fig. 4 shows an example of an ICO integrated in the moodle system. The students have the task to explore the interconnection between a truth table of three Boolean functions (right hand side of the figure) and the visualization of these functions as sets of indexes, representing the truth values of the functions (left hand side). Therefor they have to insert the values, given in the set diagram into the truth table. After finishing they can get feedback about right or wrong values. All actions as well as the result are monitored by the assessment system for a later estimation. Fig. 5 gives an overview of the relations between the different components of the assessment system during the educational process. The user interface to the students is the extended moodle learning management system.
Beginning with the login in moodle system all other components can be called. That means, only one login process is required to use all parts of our modular system. The three main components are developed by different authors: the eassessment system "AskMe!" developed by Fraunhofer IDMT [4], subject-specific interactive content objects (ICOs) developed in HTML5, CSS3 and JavaScript and the remote lab developed by University of Technology Ilmenau [5], [6].
All of these components have their own objective related goals. The abstract topics of these goals are defined by Bloom's and further taxonomies [2], more exactly: the recalling, comprehension, application, analysis, evaluation and synthesis of knowledge. As can be seen, the level of thinking skills increases with each of the objectives. The AskeMe!system in combination with ICOs cover the lower-order thinking skills up to the application and in parts the analysis of systems and problems. This application of the taxonomy includes multiple choice and right-or-wrong questions but also interactive animations.
The challenge is to bring also the other -the higher-order objectives -in our system. If we can do this, we are able to improve both the learning and teaching methods. Now, with the help of the remote lab, however, we are able to cover all of the higher-order thinking skills. These skills include the analysis and evaluation of an unknown system and the synthesis of new solutions. For example, the student can develop a solution for an elevator control, conveyor or high rack warehouse on the basis of finite state machines and the description of sensors and actors of the particular system.  During the learning process we collect data about each user of the system. These data are anonymous and only consist of an "actor", the user id, a "verb" -describing an activity -and an "object", which mainly represents one component of the whole system. This division of data into statements composed of "actor", "verb" and "object" is defined by the "xAPI" or "Tin Can API" Some examples for this kind of data are information about the activity of a user, the using of external links, ICOs or the assessment system.
With these -for unauthorized persons hidden -information we are able to generate user statistics and hints for the user in order to increase the learning throughput.
Apart from these general hints for the user the system is able to give feedback during the learning process. For example, the system can offer the user his or her mistakes after evaluation of the solution. In that way both the user and the system -and in this way also the teacher -are aware about the strengths and in particular the weakness of the user.
In the following example of an elevator control task the learning and assessment process will be demonstrated. The task of the student is to design a control algorithm for a three or four floor elevator (see Fig. 6).
For that, a set of parallel finite state machines needs to be developed. Normally, the student will make a sketch of an automaton and will revise his or her idea a few times. As can be seen, there is a virtual elevator animation at the left margin of the screen. In the middle, there are live data transferred from the web cam. And at the right margin, there are input options for the user to insert equations. After the student believes the solution is correct or wants to test something, he or she can start the experiment.
During execution of a student solution in the remote lab errors can occur. If these errors are not a result of architectural problems of the physical system they are a consequence of a faulty user design. These errors can be divided into three classes: critical errors, major defects and minor defects and are explained below.
Critical errors can cause damage to the physical system. It is crucial that we prevent these kinds of errors. Examples for critical errors are: on reaching the "open" position the door is controlled to continue motion or the elevator cage is on the top floor and would drive upwards.
Major defects don't cause damage to the physical system. This class of errors is of first priority in real systems and is divided into three parts.

1) first order errors:
can cause damage to humans and are of first priority. Examples for first order errors are: an overload of the elevator cabin is detected and the cabin is about to move the door of the cabin is open and the cabin is about to move.

2) second order errors:
can cause damage to the physical system. There is also an opportunity to damage humans. Examples for this kind of errors are: the door of the cabin locks up even though a passenger crosses the light barrier. the drilling machine (as another example of a controllable device -the "production cell"-in the lab, see Fig. 8) is about to drill and to move simultaneously.

3) third order errors:
can stop the process. Examples are: work piece in the "production cell" fails the next conveyor belt. jerky movements.
Minor defects don't influence the execution. This kind of error makes the process inefficient. Examples are: unnecessary moves of work pieces in the production cell.
an engine runs without reasons.
As consequence of an occurred error the whole system stops the execution of the program. After that, the error is displayed (with error number and description). Thereafter, the user can revise his or her solution and can run the system again.
To find bugs in the application, the user can run the program step by step. The physical system will follow them incrementally.
But if the user has no idea to find a solution to a problem, the e-learning system will give hints. These hints can be poor but also very strong in order to help the student.
For the example of the elevator control, hints can be links to relevant lectures like automata basics, references to programming languages, but also information about the nonfunctioning parts. This approach increases the motivation of the user while learning.
During the whole experiment all user actions are recorded in the learning record store (see Fig. 5) and are available for adaptation algorithms whether to give an immediate feedback or to assess the student's competencies in higher order thinking skills [7].