Paper— Teaching Virtual Experiments: From Course Design to Learning Assessment Teaching Virtual Experiments: From Course Design to Learning Assessment

— Online learning has advantages over traditional learning due to its flexibility in terms of expanding learning scale and enhanced students’ learning enthusiasm. On the other hand, virtual experiments can reinforce concepts of lecture materials and illustrate applications to complement textbooks. In order to enhance learning efficacy, we proposed a comprehensive method to implement online virtual experiments that incorporates learning assessment and course design based on the unique nature of virtual experiment. In addition a controlled experiment was adopted to explore its significance. The results show that it can help students to control the pace, frequency, time of the experiment and to evaluate their learning achievements effectively. That enhances students’ autonomous learning ability greatly.


Introduction
Online learning is not only an important part in China's higher education, but is also an important means to improve the quality of current education, expand the scale of education and further improving educational reform. The major advantages of online learning are its flexible form to conveniently expand learning scale and reduce learning cost [1]. Through this method, students can engage in independent study according to their interests, so as to achieve the objective of cross-school and crossregion education [2]. However, for some disciplines of engineering, experimental teaching is an important part and lecture courses are difficult to achieve the desired learning effectiveness. Virtual experiments are practical and effective educational tools, in addition they are cost effectiveness and convenient for students. These may be used to allow students to have a more comprehensive and flexible experience. With a virtual simulation, a procedure can be repeated without cost of materials and labour. Moreover, students can control the pace, frequency and time of the experiment [3]. Virtual laboratory experiences can reinforce concepts from lecture material, convey practical issues associated with experiments and illustrate applications to complement textbooks [4,5]. So, it is a feasible method to integrate the virtual simulation experiment with the online learning in order to conserve resources and improve the efficiency. In addition to making online experiment learning more effective, an appli-cable learning assessment is indispensable. The popular methods of learning assessment mainly include linear regression, partial least squares, multivariate statistical analysis, grey relational analysis and analytic hierarchy process [6][7][8][9]. These methods assume that there is a linear connection between the learning efficacy and assessment indicators. In fact, the relationships among them are nonlinear. Therefore, it is difficult to accurately describe the problem by using the linear model, which leads to the crucial difference between the assessment results and the actual impact. Hence, the evaluation system needs the transparency, objectivity, logical reasoning and easy implementation and that could be provided by the fuzzy logic comprehensive evaluation [10,11]. To solve the problem, this paper designed a method based on fuzzy comprehensive evaluation in order to assess the learning effectiveness of the virtual experiment.
The paper describes the integration of virtual experiment with online learning and an effective learning assessment method for better learning practice. The rest of the article is organized as follows. In section 2, we discussed the course design of the virtual experiment based on online learning. Section 3 proposes a fuzzy comprehensive evaluation model integrating with the distinguished learning behaviors that considers all aspects in learning process. The model can give a comprehensive assessment result of the learning achievement and is effective to evaluate students' learning quality. Through an example, section 4 describes the implementation of the proposed system along with obtained results. In Section 5, conclusions are drawn.

The Course Design
The course design is performed to improve the effect of teaching with virtual experiment. Fig 1. Shows the course design structure of online virtual experiments. There are three main components of the course design: teaching content preparation and production, online teaching process and after class interaction. The details of the process are described as follows.

Teaching content preparation and production
Preparation of course materials should involve a detailed and thorough design of teaching videos. Firstly, there are great differences in the motivation, interest, starting point and cognitive process of the students. All these need to be addressed while designing curriculum goals. Teachers should understand the students' learning background, which can be carried out through students' personal introduction and lecture notes. Secondly, teachers need to choose the topics and to determine the teaching contents as well as teaching targets. The process should be accompanied by the students' counseling. In the design of course content, the virtual experiment platform should be set up to guide the students; in order to explore the comprehensive and innovative experiments under the premise of completing the basic experimental objectives. They need to control the teaching time within 20 minutes, so that the teaching video can concentrate more on key points. Thirdly, virtual laboratory platform design is based on the virtual reality and multimedia technology. So, they need to ensure the effectiveness and completeness of the teaching video. It should be carried out in a flexible form through the design, the debugging, and the output final of virtual experiments. This will help to improve the students' interest in learning and to present students' abstract thought in an intuitive way. In addition this will ensure a proper, quick lead-in, a clear demonstration and a concise conclusion should be guaranteed [13][14]. According to the actual teaching requirements, teachers can search on the web for concise video clips addressing the main principle between theory and virtual experiment, which can stimulate students' studying enthusiasm and broaden students' mind. Finally, the teachers need to choose the proper production tools considering actual situation to produce virtual experiments' video. Various screen recorder software can be used to record the screen contents and voices inside or outside the computer. This method applies to multi-video clips, which ensures the fidelity of pictures and voices recorded.

Online teaching and after class interaction
During the online teaching, neither is a teacher or the traditional experiments instructor, nor a student is the passive receivers of knowledge or information. Teachers can supplement some electronic documents to satisfy various learning requirements of students. Then, the most frequently asked questions about the learning contents on the web will be answered by the teacher. Question discussion and practice are the two important parts in the teaching of virtual experiments. Question discussion aims to solve common doubts of students. The teacher should create a free exchange situation for students and guide them to discuss. This flexible way is used to preliminarily convey the difficult teaching points. During this process, teacher needs to strengthen the organization and instruction of students' group discussion as well as to enlarge their cognitive structure step by step in order to improve students' ability.
To further enhance the effect of classroom teaching, students can exchange and communicate online with the teacher through the platform within specific time periods. After-class Introspection and Interaction are important means to improve students' learning efficacy. After introspection, students can go on discussing what they have learnt in class on the internet, expressing their opinions and interacting with their teachers. To make the interaction more effective, the evaluation methods should be also more flexible and diverse. In addition to the traditional evaluation of teachers and teaching assistants, more peer assessment could be carried out. Peer assessment is very important in online education, which can develop the sense of cooperation in mutual evaluation. The teacher should make full preparations and quantitative standards of evaluation to facilitate the activities of mutual evaluation for carrying out smoothly. In the process of peer assessment, teachers or teaching assistants should guide, inspect and supervise the activities of the students' mutual evaluation through a variety of ways. The process evaluation and summative assessment should be combined in the online learning evaluation. The process evaluation mainly pays attention to the students' online learning during the course time, frequency and participation in the discussion and speech. Summative assessment is usually carried out in the final examination in which students can be divided into several online learning groups. Besides, students can upload the relevant video clips, which, after approved by their teacher, will be watched by other students for further discussion. Nowadays, SNS (Social Networking Services) have been widely used in communication among college students. Therefore, students can share the micro-lesson video clips in various SNS to be learnt and discussed in a broader way.

Learning Assessment of Virtual Experiment
It is necessary for the teacher to assess students in an all-round way under the online education mode. Targets of assessment involve degree of students' participation in learning activities, records of applying the online course platform, students' learning ability and the learning attitude. To improve the students' learning initiative and measure the students' learning achievements, we designed a comprehensive fuzzy model to assess the online learning performance of students. The comprehensive learning assessment is the final target in this model which is affected by many aspects of learning behavior. The various learning behavior can be quantified as distinguished model indexes. Considering that the main indexes include many sub-indexes. Our design analyzed the whole learning process and divided them into hierarchical indexes, which is shown in Fig 2. The final target is in the center as the figure shows, which is the evaluation of the comprehensive learning performance. It can be divided into four main factors called the first level indexes. The four main factors are surrounding the main target as the figure shows, which are "learning attitude", "learning activity", "discussion and cooperation" and "learning ability" respectively. To evaluate the learning performance more delicate, we divide each first level index into four subfactors are called the second level indexes. The sub-factors are surrounding with the first level indexes, which are shown in the outermost periphery of Fig 2. Every index has the corresponding weight attributing to its different impact to the students' behavior and each index has its own evaluation result called comment set. For the convenience of the description, weight index set and comment set is presented as follows: The first level index set is defined as U = {U 1 , U 2 , U 3 , U 4 } = {"learning attitude", "learning activity ", "discussion and cooperation", "learning ability"}, and the corresponding weight index set is defined as W= (W 1 , W 2 , W 3 , W 4 ). The second level indicator set is defined as U i = {u i1 , u i2 , u i3 }, (i=1, 2, 3,4). Meanwhile, the corresponding weight index set is defined as A i ={ a i1 , a i2 , a i3 } which is often obtained by the comments of the related domain experts. The comment index set is defined as V={ V1,V2,V3,V4} ,which represents { Excellent ,good, middle, poor}. The whole evaluation procedure includes three steps and is shown as follows: The first step is fuzzifization which means the crisp comment value should be transformed to the fuzzy member function, and the fuzzy member function can be determined from the evaluation of the students for the distinguished index. For example, if there are 30 students thinking the second index U11 is "excellent" and the total number of the students is 100, the index U11 belongs to "excellent" would be 30/100=0.3.
The second step is fuzzy inference: the value of the first level indexes can be calculated according to the value of the second level evaluation, which can be implemented by the fuzzy compound calculation between the membership degree of second level indexes and the weighted vectors given by the domain experts.
The third step is defuzzification: the result of the global evaluation can be calculated from the first level evaluation, which can be implemented through the fuzzy compound calculation between the first level indicator membership and the global weighted vector.

Result Analysis and Discussion
We take the "digital electronic technology" course as an example to show the design of the virtual experiments. The target is to improve the teaching results of the "Virtual Experiments" in the online environment, utilizing the interaction between the course design and the learning evaluation. The course design of virtual experiments is based on the learning evaluation, meanwhile the good evaluation method can give positive feedback for the course design. The design task is to implement a specific experiment effectively and the evaluation of the student's design is based on both the peer assessment among students and the teacher's assessment. Though the fuzzy evaluation results can only give students comprehensive quality evaluation of the developed solutions, but there is an important step in the process of evaluation called peer assessment which can help students to obtain positive feedbacks through comparison to improve their designs. To stimulate students' creativity, there is a reference design in the teacher's side, but there isn't the best design in the students' side. Students need to try their best to find better experiment design.
The course content is divided into independent modules called micro lessons and each module has a definite theme. For example, the chapter of the combinational logic circuit can be divided into half adder, full adder, data selector, decoder, and so on. We evaluate the overall achievements of virtual experiments including the homework of experiment design, the written exams of design method and the evaluation of experiment results. The simulation software used in this experiment is Multisim. Multisim is a complete electronic design tool, which provides a generic component database from single gate circuit components to large scale integrated circuit components. Besides, comprehensive virtual experiments are arranged to further broaden the students' thinking.

Fig. 3. A case of virtual experiment
A design case of a comprehensive experiment is selected as one micro lesson and the reference design is shown in Fig 3, which is designed based on Multisim. Before this comprehensive experiment would be carried out, it is supposed that students have completed such simple examples as counter and trigger. The comprehensive experiment is the integration of above simple experiments which can be realized by the combination of counter 74LS192D and basic trigger. At the same time, students can make use of the advantages of virtual experiments and explore the output diversity in the virtual experiment platform. For example, the results of the experiment can be shown by the virtual LED display. Meanwhile, the virtual oscilloscope can be used to observe pin signal of the virtual device. After the students have completed this comprehensive experiment through online operation in the fixed time, the fuzzy evaluation method could be used to assess students' performance, which is shown in the next section.

Experiment design
The goals of the experiment were to evaluate the learning achievements based on the online platform. To show the effectiveness of this evaluation method, a questionnaire was applied to investigate the various learning behaviors which are shown in Table 1. Indexes of U1 and U3 are obtained according to the record in the platform. Meanwhile, indexes of U2 and U4 are formed through the mutual evaluation of students. The weights of different learning behaviors were given by ten experts in education domain. Moreover, there is no significant difference in the gender of students as well as the age and physical quality. To analyze the experimental data, SPSS13.0 statistical software and Microsoft Excel 2010 software were used. Besides, the fuzzy comprehensive evaluation was applied to implement the final learning achievement assessment.  Table 1 shows the evaluation of a certain student in the various learning behaviors, which are the statistical results of the questionnaire survey about the learning behaviors. According to the model we have set up in the last section, the global evaluation set is {V1,V2,V3,V4}={ Excellent, good, middle, poor}, which are corresponding to the first level indexes Ui (i=1,2,3,4)={"learning attitude", "learning activity", "discussion and cooperation", "learning ability"}. The weights of the second level indexes are also listed in Table 1, in which each first level index is decomposed into the four second level indexes. For example: the first level index U1 "learning attitude" are decomposed into four second level indexes {u11, u12, u13, u14}. The result of the first level evaluation can be inferred from the second level indexes and the value of the second level indexes are determined based on the mutual comments of students and the learning records on the platform.

Results
The evaluation matrix of the second level indexes is shown as equations (1) and (2), which means the membership degree of the second level indexes belonging to the comment set {V1,V2,V3,V4}={ Excellent , good, middle, poor}: The weighted vectors of the second level indexes are represented as equations (3~7), which mean the importance degree of the second level indexes relative to the first level indexes. This can be obtained according to Table 1, which is determined by the related domain experts. Because there are four second level indexes and four corresponding weighted vectors, which are shown as follows: The second step is fuzzy inference. The fuzzy compound calculation is selected to infer the first level indexes according to the second level indexes, which is shown in equation (7).
The four first level indexes can be inferred according to equations (1~7): ! !!! !!! !!! !!! So, we can get the four first level indexes as shown from Fig 4 to Fig 7, which mean the learning performance in four various aspects.

#%'$
The last step is defuzzification, which is implemented according to the four first level indexes results. The global evaluation matrix is determined according to the four first level indexes: Because the first level weight vector is: And the global evaluation vector can be calculated: The final evaluation value can be calculated from the global evaluation vector, which is implemented based on equation (14), in which j represents the grade score and ! ! is the corresponding weight: According to the final results, we draw a figure to describe the result as shown in Fig 8. Apparently, we can get a conclusion that the global evaluation is: The percentages of "excellent" and "good" are both 30%, the percentages of "middle" and "poor" are both 20%. The top-grade score is 4, which means "excellent", and the worst grade score is 1, which means "poor". This student's grade score is 2.7, which can be calculated based on equation (14). The score of centesimal grade is 2.7/4*100=67.5. The result is convenient to decide the students' learning performance on the online platform.

#%'$
Another evaluation method is called the Simple Additive Weighting (SAW) that is selected for comparison. SAW [15] is a simple and widely used multi attribute decision method. The method is based on the weighted linear combination and an evaluation score is calculated for each student by multiplying the weighted value given to his attribute. Fig 9 shows the comprehensive evaluation of learning performance utilizing SAW and applying the same indicator and the corresponding weight which is shown in Table 1. The disadvantage of SAW is it's proportional linear transformation of the raw data. However, there isn't a linear connection between the learning performance and assessment indicators. SAW ignores the fuzziness of experts' judgment during the decision-making process. Besides, some criteria could have an uncertain structure which cannot be measured precisely. Fuzzy evaluation can deal with fuzzy objects and make a more scientific, reasonable and practical quantitative evaluation.

Conclusion
Online learning of virtual experiments can help students to integrate knowledge from several discipline areas and to understand the relationship between theory and practice. In addition, it is important to evaluate the learning performance, which is an important guidance for both students and teachers. Therefore, an excellent evaluation method of learning performance is indispensable to guarantee a high quality educational experience. This paper addressed the key points in the course design of virtual experiments. It also proposed an effective evaluation method targeted to identify students' learning achievements according to their distinguished learning performance based on fuzzy comprehensive evaluation. This evaluation method is advantageous to handle uncertain information full of vagueness, uncertainty and subjectivity. As a result it will be more significant to improve students' learning autonomy and the ability of combining theory with practice, which can improve their learning efficiency greatly. Future work will consider the implementation of hybrid online labs and the corresponding evaluation method. #$ #%*$ #%&$ #%+$ #%'$ #%,$