An Intelligent Assessment System of Teaching Competency for Pre-service Teachers Based on AHP-BP Method

— The Education Informatization 2.0 Action Plan issued by the Chinese Ministry of Education calls for further improvement on teachers’ teaching competency in the era of "Internet+ Education", which also brings new requirement for the training of pre-service teachers who are enrolled in university teacher education courses. This study put forward three principles to construct a new framework to assess the teaching competency for university students majoring in teacher education. Developed on the basis of the Analytic Hierarchy Process (AHP) methods, the assessment system included six dimensions contributed by 21 indicators. This system is supported by Back Propagation (BP) neural network and data label technology to realize intelligent assessment of teaching competency. The trial results of real data samples showed that the system could meet the requirements of large-scale assessment for Pre-service students in higher education sectors. The assessment framework could well cover pre-service teachers’ qualification standards, facilitate the assessment of their teaching competency, and provide a clear guidance for effective trainings for the pre-service teachers to meet the requirement of future teaching supported by the Internet technology. Paper Intelligent Assessment System of Teaching Competency for Pre-service Teachers Based on … Teaching Competency for Based on …


Introduction
In 2018, Chinese Ministry of Education issued the "Implementation guideline of the Excellent Teacher Training Plan 2.0" and proposed to apply rigid evaluation of preservice teachers' teaching competency and strengthen the training on pre-service teachers. The rigorous assessment on pre-service teachers' teaching competency is essential to ensure that they are qualified to enter the profession and gain the professional excellence in their future career. However, there are still some issues emerging in the assessment of pre-service teachers in higher education sectors in China, such as a lack of research on teaching competency of pre-service teachers, lack of clear and operational assessment system; overreliance on the assessment without clear standards and neglect of quantitative formative assessment. The current era of educational informatization is moving from Web 1.0 to Web 2.0. This is a process from the construction of educational infrastructure to the training of talents in the "Internet+ Education" era [1], [2]. Big data is transferring education from traditional to data-driven teaching mode [3], which supports learning analytics and educational research. This study combined the advantages of AHP's scientific modeling and BP's intelligent assessment to establish a data-driven teaching competency assessment system for pre-service teachers [4]. The system was proved to be a valid assessment tool for the university students of teacher education and help to optimize the training program for cultivating new teachers with expertise in the in the era of "Internet+ Education".

2
Research design

Research route
In this study, the AHP and BP methods were applied to construct teaching competency assessment system for pre-service teachers, as shown in Figure 1. Firstly, the researchers analyzed relevant policies and research work regarding the practice and assessment of teaching competency in some countries. Secondly, the Delphi method was applied to conduct two rounds of surveys to establish a teaching competency assessment system for pre-service teachers. The researchers consulted from field experts to have the first-round revision on the assessment system. Then, the AHP method was adopted to calculate the weight of each indicator and test the validity of the assessment system. Finally, the data related to teaching competency from an online education practice platform for pre-service teacher was extracted in order to test and verify the assessment system with a BP neural network model.

Research methods
Delphi method, and AHP method are widely used to construct and revise assessment indicators and determine the weight of indicators. AHP is a typical linear multi-attribute assessment method, but its assessment results are easily affected by the limited number of experts and individual subjective experience bias, which makes it difficult to accurately quantify and objectively evaluate the indicators [5]. With the development of artificial neural networks, BP algorithm has become one of the most widely used and successful neural network models with its non-linear representation ability [6]. Some studies have shown that AHP-BP is an effective modeling and assessment method in teaching context [7], [8], [9]. In AHP-BP modeling, BP neural network technology can effectively solve the problem of subjective fuzzy expert assessment in AHP method. It can automatically adjust the weight of the model with the increasing number of training samples to improve the accuracy of the prediction results. Moreover, the experts' scores collected by AHP method can ensure the rationality of model construction, and remedy the deficiency of single modeling in BP method. Therefore, the researchers aimed to apply the AHP-BP method to construct a teaching competency assessment system for pre-service teachers and conduct an empirical study to examine the effectiveness of this assessment system.

The conceptual basis
This study generated the prototype framework of the teaching competency for preservice teachers mainly from two aspects, as shown in Figure 2. Firstly, the standards of teaching competency in some countries are compared and analyzed, including the core content framework of pre-service teacher training of UK [10], the NBPTS Teacher Professional Standard [11] and InTASC core Teaching standard of USA [12], the Australian pre-service teacher Certification standard [13], the Quality Teacher Certification Standard of New Zealand [14], the Chinese Teacher Professional Standard [15] and the Accreditation Standards for Teachers Major [16], etc. Secondly, the existing researches [17], [18], [19], [20], [21], the training program and the content of educational practice course in some teachers' universities, have also provided reference for the development of teaching competency framework.

Design principle
Dual role orientation of pre-service teachers. The design of teaching competency assessment system should reflect pre-service teachers' dual roles as learners and teachers at the same time. The population of pre-service teachers in China mainly consists of the students specializing in teacher education in universities. From the perspective of student's role, professional training in universities not only provides a foundation for them to enter the teaching profession, but also facilitates the transfer of teaching capability to practice. Therefore, the assessment indicators of teaching competency should not only evaluate the students' professional knowledge, but also their capability of reflecting on their own teaching practice and self-development. From the perspective of teacher's role, pre-service teachers' teaching competency is oriented by the qualification standard of in-service teachers, but varies from the latter. Therefore, the teaching assessment present the requirements of training programs and professional practice set by the universities.
Core competence of future teachers. In recent years, the studies on citizen information literacy, digital competency, data literacy, adaptive teaching competencies [22], teaching competency of computational thinking, teacher competency in a smart teaching environment and other related issues have indicated new requirements for future teachers. In 2019, the Ministry of Education of China issued the Implementation guideline of "National Project of Information Technology Application Ability Enhancement for Primary and Middle School Teachers", which pointed out that "information technology application ability is the core literacy of expertise teachers in the new era". In the "Education 4.0 Framework" [23], "technical skills" is regarded as a key requirement for future learners. Therefore, the ability of applying technology in teaching should be regarded as an important competency for pre-service teachers in the new information age.
Data-driven process assessment. To solve the issues of data collection and poor readability of results in the process of professional practice, several researchers designed virtual practice space with artificial intelligence to facilitate the procedural data collection during university students' teaching practice [24]. VR technology has been used to build micro-teaching system to record the teaching practice process, so as to make a comprehensive quantitative assessment for the pre-service teachers [25]. In ad-dition, for the purpose of collecting and visualizing multi-dimensional data, the mapping between the indicators of pre-service teachers' teaching competency and data labels ought to be considered during the formulation of the competency indicators.

Indicators and weight
As a hierarchical weight decision method, AHP can decompose complex decision problems into decision objectives, criteria level elements, indicator schemes and other levels, and at the same time, make qualitative and quantitative analysis. The main process is to construct the corresponding hierarchical structure model according to the decomposed decision problems, construct the judgment matrix, conduct the consistency check, and finally calculate the corresponding weight to get the conclusion. The Yaahp software is often selected as an auxiliary tool for AHP model construction to support calculation and analysis for the decision-making process, as manual calculation is time consuming and lack accuracy.

Indicators generation
Through the analysis result of teaching competency standards and research findings, the draft of teaching competency assessment indicators for pre-service teachers was completed. An expert panel, including the dean of the department of teacher education, mentors for teaching practice, and course coordinators for teacher education with over 20 years of teaching and research in teach education, were invited to participate in the first-round survey to review each teaching competency dimension and its indicators.

Weight calculation
After the assessment dimensions and indicators were determined, the expert panel, who were involved in the first-round survey, participated in a second-round survey to further examine the weight of each indicator and provide consistent feedback.
Construct judgment matrix. Combined with the final version of assessment indicators, an AHP questionnaire was developed by comparing two elements at the same level. After the questionnaire was administered, the judgment matrix A of each expert at each level was calculated. The "m" is the number of indicators, and the importance degree of the indicator "i" relative to the indicator "j" was aij.
Consistency check. The consistency check of judgment matrix is to determine whether the judgment matrix of expert group is acceptable. Only when the judgment matrix meets the consistency requirement, the subsequent weight calculation can be carried out. Generally, when C.R. <0.1, it is considered that the judgment matrix passes the consistency check; otherwise, there is no consistency. The calculation process is shown as follows, where W is the weight vector of each layer of elements relative to the upper layer of elements, n is the order of the matrix, and R.I. is a constant, whose value is related to the order of the judgment matrix. The following are three formulas to calculate the maximum, consistency indicator and consistency ratio of the judgment matrix respectively.

(4)
Weight calculation. After the consistency check was completed, the weight of each indicator at a single level was calculated. The weight of the criterion layer was multiplied by the corresponding weight of each indicator layer to form the combined weight of the assessment system.

System analysis
Yaahp was used to generate expert judgment matrix and provide consistency check. The expert data was aggregated by means of arithmetic mean method, and the indicator weight value of teaching competency system was obtained, as shown in Table 1.
In the assessment system, the more weighted dimensions indicated professional foundation, teaching implementation and instructional design. At the level of specific indicators, professional basic knowledge, teaching methods, teaching objectives and process design were more valued by experts. This result can be analyzed from the following aspects.
Professional foundation, instructional design and teaching implementation were regarded as the basic teaching competencies. As the accumulation of knowledge at an early stage of pre-service teacher training process, professional knowledge is an important cornerstone for the development of students' teaching competency. On the other hand, instructional design and teaching implementation indicated pre-service teachers' ability to continuously strengthen training in practice. They are the core contents of educational practice and the foundation for teachers' professional development.
There is still room for the improvement in the students' capability of self-reflection and teaching evaluation. The attribute of "learning" possessed by pre-service students outweighs that of "teaching". There is insufficient attention paid to reflection and evaluation and fewer opportunities were provided for students to practice reflective thinking. In addition, the ability of self-reflection and assessment needs to be supported by years of teaching experience, so it is a relatively higher requirement for these students.
Technology application has become an indispensable ability for teachers in the new era. For a long time, the ability to use technology in teaching is usually regarded as a special requirement for students majoring in ICT Education, while there are no clear requirements for the pre-service teachers in teaching the other subject areas rather than ICT, such as mathematics, language, science, and so on. With the continuous promotion of education informatization and education modernization in many countries, increasing number of teachers and subject experts have begun to value the impact of technology in teaching and learning. Therefore, it is necessary and reasonable to include technology application into the six dimensions separately.

BP Neural Network
Most neural network models adopt BP network and its variation form in practical application. It is a multi-layer feed forward artificial neural network that adjusts and optimizes the weights and thresholds of each layer by using the errors of the output value and the expected value. Its ultimate goal is to achieve the maximum fitting of the output value and the expected value. In the training process, the technology has become an effective academic assessment method due to its strong autonomous learning ability, parallel processing of large-scale data, self-organization optimization, nonlinear approximation and other advantages, as shown in Figure 3.

Model design
Parameters setting. The BP neural network model with a single hidden layer is adopted in this study. Combined with the teaching competency assessment indicator of pre-service teachers, the number of neuron nodes in the input and output layers m and N were set to be 21 and 1 respectively. There is no specific reference standard for the number of neuron nodes in the hidden layer "q". According to the existing research experience " = √ + + ", the change of the determination coefficient is taken as the judgment basis in combination with the formula. Here "a" is an integer between 1 and 10. The transfer function of the hidden layer adopts S-type function and the output function chooses linear function.
Algorithm optimization. The maximum descent algorithm of BP neural network cannot always guarantee to find the global optimal solution. Because the weight of the network is always adjusted gradually along the direction of local improvement, it is very easy to converge to the extreme value, so that the network can't not find the best weight of the solution space. Therefore, it is necessary to optimize BP algorithm. The common methods include additional momentum method, adaptive learning rate, quasi-Newton method and L-M (Levenberg-Marquardt) method. Because the L-M method can realize faster convergence speed, it is selected to reduce the computation of training process in this study.
Cross validation method. Cross-validation is a sample processing algorithm that obtains as much information as possible from limited learning data. It learns samples from multiple directions, so as to effectively avoid falling into the local minimum value and avoid the problem of over-fitting to a certain extent. The basic principle is to divide the learning samples into n groups. Each time, the data of one group is selected as the test set, and the data of remaining n-1 groups is used as the training set. This process is repeated n times until the data of each group is selected as a test set for training verification. This algorithm is suitable for maximizing the use of sample data when the sample size is relatively small.

Model Application
Data label mapping. According to the description of each dimension and indicator of the assessment system, the collection category is demarcated for the data label. From an online platform for education practice in a university, the learners' data are collected and matched with the selected data labels reflecting each student's teaching abilities. The mapping relationship between each competency indicator and data labels of students is summarized and sorted out. There are two types of mapping relationships, oneto-one and one-to-multiple, as shown in Figure 4. When multiple data tags match a competency indicator, the average score of these data labels will be taken as the score of the competency indicator. BP neural network training. The data of 200 students' teaching competency was used as the sample in a university. The Matlab software was used to establish the BP neural network, and the sample data was divided into five groups to implement cross validation. The values of 21 competency indicators were used as input values, and the value of the overall teaching competency was used as the output value. The total cost of the number of hidden layer nodes was set to 8 before the training of BP neural network. It can be seen that the accuracy was 8.239*e-6 after eight iterations, as shown in Figure 5. Its determination coefficient R was close to 1, indicating that the fitting effect was good. Another 40 students were selected and predicted by the BP neural network model to obtain the absolute values of the relative error with the experts' scores. The results showed that the relative error was small and the fluctuation range was between 0 and 0.2, which indicated that the BP neural network model can be used to evaluate the students' teaching competency on a large scale. Teaching competency radar chart presentation. In order to visualize the students' teaching competency, it is necessary to use the BP neural network model to implement large-scale quantitative assessment for each student. According to the assessment results, the scores of each student's teaching competency are obtained, and the radar chart of teaching competency will be automatically generated. By comparing the evaluation results of large-scale students, the development characteristics of their teaching competency in different majors and universities can be identified. It is helpful to improve the quality control and improvement on pre-service teachers' training.

Conclusion
In this study, the AHP method and BP neural network technologies were used to construct a clear and operable teaching competency assessment system for pre-service teachers. The researchers comprehensively used AHP-BP algorithm to conduct the large-scale and intelligent assessment on pre-service teachers in universities and commenced an innovation in the development of teaching competency assessment tools. This study also has some limitations. Since the teaching competency is a relatively complex concept, it is necessary to adjust the indicators and weights of the assessment system dynamically according to the specific subject areas and teacher training requirements. In addition, the limited number of experts may affect the assessment result, so the assessment system still needs to be tested and improved in future.