A Diagnosis Model for College Teachers' Teaching Ability Based on Big Data and its Evaluation

The mining of big data provides new ideas, methods, and technical support for the evaluation of college teachers’ teaching ability. Existing studies generally over-emphasize outcome evaluations and the evaluation methods are not scientific or objective enough, thus the evaluation results are often trapped in large errors and single pattern of manifestation. To overcome such defects, this paper took college English teaching as an example to develop a diagnosis model for college teachers’ teaching ability based on big data and evaluate its feasibility. At first, the evaluation indexes of college teachers' teaching ability were determined and the entropy weight method was adopted to assign weight values to the evaluation indexes. Then, based on the Gradient Boosted Decision Tree (GBDT), the diagnosis model was constructed and the steps were detailed. After that, an improved Particle Swarm Optimization (PSO) algorithm was adopted to optimize the proposed model. At last, experimental results proved the feasibility of the proposed diagnosis model. Keywords—big data, college teachers, teaching ability, diagnosis, Gradient Boosted Decision Tree (GBDT), Particle Swarm Optimization (PSO)


Introduction
The reform of higher education system is progressing in China, and an effective improvement could be seen in terms of the teaching quality of Chinese colleges and universities [1][2][3][4][5]. The teaching ability of college teachers runs through the entire teaching cycle and all aspects of school development, and it is the lifeline of higher education [6][7][8][9][10]. In the context of ever-increasing student enrollment and expanding school scale, the evaluation system of college teachers' teaching ability should be updated accordingly and continuously [11][12][13][14]. The fast advancement of data science, Internet, and information technology has brought novel teaching modes such as Smart Teaching and Internet+ Education, and some of them have been already applied in colleges and universities [15][16][17][18][19]. The multi-type and multi-channel education big data have become an important basis for the comprehensive evaluation of college teachers' teaching ability, and the mining of massive education big data has turned into a good way to enhance such ability.
Teachers are the backbone force and valuable resource of higher educational schools, and their teaching ability is a guarantee for the teaching level of these schools. However, during the actual evaluation process of college teachers' teaching ability, data support is still insufficient, which has resulted in defects such as unscientific and unreasonable evaluation process, and it's difficult to objectively and fairly reveal the true teaching ability of teachers. Scholar Zhao [20] analyzed a comprehensive evaluation model of college teachers' teaching ability based on the theory of big data, and studied the application effect of this model. Bao and Xu [21] proposed that specialization and diversification of teachers' teaching ability is the kernel of the development of higher education; combining with big data information analysis and taking teachers of the textile science as subjects, they applied big data to the analysis of teaching, summarized the current status of the teaching ability of these teachers, and figure out its impact on the textile industry. Luo [22] argued that among the indexes that can measure and decide an excellent PE teacher, teaching ability is the most important one, then the study applied big data technology to study countermeasures for enhancing the teaching ability of PE teachers in application-oriented colleges. Jiang [23] discussed the advantages of education informatization and pointed out that educational informatization is the only way to realize education modernization, the paper emphasized that college teachers must be proficient in professional knowledge and make efforts to improve their information technology skills, increase pedagogical wisdom, and cultivate educational critical and reflective abilities.
There are obvious problems with existing evaluation systems of college teachers' teaching ability. Most relevant literatures ignored process evaluation, the extracted evaluation indexes couldn't cover every facet, and some of them employed unscientific or non-objective evaluation methods, which eventually led to large errors and single form in evaluation results. The mining of big data provides new ideas, methods, and technical support for the evaluation of the teaching ability of college teachers. Thus, this paper took English teaching as an example to develop a diagnosis model for college teachers' teaching ability based on big data and evaluate its feasibility. The main contents of the paper are: 1) extract evaluation indexes of college teachers' teaching ability, and use the entropy weight method to determine the weight values of these evaluation indexes; 2) use GBDT to construct the diagnosis model; 3) use improved PSO to optimize the optimal-searching process of the GBDT model; 4) verify the feasibility of the proposed model using experimental results.

Weight value assignment of evaluation indexes
Index weighting is an important step in the comprehensive evaluation of college teachers' teaching ability. The conventional Principal Component Analysis (PCA) relies on the judgment matrix scored by experts and can hardly attain the weight value of a single index. Hence, this paper employed the entropy weight method to determine the weight values of evaluation indexes, including 8 first-level indexes of teaching idea TS1, teaching basis TS2, teaching integration ability TS3, teaching design ability TS4, iJET -Vol. 17, No. 03, 2022 teaching implementation ability TS5, teaching organization ability TS6, teaching evaluation ability TS7, and information-based teaching adaptability TS8.
Specifically, teaching idea (TS1) includes: awareness of teaching quality improvement, teaching attitude, and teaching responsibility; teaching basis (TS2) includes: knowledge reserve, teaching resource reserve, teaching skill reserve, and teaching method reserve; teaching integration ability (TS3) includes: knowledge point integration ability, and professional knowledge expansion and integration ability; teaching design ability (TS4) includes: teaching content design, teaching scene design, and teaching process design; teaching implementation ability (TS5) includes: knowledge teaching ability, teaching management ability, guiding and tutoring ability; teaching organization ability (TS6) includes: class teaching ability, group study guiding ability, and individual tutoring ability; teaching evaluation ability (TS7) includes: student evaluation ability, and teacher evaluation ability; information-based teaching adaptability (TS8) includes: autonomous learning ability of information-based teaching, communication and cooperation ability, and research and innovation ability.
Assuming: N represents the number of evaluation indexes; M represents the number of study years; Ai represents the i-th study year; Aj represents the j-th evaluation index; Formula 1 gives the data matrix of evaluation indexes: The specific steps of index weighting of the target problem are elaborated below: Step 1: Since there are dimensional differences between evaluation indexes, dimensionless processing was performed on the evaluation index data. Assuming: maxAj and minAj respectively represent the upper and lower limits of the j-th index.
For negative indexes, the smaller the values, the better the results; Formula 3 gives the dimensionless processing formula of negative indexes: Formula 4 gives the constructed normalization matrix: Step 2: the proportions of the data of evaluation indexes were calculated using Formula 5: Step 3: It's assumed that ln represents the natural logarithm, then Formula 6 gives the calculation formula of the entropy value of the j-th index: where, the value range of oj is [0,1].
Step 4: based on Formula 7, the coefficient of variation of the j-th index was calculated: The greater the value of hj, the greater the variation of the evaluation index, the greater its contribution to the teaching ability evaluation of college teachers, and the higher the assigned weight.

─
Step 5: based on Formula 8, the weight value of the j-th index was calculated: Construction of the diagnostic model Diagnosing the teaching ability of college teachers is actually a multi-classification problem of evaluation index data, that is, to grade the teaching ability of college teachers. According to the different characteristics of each evaluation index, this paper constructed the GBDT diagnosis model based on the improved PSO. Figure 1 shows a diagram of the principle of GBDT. According to the figure, the basic idea of GBDT is to iteratively train multiple weak learning classifiers and combine them to generate one strong learning classifier.  This paper applied GBDT to construct the diagnosis model, and its structure is given in Figure 2. The corresponding input evaluation indexes were sample sets of 8 aspects, including teaching idea TS1, teaching basis TS2, teaching integration ability TS3, teaching design ability TS4, teaching implementation ability TS5, teaching organization ability TS6, teaching evaluation ability TS7, and information-based teaching adaptability TS8. The specific steps of the model are detailed as follows:   The input τ evaluation index training samples are {TS1, TS2, TS3, TS4, TS5, TS6,  TS7}; G represents the function composed of all trees; gl represents a single decision tree; the initial value of g0 is 0, then Formula 9 gives the expression of the constructed decision tree model: Assuming: Ψ represents the complexity of the decision tree; loss represents its loss function, then Formula 10 gives the objective function of the model: Ψ could be obtained through the formula below: Assuming: hi and fi respectively represent the first-order and second-order derivatives in the loss function, introducing the objective function into the decision tree model, Formula 12 gives the expanded form of Taylor's formula: where, Hj=Σhi, Fi=Σfi. Let the first derivative of be equal to 0, then the values of the leaf nodes in the decision tree can be obtained through the following formula: * = − + Formula 14 gives the objective function at this time: Then, the * value corresponding to leaf nodes was predicted to minimize the objective function value shown in the above formula, and then the newly generated gφ(a) was added to the diagnosis model. After that, iteration operation was performed continuously until reaching the maximum number of iterations, and the final diagnosis model was output.
Whether a GBDT performs regression or classification depends on the loss function used. The loss functions used in regression include absolute loss function, mean square error loss function, Huber function, and quantile loss function, which are represented by Formulas 15-18: The negative gradient errors of the absolute loss function and the mean square error loss function are represented by sign(b-G(a)) and b-G(a), respectively. The negative gradient errors of the Huber function and the quantile loss function are given by Formulas 19 and 20: When GBDT is used for classification, the loss functions used include likelihood estimation function and minimized negative logarithmic loss function, corresponding to binary classification and multi-classification, respectively. In this paper, the problem of diagnosing and evaluating the teaching ability of college teachers was converted into a multi-classification problem, and the adopted minimized negative logarithmic loss function is given by Formula 21: If the output type of an evaluation index sample is l, then the value of bl takes 1, otherwise it takes 0, and there is: Its pseudo residual could be expressed as bil=bil-Ul,φ-1(a). Formula 23 gives the residual fitting calculation formula of leaf nodes:

Optimization of the diagnosis model
The optimization algorithms of conventional models generally have problems such as high computation load and lack of theoretical guidance for parameter setting. For this reason, this paper adopted the improved PSO to optimize the optimal-searching process of the constructed GBDT model, and the steps of the algorithm are detailed as follows: ─ Step 1: Initialize model parameters including the number of iterations, initial position and velocity, and particle swarm size, etc., and divide the original evaluation index sample sets into training set and test set. ─ Step 2: Calculate the fitness of all particles. ─ Step 3: Determine the individual optimal position and the global optimal position. ─ Step 4: Assuming ai represents the position of the particle in the i-th iteration, υi represents the velocity at this time; Pbest and Gbest respectively represent the local optimal position and the global optimal position of the particle swarm; α1 and α2 represent two random numbers between (0,1); λ1 and λ2 represent the learning factors describing the algorithm's self-learning ability and overall learning ability, ω represents the weighting factor, the velocity and position of particles could be updated according to Formulas 24 and 25: Step 5: Perform algorithm iteration until the termination conditions are met, then output the optimal combination, otherwise return to Step 2 and continue the iteration.
Based on the changes in particle fitness, the algorithm judges whether the output optimal solution is local optimal or global optimal. Assuming: t represents the size of the entire particle swarm, SUmean represents the average fitness of the current position of the particle swarm, SUi represents the fitness of the i-th particle, SU represents the normalization scaling factor used to limit the size of FC, FC represents the standard deviation of the fitness of the particle swarm, then, the discriminant formula is given by Formula 26: The larger the density of the particle swarm, the smaller the FC value; when particles exhibit premature convergence, FC value is close to 0; when the particle swarm is in a random search state, FC value is larger. Therefore, this paper set a threshold, denoted as FCth. If FC<FCth, then the particles show premature convergence. Assuming: P l best represents the global optimal solution of particles in the k-th iteration, RA () represents the random number, then, based on Formula 27, the optimal position of the particle swarm is randomly perturbed as: The weighting factor ω characterizes the particle's ability to inherit previous velocity. Setting a larger ω is beneficial to the global search, and setting a smaller ω is beneficial to the local search. The algorithm in this paper adopted a more intuitive inertia weight strategy with better optimal-searching ability. Assuming: ωmax and ωmin represent the maximum and minimum inertia weights, l and L represent the current number of iterations and the maximum number of iterations of the particles, then Formula 28 gives the adopted inertia weight formula:

Experimental results and analysis
According to the analysis in the previous section, the YAAHP software was used to test the consistency of the evaluation index system of college teachers' teaching ability, and the test results are given in Table 1. According to the data in the table, the CR values of the judgment matrices of all selected evaluation indexes were less than 0.1, indicating that all indexes were of satisfactory consistency. Therefore, the entropy weight method can be used to calculate the weight values of the evaluation indexes, and the calculation results are shown in Table 1. Figure 3 shows the radar map of the distribution of first-level evaluation indexes. It can be seen that: teaching integration ability, information-based teaching adaptability, and teaching implementation ability took up higher proportions, and the proportion of teaching design ability was relatively high as well, while the subject's degree of recognition of teaching idea and teaching organization ability was not optimistic. Figure 4 shows the statistical results of the evaluation indexes of all subjects. Overall speaking, in terms of teaching ability, more than 70% of them have a qualified teaching design ability; more than 60% of them have a qualified teaching evaluation ability; less than 40% of them said they could teach the class independently, guide group study, and give individual tutoring; only less than 20% of them could carry out teaching activities by skillfully using electronic whiteboards, mobile terminals, or online open course platforms. These results suggest that, the information-based teaching adaptability of the subjects is not ideal at this stage.
Mean value and standard deviation can reflect the distribution state (concentrated or scattered) of the sample data sets. This paper used mean value to describe the situation of the first-level evaluation indexes, and used the standard deviation to reflect the differences among these first-level evaluation indexes. Figure 5 shows the statistics of the mean value and standard deviation of the subjects' teaching ability.    TS1  TS2  TS3  TS4  TS5  TS6  TS7  TS8 iJET -Vol. 17, No. 03, 2022

Fig. 5. Statistics of mean value and standard deviation of subjects' teaching ability
According to Figure 5, the standard deviation tended to be stable, indicating that for these subjects, there're not much differences among the first-level indexes, but the mean value fluctuated greatly, indicating that there're large differences in the ability levels of different first-level indexes. The mean values of TS3 and TS4 both exceed 3, indicating that most of the subjects have outstanding teaching integration ability and teaching design ability; most of them have a certain level of knowledge integration ability, and professional knowledge expansion and integration ability; at the same time, they have their supervisory opinions for teaching content design, teaching scene design, and teaching process design of the relevant courses of the major.
To further verify the effectiveness of the proposed model, this paper compared the experimental results of 3 performance indicators (precision, recall rate, and F1 value) of three models, see Table 2 for details. Via comparison, it's known that the three performance indicators of Support Vector Machine (SVM) fluctuated greatly; while the fluctuation ranges of the three performance indicators of the Classification and Regression Tree (CART) and the proposed model were relatively consistent. The proposed model outperformed the other two models in terms of average effect. As for sample sets from different school types (higher vocational schools, colleges or universities, or secondary vocational schools), the proposed model also exhibited high precision and stability, while the performance of CART and SVM was only average.

Conclusion
Taking English teaching as an example, this paper proposed a diagnosis model of college teachers' teaching ability based on big data and evaluated its performance. At first, the paper extracted evaluation indexes of college teachers' teaching ability and determined their weights using the entropy weight method; then, based on GBDT, the diagnosis model was constructed and optimized by improved PSO; after that, this paper plotted a radar map of the distribution of these evaluation indexes, gave their statistical results, compared the mean value and standard deviation, and obtained corresponding analysis results; at last, on different sample sets, this paper compared three performance indicators (precision, recall rate, and F1 value) of three models, and the experimental results proved the feasibility of the proposed diagnosis model.