Paper—Evaluation of Talent Cultivation Quality of Modern Apprenticeship Based on Context-Input... Evaluation of Talent Cultivation Quality of Modern Apprenticeship Based on Context-Input- Process-Product Model

Based on context-input-process-product (CIPP) model, this paper proposed an evaluation system of the talent cultivation quality of modern apprenticeship in high vocational education schools, and applied the proposed system to evaluate the talent cultivation quality of modern apprenticeship in an example school H. The research findings of this study revealed that, on the one hand, it will greatly improve the quality of cultivated talents if the talent cultivation objectives are kept in line with the requirements of employer companies, the apprenticeship system is built according to the law of student growth and development, and there’re good apprenticeship project resources, support and management; on the other hand, it will have an adverse impact on the effect of apprenticeship projects if students’ employment willingness has not been fully motivated, or the implementation environment is poor for the apprenticeship, or the tutors in enterprises change frequently. Based on these findings, this paper proposed a few countermeasures for improving the talent cultivation quality of modern apprenticeship, that is: fully consider students’ appeals and boost their desire for employment; forge a team of enterprise tutors capable of and good at teaching; improve the quality of apprenticeship projects for junior year students; deepen the integration between industry and education and provide strong guarantee for both school education and enterprise training. Keywords—Higher vocational education, modern apprenticeship, evaluation system, context-input-process-product (CIPP) model


Introduction
In January 2019, the State Council of China issued the National Vocational Education Reform Implementation Plan, which emphasized the need to promote the integration of industry and education, and summarized the experience of modern apprenticeship and new enterprise apprenticeship pilots. Since then, a few provinces in China have begun to be aware of the quality issue of the modern apprenticeship education mode and attempted to formulate standards for modern apprenticeship to facilitate supervision and guidance. However, until now, the quality standard system of modern apprenticeship education hasn't been established yet, and the evaluation framework related to it hasn't been formed either. If the standard formulation is to give guidance to the pilot modern apprenticeship projects from the front, then the project evaluation is to regulate the implementation of modern apprenticeship from the back. After gaining certain experience from practical applications, now the research on modern apprenticeship should pay more attention to evaluation and reflection. In such context, constructing a talent quality evaluation system for modern apprenticeship in higher vocational education schools is of great significance for deepening the integration of industry and education and improving the quality of cultivated talents [1][2][3].

CIPP Model and Talent Quality Evaluation
It's undeniable that there're many problems with the evaluation of vocational education in China, and there're flaws in the evaluation methods, standards, and modes. In terms of theory, the current evaluation system largely relies on the experience of other countries, especially European and American countries, in terms of practice, the current evaluation system is migrated and transferred from the evaluation system of general education [4][5][6][7][8][9]. Therefore, for the research on the evaluation of vocational education, theoretical construction and practical exploration are both new topics. The CIPP model can effectively obtain formative and summative results, support decisionmaking and problem-solving, and it is flexible and standardized, due to such merits, foreign educational organizations and groups often adopt this model to distinguish the differences of different practical teaching methods [10,11]. In order to evaluate the effects of pilot modern apprenticeship projects, this study also employed the CIPP model and constructed a talent quality evaluation system of modern apprenticeship. When determining the evaluation indicators, we have consulted the opinions of vocational education experts, teachers, and enterprise managers, and referred to existing literatures concerning the evaluation system of the post practice of higher vocational education and the evaluation system of applied undergraduate practical teaching. Based on the four key elements of the CIPP model, this study deconstructed the evaluation criteria corresponding to the talent cultivation background, input, process, and outcome of the modern apprenticeship and constructed a new talent cultivation quality framework, as shown in Figure 1.

Application of CIPP Model
To figure out problems existing in the talent cultivation process of modern apprenticeship and provide a useful reference for the formulation of apprenticeship standards, this study took the property management major of a higher vocational education school H as the research object and combined with the four key elements of background, input, process, and outcome in the CIPP model to deconstruct the evaluation indicators and establish the evaluation system.

Background evaluation
The purpose of evaluating the development of apprenticeship projects is to clarify the positioning the apprenticeship projects, assess the basis of school-enterprise cooperation, and determine the goals of talent cultivation. First, it is necessary to identify the position of the apprenticeship projects in the entire talent cultivation system, and make sure the setting of such projects is necessary and reasonable. Second, since the basis of school-enterprise cooperation is very important for the development of apprenticeship projects, it's necessary to evaluate such basis and define the environmental requirements for implementing these apprenticeship projects. Third, the evaluation on talent cultivation goals mainly examines whether the goals are specific and clear, whether they are compatible with the regional economy and the requirements of employer enterprises, and whether they are suitable for the actual abilities of the students. The evaluation of the development of apprenticeship projects includes 3 second-level indicators and 9 third-level indicators, as shown in Table 1. A2. Implementation environment A21. The ideas of the school and the enterprise are consistent and the guarantee system is sound; A22. The history and origin of schoolenterprise cooperation provide a good environment for project implementation; A23. Judging from current situation, the development prospects of school-enterprise cooperation is very promising.
A3. Talent cultivation goals A31. The goals are specific and clear; A32. The goals are compatible with the development level of regional economy and the requirements of employer enterprises; A33. The goals are suitable for the actual abilities of the students, and the job positions have a good room for growth. B2. Project support B21. The scheme conforms to the career growth law of students and suits the features of talents in the service industry; B22. The scheme has standardized teaching process management system and methods, as well as system standards for the master-apprentice pattern; B23. The scheme has organizational conditions to guarantee the implementation of the apprenticeship projects.

Input evaluation
B3. Project management B31. There's a sound evaluation method for assessing students' performance; B32. There's a sound evaluation mechanism for assessing the work quality of enterprise tutors and school teachers participating in the apprenticeship projects.
The essence of input evaluation is to judge the feasibility and effectiveness of the scheme of apprenticeship projects, focusing on the preparation work of the projects, it includes three second-level indicators of project resources, project support, and project management. First, in terms of project resources, the evaluation needs to consider three aspects: hardware (venues and equipment for the apprenticeship projects), manpower (tutor team selection methods, responsibilities, and requirements), and funds. Second, in terms of project support, the evaluation needs to consider three aspects: whether the work tasks and content designed for the projects conform to the career growth law of talents in the service industry; whether the project-based curriculum system conforms to the career growth law of students and whether it fits the features of the service industry; whether the implementation of apprenticeship projects has sufficient organizational conditions. Third, in terms of project management, the evaluation needs to consider two aspects: whether there is a sound evaluation method for students' performance; whether there's a sound evaluation mechanism for the work quality of enterprise tutors and teachers participating in the projects. The input evaluation of apprenticeship projects has 3 second-level indicators and 8 third-level indicators, as shown in Table 2.

Process evaluation
The process evaluation is to evaluate the implementation process of the apprenticeship projects, and reflect the actual situations in the projects. Process evaluation has two second-level indicators, project content (to evaluate whether the features of the project content are reasonable), and project feedback (to evaluate whether the project progress is normal based on the feedback of all parties). In terms of project content, the evaluation needs to consider three aspects: whether the difficulty of the project content is appropriate and whether the students can master it through hard work; whether the project content is in line with the reality in the industry; whether the various work tasks can promote the implementation of the project. These aspects can only be evaluated during the project implementation process, and the rationality of the project content can be evaluated through these indicators. In terms of project feedback, the evaluation also needs to consider three aspects: whether the tutors and teachers keep a track of the project implementation process; whether the students have enthusiasm for the project; and whether the project operates normally judging from various supporting materials and assessment results. The process evaluation includes 2 second-level indicators and 6 third-level indicators, as shown in Table 3. C2. Project feedback C21. Enterprise tutors and school teachers can keep a track of the implementation process of the project and constantly optimize the scheme according to the problems that arise; C22. Students have a correct, positive, and enthusiastic attitude towards the apprenticeship project; C23. Judging from various supporting materials and assessment results, the apprenticeship project is progressing smoothly. The outcome evaluation is to evaluate the performance of the students participating in the apprenticeship projects from the multiple perspectives of different parties in the society. It mainly includes two second-level indicators, comprehensive vocational ability, and multiple social evaluation. The evaluation of comprehensive vocational ability of apprentices is given by both the enterprise tutors and the school teachers. School teachers give evaluations to students' task completion situations based on some long-term and difficult projects; while enterprises tutors are responsible for assessing students' job competence, including vocational quality and vocational skills, wherein the vocational quality focuses on the students' service awareness and their ability to adjust psychological stress, and the vocational skills need to be examined comprehensively based on the students' theoretical knowledge and practical skills. The multiple social evaluation is mainly carried out based on the feedback information of the project quality obtained from different parties such as the government, employer companies and units, and student apprentices. The information data include the employment rate of recent graduates, the return visit data of previous graduates, and the personal development of graduates over the years, etc., these information data can be collected easily and they are of great reference value. The outcome evaluation includes 2 second-level indicators and 7 third-level indicators, as shown in Table 4.

Fig. 2. Statistical analysis strategies
The survey objects mainly involved four groups of people, students, teachers, enterprise tutors, and enterprise and school leaders. Since their involvement in the apprenticeship projects is of different degrees and forms, and their understanding of the projects has different emphases, this study selected some evaluation indicators from the CIPP model and designed four different questionnaires for different groups of respondents. The analysis strategies are shown in Figure 2.
During the survey, a total of 200 questionnaires were distributed. Among the respondents, 98 were students at the school, 65 were previous graduates, 10 were higher vocational education experts and scholars, 8 were school teachers, 7 were manager of cooperative enterprises, and 12 were enterprise tutors. 186 effective questionnaires were collected, with an effective rate of 93.00%. The statistical analysis results obtained are as follows.

4.1
Background evaluation 1. The cultivation goal has considered the talent requirements of enterprises, but hasn't fully mobilize students' employment willingness For evaluation indicator A12 "the projects meet the requirements for talents in the property management industry", among all respondents, 89.2% of them chose that the description is very consistent or consistent. In China, the service area of the property management industry exceeds 25 billion square meters, the industry market value exceeds 1,000 billion yuan; the property-related capital has very strong momentum, and the valuation of the sector continues to rise. The cultivation of talents majoring in property management must grasp the opportunities of industry development, keep up with the trend of the industry, and take up the dominant position of talent cultivation. In the Vanke Modern Apprentice Class, the goal of talent cultivation is set to train high-quality, skilled, creative talents such as housekeepers, customer service assistants, and engineering assistants, who have strong service awareness, good interpersonal communication skills, and high psychological tolerance for the "Smart Service" management system. The talent cultivation goal has well fitted the requirements of the enterprises.
For evaluation indicator A13 "the projects conform to the age and personality characteristics of higher vocational students, and can motivate their willingness for internship and employment", only 45.3% of the respondents chose that the description is very consistent or consistent. Most of the students from Zhejiang Province have good family conditions, especially in recent years, as the admission rate of junior colleges to continue education in universities has continued to rise, fully mobilizing their employment willingness has become the premise for shaping the vocational quality of "dedication and hard-working" in students, and for cultivating their perseverance to adapt to the characteristics of the service industry. Without strong employment intention, students would have doubts about internship and employment and even try to avoid them, and the goal of talent cultivation couldn't be achieved.

Enterprises with a poor environment of apprenticeship can hardly guarantee the number or quality of talents
The environment for apprenticeship in an enterprise is the basis for implementing apprenticeship projects, and it is a condition for guaranteeing the execution of various regulations and rules. Such poor environment for apprenticeship projects is mainly due to the facts that the enterprise doesn't have a good grasp of the development trend of the industry, or the enterprise has difficulty in management and operation, which has resulted in that the enterprise tutors and money input can't meet the demands. According to the scale of the enterprise, the basis of school-enterprise cooperation, and the degree of emphasis on apprenticeship projects, the cooperative enterprises of school H's property management major were graded, such as Vanke Property was rated as Grade 5, the Greentown Property was rated as Grade 4, and the other property companies were rated as Grade 3, 2, or 1. Based on the survey results, company grade, student ability improvement, and student vocational quality improvement were subject to the 2-tailed Spearman correlation analysis, and the. The data showed that the correlation coefficient between company grade and student vocational quality improvement was 0.729, and the correlation coefficient between company grade and student vocational ability improvement was 0.891, and the corresponding p values were 0.001 and 0.000, respectively, both were less than 0.01, indicating that the company grade is significantly related to the improvement of students' vocational ability and quality.

Input evaluation
1. The overall evaluation of the resource, support, and management of apprenticeship projects is good Input evaluation mainly examines the resource allocation, support conditions, and management mode of the apprenticeship projects, its results can reflect the specific situations of project preparation works. The survey data suggested that, the proportions of respondents who chose very consistent and consistent generally distributed between 60% and 90%, indicating that students, teachers, enterprise tutors, and school and enterprise leaders generally believe that the project preparation works were well done.
For evaluation indicator B21 "the scheme conforms to the career growth law of students and suits the features of talents in the service industry", 81.7% of the respondents chose "very consistent, consistent" in the survey. The property management major in school H, Vanke Property, Greentown Property, and other property companies have jointly formed a professional construction steering committee, and joined their efforts to formulate a project-based curriculum system that conforms to the law of students' vocational growth, they have implemented modern apprenticeship in talent cultivation, and realized class-enterprise pairing, and tutor-teacher cooperative and interactive teaching.
2. The mechanism for enterprise tutors to help apprentices grow is not sound According to Figure 3, within an apprenticeship cycle (three years), the ratio of tutors who can assume their job throughout the cycle is less than half. Most of the tutors are from first-line positions in the property company, and they are mainly housekeepers or customer service supervisors. The changes in these job positions have caused the unstable tutor team. Such frequent changes make students unable to master the core skills of the job positions, and affect the effective communication between tutors and apprentices, further, students would have doubts about whether they should hang on and complete their internship in this position. Therefore, the high mobility of enterprise tutors will affect the effect of apprenticeship from different aspects. To give full play to the guiding role of enterprise tutors, it is necessary to consider how to stabilize the tutor team and form clear and detailed instruction and evaluation rules, however, until now the relevant mechanisms are not yet perfect.

4.3
Process evaluation 1. There are significant differences in students' willingness to engage in the property management industry after graduation in terms of process evaluation Six third-level indicators of process evaluation were extracted from the questionnaire for students, and the two groups were subject to the independent sample T-test on their willingness to engage in the property management industry after graduation.
In terms of project content evaluation, the p-values of difficulty and suitability are both less than 0.05, so there is a significant difference between the two groups in whether they are willing to engage in the property management industry after graduation. This shows that the two groups have significant differences in the evaluation of the two dimensions of project content, moreover, combining with the mean analysis, the group that is willing to engage in the property management industry after graduation more approve of the difficulty and suitability of the projects. The p-value of project diversity is greater than 0.05, indicating that there is no significant difference between the two groups in this dimension. In terms of project feedback, the p-values of the attitude of teachers/tutors, the students' learning enthusiasm, and the exam results are all less than 0.05, indicating that there are significant differences between the two groups in the evaluation of project feedback, again, combining with the mean analysis, the group that is willing to engage in the property management industry after graduation has a higher evaluation on the project feedback. This shows that students who are willing to engage in the property management industry after graduation more approve of the project process, and those who do not want to enter this industry after graduation do not approve of the project process.

Students from different grades have significant differences in the evaluation of project process
To figure out the grade differences of students in the evaluation of project process, the six third-level indicators of process evaluation extracted from the questionnaire for students were subject to one-way ANOVA. The results show that the p-values of various indicators including diversity, teacher (tutor) attitude, student's learning enthusiasm, and exam results are all less than 0.05, indicating that the students from different grades have significant differences in these aspects. The p-values of the difficulty and suitability of the project content are all greater than 0.05, indicating that there is no significant difference between each grade. Moreover, the multiple comparative analysis between the various grades suggests that, in terms of the evaluation of the difficulty of project content, there is a significant difference between the freshman students and the junior year students; in terms of the evaluation of difficulty, there's a significant difference between the junior year and the senior year students; in terms of the valuation of learning enthusiasm, there's a significant difference between the freshman students and the junior year students; in terms of teacher/tutor attitude, there is a significant difference between the freshman students and the junior year students. Except for these, there's no significant difference in other evaluation indicators between students of each grade. In addition, by comparing the mean values it's found that, the evaluation of the junior year students on each indicator is lower than that of students from other grades, this indicates that, in the project implementation process, there're weak links in the project content and project feedback of the junior year projects, and it needs to be improved.

4.4
Outcome evaluation

The comprehensive vocational ability of students has been significantly improved
The comprehensive vocational ability is taken as a core indicator to measure students' ability to apply theoretical knowledge and practical skills to solve specific problems in the process of internship. Based on the actual situation of the apprenticeship projects of the property management major, the students' comprehensive vocational ability was evaluated from five dimensions of acceptance of corporate culture, self-adjustment ability, ethics and behavioral norms, apprentice growth evaluation, and apprentice job competence. The statistical data of students' comprehensive vocational ability before and after the apprenticeship projects were compared, and descriptive statistics and paired sample T-test were conducted, the analysis results are shown in Table 5. The table also includes the content of interviews with the respondents, which is used to explain the feeling or evaluation of the respondents of different roles. In the table, the p values of the paired-sample T-test of the five dimensions are all less than 0.05, indicating that there're significant differences in the comprehensive vocational ability of students before and after the apprenticeship projects, and the apprenticeship projects have significantly improved the students' comprehensive vocational ability.

The employment quality of students has improved steadily
As a third-party evaluation organization, Zhejiang Educational Evaluation Institute conducted follow-up surveys on graduates and wrote the Investigation Report on the Career Development and Talent Cultivation Quality of College Graduates in Zhejiang Province. This study extracted relevant data of graduates graduated from the property management major of school H for one year. One year after graduation, the graduates' psychological state has generally stabilized, their career development path has be-come clearer, which can better reflect the real employment quality of the students. Among the statistical indicators of the employment situations of property management majors one year after graduation, indicators of employment rate, relevance of profession, overall satisfaction with the alma mater, and salary level were compared. The four indicators are growing steadily, and it should be noted that, the relevance of profession of students one year after graduation is between 40% and 60%, which is at a relatively low level.

Measures to Improve the Talent Cultivation Quality of Modern Apprenticeship
1. Improve the fit between the talent requirements of enterprises and the personal development of students, and fully consider the appeals of students Apprenticeship should not be a stopgap for enterprises to alleviate labor shortages and obtain cheap labor, and students' participation in apprenticeship should not be a perfunctory effort to the requirements of schools. In the context of industry-education integration, modern apprenticeship should not only be able to accurately position the requirements of enterprise for talents and absorb and reserve outstanding talents for the development of the enterprise, but also offer the apprentices with a platform for close contact with industry sectors and create opportunities for them to quickly grow into leading talents with high quality and skills. Only by unifying the goals of enterprise employment and the personal development of students can the apprenticeship projects be sustainable and productive. To this end, it is necessary to fully consider the appeals of students and guide them to make career plans, and school teachers and enterprise tutors should jointly assist students in implementing the details. Specifically, we can employ new media means such as creating a WeChat official account for student internship and publishing students' internship process, experience, projects, activities, and the subsequent development of previous graduates in the form of images, reports, interview videos, and series of topics. In this way, we can guide students to actively face the difficulties in the internship and do a good job in solving problems, so that they won't feel confused before participating in the apprenticeship projects and won't regret after participating.

Build a tutor team that is capable of teaching and good at teaching
The construction of modern apprenticeship in higher vocational education schools is an interactive process of multiple parties. As important participants, enterprise tutors should pay more attention to future policies. The apprentice's professional ability should be taken as the criterion for the evaluation of tutors, and schools should formulate the specific teaching awards rules as the funding party, so that tutors can recognize their role as "enterprise tutors" from the bottom of their hearts, and position themselves as enterprise tutors to improve their own teaching ability. For enterprise employees who serve as tutors in apprenticeship projects, it's suggested the enterprises give priority to the job promotion and salary raise to the tutors according to the time and energy they spent in the apprenticeship projects and the performance of apprentices taught by the tutor, so that tutors won't have the worry of being squeezed out of the market by the excellent apprentices they taught, then they can focus on how to better serve as tutors and impart all the experience and knowledge they have to their apprentices, and smooth the process from "knowing the skills" and "teaching the skills". In this way, we could build a tutor team that not only excels at professional skills of property management, but also knows well about the laws and requirements of talent cultivation of higher vocational education schools.
3. Make efforts to improve the quality of the apprenticeship projects for junior year students Above analysis shows that the apprenticeship projects for junior year students received low evaluation results in terms of project implementation process, therefore it's necessary to take measures to improve the quality of apprenticeship projects for junior year students. The apprenticeship projects in the junior year stage are mainly post practice as housekeepers and the comprehensive practice before graduation, students can get to know the operation process of property management projects and enhance their ability to solve practical problems through these apprenticeship projects. First, in terms of project feedback, more effective feedback information of all parties should be collected from various channels to reflect the actual implementation process of the apprenticeship projects truly and effectively. Besides overseeing the project progress through periodic time node data such as work manuals, record sheets, weekly diaries, and monthly reports, it's also necessary to smooth the communication between schools and enterprises, and build effective platforms or channels for the aperiodic communication between teachers and tutors, and between teachers and students. Besides on-site visits and regular discussions, the information collection methods can also be online platforms such as DingTalk, WeChat, and WJX.cn, which can break through time and space constraints and improve the timeliness of communication. Second, the project content needs be optimized based on the feedback information, and the width and depth of the work tasks need to be adjusted accordingly; also, according to the development trend of the industry, the project content needs to be modified constantly so that it could better fit the goals of talent cultivation; at last, we also need to continuously explore flexible online and offline implementation methods for the work tasks.