Paper—Evaluation of Programme Outcomes Under the Psychomotor and Affective Domain for ... Evaluation of Programme Outcomes Under the Psychomotor and Affective Domain for Diploma Civil Engineering Students Through Industrial Training: A Statistical Study from Employers' Perspective in Malaysia

The psychomotor and affective domains are important to assess student's achievement during the industrial training period. Students are exposed to engineering practices, real civil engineering environments, and the required ethical and communication skills through industrial training. Previous research primarily has relied on the employers 'perspectives on non-technical skills of engineering graduates. Still, there are none of the studies depends on student achievement in the psychomotor and affective domains. This study aims to identify the employer's evaluation of the student's performance for psychomotor and affective domains during their industrial training. 272 students undergoing the Industrial Training in session II 2019/2020 are involved in this study. The portion of marks from the industry evaluation for the psychomotor domain is 40% and for the affective domain is 10%. The marks were analysed using descriptive statistical analysis. The Pearson correlation (r) was used to identify the relationship between each Program Outcomes (PO) with the related domain. The t-test analysis was completed to evaluate any significant difference between the performance evaluation of student’s gender. The statistical analysis discovered that most of the students were excellent in the psychomotor and affective domains. Furthermore, the r-value for each POs shows a moderately positive correlation. Therefore, the findings indicate that the designed curriculum of the program meets the industrial requirement. However, improvements will always be made to keep nurturing a good quality of future civil engineering technical personnel. Keywords—assessment, psychomotor, affective, students’ performance, employers 70 http://www.i-jep.org Paper—Evaluation of Programme Outcomes Under the Psychomotor and Affective Domain for ...


Introduction
. Programme Outcomes (Pos) in the Engineering Technology Accreditation Council Manual, 2020 PO1 Students should have fundamental knowledge related to science and mathematics.
PO2 Students should be able to analyse the specific problem related to the application of engineering.
PO3 Students should occupy with the design of the solution for a specific problem by considering the health and safety, environmental, and social aspects.
PO4 Students should be able to know how to conduct the investigation using the catalogues, standard tests, and measurements on given problems.
PO5 Students should be able to use appropriate modern techniques, resources, and information technology to solve a specific engineering problem.
PO6 Students should be able to know about the safety, legal and social. Thus, this can help to solve a specific problem related to the application of civil engineering.
PO7 Students should have knowledge and understanding of the environment and sustainability. Thus, this helps them to solve any given problem that related to the environment and society.
PO8 Students should understand their role and should adhere to professional ethics as a civil engineering technicians.
PO9 Students should have the skill to act efficaciously as an individual and as a member of a team.
PO10 Students should be able to have effective communication skills; thus, this leads them to communicate on the given problem, give and receive the instruction when involved with the engineering project.
PO11 Students should understand how to manage engineering projects and finance and able to manage projects in various disciplinary. They should be able to have the skill as a technical team or as a leader in the assigned project.
PO12 Students should be able to have the skill to participate in independent learning and able to engage with the latest information or techniques in civil engineering.
Recently, the education system in Malaysia, including the engineering education system, has been criticised by the industry for the low performance of graduates and the inability to apply the knowledge and skills learned to the job task. Therefore, the government has prepared a comprehensive action, namely the Education Blueprint, to overcome this problem [24]. Thus, this paper only focuses on the engineering education system in Malaysia to see the extent to which the effectiveness of the action taken by the government by getting perspectives from employers on civil engineering students undergoing industrial training before they start working.
Technical skills are an important element that needs to be developed for every engineering student. With good technical skills, students have a better chance of getting a promising career after graduation. In Malaysia, students in engineering must engage in industrial training to improve students' technical skills and fulfil graduation conditions. Industrial training is a program conducted to provide exposure to professional engineering practices and the real civil engineering industry. The objective of industrial training is to provide practical experience by exposing engineering professionals' practice in civil engineering to produce competent engineers or assistant engineers. Students' work experience while undergoing industrial training is beneficial as they enter the career field [17]. The industrial training has been incorporated in the Diploma of Civil Engineering curriculum by the Faculty of Civil Engineering UiTM Pahang, Malaysia, since the 2013/2014 session to align with the university's desire to increase students' diversity of experiences. The faculty has allocated a course code with 9 credits hours for the industrial training. The industrial training is compulsory for students in semester six (6) for a minimum period of 18 weeks and as a requirement to meet the 'Engineering Technician Accreditation Council' (ETAC) accreditation qualification.

Literature review
Like other universities in the world, universities in Malaysia are also concerned about graduates who can be employed and can meet the needs of the industry after graduation. In realizing the importance of producing a high competence engineering graduate for the future, the Malaysian Ministry of Higher Education (MOHE) has placed a big priority to the high learning institution to produce engineering graduate who is competitive in the future marketplace. Currently, having knowledge in the academic field alone is no longer enough to ensure that students able to get good jobs. There is an urgent need to be improved for engineering education programs in Malaysia, especially in non-technical skills among students [14]. The seven non-technical skills verified by the Ministry of Higher Education Malaysia (MOHE) that need to be applied to students are communication skills, creative thinking and problem-solving skills, teamwork skills, lifelong learning and information management, entrepreneurial skills, morale, professional ethics and leadership skills [24].
Through an industrial training program, cooperation between universities and industry can help universities provide high skills and knowledge to students and meet the industry's requirements where they are working later [26], [8]. Industrial training is defined as training that exposes students to professional skills and experience in industrial engineering practice. The Engineering Accreditation Council (EAC), Board of Engineers Malaysia (BEM) explained that industrial training is exposure to professional engineering practice in an engineering-practice environment [11]. Mat Isa [22] stated that many lecturers lack the practical experience to relate the theories and exercises presented to students. This situation causes students not to see the actual case related to engineering. Apart from gaining practical experience and new knowledge while undergoing industrial training, the students also could practice the knowledge they have learned in class [10], [20].
The outcomes of an industrial training program are not limited only to providing and enhancing students' skills but also polish their professional growth and experience. Various researchers have argued that industrial training programs are beneficial to the trio involved in the programs: the students, the universities, and the host organisations [1]. Kamarulzaman [17] and Mat [21] reported that engineering students perceive industrial training as beneficial. It can increase their knowledge, skills, and attitude. The engineering students also agreed that it offered guidance for their future careers, added value to their career opportunities and improved their qualifications after graduation. According to Zehr [36], while students undergo industrial training, they can increase their knowledge in experimentation, fieldwork, and workplace learning. Through industrial training programs, employers can also provide feedback on whether the institution's curriculum is appropriate or needs to be improved [2]. In addition, industrial training can also provide an opportunity for employers to guide and assess student talent. Mohd Shariff and Saad [25] stated that industrial training needs to be well planned so that engineering students can obtain optimal professional skills and experience that can be used in their future careers.
Several previous studies in Malaysia highlighted the importance of employability skills required from Malaysian engineering graduates based on the employers' perception. The employability of an individual depends upon assets in terms of knowledge, skills and attitudes, the way these assets are used and deployed, the presentation of assets to potential employers and the context within which the individual works [15]. Saad [29] revealed that most employers of different categories (government-linked companies, multinational companies, government agencies and small-medium enterprises) ranked effective communication skills as very important because they will affect productivity. According to Zaharim [34], most employers stated that engineering graduates still lack communication skills, and employers are not satisfied with it. Employers also reported that health and safety skills, self-management skills and teamwork skills are among the generic skills of the highest importance for employability [27].
Based on the previous studies, it was found that most of the research conducted until recently has focused on the advantages of the industrial training program and its benefits to the students. Some studies show that employers' perspectives on engineering graduates in terms of non-technical skills. However, no studies were conducted to obtain a comprehensive employers' perspective on student achievement in the psychomotor and affective domains. The evaluation and feedback from industry supervisors are essential for the faculty to ensure the curriculum content aligns with the industrial requirement. Besides, to maintain the performance and quality of industrial training programs and engineering students' professional learning. Therefore, this paper aims to present the employer's perspective on students' psychomotor and affective domains undergoing the industrial training program.

Research questions
Research questions can be summarised as follows.
1. What is the trend of students' performance for the psychomotor and affective domain? 2. Is there any significant difference between gender for students' performance according to the psychomotor and affective domain?

Description of the Industrial Training course
Industrial training (course code ECM377) is a course that civil engineering students must take before they graduate. Through this industrial training, students are exposed to real experiences related to civil engineering work. The students must complete 18 weeks of Industrial Training during the final semester at a relevant organisation related to civil engineering works. The application for the placement was on their effort to apply with proper resume and supporting documents. The faculty supervisor was appointed to monitor and guide the students along the process. This course addresses the programme outcome which related to; (1) able to design solutions for well-defined problems, (2) able to explore the well-defined problems, (3) able to use techniques, resources, engineering tools and IT equipment to solve well-defined engineering problems, (4) understand the work ethic of the technician and be committed to the job, (5) able to communicate well on well-defined engineering activities, and (6) able to engage in innovations related to technical knowledge. The assessment of Industrial Training involved all domains in learning outcomes: cognitive, psychomotor, and affective domains. The faculty supervisor and industrial supervisor made the evaluations of this course.

Assessment method
Assessment is a method required to evaluate the developmental level of student learning and the achievement of skills. Findings from student assessment may help the lecturers and university to improve their weaknesses and plan further steps for the improvement of teaching and learning [9]. The assessment for this course consists of an Industrial Supervisor evaluation which contributes 50% of the total assessment and will be highlighted in this paper. Another 50% was distributed as follows; 15% from evaluation by Faculty Supervisor, Student Placement report contribute 15%, and 10% contributed by student logbook and colloquium respectively. In addition, the domain involved in this subject are Psychomotor 50% ( 40% from evaluation by Industrial Supervisor and 10% from evaluation by Faculty supervisor), Affective 40% ( 10% from Industrial Supervisor, 5% from Faculty Supervisor, 10% from student placement report, 5% from the Student logbook and 10% from the colloquium. The cognitive domain contributes 10% from student placement reports and student logbooks with 5% each. The summary for the respective assessment is depicted in Table 2. The assessments were measured through the designated rubric to assess the respective domain according to the assessment method. The students were assessed based on their performance during the tenure of industrial training.

Sample and method
272 students of ECM 377 involved in this study, and the selection of these students are according to the total number of students enrolled in this course in Session II 2019/2020. The students' results were analysed to evaluate the psychomotor domain and affective domain's attainment from the employer's perspective during their indus-trial training period. The analysis was made based on the overall achievement comprising the percentage (%) of marks obtained for the domain involved. Besides, descriptive statistical analysis of the data involving mean, median, mode, minimum, and maximum marks is also presented. The Pearson correlation (r) was used to measure the relationship between each POs and domain, thus identifying the direction and strength of the relationship. Finally, the t-test analysis was completed to identify any significant difference between male and female students' performance.
This study focused on the Industrial Supervisor's evaluation that consists of 40% of the psychomotor domain and 10% of the affective domain, as shown in Table 1. Meanwhile, Table 3 displays the POs that mapping with the psychomotor and affective domains. The specific rubric has been provided to the industrial supervisor as a guideline for them to assess the students' capabilities. Table 4 present the evaluation criteria of the psychomotor and affective domains.

PO4
Ability to recognise the given task and capable to relate with the relevant codes/ standard measurement.
Ability to conduct well-defined problems, locate and search relevant codes/ catalogue/ standard. Ability to execute the task given with the application of knowledge / appropriate technique.
Ability to perform, gather and interpreting data which leads to the findings on the given problem.

PO5
Able to recognise the suitable standard/ appropriate techniques and modern engineering or IT tools to the civil engineering problems.
Able to imitate the appropriate techniques and modern engineering or IT tools to solve civil engineering problems.

PO8 Comply with time management and punctuality
Adhere to ethics as a civil technician and behave according to safety integrity.

PO10
Aware and receive well to the task given.
Communicate effectively oral and written to the task given and being able to comprehend the work.
The overall performances for each POs that reflected psychomotor and affective domain were analysed by a percentage of 100. It was then classified according to the scoring guide referred to the Examination Unit, Academic Affairs of the university. The scoring guide is shown in Table 5.  Figure 1 presents the distribution of students' performance in the psychomotor domain. According to Figure 1, it shows that 262 out of 272 (96.32%) students were excellent in PO4, which obtained marks between 75% -100% and another 10 students (3.67%) are considered good in the ability to conduct an investigation using standard measurement on the given problem. These performance's pattern slightly differs in PO5. It indicates that one of the students (0.3%) performed a satisfactory level of achievement. Meanwhile, 16 students, which contributed to 5.88%, are good, and 225 students (93.82%) are excellent in using modern techniques and tools to solve the specific engineering problem during industrial training. These, due to the students, has given the proper practical scale and updated knowledge regarding new technologies in the market [30].

Psychomotor and affective domain performance
Typically, students' future behaviour influenced by affective variables such as students' attitudes, interests, and values [33]. Indicators of academic achievement and other factors such as awareness, self-efficacy, self-performance effectiveness, and time management are also essential. Therefore, the student's achievement of the affective domain related to PO8 and PO10 is displayed in Figure 2. During industrial training, the affective domain was observed by a respective supervisor on the students' attitude and communication skills. About 99.26% (270 students) and 95.96% (261 students) obtained excellent evaluation from their industrial supervisor in PO8 and PO10, respectively. Another 0.74% (2 students) are good at understanding their rules and ethics, and 4.04% (10 students) are good in communications skills.

PO10 -Effective communication skill
among each other except for the mode and mean analysis of PO5. It indicates that the mode value is 80%, and the minimum is 53% compared with PO4, PO8 and PO10. It was believed that some of the students are not well familiar with new software or tools used in their placement organisation since the industrial training was their first exposure to the real environment of works. Thus, they need additional time to learn that stuff. Moreover, the analysis demonstrated that students obtained excellent evaluation for the affective domain (PO8 and PO10). The mean, median, mode and maximum marks were in the range between 88% to 100%. Gupta [12] described that students refined their social competencies, attitudes, and professionalism towards developing the profession from industrial training. Skill obtained by going through lots of exercises at university that involved communication such as group work, lab work, and presentations before undergoing industrial training has minimised conflict occurrences to understand their roles. It is important to develop soft skills through the courses at university because it positively impacts cooperation at work and personal relationships [16]. Thus, this helps them expedite the job and perform well during the industrial training session [28].  Figure 4 presents the matrix correlation between the psychomotor domain and the affective domain. It shows that the correlation (r) between PO4 and PO10 is 0.673, while the correlation (r) between PO4 and PO5 is 0.555. Both are among the highest rvalue compared to others. Referring to Akoglu [3], the r-value in the range +0.4 to +0.69 was interpreted as moderately positively correlated. Thus, it also considers that another POs has a moderate association with correlation (r) value +0.44 to +0.48. Overall, it shows that communication has a significant relation with every PO. The ability to integrate effective communication (verbal, written, virtual and graphical) would turn into work products [19]. Without proper communication skills, they will not be able to ask questions, clarify doubts, cooperate with team members, and share ideas. Hence, it will affect the works' productivity [12], [29].

Students performance based on Gender
There are 142 females and 130 males' students in this sample. Figure 5 shows the comparison of female and male attainment for each PO. From this finding, the overall results indicated that female students performed slightly better than male students in the psychomotor and affective domains. Thus, the t-test was completed to validate any significant difference between females and males. It was tested for each PO among female and male students with the null hypothesis (Ho) that female and male students' performance is equivalent. This finding seems to be agreed with the study conducted by Woodfield [32] and Megat Mohd Nor [23], where the female students performed better than the male students during the industrial training.

MALE FEMALE
The results obtained through the t-test correlation coefficient should be able to fulfil one of the following hypotheses: H0 : there is no significant difference in achievement (p-value more than 0.05). H1 : there is a significant difference in achievement (p-value less than 0.05). Based on the t-test result in Table 6, it is proved that the differences in achievement for PO4, PO5 and PO10 among male and female students were not significant. The data shows the t-Stat is less than t-critical, and the p-value is greater than the significance level (α). However, PO8 shows a little significant difference between females and males when the p-value is less than 0.05. These are due to the considerable difference in variance value even though the mean value has slightly different. If compared with PO5, even its variance differs, but it has an equivalent mean value. Furthermore, the rest of the comparison only have slight differences in means and variances, as presented in Table 7. These seem to be agreed with Sena [31] based on the study on the gender differences in students' assessment in fluid mechanics, which indicated that the female students having the highest mean marks compare to the male students but statistically insignificant between female and male students.

Strength, limitation, and usefulness of the study
The study provides information on the evaluation of the industrial employer towards the undergraduates' students while undergoing the internship at their organisation. Hence, the findings can be considered an indicator of whether the curriculum and syllabus designed have met the industrial needs of the technical skill and soft skills required for diploma-level students or vice versa. The evaluation from the organisation reflects the quality of the students. The data obtained could also help identify psychomotor and affective elements that need to be strengthened in the syllabus. The feedback from the industry is essential to ensure the quality of the syllabus contents relevant with time. Consequently, the employability rates of graduates could be increased.
The findings of this study have to be seen in the light of some limitations. The data of students' evaluation only from one of the universities in Malaysia that offered the Diploma in Civil engineering program; thus, this represents the students' performance from that institution. Besides that, there is a lack of references to similar studies, especially regarding the psychomotor and affective domain of Civil engineering students and the involvement of the industrial evaluation. Hence, this study could also motivate higher education institutions to undertake similar studies to cover a more comprehensive comparative study. Then, strategies to enhance the syllabus content and teaching method can be developed to prepare a better quality of graduates.

Conclusion
This study indicates that most students excelled in the psychomotor and affective domain based on the employer's evaluation. The overall finding shows that 96.32%, 93.82%, 99.26% and 95.96% of students were excellent in fulfilling the elements of PO4, PO5, PO8 and PO10, respectively. The main reason for that achievement was that the industrial training took place in the final year and final semester of the study period due to the curriculum design. The students have been exposed to the industry's related skills needs while learning all the courses from Semester I until Semester V. They have learned while doing laboratory works or experimental learning related to psychomotor skills. Besides that, they also learned from the tasks that involved communication skills like group work and presentations that improved their affective domain quality. All the knowledge they acquired were implemented during the practical training. As a result, they obtain good evaluations from their industrial supervisor.
Moreover, the study revealed that the r-value for each POs showed a moderate positive correlation. It means that each PO are related to others. The study also discovered that female students performed slightly better in the psychomotor and affective domain than male students. However, the t-test result confirmed that the differences in achievement for PO4, PO5 and PO10 among male and female students were not significant, while PO8 indicated little significant difference. Finally, perhaps the findings of this study can help educators recognise the most reliable and beneficial elements in preparing the graduates full of the employability capacities required by the industries.