Teaching and Assessing Engineering Professional Skills

Engineering students are required to have, by the time of graduation, a set of professional skills related to teamwork, oral and written communications, impact of engineering solutions, life-long learning, and knowledge of contemporary issues. Teaching and assessment of these skills, as part of ABET accreditation, remains problematic. A systematic methodology to integrate these skills and their assessment in the curriculum is described. The method was recently applied in several engineering programs and proved to be efficient in generating data and evidences for evaluation and continuous improvement of these outcomes.


INTRODUCTION I.
ABET Engineering Accreditation Criteria define 11 student outcomes (enumerated from "a" to "k") that describe what the students are expected to know and be able to do by the time of graduation. These relate to the skills, knowledge, and behaviors that students acquire as they progress through the program to prepare graduates to attain the program educational objectives [1]. These 11 outcomes are as follows: a. an ability to apply knowledge of mathematics, science, and engineering, b. an ability to design and conduct experiments, as well as to analyze and interpret data, c. an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability, d. an ability to function on multi-disciplinary teams, e. an ability to identify, formulate, and solve engineering problems, f. an understanding of professional and ethical responsibilities, g. an ability to communicate effectively, h. the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context, i. a recognition of the need for, and an ability to engage in life-long learning, j. a knowledge of contemporary issues, and k. an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.
Among these 11 student outcomes (SOs), 6 are being designated as professional skills, namely d, f, g, h, i, and j. In contrast with the 5 technical outcomes, these 6 profes-sional skills are somewhat open to interpretation by individual programs [2] taking into consideration the role of the student outcomes to foster the attainment of program educational objectives. Since the appearance of the outcome-based ABET criteria by the beginning of the new millennium, engineering programs struggled to define, teach, and assess these professional skills [3].

KEY PERFORMANCE INDICATORS II.
Infusion of student outcomes into curriculum is the way by which the program is giving to the students an efficient learning experience to master the skills, knowledge, and behaviors defined in student outcomes before graduation. Since Student Outcomes are common to all King Abdulaziz (KAU) engineering programs, the academic accreditation unit (AAU), which was established in the year 2005 to coordinate accreditation activities and assist the programs in implementing effective assessment practices, defined a set of key performance indicators for each SO. These are statements of observable student actions that serve as evidence of achieving the set of knowledge, skills, and attitudes defined by the student outcome. Over the past decade outcome indicators were efficiently used by several KAU engineering programs to: 1. Map courses learning outcomes into SOs to define the learning depth and breadth of each SO, [4] 2. Prepare outcome assessment rubrics to assess direct achievement of SOs, and outcomes surveys used as indirect assessment tools to measure the confidence level of the student in attaining the SOs.
SPECIAL FOCUS PAPER TEACHING AND ASSESSING ENGINEERING PROFESSIONAL SKILLS ing the skills that students need to master the outcome. Program enhancement requires that all opportunities for improvement be considered in both key and related courses. The idea of key courses is intended to minimize the faculty workload associated with the compilation and assessment of outcomes. It establishes an efficient process for collecting the convincing evidences required by ABET. It also solves the problem of courses taught outside the Faculty of Engineering (math, physics & humanities) and those taught outside the program. These courses are considered as related and are not required to present evidences of attainment of student outcomes.
On the other hand, courses are used to build up skills, knowledge, and attitudes specified for a student outcome through the course learning outcomes addressing that one. From this point of view the student outcome is introduced (I), reinforced (R) or demonstrated (D) throughout the curriculum. Assessment of student outcomes in (I) and (R) courses is considered as formative assessment while their assessment in (D) courses is considered as summative assessment.
Formative assessment [7] is normally used as part of the instructional process to provide the information that help to adjust teaching and learning activities in the middle of the education process. In KAU they are used in junior and sophomore courses where the outcome is introduced or reinforced. In these courses it may not be possible to measure the achievement of all KPIs of an outcome and the students are not hold accountable, as part of the course grade, for all skills, knowledge, and attitudes specified for that outcome.
Key courses for an outcome are a subset of the (D) courses addressing that outcome where summative assessments are used to generate convincing evidences of the attainment of the student outcomes. (D) Courses identified for summative assessment are normally capstone courses taken near graduation. In these courses the students should demonstrate the abilities to master all KPIs of addressed student outcomes.

SUMMATIVE ASSESSMENT OF PROFESSIONAL IV. SKILLS
Capstone Senior Project in KAU is a two semester course taken in the last year before graduation. In this course, the students are exposed to a major design experience based on the knowledge and skills acquired in earlier course work and incorporating appropriate engineering standards and multiple realistic constraints. The students work in teams in an industry simulated environment and apply modern engineering tools and project management techniques. They communicate the details of their design both orally and in writing and prepare necessary documentations, engineering drawings, technical specifications and user manuals depending on the nature of the problem. The course is used as the key course for the summative assessment of several technical and non-technical outcomes including design (outcome c) teamwork (outcome d), communication skills (outcome g), impact of engineer-ing solutions (outcome h), and modern engineering tools (outcome k) using assessment rubrics and surveys developed for these outcomes. Details of these assessment rubrics for the professional skills represented by outcomes d, g, and h are given in Tables I, II, and III successfully. On the other hand the KAU engineering programs are required to identify at least two key courses for the summative assessment of the 3 remaining professional outcomes. These courses use two standard assignments in the form of two term papers, explained here-after to assess professional and ethical responsibility (outcome f), lifelong learning (outcome i), and knowledge of contemporary issues (outcome j).
In addition to the key course for outcome f, companies' evaluation of the students' performance during industrial internship, which takes the form of either summer or coop training, is used as an additional assessment tool [6].
Summer training is taken by the students in the last summer before graduation. It consists of 10 weeks of supervised hands-on work experience at a recognized firm in a capacity which ensures that the students apply their engineering knowledge and acquire professional experience in their field of study at KAU. Students are required to communicate, clearly and concisely, training details and gained experience both orally and in writing. They are evaluated based on their abilities to perform professionally, demonstrate technical competence, work efficiently, and to remain business focused, quality oriented, and committed to personal professional development.
Coop training is similar but longer, since it consists of summer plus one semester of hands-on work experience for a total of 26 weeks. In both cases the training company is required to evaluate the performance of the trainee using the internship evaluation form given in Figure 1 AN ENGINEERING ETHICS ASSIGNMENT I. Figure 2 gives the write-up of the term paper assignment for engineering ethics as given to students. The checklist is given in Figure 3 and the assessment rubric for the assignment (outcome f) is given in TABLE IV. Students are required to use both tools to self-assess their work before submission.
A CONTEMPORARY ISSUES AND LIFE-LONG I.
LEARNING ASSIGNMENT Figure 4 gives the write-up of the term paper assignment for knowledge of contemporary issues and life-long learning, as given to students. The checklist mentioned in this write-up and used for assessing step 1 of the assignment is given in Figure 5. The assessment rubric for recognition of and ability to engage in life-long learning (outcome i) is given in TABLE V, and that for knowledge of contemporary issues (outcome j) is given in TABLE VI. Both rubrics are used for assessing the final term paper. The students are also required to self-assess their work using these assessment tools before submission.  Demonstrate deep understanding of the immediate and long-term implications to society in the creation and/or use of a product or project, and the overall potential benefits and risks to society.

G
Demonstrate good understanding of the implications to society in the creation and/or use of the product or project, as well as its relation to general societal issues.

Level Description
Mark Score E Demonstrate an excellent understanding of all major topics presented and argued with clear links between successive ideas using superb organization from a capturing introduction to a clear conclusion that builds on and provides support to the subject matter.

G
Demonstrate good understanding of the subject matter with its major points stated and argued with clear links between successive ideas, but with occasional lack of logical flow from an acceptable introduction to a reasonable conclusion.

NI
Demonstrate some understanding of the subject matter with main topics that are limited, partially covered, incorrectly argued, lack logical flow, weakly introduced, or have a non convincing conclusion.

D. Evaluation of the Student's Performance:
Please give an appropriate grade that evaluates the student ability to perform professionally using the rubric below.   2 NI Need improvement to compete with others as a job seeker. 1 U Fail to compete with other as a job seeker. 0 E: Excellent, G: Good, NI: Needs Improvement, U: Unacceptable * (Please do not allow the student to continue the training if he is absent for five days or more) This is to certify that I personally supervised the On-job training of the student whose name is shown above for the period from: / / 14 H. to / / 14 H. Corresponding to: / / 20 G. to / / 20 G.
(Please notice that the Academic Affair and Training Department at the Faculty of Engineering will not consider this evaluation unless it is signed and officially sealed) Signature of the on-job training Supervisor

ENGINEERING ETHICS TERM PAPER Assignment:
It is required for the engineering students to understand their professional and ethical responsibility. This is because the main principles that engineers should work and live by are "to hold paramount the safety, health, and welfare of the public, perform services only in areas of their competence, act for each employer or client as faithful agents or trustees, avoid deceptive acts, and conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession" (National Society of Professional Engineers, NSPE, Code of Ethics). It is the engineer's responsibility to uphold his/her position to the fullest in taking everything into account before making a critical decision. Ethical and moral decisions often have severe consequences. In this assignment, students will be required to analyze an ethical situation using codes of ethics. The assignment consists of four steps: Step 1: Each student has to select three Engineering Ethics cases from On-Line Ethics Center for Engineering and Science (http://www.onlineethics.org/) relevant to his/her engineering discipline (e.g. civil, electrical, industrial, etc.) -and submit them to the course instructor. Duration of this step is one week.
Step 2: Each student has to discuss the selected cases with the course instructor & get his approval for one of them. If the student fails to get the instructor's approval, the instructor will assign a case for him. Duration of this step is one week.
Step 3: Each student has to print out the Saudi Council of Engineers (SCE) and the National Society of Professional Engineers (NSPE) Codes of Ethics in addition to a code of ethics relevant to his/her discipline and read them. The student will use these codes to justify the arguments in the write-up of the term paper in step 4. The codes should be attached to the term paper. Duration of this step is one week.
Step 4: Each student has to read the selected case, to answer the question(s) provided, and to prepare a final report in the form of a term paper that follows the standards of KAU Engineering Journal paper. The student has to prepare his/her work according to the checklist entitled "Presentation of Engineering Ethics Case" and the rubric entitled "Assessment Rubric: Understanding Ethical and professional responsibility (Student Outcome (f))" attached to this assignment. Duration of this step is one week.

Grading:
The assignment will be graded as follows: 1. Students who do not submit the selected cases within (3) working days or the final report within (5) working days after the due date will get a ZERO Grade for the entire assignment. 2. Checklist entitled "Presentation of Engineering Ethics Case" will be applied: 3. Students who receive "Needs Improvement (NI)" grade can improve their work and resubmit it within one week of receiving their reports. No late reports for resubmission will be accepted. 4. Students' work that gets "Good (G)" will be graded using the attached Rubric. The first two KPIs in the rubric evaluate the professional appearance and attitude of the student in all class settings, while the other two KPIs evaluate the technical content of the term paper. Details of rubric will be explained by course instructor.  A. Did the student submit the selected cases on time? (If No, enter 2 points for each day late (Max 3 working days )) B. Is the report submitted on time? (if No, enter 2 points for each day late (Max 5 working days )) C. Is this a resubmitted work, enter 10 points D. Other self-regulation issues (e.g.: the student was unprepared during the discussion of the selected cases). Enter the total points (A+B+C+D) 1. The Cover page includes: Course Name, Section No, Semester and year, Report Name, Student name and SSN, Report due date, Report submission date. 2. The introduction marks the context of the new work and orients the reader (i.e., gives the reader what is the report about and some sense of what follows)? 3. In the write-up, there is a clear reference to SPECIFIC RELEVANT PORTIONS OF THE CODES OF ETHICS that supports the student's answers (for example, "per SCE code 1-2") 4. In the write-up, the student included his own and societal values as they apply. 5. The conclusion that is at the end of the work, discusses and reflects upon the work done (i.e. what was learned, justifications about subject and what will happen next) 6. The work is professional and ethical (i.e., all claims are proved or referenced and references are given in detail at the end of the work, and no plagiarism) 7. The work followed KAU Engineering Journal Format Template posted at www.aaueng.com  3 G Be punctual, enthusiastic, business focused, quality oriented, take personal responsibility for his/her actions, but usually concentrate on establishing good relations with superiors or relations based on personal benefits.

NI
Underestimate the importance of punctuality, tend to have things done with minimum level of quality and/or effort, if any, or do not recognize the need to take personal responsibility for his/her actions.

Assignment:
It is required for the students in an engineering discipline to know about contemporary issues that are recent/modern events or problems that may affect the engineering discipline or are affected by the engineering discipline. Contemporary issues can be environmental, societal, economical, technical, non-technical, or political. In this assignment, students will prepare a term paper about one of the contemporary issues. Furthermore, the term paper will be used to evaluate the student's ability to engage in life-long learning. The assignment consists of two steps: Step 1: Students will search contemporary issues and report out (oral and in writing) at least five of them using "Presentation of Contemporary Issues Checklist" attached to this assignment. Duration of this step is two weeks.
Step 2: Students will select one of the contemporary issues with approval of their instructor and search in details about it. Then, they will prepare a final report that includes Step 1 and Step 2 in the form of a term paper that follows the standards of KAU Engineering Journal paper. Students will follow the rubrics attached to this assignment. Duration of this step is four weeks. Grading: 1. The assignment will be graded as follows: 2. Students who did not submit the report within (5) working days after the due date will get a ZERO Grade for the entire assignment.

Checklist entitled "Presentation of Contemporary Issues
Checklist" will be applied. Students who receive "Needs Improvement (NI)" grade can improve their work and resubmit it within one week of receiving their reports. No late reports for resubmission will be accepted. 4. Students' work that gets "Good (G)" in step 1, will be graded using the attached Rubric. Details of rubric will be explained by course instructor. Any evidence of plagiarism will result in a ZERO grade for the entire assignment

Lapses
Self-regulation issues (this part is filled by instructor) A. Is the report submitted on time? (if No, enter 2 points for each day late (Max 5 working days )) B. Is this a resubmitted work, enter 10 points C. Other self-regulation issues (e.g.: the student was unprepared during the discussion of the selected cases    The two term paper assignments were applied for the first time in one of KAU engineering programs in fall 2012 semester. Fifteen students were able to produce acceptable journal papers for both contemporary issues and ethical responsibility. Figure 6 and Figure 7 represent the results of applying the rubric of outcome f described in Table IV. Analysis of Figure 6 indicated that the students have problems in achieving the requirements of the KPI #3: Objectivity. It was noticed that the majority of the students are in the "Needs Improvement" category. They "Evaluate and judge a situation using personal understanding of the situation, possibly applying a personal value system." This required the attention of the instructor. After re-explaining this issue, weak students were required to re-submit a corrected version of the paper in order to obtain a passing grade on this KPI. Figure 8 and Figure 9 represent the results of applying the rubric of outcome j described in Table VI. The results in both figures indicate that the students have attained the knowledge and skills specified in the contemporary issues outcome. Nevertheless the results related to their confidence level in their abilities do not match the direct assessment results. It seems that this is related to what one of the authors had identified in an ASEE paper presented in the year 2007 (See Ref. [9].) In this paper students' cultural boundaries related to the fact that results obtained from surveys, the first time they were applied, were somewhat questionable. This, in fact, makes reference to the work of Yopp and Brown [10] who defined a cultural boundary as "any attribute of a culture that inherently limits the equivalency, and fairness, of tests or other forms of quality assurance." They include in these attributes "social forms or behavioural and educational traits, or customs specific to a particular racial, ethnic, social, or religious group." In particular difficulties associated with English language skills have also been reflected in the first surveys that were used. Surveys needed to be explained to the students, question by question. It was necessary to wait until the students answer a question before moving to the one after. This process, although very time-consuming, proved to be more reliable than using bilingual surveys.

CONCLUTIONS
A methodology for teaching and a rubric-based assessment approach for engineering professional skills are presented. Skills, knowledge, and attitudes specified for a student outcome are built up through the learning outcomes of a set of courses. Depending on the level of the course, the student outcome is introduced (I), reinforced (R) or demonstrated (D) throughout the curriculum.
The assessment method is designed to produce convincing evidences of the attainment of these skills by the engineering students before graduation. The method is general and can be applied by all engineering programs to satisfy ABET accreditation requirements. On the other hand analysis of the results obtained opens gates for deep understanding of the student performance and simplifies the continuous improvement process for better teaching and learning.