Beyond Letter Grades: A Case Study in Developing an Evaluation System to Generate Project Performance Transcripts

—Traditional academic transcripts are usually restricted to demonstrating students’ achievements and experiences during college life. As an example of extended transcripts, this study aims to introduce the development process behind an end-user product for a project performance evaluation system (PPES), which supports conducting project performance evaluations and gener-ates project performance transcripts at a university. The first part of the study described the design process of the PPES. A user group of instructors, which continuously provided feedback on the prototype design, was formed. Conse-quently, the PPES was successfully developed and several main interfaces of the PPES were presented. The second part of the study analyzed user satisfaction with the PPES and the project performance transcripts it produced. In gen-eral, students perceived the project performance transcripts positively, and instructors were satisfied with both the PPES and its transcripts. The implications of developing a project performance evaluation system were discussed.


Introduction
Higher education faces many challenges, such as the proliferation of open and online educational resources, high expectations from businesses and organizations, and uncertainty in the labor market. Universities are striving for innovation that secures reliability, sustainability, and competitiveness. As part of that endeavor, one of the changes in teaching and learning across institutions could be a transition from "what students have been taught" to "what students have learned and are able to do." Particularly, the private sector and society-at-large want college graduates to have versatile knowledge and a broad range of skills, rather than just narrow technical skills. Students are now required to exercise their knowledge and skills to solve realworld problems [1]. Thus, universities now employ student-centered instructional strategies and encourage experiential learning, so that students can learn creative problem solving, critical thinking, collaboration, communication, and interpersonal skills. For example, project-based courses, capstone design, internships, and extracur-ricular activities provide valuable educational experiences that promote these competencies [2][3][4][5][6][7][8][9].
As the educational environment is evolving, critique of traditional academic transcripts is being raised. Academic transcripts provide a list of subjects completed and grades achieved during college. The question is whether official academic records that consist of chronological course lists and letter grades sufficiently capture student achievement [10,11]. Educational outcomes are no longer confined to grade point averages (GPAs); outcomes also include cognitive, behavioral, affective, and social development [1]. The Association of American Colleges and Universities (hereinafter referred to as "AAC&U") [12] recommends that colleges and universities "shift the focus-at all levels of education-from course categories and titles to the quality and level of work students are actually expected to accomplish" [10, p. 11]. The AAC&U also suggests that extended transcripts may allow higher education institutions to better demonstrate students' learning and growth. Another criticism of traditional transcripts concerns usability. University transcripts are frequently used for employment and school admission purposes. Employers are the primary consumers of university transcripts, but they often complain that academic transcripts do not effectively convey what students experience or how well they can perform certain tasks. Employers prefer to assess capabilities rather than read course lists [1,13,14].
In recent years, several universities attempted to develop an extended transcript [e.g., 11,14]. Some examples are co-curricular transcripts (CCTs), competency transcripts, experiential transcripts, and data-enabled electronic transcripts [14]. CCTs or experiential transcripts provide records of participation in extracurricular or highimpact activities like research projects and internships. Competency transcripts present student proficiencies achieved during coursework [14]. Each school has their own way of designing extended transcripts, but they share the common goal of overcoming the limitations of traditional transcripts and highlighting students' transferrable skills.
So far, the CCT is the most widely used type of extended transcript. It validates student participation in extracurricular activities with an official document. Although it certainly adds valuable information, CCTs provide a participation record rather than an in-depth description of student experiences or capabilities. Furthermore, CCTs focus on involvement in extracurricular activities, not academic courses. This means that the rich learning experiences of comprehensive assignments and group projects in academic courses are still excluded in official academic records. According to the National Leadership Committee for Liberal Education and America's Promise [1], national surveys from employers indicated that they want to assess graduates' ability to apply their learning to real-world challenges. Furthermore, more than two-thirds of employers believed that an evaluation from a supervisory professor who led a student's internship or project would be very useful for determining graduates' potential. This implies that employers look for specific evidence and comprehensive information about students' contexts and performance. Therefore, academic transcripts should be upgraded to include better descriptions of a student's academic accomplishment and experiences.
Developing and launching a new form of transcript that applies to all academic courses in schools can be a very complex task, but it is possible to find solutions that work in courses geared toward high student engagement including project-based courses, capstone design, and field work. New transcripts showcasing project-based work students engaged in, how they did it, what they accomplished, and their level of competency, would be useful for stakeholders to recognize students' interests, experience, and performance potential. In this context, this study aims to introduce the case of a single university that developed project performance transcripts for project-based courses.
The university in this study chose the strategy of developing a Web-based Project Performance Evaluation System (PPES) to implement new assessments and transcripts for several reasons. First, in order to employ new evaluations and new transcripts as official academic records, there must be a common evaluation framework that can be shared among appraisers. The reliability of evaluations must be assured through systematic evaluation procedures. In addition, due to the increasing workload of evaluators, methods should be sought to mitigate the burden on instructors. If the students' evaluation is carried out using a system, the resulting framework can guide a standardized process and provide appropriate assessment standards, which support systematic assessment and increase instructor convenience. As a consequence, instructors are more likely to adopt the new evaluation methods. A key factor in the development of such a system is the design of the interface. Depending on the interface design of the PPES, it is possible to determine the flexibility of the evaluation environment and reduce the burden of the evaluation, effectively presenting differentiated performance results to each student. Moreover, usability can affect the instructors' motivation to continue using the new assessments. Therefore, this study focuses on the PPES interfaces that enable instructors to evaluate the project performance of their students. The present study consists of two parts: The first part introduces the interfaces of the PPES and the generated project performance transcripts. The second part assesses instructor satisfaction with the PPES and student satisfaction with new extended transcripts.

Project-based learning and competencies
Project-based learning is an instructional strategy, wherein students actively construct knowledge and learn skills while they define strategies, collect information, analyze data, and find solutions to authentic problems [15]. Project-based learning can occur individually or as a team, but in most cases, it is done in teams to mirror realworld situations. However, working on a project with diverse people is challenging. Simply knowing relevant knowledge is not enough, and it requires a wide range of competencies. First, problem-solving skills are critically important. Problem-solving processes include "exploring and understanding, representing and formulating, planning and executing, and monitoring and reflecting" [16, p. 9]. Furthermore, students need to manage project schedules and resources. They need to work on presentations and write reports. Besides, students may experience interpersonal conflicts on the team or face unexpected obstacles. Efficient and effective communication and open collaboration are required within a team.
In the literature, project-based learning is beneficial for students to be motivated and engaged in their work [17], to integrate theories with practice, and to take ownership of their own learning [17,18]. However, Häkkinen et al. [19] criticized the current educational environment for failing to meet collaborative and social learning needs of 21st century society; they suggested merging problem-solving learning environments with digital technology. Similarly, Griffin and Care [20] stressed that schools should teach collaborative problem-solving, which requires the integration of critical thinking, problem-solving, decision-making, and collaboration skills. Employers, as well as scholars, prefer employees with collaboration skills and social learning in diverse working environments [21,22,23]. In this perspective, team project-based learning is highly suggested.

Assessment in project-based learning
As a common method of assessment, project-based learning rubrics are used widely. Although some schools or instructors may develop their own assessments, one can easily adopt the AAC&U's VALUE (Valid Assessment of Learning in Undergraduate Education) rubrics. AAC&U [24] developed assessment tools for a wide range of learning rubrics and competencies, including inquiry and analysis, critical thinking, creative thinking, written communication, oral communication, quantitative literacy, information literacy, reading, teamwork, problem-solving, civic knowledge and engagement, intercultural knowledge and competency, ethical reasoning and action, global learning, foundations and skills for lifelong learning, and integrative learning. Among sixteen rubrics, creative thinking, problem-solving, information literacy, inquiry and analysis, collaboration, and communication are closely related to projectbased learning. As another example, the project rubrics from Greenstein [25] include identification of topics, planning, research skills and strategies, analysis and synthesis, organization, accountability, recordkeeping, problem-solving, creativity, metacognition, multimedia, collaboration, and presentation.
However, evaluating projects with rubrics does not overcome the limitation of traditional transcripts. Although students may receive rubric-based feedback from instructors, traditional academic transcripts only show one letter grade of A, B, C, D, or F. Letter grades are insufficient to capture details about the project, competencies demonstrated, or results-based outcomes. Therefore, it is imperative to find new ways of demonstrating student development and performance in project-based learning.

Extended transcripts
Although university transcripts are primary documents in the hiring process, traditional academic transcripts are limited in scope. Employers consider experiential learning as a key qualification. Paid internships before graduation show a positive relationship with employability after graduation [27]. Experiential learning, according to Moore [28], occurs when knowledge and meaning are taught and found in the real world, not in the classroom. Moore stated that experiential learning includes "how the student handles pressure, deals with authority, works with people different from them, how hard they work, and so on" (p. 82). DelBanco [29] emphasized extra-academic experience at colleges and universities, which can help students find their talents and passions. DelBanco criticized that if a university seeks to develop students fully, then traditional transcripts are an inadequate way of presenting students' development.
Transcript innovation has begun at several institutions [30,31]. For example, Elon University in North Carolina has provided extended transcripts, called the Elon Experience Transcript (EET). Elon is a mid-sized, private university that for over 20 years has offered extracurricular programs and emphasized experiential learning. The extracurricular activities at Elon University focus on five key areas: global education, service, leadership, undergraduate research, and internships. Students need to participate in at least two extracurricular activities before graduation. Previously, the Office of Student Life at Elon printed the extracurricular participation records at students' request, but few students used the service. The document was issued separately from academic transcripts, making it unattractive to students. However, starting in 2013, extracurricular participation records were integrated into the official transcript issued by the university registrar. Since then, EET utilization has increased rapidly. Parks and Taylor [31] conducted a survey to investigate perceptions of the EET. Surveys were sent to EET recipients, including employers, college admissions personnel, Elon students, and alumni. Results showed that respondents were generally satisfied with the EET, and employers perceived that EETs helped each applicant differentiate her/himself from other applicants.
Recently, Elon visually redesigned the EET to enhance simplicity. Parish, Park, and Fryer [32] found that the majority of respondents reported that the visual experiential transcripts were appealing and easy to understand. They concluded that the utility of the EET would improve with greater use over time. Furthermore, the information that students preferred to add to the EET were examined. Among students, leadership positions, information about where a student worked, on-campus work experience, skill acquisition, and learning outcomes were ranked as top priorities. Besides Elon University, the University of California (UC) -San Diego [33] also provides co-curricular records that highlight student involvement and achievement in the following areas: research and academic life, campus engagement, communitybased and global learning, and professional and career development. Transcripts at the State University of New York at Geneseo include participation records in high-impact activities (e.g., undergraduate research program, internship, service learning, etc.). Stanford University currently provides eTranscript services and is expanding official transcripts to include more details about courses and outcomes [10]. In summary, extended transcripts provide more detailed information about a student's experience, achievement, and characteristics.

3
Part I: PPES (Project Performance Evaluation System) Design and Development

Methods and materials
Research context: Building student competency is a major agenda item in South Korea (hereafter referred to as Korea). Universities in Korea redesigned their curriculum to focus on competencies and developed various extracurricular programs for student development. The present study was conducted at a large, four-year private university located near Seoul; the university operates a 16-week semester. As a part of the school's educational innovation plan, the intensive four-week project-based courses (IPjBCs) for third-year students in several academic disciplines were developed and implemented during the second semester of the 2019 academic year. The course is intentionally intensive to eliminate obstacles to team projects. According to previous research [34,35], although students recognized the value of team-based learning, they often complained that team projects were time-consuming. They were frustrated by the unfair workload and unequal participation among team members. Students were busy with other classes and work schedules, making it difficult to find enough time to work together on group projects. A study [36] reported that college students prefer to divide work fairly and effectively reduce team interaction. However, team creativity can be achieved when a team has frequent interaction and close collaboration. Since the core learning objective of IPjBC is creative problem-solving, the university reorganized the academic calendar for third-year students. Students complete all their other classes, except IPjBC, on a 12-week schedule and reserve the four-week block at the end of the semester for IPjBC. Therefore, students enrolled in IPjBCs complete their other classes before the start of IPjBCs, and they can fully commit to their projects during those four weeks.
The school then pondered how to motivate students, facilitate educational experience at IPjBCs, and appropriately demonstrate involvement and achievements. Academic transcripts were a major concern. Even if instructors monitored and evaluated team products and presentations, student transcripts would show an over-simplified letter grade. In this context, the school decided to develop a project performance transcript for IPjBCs. As a part of the official academic record, it describes student competencies on team projects. Furthermore, a web-based system for evaluating students' project performance was developed to support instructors in using the new assessment method.
PPES design and development procedure: The guidelines established for developing the PPES were to develop a system that is simple, easy to use, flexible, and supportive. The development of the PPES went through five steps (analysis, design, development, implement, and evaluation), and several prototyping tests with users were done during the design phase. Having recursive prototype testing with users in the early stages helped detect problems early and tailor functionality to meet user needs. It saved time and costs by remedying identified problems before production. Seven of the total 33 professors who will teach IPjBCs in the next semester were chosen as the PPES user group. They were intentionally selected from various academic majors to take into account a wide range of subject themes and project types. Their specializations were business, engineering, science, Korean literature, health science, and education. The role of user group was to provide feedback on the system design.
First of all, the researchers conducted a task analysis to construct the project performance evaluation procedure and design an interface for each step. The main steps involved 1) setting competencies and detailed assessment standards, 2) setting peer evaluation, 3) setting project teams, 4) conducting project performance evaluation, and 5) reviewing and finalizing student evaluation results. In addition, it was decided to develop the PPES as an extension of Moodle, the current learning management system (LMS) in the school. This streamlined access allows instructors to manage courses and conduct evaluations in one system.
During June 2019, the researchers held an initial meeting with the user group to introduce the new assessment and the process of assessment. At the meeting, the user group understood the need for performance evaluation and discussed about some ideas to improve the feasibility, flexibility, and usability of project performance evaluations. For example, it was suggested that the name and photo of students should be displayed on the evaluation screen to make it easier to identify students. In addition to items in the rating scale in peer evaluation, they also suggested a feature for comments from team members. Afterwards, the researchers created an evaluation system prototype with an animated slideshow based on the results of the discussion at the first meeting. The design document with detailed explanation was sent to members in the user group for review, and the written feedbacks from all seven members were collected in two weeks. The key questions were provided regarding the steps of selecting competencies and the standards, the screen interface of assessing students, the opinion about the printed version of assessment framework, the final appearance of the assessment results, and peer evaluation interface. In general, they responded positively about the assessment procedure, peer evaluation, and the interfaces. But they expressed that they preferred an even a simpler step for selecting competencies and the assessment standards. They preferred to click boxes rather than using a dropdown menu to mark a particular standard. They requested a memo function for instructors to write comments on each team's project, and they suggested developing a manual for users.
Interfaces were revised based on their feedback, and the revised design document was sent to the user group. The key design questions were regarding the appropriateness of assessment processes and functions, the intuitiveness of understanding the menu and assessment standards, and the feasibility of conducting the assessment with using this system. All members provided written feedback. They expressed that the procedure was simple and clear, and the interface of assessment was very easy to use. They also liked the review step before finalizing the students' evaluation results. However, a couple of users suggested reducing the number of detailed standards and using simpler language.
After compiling feedback from the user group, the second meeting with the user group was held to update the status of the system interface design. Users shared and discussed their opinions of reviews and explored further issues. The question was raised whether to include a student him/herself in the team peer evaluation. The decision was made to provide an option for instructors to decide in the peer evaluation settings. In addition, using the PPES in individual project situations was also considered. The third interface detail document was created and sent to the user group for final confirmation. All members were agreed on the PPES design. Development proceeded based on the final system design document. At the end of PPES development, user acceptance testing was done by the researchers and system administrators at the school. In summary, working with the users was very helpful to improve the feasibility of PPES implementation because various circumstances were taken into account, and three rounds of revisions resulted in a more user-friendly PPES. The following section describes several key interfaces of the PPES.

3.2
The results of PPES development Step1: Setting competencies and detailed assessment standards: The menu for project performance evaluation is provided in the LMS (e.g., Moodle). If the menu is selected, then several tabs in the next window appear in order of the evaluation procedure (see Figure 1). The first step is to determine competencies and assessment standards. The project performance evaluation in this study was based on the observation of behaviors and attitudes while completing the team project; evaluation was not based on the outcome of the project itself. The competencies and standards for evaluating IPjBCs were pre-determined to share consistency across the various IPjBC classes. Details about the process of developing the standards used in the PPES are discussed in Development of project performance evaluation framework in project-based learning at universities: Design-based research [37]. The pre-determined competencies for IPjBC are creative problem-solving, inquiry and analysis, project management, collaboration, and communication in written reports and oral presentations. PPES provides 10-12 detailed assessment standards for each competency. For example, "applied optimal solution among many other solutions and strategies" is one creative problem-solving assessment. "Accurately analyzed data using the most appropriate methods and procedures" is a sample assessment standard for inquiry and analysis. "Predicted risks and/or responded flexibly and quickly to unexpected situations" is a sample assessment standard for project management; "created a supportive team atmosphere, by suggesting practical solutions rather than placing blame for mistakes" is a sample assessment standard for collaboration; "utilized materials (pictures, videos, tables, etc.) systematically to suit the purpose of the presentation" is a sample assessment standard for communication. The interface for instructors to set competencies and specific assessment standards is shown in Figure 1. Instructors of the IPjBCs simply select the button for the pre-determined competencies and evaluation standards that they want to use. Instructors are also free to edit or add new competencies and standards by clicking the edit button.
Step2: Setting peer evaluation: The next step is to set peer evaluation (see Figure  2), which applies to the members within a team. Instructors set the peer evaluation items, including the activation period and whether a student can evaluate him or her-self. Two items per each competency were developed for peer evaluation; a total of ten items were provided in the PPES. Some sample items are: "Did he/she contribute to systematically organizing the project and advancing the project?" "Did he/she immediately share information and contribute to open and active communication?" "Did he/she contribute to problem-solving by suggesting new ideas and solutions?" Four items related to project management and collaboration were designated as mandatory. Other items were freely selected and edited by the instructors. Besides using preexisting ratings, peers can write a comment for each team member. Students can use the peer assessment feature for a pre-determined period of time.
Note: Some parts of Figure 1 are translated from Korean to English.  http://www.i-jet.org Step 3: Setting project teams: Once the assessment standards are set, instructors need to set the project teams. The interface is shown in Figure 3. A list of students enrolled in the course appears on the left. Instructors create a team by assigning students to each team using the drag-and-drop feature. Instructors can also add a project title for each team by clicking the edit button. The project title is displayed under the team's name. In the case of individual projects, then instructors assign one student to each team.
Step 4: Conducting project performance evaluation: The most important interface in the PPES is the student assessment tool. As shown in Figure 4, team members are listed, and selected competencies and standards are displayed. Instructors click a specific standard if a student has met that standard in the IPjBC. Competency either can or cannot be demonstrated. Therefore, Martin [38] suggests that competency should be attributed either a pass or fail grade. Only clicked items will be shown in the project performance transcript. Instructors can use the PPES anytime, either while coaching students or providing student evaluation. Often, feedback from team members is valuable for instructors to assess student involvement, particularly in the area of collaboration and project management skills. Instructors can access the comprehensive peer evaluation results by clicking the peer evaluation button below the student's photo. The results of peer evaluation display in a pop-up window (see Figure 5) and are summarized by averaging the score from team members who complete the peer evaluation. The comments from team members are shown as well. The reason for implementing peer evaluation as a pop-up window on the main evaluation screen is to enable instructors to refer to the evaluations and comments from team members at the same time that they assess each student.  Step 5: Reviewing and finalizing the student evaluation results: The content of the project performance transcript is automatically generated according to the evaluation of the project performance entered by the instructor through the PPES. For example, if a student has demonstrated four out of ten listed competencies in creative problem-solving and three out of nine listed competencies in collaboration, the professor would only check the boxes for those demonstrated competencies. Therefore, the project performance evaluation results will only show the specific competencies that the student has demonstrated mastery. Before finalizing the project performance transcripts, instructors will go through them for final review. If necessary, instructors can add or modify student-specific content. Figure 6 compares two students' results. This indicates how each student's role, competency, and degree of participation differed. A message is automatically generated in red to draw the instructor's attention when none of the detailed standards of a certain competency have been met. It says "no significant contribution was observed in regard to [insert particular competency];" PPES will auto-fill the competency field. Instructors can leave this statement as is, delete it, or edit it accordingly. When the final project performance transcript is issued to students, all text is displayed in black. The final project performance transcripts also include the following items entered in the PPES: the team project title, a brief description of the project, and a review opinion on the project outcome. When review is complete, the evaluation data will transfer to the Office of the Registrar, and project performance transcripts will be generated. Part II: User Satisfaction Evaluation

Methods and materials
Research context: A total of 33 IPjBCs were opened during the second semester of the 2019 school year; 805 students completed the courses. The project topics varied according to their academic disciplines. For example, students in the mechanical engineering major developed self-driving cars for use in mazes, while students in archi-tecture developed and implemented renovation proposals for a space in their building. Students majoring in Korean literature reached out to a number of community centers in the local area and created easy and accurate brochures for services. In other classes, some teams developed Apps for searching and introducing historical sites, while others proposed products and services for people with special needs. The classes were scheduled from 9 to 12 o'clock from Monday to Friday, but many teams worked extra hours according to their own project schedule. Students managed and worked on their project for four weeks. Instructors coached the project teams and sometimes field experts were invited to provide consulting to students' work. During the last week of IPjBCs, peer evaluation was conducted using the peer evaluation function of the PPES.
Participants: Students: When IPjBCs ended, 805 students received their grades and project performance transcripts. An online survey was conducted for students. Survey participation was voluntary and informed consent was obtained. Among the 112 undergraduate students who responded, 51 were male and 61 were female. Their majors varied, and included computer science, mechanical engineering, architecture, business, social welfare, Korean literature, design, and statistics.
Instructors: During the 2019 academic year, 22 academic departments opened 33 IPjBCs. Some departments divided the IPjBC into two courses due to the large number of students registered. A total of 33 instructors taught IPjBCs, and they were all full-time professors in the target university. An online survey was conducted for the instructors after the courses were completed. Four of the seven instructors in the user group were among the twenty total instructors who responded to the survey.
Instrument: User satisfaction scale for students: The student questionnaire was developed to identify satisfaction with the new project performance transcripts. The survey measured 10 items. Students responded to the questionnaire on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). The internal consistency of this scale was .94 (Cronbach's alpha). Cronbach's alpha [39] is widely used to demonstrate instrument reliability or internal consistency [40,41]. This is important as it affects the reliability of the research results based on how they were measured. While there is no absolute cut-off to an appropriate Cronbach's alpha value [41], the Cronbach's alpha value of .94 obtained in this study can be considered as high, given that a Cronbach's alpha value of ≥ .70, is generally considered an acceptable reference value in many social science studies [42]. At the end of the survey, an open-ended question was provided for optional comment.
User satisfaction scale for instructors: After the instructors completed the project performance evaluation for their students, a PPES user-satisfaction questionnaire was administered. The questionnaire consisted of 10 items and responses were recorded on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). The internal consistency of the PPES instructor satisfaction scale was Cronbach's alpha = .
89. An open-ended question was provided for optional comment.

Results
Student satisfaction results: The student survey results are presented in Table 1. In general, students positively acknowledged the new assessment approach. About 59.82% of students thought that project performance evaluation was an appropriate evaluation method for project-based courses (see Q1), and 66.07% perceived that the adoption of project performance evaluation was worthwhile (see Q3). Furthermore, the item with the highest score explained that students put more effort into their project as a result of the new evaluation process (see Q4). About 60.71% of students showed satisfaction with their project performance transcript results (see Q5) and they thought project performance transcripts reflected their project engagement and performance skills (see Q6). In addition, 53.57% of students expected that project performance transcripts would help the hiring process and graduate school admissions (see Q10). The items with low negative responses were Q3, Q4, and Q6. These results imply students' overall satisfaction with the project performance assessments and transcripts.
The results from all the items showed more positive responses than negative ones, but there were a couple of questions that had relatively high negative responses. For example, about 32% of students did not respond positively to applying this new assessment method to other courses (see Q9); For Questions 7 and 8, which are related to peer evaluation, and about 20% of the negative responses reflected that students felt that peer evaluation did not fairly evaluate participation. Instructor satisfaction results: The instructor survey results are presented in Table 2. The instructors were highly satisfied with the ease of using the PPES (M = 4.00). They were pleased with features that allowed them to review each student's results before confirming the transcript (see Q3, M = 4.10). They reported no difficulty using the interface to assess student performance (see Q2 and Figure 4), and they showed high satisfaction with the use of the PPES (see Q6). The instructors appreciated the value of project performance evaluation (see Q8, M = 4.20), and they preferred conducting project performance evaluations using the system, as compared to not using the system (see Q9, M = 4.05). In all items, negative responses were rare. However, the comments from the survey indicated that almost all instructors were able to use PPES easily, but a couple of the instructors, who had a large number of students, reported difficulty identifying each student's progress and performance within a team. This issue is crucial and could be helped by reducing class sizes. This feedback implies flaws in class registration, rather than the use of the PPES itself.

Discussion
This study detailed the development of a project performance evaluation system that produces integrated project performance transcripts. The design of the PPES was carried out on the basis of the results of the task analysis. The PPES consisted of five steps: 1) setting competencies and detailed assessment standards, 2) setting peer evaluation, 3) setting project teams, 4) conducting project performance evaluation, and 5) reviewing and finalizing students' evaluation results.
The first part of the study explained the interfaces utilized in each step of the PPES. According to new technology acceptance models (e.g., TAM), 'perceived ease of use' is a crucial factor for users to adopt new technology [43,44]. Since usability is closely related to interfaces, interfaces play a critical role for determining user satisfaction and intent to adopt the new system. In this study, efforts were made to pursue the PPES simple, easy-to-use, and flexible. The interfaces of PPES were developed to be as intuitive as possible and to minimize the workload of the primary user groups (instructors). Specifically, in order to reduce the workload for instructors, the project performance transcripts were automatically generated by clicking pre-set performance standards. Providing pre-determined competencies and standards within the PPES had the advantage of providing a framework for evaluation and leading instructors to easily complete the evaluation. PPES also achieved flexibility by allowing instructors to freely modify competencies and assessment standards according to course objectives.
The second part of this study dealt with user satisfaction. As a result, both students and instructors involved in IPjBCs recognized the need for and the value of project performance evaluation. They perceived that the transcripts generated helped students prove their competencies and add educational value. Particularly, the results showed that this new method of evaluation which assess students' detailed proficiency in various competences drives students to engage more in their projects. Furthermore, instructors responded that using PPES for performance evaluation was helpful, and they were very satisfied with the ease of using it.

6
Contribution to the Field First, this study provided higher education institutions with an idea and tangible example of performing competency-based assessments systematically and presenting students' capabilities using of a new transcript. Many educational practitioners have emphasized on instructional transformation in classrooms and various studentcentered instructional strategies have been applied increasingly, but the transcript system remains the same, limited to demonstrate students' experience and competencies. Therefore, this study undertook the crucial and relevant task of initiating diverse ideas and discussion around transforming the academic performance evaluations and transcript system. Second, the study presented all the main interfaces of the PPES, which were developed in LMS (Moodle). This contributes to promoting adoption and diffusion since most traditional and online universities use LMS as an important communication and learning tool. This makes developing the PPES -an extension of the current LMS rather than an independent systemeasily taken into consideration by other schools aiming to add a performance evaluation function that is both streamlined and costeffective. The interfaces introduced in this study serve as guidelines for developing similar systems in other schools. Therefore, this study imparts significant contribution for educators and the educational system as a whole by setting forth much needed design ideas for updating and evolving the evaluation system.

Conclusion
The present study generated several conclusions. First, the new proposed transcripts in this study have illustrated how students' rich experience and competencies from academic projects can be shown in the official form of transcripts. Particularly, as previous studies [26,27] discussed, student experiences such as internships and project-based courses are important considerations in employment recruitment processes and the project performance transcripts are expected to help in showcasing each individual's competence and uniqueness.
Second, it is worth noting that using a common evaluation system increases the feasibility of adopting the new evaluation strategy. Implementing changes to the status quo, such as adopting new strategy or changing existing educational policy is challenging due to people's resistance. The traditional academic transcript system has been around for so long that it may make it difficult for both instructors and students to adopt the new assessment strategy and the new form of transcript. To function as the school's official transcripts, some degree of common standards and consistency is required. At the same time, there should be autonomy for instructors to customize assessment criteria and determine the scope of assessment. From this point of view, the system approach described in this study was able to give instructors flexibility while sharing a common evaluation framework. It is believed that the PPES is flexible enough to be adopted by many other educational institutions. For example, the AAC&U Value rubric or other criteria can be easily entered in the PPES. Because the performance assessment method is suitable for student-centered learning, including capstone design and internships, the PPES can be applied to many courses and considered a comprehensive student performance evaluation platform.
Finally, as all performance evaluation results of the PPES are managed digitally, it is possible to upgrade to eTranscript in the future. As mentioned in the case of Stanford University, the electronic distribution of academic transcripts is increasing. In addition, digital data is highly compatible and easily manageable with other university systems, making it easy to convert data and manage academic records in the future.

Limitations and Future Research
This study presented a novel case for extended transcripts, but requires follow-up studies that consider the limitations. The system was newly developed and applied once in the field; it still needs improvement. Therefore, it is necessary to observe whether the system can adequately respond to various instructional situations. Repeatedly collecting feedback from users could continuously improve the evaluation system. In addition, future research to improve extended transcripts should take into account the perceptions and needs of various stakeholders.