Come Together: Peer Review with Energy Engineering Students

Based on the quantitative and qualitative analysis of collected data, this work examines energy engineering students’ perceptions towards web-based peer assessment, distinguishing their role as reviewers and as feedback receivers. It analyses the type of feedback in an attempt to evaluate students’ written communication skills and identifies which factors are related to students’ use and appreciation of feedback. Taking in account the conclusions of our previous case-study in the civil engineering course at University of Trás-os-Montes e Alto Douro (UTAD, Vila Real, Portugal) the results of this study allow us to deepen the reflexion on how to improve the design and implementation of future web-based peer assessment tasks in engineering courses.


INTRODUCTION
The use of peer review as a learning facilitator is not new and has been widely applied in various scientific domains and collaborative learning contexts [1] [2] [3]. As a tool that promotes active learning [4], it helps building capacities related to diagnosis, evaluation, synthesis and communication [5] [6] [7] encouraging at the same time the development of writing abilities [15].
In the Bologna context of higher education in Europe, the development of effective written and oral communication skills [8] is considered to be one of the goals [9] [10] [11] for students. Web-based collaborative activities, such as web-peer review can help achieving it [12]. Students who come to the University generally have limited capacity to use and apply these skills and they have poor discipline for individual learning, regardless of their prior training area. At the same time, the role of teachers is changing emerging more as facilitators [13] and with the necessity of grasping growing opportunities for learning and also for applying/using communication technologies. When doing the review of colleagues' work, students have the opportunity to critically analyse, comment and participate actively in the overall learning process [14] [15], but they do not always accept unreservedly to participate in a project of peer review [16].
In addition to general critical factors for its effective use, research in a higher education context has been recently focusing on web-based collaborative writing activi-ties and peer on-line assessment approaches. Some studies evidence students' resistance to peer review [16]. However, they are not conclusive, on which factors lay behind it. Furthermore, there is little understanding of the impact of students' perceptions of peer reviewed work [16]. Moreover, the effects of the results of a "good" feedback are far from being characterized, whether in studies which relate the type of feedback with the improvement of the quality of the work and/or with the received grades or even from the perspective of the position of students as authors or reviewers [17].
This work presents results of a study carried out in a higher education course at the University of Trás-os-Montes e Alto Douro (UTAD), in which energy engineering university students from the Industrial Management course, were challenged to reinforce a set of personal and cognitive competencies, using a web-based collaborative writing environment and a peer review approach in accordance to Bologna guidelines.
Taking in account the conclusions of our previous work with civil engineering students [18], and based on the analysis of data collected from a questionnaire applied to students who participated in the activity, and in the analysis of the assignments produced by students in the Google Drive environment, this work has the following objectives: • Describe students' attitudes and perceptions toward on-line based feedback from the perspective of the students who receive feedback, and from the perspective of the students as feedback providers. It will attempt to answer to the following question: as reviewers, are the perceptions of the students on the activity different from those as feedback "receivers"? • Analyze the type of feedback in an attempt to evaluate students' written communication skills. • Verify the impact of the inexistence of intermediate rates grading from the teacher on the use of peer's written feedback. • Test the relationship between the final grades of the teacher and of the reviewers and see if an improvement (in terms of grades) can be noted from one work cycle to another.

II. BACKGROUND
Although, helping engineering students to learn how to write and communicate efficiently offers many challenges [19], in a context where writing and collaborative skills PAPER COME TOGETHER: PEER REVIEW WITH ENERGY ENGINEERING STUDENTS should be promoted [19] [11] [12], in particular in the engineering curricula [8] [10], the implementation of the cycle involved in giving/receiving/using feedback can be considered an adequate strategy to provide students with a meaningful learning experience.
For teachers, giving feedback to a large number of students may be too difficult considering the variety of students' characteristics and the limited time available [20]. Therefore, peer review activities can be an alternative or complementary method of assessment.
Discussions on the use of collaborative environments in higher education and improvement of students' written production have been going on from several fields' perspectives. Concerning peer review activities, the literature reports some important discussions in the field of Computer Supported Collaborative Learning [21] [13], including the use of a Google Drive environment [22] [23], as well as in the field of educational psychology [4] and in studies on the impact of feedback on writing [24].
According to some authors online collaborative environments can help the development and acquisition of communication skills presented in the engineering curricula [11] [12]. Peer review activities can therefore be developed in an on-line environment, but the learning process may reveal a greater degree of complexity when compared with that performed in the classroom [4]. It also involves greater motivation and effort from the teacher. Regarding the use of the Google Drive more specifically for this type of activity, many studies report its benefits [23] [26] [23] [22] [27] enhancing the greater quality of a document when it is written collaboratively, compared to an individually written one. A widely discussed aspect amongst the on-line peer review studies is the type of review: anonymous or not. Underlined are the advantages of anonymous review in fostering a more critical and honest expression of opinions, free of any pressures of interpersonal factors [2].
Brodahl, Hadjerrouit and Hansen [22] show that students' computer skills and positive attitudes towards the use of digital tools play a crucial role in the perceived quality of the final document. However, the problems encountered are not exclusively of a technological nature. They also depend on other factors such as the course content, the pedagogical approach used, the time set for the activity, familiarization with the tool, prior knowledge and institutional and administrative restrictions [22].
In relation to the effective use of feedback by students, besides anonymity, critical factors are pointed out by the literature, such as: students' evaluation of their colleagues' competencies in giving feedback, trust existing amongst peers [28], the type of feedback and students' perception of fairness [16] in the process.
As to the benefits of peer review, besides the ones stated above, some authors [7] [5] [6] refer: promotion of active learning [4] and autonomy, increased students' engagement and social interaction, significant increase in the amount of feedback students receive, improvement of skills related to the diagnosis, evaluation, synthesis and professional communication, development of new ideas and critical thinking [29]. At the same time students provide to the peer assessed with external feedback [30], they observe how they solve problems, and learn to think critically.
Concerning the type of feedback more specifically, for Shute [20], feedback provides guidelines when it is specific and clear. It is effective when it provides information about the progress and/or on how to proceed. Therefore feedback should be clear, purposeful, meaningful, and compatible with students' prior knowledge and should also provide logical connections [31]. However, it's important to consider the influence of other variables, like students individual characteristics (e.g. level of cognitive skills, motivation) [20]. In particular some authors refer to the influence of the presence or absence of the teacher in the process of review/assessment [17] [24]. When feedback is given only by peers, it seems to be less enforcing [24] than when the involvement of the teacher is also present in the process. For Hattie and Timperley [31] effective feedback should consider three key issues that both students and the teacher should address: where am I going? (what are the goals?), how am I going? (what progress is being made toward the goal?), and where to next? (what activities need to be undertaken to make better progress?).
In order to help students have better performance in terms of writing, teachers should analyse and record the types of feedback which have more impact. Several studies focus on the features of a "good" feedback [20] [31] [24] [32] [33] [30]. In general they enhance the necessity of beginning with a summary of the evaluated colleague's performance, as happens among professional reviewers of articles in scientific journals [32] [33] [30]. Moreover, when a problem is detected and outlined, the location of the problem should be indicated by the fellow reviewer, as well as a proposal of solution to the problem [30].
Nelson and Schunn [24] presented a schematic grid dividing it into two fields of the characteristics of a "good" feedback. In the cognitive field, there are four essential characteristics which affect understanding: 1-Summarizing: the summaries are intended to make an overall assessment of the work and indicate the parts that need special attention by the feedback receiver; 2-Specificity of comments, involving three components: the identification of the problem, which allows an increased probability of implementing the feedback; the provision of a solution, with a comment that suggests how to deal with the problem; and the location of the problem/solution (with the location of the problem/solution the author gets a second chance, after identification, to detect a problem that might have been forgotten or not perceived). Comments can be global or specific, although specific ones are considered to be more useful; 3-Provision of clear and concise explanations: the explanations convey or clarify the reasons for the purpose of feedback; and 4-Scope: deals with the spectrum of the feedback (narrow or large). The only characteristic in the affective group (which affects the agreement of the author) is the use of affective language, which includes the use of praise (and not of "inflammatory language", which is considered as non-constructive criticism), or the use of comments which mitigate or soften the criticisms.
However, the results of a "good" feedback are still being quantified or characterized. The relations between the type of feedback and the quality of the work are on investigation. The discussion of its benefits from the perspective of the position of the students as reviewers or as writers [17] is still going on. On this latter aspect and according to Ozogul and Sullivan [30] the peer evaluation has PAPER COME TOGETHER: PEER REVIEW WITH ENERGY ENGINEERING STUDENTS the potential to be beneficial to both the assessor and the assessed students. However, Lu and Law [2] explain that students benefit most when they assume the role of reviewers. Other research stresses the importance of feedback being more qualitative rather than quantitative, as well as done timely [28]. Aspects related to grading by students are also discussed in references [28] [2] [17], such as the students' skills to assign grades to their colleagues and to do the task of revision (when compared to the teacher's skills [24]). Peer grading on the other side seems to have very limited effects on the learning performance of both students involved [2]. However, some benefits are pointed out by Sadler and Good [17] in peer grading: it can substitute teacher's grading, when the results of these grading practices are comparable with those of teachers. Finally, the effects of feedback on assessed students are complex and difficult to evaluate, because they need close monitorization [2].

III. CHARACTERIZATION OF THE ACTIVITY
The activity under analysis was carried out at the midcourse level (1st semester of a 6-semester program) of a Bachelor of Energy Engineering course, focusing only on one component of the syllabus of Industrial Management course. This course allows students to get acquainted with the business world and to learn about the main management function of an industrial company. In particular, one of the cognitive objectives of the component of the syllabus at stake was to familiarize students with the exercise of the SWOT (Strengths, Weaknesses, Opportunities, and Threats) economic strategic analysis approach. In this case, students had to learn how to identify and characterize the opportunities and threats that companies (and other agents) face according to their economic, technological, legal and the political/social/cultural environment.
As it was described in our previous work [18], the activity consisted of the following tasks which were to be performed individually by the author-students during the first week of the activity ( Figure 1): select an article, create a Google Drive file for their work, make the summary of their article in the file, do the article SWOT analysis and issue a personal opinion about the chosen article. Once these tasks were done, the document was shared through the on-line environment (Google Drive) with the assigned (by the teacher) student-reviewer and with the teacher. The reviewer had one week to read the document, give written feedback with suggestions for improvement to the author through the comment function of the Google Drive (similar to the Microsoft Word tool), without giving any grade. Then, the student-author had one week to make voluntary improvements to his/her original document. Once these improvements were done (or not) the reviewer student graded the final work of the "author" with a grad-ing scale from 0 to 1. Finally the teacher evaluated the entire work of the "author" and of the "reviewer" with the same grading scale.
The design of this activity took in account the results of our previous work [18], eliminating the intermediate grading by reviewers and teacher. That decision was made due to some evidence: on one hand intermediate grading by reviewers and teacher (at stage 2 of the cycle) influenced "authors" negatively for the use of given feedback (especially when the grading was already perceived as high), on another hand students appeared to overestimate the teachers' revision and grading in relation to those of their mates.
Therefore the evaluation/grading of this activity was done in a final stage by the reviewer to the author and then by the teacher to the "authors" and reviewers (in a scale going from 0 to 1). This procedure allowed to compare grades given by students and teachers on the same assignments and reduced the workload of the teacher (which, with intermediate evaluations, was very high in the previous experience [18]). The final grade for this activity consisted of the average from the teacher's grades given to the four assignments (as authors and reviewers). In turn, this final grade contributed to one third of the total grade of the curricular unit (on the current grading scale from 0 to 20).
This cycle of work tasks was done four times during the semester, with four different selected articles and with different couples of reviewer and authors. During the semester students performed as "authors" and also as "reviewers" of their classmates. Through this activity, besides the cognitive competencies related to the curricular unit, students were also meant to reinforce/develop the following competencies: writing, synthesis, analysis, interpersonal communication, use of information and communication technologies and collaborative work.
Before beginning the activity (before the first chosen article), a 40 minute orientation on its objectives and on how to perform it was presented in class by the teacher and two pedagogical consultants from the institutional elearning team. The Google Drive environment was presented and questions clarified. Several online supporting documents were also provided and shared in Google Drive, similar to those provided for the civil engineering students [18], for example, the global orientation one including indicators of final evaluation of the teacher's grading. The evaluation had to take into account evidence of the following skills from the author-student: summarizing, identification of all the variables of the economic environment present in the news article, drawing conclusions in terms of opportunity and threats, and finally the elaboration of a well-grounded final individual opinion on the news article. The evaluation to the students as reviewers took in account general aspects such as the type of feedback and its quality. Orientation for feedback was presented orally in a general way in class and before the beginning of the activity.

IV. RESEARCH METHODOLOGY
Our research methodology followed three main steps: quantitative and qualitative analysis of student perceptions through the analysis of a questionnaire made available on line to all students; description and analysis of the type of PAPER COME TOGETHER: PEER REVIEW WITH ENERGY ENGINEERING STUDENTS feedback found in the final documents, analysis of grades given by students and teacher.
In order to assess the students' satisfaction and perception of the proposed activity an anonymous questionnaire was elaborated and made available on-line through the Google Drive environment (survey). Although this questionnaire was in part similar to the one used in our previous work [18], it was somewhat expanded/completed to include more qualitative questions and to give way to the perceptions of the students in their role as reviewers. Therefore 17 closed-ended and an open-ended questions (why they liked (or not) to be reviewers) were added]. Out of 24 students who completed the activity, 18 answered the questionnaire, a response rate of 75% of the total participants. After data collection, simple counting and percentages were computed for all the questions except for the open ended ones. For these latter ones, the nature of each response was analysed and conclusions made.
The second element used as bases of our analysis was a sample (54) of the final assignments. The type of feedback made by student-reviewers and by the teacher was categorized in each document, using the feedback model of Nelson and Schunn [24], the same that had already been used in our previous work with the civil engineering study case, reinforcing at the same type our knowledge of this type of approach.
Finally, a statistical comparison of final grades given by the students and the teacher was performed.

V. EVALUATION OF THE ACTIVITY
A. Characterization of the sample 56% of the 18 students of the sample were male. Only two students (11%) already knew about or had previously used Google Drive. Thus, for almost all the participants this was a new experience. Regarding the adopted approach and strategy, the results were positive: as studentsauthors, 78% performed all the proposed cycles' tasks, as students-reviewers the same percentage did the review tasks but only 38% gave grades.

B. Overall students' perception
As a general assessment of the activity, 94% of the students considered it important for their learning process and all of them considered that it contributed to improve their synthesis skills and to develop respect for their colleagues' opinions. Most of the students agreed that this activity developed their critical thinking skills (94%), sense of responsibility (89%), of collaboration (78%) and of technology use (83%). Most of the students involved in the activity (94%) considered it a good or a very good experience and 89% intent to continue using this type of activities with their colleagues. These results confirm findings of our previous research with civil engineering students [18].

C. Use of Google Drive environment
As for the engineering students in our previous work [18], regarding the use (and the ease of use) of the Google Drive, 94%of the students considered Google Drive to be a useful and efficient platform for the proposed tasks and 67% agreed that the activity management table shared in the Google Drive made communication between colleagues and the teacher easier. Only 23% referred to have used the Google Drive chat. 83% of the students assumed that the Google Drive was easy to use during the activity. 13 of the 18 students (72%) attended the initial session of explanation and clarification of the activity. As a result of a the presentation in the class of the Google Drive environment, 94% agreed that there was no need for further tutoring about Google Drive during the activity. Finally, 94% of the students stated that they were willing to continue to use this in future academic works.

D. The perspective of the student who received feedback (student-author)
An analysis of the closed ended questions was done to identify the factors/reasons of students' perceptions of peers' feedback and of its usefulness. 89% of the studentsauthors liked to be assessed by their peers and most of them were enthusiastic about the overall activity. The students-authors recognized the usefulness of classmate feedback (89%). The majority of them (89%) stated that they used the feedback of their peers to improve their final written document. In a general way, 94% of them considered effective the review done by their peers and they are also available to continue this kind of approach in other academic activities. Although 22% of the students-authors considered that their peers were as skilled as the teacher to assess them, 72% considered that they were able to do it but not as well as their teacher. In the students-authors perspective about the feedbacks of their peers, 22% said that reviewers included only a summarization comment on the written work, and 72% said that, in addition to it, reviewers included also identification of problems and mistakes, and suggested how to improve their work. Furthermore, in the students-authors perspective, 50% of the students stated they received a positive and motivational feedback and 50% a corrective one (confronting ideas, arguments and identifying problems related with the writing and with the content). The feedback was mainly considered clear, fair, positive, valid, detailed and worthwhile (72%). Finally, 89% of the students-authors considered the peers' feedback useful since it allowed them to deepen their first analysis and acquire more knowledge.
A complementary analysis of three qualitative questions of the questionnaire was performed to identify the reasons underlying students' appreciation and usefulness of the feedback based on their roles, either as authors or as reviewers. The analysis for the authors, was structured in two main categories: (1) Peer evaluation and (2) Own evaluation. The first category, 'Peer evaluation', was divided into two subcategories: (1.1) 'Appreciation' (Factors/reasons for positive or negative appreciation of the peer evaluation, as authors); and (1.2) 'Usefulness' (Factors/reasons for using (or not) the peer evaluation, as authors). By looking at the first subcategory which corresponds to the question "Did you like to be evaluated by your colleague?" 89% of the students stated a clear satisfaction regarding the received evaluation. The reasons supporting these positive perceptions were mainly related to their predisposition to perform improvements. For instance, the evaluation by the colleague is considered "an opportunity" or "a necessity" to see other point of view (student 3 and student 7), the process was also "a nice way to develop our critical skills" (student 4), "improve our work and mistakes" (student 5) or "allowed a better analysis about the same topic" (student 12). On the contrary, 11% of the students didn't liked to be evaluated by their PAPER COME TOGETHER: PEER REVIEW WITH ENERGY ENGINEERING STUDENTS colleagues because "many times I've disagreed" (student 18) or "[it] didn't improve my work" (student 11). Concerning the second subcategory, which corresponds to the question "Did you use the feedback given by your colleague?" 89% of the students agreed in regard to the usefulness of the received feedback. The reasons supporting these positive perceptions were equal to the previous subcategory -predisposition to perform improvements. Thereby, the feedback by the colleague-reviewer was used because "I always try to improve" (student 3) or "I always respect other point of views" (student 7), the feedback was also used in order to "Improve my future work with the suggestions and criticism" (student 4), "improve my analysis" (student 15) or "showed what's missing" (student 12). On the contrary, 11% of the students didn't use the feedback offered by their colleagues because "I've disagreed" (student 18) or "Didn't need" (student 11).
E. The perspective of the student as" feedback giver" (student-reviewer) From the perspective of the students as feedback receivers, 89% liked to assess their colleagues and 94% also accepted to continue this kind of approach in other of their academic activities. Only 11% of the students thought they had the same skills as their teacher to assess and 83% replied that they had fewer skills than their teacher (the remaining 6% did not answer). While 11% of the reviewers said to have done a single general comment on the written work (summarization), 89% included, in addition, the identification of problems and mistakes and suggested how to improve the work. 56% of the reviewers claimed to have given positive and motivational feedbacks and 44% claimed to have given a corrective one (confronting ideas, arguments and identifying problems related with the writing and with the content). The majority of the studentreviewers (78%) said that their colleagues used the feedback they gave. In their perspective the given feedback was clear and detailed (83%). They also considered it fair (89%), positive (since it encouraged their colleagues to seek for a solution and to respect the reviewer's view) and valid (83%), as well as trustworthy and reliable (72%). 67% of the student-reviewers considered their feedback useful since it allowed the student-author to deepen their first analysis and acquire more knowledge. Finally, as a global perception, 94% of the student-reviewers described the assessment done to their peers' work as effective or very effective.
Finally, we focused on students-reviewers' own appreciation of performed feedback. To the question "Did you like to evaluate your colleague?" 89% of the students answered positively while the remaining 11% did not like doing this task. The identified factors/reasons underlying positive or negative appreciation of their own feedback were self-confidence, predisposition to perform improvements. Thus, some answers state: it "requires responsibility, and I don't feel able and/or have the minimum skills" (student 1), "I don't like to evaluate other people" (student 17), "we can be misunderstood by the teacher" (student 18), or "even teaching we can also learn.", "We learn to improve our mistakes through a unique experience" (student 13), and also "it improves our critical skills" (student 8).
In summary we may say that students appreciated this activity, either as authors or as reviewers. However, two discrepancies may be noted. The first relates to the type of feedback. Although 89% of reviewers felt that they gave a complete feedback (not just a general comment), in fact only 72% of the authors perceived this feedback as complete, thus showing a different perception on the kind of feedback given and received. The second relates to the reviewing skills of the students compared with the teacher's ones. Whereas 89% of the students felt less confident and less skilled as reviewers of their colleagues, 72% of the student-authors did not question so much the skills of their peers as reviewers, although felt they were less able to perform a good feedback than the teacher, thus, revealing that students have less confidence as reviewers than as authors.

F. Comparing grades
In a first stage, we compared the final mean grades given by the teacher and by the student-reviewers (Figure 2). Results show no significant difference between these two (t test for independent means, p-value = 0.73). In a second stage we analysed the teacher's mean grades between each cycle in pairs (article 1 vs. article 2, and so on) to see if there was any significant improvement. Although Figure  2 shows a slight tendency of means grades' increase, when we test all of them between cycles in pairs, results show that there is no significant differences between any of them (t-test for independent means, p-values between 0.23 and 0.79). The lower value of the 3rd cycle's mean grade may be explained by several factors: fewer students who did the task and received a zero grade, students' involvement in the academic celebration week (no time to do the overall academic work). The low amplitude scale of the grades could explain the smaller grades variability, thus influencing the results. G. Analysis of the feedback given by students 54 documents with feedback both from peers and the teacher were analysed. In addition to the categories from the Nelson and Schunn model [26], three more aspects were considered in the analysis (Figure 3): the counterargument by the student-author to the reviewers' feedback, the implementation of that feedback by him/her, and the average grades per cycle of activity (both from peers and the teacher).
In general, it is possible to observe that peer reviewers used all the presented categories more than the teacher (Figure 3). On one hand, this could be justified by the large number of documents that the teacher had to review (all articles were different since they were chosen by each student) and on the other hand, the teacher's feedback was PAPER COME TOGETHER: PEER REVIEW WITH ENERGY ENGINEERING STUDENTS given after the peers' one (with no need to be so complete).
Only nineteen summarizations were done (35%) showing that more emphasis should be given to this issue. The least used aspect of the feedback was "Identification of global problems" and, consequently the "Given solutions to global problems". Despite this, students showed capacity to locate problems, and they did show equal competence in presenting local solutions for these problems. With these energy engineering students these results are positive compared with the ones of the civil engineering [18]. Students also used affective and positive language resulting in a decrease of any negative feeling from identifying weaknesses and limitations of the work. The counter-argumentation to the feedback given was almost nonexistent (1), although it was not a mandatory task and since students already had received their grades. Eight students used the feedback from peers to improve their work. And two used the teachers' feedback for the same purpose, although it was not asked. In general, this analysis shows that students have fairly good communication skills in the delivery of feedback, since they use all of the categories presented in the Nelson and Schunn's model [24], either from the affective or cognitive domain. However an effort of clarification and specification should be made in the identification and provision of global problems and their solutions as well as in the summarization.

VI. DISCUSSION, CONCLUSIONS AND FUTURE WORK
The objective of this work was to contribute to the body of knowledge on the effects of on line (Google Drive) peer review activities on written communication skills, through a case study by energy engineering students.
Through the analysis of the answers to a questionnaire made available to the students, of their own assignments and of their final grades (peers' and teachers' grades), it intended to characterize students' attitudes and perceptions towards on-line based feedback from their perspective as "authors" or as feedback givers and to discuss the relationship between the type of feedback and students' written communication skills, as well as the role played by grades.
Although some limitations in the applied methodology can be encountered, such as the high number of indefinite open answers (suggesting the use of a complementary data collection instrument like personal or focus group interviews) and the absence of connection between the answers of the questionnaires and the assignments (since the questionnaires were anonymous), globally this work has shown to be relevant in various aspects.
The use of Google Drive allowed the planned tasks to be performed in a satisfactory way, reinforcing conclusions of other studies [5] [6] [23] [22]. As we experienced with civil engineering students in our previous work [18], former acquaintance with digital instruments [22] allowed students to participate positively in the realization of the proposed peer-review activity. No significant difficulties were found, as it might have been the case [11] on this type of activity. However, good preparation and support guidelines were essential for its success.
The type of feedback and the influence of the teacher's status (versus peers) were found to be relevant factors underlying the students' appreciation and usefulness of received feedback. An interesting finding is that students appear to feel less self-confident as reviewers than as authors and that the perceptions of the feedback given and received are not coincident (as reviewers students think that they give a more complete feedback whereas as authors, students find the same feedback not so complete). This finding suggests that it is important to create an environment in which students' self-confidence is fostered in giving feedback and in coming together for interactions.
Although the analysis and characterization of the feedback showed that students have a satisfactory level of written communication skills, some areas must be improved: the identification of global problems and their solutions, summarizations. Following Shute [20] and also Knight and Steinbach [4], it seems necessary to spend more time in developing skills on how to give a good feedback, for example, by showing in class successful assignments and delivering one or more training sessions on "what are the characteristics of a good feedback", reinforcing the Nelson and Shunn's model [24]. Also, in order to improve the production of more founded opinions and counter-argumentations, some guidelines and tools on critical thinking would be helpful, using for example the Ennis FRISCO grid [34], among others.
Results make us think that despite the absence of intermediate grades (which we thought inhibited the use of feedback, especially when grades were considered high [18]) the grading scale (with very low amplitude) and the high averages of final grades assigned by the teacher to each document influenced the use (or non-use) of peers' feedback in sub-sequential works. This experience does not show that without the intermediate grades, studentsauthors use more the feedback given by their peers, as we thought from previous work [18]. Therefore some ques-PAPER COME TOGETHER: PEER REVIEW WITH ENERGY ENGINEERING STUDENTS tions remain for future research: do the final grades of each work influence the use of peer feedback by the student author in subsequent assignments? Do other factors influence students' use (or not) of peer's feedback? For example the non-anonymous type of feedback, as some literature already has drawn attention to [2]? Or the role played by the teacher status, since from the students' perspective, there is an overvaluation of the teacher's grading compared with the reviewers' one, as other studies have already demonstrated it [30] [18]?
Besides contributing to the body of knowledge on online peer review, the answers to these questions will also contribute to the improvement of our pedagogical practice in higher education in engineering degrees. Other steps would enrich our approach, namely moving to other dimensions such as the effect of online peer review on critical thinking skills development.