Student-Collaboration in Online Computer Science Courses – An Explorative Case Study

As a result of the current Covid-19 pandemic, a shift in teaching and learning from face-to-face to video-based online instructional settings has occurred in higher education. For online teaching, strategies are required to allow collaboration and interaction between learners, similar to face-to-face teaching. Therefore, it is essential to identify how students perceive group work in online classes in order to be able to draw relevant conclusions for the design of online classes. This paper examines the learning climate and the perceived key features of collaborative learning in two different computer science courses with (N = 9) pre-service teachers in computer science and (N = 7) computer science majors, which took place in the virtual gathering space gather.town. We choose this platform to provide a wide opportunity for interaction, especially during collaborative group work. Even though we can draw no conclusions that the possibility of interaction in gather.town platform had an impact on the learning climate, results indicate that the learning atmosphere was perceived as similar to that of in faceto-face courses. Furthermore, the results based on the qualitative content analysis indicate that students perceived a high activation as working better in online collaboration than in face-to-face settings. Keywords—online teaching, student collaboration, group work


Introduction
Recently, there has been a significant shift from face-to-face teaching to online-only teaching at all levels of education as a result of the Covid-19 pandemic. Although concepts for online teaching have been the subject of research and practice for some time [1], there has been increased attention to this topic, particularly, more recently [2]. In this context, video conferencing systems, such as Zoom, WebEx and others, have been used to deliver online teaching. Although these tools are suitable for lectures and meetings, it can be assumed that they have some disadvantages during collaborative phases of teaching and learning. In particular, break-out rooms in standard videoconferencing systems have shortcomings, such as not interacting with participants in other groups [3]. This paper describes how we used an online learning environment in two higher education courses of computer science to design group work which enabled a setting of collaborative learning similar to face-to-face teachings. To this end, we choose a web-based tool that differs from standard conferencing systems and integrates an interactive virtual meeting space, namely the online platform gather.town 1 . The aims of this case study are: (1) assess the community atmosphere in an online-based learning platform that provides an environment of high interactivity between the learners and (2) identify key features of computer-supported collaborative learning that learners perceive in this particular setting. In the following sections of this paper, we first describe the theoretical background of computer-supported collaborative learning environments and introduce the research questions. Then we outline the specific course settings and the research methods we used, followed by the research findings. The paper concludes with a discussion and conclusions on integrating a virtual meeting room as a learning environment.

Background
The importance of fostering team skills through group work has already been recognized in computer science classes, where it has been found that a student-centered atmosphere has an impact on the development of team skills [4]. Small group collaboration is seen as an important factor in building social presence in online-learning [5]. According to [6], the learning climate also impacts self-efficacy, interests and active participation in a learner's group. This is also related to the student-centered approach and the classroom climate, affecting students' attitudes and learning outcomes [7], [8], [9]. Moreover, social skills, active learning and collaborative skills are essential along with cognitive and interpersonal aspects, which are relevant for the effectiveness of learning processes [10]- [13]. Online instruction also opens up new obstacles in communication, such as cameras being turned off or passivity reflected when teachers intended to include more interactive and collaborative online instructional sequences [14]. Hence, the importance of considering social interactions in online teaching is evident and it is shown that social presence, active social participation, and group phases are key features for perceiving learning environments as beneficial [5], [15], [16]. Friedrich & Mandl provide in [17] an overview of the learning strategies and how they influence learning. They highlight strategies of cooperate learning as one learning strategy. To identify how collaboration and group work is perceived, determining the degree of community provides insight into students' perceptions [18]. Computer-supported collaborative learning aims to bring learners into a situation where they solve problems together, exchange knowledge and opinions, and analyze their learning progress together with the help of digital tools [19]. However, according to [20] collaborative learning should take place particularly in complex learning situations and tasks in order to avoid cognitive overload [21]. Learning environments that include collaborative learning phases where a small learning group can achieve a learning goal, have shown to have a substantial impact on students' progress [18], [19]. Computer-supported collaborative education is designed to put students in situations where they use digital tools to solve problems together, share knowledge and opinions, and analyze their progress together [8]. The aim is to prompt learning through computer-supported collaborative instruction so that the group of students acquires more knowledge than they would have alone [20].

Research questions
Based on the theoretical assumptions on computer-supported collaborative learning environments and the impact on learning, the following research questions are driving this study: RQ1: How is the community atmosphere perceived in collaborative learning settings?
RQ2: What are key features that are perceived relating to collaborative group work on gather.town?
The aim was to explore students' experiences with the tool by assessing their perception of implemented collaborative learning through a post-questionnaire.

Method
In this section we describe the empirical examination of two virtual computer-supported collaborative learning environments, which took place in undergraduate and graduate computer science programs at two different universities. We chose a [22] mixed-methods design, because this allowed a more detailed perspective than would have been possible with an exclusively quantitative or qualitative approach [22], [23]. The design of this case study (see Table 1) was a between-subject design. Different groups of learners participated and evaluated the courses they attended. One university has an educational focus (Course 1) and the other has an engineering focus (Course 2). Even though both courses addressed different topics, both courses used the gather.town platform. The first course aimed to prepare and specialize preservice teachers to take a pedagogical perspective when appropriately choosing teaching methods in computer science. The other course aimed to enable computer science majors to take the application-oriented view of computer science and software development. Table 2 shows the number of students data were collected from: The following two subsections describe both courses in more detail.

Course 1: Didactics of computer science
The seminar "Didactic of Computer Science" is a compulsory subject within the undergraduate part of the teacher training program. The course aims to enable pre-service teachers to consider the basics of learning and instruction in their lesson planning and to bring their pedagogical knowledge to a higher level. This will allow them to be well prepared to teaching computer science at the lower secondary level. Each course session is characterized by a work phase that lasts 5-25 minutes, depending on the task. The pre-service-teachers are asked to individually choose at which table and with whom they want to collaborate. This was done to provide a setting where students can decide self-determined and thus motivated choose which topics they want to treat in more detail [24]. To provide an example, during small group work, students worked together to develop ideas for methodological applications for a given subject content from computer science. They also discussed the content of lectures, worked out questions or consulted on lesson plans. The small group phases were always initiated with a clear work assignment or framework. The course thus provides a framework for students where interactive or reflective exchange is encouraged. Gather.town offers the simple possibility to efficiently conduct spontaneously scheduled small group phases during the course, even without a technical intervention of the teacher.

Course 2: Software product line engineering
The lecture "Software Product Line Engineering" is an optional subject within the master curriculum of computer science. The course aims to enable students to develop and maintain software product lines by providing them an overview of different goals, methods, concepts, and techniques [25]. Although the course is classified as a lecture, it fosters interaction by including small surveys for individual students and breakout discussions for groups. For breakout discussions, students form groups of up to six people at the tables and are tasked to apply a specific method presented in the lecture. During the breakout discussions, the lecturer visited the tables to answer the students' questions. After the breakout discussion, the students successively gathered around the group tables, where one of the group's members presented their solution (on the virtual whiteboard) to the other students. Usually, students highlight essential questions and design decisions during that phase. The breakout discussions' tasks employ an ongoing example; in that each task focused on a different phase in the software product line engineering process. This permitted students to retain previous results on the whiteboard and continue their discussion in stable groups throughout the course.

4.3
Online gathering tool gather.town The virtual platform gather.town was employed as the course platform. Since it is possible to customize the platform's map, we designed it like a seminar room. Figure 1 shows how this platform was tailored to our needs. We chose to set up a plenary seating arrangement in the center of the room and surrounded it by five group worktables with virtual whiteboards at each table. The system works so that students and the lecturer as avatars can move freely on the map using keyboard controls. If two avatars come into close proximity, the video function is activated, and they can talk via the integrated video conferencing system Jitsi 2 [26]. If a person moves to the lecture table in front of the room, they can be heard and seen by everyone in the room. Each virtual whiteboard can be used collaboratively by everyone at the corresponding table.

Methodological procedure
We chose a quantitative and qualitative approach to analyzing the empirical data. To examine the learning atmosphere and the perceived collaborative group work, we selected the following assessment scales, which are based on instruments that have proven to deliver valid and reliable results.
─ Community questionnaire: The community questionnaire based on [27], [28] included group work characteristics as measuring aspects of community building. Examine the degree of community is essential for assessing perceptions of group work as it is an indicator for promoting effective collaborative learning [18]. ─ Perception of group work: To identify students' group work perceptions, additional items from [29], [30], were included to determine students' experiences in group work on a community and technical level.
─ Furthermore, we asked students, if they have learned less/equal or more during online group-work compared to on-site courses. ("Compared to on-site courses, I learned less/equal/more during group work in this online course.") Furthermore, we assessed the student's perception of group work in their collaborative learning phases through the following open-ended questions.
─ Open text responses: To allow a more detailed perspective on students' experiences of group work and learning in groups, we asked them the following questions: • How did you perceive group work in this course online compared to face-to-face courses? Briefly describe it. • Which aspects did you perceive to be efficient in group work in this online course?
Briefly describe it. • What problems did you experience with group work in this online course? Briefly describe it.

Reliability of the questionnaire
To examine whether the items worked, we examined the reliability of the items and the scales. Analyses of the internal consistency are presented in Table 3 and revealed a Cronbach's alpha for the scale learning environment of 0.91 which is excellent. For the scale perception of group-work a Cronbach's alpha of 0.70 was examined, which is sufficient. All items in this scale (perception of group-work) were not normally distributed. Those items which were normally distributed in the scale of learning environment are listed in the Table 3. All other items were not normally distributed. The item that assessed learning was also not normally distributed. Experiencing connectedness and community n.d.
The group is perceived as a whole rather than the sum of its parts n.d.

Analysis
We used descriptive analysis for the assessment scales and identified the open-ended questions by means of the qualitative content analysis [31], [32]. The assessed key features of technology-enhanced learning in collaborative settings were analyzed applying a theory-based approach that was oriented on aspects of technology and pedagogy. This enabled in a further step to classify the two seminar groups into key features that worked well or less well from the educational and technical points of view. Two trained raters rated them using a coding guide (see Table 4) and sorted them into the categories of constructive and obstructive key features and into the subcategories of technical, educational or other issues. We additionally assessed the categories according to their degree of expression (-1= worse than online teaching, 0 = same as face-to-face teaching, 3= online worked better than face-to-face). This was done in order to gain a deeper understanding of the features of technology-enhanced learning. The aim was to identify initial trends to adapt a study design for further investigations. [31], [32]. "Connection problems with the learning environment." "It is difficult to interact with those group mates whose microphones did not work properly." Whenever there is a new semantic meaning, the sentence will be split into two or more categories. In particular when technical, educational or other issues are mentioned in the same statement.

C2.2: Challenging
Educational issues concerning Technology-enhanced Learning The student describes a situation, where educational issues in respect to learning worked less well.
"If the other group mates are not motivated to communicate, it is difficult to deal with the topics in more detail."

C2.3: Other challenging issues concerning Technology-enhanced Learning
The student describes a situation, where other aspects worked less well.
"I had another course at the same time." The analysis of categories follows the qualitative analysis approach [33], where first the students' text answers are categorized, and then in a next step each category is described or interpreted in the overall context. Finally, the responses were analyzed using an inductive approach. This was done in order to develop categories and sub-categories that describe the key features that were perceived during collaborative learning.

Results
This section presents the main findings of this case study. It investigates how group work was perceived in gather.town and what difficulties were identified. The objectives were: (1) investigate the community atmosphere in an online-based learning platform (2) identify key features of computer-supported collaborative learning. We analyzed the mean scores and standard derivatives of each course.

Quantitative
The questionnaire was completed in full by a total of 9 out of 9 students in Course 1 and by a total of 7 out of 25 students in Course 2. This section describes the descriptive overview of the mean scores of all items regarding the feeling of community (see "Community Questionnaire" in Figure 2) and the perception of group work (see Figure  3). It reveals that both courses perceived the community feeling similar to that of faceto face teaching. (0 indicates that the feeling of community was perceived similar in the online course compared to face-to-face teaching). The feeling of community while using the online-based learning platform was perceived to be slightly higher in Course 2 in comparison to face-to-face teaching. Whereas in Course 1 the perception of community was not equal to that of presence-teaching. In Course 2, the participants perceived attentive listening, the ability to achieve common goals and having direct concern for the group process more likely in the gather.town-environment than in face-to-face group work. Both courses perceived the item "tolerance for ambiguity and conflict" similar: Course 1 M=0.44, SD=1.33 and Course 2 M=0.71, SD=0.95. Both courses perceived the items "having direct concern for the group process" differently: in Course 1 with M=-0.89, SD=0.93 and in Course 2 with M=1.57, SD=1.72. The multivariate analysis of variance, MANOVA, was used to examine differences between the groups systematically. As the data were not normally distributed, we have chosen a Welch-ANOVA in order to examine differences because this is robust when the variances are not homogenous. The following items revealed a significant difference between the groups: "communicating owned feelings and meaning", "having the ability/power to achieve common goals", and "having direct concern for the group process".

Fig. 2.
Change in aspects of community in both courses, questionnaire designed by [28] and integrated in the context of computer science by [27]. The bars depict the mean value, scale from -4 (declined) to + 4 (improved), n=9 (Course 1) and n=7 (Course 2) Figure 3 illustrates the second part of the questionnaire, which includes the perceptions of group work. The results indicate that both courses rated the items similarly. Figure 3 shows that the group work was not perceived particularly monotonous in As the data were not normally distributed, we have chosen again a Welch-ANOVA in order to examine differences. No items reveled a significant different between the groups. Fig. 3. Perception of group work in online courses, questionnaire based on [29], [30]. The bars depict the mean value, scale from 1 (not at all) to 5 (fully agree), n=9 (Course 1) and n=7 (Course 2) Results from the question whether students learned less (1), the same (2), or more (3) in online group work show that students from Course 1 scored M=1.89, SD=0.60 and in Course 2 scored M=2.57, SD=0.53. The average of both groups is M=2. 23. As the data were ordinally scaled, we have chosen a Mann-Whitney-U-Test in order to examine differences. There were again no significant differences between the groups. Table 5 shows the number of key features found within the groups concerning constructive and obstructive educational and technical issues. Most key features were found in educational issues that were constructive. Overall, more issues were found that were constructive than obstructive. In aspects that were obstructive, more issues were found concerning technical issues than to educational issues. The Krippendorff revealed r = 0.79, which is good.  Table 6 illustrates the results of the inductive qualitative content analysis. The subcategory has a Krippendorff of r = 0.88 and of the degree of expression a Krippendorff of r = 0.62. The examination of the category revealed a Krippendorff of r = 0.77, which is sufficient. In addition, a third trained rater rated the coding scheme that were developed by the two raters in order to assure that this coding scheme can be transferred. There was intercoder reliability in accordance with Krippendorff of r = 0.88 for the main categories. The subcategories revealed a Krippendorff of r = 70 between the first two raters and the third rater and a Krippendorff of r = 59 in regard to the degree of expression. Table 6 and 7 show the examined categories and sub-categories as well as the degree of expression with regard to key features that succeeded as well as were obstructive from the technical and from the educational point of view. Table 6 and 7 reveal the main categories and sub-categories that were found in the students' responses. It also reveals the degree of expression (-indicating that face-toface teaching works better than online-teaching; 0 indicating that it worked neither better nor worse in both settings; + meaning that these issues worked better in the onlineteaching than in face-to-face teaching). Both courses addressed technological as well as collaborative learning issues that succeeded with regard to technology. Whereas the first course mentioned stability and synchronous work as being key features that worked from the technological perspective, the second course highlighted aspects of active participation in collaborative learning and stability and usability as key aspects which succeeded from the technological point of view.

Qualitative
With reference to the main category of collaborative learning, both group of students perceived aspects of synchronous work and of active participation as being better in online-learning compared to face-to-face teaching. With regard to learning climate those issues that were constructive were perceived as neither being better nor worse compared to face-to-face teaching. With regard to the main category of group work, in both groups there were found issues like group building process, group organization and time management as factettes of student collaboration which worked rather better in the online settings compared to face-to-face teaching.
Both courses addressed communicational issues as obstructive. Whereas the first course mentioned asynchronous communication as hindering, the second course referred to the willingness to communicate as an obstructive issue. The stability and usability were addressed in both courses as technological key features that were obstructive. Group work factettes were found in both lectures as obstructive. The first course highlighted aspects of group organization and group presence as challenging whereas the second course mentioned time management as obstructive.

Comparing the results
The following examples compare the open-ended questions with items from the assessment scales. Concerning group work, the previous section showed that the group building process and group organization were key features that worked better in the online setting than in face-to-face classes.
The following examples illustrate this: In terms of group processes, in Course 1, the commitment to group work was described by student S1 as: S1: "It is easier to disengage as a group member than in on-site group work." or also: S2: "I find it difficult when the groups are too large. It is then easy to lose the overview and it can quickly become chaotic.", both were categorized as "Group organization (Group work)".
which also influenced communication in Course 1: S3: "I find communication to be difficult", S4: "One disadvantage I found was that you can't talk at the same time, you always have to wait until someone has spoken." both of which were rated as working better in face-to-face teaching "Asynchronous Communication (Communication)" category. While overall, students from Course 1 rated an average of -0.24 on the Community Questionnaire, the two students (S3/S4) rated it -0.46.
By contrast, in Course 2, students commented on the group work: S5: "Some don't participate, but still more than on-site.", which was defined as active participation (in the Group work category), or S6: "It is easier to organize in groups because you don't have to walk through the lecture hall.", which was categorized as group building process and rated to work better in online-teaching. While overall the students of Course 2 rated the perception of group work an average of 3.61, the two students (S5/S6) rated it higher at 4.14. There were noticeable differences in the item "The technical stability during the group work in the breakout sessions was very good." (Course 2: 3.86 and S5/S6: 5.00).
Although the quantitative data indicate a high level of technical stability, some problems became apparent through the students' open feedback. In Course 1, one student reported that: S7: "The white board in the middle of the group worktables could be improved." and "Occasional connection problems of the end devices and the learning environment." or in Course 2: S8: "The tool is technically quite unstable and not necessarily intuitive to use." The item of technical stability during the group work, was rated on average for both groups with M=3.50 (indicating something between disagreeing and agreeing).
However, S7 and S8 rated the item of not being confident in using technology with M=1.50 (totally disagreeing to disagreeing). The overall averages of both groups were M=2.10 for this item, thus showing a rather high self-confidence regarding technical issues.
Furthermore, the positive functionality of the platform's whiteboard was also highlighted, as in Course 1 (this student completely disagreed with regard to the statement of: I was not confident in my use of technology") and hence shows a subjective high technical experience: S3: "The possibility to work simultaneously/parallel and together on a whiteboard." In Course 1, it was also reported that the easy ability to move around the room and quickly find each other in groups are key benefits: S9: "Alternating phases: i.e., being able to discuss an aspect for just a few minutes in small groups. In other conferencing systems this always takes forever. In gather.town you can simply move freely on your own." From quantitative data, this student (S9) identified a subjective technical expertise ("I was not confident in my use of technology.") with M = 1.0 (average of both groups was M=2.10)

Discussion
Due to the Covid-19 pandemic, university teaching has been transitioned to using video in teaching in all disciplines. Due to the need to quickly respond to the urgency of using video-based systems in teaching, systems such as WebEx, Microsoft Teams, or Zoom were adopted in early 2020. Even though these systems allow a great interactivity between the participants, online-platforms enable a self-determined choice of freely moving as an avatar on a virtual map and easily communicating with other course participants. In this paper, we chose gather.town in order enable a high interactive way of cooperating with other learners. The aims of this study were: (1) assess the community atmosphere in an online-based learning platform that provides an environment of high interactivity between the learners, (2) identify key features of computer-supported collaborative learning that learners perceive in this particular setting, which lead to two research questions: RQ1: How is the community atmosphere perceived in collaborative learning settings?
RQ2: What are key features that are perceived relating to collaborative group work on gather.town?
The community questionnaire showed that compared to studies in higher education, the scores were relatively low. In related work where team-work was promoted in onsite courses in computer science, the community questionnaire showed that results are usually > 2.0. -Even though we did not assess the learning climate explicitly, results indicate, that the community atmosphere is perceived as similar as in face-to-face courses. The questionnaire in this explorative case study showed that these values of both courses investigated are below 2.0 in all cases and in most cases even below 1.0. Furthermore, it was found that students learned at least just as well in gather.town as they would have been in face-to-face settings. This indicates that the students perceived the degree of learning similar to face-to-face student collaboration. This implies, that gather.town provided a platform to collaboratively learn as well as face-to-face collaboration. So far, we have not examined student collaboration in realized face-to-facesettings.
The analysis of the open-ended questions showed more key features that were constructive -both with regard to educational and technical issues -than that were obstructive. In addition, there were found more educational key features that were constructive in contrast to technical issues in both courses. It may be assumed that the educational aspects concerned with learning issues were perceived more dominantly in this online platform due to the focus on fostering student collaboration. In other words, the focus was more so on collaborative learning than on using the technical options properly. In a course on using digital tools the focus on student's perception maybe would have been on technical issues rather than on educational issues.
Moreover, categorizing the open-ended questions through the qualitative content analysis revealed that group work or collaborative learning were identified as a dominant issue of student collaboration in both groups. This underlines the findings from above that student's focus was more on student collaboration than on technical or communicative issues. This may imply that when designing collaborative group phases, technical and communicational issues do not need to be emphasized as much as, for instance, activating participants or processes of group building.
The results of the open-ended questions revealed that synchronous work and active participation were perceived by students as being better in gather.town compared to face-to-face teaching. This may give a first hint of the impact of interactive collaborative learning on students' perception.
From the method mixing qualitative with quantitative data, it can be assumed, that higher subjective technical experience leads to higher satisfaction with course system but also shows more critical views on usability. Different perceptions of group processes from qualitative data could also be an indication that the item "Having direct concern for the group process" results in the divergences described above. In particular, group size is likely to be an important aspect of online group work. From experience and literature, optimal group size is from 2-5 students for each group [34].
In summary, this is in line with the above-mentioned high values for the item "The transitions from teacher lecture to group work were smooth".

Limitations
This study reports our experiences using the online tool gather.town for online teaching in the field of computer science. This case study is limited to various points that must be addressed in future work. First, it included only a low number of learners and a limited choice of disciplines. Therefore, we can make no systematic conclusions. Consequently, future research must include more subjects and examine a broader range of topics. Moreover, a detailed insight into the concrete student collaborative setting is needed in order to better understand learning sequences better. As a consequence, in future research video data should also be implemented as well in order to gain more insight.
Even though the data collected with questionnaires and free text answers provide a detailed picture in terms of mixed methods, the possibility of generalization is still limited. Also, the small number of learners and the unbalanced response rate of both courses causes a certain bias, which we have pointed out in the area of feeling for community. Future studies need to focus on different settings of learning to analyze which setting has the most impact on collaborative learning.

Conclusion
This paper described a case study in which student's perception of learning climate and group work in realized computer-supported collaborative learning settings were analyzed. We examined the learning climate in two courses in computer science as well as perceived key features that were regarded to have an impact on learning. We aimed to provide insight into the both learning climate in online-courses and perceived key features of student collaboration in computer science.
To examine the learning climate student collaboration, we referred to well-established items. However, it must be clearly stated that we can draw no conclusions that the possibility of interaction in the gather.town platform had an impact on the learning atmosphere, as we have not assessed the course atmosphere in courses which provide a different degree of interaction possibilities. In addition, we have not assessed the learner's atmosphere outside the course or even before it started.
In the future, we will further investigate the differences between online teaching and face-to-face teaching, different settings of collaborative learning, and how the lack of interpersonal perceptions can be compensated.

Authors
Bernhard Standl is an assistant professor for computer science education at the Karlsruhe University of Education, Germany. His main research interests are in the modeling of teaching concepts as design patterns for computer science education.
Thomas Kühn is a postdoc at the Karlsruhe Institute of Technology. His current interests revolve around inventing new ways to model and program future software