A Systematic Review of Serious Games for Collaborative Learning: Theoretical Framework, Game Mechanic and Efficiency Assessment

lusha.huang@connect.polyu.hk Abstract —In recent years, there has been extensive research on serious games for educational purpose. However, the design space for collaboration in games remains substantially unexplored. In this study, we systematically reviewed 31 empirical research articles regarding game-based collaborative learning published from 2006 to 2020 and attempted to provide new information about designing serious games for collaborative learning. We surveyed a number of games and investigated their design features that encourage collaborative learning. Twenty game mechanics were identified and grouped into six main domains: (1) Space, (2) Objects, attributes and states, (3) Actions, (4) Rules and goals, (5) Skill, (6) chance. The analysis of user studies they performed indicated that most of the game projects relied on self-report methods to test their learning effectiveness, and only a few studies adopted the data mining method based on game logs. The implications for research into facilitating collaborative learning and recommendations for future research directions are


Introduction
The video game has been recognized as one of the most engaging forms of entertainment nowadays and is making a significant contribution to the digital future. The application of games has expanded beyond entertainment and opens a new domain of learning technology, which is described as serious game, a type of game that enables the user to learn skills/information more effectively.
Recent years have shown an increase of interest in serious games for an educational purpose [1,2], and collaborative learning plays a huge role in the successful attainment of game-based learning [3]. McGonigal believed that games are exceptionally skilled at encouraging collaboration and argued that "online games are among the most collaborative people on earth" [4] (p. 268). The combination of serious game with collaborative learning may enable a new learning concept for current students who are described as the Digital Natives in a different learning style and learning culture [5]. Collaborative serious games have been recognized as a powerful tool for promoting high-level learning [6] and can help to promote students' positive learning attitudes [7], motivation, interests and higher-order cognitive skills [8].
Collaborative serious games have been highlighted and widely discussed by researchers. However, the design space for effective collaboration in games has so far been scanty. Most studies are narrowly focused on the learning effectiveness of digital games. The design of game features to promote collaboration with considerable research value has not been paid enough attention.
This study examines the most recent literature regarding game-based collaborative learning with a purpose to gain more insight into the current studies, especially the game design aspect. This paper surveyed a variety of games and investigated design features of them that encourage collaborative learning. The review aims to answer the following three main questions: 1. What are the theoretical frameworks and concepts of collaboration in game-based collaborative learning studies? 2. What game mechanics are embedded in order to promote collaboration? 3. What indicators are used to assess the effectiveness of collaboration in game playing?
The paper is structured into six parts. Section 1 provides a brief introduction and Section 2 describes the methodology this study used. Section 3 summarizes the theoretical foundations and concepts of collaborative learning employed by current studies. Section 4 investigates the game mechanics and design features of their products that encourage collaboration. Section 5 examines the game metrics they used to evaluate the collaboration established during game playing. Section 6 concludes this paper with a discussion.

Methodology
In this review, research articles published from 2006 to 2020 were searched and selected from the databases of SCOPUS and ProQuest. The searching of the literature was carried out in March 2020 following the principles developed by Creswell and Creswell [9]. Several keywords regarding collaborative learning were used as the search terms in combination with the keyword game by employing the "AND". The keywords for collaborative learning were collaborative learning, collaborative ability, collaborative skill, collaborative play, collaborative problem solving, team building, collaborative game, and the "OR" was applied to couple with all these keywords. The searching resulted in 419 articles for further examining and selection.
Then the articles resulted from above searching were further examined to ensure that they met the following criteria: 1) Design and implement at least one specific digital game 2) Focus on and aim to foster collaborative learning 3) Provide empirical assessment or descriptions of the effectiveness of the game.
Twenty-two papers were identified for review by adopting these three criteria, as demonstrated in Table 1.

The Theoretical Framework for Collaborative Learning
Designing a game that is both enjoyable to play and effective at learning requires the incorporation of both an educational dimension and a game aspect [10]. The educational dimension specifies the learning objectives of the activity, which determines how to integrate the game as a learning environment. The game dimension defines what game elements should be designed to achieve the desired experience in order to ensure the fulfillment of the learning objectives. Furthermore, the game dimension of the design process is subject to the educational dimension, and unlike games for entertainment, the elements of educational games are constrained by the learning objectives [11]. For the existing studies, the frameworks for collaborative learning they used were first identified across the introduction, research background and method, from the conceptualized theory to actual pedagogical content.
The theoretical frameworks provided in the articles varied in depth and detail, and this study began the analysis with the ones that were described as the model or theory level. Among 31 articles selected for the current review, twelve of them employed a model or theoretical framework of collaboration. Zea et al. [12] and Wendel et al. [13] applied five components of collaboration in game design, including face-to-face promotive interaction, group processing, individual accountability, social skills and positive interdependence. For example, positive interdependence means that group members rely on each other, and no one can succeed alone. For fostering the positive interdependence in game playing, Zea et al. argued that collaborative players should have only one goal and assume responsibility for the common success or failure [12], and Wendel et al. also emphasized that there should be some tasks that require a team of players to solve together [13]. Collazos et al. [14] defined five indicators of collaboration as monitoring, performance, success criteria review, intra-group cooperation and applying strategies, which allow measuring and analyzing collaborative activities in the digital game.
Susaeta et al. [15] presented the Classroom Multiplayer Presential Role Playing Games (CMPRPG), which translates the massive multiplayer online game concept to the classroom needs. It intends to incorporate the collaborative game in the classroom context and involves all students in a class participating in their learning within a collaborative environment. Echeverria et al. [11] applied Classroom Multiplayer Presential Games (CMPG) model from Susaeta to ensure students work in groups to accomplish shared objectives collaboratively. Theodorou and Kordaki [16] adapted the Jigsaw method of collaborative learning within the digital game situation, which entails players to team up to play different levels of the game and later return to their team to teach their colleagues. Culbertson et al. [17] drew from the framework of Group Task Circumplex and used the dimension of task independence to determine the degree of collaboration, which refers to the extent to which players must depend on each other to complete the task. In recent research, Gonzalez et al. presented a system of 13 patterns for designing and evaluating collaborative activities [18], including feedback, apprentice, activities, integration, facilitator, evaluation, coordination, positive interdependence, shared objects, process outcome, nature of the task, conflicts and making decisions, and the group of apprentices. The game of Circuit Runner chose five facets of collaboration skills [19，20] as outlined in the ACT's Holistic Framework [21], including maintaining a shared understanding, engagement/interaction, evaluate feature identification and strategy. Jurdi et al. [22] developed a framework for collaboration called CPSbot, revolving around four skills of communication, negotiation, organization and planning, which is based on the PISA 2015 definition of collaborative problem-solving. Cheng et al. [23] employed the Team Competition-based Ubiquitous Gaming model [24] in their game design to promoting players' collaboration.
For other articles that did not specify their theoretical frameworks of collaborative learning, this study examined the principles they used to design and analyze the collaboration embedded in the game playing. For instance, Hamalainen employed the players' engagement in discussion and negotiation as principles when designed a game for collaborative learning [25], including explaining, reasoning and elaborative questioning. Gasonpan and Temdee emphasized the positive independence and regarded collaborative learning as playing the game in a group and sharing the knowledge [26]. Warmelink et al. [27] focused on the concept of team cohesiveness and accompanying instrument developed by Seers [28]. Gillespie et al. [29] adapted the collaborative task developed by Holt and Yuill for autistic children to their game's collaborative component. Also, as for some articles that did not state the model or principles for collaborative learning, the approaches mentioned in their method sections for measuring the outcome of collaboration were investigated to identify the instructional indicators. Hamalainen et al. [25] and Bluemink et al. [31,32] employed seven categories of collaboration in the analysis of player collaborative behaviours, including social statement, instruction or order, content statement, suggestion, encouragement, question, and response. Burton and Martin [33] used five categories of collaboration to check for conversations of players, including inferencing, application, elementary clarification, in-depth clarification and judgment. Hummel et al. [34] used a questionnaire to measure players' satisfaction, which provided ten items related to their collaboration among game playing. In the study of Sung and Hwang [35], the collaboration experience of players was determined by the self-efficacy of group learning questionnaire. Sereti et al. [36] designed a game to improve players' seven types of teamwork skills as measured by a survey: ideas person, compromiser, evaluator, leader, summariser, recorder and encourager. Hou [37] examined players' behaviours patterns through game logs and identified three main indicators of collaboration: Talk (Discussions and Conversations), Group-work and Trading of items. Oksanen employed the Sociability Scale with ten statements in their research to investigate players' communication and interaction [6,38]. Wendel et al. [13] required the user to play the prisoner's dilemma game after the digital game playing and regarded the number they cooperate with others as the indicator of the effectiveness of collaborative game they designed.

Game Mechanics for Collaboration
Games are complex systems comprising of a diverse of pieces and parts, and a number of frameworks have been created to formulate the main elements in the design of games. The MDA model is a well-known framework that describes the game concerning three mutually-dependent layers: Mechanics (M), Dynamics (D) and Aesthetics (A) [39]. Fullerton et al. provided another theoretical framework that assists game analysts and designers to investigate all kinds of games from the Formal elements, Dramatic elements and Dynamic elements [40]. The Elemental Tetrad of Games is another model that has been widely acknowledged and accepted in both the academia researchers and practitioners in the games industry [41]. Shell classified all game elements into four domains: aesthetics, stories, mechanics and technology.
This study decided to use Shell's Elemental Tetrad as a framework to understand and analyzed the game elements that allowed games to be effective at encouraging collaboration. Schell's model has evident implications in the design and analysis of game elements, and its' substance resides on viewing games from the perceptive of the game design process and a development studio. Many existing models examined game elements from an extremely academic perspective, more concerned with philosophical analysis. The Elemental Tetrad arises as a suitable candidate also because it has been proven to be a powerful tool for educational games design [11,38].
Game mechanics are the most relevant component of the Elemental Tetrad and are regarded as "core of what a game truly is" [41]. Game mechanics are the key elements that enable games different from other kinds of media as they determine the interactivity through the rules and procedures. As pointed out by previous researchers, regarding the educational game's design, the game element that has a direct link with the learning objectives is the mechanics [10]. This study focuses on the game mechanics in games that enable active collaborative learning. The story of a game, technology, and aesthetics are not covered here because they are mostly independent of the desired learning outcomes.
As seen by Schell [41], mechanics are the procedures and rules in the game, describing in 1) Space 2) Objects, Attributes, and States 3) Actions 4) Rules and Goals 5) Skill 6) Chance. Table 2 summarized and described games: Mechanics for collaboration based on these six categories. For each category, this paper further explains specific characteristics that the game mechanics have and how they could help to facilitate the collaborative learning, which way players can meet the objective and what happens when they play. Schell further divided game mechanics into six subcategories: 1) Space 2) Objects, attributes, and states 3) Actions 4) Rules and Goals 5) Skills 6) Chance

Space
Space defines the game word where the play is taking place and will affect the collaboration process. A shared virtual space where all players co-exist together enables them being aware of other's presence and fosters possibilities to ask for and provide assistance [8,13,14,15,37,42]. Additionally, some collaborative games require their participants to play in physical isolation from each other [6,13,38], with a purpose to forces the group members to focus on the game playing and communicate through the game. The in-game helping system provided in the game world, such as non-player characters (NPCs), is essential for identifying obstacles met by participants and providing hints when necessary to facilitate the game playing [6,13,18].

Objects, attributes, and states
The game space has objects in it that can be noticed or controlled by players. Objects have one or more attributes (categories of information about an object), and each attribute has a current state. The tradable item or resource can be used to generate communication and collaboration among players [13,18,37], while players need to explain, justify and negotiate in order to gain what they want from others. The information division mechanic [38,42], in which each member has only a portion of the information or materials needed for the task and the members' information have to be combined in order to complete the task, promotes communication and requires the contribution of each member.

Actions
Actions refer to what players can do in the game and refer to how players coordinate and communicate in the game. The underlying action determines what operations players can take, and the strategic action describes how players use basic actions to achieve a goal. It is crucial to incorporate a set of communication tools to enable group members to share information and communicate with each other. The chatting channel [8,12,14,21,22,32] is considered to be one fundamental mechanic that enables group members to discuss, exchange information and make decisions together. It also allows team members to offer assistance or feedback to each other as well as provides the most and strongest supporting for planning and taking action on the tasks. The expression or gesture is a useful tool of non-verbal communication for enacting the communication and encouraging others since players can express promoting behaviours or praising team members quickly using emoticons [8,21,22].
The indirect action requires players to explain the quest to a partner since the problem is shown to one member who will not be allowed to solve it [6,14,27,31]. This game mechanic facilitates responding to requests from others, sending important information and establishing shared meaning. Competition with other groups [12,23,14,32] encourages all players to perform their duties and adjust strategies to enhance group achievement more effectively. The competitions organized are also an opportunity to check on each one's progress concerning themselves and others since they must do their best in order to be above other groups. In the game of MathMon, Players were teamed up to work together to solve ten maths puzzles and be the first group to complete the game in the shortest time [23].

Rules and goals
Rules and goals are the most fundamental mechanics for a game. Goals are the crucial thing that makes a game a game since the playing process is determined by the goal of the game that may be achieved. Rules describe the goals of the game and the consequences of actions. In nearly all collaborative games, players are required to form a team to discuss and accomplish the quest together [8,13,14,26,31,32,42,43]. The player team establishes rules of engagement for members to follow and offers feedback on their performances. As such, the leader of the team [12,13,18] is critical in some situation to make the decision and manage the work, who can assign members to a task, giving advises and encourage the group. Another point is that the role of a leader can be switched among different team members, which helps to prevent the game from being reduced to one player making decisions for the team. The group briefing [12,18], in which all members discuss their progress and how well they are working as a group, provides possibilities to repair the shared understanding of the problem, monitor the results and evaluate whether the common goal is being achieved. The group score that reveals what progress has been made and the contribution of each member [12,14,23,42] can be a motivation factor for promoting each player's performance in the team. The score teaches the player what is essential within the game after any action taken, the player can observe its effect on the score immediately. The game of MathMon provides a leaderboard on the screen to demonstrate the scores gained by each player when there is a progressing [23].
Many researchers believed that establishing a common goal is vital for motivating the individual and improving group awareness [8,12,14,44]. The common goal that requires agreement from all members establishes a clear objective that all group members will work towards. It requires players to assume responsibility and commitment for common success or failure, and helps to generate interaction, communication and shared understanding between the members of the group. For example, two players must collaborate to answer the quiz by indicating agreement at crucial points [29]. The partial goal emphasizes the responsibility and contribution of each member, while everyone should achieve a sub-goal before the whole team could go on to the next stage [12,18,44]. If a player is not able to pass his/her sub-goal, then the whole group will not reach the common goal. Therefore, each group member must contribute to the group efforts, and it prevents the situation of free riding in which one player does all the work, and the others miss the learning content.
Joint rewards can be provided to encourage group engagement and responding to other members' efforts, while all members share the reward after the group achieves its goal [12,13,14]. For example, in the game Escape from Wilson Island [13], solving a difficult challenge as a group is usually more rewarding than solving a simple problem alone. The group victory makes it impossible to complete tasks without the effort and contribution from other group members [8,13,14,42]. The victory depends on the outcome of the team, and winning can only be gained by working with others, not one player alone.
So far as the challenges and problems players need to solve, half of the game projects presented the simulation playing, such as designing a hotel room [25], programming [16,33], writing the business report [34]. About one third focused on the adventure or puzzle-solving, for example overcoming obstacles and escaping by finding the right password, collecting items and building tools [8,13,31,32].

Skills
Skills are the abilities that need of players' characters to complete the mission. The players' roles and unique abilities [13,15,27,44,45] require them to learn who is skilled at what in the team and promote everyone to contribute their knowledge to that of other group members. Each player is given a role with certain duties that the group needs to complete a common task. In the game Escape from Wilson Island [13], only players who have axes can fell trees to obtain wood for building the draft or the hut. In one task of the eScape [8], the player is required to wear a protective barrel to collect bees' nests, but his view is blocked by the barrel and has to be guided by other team members to solve the task together. In the game CMPRPG [15], the players can choose their own class as a hunter or shaman, while each class has a particular ability and needs the other class to collaborate appropriately. There are three roles in a team of the game AMELIO: Scientist (gathering information), Commander (illuminating the environment), and Engineer (using the device) [27]. For example, in a puzzle of repairing fuses to restore power, the Commander locates the sockets for placing fuses, the Scientist identified the different colours of fuses, and Engineer puts the fuses into the location. In the game of School Scene Investigators, players selected an independent game role, including photographer, pyro-technician, techie, and social networker, and each role was required to collect unique pieces of information and conduct unique tests [45].

Chance
The chance is the uncertainty and surprise involved in the games. It is a vital component of a fun game because the surprise is an essential source of personal pleasure. The mechanic of chosen challenger [12] in which the group must choose who solves the problem encourages planning, debating and responsibility performing. The surprise challenge assigned to players with fewer scores could balance all members' performance and motivate the participates [12].

In-Game Indicators for Collaborative Learning
To better understand and enhance collaborative process, it is imperative not only to investigate the sum of game mechanics for collaborative space but to evaluate the collaboration as well.
There are tremendous efforts in evaluating the collaboration established during game playing. The analysis indicated that most of the studies relied on self-report methods to test the learning effectiveness [6,14,19,22,27,33,34,35,38,42]. Hummel et al. [34] used the student satisfaction questionnaire in their study, which has ten items related to collaboration. Sung and Hwang [35] investigated the collaboration and communication of the group using the self-efficacy of group learning survey. Oksanen examined the group interaction through the sociability scale with ten statements [6,38]. Culbertson et al. [17] employed the inclusion of the other in the self-scale to examine interpersonal closeness and developed two other scales to measure the perceived helpfulness and ignorant partner perception.
In recent years, game logs and data mining are used to measure the collaborative game playing. Some studies have been conducted based on chatting logs of player because discussion plays a crucial role in the interaction and collaboration within the game. Hamalainen et al. [6] and Bluemink [31,32] explained the collaboration process through the chatting logs recorded in the game of eScape. They examined players' discussion before solving each task and categorized them into Content statements, Questions, Instructions or Orders, Responses, Suggestions, Social statements, and Encouragements. Hamalainen [25] collected the chatting logs of players to analyze how they engage in discussion and negotiation, including elaborative explaining, reasoning, and questioning. Burton and Martin [33] identified different forms of col-laboration by analyzing chatting logs on a detailed level. Wendel et al. [42] examined discussions of players by the messages they sent in the game. Culbertson et al. [17] numbered the lines players entered into the chat window as the indicator of their collaborative engagement. Polyak et al. extracted the raw data of conversation logs within the game and transformed these data to represent different levels of collaboration skill evidence [20].
The behaviours logs were another game metric the studies employed to assess the collaboration. Hou [37] performed a frequency analysis of behaviours logs to classify the collaboration as different types, including ten behaviours categories such as fight, talk, trading of items or conducting tasks. Wendel et al. [42] also recorded behaviours logs as an indicator of group achievements. Gonzalez et al. [18] developed a tool to register the goals, scores, the time spent in the area, and start and finish game time. The videotaping of player behaviours was also used to observe the collaboration patterns among game playing [8,15,19,23,29,31,32]. For instance, Gillespie et al. [29] videotaped participants when they were playing the game and code their interactions. Cheng et al. [23] investigated the face-to-face verbal interaction that occurred in the game playing and transcribed the data into the sentences or acts as an indicator of collaborative learning. Bressler et al. [45] recorded the audio of each player during the gameplay and qualitatively coded the transcripts into three kinds of communication responses (discuss/accept/reject) as collaborative discourse occurred in the game. The outcome of the play was viewed as another indicator of collaborative learning. For example, Hamalainen [25] analyzed the score on the final report for each player group. Boufera et al. used the game score and completion time to determine the effectiveness of their game in improving the learning outcomes [46].
Additionally, Wendel et al. [42] attempted to assess the effectiveness of the collaborative digital game by playing the prisoner's dilemma game and investigating how many times players selected to share with others. Theodorou and Kordaki [16] carried out their study rooted in the Jigsaw method of collaborative learning and asked players to work together to integrate all levels into a single one as the indicator of collaborative learning.

Discussion
This study aims to review studies of game-based collaborative learning in terms of three aspects:

1) Theoretical frameworks 2) Game mechanics 3) Game metrics
This paper examined the frameworks previous studies used to define collaborative learning, identified the game mechanics that foster collaboration, and discussed the methods for assessing learning outcomes.
The analysis shows there is still a lack of systematic framework on the concept and components of collaborative learning. Some researchers used the five components of collaboration to guide their game design, which was employed to explain how to enable all members actively involved in the collaboration. The positive independence and communication between players were significant elements for collaboration emphasized by researchers. However, for most existing studies, the theoretical framework was still vague, and no systematical concept or model for collaborative learning was provided. How to adapt theoretical foundations of collaborative learning to practical game design is still worthy of attention.
This paper examined collaborative games and identified 20 game mechanics supporting collaborative playing, such as setting common goals, sharing the rewards or penalties, and bestowing different responsibilities upon the players. The findings indicate that the player team and chatting channel are the most common game mechanics for facilitating collaborative activities. These game mechanics identified here can serve as an inspiration for researchers or developers working on collaborative games, which hopefully enable the development of better collaborative serious games. However, there is a strong need of guidance to inform the game design for collaborative learning. On top of that, the comparisons among different game mechanisms were less investigated. A considerable amount of research has emphasized the effectiveness of game learning. Digital games as a new medium can indeed support productive learning, but it is the design within the game that will determine the efficacy of learning. A demand for designing and studying collaborative games is articulated, and it urges researchers to explore game design space for fruitful collaborative activities.
The analysis of user studies they conducted demonstrated that most of the game projects relied on self-report methods to test their learning effectiveness, and only a few studies used the data mining method based on game logs. The survey provides a quick and easy way to collect empirical data and has been a widely acknowledged and accepted tool in many fields. However, when applying the survey method to the context of the digital game study, it should be noted that the self-report method itself may be an unreliable source of data that leaves room for interpretation bias. Other methodology issues included such as small sample size and the appropriateness of the instrument to assess learning outcomes. The findings perhaps indicate the need to develop appropriate indicators of game metric to assess learning outcomes [47]. Compared with the traditional self-report method, the game metric technique based on server data enables us to access a high precise recording of players' behaviours. Even though several studies in this review have examined in-game behaviours through chatting or behavioural logs, they usually presented their results in a more descriptive way, such as the number of texts inputted, or quests completed. Deeply analyzing behavioural data will be needed to investigate the relationship between in-game performance and learning outcomes.
Previous research has demonstrated the potential of using digital games to foster collaborative problem-solving skills and offer an effective learning environment. However, the area of game design for supporting collaborative play is still not well understood. This study presents a set of game mechanics for collaborative learning developed based on the analysis of nineteen collaborative games. These game mechanics provide a comprehensive framework for collaborative game design and analysis. The findings and conclusions emerging from this paper provide new insight on how to support collaborative play in games. One limitation of this article is the limited number of reviewed articles. Therefore, the inclusion of more papers in future reviews might assist in generalizing results concluded here.