Support of the critical thinking process in synchronous online collaborative discussion through model-eliciting activities

: As online education becomes common, it is essential to identify activities that support critical thinking in this environment. This paper describes the use of a model-eliciting activity (MEA) solved by groups in a synchronous online discussion. The participants displayed their thinking processes and solutions consistent with successful face-to-face (F2F) MEAs. Each group’s discussion showed evidence of critical thinking and incorporated quality logical arguments. This demonstrates that MEAs may be also used in an online environment to successfully support application of critical thinking skills.


Introduction
Developing critical thinkers is an essential goal of education. An important aspect of learning critical thinking is having opportunities to apply critical thinking skills, receiving feedback, and reflecting on their application [1]. Providing such opportunities represents a significant challenge in online discussions [2]. Although numerous activities support aspects of critical thinking, only certain types of online discussion activities support learner progression through the full critical thinking process and support quality argument construction [3]. Identifying such activities is important for facilitating online learner development. Based on face-to-face (F2F) successes and previous studies of online discussion, the hypothesis was that modeleliciting activities (MEAs), a variety of small group problem solving, would also support critical thinking in the online environment [4]. Participants formed two groups of adult learners attending a professional development workshop [4,5]. This paper assesses the efficacy of MEAs to foster critical thinking in an online synchronous discussion, and describes both, the progression through the activity itself using the model-eliciting cycle framework and the critical thinking demonstrated using the critical thinking assessment framework (TAF). Post-MEA participant discussion is also considered to help provide meaning for results.

Theoretical Framework: Critical Thinking and MEAs
Dewey [6] described critical (or reflective) thinking as a logical process consisting of five steps: "(i) a felt difficulty; (ii) its location and definition; (iii) suggestion of possible solution; (iv) development by reasoning of the bearings of the suggestion; (v) further observation and experiment leading to its acceptance or rejection". This process is expressed as a series of highly structured logical statements, which relate beliefs, data, observations, ideas, or experiences to the problem or solution, and that justify judgments and conclusions. The assessment of critical thought in online discussion requires an analysis of the overall process and of the structure and quality of the logical statements or arguments [2,5]. The TAF, fully described in an earlier paper [5], allows such an assessment of critical thinking [5].
Previous assessments of critical thinking in online discussions show a general pattern: the thoroughness and quality of the critical thinking applied by learners is often dependent on the activity or discussion prompt with which they are engaged [3,[7][8][9][10]. In particular, only a few activities support the full process as described by Dewey. Reflective prompts and nominal definition-centered prompts, which are commonly used online discussion activities, generally elicit only two of the steps: exploring the problem and suggesting a solution [3 5, 7, 8]. However, other activities like guided debates, WebQuests, and case studies can support the full critical thinking process [3,[8][9][10]. This dependence on the nature of the prompt or activity creates a pedagogical need to further elucidate the impact of various prompt types and to identify activities that support the full critical thinking process.
MEAs are small group problem solving activities which take generally an hour or two to solve, and have been found to foster critical thinking in F2F environments [4]. Groups are given a real-life situation in which an identified client needs a solution that must be applicable to other similar situations. The requested solution is a mathematical or scientific model that needs to be developed. MEAs are meaningfully situated and require solvers to express, test, and revise their thinking. Clear objectives allow the learners to continuously judge the quality of their solution by fostering multiple opportunities for reflection and explanation [11]. These aspects facilitate application and communication of critical thinking skills and processes as learners "select, filter, organize, and transform information" [12]. The short time frame, group discussion focus, and simple objectives suggested that implementation of MEAs in the synchronous, online environment might also support critical thinking and learning.

Methodology
In this study, we assessed critical thinking as two groups solved an MEA via synchronous online discussion. We adapted the TAF, which allows an assessment of process, structure, and quality of asynchronous thread-based discussion [5]. The unit of analysis for process codes was modified to support transition identification rather than post-description. The primary metric was the percentage of discussion devoted to each step in the critical thinking process, identified as: initiation, exploration, solution, judgment, and resolution. We also considered the claim and evidence structure of arguments in the discussion, including sub-structure content, relevance, and reliability quality ratings.
An overall assessment was done on the validity of the problem solution and consistency with the MEA learning goal, in this case mathematical development. Coding based analysis of this cyclical framework and analysis of the final solution were used to assess critical thinking. Cycles were tallied based on two criteria: (1) the model cycle includes: initial discussion of aspects of the problem and/or of individual ideas and analysis of the model solution; and (2) the model is disparate from the previous one in that it: addresses more aspects of the problem, is more general, or uses a different representation.

Model-eliciting Process and Results
Both groups participating in the online MEA engaged in modeling cycles and produced a final solution consistent with those produced in a F2F MEA implementation. Overall assessment showed that each solution was mathematical, generalized, and applicable to the problem. Groups engaged in six and three modeling cycles, respectively, to produce their final solution. Participants who had previously completed a F2F MEA reported that the two experiences proceeded similarly and successfully [12]. The primary difference reported was that the number of cycles was fewer in the online environment than in the face-to-face environment. They hypothesized this decrease may have been due to an increased ability to develop multiple ideas simultaneously in the online environment or may have been because the cycles took longer to complete in the online environment; however, solvers did not feel this difference had a negative or positive impact.

Critical Thinking Analysis Results
After modifying the framework for synchronous discussion analysis, the two coders employing the TAF achieved substantial agreement (> 70%, Cohen kappa > 0.6, n = 358) on each discussion aspect: process, structure, sub-structure, and quality. Initial analysis of the resulting data showed that a significant percentage of each group's discussion (61.4%, 32.1%) was devoted to social or logistical issues rather than to solving the problem. Only statements identified as contributing to solving the problem were further analyzed (n = 178). Figure 1 shows the distribution of statements for the steps in the critical thinking process. It is noteworthy that judgment and resolution, accounting for 69% of the statements are often lacking in other online discussions [3,5,7,8]. This evidences how MEAs strongly support the full critical thinking process.
Analysis in the structure and quality of the discussion shows further evidence for strong critical thinking when solving an MEA in the synchronous online environment. The average claim rating (2.4, n = 95) indicates that claims were: generally pertinent to and contributed to the discussion; supported by several pieces of evidence, or one very strong piece of evidence; and, consistent with the information and ideas presented. Additionally, the evidence ratings showed strong relevance and reliability indicating welldeveloped and well-justified arguments.

Participant Discussion Results
Participants' post-MEA discussions comparing the F2F and online environments also provided valuable results. Participants identified vocal communication in the F2F environment and multi-lateral communication and permanence in the online environment as the most distinctive features of each. They reported feeling that vocal communication would be beneficial in the online because they sometimes misunderstood chat responses and because of the time and effort involved in typing. Some felt that online multilateral communication facilitated having everyone's ideas included as different members worked simultaneously on multiple aspects of a problem. However, others felt that multilateral communication created confusion about the topic being discussed and detracted from understanding and development. Likewise, some participants positively associated online chat permanence with being able to review previous ideas, being guaranteed a voice, and being able to catch up and not miss things if one was temporarily distracted. Others reported that they held back and contributed less because of nervousness that "stupid" statements would be permanent. Positive or negative views of these unique aspects of the online environment were given by participants reporting high or low general usage of online communication tools, respectively.