How Do Mixed Teaching Teams Influence Learning in IT MOOCs?

The impact of mixed teaching teams on learning quality and the outcomes of diverse target groups in a massive open online learning context in the workplace is scarcely analyzed. Insights from gender studies on female role models, research on team teaching in computer science as well as digitally supported learning do not transcend these separate areas and are therefore of restricted explanatory power. Accordingly, this session will answer the research question on how male-only and mixed teaching teams influence IT MOOCs (Massive Open Online Courses) in regard to the proportion of female learners, participation in the discussion forum, and success rates. This study will be backed by the learning data gathered in the time of running two courses about blockchain with 14,000 learners on our MOOC platform openHPI with a mixed teaching team and a male teacher. The courses are conducted entirely online and are open and free of charge. All the material provided can be accessed from any device connected to the Internet. As our MOOCs in the enterprise context address not only employees, but also partners, students and other interested (lifelong) learners we will include a closer look at learners’ socio-demographic background, such as career status, highest degree, professional life, and position. Additionally, we statistically analyze accompanying survey data using research methods in social sciences. Finally, our session will highlight and discuss challenges of mixed teaching teams in IT MOOCs and possible solutions to support gender diverse online learners in the workplace. Keywords— Gender, Lifelong learning, MOOCs, STEM


Introduction
Social participation and co-creation are tied to the digital world. While every man and woman should be able to move autonomously in the digitalized world, men have been generally shown to have a greater affinity for the digital world (digital index of 63 points from 100) and on the average higher competency than women (digital index of 54 points) in dealing with the challenges of digitization [1]. Sixty percent of the digital pioneers, who exhibit diverse usage behavior, pronounced digital competency, and strong interest in new technologies, are male [2]. "Even though the hard sciences and education overall have become more welcoming toward females, social biases against women's 'abilities' in the STEM fields continue to exist and negatively affect their professional development and their judgement of others" [3]. Accordingly, women are underrepresented in science, technology, engineering, and mathematics (STEM) [4]. Could new formats of further education be a chance for these underrepresented groups to establish themselves?
Massive Open Online Courses (MOOCs) provide a promising mix of structure (new learning content on a weekly basis, deadlines) and flexibility. Participation scalability is almost unlimited without entrance criteria. High quality learning content is available online, covers a few weeks, and gender seems to go in the background. Indeed, digital teaching and learning do not fulfil gender diversity in a wide scope: female instructors as well as participants are still a minority in IT MOOCs. We intend to identify influencing factors to facilitate female contribution and attendance in digitally supported learning. According to the important discussion about the relevance of female role models to arouse, strengthen, and maintain girls' and women's interest in IT [5], we focus on the influence of mixed teaching teams on learning in IT MOOCs.

Research Questions and Hypotheses
This paper will answer the research question of how male-only and mixed teaching teams influence the proportion of female learners, their participation in the discussion forum, and success rates in IT MOOCs. Female instructors may imply a gendersensitive course design, lower the inhibition threshold for girls and women and make it easier for them to get started. Therefore, we assume that in MOOCs with at least one female instructor more women will participate than in courses with only a male instructor (H1). We believe that the higher the percentage of women enrolled at the beginning of the course, the higher the likelihood of their active participation in the forum and level of comfort in the online learning environment. We assume that in MOOCs with a female instructor a higher proportion of women are likely to participate more intensively in the discussion forum than in courses with only male instructors (H2). Furthermore, as social elements are fundamental for a rich learning experience of reconstructing and expanding complex knowledge and understanding [6], we expect that a higher proportion of women complete the MOOC of a mixed teaching team more successfully than online courses with only a male facilitator (H3).
Another special focus of our analysis lies in the similarities and differences of participants enrolled in both iterations of our MOOC about blockchain with different teaching team constellations. As our MOOCs in the enterprise context address not only employees, but also partners, students and other interested (lifelong) learners, we will include a closer look at learners' socio-demographic background, such as career status, highest degree, regular computer use, professional life, and position.

Related Work
In a collaborative (co-)teaching setting, at least two facilitators share resources to plan and implement a course, as well as to assess learning. Even though the "teacher's level of participation may vary and change throughout teaching" [6], co-teaching inclusively welcomes and appreciates members of a learning community with different abilities [7]. The effectiveness of co-teaching skills depends on teachers' positive social interdependence and interaction, e.g., mutual encouragement and facilitation of each other's efforts, and complimentary roles. Those characteristics help foster positive energy in the teaching and learning community, increase achievement, and promote self-esteem. Nevertheless, practice has shown that teachers rarely communicate with each other in front of the learners to avoid confusion and that to plan co-teaching is more time consuming than to plan lessons on their own [6]. Self-selected teaching teams show significantly more shared responsibility and enjoyment with the coteaching process, but it is not a guarantee for a higher quality of collaborative teaching [8]. Heterogeneous gender-mixed teams exhibited particularly strong co-support and cooperated more effectively than homogeneous teams whereas "high heterogeneous teams in tenure were less supportive and effective than low heterogeneous teams in that attribute" [9].
Even though all gender students actually participate more in answering questions when the teacher is a woman [10] they still rate male professors better than female [11]. This is particularly negative in light of an exemplary study about a math class, which shows that female students increase their fondness for math and identify more with the subject when the professor is female [10]. In engineering majors, effective female role models significantly relate to persistence in engineering [12]. Gender role socialization [13] and stereotypes of people in STEM fields significantly decrease "women's interest and anticipate success in computer science. Effects occurred even when the learning material, gender of the professor and gender ratio of the classmates were identical" [14].
Especially female students are often shy or fear classroom dynamics [13]. In line with the YPQA pyramid with a safe environment at the lowest level, followed by a supportive environment, as well as interaction, and engagement at the highest level, women develop as knowers in a course when instructors guide, collaborate with, and encourage students to birth their own knowledge [15]. [4] shows that female instructors "positively impact girls' performance on advanced programming tasks and reduce the number of gender differences between boys and girls in their mastery of programming concepts".
Typically, online learners are on average well-educated, working men in their mid-40s and diversity is still a large issue [16]. In general, women receive worse performance ratings than men when they have a share of less than 20%, "even after male-female cognitive ability, psychomotor ability, education, and experience differences were controlled" [17]. In contrast, some studies found out that females academically perform slightly better than males in online-learning environments [13,18].
The impact of mixed teaching teams on learning quality and outcomes of diverse target groups in a massive open online learning context in the workplace has scarcely been analyzed yet. Insights from gender studies on female role models, research on teaching as a team in computer science, and digitally supported learning do not transcend these separate areas and are therefore of restricted explanatory power.

Data and Research Methodology
The paper at hand is backed by the learning data gathered at the time of running two comparable courses in German on blockchain with more than 14,000 enrollments: "Blockchain: Hype or Innovation?" (blockchain2018) conducted by a male professor and a female research associate and "Blockchain-Security even without Trust Center" (blockchain2019) with only a male teacher [19]. The gender of the tutors was clear right from the beginning of the sign up process. The courses are entirely online and are open for everyone and free to use. All the material provided is accessible from any device connected to the Internet. The courses cover two weeks each. The first week consists of ten and the second week of eleven videos.
In 2018, in all videos, both the male and the female teacher stand in front of the camera to explain the learning content, which is a change compared to the majority of our courses offered by male instructors only. It was our first course on blockchain and it was unclear whether it would take place again. These circumstances may have contributed to a high motivation among the participants. Women's and men's main motivation for joining the course was professional interest and further training. By contrast, women joined the second iteration of the course by a male professor mainly out of personal interest and as a free-time activity (statistically not significant).
On average our female participants use their computer regularly mainly on an intermediate level, e.g., for text processing, spreadsheets, or database administration (54% of female versus 39% of male learners). By contrast, the majority of our male participants reported a high level of regular computer use, e.g., for software engineering, adaption of computer games, programming in languages as Java, SQL, PHP, or PYTHON, or network administration (57% of male versus 27% of female learners, Phi and Cramer-V=0.264***).
Independent of the teaching team constellation, a larger proportion of female learners than males prefer anonymity in MOOCs by participating without a profile picture, but the difference based on gender was slightly smaller under a mixed teaching team than with a male teacher (statistically not significant).
Almost 1,500 participants enrolled in both courses. We investigate this subtotal separately to find the underlying cause of the role models' effectiveness in a mixed teaching team compared to with only a male teacher. We use descriptive and multivariate research methods in social sciences to statistically analyze participants' learning data and outcomes as well as accompanying survey data.

5
Results and Findings

Proportion of female learners in relation to the gender of the teaching team
In the first iteration of the course on blockchain taught by a mixed teaching team in 2018, 17% women participated. Against our expectation, the proportion of women was a little bit higher in the second iteration of the course taught by a male teacher (21% women, Phi, Cramer-V, and contingency coefficient=0.048***), but the total number of female participants was smaller than in the course given by the mixed teaching team (956 women versus 1,500 women). This is also true for our first-time attendees. In the course held by the mixed teaching team, 21% were women (337 in total) compared to 23% female first-time participants (91 in total) in the course of the male teacher. Even if female instructors imply that a gender-sensitive course design might lower the inhibition threshold for girls and women and could make it easier for them to get started, the gender of the course facilitators is not a guarantee for a high amount and high proportion of female participants. Innovative continuing education formats such as MOOCs still have different activation potentials for male and female learners even though a mixed teaching team leads the course. This particularly applies when participants are recruited from the same pool as other IT MOOCs.

Communication in the discussion forum by gender
In principle, women seem to profit from the relative anonymity and additional time to formulate answers in online discussion forums [20]. Independent of the teaching teams' gender, a slightly higher proportion of female than male learners participate actively in the discussion forum (statistically not significant). Under a mixed teaching team, 2% of women and 1% of men open a new discussion thread (Phi=0.029 n.s.); 3% of female learners and 2% of male learners post a question, answer, or comment in the forum (Phi=0.52 n.s.). The small difference between female and male participants appears on a slightly higher level also in the course of a male teacher with lower enrollments in total (2% women and 1% men open a new discussion thread, contingency coefficient=0.027 n.s.; 4% women and 3% men write posts, Phi=0.061 n.s.).
Under a mixed teaching team, women and men open on average a similar number of discussion threads (0.03). By contrast, under a male teacher women open on average very slightly, but statistically significant less discussion threads than men (0.02 versus 0.03 threads**). Independent of the teaching teams' gender, women seem to post somewhat fewer questions, answers, and comments than men (under a mixed teaching team women write on average 0.14 posts and men 0.24 posts, under a male teacher women write on average 0.13 and men 0.2 posts, both differences n.s.).

Success rates by gender of the teaching team and the participants
In the course taught by a mixed teaching team with large enrollment numbers, participants' learning outcome is on average at a lower level than in the course given by a male professor with less enrollments in total. Apart from this, the difference between female and male learners is in the first iteration of the course taught by a mixed teaching team smaller than in the course taught by a male teacher. Women visit on average 55% and men 57% of the learning items of a mixed teaching team (F=0.501 n.s.) compared to 66% and 73% of a male teacher (F=44.342***).
With a mixed teaching team, 36% female participants and 40% male participants received a confirmation of participation by completing at least 50% of the course material (Phi, contingency coefficient=0.036***). With a male professor, 38% female learners and 48% male learners completed the course successfully with a confirmation of participation (Phi, contingency coefficient=0.078***). In the course given by a mixed teaching team, women reached on average 23% and men statistically significant slightly more (26%, F=14.943***). In courses taught by a male professor, women and men get slightly higher results: women receive one quarter of all points and men nearly one third (31%, F=59.486***).
In the first iteration of the course with a mixed teaching team, one quarter of our female learners and 27% of our male learners received a record of achievement by earning more than 50% of the maximum number of points from all graded assignments (Phi, contingency coefficient=0.025 n.s.). In the second iteration with a male professor and fewer enrollments, the proportion of participants, who successfully complete the MOOC with a record of achievement is slightly higher, but the gap between female and male learners increases five percent: 31% of women and 38% of men got a record of achievement. Likewise, in the first iteration of the MOOC with a mixed teaching team and large enrollment numbers, 5% of our female participants and 6% of our male learners were among the top 5, top 10, and top 20% (Phi, contingency coefficient=0.025 n.s.). With the male professor, 7% of our female learners and 9% of our male participants scored particularly well (Phi, contingency coefficient=0.035 n.s.).

Similarities and differences in the participation and success of learners enrolled in both courses
There were 1,434 participants enrolled in both courses (83% men, 16% women). Based on this so-called paired sample, we attempt to analyze the main effect of the different gender of the teaching team in both courses, which means the variance within the persons (intra-subject effect, not differences between persons). We calculate a general linear model also known as single factorial analysis of variance (ANOVA) with measurement repetition [21].
Learners, who participated in both courses, opened more discussion threads, and posted more questions, answers, and comments in the forum of the course with the mixed teaching team than in the course of the male teacher (0.07 versus 0.04 forum threads and 0.65 versus 0.31 forum posts). Altogether, a mixed teaching team has a positive effect on the participants' willingness to communicate in the discussion forum. Furthermore, they visited more items and received more points in the course with both a male and a female teacher than in the second course iteration with only a male teacher (65 versus 56% of items visited and 45 versus 35% of points received). In all, a mixed teaching team positively influences the learning achievement. The intra-subject effects refer to the F-and significance test of the main (direct) effect of the independent factor of the teaching teams' gender to the dependent variables. The results reveal that a mixed teaching team has significant positive main (internal subject) effects on learners' participation in forum threads (F=6.389**) and forum posts (F=4.994**), the percentage of items visited (F=58.699***), and their points received (F=67.704***). The partial Eta-squares (partial η 2 ) measure the effect size or variance explained by the gender of the teaching team. The small values of the partial Eta-square below 0.05 indicate nevertheless only very little direct effects of the teaching teams' gender on differences in intra-subject participation and achievement.
To examine potentially different inter-subject effects of the teaching teams' gender on women's and men's participation and achievement, we integrated participants' gender (which varies between persons, is constant inside a person, and could bias the single factorial analysis of variance mentioned above) as an intermediate subject factor into our model, [21]. Table 1 shows the means of our metric dependent variables by participants' gender for both course iterations. In addition to the main direct effect of the teaching teams' gender, we report the test results of the interaction between the inter-subject effect and the course repetition as well as the inter-subject effects (F-tests) of participants' gender. The overall positive effect of a mixed teaching team on participants' willingness to open discussion threads applies especially for female learners. Fig. 1 visualizes the differences between male and female learners in the course of a mixed teaching team and a male-led MOOC concerning active participation and achievement. With a mixed teaching team, women start on average significantly more discussion threads than in the course of a male teacher (0.08 threads of women in the course of a mixed teaching team and 0.03 threads of women in the course of the male teacher versus 0.07 and 0.04 threads of men). In fact, in the course of a mixed teaching team women have the courage to open even more new threads than men do (0.08 versus 0.07, inter-subject effect F=27.193***, partial η 2 =0.018).
With regard to answers and comments in the discussion forum, the picture looks somewhat different. Concerning forum posts, Fig. 1 shows almost parallel lines with little interaction. This indicates that the teaching teams' effect on learners' motivation to post answers and comments in the discussion forum is mainly additive [22], meaning that the effect of participants' gender is similar across both courses and the effect of the teaching teams' gender is similar across male and female learners, too. Independent of the teaching teams' gender, female participants in both courses react on average slightly less to their fellow learners' discussion posts than men (0.53 posts of women in the course with a mixed teaching team and 0.12 posts in the course of a male teacher versus 0.68 and 0.35 posts of men). However, this is statistically speaking not a significant difference (inter-subject effect of F=0.449 n.s., partial η 2 =0.006 and interaction between inter-subject factors and course repetition: F=0.0360 n.s., partial η 2 =0.000). Nevertheless, women contribute over-proportionally in the course of a mixed teaching team (on average 0.53 posts in the course of the mixed teaching team versus 0.12 posts of women in the course of a male teacher compared to 0.68 and 0.35 posts of men). In this view, a mixed teaching team could encourage women, who tend to hold back in a male-directed course, to contribute actively to discussions about the learning content and to catch up in some degree.
The same is true with regard to women's and men's intensity of participation and achievements. The overall positive effect of a mixed teaching team remains valid for both subgroups (women visited 59% of the course with a mixed teaching team and 44% of the course with a male teacher and men visited 66% and 58% of the learning items). Although, women engage significantly less with the online learning content in STEM than men (inter-subject effect of participants' gender of F=16.846***, partial η 2 =0.012; interaction between the gender of the participants and the teaching teams of F=4.0580**, partial η 2 =0.003). This difference is considerably smaller in the course with a mixed teaching team than in the course of a male teacher. Based on this result, a mixed teaching team seems to support those women who feel less attracted by a maledominated course in following through on online learning in STEM.
Instructed by a mixed teaching team, women as well as men achieve significantly higher results than in the course given by a male teacher (women 40% in the course of a mixed teaching team and 29% in the male-led course; men 46% and 37%). In the course conducted by a mixed teaching team, women's backlog compared to men's achievements tends to be somewhat smaller than in a male-led course (6% with the mixed teaching team versus 8% with a male facilitator, inter-subject-effect of F=7.885***, partial η 2 =0.006; interaction of F=0.4030 n.s., partial η 2 =0.000). Accordingly, especially female participants, who find it easier to learn from female role models than men, could profit from a mixed teaching team in STEM.
Via a multivariate ANOVA (MANOVA) [23], we tested potential influences of additional socio-demographic characteristics, such as learners' regular computer use, highest degree, professional experience, career status, position, and their main motivation to join the course on women's and men's active participation and achievement. We found significant, but very small main effects of highest degree and career status on items visited and points achieved as well as inter-subject-effects of highest degree on learning achievements (partial η 2 smaller than 0.015), while some intra-subject effects of the teaching team constellations as well as inter-subject effects of gender become partly statistically not significant. Further research needs to be done in this area.

Participants' evaluations of courses with different teaching team constellations
There were 1,870 learners who participated in the surveys conducted after the end of both courses-309 women and 1,528 men. Among other questions, we asked how they rate the quality of the course to examine whether participants' evaluation of the courses reflect positive effects of mixed teaching teams. In the bar chart (Fig. 2) the results of our descriptive analysis show that the majority of participants rated the quality of content as good, especially the quality of the MOOC held by a mixed teaching team. Nevertheless, particularly female participants evaluated the quality of content of the MOOC given by a mixed teaching team as (very) good (90% of women versus 83% of men), whereas male participants evaluated the second course given by a male teacher better than the female learners did (79% of women versus 82% of men). Up to now, most MOOCs in STEM have been created by men for men, but "online course design integrates inclusive practices across the learning environment for all learners to enjoy" [24]. We have shown that mixed teaching teams have positive effects on active participation and learning outcomes of female as well as male learners and reduce differences between female and male participants in online courses. Heterogeneous teaching teams are fundamental for inclusive further education. Beyond teaching team constellations, single participants profited from a high status of the teacher and a smaller course size.
We nevertheless state that our case study is based on a self-appointed sample and not a strict experiment. The female teacher as a junior member of the mixed teaching team may have an effect on the results, too. Due to the data situation of voluntary sociodemographic profile information, we had to exclude some independent variable from our model. Besides, the participants of our post-survey are generally successful learners who are still active at the end of the course. Accordingly, the results have to be interpreted cautiously.
In our future research, we want to focus on the behavior of the teaching team by gender. A well-functioning mixed teaching team might be reflected by a wellfunctioning diverse learning community whereas poorly-functioning mixed teaching teams in which, for example, the male facilitator repeats a woman's words in his own words, can also result in discriminating behavior in the online discussion forum and learning context.