The Gender Gap in Engineering Education During The COVID-19 Lockdown: A Study Case

borja.bordel@upm.es Abstract — Higher education in Spain, especially in Madrid, was suddenly and unexpectedly shut down on March 9 th 2020 because of the beginning of the COVID-19 first wave emergency. In engineering education, where practical laboratories are a relevant part of the educational process, professors followed different approaches (sometimes concurrently), designed after only three or five days of discussions. Although, globally, the obtained results are acceptable considering the situation, after informally analyzing the data and observing the post- lockdown students’ profile, some collectives seem to have suffered a higher impact than other. Three research questions are analyzed. The objective of this work is to analyze if the performance of women in engineering courses, specifically in computer engineering, is different from male students. The study case is carried out at Universidad de Madrid, particularly, in the computer engineering degree. Furthermore, if a statistically relevant difference is discovered (based on Mann-Whitney U test), the final causes of this worrying situation will be studied. Official academic results and other similar indicators were analyzed. Besides, more than one hundred surveys among students were developed. Percentage of women (participants) is above 30%. Results clearly show a deterioration in all indicators for all collectives and students, comparing the performance during the lockdown and the performance of previous years. However, this impact is not homogenous, and results also show a gender gap placing women in engineering education during the lockdown (an after) in a disadvantaged


Introduction
Technical universities in Spain have a long tradition of fully presential classes, laboratories, and other similar activities [1]. In general, professors are prepared for years to control methodologies such as project-based learning, challenge-based learning, etc. [2] On the other hand, online teaching is associated to some different universities (such as Universidad Nacional de Educación a Distancia, UNED) focused on methodologies such as flipped classroom or autonomous learning, where presential classes are sparse and mainly limited to exams and similar evaluation activities [3].
This ecosystem, usually, allows higher education institutions to adapt their facilities and teams to a set of only three or four methodologies (with a lot of common characteristics), so the cost and investment efficiency is clearly higher than in a more heterogeneous approach. For long years, nobody envisioned a critical problem in this strategy, working now for (approximately) sixty years [4].
This equilibrium got wildly broken in 2020.
In February 2020 the first cases of COVID-19 (a new disease generated by a coronavirus originated in Wuhan at the end of 2019) were detected in Madrid [5]. Although the same virus had caused a terrible social and sanitary crisis in China (first) and Italy (in January 2020), including a hard lockdown and the suspension of all presential classes (at all educational levels), no alarm was triggered in Spain. In fact, in the early days of March, the government ensured the control on the pandemic and the normal development of the second term (from February to June) in all Universities and schools [6]. Considering this information, no infrastructure, methodology or roadmap was prepared or, even less, deployed in universities to palliate the potential impact of this crisis in education. And despite all these messages, higher education in Spain, especially in Madrid, was suddenly and unexpectedly shut down on March 9 th 2020 because of the beginning of the COVID-19 first wave emergency.
Then, a non-systematic reorganization of higher education was triggered [35]. In general, although the final objective was to guarantee the continuity of the educative process in all institutions, professors were totally free to manage this situation according to their preferences [36]. In particular, in engineering education, where practical laboratories are a relevant part of the educational process, professors followed different approaches (sometimes concurrently), designed after only three or five days of discussions. From webinars to virtual laboratories, flipped classrooms and the absolute cancellation of every kind of practical activity. Because of this heterogeneity, results and efficacy of the different subjects were very different [31]. For example, in some institutions (such as Universidad Politécnica de Madrid, UPM) more than three different tools for online meetings were employed, different policies about online classes and videos were stablished (synchronous, recorded, no classes), and evaluation methods, many times, were transformed regardless the social situation and the schedule and changes proposed in other coincident subjects.
In June 2020, when the term was over and the first COVID wave in Madrid was under control [7], observations showed that, globally, the obtained results were acceptable considering the situation. However, after informally analyzing in a deeper way the data and observing the post-lockdown students' profile, some collectives seem to have suffered a higher impact than other [33]. Socioeconomical differences were early detected, and universities deployed policies to provide vulnerable students with computers, internet connections and other similar resources. Nevertheless, other biases have not been formally detected until now. In particular, it is clear than the number of women in the post-lockdown engineering courses in Madrid was reduced. This effect seems to refer to a higher impact of the COVID lockdown and pandemic in the female students (in engineering and technical degrees) [32]. However, before concluding any assessment, causes or problems to be addressed, a scientific analysis must be carried out.
The gender gap in technological and science degrees is known. Many different studies have analyzed the gender equality is STEM (Science, Technology, Engineering, and Mathematics) programs since 1971 [26]. In general, the percentage of graduated women in engineering is much lower than the percentage of men [29]. And this gap is not evolving in a linear way, so in some periods it has increased and in other ones it has decreased [27]. Nowadays, especially in Spain, this gap seems to increase, regardless the COVID crisis. However, the gender gap evolution is slow, and changes typically require some years to be consolidated and observable [28]. On the contrary, in 2020 a fast, relevant and very worrying decreasing in the number of female students has been detected [30], whose causes cannot be only related to the traditional gender gap, as its characteristics do not match the features of slow societal changes. Considering the great, and still unknown, impact of COVID crisis in society, including education, [34] the fast increase in the gender gap in 2020 could be caused by this global crisis.
Therefore, the objective of this paper is to analyze if, during the 2020 lockdown and COVID crisis, the performance of women in engineering courses, specifically in computer engineering, was different from male students. Causes of this difference, if exists, are also investigated. In particular, we are studying if the academic performance during the lockdown was different, if this difference had a relevant impact in academic results and, moreover, if this situation has permanent consequences with a higher and anomalously high dropout rate among female students. The study case is carried out at Universidad Politécnica de Madrid, particularly, in the computer engineering degree.
Furthermore, if a statistically relevant difference is discovered, the final causes of this worrying situation will be studied. Problems and bad educational habits [24] that have helped the gender gap to increase during this sanitary crisis will be described, and some recommendations and probable actions [25] to, at least, partially solve and revert the caused damage will be proposed.
Three basic research questions are guiding all our work: ─ RQ#1: Was the educational performance (attendance rate, participation, etc.) of women during the lockdown worse than male students' performance? ─ RQ#2: Were academic results of women during the lockdown worse than results of men? ─ RQ#3: Was the gender gap increased after the lockdown?
Contrary to other previous works, in this study we are not only focusing on how the gender gap in engineering education evolves or on how the COVID crisis has affected education, but on both aspects at the same time. In that way, we'll be able to determine if the gender gap has suffered a relevant and sudden increase because of COVID emergency. To the best of our knowledge, this is the first work considering this approach.
The rest of the paper is organized as follows: Section 2 describes previous works on the impact of COVID in education and the gender gap. Section 3 describes the methodological approach and the analysis techniques employed in this study. The groups under study and the considered data inputs are also presented. Section 4 re-ports the obtained results and some discussions. Moreover, some recommendations to address the observed challenges are also included. And Section 5 concludes the paper.

Previous studies about education, COVID and the gender gap
Although COVID first wave emergency has been a recent event of the last year many different reports about its impact in society and education have been published [49].
Most of these works, however, are still qualitative and descriptive. One of the most affected disciplines by this crisis is medicine, where practices in hospitals and other real scenarios are essential. Thus, several reports about teaching medicine in the COVID era may be found [8] [9]. However, no quantitative analysis is typically provided, and only some new platforms and methodologies for medicine education [10] or literary descriptions about the experience of teaching medicine under a lockdown are reported [11]. Similar works (although much less numerous) have been reported in disciplines such as science [12] or orthopedic [13]. Besides, some papers with a geographical point of view describing (in a qualitative manner) the impact of COVID emergency on the Indian [14] or Philippine [15] education system may be also found.
Quantitative analyses are very sparse, and they are still very initial, where only some few variables are compared using heuristic techniques [16] [17]. On the contrary, only some studies have reported some important and clear conclusions, such as the relevant reduction in the professor-student interaction caused by COVID in 2020 in Germany [18].
In this context, analyses about the evolution of the social gaps (including the gender gap) in education are still very limited. Although general reports about how gender violence [19] or the socioeconomical differences [20] or, even, the gender gap [21] have evolved in society during this unexpected situation have been commonly reported; studies focused on the higher education context are, on the contrary, very sparse. In fact, only some initial works may be found. Specifically, in India, it has been analyzed those asynchronous methods are those that increase the least the socioeconomic gap among students [22], mainly because of the limited access to internet in the country. On the other hand, in Indonesia, authors have found (in a preliminary study) how the strong gender roles in the country have greatly affected the learning of female students during the lockdown [23].
Contrary to this situation, many different authors have analyzed the gender gap in engineering education, regardless the COVID emergency. Some methodologies to assess the evolution of the gender gap (using questionnaires, for example) have been reported [29]. On the other hand, analysis methodologies [28] may be also found, for example based on key performance indicators. Regarding works reporting specific results about the gender gap (size, causes, etc.), different studies focused on different geographical areas may be found. From works supported by international institutions, such as UNESCO, with a global scope [27], to works focused on specific cultural areas (such as Europe) [37], countries [43] or even on specific Universities [38] [39]. Finally, some authors have investigated different strategies to reduce this gender gap [42], ranging from promotion activities among women in high schools [40], to support STEM entrepreneurship policies [41].
Nevertheless, no formal analysis about the gender gap in the post-pandemic higher education institutions have been reported. Including the temporary and permanent effects, and their causes; and specially in degrees (such as technical degrees in the European countries, including Spain) where policies to reduce the gender gap have been applied for the last ten years. This paper provides an initial study addressing this challenge.

Participants, data collection and analysis methodology
The proposed study case was carried out at Universidad Politécnica de Madrid (UPM) between June and September 2020. In particular, professors, coaches and students from the computer engineering degree at the Computer Science School participated in this analysis as volunteers. In order to guarantee a relevant sample, students, professors and coaches of all ages, genders and knowledge areas were involved. Students and professors with full-time and part-time schedules were also considered.
The analysis described in this paper was planned, guided, monitored and evaluated by its authors (hereafter experts), who have more than five years of experience in computer science and data analysis. The experts chose, from the general population, specific students, professors, and coaches to participate in the study. The population under study was selected and configured to guarantee its heterogeneity and statistical relevance. The selection process performed by experts took into account different profiles, as said before. Moreover, participants were selected considering the principles of gender equality.
All the participants were treated anonymously by experts. No personal data related to the participants' identification were stored or diffused outside the official platforms. All the experiments were performed under the conditions of respect for individual rights and ethical principles that govern research involving humans. Table 1 and Table 2 show the characteristics of participants involved in this study.  Two basic information sources were collected and analyzed together in this work. On the one hand, official academic results (number of passed subjects and marks). On the other hand, more than one hundred surveys among students were developed. These surveys were based on a Likert scale, with questions regarding the educational performance, the affective, emotional and familiar situation and, also, the economicrelated problems. In this scale (1)(2)(3)(4)(5), the unit represents "strongly disagree", and the maximum value (five) represents "strongly agree". The entire survey included fortyseven different questions. Besides, a space for free-text comments where participants also could describe the reasons why they had problems to develop a normal educational process, or why they dropped out college, was also included. Table 3 shows a brief extract from this survey.
Professors also answered a survey about their observations during the lockdown, regarding the educational performance of students. In this case, only twenty-seven questions were included in the survey. Educational coaches were also provided with a similar survey, regarding the family, economic and sanitary situation they have addressed and the profile of affected students. In this final case, the survey included twenty-eight different questions. From the analysis of these surveys, five basic indicators about the academic performance of students were extracted: the attendance rate; the participation level; the total number of studying hours; the total number of hours dedicated to childcare, domestic activities and similar tasks; and the difficulty level to conciliate housework and academic tasks. To do that, the punctuation of questions measuring those indicators in the surveys were added to obtain a final result for each participant.
In order to create a control group, the academic results and Key Performance Indicators (KPI) extracted from data about previous courses (2018-19 in a general way) were also considered to discover statistically relevant differences between a normal educational process and education in COVID-19 era. Considered KPI include: the success rate, the attendance rate, and the efficiency rate.
Besides, and finally, the dropout rate was also analyzed and controlled. Results were analyzed using three different tools. First MATLAB and R software suites were employed to digitalize and process all surveys. Besides, using statistical libraries, data were visualized and compared using commonly accepted tests, specifically the Mann-Whitney U test. The Mann-Whitney U test is a nonparametric test of the null hypothesis that two samples come from the same population against an alternative hypothesis, comparing the mean values of the two samples. It is used to evaluate if two different data populations are similar or different (higher or lower). The pvalue indicates the significance level of Mann-Whitney U test.
On the other hand, in order to analyze free-text comments provided the participants, unlabeled Machine Learning classification techniques and Big Data grouping technologies were employed to cluster comments describing equivalent problems and situations and to label the description as a negative or positive reference. Statistical results from this analysis will be also considered.

4
Results, discussions and recommendations Table 4 shows the results from the statistical test, comparing the academic performance of female students (pilot group) during the lockdown, compared to the performance of men in the same period (control group).
As can be seen, a clear gender gap is observed. Statistically relevant differences are observed for all indicators, showing that female students had more difficulties to conciliate the domestic and academic tasks, so they could not attend classes and participate at the same level than male students. This gap is also observed in the work and domestic hours, which tend to be highly different depending on the gender (with a much higher dedication of female students to family care than male students). Actually, some previous proposals focused on breaking the traditional gender gap in engineering education are looking for new conciliation policies, and safer spaces to promote the women participation in class [40]. NS not significant; * significant at p < 0.05; ** significant at p < 0.005; *** significant at p < 0.001.
Before answering RQ#1, we are analyzing how this gap evolved in comparison with the previous year. To do that, academic performance indicators of students during lockdown (pilot group) are compared to existing registers (control group) when possible. Table 5 shows the results. In this case, only the attendance rate and conciliation level were registered during 2018-2019 course. NS not significant; * significant at p < 0.05; ** significant at p < 0.005; *** significant at p < 0.001.
In general, academic performance of students decreased during the lockdown. However, not in a homogenous manner. From these results, it is clear that, although the gender gap existed before the COVID emergency [47], this situation has increased exponentially its impact and visibility. Actually, differences in global performance between female and male students in STEM programs are well-known [47], and different techniques (such as the introduction of "friendly instructors") have been investigated [48]. However, previously reported differences [47] in very recent works are much lower [48], and societal changes (which are very slow) cannot explain this fast increase. COVID crisis emerges as the most probable explanation.
Therefore, and finally, we can answer the first question RQ#1. The female students' performance was worse than the performance of male students during the lockdown in computer engineering subjects.
Nevertheless, these results have not analyzed the cause for this gender gap increase. Thus, Table 6 shows the punctuation reported in the surveys by female and male students regarding to key aspects about the physiological situation: stress and worry. As can be seen, both indicators are much higher in female students than in men, so the underlying society roles regarding women as a family caring role impacted in a negative a very relevant way on female students during the lockdown and the sanitary crisis. Previous works [44] have already identified the work-family conflict as one the main barriers for society during COVID crisis, but it is clear the impact is not homogeneous (at least in an educational context). Thus, we highly recommend Universities and other higher education institutions to create specific policies focused on conciliation and psychological services to reduce this gap in those crisis situations. NS not significant; * significant at p < 0.05; ** significant at p < 0.005; *** significant at p < 0.001.
Although academic performance was in general worse in female students, we are now evaluating if this negative impact was also transferred the official academic re-sults. Two indicators are considered: number of passed subjects (in percentage) and marks. In general, in this case, the gender gap is also present, but in a weaker manner. NS not significant; * significant at p < 0.05; ** significant at p < 0.005; *** significant at p < 0.001.
As can be seen, the percentage of passed subjects (in respect to the initially enrolled subjects) was almost similar between female and male students, although marks do show a clear decreasing in the case of women. It is clear that students tried to maximize the first indicator, although the hours invested in each subject was lower and then, the learning level was lower as well.
At this point, other factors also contribute to this gap, such as stereotypes and roles [46], that affect the self-perception and self-confidence, especially in mathematical abilities (which as critical in engineering) [45]. Although this effect is known and a challenge to be addressed, differences are usually greater in secondary educative levels, and tend to reduce (or disappear) in higher education. The appearance of these large differences among pre-graduated students during COVID crisis needs to be analyzed, because of its fastness and rarity.
Before answering the second research question RQ#2, we are analyzing how this gender gap evolved in the last years, comparing previous results to existing records from 2018-19 course. Table 8 shows the results of this comparison. NS not significant; * significant at p < 0.05; ** significant at p < 0.005; *** significant at p < 0.001.
The conclusion is coherent with all previous results and conclusions. In general, students tended to maximize the number of passed subjects (where globally, no statistically relevant difference was observed), although in this process the learning was weaker and then, obtained lower marks than in previous courses. Nevertheless, the gender gap in this case was greatly increased. In general, their marks and the percentage of passed subjects suffered a very relevant decreasing. On the hand, this effect is clearly related to the decreasing in the academic performance during the lockdown. But, on the other hand, the flexibilization of the lockdown in May 2020, and the backup exams in July 2020 (as reported by female students in the surveys) helped a very relevant manner to reduce the initial negative impact of the sanitary crisis. So, the percentage of passed subject did not decrease as much as expected.
Therefore, at this point, we can answer the second research question RQ#2 and confirm that academic results of women during the lockdown were worse than results of men. Considering their work capabilities and disposal, we are recommending defining special evaluation activities, process and periods for people with special needs because conciliation or their family situation (specially for female students). This approach may palliate this situation and reduce the gender gap in those kinds of extraordinary emergencies and circumstances. We need to remember that the workfamily conflict has arisen as one of the main challenges in this COVID crisis [44].
At this point, analyzing results in Table 5 and Table 8, we could also answer the third and final research question RQ#3. However, first, we are analyzing the evolution of the dropout rate in the last courses. As control group we are considering the results from course 2018-19, while students in the lockdown were taken as pilot group. Table  9 shows the results. NS not significant; * significant at p < 0.05; ** significant at p < 0.005; *** significant at p < 0.001.
As showed, the dropout rate slightly increased for female students, while globally and for male students the change was not statistically relevant. In global numbers the situation is not critical or even worrying, but it clearly refers an increase in the gender gap in the context of computer engineering. In this case, moreover, with permanent effects, as those female students will have a lot of obstacles to return to the University and finish their degrees. Although this difference in the dropout rate is one of the main traditional challenges in terms of technology, education and gender gap [27], this fast increase caused by COVID cannot be ignored. As this crisis can last several years, it could destroy the advances in the last years, when the number of graduated women in computer engineering was growing up slowly [28].
Therefore, we can now answer the final research question RQ#3 and confirm the gender gap in computer engineering degrees have increased during the lockdown.
As a final and global discussion, results clearly show a deterioration in all indicators for all collectives and students, comparing the performance during the lockdown and the performance of previous years. However, this impact is not homogenous, and results also show how there is, actually, a gender gap placing women in engineering education during the lockdown (an after) in a disadvantaged situation.
Although deeper analyses may discover more complex conclusions, some results can be already presented and discussed. The performance of women in computer engineering during the lock down is significantly lower than men. The most common reasons are they felt more stressed during this time, as well as their role at home, where they tend to help third parties and care about other significance more than male students. Most female students report that they were unable to conciliate educational time and home duties. After the lockdown was relaxed in May 2020, with less pressure and need for involvement at home, a percentage was able to address the situation and, although still worse than results of male students, academic results of women were closer to results obtained by men.
Despite this apparent partial solution, other permanent effects have been besides reported. As the sanitary crisis does not reduce its intensity, a statistically higher number of women have dropped out college, under the argument they cannot manage family issues, sanitary problems (sometimes parttime or full-time work) and college at the same time. In this case, women demand from universities and professors more flexibility and conciliation plans.
As a global conclusion, we found out the gender gap in engineering courses has increased during lockdown.

Conclusions
Female students have suffered the consequences of the lockdown more than men. They felt more stressed and worried and, besides, they tended to contribute more to home duties. They also helped third parties more than male students. Consequently, their educational performance during the lockdown was worse than men's. This situation, although much smoother thanks to the hard work of our female students and the relaxation of lockdown since May 2020, is also reflected in academic results. The most worrying situation, however, is the finding of relevant and permanent effect of unnormal drop our rate among all students but, specially, among women. This is clear that conciliation policies should be extended to students when they are enforced to remain at home. In that context, women are more vulnerable, and a clear methodology and strategy to address the gender gap in this scenario are essential.
Future works will analyze data with a higher granularity, considering potential differences depending on the subject, the course, and the age of the participants.