Rocky Mountain Scholars Program: Impact on Female Undergraduate Engineering Students – Social and Academic Support, Retention, and Success

The Rocky Mountain Scholars Program (RMSP) was developed, in part, to improve student success and persistence in Engineering disciplines at Colorado State Universi-ty through a portfolio of engagement activities focused around undergraduate research experiences. Female RMSP participants exhibited substantially higher retention rates and grade point averages relative to other female engineering students at CSU. To better understand the impact of the RMSP, and its effectiveness among female engi-neering students, researchers focused on whether, and how, experiences and percep-tions differ between male and female students in engineering programs. That is, how do male and female students differ, if at all, in their subjective perception of life as an engineering major at CSU? A survey was developed measuring resilience, self-efficacy, motivation, social support, academic support, and perceived sexism. Data was obtained from 144 first-year engineering students at CSU. Results indicated that social support from extracurricular activities is particularly important among female students. This points to an increasing need for programs like the RMSP to create social networks among students and faculty, link students to the broader impacts of their work, and ultimately improve the undergraduate experience of under-represented groups in STEM programs.


The program
In 2011, Colorado State University launched the Rocky Mountain Scholars Program to improve the support, success, and retention of students in Science, Technology, Engineering, and Mathematics disciplines. The program provides a cohort-based experi-ence built on a foundation of student engagement in faculty-mentored research experiences designed to provide the critical support mechanisms linked to student persistence in college and career.
Once recruited, students enter one of three research cohorts that explore the applications of engineering on medical treatment and global health disparities. Additional attributes of the program include: 1) participation of students in an orientation program; 2) co-enrollment of students in a one-credit engineering research seminar section open only to students in their cohort; 3) training workshops to prepare them for participation in CSU's undergraduate research symposium; 4) training to serve as peer mentors; 5) engagement in a community outreach program; 6) participation in CSU's STEM Career Fair; and 7) involvement in a range of social activities. The program is designed to accommodate a total of 45 women engineering majors. This program is housed in CSU's Center for Undergraduate Research where participants have designated space and access to the Undergraduate Training Laboratory, the Undergraduate Research Library, the Student Scholars Lounge, and the full breadth of additional resources offered by the Center.
The program is based on Tinto's model of Student Retention [1] as well as his more recent research and that of his colleagues, which emphasize student engagement as the most critical factor in retention programs for undergraduate students [2], [3]. "These studies argue that if students do not feel engaged within the context of their program, they will leave the university prematurely." [1] Engagement is defined as student-focused activities that link students actively in both social and classroom settings. Examples of social engagement include peer study groups and learning communities, while involvement in faculty-mentored research and co-enrollment in a research seminar are examples of academic engagement. The RMSP provides both social engagement and academic engagement to participating students.

Social engagement: Increasing social support and belonging
Research on individuals enrolled in undergraduate STEM programs [4], [5], [6], [7] consistently report that the women who leave do not have a feeling of connection to others, their peers or their instructors. They do not feel engaged or welcomed in their classrooms or in their living environments. Often, these students become frustrated as they lack confidence in their ability to navigate the complexities of the systems in their home department or institution [8], [9].
The RMSP utilizes several mechanisms that may increase social support and build both strong and weak ties [10], including providing direct mentoring, connecting students across STEM disciplines, and creating opportunities for social engagement.

Academic engagement: Research experiences that connect theory to practice
There is also increasing evidence indicating that undergraduate students who are involved in research achieve better academic outcomes. Undergraduate research can pro-vide students with practical problem-solving skills [11], [12]. While involved in undergraduate research, students' are exposed to and respond to real world problems. Subsequently, they can then reflect on the outcomes of their actions and this serves as a means for development and evolution of knowledge related to their areas of research [13].
Studies on engineering students have indicated that undergraduate research experiences lead to better academic performance and higher rates of retention [14], [15], [16], [4], [13]. Undergraduate research is particularly suitable for bridging difficult subjects across multiple disciplines [17] as is increasingly required in the STEM workforce.
Tinto [1] has also more recently emphasized the importance of faculty/mentor contact for students within a context of active student engagement, including tasks like research.
The RMSP provides engaging research working alongside CSU faculty to connect classroom concepts with real-world applications. The opportunities for research include: 1) design, synthesis, and testing of small molecule inhibitors with therapeutic applications; 2) design, fabrication, and testing of medical devices; and 3) civil and environmental engineering with applications in global health disparities.

The research
The RMSP program has been successful. Interestingly, it is particularly effective at increasing the recruitment, academic performance, and graduation of women majoring in biomedical engineering. Most women in this discipline at CSU average a 60% rate of retention and a 3.41 average GPA. However, among the dozens of women participants in CSU's RMSP program, there is a 100% rate of retention and the average GPA is 3.89. All of these women either entered the engineering workforce or entered engineering graduate programs upon graduation from CSU.
Though the researchers broadly know the challenges facing female engineering majors, and the benefits of social support networks and research experiences on undergraduate students, it was unclear what elements of the RMSP were impacting this particular group of women.
This study explored factors that could explain the persistence of female engineering students at CSU. This exploratory research investigated individual resilience, social support networks, and motivations among male and female students in an engineering degree program at CSU to answer the question: How do experiences and perceptions differ between male and female students in engineering programs?

Methodology
Researchers focused on broader patterns and factors among all first-year engineering students that could ultimately inform programmatic interventions for female engineering students. The use of a survey provided an opportunity to examine the balance of individual and social factors influencing student retention, persistence, and success. Questions were developed to examine individual measures of resilience, self-efficacy, and motivation with an exploration of social context through measures of social and academic support, as well as perceived sexism.
Ultimately, this research utilized two measures of self-efficacy, two measures of resilience, four measures of motivation, two measures of perceived sexism, three measures of social support, and one measure of academic support. Responses were gathered on a five-point Likert scale from "Strongly Disagree" to "Strongly Agree." Open-ended questions were included before Likert-scaled measures of motivation to gather more detailed information and avoid steering students towards particular answers regarding their motivations.
Because the impact of the RMSP has been different across degree programs, ranking questions were added to gather information on student perceptions of different degree programs, and to understand how students choose, or potentially, avoid certain degree programs. Ranking questions asked students their perceptions of which programs were most supportive, most difficult, and most beneficial to career prospects. Open-ended questions were also utilized to understand the barriers and benefits created by each degree program. These were asked before ranking questions on which degree programs were most desirable, most difficult, and most supportive to avoid narrowing student responses. Table 1., below, details the measures used to identify self-efficacy, resilience, motivation, social support, academic support, and perceived sexism.

Survey methods
An online survey was created using Qualtrics and piloted by the CSU Walter Scott Jr. College of Engineering (COE) for validity. The COE provided a list of 686 firstyear student engineering majors. To encourage participation, researchers conducted a raffle for a $50 Amazon gift card for students who completed the survey.
After the responses were reviewed, it was determined that biomedical engineering students were underrepresented. As a result, researchers contacted the COE and asked that an email be sent with a survey recruitment message through an undergraduate advisor. A link was provided to one advisor and one administrator at the university, along with recruitment text explaining the purpose of the study and sharing the protections in place to ensure student anonymity. This resulted in 52 additional responses. Researchers attempted to increase participation among RMSP students by utilizing similar strategies but were unsuccessful.

Data cleaning and analysis
Quantitative analysis: In total, 163 responses were received. Nineteen responses were ineligible and removed during the data cleaning process. All remaining survey responses were analyzed. Comparisons of groups were conducted using non-parametric Kolmogorov-Smirnov tests. This test compares the difference between groups across the entirety of the empirical distribution, as opposed to only comparing the central tendencies. Differences were considered significant at p < 0.05, the standard cutoff in the social sciences. A measure of "grit" was developed as a composite of three Likert scale items measuring subjects' self-assessment of their own resilience and ability to respond to setbacks. This composite measure is the result of an iterated principal factor analysis conducted on the polychoric correlation matrix of the three items. We performed factor analysis on the polychoric correlation matrix due to the non-continuous nature of the underlying items. The results suggested one underlying latent factor accounted for most of the joint variability between these three items. That is, the eigenvalue on the first factor exceeded one, and no other factor exhibited an eigenvalue near one [18]. Items of this composite factor were also analyzed separately and no significant gender differences were found, either for the composite factor or the underlying items Qualitative analysis: An initial analysis of the qualitative data generated an exhaustive list of themes; further iterations refined categories across responses. Finally, each theme was coded as a binary variable, with "0" representing that the theme was not present in the response and "1" signifying that the theme was present in the response. Qualitative responses were compared with demographic information (gender, major, etc.) to suggest relevant patterns. Demographic information and binary variables were imported into Stata 15.1 for preliminary distributional analyses.

Respondent demographics
The survey resulted in 144 responses from current first-year engineering students, including 59 men and 76 women. This represents an oversample of female engineering majors, who make up just over 25% of incoming engineering majors at Colorado State University (see Fig 1).

Fig. 1. First-year engineering students and survey respondents
Survey responses represented all eight majors in the College of Engineering. The response rates differed across degree programs, ranging from 8% (Engineering Science) to 38% (Biomedical Engineering) (Fig 2).

Fig. 2. Response rate by major
Survey respondents self-reported their race and ethnicity. A comparison of survey respondents to the reported race and ethnicity of incoming freshman in the College of Engineering suggests that our sample may be slightly more diverse than the college as a whole. About 74% of incoming freshman in fall of 2018 were white; among survey respondents, 68% identified as white.
Just over 10% of respondents identified as first-generation college students. Across the College of Engineering, about 15% of students identified as first generation.

Key themes
The researchers explored the journey from choosing an engineering major to entering the workforce through three key themes: pathways to an engineering major, building resilience, and planning for future success.
Pathways to an engineering major: "I love math and science, and I want to make a difference in the world." -Female Mechanical Engineering Student.
Pre-college experiences provide important pathways into engineering majors; however, it is not vital that these experiences be explicitly engineering-related. Math, science, and engineering courses in high school, as well as participation in camps and clubs, help inform students' major choices. There was no difference in pre-college experiences or interests between male and female respondents.
When asked why they chose to study engineering, the most common responses among students dealt with their personal interest in subjects like math (28 responses) or science (20 responses). Collectively, these codes appeared in 42% of responses, with no difference between male and female respondents. As one male mechanical engineering student stated, "I chose to study engineering because I was always better at math and science majors than other classes, and engineering seemed more interesting than other options." An interest in building, inventing, and problem solving was also a common pathway. Nearly one-third of respondents (with no difference by gender) identified this as a critical pathway to their engineering major. A female chemical engineering student stated, "I chose to study engineering because I like to problem solve and know why things happen." A male computer engineering student responded, "I really enjoyed math in high school, and I have always wanted to work with my hands to create and design new things. I am also good at problem solving." Just over a third of students (35%) had participated in engineering activities before entering college, including engineering classes in schools, clubs, camps, etc. Most of these students (75%) had taken engineering classes in school; more than half (60%) participated in multiple pre-college engineering experiences, including classes, job shadows, clubs, or camps. There was no difference in pre-college participation between male and female students. The importance of pre-college experiences, specifically of high school experiences and early role models in engineering (including teachers and family members), appeared in about 15% of students' responses when asked why they selected engineering as their major.
Participation in pre-college engineering activities may bolster confidence in one's engineering knowledge and abilities. Students who had participated in pre-college engineering activities were more confident about their ability to do well in their degree program.
Building resilience: "The degree is very difficult and sometimes hard to manage. The requirements sometimes seem impossible on hard days and the workload and required effort to put in makes studying this very difficult yet very worth it."-Female Electrical Engineering Student. Students recognize that completing an engineering degree can be challenging, and demonstrate a high level of motivation, resilience, and pride. In open-ended responses, both male and female students highlight the challenge of engineering as particularly appealing, and some indicate that completing difficult coursework now will prepare them for future challenges.
Respondents reported a high level of self-efficacy and resilience (Fig 3). Nearly 90% of students agreed or strongly agreed that they "always" met the goals they set for themselves. More than 70% said they were able to handle unexpected problems well. More than 80% said that failure just made them try harder. There was not a statistically significant difference between male and female respondents on any of these measures.

Fig. 3. Male and female students' levels of self-efficacy and resilience
Ranking questions on the perceived difficulty of, and opportunities associated with, different engineering majors revealed that students largely believed their degree program was the most difficult. When asked how their degree program made it more difficult to succeed, 60% of responses highlighted the difficulty of course material. About a quarter of responses dealt with the time and effort required to complete the degree. Some students connected the time and effort needed for the degree requirements with trade-offs in their personal and professional lives, mentioning, for example, that they would have less time available for "networking" or internships. As one female civil engineering student stated, "It can take away time that could be spent working in the field and getting hands-on experience." Others felt that the workload isolated them from their peers in other degree programs. Students from multiple degree programs shared that their degree program set them apart, not from other engineering students, but from their friends and peers in other departments. When asked how their degree program makes it more difficult for them to succeed, one male chemical engineering student stated, "[My major] limits my options for other academic, social, and recreational growth during my college years as it will take up so much of my time." This social isolation from peers in other degree programs does not appear to be mitigated by social support within the department. Though academic support appears to be a strength across degree programs (77.5% of all respondents said they had access to academic support), fewer students felt supported socially. When asked how their degree program made it easier to succeed, just four students specifically identified social support as a strength of their department (three female biomedical engineering students and one male mechanical engineering student). About a third of students agreed or strongly agreed that they didn't fit in with their department, and nearly 10% said they didn't have anyone at CSU they could turn to for advice. The percentage of students who felt they didn't fit in was similar between male and female respondents. Non-white students were more likely to say they had someone they could turn to for advice. Less than half (43.7%) said they had a role model in their engineering program. There was no difference between male and female students, or between students in different degree programs on this measure. Extracurricular activities may be an important source of social support. Students who participated in engineering-related extracurriculars were less likely to say they didn't fit in with the rest of the department. Female students may experience greater benefits from extracurricular activities. Women were significantly more likely to report participating in an engineering-related extracurricular. Nearly 60% of female respondents reported extracurricular involvement, compared to just a quarter of male respondents. Students who participate in the Society of Women Engineers were more likely to report having a role model in engineering that they look up to.
Twenty percent (20%) of student responses to the question of why their degree program made it more difficult for them to succeed specifically stated that they felt engineering was more difficult than other majors. Participation in pre-college engineering activities was not related to increased participation in engineering-related extracurriculars in college.
Future success: "I want to go into product design and development and try to make things more sustainable. I want to make them use less materials, energy, make them last longer, or make them out of more sustainable materials."-Female Mechanical Engineering Student." Male and female students identified financial and career benefits as advantages of studying engineering. More than 90% of respondents said that an engineering degree set them up to be more financially successful than other degrees and provided more career opportunities than other degrees. This pattern was also seen in qualitative responses. Career opportunities and financial success were significant motivators for declaring an engineering major. About 20% of responses to the question of why students chose to study engineering mentioned future career and/or financial benefits of studying engineering. As one male mechanical engineering student stated, "Engineers have jobs everywhere. I could take the degree anywhere in the world when I graduate." Though there was no differences between male and female respondents on whether a degree in engineering would help to "solve problems in the real world," female respondents were more likely to indicate the importance of real-world impact in openended responses. As one female civil engineering student stated, "I'm passionate about designing things that will help make people's lives better or a little easier." Only four males (6.8%) mentioned using their degree to impact or improve the world compared to 15 female students (just under 20% of all female respondents).
When asked about their future plans, 73.2% of respondents planned to begin working after graduating with their degree. Just 14.4% said they planned to continue on to graduate school. Even fewer, just 7%, mentioned the impact of their work when discussing future plans. Five of these seven students were female.
Female respondents were more likely to anticipate future challenges associated with sexism. They were much more likely to say that women had to work harder to succeed in engineering, and much less likely to say that sexism in engineering is "overexaggerated." The largest difference occurred among environmental and electrical engineering students; the smallest difference occurred among chemical and mechanical engineering students.

Fig. 4. Women have to work harder than men to succeed in engineering
This perceived imbalance in effort required was correlated with a decrease in confidence about individual ability to succeed in an engineering degree program at Colorado State University. Essentially, students who perceive more barriers to women succeeding in the engineering field are less likely to feel confident in their own ability to succeed, despite seeing no gender differences on either measure.
Male respondents were more likely to somewhat agree or strongly agree that sexism in engineering is overexaggerated. This measure was not correlated with insecurity around individual abilities, or with variations in feelings of belonging or "fitting in" with their degree program.

Implications for Women in Engineering
Overall, few gender differences existed in the data set. Women expressed more pride in their engineering major, but male and female respondents did not differ in their perhttp://www.i-jep.org ception of financial rewards or career opportunities, in their assessment of the availability of support within their department, in their "grit," or in their confidence in their ability to succeed within the department.
Gender differences did emerge when students were asked to assess challenges facing women in engineering. Female students were significantly more likely to say that women have to work harder to succeed, and less likely to say that sexism in engineering was "overexaggerated." Female students were also more likely to highlight the desire for their work to have impact through helping individuals or addressing larger social problems like poverty and inequality.
Finally, female students were much more likely to participate in engineering-related extracurricular activities. Analysis on participation in one organization, the Society for Women Engineers, found a significant increase in access to role models in engineering among participants.

Discussion
The results detailed above suggest an important role for extracurricular programs like the Rocky Mountain Scholar Program that link students to peers and role models and demonstrate the impact of engineering concepts.

Creating learning and support networks
Extracurricular activities appeared to be an important source of social support for students, particularly for female respondents. Despite studies that suggest women in engineering programs experience social isolation, our studies identified no gaps between male and female engineering majors at CSU in social support. There was a difference, however, in participation in engineering extracurricular activities. These extracurricular activities may be providing social support networks for female students, and ultimately mitigating social isolation among female engineering majors. Extracurricular activities like the Society of Women Engineers also facilitated connections between female students and role models that those students may not otherwise interact with in courses or other settings.
Programs like the RMSP provide both peer-to-peer and student-to-faculty connections, helping students identify role models, sources of knowledge, sources of social support, and other resources.

Connecting knowledge and impact
Applied research by students provides an opportunity to build social and academic networks. It can also strengthen the connection between the time and effort required for coursework, and the future impacts of an engineering degree on pressing social problems like climate change and global health disparities. This suggests that participation in applied research may be particularly important for female engineering majors, who are more likely to highlight the importance of impact in their work.

Activating male allies
This research suggests an important role for programming and awareness for male allies in engineering departments. The disparity between male and female respondents in perceived sexism suggests that male students and professors could be made more aware of the ways that gender bias can impact classroom and workplace environments. The RMSP creates connections between students and existing faculty-many of whom are male-who can be activated as allies of female engineering students in the classroom, and in the college as a whole. The finding that respondents who felt that women had to work harder to succeed also felt more insecure about their own ability to succeed in their degree program suggests that creating a more supportive environment for female students could also improve outcomes for male students.

Limitations and Challenges
A lack of participation among RMSP students makes it difficult to align broader trends in the College of Engineering at CSU with trends among the subgroup of students in the program. More research is needed to understand how RMSP programs experience social support.
In addition, this research relied on a binary classification of gender as "male" or "female." Respondents were given the option to self-identify their gender, and three respondents (2%) identified as transgender or agender. These responses were included in qualitative analysis, and in descriptive statistics, but, due to the small sample size, were removed from comparative analyses of responses based on gender. More research is needed to understand the experiences of transgender and agender students in engineering.

Future Research and Interventions
The findings detailed above provide a foundation to aid in the development of future research on social support and isolation in engineering programs, as well as the role of undergraduate research experiences for connecting students and bridging the divide between course material and "real-world" impact.
Future research should explore the importance of applied undergraduate research experiences like the RMSP on persistence and performance in college. In addition, a deeper understanding of the impact of the RMSP on the creation and evolution of student networks, including learning networks, peer support networks, and mentoring networks, is needed. This can be supported by further exploration on the role of extracurriculars in providing social support, as well as the role of extracurricular activities for female engineering students.
Social isolation between engineering students and their peers in other degree programs should also be more closely studied. This can be accomplished by examining the impact of students' social isolation from their peers in other degree programs and by asking engineering students if this isolation is because they believe their peers "don't understand" their course requirements or workload. In addition, the use of social media and social networking platforms to mitigate isolation should be explored [19].
Furthermore, future research should explore how narratives in engineering orientations and courses in higher education shape the way that students interact with their peers within and outside of engineering degree programs and potentially in K-12 and informal settings [20], [21]. More specifically, where does the idea emanate that an engineering major sets you apart from your peers? How might these ideas and perceptions affect the support networks of students while in school? After graduation? And if support and social networks become more limited, what is the effect on knowledge generation and creativity in the engineering field? In addition, from a structural perspective, how might degree or course requirements change the support networks of students?
Lastly, our findings detailed a significant disparity in perceptions of sexism between male and female students. Thus, an important question for future exploration is, how can male professors and male students become allies for women in engineering programs?

Conclusion
The RMSP provides experiences for engineering majors that tap into important, research-based mechanisms for student success. This research explored the experiences and perceptions of male and female engineering students through surveys that measured individual motivation and resilience, as well as social support mechanisms and perceived sexism to investigate the relationship between individual characteristics and institutional context.
The differences in responses between male and female students occurred, not on measures of resilience or motivation, but on questions regarding the institutional context within which they are studying. Female respondents suggested extracurricular activities and meaningful research as important components of a successful engineering experience. This points to an important role for the RMSP in creating social networks among students and faculty, linking students to the broader impacts of their work, and ultimately improving the undergraduate experience of underrepresented groups in STEM programs.
The RMSP is continuing and expanding on the CSU campus. Moving forward the program plans to continue to review existing studies [22] as well as gather key stakeholders, including students and faculty, to develop a research and evaluation plan based on the findings of this study that contributes to the success of engineering students in Colorado and across the nation.