Student Perspectives on the Learning Resources in an Active, Blended and Collaborative (ABC) Pedagogical Environment

Digital technologies have become central to higher education teaching and learning practices. This paper presents the results of a qualitative study of students’ experiences within a resource-rich and digitally-driven pedagogical system in a second-year engineering dynamics course. Course instructors created an Active, Blended, and Collaborative (ABC) learning environment providing students with multiple learning resources. The resources included a hybrid lecturebook (a combination of textbook and workbook), an online discussion forum, a large repository of online videos, and a tutorial room staffed by teaching assistants. A thematic analysis of semi-structured interview transcripts characterized how students perceived the utility of the available resources, especially given the complex integration of pedagogies and resources available in person and across the technology platform. The findings revealed that students valued the blended structure because it provided them with both immediate and asynchronous access to the learning resources. Students used the resources to help them solve homework problems, study for exams, and improve their conceptual understanding in the course. Students reported engaging with their peers to receive feedback constantly throughout out the course. The results highlight how the learning resources were aligned with each other and in-class pedagogies, and they worked in tandem to support students’ academic success. The results from this paper contribute to the literature on student perceptions of technology-mediated learning environments in core engineering courses. Keywords–Active learning, blended learning, collaborative learning, student


Introduction
Pedagogical reforms that leverage technology are one of the ways universities and instructors work to transform undergraduate engineering courses. Universities cater to large and diverse undergraduate populations, and it has become critical for engineering faculty to include technology-based teaching practices in their courses [1]. Teachers can inform their instructional design by understanding how students engage with and utilize their learning resources [2]. Prior studies have focused on the perspectives of the instructors who use technology tools for teaching [3], but it is also necessary for instructors to acknowledge the adjustments that students make while transitioning from a traditional lecture to a blended or online format. Developing this understanding will enable instructors to develop student-friendly, technology-based resources that support student success.
In 2010, instructors from the School of Mechanical Engineering at Purdue University began to develop an Active, Blended, and Collaborative (ABC) learning environment in a core Dynamics course typically taken by students in their secondyear [4]. This ABC learning environment, known as Freeform, was designed to better support students and improve both their conceptual knowledge and problem-solving skills. Instructors teaching in the Freeform environment utilized active and collaborative learning strategies in an attempt to increase students' engagement in the classroom. The students were provided with multiple blended learning resources aligning instructional activities, via a digital platform, across in-class and out-of-class time. These included a newly designed lecturebook for the course, a repository of videos demonstrating problem-solving for lecture examples and homework questions, and a discussion forum enabling asynchronous collaboration. The students had access to office hours and a tutorial room staffed by teaching assistants for additional support. More detailed information about the course design and range of learning resources is provided later in the background section (see Section 2.2). After several years of materials development, the Freeform paradigm became the de facto environment for all the Dynamics classes in the department. Longitudinal regression analyses determined that since implementing the Freeform learning environment, the rate at which students received the non-passing D, F, and W (withdrawal) grades (the 'DFW' rate) decreased significantly [5].
The drop in DFW rate provided statistical evidence that students were performing better overall in the Freeform environment. However, we identified the need to analyze the students' engagement with the learning materials in order to better understand how the alignment of in-class pedagogies and blended content contributed to increased student success. More specifically, we sought a qualitative understanding how and why students used the blended resources available in this complex ABC learning environment. The results from this analysis augment existing literature on student perspectives about integrating ABC learning resources in a learning environment, and they inform future STEM educators as they consider transformations of their undergraduate courses. In this paper, we address two research questions: • RQ1: In what ways do students characterize the utility of the ABC learning resources in the course?
• RQ2: In what ways do students perceive the resources available in an ABC learning environment to work in tandem with each other to impact their experience in the course?
To fully understand the value of the ABC learning resources from the students' perspectives, we chose a qualitative approach. Our aim was to understand in detail their perceptions of the ways in which the ABC learning environment enhanced their success in the course.

Benefits and challenges of ABC learning environments
There is extensive literature detailing how Active, Blended, and Collaborative learning resources individually help students learn better in university engineering/STEM courses, although research on how such resources work in tandem for students is more sparse. Freeman's meta-analysis of 225 studies reported that students in an active learning classroom had six percentage point higher examination scores as compared to those in a traditional lecture [6]. Incorporating active learning techniques in the classroom provides students the opportunity to build creative and critical thinking skills [7] and reflect more deeply on the course content [8].
A meta-analysis of students in a blended learning environment, which combines face-to-face and online instruction, reported that students in both the K-12 and higher education settings performed better relative to strictly face-to-face instruction [9] [10]. Blended learning helps build the critical thinking and problem-solving skills of students [11], thus catering to large and diverse student groups who may, in the case of video content, have different viewing habits [12]. Students in blended classrooms have exhibited sustained interest leading to cognitive engagement, and they have demonstrated increased peer-peer and peer-instructor interaction [13]. Studies have reported that embedding blended learning activities in the students' experience positively impacted their performance in the course and decreased student attrition [14]. Blended learning activities have also improved the engagement of students in a course [15] and fostered a positive attitude towards collaborative learning [16].
Collaborative learning has been shown to have a positive academic and social impact on students. Studies report that encouraging collaborative learning inside of a classroom increased student engagement, performance, and retention in a course [17]. Other benefits of learning in a collaborative environment include improved interpersonal interactions, self-esteem, and student attitude towards one another [18] [19]. Similar results were reported by a meta-analysis of collaborative learning studies in undergraduate STEM courses concerning student performance and persistence [20].
Despite the positive outcomes noted above, each teaching and learning practice places demands and constraints on both the instructor and students. Implementing active learning might come with challenges that include restructuring an existing lecture-heavy course, the creation of new learning materials, and the need to develop expertise in active learning methodologies [21]. Similarly, faculty need to be able to manage their time effectively in the class, as implementing active or collaborative learning might require a change in how time is spent in the classroom.
Designing blended learning environments has challenges which include facilitating interaction among students, building connections between the students and their instructor, and fostering an effective learning climate [22]- [24]. Blended learning has the potential to increase the distance between a faculty member and student (a consequence of the reduction in the number of in-person classroom sessions or increased dependence on the blended resources), and it is essential for instructors to find ways to monitor the learning of the students and help them maintain their motivation. Another challenge of blended learning is the structure of the learning environment. Prior studies reported that the inclusion of multiple blended components in a single course without proper alignment (to pedagogies or assessments) can increase the cognitive load on students [16]. Students collaborating on blended learning platforms have reported a lack of trust while working with peers or team members, especially when they have insufficiently frequent face-to-face interactions [25]. It is therefore important to ensure that the structure of a course incorporating blended learning aligns well with the other blended and non-blended components of the course.
While most of the aforementioned studies reported the benefits and challenges of the individual dimensions of ABC learning environments or a combination of active and collaborative learning [26] or blended and collaborative learning [27] [28] [25], new challenges may arise when one simultaneously combines the ABC learning environments, making their implementation complex. This may explain the lack of wide-spread development and simultaneous adoption of ABC learning environments in core STEM courses. To illustrate the paucity of combined ABC literature, we conducted a brief literature review using the key words ("active" AND "blended" AND "collaborative") OR ("blended" AND "collaborative") on Education Source (which lists full text education journals from early childhood to higher education), and the results revealed 42 studies out of which only 20 focused on higher education. While there are a number of precedents for applying such a complex learning environment to introductory courses, such as introductory physics [29] [30], we found no studies available for core second-or third-year engineering courses that integrate ABC learning environments.

Integration of ABC learning resources in a single course
Freeform integrated multiple resources that facilitate Active, Blended, and Collaborative learning into one course in an attempt to provide students with resources that are aligned to complement each other. Figure 1 illustrates the core functions of the resources in a Venn diagram format. We detail here each of the resources and the way in which each was utilized in the class as an Active, Blended, and/or Collaborative tool. Students enrolled in the course had access to a unique course textbook, the Dynamics "lecturebook", that was designed to facilitate active learning [4]. The lecturebook contained streamlined descriptions of Dynamics concepts, example problems, and additional extensive white space for students to solve the problems and take notes. These problems were the same problems that the instructor solved during the class, and the white space in the lecturebook enabled and students to take active notes in class [31]. Instructors also enabled active learning by asking, and allowing students to ask, frequent questions in the classroom. A semester long classroom observation of the Freeform classroom revealed that over 10% of the classroom time (over the entire semester) was spent on asking and answering contentrelated questions [32].
To promote collaborative learning, students were given team-based problems and group quizzes to solve in the classroom. In the entire semester, the number of group quizzes provided to the students ranged from 4-10 (the number varied across sections depending on the instructor), and they contributed to around 5% of the course grade. Students in the class were also encouraged to work with their peers to practice problem-solving in the class, as the collaboration allowed easy and convenient knowledge sharing [33]. To extend the collaborative learning environment outside of the classroom, the students were provided access to a course website with an online discussion forum. The discussion forum (colloquially called the course "blog", and referred to as such in the rest of this paper) on the website was both a blended and collaborative learning resource and was intended to enable student interaction with their peers outside of the classroom. We also note that many of the resources could be used in ways not indicated on the figure; for instance, the lecture example videos (a core blended resource) could be used collaboratively to spark discussion among students in a study group. iJEP -Vol. 10, No. 2, 2020 The course website also contained other blended resources such as multiple videos of solutions to the problems in the lecturebook, video solutions to homework problems, and "Visualizing Mechanics" videos that provide a visual live-action demonstration of Dynamics concepts. homework assignment contained its own comment thread within which students could discuss potential solutions to the problem. The blog was organized with convenient navigation features to enable quick access to course resources.
Students in Dynamics submitted homework assignments three times per week, and the homework contributed to 17% of the class grade. Apart from the homework, students completed three mid-term examinations, and a final exam at the end of the semester. While the homework solution videos were available to students after they receive their graded homework, the lecture example videos were available throughout the duration of the course. Prior research suggests that the videos could allow the students in the blended classroom to feel better supported during a course [34] and to access problem-solving demonstrations as necessary. The lecture example videos were solved examples of the problems in the lecturebook, which enabled the students to revisit the instructor's problem-solving techniques at their convenience. The homework solution videos provided the students with detailed homework solutions. Students were provided with office hours to meet the instructor, and a tutorial room staffed by TAs who were available more than forty hours per week. Shown at the intersection of ABC elements in Figure 2, the course blog organized discussion threads around homework problems, and students were encouraged to help each other by leaving comments on the homework questions. Students were provided with five bonus points (which was a fraction of the value of one homework assignment) as an incentive to use the online course blog. To receive the five bonus points, the students were asked to post at least five comments on the blog. The online resources in the course provided students with multiple avenues to access and review the course content, collaborate with their peers, and review solutions to dynamics problems.

2.3
The student perception of ABC learning resources Few studies have examined student perceptions of their learning resources in active and blended courses [35], and prior results are mixed. A study on the strengths and challenges of active learning revealed that students believed active learning improves their understanding in the class and provides them more practice as compared to a traditional lecture format [21]. Another study found that students believed that active learning positively impacted their learning [36]. On the other hand, students enrolled in a blended course reported developing negative perceptions of the course when the online and face-to-face course components were misaligned [37].
Prior research suggests that a complex learning environment could interface with students' backgrounds and expectations in a number of positive or negative ways. Gedik et al. examined a blended course, which provided students with online reading materials and a discussion forum, and reported that students perceived the blended learning resources to improve their engagement with the course content and to aid in clarifying confusing points in the course [38]. However, students also reported that the course combined the workload from face-to-face and online instruction and thus increased the total time commitment needed for the course. The perceived increase in workload was attributed to the need for students to attend the face-to-face classroom sessions and participate on the online forum.
Students studying in a collaborative learning environment believed that they were more empowered over their learning as compared to traditional instruction [39]. However, students believed that collaborative learning inside of the classroom introduced multiple challenges, as not all of the students participated equally in discussions [28]. Students felt it was important for the instructor to enforce course policies or mechanisms that would ensure equal participation from all of the members of a group and contribute to group discussions.
Taken together, studies like these highlight the need for more systematic study of the student experience in resource-rich learning environments and led directly to our research questions. The Freeform learning environment provides students with a learning experience that intentionally integrates ABC materials into a resource-rich environment that provides students significant freedom in how they approach the course (and especially course-related help seeking).

Data collection
Participants. In the spring of 2016, 384 students were enrolled in four sections of the course, each taught by a different instructor. Enrolled students belonged to multiple racial/ethnic groups, although our institution's registrar collects race/ethnicity data only for domestic students, and all international students are identified together as 'international'. While white domestic students represented the majority of the sample population, international students represented more than 20% of the class. Female students represented 25% of the class. The demographics of students enrolled in the course were representative of the academic department in which this study took place (with the exception of women, who were over-represented in the enrollment during this semester).
This research was driven by semi-structured interviews with 28 students who took the Dynamics course in the spring of 2016. The interviews were conducted during the last week of the semester and the students who participated were provided with a gift card as an incentive. The interview participants were recruited as a stratified sample based on their incoming GPA, international status, and the course section in which they were enrolled. We used the students' incoming GPA as a proxy for a number of factors that prior work suggests are related to academic performance, help-seeking behaviors, and engagement with active, blended, and collaborative activities [40], [41]. Other research suggested that students with high and low prior performance may seek help, self-regulate their learning, and utilize the resources in the homeworksolving process in different ways [42]. We therefore stratified students into a low-GPA/high-GPA (split at the median GPA) to ensure a sufficient number of interviewees in each stratum.
Prior literature suggested that international students' experiences in US classrooms differed from their domestic counterparts while they adapted to the active learning pedagogy that is more prevalent in American classrooms [43]. International students who are not proficient in speaking English have reported challenges with the language of instruction, which could influence the nature and extent of their interactions with the instructors and peers, both inside and outside the classroom [44]. We therefore stratified by domestic and international students.
Prior studies also suggest that the instructor's role in the clear articulation of course expectations and available resources is crucial to students' success in the course [2]. We therefore hypothesized that instructors in different sections would be an important mediator to facilitate active and collaborative learning in the classroom and to direct students' usage of the online learning resources. We sampled from all four of the sections during this semester. The stratification was further informed by the results of a prior quantitative study from the same project, which reported that their incoming GPA, international status, and the section were significant predictors of the grades students received in the course [45].
Our goal was to recruit interview participants according to our stratification variables (GPA high/low, nationality domestic/international, 4 sections of the course), including 2 students from each stratum, which would have resulted in a total of 32 participants. Despite our recruiting efforts, this goal was not achievable and the final interview sample included 28 students distributed across our stratification variables as shown in Table 1. Interview protocol: The interview questions spanned a wide range of topics including the students' perception of the Freeform learning environment, usage of and feedback on the available resources, and comparison of the ABC learning environment to past lecture-based experiences. All of the interviews were audio recorded, transcribed using a third-party vendor, and then analyzed by our research team.

Data analysis
Codebook development: The first step in the data analysis was to develop a codebook. Given that a significant amount of data was collected on a broad range of interview topics, it was necessary to organize the data using different codes before further analysis. Multiple steps were taken to develop the codebook. Two researchers started to first develop the codebook through structural coding, which helped to label and index the data and allowed the researchers to access the most relevant passages in a larger dataset [46]. Because the interviews were semi-structured and addressed the high-level research questions of the project, the initial codes that were generated aligned with the research questions related to the overall project (not necessarily the RQs related to the study reported here). For example, one of the research questions of the overall project is, "What affective, attitudinal, and emotional perceptions of the Freeform environment can we observe from students and faculty, and how do these perceptions compare to those of more traditionally-structured classes?", and the codebook included the code "attitude" to categorize these data. In the next step, researchers examined emotions that were detectable in the transcripts based on a student's reported feelings and emotions during the course [47] [48]. Emotions observed in the interviews were mentioned in the inclusion and exclusion criteria of the codebook to help the researcher identify the proper code. The final step involved looking at the values and beliefs about the (Freeform) learning environment as expressed by students during the interview [49] [50]. Table 2 provides an example for the code "attitude" from the codebook that we developed.
Twenty-six codes emerged during the coding process, and they remained consistent across the multiple interviews that were coded. As noted in Table 2, a definition, short description, inclusion and exclusion criteria, and an example were provided with each code in the codebook [52]. To establish the reliability of the codebook, two researchers used the codebook to code the same interviews separately using NVivo (licensed software for qualitative data analysis) and then manually checked for interrater reliability in the coded data [53]. Reliability was calculated by the ratio of number of agreements to number of agreements and disagreements. A reliability of over ninety percent was achieved by the researchers, which met an acceptable level [54]. After this development and validation stage, we used the codebook to examine the twenty-eight interviews included in this study. Table 2.
Structure of the codebook developed to organize data (one of twenty-six codes).

Code Description Inclusion / Exclusion Criteria Example
Attitude (critical/ endorsing) The students' attitude towards the Freeform, environment, resources, and classroom.
An attitude is the way someone thinks and feels about themselves, another person, thing, or idea. Think of attitudes as evaluative perceptions and sets of cumulative reactions, reflecting the beliefs people learn through time [51].
Include attitudes that are both critical and endorsing. Thematic analysis: To analyze the interviews, a thematic analysis was conducted on the coded data [55]. The thematic analysis was conducted on the portion of data that was organized under the codes for each of the ABC resources: Freeform instructors, lecturebook, lecture example videos, Visualizing Mechanics videos, homework solution videos, blog, office hours, and tutorial rooms. Data coded under the codes attitude, beliefs, emotion, and value were also included in the thematic analysis. We selected these codes to confine our analysis to the two research questions for this paper regarding the students' perceived value and the integration of the resources in the Freeform learning environment. We defined a 'theme' to be a common experience expressed in interviews by at least five students. After observing the themes that emerged from the data under each code, we checked for overlapping themes across the different codes, and coalesced overlapping themes into a single theme as appropriate. We aligned theme names with the two research questions in this study. For example, the theme "convenience" was observed as we individually analyzed the data coded under each of the different learning resources. The theme "convenience" was therefore identified, and renamed as "Convenience while understanding Dynamics concepts, solving homework problems, and studying for tests". Each of the themes along with relevant quotes from the students is detailed in the next section. Table 3 presents the themes based on how the students' characterized the utility of the Freeform resources. An illustrative quote is provided for each theme from the dataset to give additional context. Immediate and asynchronous access to learning resources: Students appreciated that the blended and collaborative learning resources were asynchronously and immediately accessible to them. The course website provided students with an asynchronous course blog to interact with each other outside of the classroom. The blog was common to students from all of the sections of the Dynamics course and was designed to increase the students' interaction with their peers beyond their classroom. Most of the students interviewed found the blog to be helpful as its asynchronous nature made it accessible anytime based on their convenience. The asynchronous nature of the blog helped students whose homework completion occurred over a longer time scale, because they could periodically receive feedback and support via the course blog until the submission deadline (student 19 on Table 4). Our data suggests that the asynchronicity was not useful for students who preferred to complete their homework in a single sitting, or who completed the homework early in the submission window (the majority of the blog comments were typically posted closer to the homework deadline). More than half of students saw value in accessing the blog even when the tutorial room was open and viewed the blog as a useful and convenient alternative that was accessible all of the time (students 26 and 20 on Table  4). Students talked about how the blog allowed them to work on the homework around their schedule instead of the tutorial room availability. The tutorial rooms were accessible to students on 5-6 days of the week and were open 8 -11 hours a day, but they were not open continually and might have been inconvenient to students who required assistance during late hours (student 5 on Table 4). The lecture example videos were available to the students online throughout the semester, providing them with immediate access to worked examples. All of the lecture example videos were posted online at the start of the semester, and the asynchronous availability made these videos beneficial to the students, especially while working on homework or studying for exams (student 5 on Table 4). Student dependence on the instructor and TAs (who were available only at specific times and in specific locations) would perhaps increase if such resources were not available (student 13 on Table 4). The videos available in the Freeform learning environment aided the students by reducing their dependence on the instructional staff outside of the classroom while still providing instructor-created material (student 27 on Table 4). Both the immediacy and asynchronicity of multiple Freeform resources were perceived as important characteristics of the learning environment.

Students' characterization of the utility of ABC learning resources (RQ1)
Relevance of the resources to understanding dynamics concepts, solving homework problems, and studying for tests: The lecture example videos, homework solution videos, and the Visualizing Mechanic videos available to the students were strategically aligned to ensure that the resources were relevant to concepts and problems solved in the classroom. The online videos were avenues for the students to re-engage with the course content outside the classroom: "In the traditional class, it's like 50 minutes' class. You need to learn everything in that 50 minutes... If you want to learn more, you need to go to office hours or you need to face-to-face talk to the professor. With the online resources, you can go home and just watch the video if you don't understand" (Student 20). Many of the students who were interviewed mentioned having watched the videos to clarify the concepts taught in the class while they were completing their homework: "I usually jump right into it [homework problem]. If it seems like I'm misunderstanding something a little bit, I'll try to find a lecture example video similar to the question" (Student 8).
Student 6 reported that the videos helped refresh his memory and revisit the concepts exactly as they were taught in the class since the instructors demonstrated the same problems solved in the lecture example videos: "It's nice to supplement maybe something you missed in lecture taking notes with a video in this case of the problem worked out. It was nice to be able to refresh your memory or see the steps worked through again as opposed to only seeing it once in class". This might help students who regularly engage in note-taking and could not multitask by listening to the instructor at the same time. Student 4, who reported having a shorter attention span, saw value in having the online videos as a support outside the classroom: "I liked these videos more than any other class where you just have to pay attention every single second in class to make sure you get every detail, because I miss stuff a lot, no matter how hard I try to pay attention, so the videos helped me a lot." The opportunity to revisit the concepts taught in the class through the lecture example videos was valuable to students who had a busy schedule. The lecture example videos provided students with the flexibility to focus their attention on the course deliverables that they had that day: "If there is an exam coming up I give it more attention, and I stop paying attention in all the other classes. It helps me balance out I feel like with dynamics, even if I miss lectures or even if I'm too tired that day, I can go through the lecture slides [lecturebook] and lecture videos and understand what I missed" (Student 11). A similar circumstance was reported by Student 13 who had a lab right before the class and could not always focus completely during the class: "I think the main benefit of having online videos is that it allows people to study whenever they want. Especially whenever they're ready to learn because I have a two-hour lab before my [Dynamics]

classes on Wednesday and Friday and a lot of times I'd get out of that and be like I just can't learn right now".
Instructors teaching the Dynamics course sometimes had live demonstrations in the class to help students visualize Dynamics concepts about moving objects. The instructors also developed a series of Visualizing Mechanics videos to provide the students access to many more live action demonstrations of Dynamics concepts outside of the class. Student 9 mentioned that the Visualizing Mechanics videos helped him to mentally picture and conceptually understand certain Dynamics concepts:

"I used Visualizing Mechanics videos to get a better understanding of it [Dynamics concepts]. If you give me a schematic drawing and you expect me to like to understand the angular velocity I can't do that. I need to actually see it in action".
A few students mentioned that they got better at visualizing the Dynamics concepts as the course progressed: "I did [use the Visualizing Mechanics videos] at the beginning of the semester for some things. After the beginning, I feel like it was not as helpful because I had gotten better at visualizing that without the videos" (Student 7). However, most reported not utilizing the Visualizing Mechanic videos and the data extracts mentioned above represent the experience of only few of the students.
Multiple avenues to provide and receive feedback from peers: Students perceived that the computer-based and classroom resources in Freeform provided multiple opportunities to receive feedback inside and outside of the classroom. In the class, the students often worked in groups to complete the in-class quizzes. Students reported that working on in-class quizzes in groups helped them receive immediate feedback from their peers. This was helpful for the students especially while working on conceptual problems, as this helped them identify misconceptions early in their learning process: "It's really cool because a lot of the conceptual components of the course are really confusing and easy to mix up. So, it's helpful to be able to work on that and be able to immediately get feedback [from peers] on you know, oh you're doing it wrong. I guess that's really helpful" (Student 5).
Outside of the class, many of the students believed that working with their peers allowed them to ask questions when necessary and find efficient problem-solving strategies: "I would use my peers by asking questions about certain parts on how to do certain little tricks in each problem, and it was nice" (Student 6). More than half of the students interviewed mentioned reaching out to their peers outside of class to get feedback and found it helpful to identify the mistakes that they made: "This [working with peers] was especially helpful because we could teach our peers when we knew what was wrong and we could learn from our peers when we didn't know what was wrong" (Student 6). Students therefore valued working with their peers and attributed their improved understanding to it: "I found often you can actually learn a lot more by talking about a subject than just by simply reading it or going over something" (Student 3).
The course blog provided students an additional avenue to collaborate with all of their peers and receive feedback outside of the class. The students mentioned using the blog to provide and receive help while they solved the homework problems: "I'll try the problem. If I can't get through it on my own, then I'll check the various blog comments that people have left, and then pitch in if I can, and then go back and just finish up the problem" (Student 7). Student 27 reported similar practices when solving the homework problems and wanted to confirm her answers with others: "if I just feel a little bit uneasy about my answer, checking the blog and seeing what other people have asked and answered, cause sometimes I like to just confirm that I'm heading in the right direction". This attitude of working and helping each other contributed to the development of a collaborative learning environment among the students: "Well our professor already has the blog for all the four sections together. Obviously, there are going to be some people without questions and some people who want to help out too, I think that promotes a collaborative learning environment" (Student 19). This appreciation and willingness of students to collaborate with their peers was common among students in this academic program as reported in a different study focusing on departmental culture [56].
The students also found the feedback from their peers to be sometimes more meaningful than the feedback from the instructor: "Especially the comments on the homework problems I thought that was really helpful because sometimes a professor knows way more than you and if they try to help you, they don't always get at what level [novice, intermediate or expert] you need the help. But the other students can understand because they've done the same class as you've done, so they have the same base of knowledge" (Student 4). This student perception about peer feedback is aligned with prior literature, which reports that peers, given their recent experiences with learning the content, are better able to recognize challenges with the subject matter as compared to the instructor [57]. Feedback from peers is also considered to be more comprehensible as they are often able to speak the 'same language' as the students (i.e., expressed at the right level of expertise and potentially using informal, rather than highly technical, vocabulary). This allows students to provide more relevant and immediately applicable feedback when compared to the instructor.

Integration of ABC learning resources as perceived by students (RQ2)
In Table 4, we list the three themes that describe how students perceived the ABC resources to work in tandem with one another. These are the three most commonly mentioned combinations of resources that students used in conjunction with each other, and they span the domains shown on the Venn diagram of Figure 1. Table 5.
Common learning resource integration strategies. The specific ABC components are listed in parenthesis.

Resources Quote
Instructor "I would first read the section before the problems where all the material is introduced, and then I would look at the problem and solve as much as I can and refer it back to homework solutions that I can find, and then go on the blog to see and look for solutions even if I don't have ideas how to solve them." (Student 12)

Instructor and lecture example videos:
The students perceived that the lecture example videos online were well integrated with the lecturebook problems introduced during class meetings, whether the lecturebook was used in a guided-solution format (the instructor solves the problem) or an active learning format (the students solve the problem). The lecture example videos were explanations of the Dynamics problems available in the lecturebook. While the basic problem-solving strategies were taught by the instructor in the class, the lecture example videos sometimes provided alternate approaches to the problems, as the videos were designed by veteran instructors who authored the lecturebook: "it's just basically stuff we did in class but explained differently" (Student 15). The students believed that having the lecture example videos was like having a virtual instructor outside the classroom, but in this case accessible all of the time: "They [lecture-example videos] are super, super, helpful. It's pretty much just having the instructor but outside the lecture" (Student 9). The same was reiterated by Student 17: "The lecture example videos supplement whatever happens in lecture". The students also mentioned the use of the lecture example videos as an alternate instructor when seeking help: "Like if I don't understand something, I'll basically just go to like the second teacher, which is just the lecture example videos" (Student 15).
Lecture example videos and the lecturebook: The combination of the lecturebook and lecture example videos was commonly used by students as they completed their homework. When students encountered challenges while completing the homework problems, they referred to the lecturebook to find a similar problem. Students then used the lecture example videos as they provided solutions to problems taken directly from the lecturebook: "for a lot of the homework if I'm not understanding something, I'll look for a problem in the lecturebook. I'll look for a problem similar to the homework problem and then there are videos explaining every single lecture problem, so I'll just watch that video and see if I can get a better understanding" (Student 22). Student 12 reported that the combination of lecturebook and lecture example videos improved the efficiency in his learning process: "Having this special textbook that our professors actually wrote and lecture examples online, it helped me learn more efficiently". This is a consequence of the hybrid nature of the lecturebook, where the content in the lecturebook is integrated with the lecture example videos in an intentional and well-aligned way.
Lecturebook, lecture-example videos, course blog and peers: The students reported using multiple learning resources in tandem with each other especially while working on their homework. If they had difficulty while solving a homework problem, students would first resort to the lecturebook and lecture example videos, then turn to their peers via the blog or in person, and finally visit the tutorial room or meet the instructor for further assistance: "I usually look at the problem, then I'll go into the lecturebook, itself. I'll see if there's a problem in the lecturebook that's similar to that I'll watch the lecture video online for that example. Once I do that I'll try to actually solve the problem and if I run into a road block, I'll go to the tutorial room and if the tutorial room isn't really helpful, then I'll go ask the instructor" (Student 9). Student 13 who used the blog said: "I would first read the section before the problems where all the material is introduced, and then I would look at the problem, solve as much as I can. I would then go on the blog to look for solutions". We observed that students reached out to the instructor and tutorial rooms as a last resort. While this help-seeking behavioral pattern was found to be common among students in the department [56], the instructor-curated resources available to the students are specific to the Freeform learning environment and support their help-seeking pattern for this course. Thus, providing a combination of ABC learning resources was perceived by a majority of the interviewees as beneficial and aligned with the multi-faceted problemsolving strategy that they described.
While students did not mention using the homework solution videos and Visualizing Mechanic videos in combination with the other learning resources, they used one or more of the video sets (lecture examples, homework solutions, and Visualizing Mechanics) as standalone resources to help them complete homework assignments and prepare for other assessments such as quizzes or exams. Not all students reported using all three video sets. As mentioned in Section 4.1.2, the Visualizing Mechanics videos were helpful to students who needed help in visualizing live action demonstrations of Dynamics concepts. Students mentioned specifically using the homework solution videos after receiving their homework scores to clarify some of the mistakes they might have made while submitting it. For example, Student 3 mentioned: "The homework solution videos, I look at them occasionally because I used them only if I didn't understand homework completely or if I wanted to go back and look at the homework and see what I did wrong or what I didn't get". So, while students used the lecturebook, lecture example videos, course blog, and peers in tandem to solve the homework problems, they used the homework solution videos in a more narrow scope to clarify errors noted in their graded homework assignments and to prepare for exams.

Discussion
The thematic analysis of the interviews revealed the predominant perceptions of utility of the Freeform resources from the students' perspective. The alignment of the digital learning resources with the content taught in the classroom and their asynchronous and immediate access provided convenient and effective support to students in their learning process. The integrated functioning of the learning resources in tandem with each other guided students in their problem-solving strategies especially while working on homework problems. Analysis of the student interviews highlighted the role of the instructor in each student's effective utilization of the learning resources. In this section, we discuss how the blended and collaborative resources helped the students to be efficient in their learning process, and how the instructor was an important mediator to direct student usage of the blended resources.

Efficiency in the learning process
Efficiency is defined in this paper as instances when a student perceived that they completed a particular task faster than the usual time required or when they could improve their conceptual understanding or the problem-solving strategies in the stipulated time. The lecture example and Visualizing Mechanics videos allowed the students to revisit the concepts taught in the class multiple times, which helps in transferring the knowledge to long-term memory and improves the retention of that knowledge [58]. The students used the homework solution videos to not only check the answers to their homework problems but also optimize their problem-solving strategies: "With [Dynamics], you could work through the problem and then watch the solution and be like, Oh, that would have been way quicker. I could have done this or that to save time or do it in a better way" (Student 7). This way, students reported that the learning resources helped them to not only achieve better performance but constantly improve their problem-solving process to make it more efficient. We do not have independent verification of this process improvement, and people in general are notoriously poor at self-assessing their competence in certain domains [59]. Nonetheless, this sentiment was reported by students.
The availability of multiple learning resources, especially the online resources, allowed the students to organize their studying as around their overall schedule or according to their learning preferences [60]. This flexibility may allow the students to be more self-directed in their learning process [61] and work collaboratively anytime based on their convenience. Students often could be distracted and miss what was being taught in the class. Having additional support for the students outside of the class helped them because they had reinforcement of the content presented in class [38]. Moreover, the students had direct and near-real-time opportunities to receive feedback from their peers, another affordance of blended learning solutions that benefits students [62] [63]. Students mentioned in their interviews that the support available outside of the class was instantaneous and allowed them to be more efficient with time. The students reported use of their peers on the blog and offline to provide and receive feedback throughout the course. Students frequently mentioned that helping their peers often helped them to identify their own misconceptions in the process.

Instructor as a mediator for directing student resource usage
It is important for instructors to encourage students to utilize the learning resources from the start of the course and make sure that the resources are convenient to access and utilize. We observed that some of the students needed to first use the learning resources in order to recognize their benefits and build an affinity towards them, which aligns with prior studies [64]. For example, Student 5, despite seeing the benefit, did not use the blog: "I mean the blog I personally really didn't use it at all, but I think it would be really helpful if it was used, and again I've never seen that in another class really". Prior literature suggests that the lack of utilization can be attributed to multiple reasons, including a student's perceived ease of using the resource [65], cultural barriers such as the language, and technical issues [38].
We observed that providing an initial incentive was a catalyst for encouraging some students to use resources at the start, which later continued as they were convinced about the benefits of utilizing the available resources: "Well there was like five bonus points offered, so I definitely started. I probably wouldn't have done it honestly if there weren't those points, but once I did it I thought it was a good resource" (Student 13). Student 24 reiterated the same: "I definitely think it [the comments on blog] was posted by people who want points since they offer points if you posted five comments. Then sure, some people just got into the habit of posting once they did those five comments. They found it convenient to continue doing that [interact over the blog]".
Rogers's theory of diffusion characterizes individuals into five different categories based on the time they took to adopt a particular innovation or technology: innovators, early adopters, early majority, late majority, and laggards [66]. When we compare the students' behavior and perceptions to the different categories, the students in the course might fall under the early majority, late majority, and the laggard's categories. Rogers revealed that students in all of these categories need success stories or the evidence that shows the usage of the blog is beneficial to them in their learning process. This is consistent with other studies that suggested that the instructor should make the value of using online learning resources explicit at the start to the students, especially if the instructor wishes the students to participate actively in online discussions [67]. The instructor therefore has a critical role as a mediator of the students' buy-in to utilize learning resources in the course.

Implications for engineering education practice
As students in their sophomore year are reported to struggle with increased academic expectations and an intensified curriculum [68], instructors are looking for evidence-based teaching practices that could support student learning and improve their retention in core engineering courses. The Dynamics course considered in this study is widely regarded as one the most difficult courses that engineering students take during their undergraduate study [41]. The ABC learning environment presented in this study was applied to the Dynamics course, which resulted in a drop at which students received the non-passing D, F, and W grades. The results from this study provide insights into how students characterized the utility of a range of active, blended, and collaborative learning resources made available in the learning environment, and how they used them to navigate some of the expectations in a challenging core engineering course. In particular, we suspect that the tight alignment of the lecturebook (with its many examples and ample white space) serves as the inclass launching pad for students to explore the blended resources. The in-class pedagogies leverage the blended resources in powerful ways as articulated by students included in this study. Our results provide instructors with an evidence-based template for how in-class actions (lecturebook and pedagogies) and blended resources (blog, videos) can be thoughtfully merged for to support student academic achievement.

Limitations and Future Research Opportunities
This research was conducted within a specific context and student population, and as such its outcomes are likely not transferrable to other environments. For instance, our prior work [69] suggested that in a smaller-enrollment context, student resource usage was, unsurprisingly, different than that reported for the large-enrollment course we studied. Moreover, the setting for the study promotes a highly collaborative undergraduate student culture [70], and students who experience difficult organizational culture at other institutions might illustrate different resource usage patterns. Our conclusions also represent the perspectives of a small number of students, and there are opportunities to revisit our stratification variables and our population sampling to gain a more comprehensive dataset. With a larger sample, we may be able to apply this qualitative approach to understanding differences in experiences across the stratification categories; we have reported quantitative results along these lines recently [71] although there is ample opportunity to further advance that line of research. While we are confident in the conclusions of this qualitative study, there is clearly an opportunity to enrich our understanding through wider-scale data collection.
The results of this work are further limited by the small subset of instructors represented in the dataset. Four distinct instructors taught the sections of the course from which our participants were recruited, and each of the four executed the course in ways that reflect their underlying preferences about teaching -despite their adherence to active learning strategies in the classroom and the ABC paradigm in general. Some instructors may have emphasized certain blended resource (the blog, or the lecture example videos) more than others, which in turn could drive student usage of those resources. We do not have a systematic accounting for how these instructor preferences or in-class actions may have influenced the experiences reported by our participants, but we acknowledge their potential role in shaping the outcomes of this work.

Conclusion
Our results provided insights into students' perceptions of the resources available in an Active, Blended, and Collaborative learning environment at a large public university's engineering classroom. Students perceived that the Freeform learning environment provided continuous and quick access to learning resources. The suite of online videos aligned with and supplemented the course content and in-class pedagogies. Students experienced strong alignment between their in-class experiences and blended content. Many participants emphasized the value of their peers, both in person and online, while working on homework or studying for exams as sources of feedback or problem-solving strategies. The results show that the perceptions of utility described here are clearly articulated across our interview strata, and they illustrate in rich detail how students describe and perceive the benefits associated with the complex set of learning resources and in-class pedagogies of this ABC environment. 8 who were involved in the data collection process and Kavitha Ramane for her contribution in developing the codebook.