Developing a Framework for Creating Effective Instructional Video Podcasts

The purpose of the following study was to develop a comprehensive, theory-based framework for creating instructional video podcasts designed to present worked examples. Sixteen design characteristics, organized according to four categories (establishing context, providing effective explanations, minimizing cognitive load, and engaging students), were used to develop 59 pre-calculus videos for 856 first-year university students. Overall, the vast majority of students noted that the video podcasts were useful and helped them understand mathematics better. With respect to establishing context, the evidence suggested that problem selection was appropriate and video podcasts were clear, straightforward, and detailed. Regarding the quality of explanations, a number of students commented on the effectiveness of the step-by-step presentation of solutions and the use of visuals to support learning. Students agreed that video podcasts were easy to read, but did not directly mention other issues involving cognitive load. Students also noted that video podcasts were engaging and better than using textbooks. They also enjoyed working on the interactive student-problems. Finally, significant gains were observed in all five pre-calculus knowledge categories evaluated. It is concluded that the framework proposed in this study is a reasonable starting point for creating effective workedexample video podcasts.


A. Overview
Video podcasts are audiovisual files that are distributed in a digital format through the Internet to personal computers or mobile devices [1]. According to a recent review of the literature [2], at least four different kinds of instructional video podcasts are used in higher education including lecture-based [3], enhanced [4], supplementary [1], and worked examples [5]. The current paper focuses on worked-example video podcasts which provide video explanations of procedural problems often associated with mathematics or science courses [6]. Video podcasts provide a potentially effective teaching method for addressing gaps in student knowledge. Public acceptance of this form of teaching is reflected by recent grants of seven million dollars from the Gates and O'Sullivan foundations to the Khan Academy (http://www.khanacademy.org), one of the largest and best known, non-profit depositories of worked-example video podcasts [7]. Limited research has been conducted on identifying characteristics that maximize effectiveness of these tools [5,8]. The purpose of the following study was to develop and assess a comprehen-sive, theory-based framework for designing workedexample video podcasts.

B. The Role of Video Podcasts in Education
Since 2006, research on the use of video podcasts in higher education has grown markedly [3,5]. Students have reported that this medium is motivating [9][10], useful, helpful, and effective with respect to improving learning [4,[11][12][13]. Students especially like controlling when and where they learn [10,14,15], what they need to learn [3,10], and the pace of learning [3,16,17]. In addition, researchers have observed that the use of video podcasts has resulted in significant increases in skills [9,18], test scores [5,19] and grades [20,21].
While previous research suggests that video podcasts have had a positive impact on student attitudes and learning performance, the format of presentation has been predominantly passive (e.g. lectures, PowerPoint summaries, or supplementary materials). Very few studies have examined video podcasts designed to present worked-examples [1][2][3].

C. Worked-Example Based Video Podcasts
As stated earlier, worked-example video podcasts are relatively short (e.g., under 10 minutes) video explanations of how to complete procedural problems in subject areas such mathematics or science [6]. To date, only two peer-reviewed studies have been conducted on this type of podcast. Loomes et al. [8] discussed the potential benefits of worked-example video podcasts but did not formally study their impact on student attitudes or learning. Crippen & Earl [5] reported that students had positive attitudes toward the use of video podcasts in an undergraduate chemistry class and that there was a significant correlation between the use of problem-solving video podcasts and test scores.
In spite of the limited research, the use of workedexample video podcasts has increased rapidly since 2005 and the launch of You Tube [22]. As of January 2012, YouTube was viewed over four billion times per day [23]. Often used for entertainment purposes, YouTube is also a free source of numerous worked-example video podcasts. In addition, new portals such as the Khan Academy, exclusively designed to distribute worked-example video podcasts, are used extensively (see http://www.khanacademy.org). However, limited attention has been directed toward improving the effectiveness of worked-example video podcasts. If students are going to use these tools, it is critical to identify characteristics that influence their quality and impact on learning.

D. Criteria for Creating Effective Worked-Example
Video Podcasts In order to identify key features that might alter the effectiveness and impact of worked-example videos, three tangential areas of research were consulted: written worked examples [6,[24][25][26][27], multimedia learning [24,28], and how people learn [29,30]. Based on an extensive review of the research, sixteen features were identified as potentially important in designing effective video podcasts. These features were organized under four main categories including establishing context, creating effective explanations, minimizing cognitive load, and engaging students. Each category and its sub-components will be discussed in turn.

E. Establishing Context
To establish the context of worked-example video podcasts, a review of tangential research indicates that the following areas need to be addressed: problem type, appropriate problem labels, background information, and identification of key problem elements.
Problem type. Creating well-designed problems is a critical first step to establishing an effective learning context [31][32][33][34]. Willingham [34] claims that teachers concentrate far too much on answers and solutions to procedural problems and do not devote enough time to creating meaningful and effective questions. Ball & Bass [31] add that acquiring the requisite knowledge for understanding how to create effective problems is extremely challenging and illusive for many teachers. Greer [32] notes that not enough detail or context is provided in mathematical word problems for students to establish personal meaning or connection to what is being asked.
Clear problem label. According to Bransford et al.'s [29] extensive review of how people learn, wellstructured, organized knowledge leads to better problem solving and retention than memorizing a series of facts or procedures. Clear, meaningful labels for problems within a larger organized framework are an important aspect of setting the context for student learning with workedexample video podcasts. Catrambone [35] adds that careful labelling of problems and sub-goals are influential in guiding effective problem solving.
Background information. Eliciting pre-existing knowledge and conceptions related to concepts being taught is an important aspect of teaching students to solve problems, because it provides opportunities to build on or challenge current understanding [29]. Willingham [34] suggests that it is also important to establish why a specific question is important in order to engage and motivate students in the problem solving process.
Explain key elements. Willingham [34] advocates that enough time needs to be spent on explaining what a problem is asking before delving into a solution. If the student does not understand the structure of the problem, what is being asked, and the necessary key elements required to solve the problem, he/she will have difficulty proceeding. With respect to multimedia presentation of problems, Clark & Mayer [24] maintain that it is essential that the student understand key pre-concepts before solving a problem so that he/she is not cognitively overwhelmed. They refer to this guideline as the pretraining principle.

F. Creating Effective Explanations
Once a student has been primed to solve a workedexample video podcast, an effective explanation has to be provided. Minimally, this process should include breaking problems into meaningful cognitive steps, explaining the reasoning for each step, and using visual supports.
Meaningful steps. Considerable evidence suggests that procedural problems, typically used in worked-example video podcasts, need to be broken down into explicit, meaningful, bite-size chunks, particularly when taught to novice learners [24][25][26][35][36][37]. However, more experienced or knowledgeable learners may not find detailed, step-by-step, worked examples particularly helpful because the explanations are too slow and simple [24].
Explain all steps. Once a problem has been broken down into meaningful steps, it is critical to explain the reasoning for each step and avoid abrupt cognitive leaps. Renkl [25] supports the use of principled explanations where each step is meaningfully connected to the entire explanation.
Use of visuals. The use and benefits of visual aids to increase the effectiveness of problem solving is well documented in the literature [6,38,39]. Clark & Mayer [24] refer to the multimedia principle and suggest that words and graphics be used together as opposed to words alone. However, they also note that graphics should be chosen carefully to help the learner understand the material, as opposed to simply decorating the page.

G. Minimizing Cognitive Load
When presenting procedural problems, instructors need to minimize extraneous cognitive load (engaging in processes that are not beneficial to learning) and optimize germane cognitive load (engaging in processes that help to solve the problem at hand) [40][41][42]. In worked-example video podcasts, extraneous cognitive load can be minimized by clear presentation of text, writing down key information, providing a clear, well-organized layout, and highlighting key areas of the page when they are being explained.
Readability. The writing in a worked-example video podcasts needs to be clearly legible or students will spend excessive processing time trying to decipher words and symbols. This fact is particularly important in mathematics-based video podcasts which are typically hand written.
Write down key information. Writing down key reference information is also critical when solving procedural problems so that the student's working memory is not overloaded. Clark & Mayer [24] add that the instructor needs to be selective in what is written down and what is spoken (modality principle). Writing down too much information can result in cognitive overload.
Layout. Keeping the layout well organized, clear, and uncluttered is particularly important when presenting a procedural-based problem. Crowded, dense displays of text over numerous screens can overwhelm a novice student [24]. Referred to as the coherence principle, Clark & Mayer [24] suggest that instructors should avoid adding any information that is not relevant to the instructional goal.
Highlighting. The final way to minimize extraneous cognitive load in worked-example video podcasts is to highlight key areas of the screen when providing an ex-SPECIAL FOCUS PAPER DEVELOPING A FRAMEWORK FOR CREATING EFFECTIVE INSTRUCTIONAL VIDEO PODCASTS planation. Without the highlighting, students can spend an inordinate amount of time trying to figure out what aspect of the problem is being discussed. Willingham [34] adds that if a student's attention is not fully directed to the area of a problem being presented, learning is unlikely to occur.

H. Engaging Students
Student engagement is also critical when creating worked-example video podcasts and instructors need to be cognizant of features such as voice intonation, pace and length of explanations, potential distractions, and opportunities for interaction.
Engaging voice. Research on discourse processing [43], the voice principle [44], and the personalization principle [24,45,46] suggest that worked-example video podcasts need to be communicated with an engaging voice. According to Beck et al. [43], students work harder on problems when information is presented in a conversational manner. Atkinson et al. [44] add that students perform better when human, personal voices are used compared to computer generated voices. Finally, Clark & Mayer [24] maintain that a conversational speech is significantly more effective than formal speech with respect to improving student performance.
Pace. The pace of explanation can have a significant impact on learning, particularly with respect to cognitive load [40][41][42]. While the impact of pace can be partially moderated by the pause and rewind features of video podcasts, the overall flow of learning may feel somewhat disjointed. Willingham [34] suggest that effective pace is critical for avoiding working memory overload.
Length of Clip. According to Tapscott [47], today's students, referred to as the Net Generation, are quick switchers who like to multitask and take frequent mental breaks. Lengthy worked-example video podcasts might be problematic in sustaining student attention. Medina [48] provides some evidence to suggest that the human attention span wanes after about ten minutes of listening to passively presented instructional material. Finally, Renkl [25] advocates the use of minimalist explanations, focussing on relevant meaningful details and minimizing extraneous information that could increase cognitive load.
Distractions. Clark & Mayer [24] provide considerable evidence to suggest that distractions in the form of dramatic stories, pictures and background music have a detrimental impact on student problem solving in a multimedia environment. It is argued that these distractions, often intended to garner interest or entertain, shift attention away from learning. Student problem. Proponents of constructivist learning might argue that worked-example video podcasts over emphasize passive presentation of content instead of active problem solving and knowledge building [25,26]. However, considerable research suggests that complex learning tasks require more extensive guidance, particularly for novice learners (see Kirschner [26] for a review).
Presenting worked examples in conjunction with relevant practice problems increases problem solving effectiveness [49].

I. Summary
The schema proposed for creating worked-example video podcasts consists of four key areas: establishing context, creating effective explanations, minimizing extraneous cognitive load, and maintaining student engagement. This schema is summarized in table 1.

Establishing the Context
Problem Type: An appropriate problem is chosen for the concept being presented (e.g., focuses student on specific concept, numbers are select carefully) [31][32][33][34] Clear Problem Label: The problem is clearly labeled and displayed at the beginning of the clip [29,35] Background Information: The context and type of problem is clearly articulated at the beginning of the clip. [29, 34, p. 20 & p. 84] Explain Key Elements: Key elements are clearly articulated before trying to solve it. Don't not simply read the problem, Rather, highlight key features that learners should attend to [24, 34, p.75]

J. Research Questions
The preliminary framework for creating workedexample video podcasts in Table 1 was used to develop and deliver 59 pre-calculus problems to university students. Two primary research questions were used to assess effectiveness of the proposed video podcast design framework: 1. What were student attitudes toward worked-example video podcasts? (survey data and comments) 2. How did student knowledge change as a result of using video podcasts? (self-report data)? http://www.i-jet.org Students. The student sample (n=856) was chosen from a small university located within a large, metropolitan area of over three million people. Over 70 percent (n=603) of the sample population were males. Students coming from engineering (n=463), science (n=342) or education (n=51) backgrounds were enrolled in a first year university Calculus course and reported high school grades of 50-59% (4%, n=34), 60-69 (13%, n=107), 70-79% (36%, n=309), 80-89% (35%, n=295) and 90+ (13%, n=101).

B. Video Podcasts
Development. Two female secondary school instructors with advanced knowledge in mathematics were trained to develop video podcasts based on the framework outlined in this study (Table 1). Practice video podcasts were developed, evaluated and discussed to ensure consistent quality and adherence to the 16 key design principles ( Table 1). The training process took place over a period of two weeks. During the subsequent two months, 59 video podcasts were created using a screen casting program called Camtasia (version 5). Each finished recording took approximately 60 to 90 minutes to complete. All video podcasts were then loaded and organized on a central web page for students to access when needed. See Kay [60] for a link to the complete library of video podcasts developed and delivered.
Content. The course instructor created 59 problems based on difficulties that students had experienced over the previous two years in the first year university Calculus course. Care was taken to select the correct problem type and challenge level ( Table 1, Item 1). Five content areas were covered including operations with functions, solving equations, linear functions, exponential and logarithmic functions, and trigonometric functions. A detailed description of five pre-calculus content areas covered by the 59 worked-example video podcasts is provided in table 2.
Length. An effort was made to provide high quality, effective explanations that where less than 10 minutes in length, as prescribed by Medina [48] and Tapscott [47] ( Problem format. Each video podcast included the problem solved by the teacher in a step-by-step fashion (teacher problem) and a corresponding problem to be solved by the student (student problem). The student problem was included to increase engagement and accommodate the recommendation for students to solve problems in conjunction with the instructor's explanation (see Table 1, Item 16). The teacher would start by explaining the nature of the problem to be solved and then proceed to discuss the first step. The video podcast would then automatically stop so that the student would have time to work through the first step of his/her assigned problem. When the student wished to continue, he/she would click a button and the answer for the first step to the student solution would be presented. The process of explaining a step and pausing the clip to allow the student to complete the corresponding step in his/her assigned problem continued until the full solution was presented. Students could also control the video podcast with a pause, stop, or play button as well as a dragging tool which a permitted movement to anywhere in the clip (see http://tiny.cc/calcvp1 for a sample clip).

C. Procedure
Students were sent a link to the video podcast depository three weeks prior to a pre-calculus diagnostic test. They were also reminded of the link, posted on the course web site, during the first week of classes. After they received their results from the diagnostic test, they were asked to fill in a 10 to 15 minute survey inquiring about their use of and attitudes toward video podcasts. Participation in this study was voluntary, anonymous and in no way impacted a student's grade. The survey questions about video podcasts are presented in Appendix A.

D. Data Collection
Data was collected in two formats: a formal student survey focussing on overall attitudes toward video podcasts and an open-ended question asking students why they used video podcasts. Specific questions targeting the four categories(establishing context, explanation, cognitive load, and engagement -see Table 1) outlined in the framework for developing video podcasts were not asked in order to keep the learning environment as natural as possible and avoid leading students toward answers they would not have provided on their own. For example, it is unlikely that students would notice the impact of a clear problem label or that key information was being written down in order to reduce extraneous cognitive load.

E. Data Sources
Background information. Students were asked their gender, program of study, and Calculus grade in highschool.
Open-ended response question. Students were asked to explain why they chose to use video podcasts. These qualitative comments were reviewed and categorized by theme. The coding scheme is presented in Appendix B.
Survey Data. Students were asked to rate overall usefulness and six features of video podcasts (Appendix A). The internal reliability of the scale was 0.83. However, each item in the scale was analysed individually in order to gain specific insights about student attitudes toward video podcasts.
Student knowledge. Students were asked to assess their pre-calculus knowledge before and after using the video podcasts in five areas: working with functions, solving equations, linear functions, exponential and logarithmic functions, trigonometric functions. This approach to assessing performance, while less precise than an actual test, was used to preserve anonymity and authenticity of the survey data. It is worth noting that Kuncel, Crede, & Thomas [61] conducted a meta-analysis on self-reported outcomes and concluded that students are accurate with respect to predicting their actual grades.

A. Use of Video Podcasts
Almost 60% (n=488) of the student sample chose to use video podcasts in at least one of the five content areas. Regarding specific content area, 41% (n=352) selected functions, 36% (n=304) chose solving equations, 33% (n=278) picked linear equations), 43% (n=366) opted for exponents and logarithms, and 38% (n=322) used trigonometric functions. Total time spent using video podcasts ranged from 8 to 240 minutes with an average of 67.2 minutes (SD=70.0).

B. Student Attitudes Video Podcasts
Five categories of student attitudes will be discussed based on the developmental framework for creating video podcasts proposed in this study: general impact, establishing context, explanation, minimizing cognitive load, and engagement.
General impact. Overall, nearly three quarters of the 453 students who used video podcasts rated them as being useful (n=130; 29%) or very useful (n=203; 45%). Only three students rated video podcasts as not at all useful. Students rated the worked-example video podcasts very highly with average item scores ranging from 3.9 to 4.4 on a five-point Likert scale (1=Strongly Disagree, 5 = Strongly Agree). Just over 80% of the students agreed or strongly agreed that the video podcasts helped them understand mathematics concepts better (see Table 3). Based on the student comments, almost 50 students noted that the video podcasts were helpful or useful. Typical comments were: "I used them and found them helpful." "[They were] very detailed, interesting, and helpful." "The clips were useful and informative." "I thought that they were very helpful and clear." Establishing context. Student comments suggested that problem selection was effective. The problems helped them review (n=22 comments) or remember (n=11 comments) previously forgotten pre-calculus concepts, address specific gaps in knowledge (n=23 comments), or solve problems in which they were having difficulty (n=19). Sample comments were: "The clips helped [me] to remember operations." "If I didn't remember how to solve a problem, watching the videos helped me." "They were a good way to review." "The clips where great tools as they helped me with the problems that I was having related to certain topics." "I used to it help me in solving some of the questions on the practice tests. Students were divided on the level of problem difficulty. Some (n=10) regarded the problems as being too easy (e.g. "I wanted to see the more complicated problems.", "The examples were too easy.") while others (n=7) thought they provided a sufficient level of depth (e.g., "They showed how to do harder questions.", " The clips were very useful in bringing some depth").
Indirect evidence of establishing context came from survey data and student comments about the quality of explanations (n=39 comments). In the survey, approximately 90% of the students agreed or strongly agreed that video podcasts were easy to follow and of high quality (Table 3).
Students also commented that the video podcasts were easy to follow, clear, simple, straightforward, and detailed. Sample comments included: "I used the clips because it was easy to follow along." "The clips [were] very thoroughly explained." "I used them because it broke everything down and went quite slow." "Because they gave a good explanatory guide to many problems." "They were very in-depth and idiot proof."

SPECIAL FOCUS PAPER DEVELOPING A FRAMEWORK FOR CREATING EFFECTIVE INSTRUCTIONAL VIDEO PODCASTS
In addition, students noted that the video podcasts were beneficial for understanding new concepts. Again, this is tangential evidence that the approach to creating video podcasts was effective in terms of establishing the overall problem context and key features. Sample comments were: "[The clips] helped me to grasp the knowledge." "[They] to better understand the material." "They helped me understand clearly how to approach the problem." "It helped me understand how to actually apply my knowledge when given certain questions." Explanation. In addition to commenting on the overall quality of explanations and increasing understanding, students explicitly and frequently noted the effectiveness of the step-by-step process. This was the feature emphasized most by students (n=75). Typical comments were: "I used the mini clips because I liked to have seen the example being done so I know what steps to take figuring out the question." "They help[ed] me and guided me step by step and that is what lead to my greater understanding of how to answer the question." "I used them because they go through all the examples making sure to go through all the simple and complex math." "I used the mini clips because it showed step by step solutions." Students also reacted positively to the use of diagrams and other visuals to help organize, clarify, and/or illustrate key aspects of problems (n=35 comments). Representative comments were: "They helped a lot to visualize and understand what was happening." "It helped watching a visual of the concept." "I used the clips because they illustrated the written documentation." "The visual aids helped me see how to do the problems." Minimizing cognitive load. Almost 90% of the students agreed or strongly agreed that the writing in the video podcast was easy to read (Table 3). However, students did not address cognitive load issues such as writing down key information, layout, or highlighting directly with their comments. Indirectly, numerous comments about step-bystep explanations indicated that many students were not overwhelmed by the cognitive load experienced while watching video podcasts.
Engagement. Three quarters of the students agreed or strongly agreed that using video podcasts was better than using textbooks and that good learning tips were provided. Nearly two thirds of the students agreed or strongly agreed that they liked doing the student problems (Table 3).
In addition, open-ended responses indicated that students liked completing the student problem (n=25 comments). Typical comments were: "I liked these because you actually got to try the problems with someone there helping you along." "[I liked that] every mini-clip has a sample question to answer on our own with the solution being provided later on. "Having the chance to solve the problem before the answer was given was great." Furthermore, a number of students (n=24 comments) preferred video podcasts over textbooks noting that "It's easier to understand when you see it being done than when you look at an already finished solution" or that "It's much easier to focus on and interesting than reading instructions on how to do something".
Moreover, some students felt that the video podcasts were engaging (e.g., "[The clips" kept your attention" or "[The clips were] much more fun and interesting") and personal (e.g., " [I] like having someone right there to walk me through it ... kind of like my own personal teacher").
Additionally, a sizeable number of students were more engaged because video podcasts suited their personal learning style (n=22 comments). Sample comments were: "I am a very visual learner, and to fully understand a concept I need to see it being done." "I am a visual learner, they helped a lot." "Having the voice explaining and seeing it happen works well." It should be noted that a handful of students (n=5) appeared to disengage because they thought the video podcasts went too slow ("She goes really, really slow and it's quite frustrating", "At times the person spoke too slowly").

C. Video Podcasts and Student Understanding
Students self-assessed five areas of pre-calculus knowledge before and after a two week period of using video podcasts. Paired t-tests revealed significant gains in all five pre-calculus knowledge categories evaluated (Table 4). The effect sizes (based on Cohen's d) ranged from 0.30 to 0.53 and are considered to be moderate [62,63].

A. Overall Impact on Student Attitudes
The evidence from this study suggests that workedexample video podcasts were well used with almost 60% of the student population viewing this medium for over an hour, on average, to prepare for the pre-calculus diagnos-SPECIAL FOCUS PAPER DEVELOPING A FRAMEWORK FOR CREATING EFFECTIVE INSTRUCTIONAL VIDEO PODCASTS tic test. Both survey data and open-ended comments indicated that the vast majority of students rated the workedexample video podcasts as useful tools that helped them learn mathematics concepts better. It is reasonable to conclude most students had positive attitudes toward workedexample video podcasts created using the framework outlined in Table 1. It is important, though, to examine student ratings and comments in more detail to determine whether specific criteria outlined in the design framework proposed for the current study contributed to positive attitudes experienced by students. Each of the four main categories in Table 1 will be addressed (establish context, creating effective explanations, minimizing cognitive load, and engagement).

B. Design Framework and Student Attitudes
Establishing context. While there was some debate on the difficulty level of questions, many students in this study responded positively to the problem selection claiming that the video podcasts were successful in helping them remember or review, previously forgotten topics. In other words, the knowledge was relevant and helpful to students with respect to bridging learning gaps in targeted content areas. Specific comments about the use of clear problems labels (Table 1, Item 2), providing sufficient background knowledge (Table 1, Item 3), and explaining key elements before solving a problem (Table 1, Item 4) was not offered by students. Nonetheless, survey data and student comments strongly suggested that workedexample video podcasts were easy to follow and of high quality. If sufficient context was not developed, it is unlikely students would have rated ease of use and quality so highly.
Regarding difficulty level, some students thought that the problems selected were too easy while others noted that they provided the correct level of depth. This result is consistent with research suggesting that worked examples are effective for beginner level students and frustrating for more knowledgeable peers [24-26, 28, 35-37].
Creating effective explanations. The data from this study indicated that large number of students noted and appreciated the effectiveness of the step-by-step explanations. This result was predicted by previous literature [24-26, 28, 35-37] but has not been explicitly confirmed in a worked-example video podcast learning environment. Furthermore, visual aids offered significant learning support. There was a clear preference for the dynamic visualization of a problem as opposed to the static presentation of a text-based format. This result is consistent with previous research claiming that visual information is advantageous in the problem solving process [6,24,50,51]. It is reasonable to conclude that students were positive about step-by step explanations and visual supports provided in workedexample video podcasts.
Minimizing cognitive load. Students did not comment directly about cognitive load issues such as writing down key information (  Item 11). A majority of students agreed that the writing in the video podcasts was clear, although a direct link to cognitive load was not established. An argument could be made that extraneous cognitive load was effectively managed because most students rated video podcasts as useful, clear, easy to follow, and effective at helping them understand and learn pre-calculus concepts previously forgotten. If students were experiencing high extraneous cognitive load, they would have commented that the video podcasts were difficult to understand, hard to follow, went too fast, or were confusing. No such comments were made. Nonetheless, more research is needed to directly establish a link between minimizing extraneous cognitive load and the effectiveness of worked-example video podcasts.
Engagement. Students in this study commented sparingly if at all on specific design features such as voice (  Item 15). However, nearly two thirds of the students enjoyed working through the student problem as the video podcast was being presented. A number of students added that attempting the student problem assisted their learning. Other evidence that students were engaged included direct comments about the video podcasts keeping their attention and being interesting. In addition, three quarters of the students noted that working with worked-example video podcasts was better than using textbooks. As was the case when analyzing student attitudes regarding cognitive load, the evidence does not explicitly support the impact of specific design features used to increase engagement, with the exception of incorporating a student problem. Nevertheless, the data suggest that students were engaged when using worked-example video podcasts.

C. Learning Performance
Significant increases were observed for all five categories of pre-calculus knowledge areas with a moderate effect size. This result is consistent with student beliefs that that the worked-example video podcasts had a significant impact on their learning. Gains in understanding and knowledge are consistent with previous research and the impact of lecture-based video podcasts [5,9,18,19] as well as the single study conducted using worked-example video podcasts [5]. It is important to emphasize that any conclusions about knowledge gains are limited because the data collected was based on self-assessment. In addition, it would be spurious to assume that the only influence on learning was the use of worked-example video podcasts. Students may have consulted textbooks, other web-based resources and friends to augment there knowledge of pre-calculus concepts. A more rigorous evaluation would involve the completion of actual preand post tests.

D. Summary
Based on a thorough review of the literature on written worked examples, multimedia learning, and Bransford et al.'s model of how people learn, 16 criteria for designing worked-example video podcasts were proposed and organized under four main categories: establishing context, explaining the problem, cognitive load, and engagement. A holistic as opposed to an experimental approach was used to evaluate the general impact of the proposed design characteristics. The results indicated that, overall, students had positive attitudes toward the use of worked-example video podcasts, noting they were useful and improved their understanding of pre-calculus concepts. In addition, self-assessed knowledge in five pre-calculus concepts increased significantly, based on pre and post evaluations. Direct evidence from survey and open-ended questions suggested that specific design features such a problem selection, step-by-step explanations, use of visual suphttp://www.i-jet.org SPECIAL FOCUS PAPER DEVELOPING A FRAMEWORK FOR CREATING EFFECTIVE INSTRUCTIONAL VIDEO PODCASTS ports, and providing a corresponding student problem to work through during the video podcasts were effective. However, more research needs to be conducted to establish a direct link between the remaining design features, student attitudes, and learning performance.

E. Caveats and Future Research
This study is a first attempt to examine and evaluate a research-based framework for designing and creating worked-example video podcasts. A concerted effort was made to ensure the quality of the analysis by conducting a comprehensive review of video podcasts design principles, providing a detailed description of the video podcasts used, collecting data from a large sample, and employing multiple data collection tools. Nonetheless, several notable caveats are worth mentioning to guide future research.
First, actual student behaviours and reactions while using worked-example video podcasts need to be examined in order to determine whether specific design features are having their intended impact. For example, the use of think-aloud protocols where a student talks out loud while using a video podcast could provide more detailed information on what students attend to while solving problems and the impact of various design elements on cognitive load. In addition, collecting data using interviews or focus groups with a small sub-sample of students might help gather more information on which characteristics of video podcast use are associated with negative and positive outcomes. Furthermore, data collection tools on the attitudes of students toward video podcasts could be further refined to target specific design components of the proposed framework. Finally, formal pre-and post-tests on precalculus would provide more persuasive data regarding the impact of video podcasts on learning performance.