The Effects of the Use of Renzulli Learning on Student Achievement in Reading Comprehension, Reading Fluency, Social Studies, and Science

Renzulli Learning is an on-line educational profile and educational learning system designed to match student interests, learning styles, and expression styles with a vast array of educational activities and resources designed to enrich and engage students’ learning process. In this experimental study, quantitative procedures were used to investigate the use of Renzulli Learning on oral reading fluency, reading comprehension, science achievement, social studies achievement of 383 elementary and middle schools students. The research took place in two schools, an urban middle school in Georgia where half of all students are considered to be at risk due to poverty or other factors, and a suburban elementary school in southern California. Students in grades 3 5 (n = 185) and grades 6 8 (n = 198) were randomly assigned to use Renzulli Learning for 2-3 hours each week for a 16-week period. Students in the treatment groups were compared to students who did not have the opportunity to use Renzulli Learning in control classes in the same schools. A two-way repeated-measures ANOVA was used to explore differences between treatment and control students. After 16 weeks, students who participated in Renzulli Learning demonstrated significantly higher growth in reading comprehension (p < .001), significantly higher growth in oral reading fluency (p = .016), and significantly higher growth in social studies achievement (p = .013) than those students who did not participate in Renzulli Learning.

INTRODUCTION Despite some research that links technology and educational attainment in U.S. classrooms [1,3,17,19,23], little research exists about whether the use of Internet technology can increase reading fluency and comprehension. This quantitative study explored the use of an Internet-based program, Renzulli Learning (RL), to increase student achievement, specifically in reading fluency, reading comprehension, science and social studies. This interactive on-line program matches student interests, learning styles, and expression styles with a vast array of educational activities and resources designed to enrich students' learning process. In this study, students using Renzulli Learning were provided with opportunities to explore, discover, learn, and create using the most current technology resources in a safe environment. Many students spend hours reading independently on the Internet while using Renzulli Learning, and this study was conducted to explore whether the use of this on-line system could result in increased reading fluency and comprehension, as well as science and social studies achievement for elementary and middle school children.
II. REVIEW OF RELATED RESEARCH According to a national survey by [26], with over 185,000 student respondents from all 50 states, students who use the Internet as a resource at home are frustrated because they can not use technology how, when, and where they would like to in school. Major findings from [26] found that teachers and students believe there is a need to gain access to up-to-date technology tools at school when students need them. This survey also found that students want to use the power of technology to enrich their learning experiences [26]. Neuman [27] pointed out the changing nature of school libraries that incorporate the Internet to enhance student learning, and cited differences in "virtual as well as physical space" between schools using technology and those that are not.
III. DIGITAL DIVIDES BETWEEN ADULTS AND CHILDREN Prensky [31] studied the paradigm shift in the way adults and children (teachers and students) perceive differences in technology, finding that students and teachers experience a digital divide, and that the American educational system is not designed to teach today's digital natives. The chasm between school-aged students and their adult teachers is explained by the four-word title of an article, "Digital Natives, Digital Immigrants" suggesting that today's students (K -college) are digital natives who have grown up with the new technologies of the 21 st century [31]. These "native speakers" of the digital language, including the Internet, computers, and video games. Recent college graduates have spent as many as 20,000 hours watching television, over 10,000 hours playing video games, and less than 5,000 hours reading [31]. A critical question, according to [31], relates to how many of those students would read more often with increased incentives due to their use of technology and the Internet. Levin and Arafeh reported that 71% of students in the U.S. between the ages of 12 and 17 cited the Internet as their primary resource for help with homework THE EFFECTS OF THE USE OF RENZULLI LEARNING ON STUDENT ACHIEVEMENT IN READING COMPREHENSION, READING  FLUENCY, SOCIAL STUDIES, AND SCIENCE: AN INVESTIGATION OF TECHNOLOGY AND LEARNING IN GRADES 3 -8 assignments [22]. "The study is based primarily on information gathered from 14 gender-balanced, racially diverse focus groups of 136 students, drawn from 36 different schools" [22]. According to [20], in a survey of 754 American students, they found that 94% of the sample between the ages of 12 and 17 used the Internet as their primary "textbook" for information and school research. When students spend more time reading online and use the Internet as a primary resource, they may be improving their reading skills in the process. When students are given an opportunity to pursue their interests both electronically and in traditional learning environments, they may become more invested in their learning and more engaged in school [35]. Some teachers, also known as digital immigrants, are more comfortable with technology than others, yet an important distinction exists between the students (digital natives) and their teachers (digital immigrants). "As Digital Immigrants learn -like all immigrants, some better than others -to adapt to their environment, they always retain, to some degree, their "accent," that is, their foot in the past. The "digital immigrant accent" can be seen in such things as turning to the Internet for information second rather than first, or in reading the manual for a program rather than assuming that the program itself will teach us to use it" [31]. Prensky believed some Digital Immigrant teachers speak an outdated language in school classrooms, and often struggle when they try to teach students who speak an entirely new and different language [30].
Among an identified population of advanced and high potential students with technological strengths and interests, technology is viewed as an appropriate way to maintain interest and engagement in learning [39,40]. "While a myriad of quality educational software programs exist, some of the best programs resemble an empty box in that they begin with a blank screen upon which children can apply their creative imagination and talents" [39]. Precocious and academically talented students appear to be in the generation of digital natives who incorporate technology into their cognitive learning processes; while many of their teachers are from a generation of digital immigrants who struggle to master an ever-changing language of technology [39].
Other research has demonstrated the power of technology and its influence on student learning [22,4,8,9]. In one eight-year longitudinal study, it was found that careful alignment between content-area learning standards and carefully selected technology can significantly increase student achievement scores [2]. Cradler and Cradler [4] reported that teachers observed significant changes in their students' skills and knowledge acquisition upon completion of their first multi-media project. In a follow-up study, "teachers reported increased student knowledge in: research skills; ability to apply learning toward real-world situations; organizational skills; and interest in content" [5]. One study also found that student achievement increased in science when students created multimedia projects illustrating the concepts they were learning in school [38]. Positive changes were reported in classrooms where teachers used the Internet as part of their instruction [8,9].
No research, until now, has linked Internet technology to significantly higher scores in students' reading fluency and comprehension. Pikulski and Chard [29] found a correlation exists between reading fluency and reading comprehension scores, using data from the National Assessment of Educational Progress in Reading. "In that study, 44% of the subjects were not fluent when reading grade-level appropriate materials; the study also showed a significant, positive relationship between oral reading fluency and reading comprehension" [29]. In another recent study investigating the outcome of the use of an enriched reading program, the correlation between reading fluency and comprehension was found to be high [32].
The question of how technology can increase and enhance learning is a growing area of educational research in the United States. Some researchers have investigated teachers' perceptions of self-efficacy in relation to their technology usage [14,24,25]; perceived benefits for students who use technology, [1,3,23]; and perceived benefits among teachers who use technology in the classroom [21]. Little research has addressed the use of technology to increase oral reading fluency or comprehension. Research shows that reading rate norms often increase in the middle grades, and after sixth grade an identifiable trend in reading fluency scores shows that those scores flatten as children get older [13].
Renzulli Learning is an electronic search-engine and profiler that matches students' perceived interests, abilities, learning styles, and expression styles to thousands of enrichment activities. "This program is not a variation of earlier generations of popular e-learning programs offered by numerous software companies. It is a totally unique use of the Internet that combines computer based strength assessment with search engine technology, thus allowing true differentiation in the matching of thousands of carefully selected resources to individual strengths" [36].
Renzulli Learning is based on the Enrichment Triad Model [34] and the Schoolwide Enrichment Model (SEM) developed by Renzulli and Reis [35], representing over 30 years of research conducted at the University of Connecticut's Neag School of Education. The SEM is recognized as one of the most widely used plans for enrichment and talent development in the world [35]. The SEM is defined as, "a systematic set of specific strategies for increasing student effort, enjoyment, and performance, and for integrating a broad range of advanced level learning experiences and higher order thinking skills into any curricular area, course of study, or pattern of school organization" [35]. In its original paper-based format, the SEM instruments that are now a part of Renzulli Learning have been field tested for over 20 years in thousands of schools. Since the advent of Renzulli Learning in 2005, its licensed user base has grown to over 300,000 students and 40,000 teachers in 3,000 schools and 450 school districts across 40 U.S. states, Canada, Bermuda, and the Middle East.
IV. RESEARCH QUESTIONS The purpose of this study was to investigate differences in students' reading fluency, reading comprehension, science achievement, and social studies achievement between treatment and control conditions (using or not using RL) on pre and post-tests using the Iowa Test of Basic Skills (ITBS). The following research questions, addressed through quantitative analyses, guided this study: 1) Is there a significant difference from pre-test oral reading fluency scores to post-test oral reading fluency scores for students who participated in RL and students who did not participate in RL? 2) Is there a significant difference from pre-ITBS reading comprehension scores to post-ITBS reading comprehension scores for students who participated in RL and students who did not participate in RL? 3) Is there a significant difference from pre-ITBS science achievement scores to post-ITBS science achievement scores for students who participated in RL and students who did not participate in RL? 4) Is there a significant difference from pre-ITBS social studies achievement scores to post-ITBS social studies achievement scores for students who participated in RL and students who did not participate in RL?
V. METHODS Quantitative procedures were used to investigate the use of Renzulli Learning on reading fluency, reading comprehension, science achievement, and social studies achievement in two schools, an urban middle school where a majority of students are placed at risk due to poverty or other factors, and a suburban elementary school in a middle class neighborhood. The sample included 383 students (n = 203 males; n = 180 females) from two schools with administrators who volunteered to have students in both schools participate in the study. An experimental design was used with random assignment of students to treatment and control groups, and the use of recommended follow-up procedures to explore specific group and individual differences was employed.

VI. INTERVENTION
The two participating school administrators sought and received their district superintendent's approval, the administration and faculty at both sites agreed to participate in this study, and Institutional Review Board approval was sought and gained. One of the schools selected for this study was a middle school in Atlanta, GA, where treatment groups in 6 th , 7 th , and 8 th grades (n = 106) used Renzulli Learning for 16 weeks for approximately 2-3 hours each school week in the heterogeneously mixed, randomly assigned technology classes. Students at the middle school site were randomly selected, One Technology Connections teacher was assigned to teach all technology classes so random assignment of the teacher was not possible. Control groups in 6 th , 7 th , and 8 th grades (n = 92) at Inman, however, were not given access to Renzulli Learning for the same 16-week period, but had access to the system after the study was completed and all data were collected.
The second research site was an elementary school in Oak Park, CA where treatment groups in 3 rd , 4 th , and 5 th grades (n = 96) used Renzulli Learning for 16 weeks for approximately 3 hours each school week in randomly selected, heterogeneously mixed classrooms. Both teachers and students at Oak Hills Elementary were randomly selected to participate in treatment and control groups in the study. Control groups in 3 rd , 4 th , and 5 th grades (n = 89) at Oak Hills did not have access to Renzulli Learning for the same 16-week period, but were going to use the system after the intervention. Repeated-measures ANOVA procedures were used to explore group differences in each school.
Recruitment of schools for the study was completed the year before implementation. Several meetings were conducted with principals and superintendents from various school districts in professional development partnerships with the university researchers describing the study between the fall and the winter. Subsequent discussions were conducted with interested principals about study participation and the need for random assignment to treatment and control groups, as well as a minimum of 16 weeks for the intervention.
Two principals made a commitment to participate, schedules were arranged, and all students (and teachers at Oak Hills Elementary) were randomly assigned to treatment or control groups after Institutional Review Board (IRB) approval was received. The principals in both schools identified a research liaison to facilitate the implementation of the study, and professional development sessions for the treatment teachers were delivered in the summer preceding the intervention The sample for this study, as well as comparative demographic data at state levels and within each school, is summarized in Table 1 for the middle school, and Table 2 for the elementary school. VII. INSTRUMENTATION The following instruments were used to conduct pre and post assessments with all treatment and control students in this study.

A. Oral Reading Fluency Assessments
Curriculum-based measures of oral reading fluency were individually administered as a pre-and post-test to assess students' speed and accuracy when reading connected text [6]. Because oral reading fluency reflects the complex orchestration of both lower-level and higherlevel processes, it is considered a reliable indicator of overall reading proficiency [7]. To measure oral reading fluency, each student read an unpracticed grade level passage of connected text for 1 minute. The score is the number of words read correctly. Test-retest and alternateform reliability of oral reading fluency measures are consistently above .90 and criterion-related validity with other standardized measures of reading decoding and comprehension average .80 or higher [12].

B. Iowa Tests of Basic Skills
Reading comprehension, science achievement, and social studies achievement were measured using the Iowa Tests of Basic Skills (ITBS) subtests from each subject area (Form A, 2001). The ITBS measures achievement in 15 subject areas for students in grades K-8, and the three subject areas of reading comprehension, science, and social studies were administered at grade-level to all students in the study in grades 3 -8 in both treatment and control groups. The Reading Comprehension subtests of the ITBS, which are administered in two parts, measure how students derive meaning from their reading [15]. The Social Studies subtest assesses students' understanding of history, geography, economics, and government through a variety of concepts, principles, and selected visual illustrations [15].

C. Renzulli Learning Treatment
A 16-week overview for the activities that teachers implemented within the RL system is summarized in Table 3. These activities were provided to help the treatment teachers incorporate all aspects of Renzulli Learning into their classroom pedagogy.

VIII. RENZULLI LEARNING PROFILER & ENRICHMENT ACTIVITIES The Renzulli
Learning Profiler is an on-line questionnaire about students' interests, abilities, learning styles, and modes of expression that takes between thirty minutes and one hour to complete. The diagrams below illustrate the types of questions students answer in the profiler (Figure 1), and the screen that appears upon its completion ( Figure 2).
Students in the experimental group at both schools registered to use Renzulli Learning and began using the program in their heterogeneously mixed and randomly assigned classes for a time commitment of between 2 -3 hours per week over a 16-week period. After the Profiler was completed, students had access to their own Enrichment Database on Renzulli Learning, including the following activities and resources: virtual field trips; real field trips; creativity training; In this study, the teacher's role changed from the traditional model where the teacher planned and prescribed what is to be learned to one who facilitates enrichment learning. Renzulli and Reis [36] described the teacher's new role as the "guide-on-the-side" by approaching the teaching/learning interaction from the perspective of a coach or mentor rather than the teacher who "fills" the student with knowledge. The basic characteristics of enrichment, according to [35], learning include enabling students to select a topic that may be related to the regular curriculum or an independent topic based on his or her interest; the production of a product and/or service that is intended to have an impact on a particular audience; and the use of authentic methods, technological resources, and advanced level content by a student to produce a product or service.
IX. DATA ANALYSIS Several options were available for data analysis in this study, including analysis of covariance (ANCOVA) procedures, often used for a pre-test/post-test design when comparing the impact of two different interventions [28]. However, the assumptions for ANCOVA were not met when heterogeneity of slopes was found between pre-test and post-test scores for identified gifted and non-identified gifted students. "In pretest-posttest nonequivalent group design, naïve use of analysis of covariance or multiple regression can lead to Type I errors" [18]. For nonequivalent groups, it was recommended repeatedmeasures analysis of variance and a thorough analysis of the interaction term to help identify areas of significant change [18]. Therefore, it was decided that repeated measures analysis of variance (ANOVA) was the most appropriate procedure to measure differences in scores over the course of the 16-week intervention.
Multivariate assumptions of normality and a representative random sample assumption were met before repeated-measures ANOVA procedures were conducted [11]. Wilks' lambda (Λ) is the multivariate statistic used in this data analysis for a main or interaction source, ranging from 0 to 1 (0 indicates no relation between a repeated-measures source and the dependent variable and a 1 indicates the strongest relationship possible) [11]. The effect size is reported as Partial Eta squared (partial η 2 ), and should be interpreted by using the commonly followed effect-size guidelines of: .01 -.05 = small effect; .06 -.13 = moderate effect; and anything larger than .14 = large effect.
The quantitative analysis was completed using repeated-measures ANOVA procedures. For research question 1, the between variables were students using Renzulli Learning (treatment) and students not using Renzulli Learning (control). The repeated measures were the students' oral reading fluency pre-tests and post-tests. For research question 2, the between variables were students using Renzulli Learning (treatment) and students not using Renzulli Learning (control). The repeated measures were the students' Iowa Test of Basic Skills (ITBS) reading comprehension pre-tests and post-tests. For research question 3, the between variables were students using Renzulli Learning (treatment) and students not using Renzulli Learning (control). The repeated measures were the students' ITBS science pre-tests and post-tests. For research question 4, the between variables were students using Renzulli Learning (treatment) and students not using Renzulli Learning (control). The repeated measures were the students' ITBS social studies pre-tests and post-tests.
Differences in reading fluency, reading comprehension, science achievement, and social studies achievement were examined between treatment and control groups. Finally, effects for a significant interaction between participation in Renzulli Learning and students' identified or not identified as gifted were examined in relation to reading fluency, reading comprehension, science and social studies achievement. Gifted students at each school were identified by using multiple criteria required by their respective states. The state of California requires identification procedures be equitable, comprehensive, Creativity, such as a student generated product or performance judged by a panel of experts.
X. RESULTS A two-way repeated-measures Analysis of Variance (ANOVA) was used to explore differences between treatment and control students. Due to the research that identified some of the differences in the use of technology for gifted and advanced students [40], differences were also investigated between those students who were identified as gifted and those who were not identified gifted students in the statistical analysis and no significant differences were found between these two groups, meaning that the benefits of RL were extended to both groups.
After only 16 weeks, students who participated in Renzulli Learning demonstrated significantly higher growth in reading comprehension (p < .001), significantly higher growth in oral reading fluency (p = .016), and significantly higher growth in social studies achievement (p = .013) than those students who did not participate in Renzulli Learning. No differences were found in science achievement.
The mean scores by groups for reading fluency ( Table  4); reading comprehension ( Table 6); science achievement ( Table 8); and social studies achievement (Table 10), as well as the Analysis of Variance (ANOVA) for Main Effects and Interaction Effects for reading fluency ( Table  5); reading comprehension (Table 7); science achievement (Table 9); and social studies achievement (Table 11) are provided.   The main effect of Time in the ANOVA for Main Effects tables, which is the effect of time averaged across groups, was estimated for each dependent variable in addition to Time X Treatment and Time X Treatment X Gifted interaction effects. A significant Time X Treatment interaction effect was found for each of the following: Reading Fluency, Reading Comprehension, and Social Studies. No interaction effect was found for Science. These results indicate that averaging across gifted subgroups, the effect of time on the dependent variable varies by treatment group. This is illustrated by different slopes over time for the two treatment groups, which are students using Renzulli Learning (treatment) and students not using Renzulli Learning (control).
A significant Treatment X Gifted interaction indicates that averaging across time, the effect of being assigned to the treatment or control group on the dependent variable differs for groups who are and are not identified as gifted. This is illustrated by different average levels of performance on the dependent variable by subgroups (gifted or not) within treatment (control or treatment). No significant Treatment X Gifted interactions were found. Finally, for each dependent variable, a three-way Time X Treatment X Gifted interaction was tested. This interaction is interpreted as a difference in the effects of time by treatment group that varies by gifted subgroup. No significant Time X Treatment X Gifted interaction was found.
XI. DISCUSSION AND IMPLICATIONS When students have an opportunity to learn in ways that support their learning styles and interests, they may become more invested in their learning process, as well as the content and materials with which they are interacting and that they are attempting to master. For the last 30 years, Renzulli and Reis have advocated the use of interests to guide students' work and projects, and their research suggests that more interest leads to more student engagement and success [33,34,35,36]. This study suggests that when students are given access to an interest based Internet program, such as Renzulli Learning, to read, research, investigate, and produce, student achievement improves, specifically in relation to reading fluency, reading comprehension, and social studies achievement.
One study reported an increased likelihood for projectsbased learning and problem-solving activities for students with access to five or more computers in the classroom compared to students in classrooms without access to as many computers [41]. "Similar findings were reported in Dirksen and Tharp's (2000) work, indicating that general education teachers with access to three or more classroom computers were more skilled in integrating technology into the curriculum through higher level thinking activities than their colleagues with access to only one classroom computer" [37].
Jennings and Lucca investigated the impact of virtual field trips to support and increase student achievement, finding improvement in treatment group students on reading comprehension skills, measured by a standardized reading test when compared to control group students [17]. This study supports such findings suggesting the power and potential of learning through the use of technology. XII. LIMITATIONS There were a few limitations to this study, including unscheduled interruptions to the class schedule, unforeseen potential for a novelty effect, testing fatigue, and a diversity of tests. At both schools there were a couple of unscheduled interruptions to the class schedule, including an unannounced field trip and impromptu school meetings. As a result, the length of the intervention was only 16-weeks. Also, given the duration of the study, one may consider that a novelty effect could have played a role in the results. While that is possible, it is not necessarily plausible. One could also argue that if the intervention was longer, treatment students might have experienced greater gains. The exact number of assessments completed by each student throughout the course of the study was 10, including pre and post oral reading fluency assessments; pre and post ITBS reading comprehension (Parts I & II); ITBS social studies, and ITBS science. As a result of so many pre and post assessments, students may have experienced testing fatigue.
One may consider it a limitation that the same passage was read by students during the pre-and post-testing for oral reading fluency. Perhaps some students remembered what they read 16 weeks earlier and as a result their familiarity with the passages may have affected the scores. Finally, one may consider the possibility that the ITBS science subtest required more background knowledge and content-based information than the ITBS reading comprehension and social studies subtests.
XIII. CONCLUSION It is important to recognize the role technology plays in the lives of students in elementary and middle school grades. In The Road Ahead, Bill Gates wrote, "One thing is clear. We don't have the option of turning away from the future. No one gets to vote on whether technology is going to change our lives" [10]. Gates was correct, for technology has changed the lives of children both in and out of school and more teachers and researchers should explore the relationship between technology and student achievement.
This study suggests that using Renzulli Learning 2-3 hours per week for only 16 weeks has the potential to increase student achievement in terms of reading fluency, reading comprehension, and social studies achievement. If students are reading what they enjoy and are engaged and motivated to do so, one may conclude that achievement gains in reading will occur. While Renzulli Learning is not a panacea for reading motivation among young readers and adolescents, it does offer a variety of resources to choose from based on their interests, learning styles, and expression styles. Thus, the more a student reads, the more his or her reading will improve. This study is an important first step to exploring the Internet-based technology to improve student achievement.