The Ecology of Analytics in Education Stakeholder Interests in Data-Rich Educational Systems

— In this paper, we suggest an ecological perspective on the role of analytics in education. We discuss how different stakeholder positions in education points to different interests in using analytics. As a point of reference, we examine the Danish case of ICT integration in primary and lower secondary school (Danish: Folkeskolen) in order to study cases of emerging and at times conflicting stakeholder interests. On this basis, we discuss how this complexity of the educational ecosystem affects different stakeholder positions within the field.


Introduction
In recent years, large investments in digital educational infrastructure, and digital teaching and learning environments has led to an extensive interest in learning analytics from both public and private stakeholders. This digital transformation of education has evoked an unprecedented level of datafication of learning and teaching with large amounts of data collected on all levels of the educational system [1], [2]. This data is collected from a wide variety of systems and applications and includes information on everything from attendance to test results, student interaction, digitized student products and now even biometric data. To an increasing extent, data on behavior and performance generated in digital teaching and learning environments has become the filter through which learning and teaching is monitored and evaluated. These technological developments challenge traditional views of learning and teaching as well as our understanding of the role of analytics in education.
According to Ferguson [3], there are several drivers behind the growing interest in applying data analytical methods in education. These drivers are motivated by a range of political, educational, academic, and economical factors. Thus, the field of analytics in education has become a complex field of different stakeholder positions and interests. New partnerships between public and private actors emerge and challenge the traditional power balance in public education [4], [5]. This is, among other things, because policy reforms on digitalization rely on business actors to help schools fulfill the demands [6]. These new dynamics open for a much more complex conception of the role of analytics in education, because new emerging stakeholders hold different and at times competing or conflicting interests.
In this paper, we propose an ecological approach to the conception of analytics in education. An ecological approach allows us to describe the complexity and interrelatedness of the different levels of the educational ecosystem. Our aim is to show that learning analytics is only one out of several different interests in educational data, and we suggest purpose and success criteria as important standards for distinguishing between different types of stakeholder interests in educational data. As a point of reference, we examine the Danish case of ICT integration in primary and lower secondary school (Danish: Folkeskolen) in order to study cases of emerging and at times conflicting stakeholder interests. On this basis, we discuss how the complexity of the educational ecosystem affects different stakeholder positions within the field.

Background
Over the last decade, the field of learning analytics has emerged as a subdisciplinary field of educational research [7], [8], that focuses on utilizing computational techniques and tools for analyzing large amounts of data generated in educational contexts for purposes of understanding and optimizing learning and learning environments [9], [10], [11], [12]. Early learning analytics tended to focus on analyses that could inform strategies at an institutional level [13], e.g. by predicting overall academic performance or student retention. However, with the continuous increase of available digital data generated by individual students, learning analytics has moved the analytic scope towards the level of the individual learner [14], [15], in order to examine learning as it occurs in the context of formal education. This shift in focus is, among other things, caused by the emergence of e.g. digital learning platforms and learning management systems, online learning environments, digital learning resources, student programming and multimodal text production, that all leave behind digital traces containing rich information about both learning activities and learner behavior [16], [17], [18], [19].
The increased interest in data on individual learning progression has not meant, however, a decrease in the interest in other types of data generated in educational contexts. On the contrary, what we currently see is an overall increase in the generation and collection of data for multiple purposes in all parts of the educational system. Obviously, doing learning analytics, i.e. collecting and analyzing data for optimizing learning, is still a major interest. It is equally obvious, however, that the accelerating datafication of education presents an opportunity to do analytics for various other purposes than optimizing learning.

3
The Ecosystem of Education

Social contexts as ecologies
In order to describe an understand the increased complexity of educational analytics, we employ an ecological perspective to explore how different stakeholders in various parts of the ecosystem pursue different interests. In doing so, we follow a growing trend in recent years of framing social context and human interaction in terms of ecology. Pioneered by, among others, Gibson [20], [21] and Bronfenbrenner [22], notions such as ecosystem, environment, diversity and change have entered the terminology of various disciplines such as psychology, anthropology, and science and technology studies. Even though Gibson focused almost entirely on human interaction with the physical environment, his ecological account has been very influential in understanding how the surrounding environment shapes and enables human action and interaction. Bronfenbrenner focused more explicitly on human interaction in social contexts as he developed his ecological systems theory in order to account for human development. This account too has had great impact and inspired research with an ecological perspective in many fields.
Being conceived in the 70's, neither of the two approaches pay much attention to the impact of (digital) technologies on social ecosystems. Later, an important attempt to include technologies is made by Nardi & O'Day [23]. In order to describe technology-rich social contexts, they introduce the notion of information ecology, which they define as an ecosystem consisting of people, practices, values, and technologies. Thus, in their account, technologies are put on a par with actors and their values and practices. For the purpose of describing the complexity of contemporary data-rich educational systems, we need, however, to be able to look at the interconnected technologies as an independent system. Therefore, following García-Peñalvo [24], we understand the ecosystem of educational technologies (in the broadest sense) as an independent part of the larger ecosystem of education. This is important because digital technologies, as opposed to non-digital technologies such as books or pencils, can disperse data and information across platforms and interact in certain ways that create synergy and cross-sectional effects.

Analytics in educational ecosystems
On these grounds, we will work with three levels of description • The ecosystem of institutions • The ecosystem of technologies • The analytics practices in technology-rich educational ecosystems.
By ecosystem of institutions, we mean the entire range of public and private institutions and companies, including all the different groups of people that make op the institutions, e.g. politicians, policy makers, leaders, practitioners, parents, and stu-dents. In other words, the ecosystem of institutions is the sociocultural level of the educational ecosystem, i.e. the level of action, interaction, decision making etc.
By ecosystem of technologies, we mean the interconnected system of technologies (both hardware and software) that make up the entire cross-sectional network of administrative systems, communication systems, test systems, learning management systems, digital learning resources, digital student products etc. It is important to notice the asymmetry between the ecosystem of institutions and that of technologies: The ecosystem of institutions can exist without the ecosystem of technology, but not vice versa.
By analytics practices in technology-rich educational ecosystems, we mean the use of data generated within the ecosystem of technologies carried out by actors within the ecosystem of institutions with the purpose of doing analytics. By this definition, we mean to emphasize the dynamic relationship and mutual dependency of the three levels: Institutions implement new technologies > the ecosystem of technologies generates new types of data > new types of data enable new analytics practices > new analytics practices affect the behavior of actors in the ecosystem of institutions etc.

Ecosystems of Education: The Danish Case
Since the early 2000's the Danish public sector has been subject to a range of different strategies for digitalization [25], [26], [27], [28], [29]. The earliest strategies only targeted the educational sector moderately. However, in 2011, the Danish Government and KL -Local Government Denmark (interest organization of the 98 Danish municipalities) made a deal to set aside 500 million DKK (approx. 67 million EUR) to strengthen the use of ICT in primary and lower secondary school [30]. The funds were in particular allocated directly to schools to allow them to purchase digital learning resources and provide all students with well-functioning ICT. In addition, the funds were to strengthen the development of digital learning resources in the private sector and further strengthen research and consultant-based projects focusing on applying ICT in teaching. As a result, the Danish primary and lower secondary school has seen an increase in the use of digital learning resources [31] and a general digitization of the practices of different agents from both within and outside the educational domain. In the following, we will take a closer look on how these changes have accommodated new stakeholder positions and interests within the Danish educational ecosystem.

Ecosystem of institutions
A key part of the 2011 digitalization strategy was to provide public funding for private producers of learning resources. The idea was to aid the development of a range of digital learning resources that could utilize the affordances of the digital format to facilitate innovative and varied teaching practices [32]. The most recent outline of funded projects shows that many small private actors were provided with funding [33]. However, studies have shown that the Danish market for digital learning resources is largely dominated by big private actors who offer omnipotent online learn-ing portals with traditional ready-to-use courses [34], [35]. For instance, Gyldendal (the biggest publishing house in Denmark) has developed an online learning platform that covers all school subjects and content areas, and it is the most frequently used digital learning resource for the teaching of Danish as L1 (mother tongue education) [31].
From a macro perspective, there is an apparent discrepancy in interests between government intentions of using small private actors as an initiator of innovative teaching on the one hand and the private market allowing big actors with business interests to dominate with more traditional courses and content on the other. This supports the idea put forward by Tomlinson [36] that in economic markets consumer sovereignty soon gives way to producer sovereignty, which allows the stakeholder interests of big private actors to dominate at the expense of national policies and intentions. This type of digital transformation -whether it is in the public or the private sector -is usually based on an economic logic that equals digitalization with either financial profit, savings or increased systemic efficiency [37], [38], [39], [40].
From a micro perspective, different stakeholder positions are emerging. Parents and teachers enter the equation as agents interested in the data generated by new digital technologies in education. However, the interests of teachers and parents may differ, as teachers tend to use performance and achievement data in a functional manner to inform instructional strategies [41], whereas parents might have more implicit interests (e.g. choosing schools for their children). Further, school leaders might use data from technologies in the classroom to allocate financial means or teacher resources to specific groups of students. This is in line with the recent trend of datadriven or data-informed school management, which is becoming increasingly popular within the Danish educational system [42], [43]. A somewhat recent Danish study shows that both school leaders and parents find quantitative data more legitimate than qualitative data, whereas teachers prefer to use qualitative data to inform classroom practice. However, at school level quantitative data are more often used as the basis of local decisions, because quantitative data is easier to collect and organize systematically [44]. Although these stakeholder interests are not necessarily conflicting, they pose different perspectives and add different uses to the data generated by digital technologies in schools.

Ecosystem of technologies
The changes described in the above section have direct implications for the ecosystem of technologies. Although Denmark in general is a highly digitalized country [45], the latest digitalization strategies have further increased the number of digital technologies in the classroom. In conjunction, these technologies form an ecosystem capable of generating data about several aspects of learning and teaching environments, including data on individual students' learning outcome. However, different technologies in the classroom accommodate new stakeholder positions. For instance, a growing number of Danish so-called iPad-schools have occurred across the country investing in tablets based on a 'one device, one student' strategy [46]. Such strategies open for further intersections between public education and private market actors, who have learning and financial profiting as their respective interests.
Specific hardware also affects the kinds of software that are implemented in education. An example from the Danish case is the implementation of learning management systems, which is part of a larger 2014 collaboration between government, municipalities and private contractors (Danish: Bruger portal sinitiativet). Because all students have been equipped with either laptops or tablets, the idea is to further expand the digital infrastructure of Danish primary and upper secondary schools by providing teachers with online platforms that can help facilitate learning activities in the classroom. One of the features of the project is that data on student learning progression and well-being can be gathered and shared with teachers, parents, school leaders and governing institutions [47]. This allows the municipalities and the state to track progress on both an institutional (schools) and an individual (students) level, which potentially allows data to be used for accountability or benchmarking purposes.

Analytics practices in technology-rich educational ecosystems
From the descriptions above, it becomes clear that the digitalizing of the Danish primary and lower secondary school has created a large and diverse ecosystem of technologies capable of producing large amounts of data. It is equally clear that this development has opened for new stakeholders with other interests in educational data than optimizing learning outcomes. As the educational ecosystem evolves, new analytics practices might emerge. Right now, however, we see three main types of analytics practices within the educational ecosystem.
The first type of practice is institutional analytics. From a macro perspective institutional analytics focuses on accountability, benchmarking and institutional governance, and it is still a very common type of analytics despite the increased focus on the individual learner. From a micro perspective, this particular interest also manifests locally when school leaders use educational data in the interest of allocating financial means and teacher resources to specific areas of the school. Thus, we see institutional analytics as a dominant analytics practice with institutional governance as its purpose and with the distinct success criteria of increasing operational efficiency.
The second kind of practice we see is business analytics. If we turn to industry, we see a range of private actors interested in using the data created within the ecosystem of technologies for competing in the economic market more effectively [48]. Particularly, the developers of digital learning resources are important actors because they dominate the teaching of specific school subjects and hence have large amounts of data from the classrooms available to them. These types of data can be utilized in the process of designing or redesigning products with the aim of aligning business planning and strategy with data-informed knowledge of consumer (i.e. students and teachers) behavior. Thus, business analytics has the purpose of consolidating and increasing specific market positions and the distinct success criteria of ensuring revenue, and we see it as another dominant analytics practice within the Danish educational ecosystem.
The third kind of analytics we see is learning analytics. Learning analytics has the purpose of understanding and optimizing learning and learning environments and the distinct success criteria of improving students' learning outcome. As such, learning analytics is concerned with a wider variety of methodological and ethical issues, than is the case in the other two analytics practices. In addition, questions of data ownership and the difficulties of operationalizing valid measures of learning also makes the practice of learning analytics the most complex [44], which might explain why learning analytics currently seems to be the least dominant analytics practice in the Danish context. We see very few examples of full-fledged implementations of learning analytics tools, and they are often met with skepticism from practitioners.

Discussion
According to Reyes [49], learning analytics -as opposed to other types of analytics -has the potential to inform and benefit most stakeholders in education, because it examines the one unit that most stakeholders share, namely the learning of students. However, as we have pointed out there are many other types of analytics in education. Learning analytics is, as such, not a replacement for other kinds of analytics, but rather it is a further addition to an already complex educational ecology. In the following, we will briefly address how this complexity affects different stakeholder positions within the field.

Implications for policy makers
At the levels of policy and governance, there is a need for a more reflected approach to analytics based on deep knowledge of the potentials and limitations of data analytical methods in educational contexts. This implies that policy makers take into account the dynamics of the complex educational ecology and carefully consider how bringing in new technologies and new stakeholders affect the different ecosystems and the ecology as a whole. This means that knowledge generated through analytics is not blindly integrated into educational policies as a replacement for teacher expertise. Rather, such knowledge should be a supplement to teachers' pedagogical thinking and professional judgement [50].

Implications for schools and private stakeholders
As data magnitude and stakeholder numbers increase, so does the opportunities for data breaches. In addition, as more stakeholders from the private sector contribute to the ecosystem of technologies, the question of data ownership becomes more apparent [51]. This only adds to the high requirements of security and transparency that are already assigned to the field. Schools have a major challenge coping with this complexity. As pointed out by Rao, Ding & Gudivada [52], successful analytics require a highly sophisticated reference architecture consisting of many different human and technical resources. As such, neither schools nor private stakeholders are left alone to cope with the many challenges of analytics, but are required to collaborate and find common solutions and workflows.

Implications for teachers
Teachers might come under heavy pressure when different kinds of analytics enter the educational ecology. Thus, data literacy skills need to become an integral part of teacher training and education [53]. Data literacy is, in this sense, not only a matter of being able to understand and interpret data, but it also includes the ability to act upon data in a critical and constructive manner. A part of this is being aware of how different stakeholder positions within the ecology implies specific interests in educational data. Thus, teachers must be able to interact with data in a way that balances datainformed decision-making, and pedagogical freedom and reasoning.

Implications for students and parents
It is arguably an open question whether students can be seen as stakeholders in analytics, especially if the students are children. Students are, however, one of the main contributors and creators of data, as it is their activities and behavior that generate a big part of the data used in analytics. Still, new evaluation formats (especially in formal education) require students to be able to understand how the resulting data are interpreted. Data literacy is, thus, a key competence for students at all levels of the educational system. Similarly, parents are faced with many types of data about their children -particularly through primary and lower secondary school (assessment data, data on well-being, performance data etc.). Hence, data literacy skills are also important for parents in order for them to gain agency.

Implications for researchers
The central point of this paper is that different stakeholders and educational technologies are all part of one coherent ecology of education. Thus, changes within one ecosystem affect the other ecosystems in the ecology. In this way, the complex dynamics of the educational ecology is a consequence of different stakeholder positions and interests. This is important for researchers engaged in analytics to keep in mind, because it poses many new questions and dilemmas of both epistemological and ethical character. E.g., how do we merge and compare different data sources? How do we incorporate private stakeholders into research projects in an ethically sound and responsible manner? These questions need to be addressed and reflected on in future projects.

Conclusion
From the description of the Danish case, we have seen that stakeholder positions in education point to different and specific interests in doing analytics. Thus, it is an important point that not all types of analytics in education are learning analytics, i.e. analytics with the purpose of understanding and optimizing student learning and learning environments. Analytics concerned with institutional governance, benchmarking, accountability, business profits etc. are all relevant and for the most part legitimate positions seeking to take advantage of the digital traces left behind by different actors in education. These different positions do not necessarily pose a problem. However, when stakeholder interests are conflicting, we might see counterproductive initiatives where the use of different types of analytics are working in opposite directions. As such, taking into account the different stakeholder positions and interests in analytics is crucial for further development and manifestation of the field of educational analytics.