Trends in MOOCs Research: Analysis of Educational Technology Journals

Massive open online courses (MOOCs) have received a great deal of interest from educational institutions and private enterprises. Hence, several empirical studies and reviews have been produced. They will be at the heart of the synthesis presented in this article, which reviews the research objects, the analytical frameworks mobilized and the methodologies adopted in the research produced specifically between January 2012 and January 2018. The trends were identified by a review of 36 selected peer reviewed journals. 100 studies were retained after quality assessment. The results show that research questions focused primarily on learning processes followed by design of MOOCs, learning experience and predictors of retention. A variety of frameworks have been adopted in empirical studies. Questionnaires were used as primary collection method and descriptive statistics were the preferred treatment processes. Keywords—MOOC, literature review, research trends, research in MOOCs.


Introduction
The acronym MOOC stands for massive open online course which is growing in number in recent years. MOOCs are open to any interested person and offer education products for a scalable number of learners who have access to the internet [1], there are hundreds of courses around the world provided to millions of registered users of MOOCS [2]. These courses have prompted major universities and institutions to take a pedagogical view toward developing new design MOOCs and make them available to a global audience in order to promote teaching and learning, and research on MOOCs has extended significantly. Specifically, in the past six years there has been an increasing amount of literature on MOOCs research.
These literature reviews have concentrated on several vast areas such as categorizing academic research relating to MOOCs [3,4,5,6,7,8], exploring research trends in terms of data collection and analysis methods [9,10] or examining a specific theme or a particular issue [11,12,13,14,15,16,17,18,19,20,21]. We believe these literature reviews provide a valuable synthesis of MOOCs, but further examination is needed to examine the literature. Indeed, the past reviews failed to examine completely or cate-gorize research trends from the point of view of research questions and the mobilized frameworks. This study differs from the past literature reviews by its focus on research questions, frameworks mobilized and methodologies adopted. It expands the previous review of MOOC research by adding more updated papers published between January 2012 and January 2018 in selected high-quality journals.
This paper is organized as follows: Section 2 presents some previous reviews analyzing the research trends on MOOCs. Section 3 describes the methodology adopted to select journals and articles. Section 4 answers the research questions by presenting and analyzing the main obtained results. Section 5 discusses the results by comparing them with previous ones. Conclusions and implication for future work are described in section 6.

Literature review
Over the past six years, researchers have begun to give careful consideration to trends in research on MOOCs. Some of these studies sought to undertake a comprehensive analysis in order to take stock of the studies at a specific moment, including Bozkurt, Keskin and de Waard [22], Ebben et Murphy [3], Gašević et al. [4], Liyanagunawardena et al., 2013 [5], Veletsianos and Shepherdson [6], Yousef, Chatti, Schroeder, Wosnitza et Jakobs [7] and Zawacki-Richter et al. [8]. The research methods used in these examined literature reviews differ. The majority of selected studies come from databases and journals. The journals were generally selected on the basis of their specialisation in the field of educational technology, their presence in recognised databases such as Scopus and Thomson Reuters or their citation in the bibliographies of identified articles. The selection criteria for the articles included the type of article (research, report of the approach, theoretical analysis, etc.), whether or not it was published within the period covered by the literature review, and the presence of the keywords considered. The review undertaken by Gasaevic et al. [4] is somewhat unusual insofar as these authors analysed research projects submitted to the MOOC Research Initiative (MRI) and no journal or database was thus explored. The review undertaken by Israel [20] was 'narrative reviews' without any systematic methodological process, which explains the absence of the journals or databases examined, or even the number of articles found.  [4] outlined the specific finding of an analysis of the research proposals submitted to the MOOC Research Initiative (MRI) funded by the Gates Foundation and administered by Athabasca University. They examined the main research themes and research methodologies used in the examined studies. Social learning was found as a theme that received the greatest interest and mixed methods was the most preferred research approach.
One year later, Raffaghelli et al. [9] explored the methodological approaches most commonly adopted in the scholarly literature about MOOCs published during the period January 2008-May 2014. According to these authors, studies have primarily focused on theoretical research and case studies. Other key findings included: surveys were the most presented data collection method and conceptualisation of dimensions were the most frequently used analysis method. The next year, Veletsianos and Shepherdson [6] reviewed 183 empirical MOOC papers published between 2013 and 2015 with a purpose to explore the geographical distribution, research components, article citations and research methodologies of MOOC studies published between 2013 and 2015. They founded that the majority of the examined studies is published by individuals who are from North America and Europe. In addition, the examined literature has favored a quantitative research method preferring the collection of data via surveys and automated methods. Recently, Zhu, Sari and Lee [10] published an article reviewing research methods and topics of 146 empirical studies of MOOCs (by searching the electronic database Scopus, and peer-reviewed journals) published from October 2014 to November 2016. This review indicated that most studies used quantitative research methods and the most frequently adopted data collection method was survey and descriptive statistics were the most adopted analysis method.
The examined reviews offer valuable syntheses required to understand trends related to MOOCs, but are partial. For instance, they fail to account for the distribution of research questions and frameworks adopted (and their cited authors) to answer these research questions. There is only a paucity of studies that presented clearly this as a main research question. Indeed, the first review is that of Kennedy [14] who conducted a review research in order to better understand the characteristics associated with MOOCs by reviewing six studies published in journals between 2009 and 2012. Specifically, this author examined the frameworks ground research in MOOCs, types of research methods that have been applied to MOOCs and the characteristics associated with MOOCs. The second review is that of Bozkurt et al. [22] who analyzed 51 theses and dissertations related to MOOCs and published between 2008 and 2015, identifying research trends and currents issues about MOOCs in terms of research methods, research themes, research models/designs, as well as theories, concepts, frameworks and models. Similar to this research, but with a different inclusion criterion (including only articles instead of theses), the third review of Bozkurt, Akgün-Özbek and Zawacki-Richter [23] reviewed 362 articles published from 2008 to 2015.
Thus, the purpose of the present review is to examine in depth these current trends and issues in research on MOOCs by reviewing recent research papers. More specifically, this literature review will attempt to respond to the following research questions: 1. What research issues or objectives have been analysed in the articles identified in this literature review? 2. What conceptual frameworks or theories have been adopted to respond to these different research questions? 3. What research methodologies have been adopted in terms of the collection, analysis, and processing of data?

Data collection method
We collected data by using the scientific research method that identifies steps to conduct research literature review [24]. Only peer reviewed journals were retained for this study. Papers from report research, conference proceedings, books and thesis were not retained. A total of 36 high quality journals focusing on educational technology have been selected (Appendix A), of which 27 are indexed by the recognized databases (Scopus and Thomson Reuters). 5/27 of these journals tended to publish MOOC-related articles (i.e., Computers & Education, British Journal of Educational Technology, International Review of Research in Open and Distributed Learning, Distance Education, and Internet and higher education).
The other 9 journals were included since they published MOOC related research such as "European Journal of Open, Distance and E-Learning" and "eLearning papers". The study had to be an empirical or review study. We excluded studies from the synthesis if they failed to provide precise research questions or objects of research and methodologies. Studies that are of engineering orientation which addressed topics such as software development or software engineers and platform development, conceptual papers, doctoral thesis and books were also excluded. We included articles according to three criteria:  (table 1). These 23 previous research reviews were also retained so as to provide some insights into the trends already observed in the literature. The number of the founded articles for each journal is described in the table above. The table reveals that most articles were published in International Review of Research in Open and Distributed Learning (n=27) followed by Computers & Education (n=13), Distance education (n=9), American journal of Distance Education (n=8), British Journal of Educational Technology (n=8) and Internet and Higher Education (n=8). The articles have been the subject of an in-depth analysis based on an analysis grid which facilitates the coding of data. The grid included both multiple-choice and open-ended questions and had three sections: writing characteristics (references, authors' description, authors disciplines, type of document, nature of document, the level of education under consideration); conceptual framework adopted (theoretical foundations, research concepts, questions or objectives); information on the empirical elements of the research (data collection method, data processing method and key findings). The grid was structured as reported in Table 2.

Analysis method
For the responses from the closed response questions presented in the grid, we used a descriptive statistic (E.g calculating the frequency of the distribution of data collection method, analysis method, etc). For the data issued from each open-ended question such as research question or goals, frameworks, we used a thematic content analysis technique. Indeed, the excerpts of the articles, in each item of the grid, were collected and read thoroughly in order to suggest thematic categories. These excerpts were divided into units of meaning (shorter segments of text that can be associated with a category). For instance, for the research that is questioning "What are the milestones that predict course completion? [62] was assigned to the category "predictors of retention", "What do key stakeholders (students, faculty, online support services, coordinators, and leaders) tell us they have learned from the AOLE experiment? [58]" was assigned to the category learning experience and finally this one "What kind of grading, if any, do professors utilize in MOOCs? [56]" was attributed to the thematic category design of MOOCs. We note that the categories must be explicit and mutually exclusive (each unit of meaning must only fall under one category) and they must make sense in terms of research in the field. Results

Research questions or objectives
We identified four categories of research objects in the 77 selected empirical articles: the learning process, predictors of retention, learning experiences and the design of MOOCs. We present below some of the results obtained. The learning process category is the most presented (38.96%) followed by the design of MOOCs (23.38%), learning experience (20.78%) and predictors of retention (16.88%) (figure 1).

Fig. 1. Distribution of research objects in MOOCs studies
The articles related to the learning process deal with determinants of learning and interactions in the MOOCs. The topics in these studies included how participants selfregulated their learning (6 studies), their motivations for participating or learning in a MOOC (5 studies), their behaviour in the course (4 studies) and the impact of educational and technical tools on learning (7 studies). The interaction in the MOOCs focused on the modes of interaction that characterised their participation in discussion forums (2 studies), the processes of interaction between users and organisers in the case of cMOOCs (1 study), the modes of communication privileged by learners (2 studies), the relationship between different levels of interaction or the participant behavior and modes of interaction (2 study) and their role in the co-construction of new knowledge (1 study).
The studies related to the predictors of retention of MOOC included analyzing the evolution of retention and/or participation of learners (1 study), typology for determining success and drop out in MOOCs (1 study), the impact of the learning strate-gies and motivation (6 studies) or the characteristic of learners (5 studies) on the level of success.
The topics of learning experience studies cover the perception of teaching, learning and design of MOOCs within MOOCs (12 study), the relationship between multiple learning styles and learner's intentions to use MOOCs (1 study) and the various difficulties encountered by students in MOOCs (3 studies).
The topics in studies related the design of MOOCs addressed efficiency of integration of MOOCs in traditional or classical teaching by examining the feasibility of using MOOCs as a learning environment with undergraduate students (2 studies), evaluation in MOOCs (5 studies), teaching methods adopted in MOOCs (7 studies) and the use of technological tools in MOOCs (4 studies). Indeed, the evaluation included research objects related to the types of evaluation proposed in MOOCs, the assessment by students of the effects of teaching and learning via a MOOC relevance of the content and the teaching methods. The 'teaching methods adopted in MOOCs' focused on analysing how instructors designed, developed and delivered MOOCs contents. Using technological tools in MOOCs' included research objects that focused on the methodological design of the MOOC, the identification of relevant post in the forum discussion and the access frequency of the elements of the platform.

Conceptual frameworks
To respond to the multiple research questions noted, the empirical studies undertaken adopted a variety of conceptual frameworks. Among the articles that have adopted these conceptual frameworks (42/77); 26 of these were explicitly presented and the other 16 were identifiable through the text. Interestingly, 35 papers out of 77 didn't employ any theoretical framework to respond to the research questions related to MOOCs, which equals to 45% of all the 77 empirical research. Table 3 shows that most of the research studies used one (n=30) framework or theory. A minority of studies used two (n=5), three (n=1) and four frameworks or theories (n=2). The conceptual frameworks refer mainly to learning theories such as self-regulation learning strategies and social learning ones (n=13). Table 3. Conceptual Frameworks (the other 31 frameworks are described in Appendix )

Articles
Conceptual framework or theories mobilised Almatrafi et al. [41] -Model to identify "urgent" posts that need immediate attention from instructors Chang et al. [34] -The relationship between learning style and learning experience  Figure 2 shows the distribution of studies per each methodology. We can see that the most common research methodology type is quantitative method (57,14%) followed by qualitative (27, 27%), and mixed method (15, 59%).

Fig. 2. Distribution of research methodologies in MOOCs studies
Findings concerning research methodologies used in MOOC research have a same trend compared to previous findings of the review conducted by Zhu et al. [10], who reported that most studies used quantitative research methods followed by mixed and qualitative research methods. The researchers largely focused on questionnaire/survey (43) (figure 2) which is in line with finding of the studies by Raffaghelli et al. [9] and Zhu et al. [10]. This collection tool is adopted by researchers to study the learners' activities and their experiences in MOOCs. The second largest collection method is data tracking for learning analytics (28) which is used to analyze co-creation of the content by the learners, the strategies adopted by MOOC instructors to teach contents and the interaction between participants in MOOCs. The third frequently used method 44 21 12

Quantitative
Qualitative Mixed is interview which is generally used to complete the questionnaire (21). Virtual ethnography (4) and narrative inquiry are the least frequent collection methods (1).

Fig. 3. Distribution of data collection methods in MOOCs studies
*One study may adopt more than one collection method. We note that learning analytics refer to the analysis and the reporting of data about learners and their contexts issued from SQL files, Quiz, Log files, discussion forum, etc.
Descriptive statistics were the most frequently used data processing method. Thematic/Discourse analysis was the second largest data processing. This result corresponds with findings in other studies. For instance, Veletsianos and Shepherdson [6] and Zhu et al. [10], stated that descriptive statistics were reported in almost all papers that they examined. Little use was observed in other methods (phenomenology, social network analysis, grounded theory, psychometric analysis and sentiment analysis).

Discussion
The 100 studies reveal interesting current trends related to the research on MOOCs published between January 2018 and January 2018 in peer-reviewed journals. These studies have been published in an assortment of peer-reviewed journals. Surprisingly, more than one-fourth of the research studies were published in International Review of Research in Open and Distributed Learning (IRRODL). Unlike the review of Raffaghelli et al. [9], literature reviews are quite present in this study (23 reviews). The findings of these 100 examined studies indicated that research questions have essentially focused on learning process (38.96%) followed by the design of MOOCs (23.38%), learning experience (20.78%) and predictors of retention (16.88%). More importantly, this result corresponds with findings of other studies. For instance, Raffaghelli et al. [9] stated that studies have focused on learning processes and pedagogical issues, impact of MOOCs on pedagogical theory and on educational institutions.
Although the conceptual frameworks used in the empirical studies vary widely, they have remained centred on learning issues. Learning processes were an important concern for researchers analysing MOOCs. This observation may be explained by the fact that given the open nature of a MOOC; researchers have focused on two phenomena. On the one hand, there is minimum direct interaction between instructors and learners, which forces learners to self-regulate their learning in MOOCs. On the other hand, interactions with peers with varying degrees of expertise, particularly in cMOOCs are centred on compensating for the inability to interact individually with instructors.
Although design of MOOCs is the second most frequent research question, a small number of studies assessing how learning occurs in MOOCs analysed the modalities and technologies used for assessment in MOOCs. Similarly, very few studies assessed the integration of MOOCs in traditional courses (blended MOOCs in a face-to-face environment). It is also worth mentioning that, as Veletsianos and Shepherdson [6] have noted, there is a paucity of studies examining content in relation to the experiences of teachers. As a result, future research can consider carrying MOOCs studies dealing with the manner in which MOOCs have been integrated in traditional courses and the difficulties experienced by teachers when offering, designing or teaching in MOOCs.
Results indicated also that most of the frameworks adopted in empirical researches generally have a focus on learning theories specifically self-regulated learning theories and social learning ones which is in line with findings of Bozkurt et al. [22], who stated that studies are framed by theories focusing on online learning communities and distance education learners. Although these frameworks are employed by many researchers to understand learning process within MOOC, they still centered on learners and they don't consider theoretical standpoints that are focused on knowledge which help researchers develop a deep understanding of the manner in which learners apprehend concepts. For instance, modeling the disciplinary structure of a content or the cognitive levels required to learn content through MOOC (what knowledge and competences are required to learn a specific discipline for example algorithmic, mathematics, etc.) (Svinicki, 2010) [76].
Finally, findings revealed that more than half of studies adopted quantitative methods and less than one sixth adopted mixed methods. As a result, surveys were the most used data collection method and descriptive statistics were the most adopted data analysis. This focus on quantitative research methods can be explained by the opportunities provided by platforms for researchers in order to easily access and study vast amounts of data [49].

Conclusion and Implications for Future Research
This study explored 100 studies published in peer reviewed journals between January 2012 and January 2018. The findings reveal that research questions have essentially focused on learning processes, design of MOOCs, learning experience and predictors of retention. The conceptual frameworks have focused on learning issues. Quantitative methods are predominant and the questionnaire is the preferred data collection tool. As a result, data processing has privileged descriptive statistics.
This synthesis reveals a number of implications for future research. Qualitative research focused on actual practices (based on ethnographic methods) may help promote a greater understanding of the learning experience of participants. This type of study may also help explain the integration/connection possibilities of MOOCs in traditional training trajectories. The follow-up of learners in these open and massive training systems benefits today from the development of tools and methods for systematic text mining, automatic language processing and recommendations generation combining user profiles, content descriptions, classification, filtering, trace analysis and so on. Consequently, a thorough study of leaning analytics deserves a particular attention in future research. Using theoretical frameworks focusing on knowledge by researchers, may help define the learning process and the factors that influence it and determine the difficulties experienced by participants when learning specific concepts. By addressing these challenges, instructional designers could improve the design of MOOCs.