Internal and External QA in HE LA Tools and Self-Evaluation Report Preparation

Learning Analytics (LA) and tools for intelligent analysis of data accumulated in the information systems used in higher education institutions (HEIs) allow quality experts to increase the effectiveness of processes for monitoring, quality assurance and evaluation of training. The paper presents LA model and a correspondent software tool designed for the needs of quality experts in Bulgarian higher education institutions. The tool allows them to monitor and improve the learning process. However, the experiments presented here show that the tool can also significantly assist in the preparation of selfassessment reports for internal and external quality assessment in HE. Research and experiments with the model and the LA tool under consideration are conducted on the basis of the information infrastructure of a typical Bulgarian university – University of Plovdiv “Paisii Hilendarski”. Keywords—Learning Analytics, Intelligent Data Analysis, Decision Making, Self-evaluation Report, Higher Education.


Introduction
In recent years, extraction and analysis of data produced by participants in learning processes has become increasingly important and has led to the emerging of a new research field, called Learning Analytics (LA). The evidence for the growing interest in LA is a large number of publications during the last decade, showing the rapid growth of techniques, methods and applications of LA [1]. LA refers to the process of collecting, evaluating, analysing, and reporting organizational data for decision making [2] to improve learning processes and optimize the environments in which they occur [3,4].
Contemporary higher education institutions (HEIs) collect data for students and their achievements. Much of these data that can be used for LA comes from the learning management systems (LMS) and student information systems. When conducting e-learning data for students' activities are stored in the LMS database, such as assignment submissions, answering self-assessment or assessment quizzes, participation in group discussions, reads of learning resources, etc. The final grade for each discipline is stored in the database of the student information system used in the HEI.
These data can help HEI improve the quality of courses and teaching methods, develop curricula, track student performance and identify students who need support, decision-making based on evidence [5,6].
LA tools are a possible way to ensure quality and improved efficiency, which is crucial for many HEIs [7]. Many HEIs worldwide [3,8,9,10,11,12,13,14,15,16] have already used LA tools to track data for institution's work, curriculum, teachers, students to improve the quality of learning, student retention, enhance student performance, deliver early interventions and immediate feedback and make significant progress in improving the learning processes. Many of these tools are developed for the needs of students, teachers and managers of institutions and provide them with improved indicators to measure the effectiveness of teaching methods, learners' engagement in the LMS, and the effectiveness of learning process using technology [5].
Less attention has been paid to the possibility for using LA tools in internal and external quality evaluations in HEIs. LA and tools for intelligent analysis of data accumulated in the information systems used in HEIs allow quality experts to increase the effectiveness of processes for monitoring, quality assurance and evaluation of training. There are successful experiments for dynamic quality assurance and evaluation of higher education through automated data retrieval from information systems and developed model and tools for automated accumulation and aggregation of data [17]. Since the primary objective of LA is to improve the quality many systems for quality evaluation of learning in HEIs, developed by independent institutions (e.g. ENQA, EFMD, Quality Matters Program, ACODE, EFQUEL, NEAA, etc.) contain indicators, typical for LA models. These indicators [18,19] allow the evaluating external experts to give a real assessment of HEIs for students' activity and success rate, teachers' activity and recommendations for improving the quality of the training at HEI. During the evaluation's procedures, HEIs have to write down self-evaluation reports with a set of proofs. Many of these proofs, especially when it comes to the assessment of distance learning [18], require collecting, analysing and interpretation of students' big data. In this regard, quality experts (e.g. members of quality committees) can use LA tools to generate proofs when they write down self-evaluation reports for external quality evaluation by independent agencies.
The paper presents LA model and a correspondent software tool designed for the needs of quality experts in Bulgarian HEIs. The tool allows them to monitor and improve the learning process. However, the experiments presented here show that the tool can also significantly assist in the preparation of self-assessment reports for internal and external quality assessment in HE. Research and experiments with the model and the LA tool under consideration are conducted on the basis of the information infrastructure of a typical Bulgarian university -University of Plovdiv "Paisii Hilendarski".

LA Model with a Set of Indicators
On the basis of a literature review in the field and an investigation of quality requirements in higher education is a model with a set of indicators that serve as a busi-ness logic basis of the developed LA tool (see Section 3). This model define what type of data should be collected from the institutional information infrastructure that quality experts of the institution will be able to use for continuous improvement and for ensuring more student-focussed provision of higher education.
The model is developed correspondingly for the needs of quality experts. It consists of measurable indicators allowing the quality experts to track data for students' learning or training for different purposes, e.g. monitoring, analysis, forecast, intervention, recommendations, etc., but finally to improve the learning and teaching processes. The model is built as hierarchies of measurable indicators of different levels. Level 1 contains (see Fig. 1) five indicators, each of which contains one or more indicators from Level 2.

LA Tool Description
Based on the developed model, a corresponding software system, called Learning Analytics Tool for quality experts (LATqe) was designed and implemented.
As a result of an analytical review of software solutions for extracting, analysing and visualizing data from various information sources, technologies and tools for software development were selected. The LATqe tool is developed by the integration of existing software solutions, namely JasperReport Server and JasperSoft Studio tools (developed by TIBCO JasperSoft) and the software framework Dynamic Presentation Framework (developed by a team working at the University of Plovdiv).
The JasperSoft Studio provides a rich set of instruments for design of report templates which can be filled out with data retrieved from a variety of data sources (relational databases, big data sources, or other types of database systems). Along with JasperReport Server it can be used to create powerful report publishing workflows.
JasperReport Server provides opportunities for organizing structured repositories, accessing data collections with a different type of organization (incl. custom -DB, XML, CSV, Hibernate, POJO) and using them as data sources for the needs of Jas-perSoft Studio when generating reports, storing reports and presenting them in the preferred by the user form. The server also offers powerful tools for integration with various software applications through shared web services.
Dynamic Presentation Framework (DPF) is a software framework for visualising dynamic user-driven views of digital objects in a web browser. DPF also allows connection to external sources through web services.
The architecture of the LATqe tool (see Fig. 2) follows the standard type of 3-tier architecture with well-known three layers -Presentation, Application and Data layers.
In the basis of the LATqe Presentation Layer is the software framework DPF, through which the user can request the generation of a report by a chosen template and view the result of the request (visualized report). DPF (using XML Parser and Style Control Module functionalities) allows users (quality experts) through predefined conditions to modify some view attributes such as color, font size, etc., to visualise the report in the web browser in a user-friendly way.
By the report templates design tool JasperSoft Studio is implemented the core functionality of the Application Layer of LATqe and its business logic.
Key elements of this functionality are modelling of the three developed models for the needs of quality experts (see Section 2) and acquisition of values for the model' indicators of different levels from digital footprints left by students and/or teachers during training in each course and/or by inspectors (responsible for programme training) in LMS, student information systems and/or other systems of HEI information infrastructure.

Fig. 2. LATqe architecture
Therefore, in the first stage, the institutional information infrastructure (including university digital repository, databases of university information systems, learning management systems Moodle, performance support system DIPSEIL [20], etc.) of a typical Bulgarian university (namely the University of Plovdiv "Paisii Hilendarski") has been analysed. The analysis has been done in terms of its use as a data source (about the training, the results achieved, etc.) when forming values of the indicators from the proposed model.
In the second stage, templates of reports were designed using JasperSoft Studio based on the proposed model as sets of indicators (see Table 1) for the needs of quality experts when generate reports needed for internal and external quality evaluations. All developed templates of reports have been stored on the JasperReport Server. Jas-perReport Server plays an intermediate role between the three architectural layers: • DPF requests the REST services of JasperReports Server to run a chosen template and generate a report through the Service Client; • The JasperReports Server Web Service interface responds to HTTP requests from the client application.
Data Layer of the LATqe application includes various databases of the institutional information infrastructure (student information system, Moodle, etc.) as well as the JasperReports Server repository itself. JasperReport Server addresses them to retrieve the necessary data when generating reports.
LATqe allows quality experts to generate dynamically reports allowing them to monitor and evaluate the quality of learning and achievement of students and faculty staff in all courses from all bachelor/master programmes for the needs of evaluation procedures. LATqe allows for each indicator of the proposed model to be generated reports with retrieved values from the information systems. Generated reports contain tables and diagrams and allow users to perform various analysis on the retrieved data.  Fig. 3 presents a part of the generated report for Indicator 1.3 with input value for course Object-oriented design and programming. The report shows the number of views of each resource in the course and the average number of view of each reports per student. Thus, e.g. if there is a significant decrease in the number of views during the learning process, the quality expert can take timely measures to improve the quality of learning materials and reduce the drop out of students. This report can be used as evidences about the use of learning resources by students during internal and external quality evaluations.

Conclusion
LATqe will be provided for real-time testing at the University of Plovdiv. The experiments will be carried out during the e-learning processes. On the basis of the results, users will take measures to improve the quality of training and students' achievements. Feedback from all users will be taken into account in the development of the final version. The final version of LATqe will be integrated into a single system, which will be the first for Bulgaria integrated system of intelligent analysis of education data, meeting the requirements and needs of all stakeholder groups.
The paper is partly supported within the project MU19-FTF-001 "Intelligent Data Analysis for Improving the Learning Outcomes" of the Scientific Research Fund at the University of Plovdiv "Paisii Hilendarski".