Development of an Information System to Enhance Students Reading Literacy

The research study at hand aims to answer the question, whether an innovative information system can be implemented that will help to enhance reading literacy of elementary pupils. Based on predefined reading tests this web-based system evaluates the reading literacy of pupils. It consists of two primary parts; the system that creates and evaluates such reading tests and the test platform itself. In order to assess the system a field test was conducted. Therefore it was tested in two school classes. In the course of this study reading tests were carried out and retrieved data and results were evaluated. Despite some minor usability problems, the system performed very well. The test system delivered good estimations of the reading capabilities of single pupils and classes. Of special interest is the system’s analysis of the created reading tests since the system is capable of evaluating reading test according to their difficulty.


A. Learning analytics
Learning analytics in general is a great promise for teachers. Phil Long and George Siemens stated [1] that "the most dramatic factor shaping the future of higher education is something that we can't actually touch or see: big data and analytics." With collected data about the acquired and mastered skills of pupils it might be possible to get feedback concerning their own teaching method. Furthermore in preferred cases the teacher reconsiders the applied methods of teaching according to their pupils' strengths and weaknesses in order to achieve satisfying results for pupils as well as for the teacher.
The field of learning analytics is a rather versatile and interesting one and has many different research approaches [2]. Some concentrate on learning efficiency others on the application of algorithms on the data to see which learning pattern is used [3]. Of course one of the main questions addressed by Erik Duval [4] is about what exactly should be measured to get a deeper understanding of how learning takes place.
Even though learning analytics is a more than interesting topic and the benefits for teachers in terms of pedagogy would be extensive also different constraints (like data security issues) must be discussed [5].

B. Tablets in class
In the current heyday of developments in technology it comes as no surprise that many children handle a tablet better than many adults. Children use tablets in everyday life, because of user friendliness and the great number of applications designed for them. Furthermore the ownership of personal mobile devices is increasing dramatically in the last years [6].
Tablets are a rather new technology but the potential to use them as an educational tool in class was recognized soon after their introduction [7]. Even though tablets were not specifically designed as a teaching tool, more and more schools use tablets in the classroom. [8] [9] Due to this fact, tablets generally offer a great opportunity to create a testing platform. The fact that tablets are simple to be used and the possibility to connect to the Internet form a great base for centrally administrated tests [10]. This combination enhances accessibility as well as usability for both, teachers and pupils.

C. Research question
The great benefit that could arise from combining learning analytics with the use of tablets in classrooms lead to the question this research study aims to answer: How can the prototype of an information system, that has the ability to enhance reading literacy of pupils in combination with learning analytics, be implemented? The decision to focus on reading literacy is based on the fact that reading is one of the most important intellectual skills people have to learn in their life.

A. Concept of the system
After research in the field of reading literacy of pupils, it was found out that only by carrying out appropriate reading tests, is it possible to assess reading literacy properly. Concluding that such tests are the only way to get data that determines capabilities of pupils, a platform to conduct such tests had to be developed.
Due to the fact that these days, classes with tablets for every pupil are increasing the ideal platform to realize a quick and accurate testing was a tablet approach.
Currently there are three main operating systems for tablets, iOS maintained by Apple Inc., Android maintained by Google Inc. and Windows maintained by Microsoft. The fact that iPad from Apple Inc. is used predominantly in school-classes in our country, led to the decision to choose the iPad as the main testing platform. Additionally, a web-based version is created since it could PAPER DEVELOPMENT OF A PROTOTYPE OF A SYSTEM TO ENHANCE STUDENTS READING LITERACY be an advantage in terms of distribution if no iPads are available.
Due to the fact that the testing platform itself cannot be used without an appropriate administration-tool also a socalled backend will be necessary which provides the reading tests. The idea is that teachers can create their own reading tests, distribute them in their classes and in return can see a quick analysis of the performance of their pupils who took the tests. A further feature of the administrationtool allows it to share the created tests through the system. This enhances the tool with a social note and supports interscholastic collaboration. Therefore the information system to determine the reading ability is based on two main components; the (mobile) testing-platform and the web-based administration-and deployment-tool.

B. Resarch design
The research design strongly follows the approach of information system prototyping. According to Maryam Alavi [11] as well as Orland Larson [12] prototyping is based on four steps: identifying basic requirements, development of a working prototype, implementation and usage, and revision.
Therefore the implementation of an administration-and deployment-tool as a web-based application and the implementation of the testing platform as an iPad application as well as a web-based version is the main goal of this research study.
In order to evaluate whether the implemented systems meet the expectations and provide benefits for the assessment of reading literacy in school a first field study had to be conducted. This study was carried out in an elementary school in order to see whether an assessment of reading abilities would be possible.

III. IMPLEMENTATION OF THE PROTYPE
The implemented prototype is called 'Lesetrainer' and is designed to be used in elementary school. Pupils from ages 6 to 10 were supposed to perform the reading tests. In order to store students' results as well as the tests created by teachers the 'Lesetrainer' needed a user management. To avoid further registration the 'Lesetrainer' uses the central user management provided by TU Graz for all their learning applications 1 .

A. Reading Tests
In order to provide proper testing, already available reading tests were collected and reviewed. The tests are all standardized and normed. These properties generally enable comparison between them.
A review of these tests showed that despite being different all of them address the same basic skills. Therefore it can be assumed that the basic skills required by those tests could be taken as indicators for the reading literacy. The skills shared by these tests are as following: speed, understanding and comprehension. Reading rate or the ability to assign a word to the proper picture out of four can be taken as an indicator for the decoding speed. The ability to understand and comprehend a sentence or a whole text semantically can be measured by asking ques-tions or by asking pupils to complete sentences. [13] [14] [15] [16] As a consequence the system provides three types of reading tests.
1. A reading rate test, which consists of short sentences. These sentences have to be checked on whether the content is true or false. Therefore, it has to be marked as such after reading. In order to increase difficulty the test can be limited to a prior defined number of minutes (the shorter amount of time the more difficult such a test is). 2. The second test addresses the skill to understand a sentence. Here the pupil has to complete a sentence. Out of four possible words, the correct one must be chosen. Like the reading rate test this one also has a time limit (shortening the time available adds to difficulty). 3. The third test assesses reading comprehension. Here pupils have to read a short text. After reading the text four sentences are provided with only one sentence, which matches the short text (the sentence either describes the text, adds to the text, etc.). Pupils have to choose the right sentence. Different to the other two tests, this one does not have a time-limitation. The test person can take as long as he/she needs to finish the test.

B. Collected data
The main collected data consists of the amount of correct and incorrect answers. Further the amount of time needed to complete a part of a test is measured and saved. The summary of the times needed to complete single parts of the test is added up in order to measure total time required for completion.

C. Administrtation and deployment tool
As stated above the administration and deployment tool has three major functions. First the creation of reading tests, second the deployment of those and third the processing of the retrieved data including presentation of this data to the teacher. This tool is realized as a web application and in order to achieve this, a framework and a database is used. The Yii 2 Framework was chosen for the implementation and a MySQL database stores the created reading tests as well as the results of the completed tests. The Yii Framework is based on PHP and is executed server-sided. To communicate with the client applications the administration tool contains a web service. It delivers the data in the form of JSON objects. The reason for using this is that it has a simple format and it can efficiently and easily be decoded.

2) Deployment of tests
Deploying a reading test is a simple task. The teacher choses the test he/she wants to use, presses the deploy button. After this the school class, retrieved from the usermanagement can be chosen. Finally, the teacher has to specify a date. This date marks the day the reading test is available for students.
The system internally creates database entries for every pupil and the integrated web service provides a list of available tests for every user. The list only contains tests if the deployment date is the current day or is behind the current date.

3) Analyse of the results
The third major function of the 'Lesetrainer' is the analysis of the test results. There are two ways the analysis can be viewed. First the teacher can see the overall result of the class and second it is possible to see the individual results of each pupil.
The evaluation of the test is based on the number of correct answers. The number of correct answers is automatically presented in percentage. The overall class result is calculated by the average. The average is calculated in two ways by the arithmetic mean and by the median with the absolute deviation to foster the final interpretation.
a) Pupil's analysis It is possible to get detailed information for each pupil. The teacher can choose out of four categories of overall results of one pupil. There is the average of all kinds of reading tests, or only of a specific kind of test.
In every one of the four possible views both the average percentage as arithmetic mean as well as the median with deviation is calculated. To allow a quick evaluation the calculated result is colored from red (0%) to green (100%). The deviation indicates the constancy of the results.
A diagram (Figure 6) shows the trend of the single results a pupil achieved in the test. Furthermore it is shown by the arithmetic mean and median of the class to which the student belongs to. This allows it to compare the students result against the class's result.
Besides the average result and the diagram, a table with the option to get the detailed results is provided. For every single task of the reading test the detailed results specify the given answers and the time it took the pupil to give it. b) Class Analysis Similar to the pupil's analysis it is possible to choose one of the four views. Here the average results are calculated from the single results of the individual pupils.
A diagram (Figure 8) shows the trend with the mean and median of the class.
Along with the two average results of the class a list of all the pupil's individual results highlighted according to their average result is shown. Also a list of all the deployed tests with the option to limit the average calculation is shown. The list of tests allows the teacher to access detailed information of the deployed tests.
c) Test Analysis The retrieved information during the test can be used to create a detailed analysis of the deployed test. The test analysis view does exactly this. It shows the average number of questions that were answered correctly, incorrectly and the ones that did not got answered. The average time a student required to give an answer is also calculated.
Further important information is the number of students who have not started the test yet and the number of students who could not finish the test because they ran out of time. This is also important for the validity of the current result.
Two diagrams use the retrieved data to visualize the results of the single questions. The first one (Figure 9) shows the number of correct, incorrect and not given answers. The second ( Figure 10) compares the average time needed to give a correct answer to the average time needed to give an incorrect answer. These diagrams allow an interpretation over the single tasks the student was supposed to do. It clarifies whether the pupils were able to understand the question or not.

4) Web service
To communicate with the client applications, a web service was implemented. Figure 1 shows the communication between client and server. It provides the following functions.
• Authenticate user: Forwards the authentication request to the user management. • Save result: This function saves a given result to the database ( Figure 2). • Get test list, returns the list of available reading tests according to the given user ID and the category of reading tests. • Get test provides the delivery of the whole reading test by the given user ID and test information. • Quick test implements a function, which returns a random reading test from the public list. This function is used in the applications to show how the reading test works without valid login information.

D. iPad Application
As we defined that there will be three different reading tests, we decided to implement three different iPad applications. The application will implement a simple and easy to use design ( Figure 3) so that the pupils will not get distracted. The three designs simply differ in the color scheme. In the source code the differences are limited to one configuration parameter, which states the category of the reading test and the view, which presents the task to the testing person. The three applications can be found in the appstore by the name 'LesenSpeed', 'LesenSatz' and 'LesenText' The iPad application communicates with the administration tool via a web service. This web service was designed to make it possible, that the test can be performed once downloaded, without internet connection. The app requests the whole test from the web service. If the iPad has no connection to the Internet after the test is finished, it stores the result and on the next launch of the application it tries to upload the stored results again. Figure 4 shows the flowchart of the application.

E. Web-based Testing application
In order to not only limit the usage of the 'Lesetrainer' to school classes with iPads, a web-based version has been developed. The first approach was to integrate it directly into the administration tool. But various problems soon arose. The problem was the server sided framework Yii.
To perform reading tests the measurement of time is necessary and this can only be done on the client side. Therefore a new approach had to be found.
Ember.js 3 was the solution to this problem. It is a Ja-vaScript framework and runs only on the client. The three applications were re-implemented in JavaScript. The fact that it runs on the client made it necessary to implement communication functions.
The Ember.js application now uses the same web service as the iPad applications. Figure 5 shows the flowchart of the web-based application

IV. FIELD STUDY AND EVALUATION
The field study was conducted at an elementary school in Vienna the capital of Austria. This school has classes where every pupil was equipped with an iPad. Two school classes agreed to perform reading tests on two different days. Each class performed a reading test of each category.

A. 'Lesetrainer's' Usability
On the first day all students were introduced to the applications. Quickly one problem arose. The students had to log in in order to get the list of tests. The problem was that the students were not familiar with the password input and this caused some problems.
It was possible to implement a solution early enough so that on the second test day this feature could be tested. No problem with the login was observed on the second day.
Conversations with teachers and test users showed that the administration and deployment tool has some problems with its usability.

B. Evaluation of the Analysis
The two testing days delivered data that would allow an evaluation of the analysis' functionality that the 'Lesetrainer' implements.

1) Student analysis
The collected results on the two test days point out that the 'Lesetrainer' is capable to visualize student's results and an assessment of the reading capability is possible. Figure 6 shows a student result with a high estimation of reading literacy. Interesting is the low deviation which indicates a very steady performance of the student.
Interesting on the diagram in Figure 7 is the fact that the student shows an improvement of reading skills in the last reading test.
Class analysis As mentioned above the average result is calculated in two different ways; the arithmetic mean and the median. The class analysis shows the difference in the two ways of calculation. The two calculated figures partly deviate widely from each other ( Figure 8).
The problem is that the arithmetic mean is greatly influenced by outliers. The median however is not affected of such outliers. This is the reason why there can be such great differences. 2

) Reading test analysis
The field test showed the potentialities of the analysis of the test results as mentioned above. One reading test showed very interesting data: The evaluation of the number of correct, incorrect and not given answers pointed out that the students had mainly problems with two questions (Figure 9). These two had clearly more incorrect answers than usual.
This information itself only says that there were problems. In addition the evaluation of the response time allows a better interpretation about the problems with the questions. Figure 9 shows that questions number 2 and 4 are the ones with the high rate of incorrect answers. Figure 10 shows the response time for incorrect and correct answers.
On the one hand it is obvious that the average time to give a false answer is very high at question number 2 ( Figure 10). In this case it is about 20 seconds and about 6 seconds to give the right answer. On the other hand at question 4 there is no significant difference in the two response times.
This additional information shows that the students have different problems with the tasks. The figures of the response time at question 4 show that there must be some textual problem with the task.
The observed behavior of the response time at question 2 show that the students have a problem understanding the question in the way it was meant.

V. DISCUSSION
The field test pointed out some weaknesses with the design of the iPad application, the usability of the administration and deployment tool. The functionality of the 'Lesetrainer' proved to be a success. All the desired functions could be fulfilled. The analysis of the retrieved data showed the high potential of the 'Lesetrainer'.

A. Password input problem
Some pupils had problems to log in to the iPad application. The observation showed that the pupils had problems to type in the password. More precisely pupils were not able to check if their password was typed in correctly. Some of them were not familiar with the behavior of a password field.
The replacement of the typed in character with a placeholder caused the problem. The solution to this problem was clear a button that toggles the password field from secure text to plain text. This made it possible that pupils and teachers could check if the right password was typed in. As already said, the implementation of this solution fixed the issue and on the second testing day there was no further problem.

B. Administration and Deploymenttool
Some problems with the usability of the administrationtool could be observed. Some users had problems to find all the functions right away. This leads to the conclusion that the navigation of the administration-tool need to be improved. In order to make this happen a usability test should be performed. This test would then point out where the major problems and specific improvements could be realized.

C. Functional Test
Besides the problem with the password in the iPad application no major problems did occur. The desired functionality was provided and the field study showed that the prototype is working according to expectations.

D. Analysis of retrieved data
The observations of the retrieved data show that the 'Lesetrainer' is capable of evaluating the reading literacy. The analysis of the tests showed interesting facts about the difficulty of single tasks and pupils' performances. This interpretation of the data can in the future help teachers to improve their tests or to change their approach to teaching reading.
The class' und pupil's analysis makes it possible to determine if the class or the student may have problems with reading. Knowing that there is a problem makes it easier to set actions to compensate them. Special and purposeful actions can be taken in order to solve the problem.

E. Future work
As mentioned above, a usability test could be a big benefit in order to improve the 'Lesetrainer'. Also, in order to ensure that the gained results can be compared to each other the tests have to be normed. In order to do this, the tests have to be examined in detail to get standardized. Only results of normed and standardized tests can be compared to each other.

VI. CONCLUSION
The usage of iPads as a testing platform in school classes is a promising approach for the future assessment of student reading capabilities.
The 'Lesetrainer' showed how a system might look like to achieve an evaluation of reading literacy. The analysis of the results of different tests showed some interesting facts. It might be possible to add some other measurements to the system to improve the analysis opportunities.
A transfer of this concept is plausible. It is possible to use the system as a simple testing platform. Teachers can perform exams with the system. The system makes grading pupils possible and by analyzing the tests the quality of the tests can be improved.
On the whole, the 'Lesetrainer' provided all desired functionality and showed some interesting facts. Therefore the 'Lesetrainer' is a proof of concept for a system to improve the reading literacy of students.