A Technological Tool to Optimize Educational Assessment

In this paper we consider a table of specification problem which allows teachers to specify number of questions of a test based on weights and importance of di!!erent topics and objectives. In order to use the table of specification properly we have modeled it as an optimization problem. To facilitate the task of the teacher, we have developed a technological tool (software) that automatically optimizes the table of specification. The teacher or the decision maker must simply enter the necessary data.


INTRODUCTION
During their professional practice teachers assess students frequently.For a summative evaluation, the preparation of a sequence of an exam or a final exam should not be made hastily.The questions to ask and the tasks to assign should be consistent with what has been taught, and what has been taught must be taken from the learning objective before the sequence or the period of teaching and learning begins.In other words, the planning of assessment processes must accompany that of activities of teaching and learning.We need a tool to bridge the gap between the views subject and objectives on the one hand and questions or problems of the examination, on the other hand, given their representation in numbers and their relative importance.This multiple use tool is the table of specification (see [1] for instance).
In the literature there are many works on admission policies and the selection decisions being made on the basis of standardized test scores and grade point averages (see for instance [2-3-4-5-6-7-8]).A Table of Specification is a two-way chart (see for instance [9]) which describes the topics to be covered by a test and the number of items or points which will be associated with each topic.To make it, the teacher must complete two tasks [10]: • Identify the domain of content (topic) which is to be assessed; • Break the domain into levels of objectives (objective).We can use the categories of Bloom's taxonomy [11] for example.By crossing the components of each domain of content and each domain of objectives, we obtain cells which indicate, most of the time, the relative weight granted at this crossing for the summative assessment, see for instance [12].The teacher must transform these weights in numbers of questions or points.However, in most cases, the resulting numbers are real numbers.They must be transformed into integers by maintaining the weight of each cell as close as possible.This task generates an optimization problem that requires the use of artificial intelligence methods to solve.
We describe the table of specification problem and we provide an example to illustrate it in the second section.In the third section we introduce our mathematical modeling of the problem.In the fourth section we describe our technological tool to solve the problem.Fifth section is devoted to numerical experiments on some real table of specification problems.Finally concluding remarks are given in the last section.

II. AN ILLUSTRATIVE EXAMPLE
To illustrate this problem, we take a simple example.Consider an exam of geography for the junior high school graduation [13].According to the standardized test established by the ministry of education in 2006, the main topics (domain of content) of the subject geography are: developed countries models (DCM1), developing countries models (DCM2) and addressing an economic phenomenon (AEP).In addition, in each topic, there are three objectives: the use of the knowledge (UK), the use of the geographical approach (UGA) and the use of forms of geographic expression (UFGE).The degree of importance of each topic and each objective is presented in Table I.The teacher shall select a total number q of questions to this exam, for example one take q = 20.Then, we convert the percentages in Table 1 into numbers a ij of questions, 1 !i ! 3, 1 !j ! 3, see Table II, for instance a 11 = 20"15/100 = 3, a 23 = 20"11.55/100= 2.31.Since each number of issues must be an integer, then we have to convert each real number a ij , 1 !i ! 3, 1 !j ! 3, into an integer number x ij , such that the total of numbers x ij is q=20 and x ij , 1 !i ! 3, 1 !j ! 3 must be as close as possible to numbers a ij .Since this is a small size problem, it can be solved manually; a solution to this problem is presented in Table III.However, in case of large size table of specification problem, manual optimization is very di!!"cult and very expensive; we have to apply a search method (artificial intelligence) to solve the problem.

III. PROBLEM STATMENT AND MATHEMATICAL
MODELING Suppose we have a table of specification of a subject decomposed into n topics (domains of content), and skills of each domain are grouped into m objectives.Let q be the total number of questions.By converting the percentages of the table of specification into real numbers, we obtain a matrix of real numbers (a ij ), 1 !i !n, 1 !j !m.From this matrix we have to find a matrix of integers (x ij ), 1 !i !n, 1 !j !m which specifies the number of questions in each objective of each topic, satisfying the following constraints: 1.The total number of questions is q; 2. For each 1 !i !n and 1 !j !m we have either x ij =!a ij " or x ij =!a ij "+1, where !a ij " is the floor of a ij .3. In each line (topic) i, 1 !i !n, we have 4.In each column (objective) j, 1 !j !m, we have 5.Each number x ij , 1 !I !n, 1 !j !m must be as close as possible to the number a ij , 1 !i! n, 1 !j !m.By minimizing the distance between x ij and a ij for each 1 !i !n and 1 !j !m, the constraints numbered 3, 4 and 5 will be satisfied.Then the optimization problem is defined as follows:

IV. A TECHNOLOGICAL TOOL TO SOLVE THE PROBLEM
To solve the problem and to facilitate the task of the teacher, we have developed a technological tool (Figure 1) that automatically optimizes the table of specification.The teacher or the decision maker must simply enter the necessary data.The teacher must enter the number of weeks for each course, the weight of each objective and the number of questions (see Figure 2).By clicking on the button "Issues", we get the table of specification for questions, and by clicking on the button "Points" we get the table of specification for points.Figures 3 and 4 give us an illustrative.We can change the color according to our choice and we can read the guide.To find tables of specification in integers, we click on the buttons "Issues" and "Points" which are just below.We have to wait about 7 seconds, the time required for our search method (artificial intelligence) to find a feasible solution.Figure 5 gives an illustrative of table of specification decision.

A. Problem 1
Consider the table of specification of Mathematics exam for the junior high school graduation [13], presented in Table IV     In this paper we presented the table of specification problem and we formulated it as an optimization problem.Our aim here is to help teachers to decide properly the number of questions or points in a table of specification according to the weight of each topic and each objective.
To solve this optimization problem we proposed a method based in artificial intelligence, and we developed a technological tool that automatically optimizes the table of specification.The teacher or the decision maker must simply enter the necessary data.Numerical experiments show that solutions obtained by our method are generally good.

Figure 1 .
Figure 1.General view of our software Now we will explain each component of this tool.

Figure 2 .
Figure 2. Data entered by the teacher

Figure 3 .
Figure 3. Table of specification of questions in real numbers

Figure 4 .
Figure 4.Table of specification of points in real numbers

Figure 5 .
Figure 5. Table of specification decision

TABLE I .
TABLE OF SPECIFICATION OF GEOGRAPHY TEST IN PER-

TABLE II .
TABLE OF SPECIFICATION OF GEOGRAPHY TEST IN REAL and V.The corresponding solution obtained by our method is presented in TableVI.

TABLE IV
Consider the table of specification of Physics exam for the Alberta Education [14], presented in Table VII and VIII.The corresponding solution obtained by our method is presented in Table IX.

TABLE VIII .
TABLE OF SPECIFICATION OF PHYSICS TEST IN REAL