A Novel Expert Evaluation Methodology Based on Fuzzy Logic

—In order to maintain the training quality and ensure efficient learning, the introduction of a scalable and well-adapted evaluation system is essential. An adequate evaluation system will positively involve students in the evaluation of their own learning, as well as providing teachers with indicators on the student's strengths, the specific encountered difficulties and the false or misunderstood studied parts. In this context, we present, in this article, a novel intelligent evaluation methodology based on fuzzy logic and knowledge based expert systems. The principle of this methodology is to reify abstract concepts of a human expertise in a numerical inference engine applied to evaluation. It reproduces, therefore, the cognitive mechanisms of evaluation experts. An implementation example is presented to compare this method with the classical one and draw conclusions about its efficiency. Furthermore, thanks to its flexibility, different kinds of extensions are possible by updating the basic rules and adjusting to possible new architectures and new types of evaluation.


Introduction
Nowadays, ubiquitous computing marks its entry into the education system; it finds a special place in scientific and technical education. The computer, the main element of this technology, serves both as a laboratory object and as a learning tool open to new trends in learning / teaching such as distance learning platforms. However, these knowledge-based learning technologies should put learners at the center of interest; it must adapt to the contextual behaviors of the learner. To achieve this, methodological research proves to be necessary and wins over the pure tests of finding balance between the systems development and the education needs. Research must also feed on evaluation to design effective training platforms [1] [2]. It is an engineering based on well-grounded foundations of scientific research to build its own existence. Thus, didactic engineering poses its singularity not by the research objectives carried out under its banner, but by the characteristics of its methodological functioning [3]. Training is no longer considered a single activity, or even an end in itself. It is a tool that, to be efficient, is divided into several parts: predict and analyze needs, build actions according to a plan, then realize and finally evaluate. Thus the content should be maintained and modified according to the reached levels for a good regulation of learning. This objective cannot be valid without the implementation of an adequate and effective evaluation methodology.
The quality approach, through its different concepts in the education field [4] [5], aims to increase the learning and teaching efficiency. Evaluation plays an important role in this context. It is a means of quantifying a teaching unit achievement. As a result, it allows classification or ordination. It is therefore important to collect relevant indicators to assess the quality and effectiveness of the training process. A good use of these indicators (essentially inherent in any evaluation) will give birth to structures that are part of a regulatory, standby and decision-making perspective throughout the educational process [7] [8]. Also, one can say that the assessment as a method of measuring the instantaneous manifestations of the student is rarely an end in itself. Evaluating is already linked to a faculty of discerning, recognizing, differentiating, distinguishing, judging, appreciating, estimating and finally gain individual and collective evaluative practices [9]. To evaluate is also to surround oneself with the conditions of success by putting under insurance the quality of all services related to the training. The human being is able to evaluate people, groups, institutions, processes or any other subject. However, to carry out his task, the evaluator must have criteria, even though they are not explicit or uncertain. This is why evaluation is a complex field, always subject to new researches on approaches and methodologies. Ensuring the evaluation quality is a real challenge for the evaluators. The evaluation of an evaluation system is then called a meta-evaluation. On the other hand, neither the criteria nor the decisions taken have precise and exact boundaries. The nature of all involved parameters seems to be fuzzy and explicated as fuzzy sets. Thus, by employing basic rules, laid down by evaluation experts, and thanks to an inference process; judgments, measurements and information can be issued to the observatory system [11] [12]. Therefore, many kind of treatment can be implemented to regulate training, report anomalies or design intelligent systems of learning or evaluation.
In general, the evaluation's mission is to support and inform the learner, but also to assist and pilot the teaching according to the learning / training objectives. The evaluation allows tracing / retracing the learning continuum and its progression. It also serves to adjust the teaching according to its needs. The data collected by the evaluation is used to generate decisions and feedbacks on the training while valuing its strengths, correcting anomalies and managing needs according to the targeted learning outcomes. This approach is concretely focused on the learner interests. It encourages the learner to become more involved in this process. The effects on motivation and encouragement of learning are indisputable. However, it is essential to communicate to learners the results, the expectations and the criteria of any evaluation in order to consolidate their learning, valorize their strengths, regulate their learning style, engage them, make them more accountable and involve them in a new process of autonomy and self-evaluation. The difference between the learner's results and the expected results according to the training goals provides the necessary information to estimate the percentage of the objectives achievement. This gap will be used, step by step, to represent the progression of the learner's skills. This evolution will help to profile the learner in a fair and valid way. Evaluation is therefore a process of linking the elements coming from an observable and a referent to produce enlightening information on the observable, in order to make decisions about its action (decision making) and not on the results of the evaluation (decision taking). Like any other information system, evaluation is subject to bias and unwanted effects that can be described, if we used engineering terms, as disruptive.
From this perspective and to best meet all these expectations, we have developed an evaluation methodology based on heuristic methods and more particularly on the notions of fuzzy logic and neural networks since the parameters of the problematic are more suited to these methods [14]. This method aims to regulate and advance learning / teaching, both individually and collectively. It involves several parameters brought into play by the evaluation complexity as described by W. A. Firestone [15], which are sometimes pedagogical and sometimes human variables, and making use of new information technologies, electronics and also fundamental mathematical concepts such as fuzzy logic. This method emits a set of indicators on the learning progression, the teaching evolution and the detecting of possible anomalies. Thus conceived, systems based on this logic are classified as an expert systems [16] [17]. This advantage will be evident once the human parameters are included, in which intra and extra evaluation decisions [18] are based on fuzzy criteria. The assessment methodology that we have established has served as a basis for developing a variety of evaluation expert systems that serve both classroom assessment and evaluation on distance learning platforms [16].

Skills Evaluation and Fuzzy Logic Contributions
In the case of the skill-based approach, the acquired knowledge's evaluation takes on a new dimension. Indeed, tools development is no longer limited to specific and operational contents and objectives representative of the reference universe in terms of contents or objectives. It proposes one or more complex situations, belonging to the kind of situations defined by the competence, which will require from the student a complex production to solve the situation [19] [20]. The evaluation by complex situations is the most relevant in the context of most education systems, which are well inscribed in the perspective of the skills-based approach. However, it presents some difficulties in terms of social acceptability. Certainly, it is not easy to transform a culture of "by heart" or mechanical application to that of problem solving, especially since it requires different ways of correcting and communicating information, and does not offer the same character of legitimacy as the classical tests where the pseudo objective answer most often leads to selection alone. However, all these obstacles must not prevent the realization and use of such assessment tools. If not, what would be the point of declaring that one wants to develop skills if the learning evaluation system is only based on mastery of reproduced knowledge and / or isolated know-how transfer?
Evaluation tests are developed by ensuring that situations respect the parameters of the situations family and can therefore be considered as equivalent to each other and secondly by respecting the 2/3 rule [21]. In addition, each test includes evaluation criteria (minimum criteria and perfection criteria 1 ) which must be detailed for each situation with indicators leading to the elaboration of a grading scale. Taking into account all the fundamental points of a skill-based evaluation, we propose a novel methodology based on heuristic methods, more particularly on fuzzy logic and on neural networks to answer some expectations and clarify some ambiguities of this evaluation.

A Novel Evaluation Methodology Based on Fuzzy Logic
It seems that the combination of fuzzy logic and multi-criteria decision making provides a promising framework for designing an effective and flexible system adapted to the different forms of evaluation. In fact, fuzzy logic incorporates concepts that are essential for multi-objective decision-making: the idea is to compare objectives and constraints with respect to their relative importance while taking into account possible compromising effects between criteria [22]. In general, classical methods have a "sharp" or "tense" representation of the notion of a class. We note from the beginning that some methods such as Bayesian discrimination methods assign to the subject sample a probability of belonging to a class. This probability represents the occurrence frequency in the case of repeated experiments. It is fundamentally different from the notion of a fuzzy set membership.
Confronted with uncertainties resulting from a lack of information for reasons of inaccuracy, lexical imperfection and inaccurate measurement, we cannot perfectly perceive the world. There are many situations in which our perception is infiltrated by concepts that are not well-defined at their edges, like when we use terms such as too, much, more, big, bigger than, young, etc. Truth and logical exactitude are not always binary. Thus, cannot they be true only to a certain degree? As a result, they are also false to another degree. In the evaluation field, a question can be difficult, less difficult, easy or too easy. An answer, in itself, is not always either true or false. Its accuracy is graded from 0 to 100%: it can be excellent, good, passable, insufficient or very insufficient. It is the same for a student's reached level. These concepts, sometimes abstract, which one is obliged to work with in order to evaluate a subject, are fuzzy concepts perfectly modeled by fuzzy logic.
Minimum criteria are those which must absolutely be learned to certify the skill mastery whereas the perfection criteria concern qualities whose presence is preferable, but not essential.
Fuzzy logic was first introduced by Zadeh in 1965 [23]. Today, it is the heart of several research projects aiming to perfect and apply this theory in different fields. This theory combines the notions of "fuzzy subsets" and "the theory of possibilities". For ill-defined problems, this approach is modeled on human reasoning rather than rigid calculations. It allows designers to better understand natural phenomena, inaccurate and difficult to model, using the definition of membership rules and functions in groups called "fuzzy sets". Fuzzy set theory is an excellent extension of classical set theory. In a classical set E, the membership function is defined by: An element x is either in E ( ( )=1) or not ( ( )=0). We know that this very strict notion is not always as rigorously verified. Indeed, in many situations, it is sometimes ambiguous to decide on the membership of an element x to a set. The membership function is the equivalent of the characteristic function of a classical set. Introducing the notion of "degree of membership" in the condition verification allows intermediate states between false (0) and true (1). This is why this logic confers a very appreciable flexibility to the problems using it. Thus, it will be possible to take into account imperfections (inaccuracies and uncertainties). Fuzzy logic allows to reify "human reasoning" in the heart of decision-making systems; some neuroscience articles [24] [25] state that it is plausible that fuzzy logic concepts are a "biologically compatible" tools.

Principle of the evaluation methodology
The introduction of fuzzy logic is self-explanatory in order to employ "expert" knowledge in the service of evaluation. Fuzzy data are then processed by an inference engine subject to basic rules for issuing results, decisions or actions. Such a treatment is schematically shown in Figure 1. One seeks to find acceptable precision in the complex system of evaluation with all its ambiguities and uncertainties. According to Zadeh, complexity and inaccuracy are correlated and added, hence the possibility of providing optimal solutions by using inference engine. The inputs and outputs of the system are fuzzy variables, so they should be quantified; this is the fuzzification stage. The inputs to our fuzzy assessment system are item and the student's answer. The output is the quantified estimation of his work or his reached level. To have numerical values from the output, we go through a defuzzification stage. The inference engine is based on rules. These rules are used to connect fuzzy input variables to output variables using operators. They are of the type: "If condition 1 and / or condition 2 (and / or ...) then action on the outputs". These rules are established by experts in the evaluation field. Examples of rules: • If the item is very difficult VD and if the response is excellent E then the student's level is excellent A. • If the item is very difficult VD and the answer is good G then his level is very good B. • And so on till we summarize all the needed rules (Table 1).
Furthermore, to obtain the overall level attained by a student following an evaluation involving a certain number of items, the questions should be sorted by homogeneous groups (for example, groups of the same set of difficulty) then subjected to the treatment represented in figure 1.

Fuzzification / Defuzzification
The first element of speech, the question, is partitioned into fuzzy linguistic classes according to its degree of difficulty: from very easy VE to very difficult VD. A strong correlation with the acquisition levels or Bloom's taxonomic levels can be established to determine the fuzzy set of an item. In the same way, we also proceed to the fuzzification of the provided answer according to its quality: from excellent E to very unsatisfactory VE. An example of answer fuzzification is given in Figure 2.  2. Fuzzification of the "answer" input using the linguistic sets: excellent E, very good VG, good G, passable P, unsatisfactory U and very unsatisfactory VU Output variables will be used to obtain graphical information in the form of finite surfaces. Defuzzification consists therefore in converting this information into quantitative values. Several defuzzification techniques are envisaged. In practice, the most used techniques are defuzzification by gravity center and defuzzification by maximum calculation.

Basic rules
Basic rules are the heart of the inference engine in our evaluation system. These rules are used to connect fuzzy input variables to fuzzy output variables using different operators. The proposed basic rules are grouped in Table 1. Table 1. Typical basic rules of the intelligent fuzzified evaluation system These rules give rise to a surface, called fuzzification surface, which illustrates the final score on accordance with the item's degree of difficulty and the provided answer ( Figure 3).

Fig. 3. Fuzzification surface Results and Discussion
Let us consider a test including a number of items grouped according to their difficulty degree into five groups (very easy VE, easy E, medium M, difficult D and very difficult VD). Suppose that the marking scheme is achieved by a uniform distribution of scores, i.e. 4 points for each group. The following example re-evaluates the scores, via the intelligent fuzzified method, for a set of students who obtained the same grade of 10/20 (50%) using the classical method. The implementation algorithm of the method is represented in Figure 4. For different distributions of the score 10/20 obtained by the classical method, the proposed evaluation method, based on fuzzy logic and inference engine, assigns scores ranging from 9.32 to 11.76 according to the case. So a maximum range difference of 2.44 point ( Table 2). The classical method adds entities of different natures using a "constant" mathematical rule, while the fuzzy method establishes relations between fuzzy sets of item and answer.
Indeed, one can notice that the fuzzy evaluation method takes into account the observed progression of the student's achieved level according to the growth of the difficulties. This evaluation system encourages the continuous progress of students, especially when the difficulty is increasing. But when this progression is discontinuous, the score decreases the more the discontinuity is observable towards the least difficult levels. As a result, it breaks the "constant" rule by establishing a sort of "dynamic" rule of variation between levels of difficulty and the progression of scores obtained.
One can conclude that the fuzzy evaluation method takes into account the progress observed in a student and pinpoints his level according to the growth in difficulty. It also provides complete information (indicators) on each student's profile by tracing their performance for each level of difficulty.

An Expert System for Catch-up Session Management
Students' success and transition terms are generally an exclusively administrative matter. The sole role of the pedagogical team in this context of assessing students is limited to the verification of constraints and imposed rules. Verification of objectives achievement and expected competencies often remains superficial and vaguely discussed among the teaching team members. The evaluation quality should be a real challenge for teachers, evaluators and researchers alike. If the quality approach, through its different concepts [4] in the education sector, aims to increase the learning / teaching efficiency, it should be, first of all, felt by students as a means of quantifying their learning of the studied unit before it is a classification or ordination process. For this work, we were particularly interested in the management of catch-up session and we propose in this part to design and implement an expert system for managing catch-up scores. The expert rules system designed for the inference engine are proposed based essentially on the intelligent fuzzified evaluation method previously described. The realization of this system has brought to light numerical results that have been compared with conventional calculation methods.
Based on the intelligent fuzzified evaluation methodology, we designed an expert catch-up system with two inputs being the first exam score (exam 1: the reason to resit the exam) and the catch-up score. For the fuzzification of the catch-up score, we slightly moved the membership functions (Gaussian modified) to the right so as to "tighten" relatively the estimate reached level while supposing that the difficulties of the two tests are substantially the same. This is to say that the student must show a better performance at the second chance that has been granted. The degree of membership in the catch-up exam will be slightly lower if one considered the same fuzzification for both inputs. The basic rules adopted to design this expert system are summarized in Table 3.

Catch-up exam
According to the first test scores (exam 1), we compared the expert system scorethus implemented according to the basic rules, the fuzzification method, and the inference engine -with the mean, maximum, and minimum of both catch-up and exam1 scores. For exam 1 scores of 30% and 50%, the Figures 5 and 6 trace the fuzzified score curves provided by the expert system, the average of the two scores, and the catch-up score.
Let n (in %) be the score of the first test (exam 1). We notice that: • When the catch-up score is near (100% -n), the final score provided by the expert system is almost the same as the classical arithmetic mean; • For an exam 1 score of less than 50% and for a catch-up score between 15% and (100% -n), the fuzzified score is below the arithmetic mean; • When the catch-up score is above (100% -n), the fuzzified score is higher than the arithmetic mean and is much closer to the catch-up score.
These results show that the proposed expert system encourages students to improve their performance and aim for higher scores than what is only needed to pass the test. In the same way, the expert system will sanction students who get a score worse than the first exam. These results are acceptable in the condition that the difficulty degree is the same for both exams.
It becomes clear that it is easily possible to establish an algorithm for calculating the catch-up score according to the issued rules and fuzzification method. This expert system reveals that it is possible to develop new catch-up management rules based on the observation and accumulated experience of evaluators. Thus we can revise the conventional methods of catch-up management -often imposed without educational justification -and create new ones. We can then conclude that the reification of human expertise in a digital engine, thanks to the present expert system, is to be considered as a basic methodology that can be used to develop new methods of evaluation and management.

Method Extensions
Thanks to its simplicity of implementation based essentially on the rules issued from an expert knowledge base, the evaluation system presented in this article can be extended and applied in any structure of education. The management rules issued by the experts are put into play numerically in order to help, optimize and evaluate. Intelligent applications are then possible if one wishes to automate the management and the decision while setting up an adequate monitoring system based on relevant indicators. Such intelligent systems are sought for the automated platforms of e-learning, mlearning, individual learning and web-based training. To prove its power of optimization, regulation, intelligence and especially its simplicity of implementation, some extensions of the evaluation system were designed. We particularly mention: • An expert system for courses, modules and full semester evaluation; • A system for the regulation of learning and assistance in orientation; • An intelligent database management system (DBMS) for e-learning and web-based training.
The proposed evaluation method and its extensions are built around a Knowledge Based Expert Systems (KBES). An expert system is a program designed to simulate the behavior of a human who is a specialist or expert in a very restricted area. It is used to model the reasoning of an expert, to manipulate knowledge in a declarative form, to facilitate acquisition, modification and updating, and to provide explanations on how the results are obtained. The use of the evaluation system is independent of the knowledge it contains in a way that non-experts can use it. Two interfaces are required for a good human-machine communication: one facilitates the dialogue with the user and the other allows the expert to consult, modify, update or enrich the system knowledge database.

Conclusion
Given its predominant role, evaluation is a promising research area that aims at both the continuous development of traditional training and the improvement of new forms of training assisted by communication and information technologies. In the light of this idea, we presented in this article a novel evaluation methodology based on the notions of fuzzy logic and knowledge based expert systems. The established evaluation methodology served as a basis for developing a variety of evaluation expert systems. For new training platforms, our evaluation methodology has a dual purpose; it serves in individualized training as a measurement tool for the regulation of learning and the correction of teaching. In this case, our method will fulfill the conventional function of evaluation-measurement method. On the other hand, it serves in the establishment of new dynamic pedagogical scenarios based on the intelligence approach incorporated into our evaluation methodology. In this case, it will fulfill its function as an intelligent fuzzified method in order to adapt the learning to the student needs. Furthermore, thanks to its flexibility, the proposed methodology gave birth to several extensions and allows identifying relevant indicators that enable the evaluation of the quality and effectiveness, and regulate the learning process.