Adaptive e-Assessment System: A General Approach

—On any type of course (on site or online), a learner is evaluated whether he has acquired the knowledge and competences provided in the course. The evaluation should be performed by evaluating his progression by means of the interaction in the classroom or assessment activities. Mostly, assessment activities are used to check the level of expertise of the learner. Typically, the assessment model and assessment activities of subjects in official programmes are the same for all the learners, since they should be evaluated having the same opportunities and conditions. However, when the learner is evaluated based on a continuous assessment model, he is demonstrating on each activity his knowledge and proficiency level and, at the same time, his reputation could be also built based on the actions he is performing within the course. Therefore, the assessment model can be particularly adapted for each learned based on this information. In this paper, we present a general system to adapt any component of the assessment process (model, activity, question…) based on different evidences gathered from the learning process of the learner.


INTRODUCTION
The aim of an adaptive e-learning system is to guide the learner through a comprehensive learning process based on the learner knowledge [1].Conceptually, adaptive learning can be performed in three different and nondisjoint ways: 1) personalization of the learning process: where the learning process of each learner is personalized according to the learner profile and behavior; 2) content personalization: where the learning objects (or the courses) offered to the learner (and the activities the learner should do) are personalized according to the learner experience; and 3) interface adaptation: where the learning management system interface is adapted according to every learner's needs, preferences and history.
Adaptive e-assessment 1 is commonly defined as an optional particular feature in adaptive learning.The learning path of the learner is adapted based on his prior knowledge and skills, the knowledge and competences he is acquiring.The learning path offers different learning resources or learning objects (material, case studies, examples, etc.) based on the learner profile.More sophisticated models also offer the possibility to adapt the assessment activities, since they can be specifically designed to assess a particular learning path.In this case, adaptive e-assessment can be applied in order to assess specific 1 From now on, we will talk indistinctively of adaptive e-assessment and adaptive assessment.knowledge and competences that the learner has to acquire at the current point of the learning process.
Adaptive e-assessment tends to analyze the prior cognitive level of the learner, and based on this analysis it proposes the next proper assessment activity.In this case, from a computerized point of view, the system evaluates a set of indicators related to the cognitive level of the learner to select the next activity to be presented to the learner.However, this approach can be seen as a particular case of a more general approach.The adaptation should not be done only based on the cognitive level.Other indicators could be also taken into account, such as the learner's behavior or reputation to extend the aim of the adaptation to multiple factors: improving knowledge and competences acquisition, authorship assurance, or even trust assurance.This new perspective opens multiples paths for extending the model.
In this paper, we present a general system that allows assessing learners in e-learning environments, usually referred in the literature as e-assessment.E-assessment can be defined as the process where information and communication technologies are used for the management of the end-to-end assessment process [2].In other words, e-assessment deals with methods, processes and webbased software tools (or systems) which allow systematic inferences and judgments to be made about the learner's skills, knowledge and capabilities [3].Furthermore, the proposed e-assessment system aims to adapt the assessment process of the learners, based on different type of indicators that could be taken into account.
The paper is organized as described next.Section II introduces the previous work performed in this area.Section III presents the score-based adaptive e-assessment as a particular case of a more general model which will be presented in Section IV.Finally, Section V summarizes an experiment performed to show the relevance of the contribution and the conclusions and future work are described in Section VI.

II. RELATED WORK
In this section, the previous work related to adaptive eassessment is described.As aforementioned, adaptive eassessment is commonly used as a part of an adaptive learning system.In these systems, there are different ways to perform the adaptation.One option is to associate a unique learning activity to a learning path.Thus, the adaption is based on the selection of the learning path.Another option is to associate multiple learning activities to the same learning path.In this case, the level of knowledge and competences of the learner within the learning path should be analyzed in order to select the best SPECIAL FOCUS PAPER ADAPTIVE E-ASSESSMENT SYSTEM: A GENERAL APPROACH suited individual assessment activity.These approaches are based on the principle that the assessment activity is a part of the learning path and, therefore, it helps to acquire the concepts and skills within the path, and it does not only serve as an activity to obtain a score.
Different systems have been developed based on this principle.Authors in [4] propose a system to provide personalized assessment activities based on the cognitive level of the learner.The proposed model learns and memorizes good learning assessment activities for different learners, and accordingly provides a personalized learning sequence for other similar learners.A similar system was introduced in [5] where the assessment questions were triggered based on the expertise proved by the learner.Authors in [6] describe a different system where learning and assessment were totally adaptable based on knowledge, competences, preferences and needs of the learner.
Research on this topic mainly focused on the automatic generation of computer adapted tests (CAT) based on the proficiency of the learner.These approaches tend to develop a large set of questions banks categorized by concept and level of difficulty, and they are triggered based on learning conditions met by the learner.There has been considerable research attention focused on Item Response Theory (IRT) [7] where the objective is to design systems to automatically provide and score questionnaires to measure abilities, attitudes, or other variables.Amongst others, systems as PARES [8], ITSAS [9], IRTT [10] and AAS [11] have been proposed using variations of the IRT approach.
Note that, the previous systems rely on the learner's profile construction, the quality of the collected information and the ability of the model to predict the knowledge and competences to be assessed.Based on the prediction model, Bayesian [11][12] [13] and Fuzzy [4][14] [15] methodologies have been used.These methodologies help to evaluate hypothesis based on some prior collected evidences.
However, all these approaches are focused on adapting the assessment based on prior acquired learning, without taking into account other variables that may be also relevant to select the next assessment activity.This paper describes a more general system that takes into account multiple evidences in order to generate the next assessment.
The next section describes the current system applied in our university that is basically a particular case of the general system that will be described in Section IV.

III. PARTICULAR CASE: SCORE-BASED ADAPTIVE E-ASSESSMENT
In this section, we explain the system broadly used in our university to evaluate the learners.Currently, most of the subjects of any programme (degree, bachelor, postgraduate studies) use a continuous assessment model which includes several Continuous Assessment (CA) activities.This base model sometimes is combined with a Final Project (FP) and/or a final assessment activity.The final mark of the learner is obtained from the evaluation of all these activities.However different assessment models can be derived from this base model, depending on the selected assessment activities (see Fig. 1): • Continuous Assessment (CA or CA + FP): It is composed of different continuous assessment activities and each activity includes several exercises.The type of the exercises is different from one subject to another (individual, collaborative, design, coding, etc.).
The number of activities depends on the number of didactic units included in the subject.The model can be complemented with a final project (FP).The final project (if any) aims to promote skill acquisition or simulating a professional environment.In general, it involves the use of some kind of software or virtual laboratory.• Continuous Assessment and Final Validation Activity (CA + FV or CA + FP + FV): Similar to the previous model, but it is complemented with a final validation activity (FV).Note that it is not an exam.By definition, the validation activity is composed of exercises to validate both continuous assessment activities and the final project, if any, done by the learner.It is evaluated as pass or fail (i.e. a qualitative mark is issued).The learner is eligible to access to the validation activity whether he has passed the continuous assessment activities and the final project, if any.• Continuous Assessment and Final Synthesis Activity (CA + FS or CA + FP + FS): Similar to the previous model, but the final activity is defined as a synthesis activity (FS).The objective is similar: to validate the knowledge and competences acquired by the learner in a subject, but the complexity is higher and the evaluation is performed in a quantitative way (i.e. a SPECIAL FOCUS PAPER ADAPTIVE E-ASSESSMENT SYSTEM: A GENERAL APPROACH score is issued).The learner is eligible to access to the synthesis activity whether he has passed the continuous assessment activities and the final project, if any.
• Continuous Assessment and Final Exam (CA + FE or CA + FP + FE): In this model, the final activity is an exam (FE) that is evaluated from a quantitative perspective.Here, there is no need to pass the continuous assessment activities to access to the final exam.• Final Exam without Continuous Assessment (FE): Exceptionally, there are some subjects with only a final exam (FE).Note that, in this case, the option of combination with a final project (FP) is not allowed.It is important to note that each assessment model can impose additional constraints when applied in a subject as the percentage of relevance for each type of activity and the minimum values to be reached on each type of activity in order to compute the final mark of the learner.Furthermore, a subject can incorporate more than one model, and a specific model is applied depending on the accomplishment degree of these constraints by the learner.All these constraints (that can be modeled by means of parameters) combined with the set of possible models generate a large number of possible assessment models.
For instance, a subject (see Fig. 2) can combine the assessment models CA + FV and CA + FE, and the first model is applied whether the score of the continuous assessment activities (CA) is larger or equal to 5 2 .
A multi-model assessment (i.e. the application of more than one model) with a conditional application based on a score could be defined as a particular case of adaptive assessment, specifically, as score-based adaptive assessment which is mostly used for the state-of-art adaptive eassessment systems, e.g.IRT-based systems.The selection of the model and the final assessment activity is based on a single indicator which is the score that denotes the knowledge and proficiency proved by the learner in the proposed assessment activities (CA and FP).Although this is a simple way to adapt the assessment, it is the basis to design more sophisticated models.In the next section, the general case is defined.

IV. GENERAL ADAPTIVE E-ASSESSMENT SYSTEM
In this section, we describe the general adaptive assessment system.The main properties of the system are (1) to collect any type of evidence and analyze it for updating the learner profile; and, (2) to create the next assessment activity to be delivered to the learner based on the information gathered and the objectives associated with the assessment activity.
Note that, a system with these characteristics is enough general to be applied to on-site or virtual learning.However, virtual learning environments (VLE) have some advantages, since most of the evidences can be obtained automatically without any intervention of an agent (instructor, administration staff, etc.) using Learning Analytics frameworks [16][17] [18].
The aim of a general adaptive e-assessment system is extended to not only be able to adapt the assessment process according to the knowledge and competences acquisi- 2 The grading system of our country grades from 0 to 10, being 5 the lowest grading grade to pass the subject.tion of the learner.Other objectives can be taken into account related to the instructional and assessment process.In this paper we have detected, at least, the next objectives: • Knowledge and competences acquisition: This is the main objective of an adaptive learning system and, consequently, of an adaptive assessment system.• Trust assurance: Some systems (not only in e-learning environments) evaluate the reputation of the user.In a VLE, the assessment process of a learner with a high level of trustworthiness can be adapted to be more flexible, i.e. a final exam (FE) can be transformed into a final test validation activity (FV) when the learner proves constantly his knowledge and proficiency through the interaction with the VLE and the collaboration with other learners.• Security assurance: It is crucial that the assessment process performed by a learner has been conducted without any infraction, ie.plagiarism detection.Some security issues such as authorship sometimes have a high level of uncertainty.The instructor detects that the learner may not be the author, but he cannot ensure it.In this case, the next assessment activity can be customized to refute or confirm the teacher suspicions.It is important to note that the adaptation can be done at different levels of granularity (the coarser levels subsume the finest ones).The top level refers to the customization of the assessment model and, therefore, all the assessment activities within the model.As aforementioned, in addition, more than one assessment model can coexist within a subject.The intermediate level adapts individual assessment activities.Here, an assessment activity is customized with all its internal questions.At the finest level, several questions within the assessment activity are personalized.
A general adaptive assessment system is composed of two main modules: • Evidential module: This module is focused on analyzing the gathered evidences and detecting which evidences are relevant to update the profile of the learner.• Adaptive module: This module is responsible for adapting the next assessment activity to be delivered to the learner.
Note that the modules have different scheduling.On the one hand, the evidential module is continuously analyzing evidences, since they are collected from the learning process and assessment results of the learner.On the other hand, the adaptive module is only used when assessment activities are created.This module will customize the activities according to the evidences collected by the previous module.In the next subsections, we describe in detail each module.

A. Evidential module
This section describes the evidential module.As aforesaid, its objective is to analyze the evidences generated during the learning experience of the learner.Figure 3 illustrates the module.
The module has inputs from different data sources.There are different evaluators that provide evidences from the learning process and interaction of the learner within the VLE system.Additionally, the module requires storing the learner profile that collects all the outcomes and information related to the learner.According to be a general system, the module can be configured with a parameterizable profile model which specifies the particular information that should be stored for each learner.Thus, the learner profile could gather many information related to the learner: information about knowledge, activity on the VLE system, reputation, interaction with learning resources.The specification of the information to be stored will be specified by the profile model.
The module is composed of the next components: • Evidence Analyzer: This component is responsible for processing the evidences and generating the internal indicators helpful to update the learner profile.Depending on the information stored in the learner profile specified by the profile model, the evidences will be evaluated in a different form (they will generate different indicators) and even some of them should be dismissed since they will not have impact on the profile; e.g. the profile model might not take into account security evidences since they are not relevant in the particular adaptive assessment system.The component should specify which type of data sources accepts and how the data should be sent 3 .Although each evaluator can follow a different data model to store evidences, i.e. user-centric or event-centric, this component will be able to collect evidences from any data format.The evidence analyzer works as an ETL (Extract, Transform and Load) process since heterogeneous data from different sources are collected, analyzed and sent in a homogeneous format to the Profile Updater.This ETL process will be conditioned by the profile model and will notify only the indicators that will have an impact on the Learner Profile.• Profile Updater: This component updates the learner profile based on the indicators generated by the pre- 3 A good review of different models to collect information based on learning analytics can be found in [19].
vious component.The profile model should be as much extensible as possible in order to be able to gather all relevant information related to the learner [20][21] and, therefore, to create sophisticated customized models in the adaptive module.For instance, for the identified evaluators explained next, the learner profile will have commonly learning information such as learner prior knowledge, competences level acquisition or current grades on ongoing and finished subjects.Additionally, events and issues collected by the evaluators will be continuously aggregated on indicators for security and presence such as misconduct or plagiarism on subjects or activity of the user on discussion forums or collaborative activities.
We have identified three evidence generators: • Presence Evaluator: Presence information is also important in some type of activities such as peer assessment or collaboration activities.This system evaluates the reputation, the trust and confidence the VLE system has on the learner.Techniques to collect trust and reputation information in general systems (not specifically in VLE systems) can be used [23][24].
It is important to note that more generators may appear in the future.The system will be able to use them as long as they fulfill the specification to send the evidences to the Evidence Analyzer.

B. Adaptive module
This section describes the adaptive module (see Figure 4) that will provide support to the creation of the adaptive assessment.
In this case, the module uses several storage components.The learner profile is shared with the Evidential module.This module only reads the profile built for the learner since the Evidential module is responsible for updating it.Next, the module also uses other relevant information of the learner such as his learning preferences or accessibility needs.In some systems, this type of information is merged within the learner profile.In this case, we split it in two different data sources (the learner profile and other learner information), because the other learner information is a more static information (i.e. less SPECIAL FOCUS PAPER ADAPTIVE E-ASSESSMENT SYSTEM: A GENERAL APPROACH frequently updated) related to learning preferences and accessibility needs and not based on evidences from the Evidential module.Finally, the assessment objects are stored in data banks.We assume that the assessment objects have been created previously.In the case that a particular system requires creating new assessment objects, an additional module should be created to add assessment objects in the data banks.However, this is out of the scope of this paper.Although different approaches can be used to design the storage system of assessment objects, there is limited literature related to management of this type of objects.However, approaches related to learning objects repositories could be used.Similarly, the assessment objects should be described with metadata.This information will be crucial for the Assessment Recommender module to select the best-suited assessment activities.If we focus on the design of the repositories, a good review of learning objects repositories can be found in [25].Additionally, a review related to open education resources repositories can be found in [26].
Additionally, the module can be parameterizable providing the context where the assessment activity will be deployed and the selective model.The selective model describes which information is used from the learner profile to create the assessment activity and the type of assessment activity that has to be created.
The module is composed of the next components: • Profile Analyzer: This component will process the learner profile to evaluate the information related to the learner.The context [27] where the assessment activity will be deployed and the selective model to be applied should be known, since the analysis will be based on this information.This component is basically a collector of information of the learner profile and in some selective models, new indicators are computed aggregating different information of the profile.The set of indicators such as knowledge, competences, their level of acquisition, trust score of the learner will be transferred to the next component.This information will be used to build the adaptive assessment activity.• Assessment Recommender: This component collects the previous data and using the selective model will create the assessment activity to be delivered to the learner.Note that the component reuses the assessment objects stored in data banks.The assessment objects could be tests, open questions, or any type of exercise that can be assessed automatically (preferred) or manually by the instructor or, even, on a peer-review basis.The component also uses the other information of the learner relevant on building a full-fledged personalized assessment.There are in the literature many approaches to select best-suited objects based on a specific query.Mostly defined as recommenders, these components suggest the best object.As far as we know, there are no specific approaches to select assessment objects.However, techniques related to recommend learning objects could be reused [28][29] [30].These approaches commonly combine user preferences, context-based and knowledge-based information to provide the next learning objects to deliver.In assessment objects selection, the knowledge-based criterion should be extended to other indicators stored in the profile model.• Assessment Adapter: This component creates the assessment object (ie. the activity) to be delivered.Note that, multiple output formats can be available (HTML, VLE specific format, etc.) based on the system where the assessment activity will be presented.The context will give information of the output system (mobile app, intelligent tutoring system, MOOC platform, Word document etc.).This component will adapt the assessment activity to be compatible with the interface presentation.Internally, the component is basically a format transformation system to provide the best-suited representation.

V. EXPERIMENTAL RESULTS
This section shows an experiment on how a general adaptive assessment is aligned with a non-adaptive assessment system.The aim of the experiment is to show the impact of an adaptive system on the selection of the final assessment activity on an online course.We have performed a simulation of the instructional process of the subject "Computer Fundamentals" of the Degree of Computer Science in the Open University of Catalonia.Currently, this subject does not have any adaptive system and, therefore, all students perform a final activity (exam) at the end of the instructional process.The experiment simulates whether the final exam should be triggered assuming different selective models.The experimental results show the correlation on the deployment of the final activity between the different selective models and the score of the final exam.That is, a high correlation between the score on the final exam and the selection of the selective score indicates that the selective score has successfully selected not to trigger the final exam since the collected evidences show that the learner has acquired sufficient knowledge and competence during the continuous assessment activities to pass the course and no issue has been detected during the instructional process.Note that the system has not been yet implemented; the simulation was only intended to see how different selective models could impact in a hypothetical adaptive assessment system.
Currently, the subject has three continuous assessment activities (CA), a final project (FP) and a final exam (FE).A web forum is used as a communication tool between learners and teachers.The subject also has an intelligent tutoring system called VerilUOC [31] where learners solve exercises related to the design of digital systems.The intelligent tutoring system offers a large set of exercises (more than 150 exercises) to practice and some exercises of the continuous assessment activities and final project can be also submitted using this tool.Therefore, where TCE and TAE are the total correct and attempted exercises, TDE is the difficulty of all as- 4 In case on existence of a final project, the selective score is computed as the average between the CA and FP.sessment activities, where we have assumed equal difficulty (set to 1), and Score_CA and Max_CA are the score and the maximum score that the learner can reach in the continuous assessment activity 5 , respectively.
• Presence-based Model: The presence shows the interaction of the learner within the course with other learners.Learners can show a certain level of trust based on the quality of his contributions to the web forum (submitted by means of messages).In this case, we added to the previous model a corrective factor using the trust-based model described in [32]: where !represents good quality messages, that is, messages with a meaningful impact in the instruction, for example, messages that help to other learners, the proposal of a solution to exercises, good questions, etc.In turn, " represents the remaining messages submitted by the learner.
• Authorship Model: In this case, plagiarism issues set the selective score to zero, otherwise the current selective score is maintained.Note that, these learners should go directly to the final exam (FE).Table I shows the correlation on the deployment of a final validation activity and the score of the final exam the learners have performed.In the case of a non-adaptive system, we assume that a score in the final exam larger than 50%, the final exam would not be necessary and a final validation activity would be enough.In the case of the adaptive assessment system, the final validation activity is triggered when the selective score is larger than the percentages showed in the table for each selective model (from 50% up to 80%).The objective of the simulation on this range of values is to check how the correlation is affected on selective models with harder constraints.For instance, the percentage of 80% denotes that the exam should be triggered only if the selective score is larger than 80% of the maximum value.Note that two starting scores have been used: the continuous assessment (CA) and the final project (FP).The final project is a harder assessment activity at the end of the continuous assessment and previous to the final exam which aims to assess in an integrated way all the concepts provided in the subject.Therefore, the correlation should be higher if the score of this activity is taken into account since the learner proves all the acquired knowledge and skills.
If the percentages set to 50% are analyzed, the table shows a high correlation in the score-based model.The selected activity by the score of the model in both starting scores (CA and FP) is highly correlated with the score of the final exam of the learners, that is, learners with a selective score larger than 50% has passed the final exam (score of the final exam larger than 50%).This result reflects that currently adaptive assessment systems based only on this indicator would infer that in many cases the final exam (FE) is not needed and a simpler final assessment activity, i.e. a final validation activity (FV), could be generated for learners with a good score.SPECIAL FOCUS PAPER ADAPTIVE E-ASSESSMENT SYSTEM: A GENERAL APPROACH However, the knowledge-based model increases the quality of the model.Results extracted from VerilUOC utilization shows a more fine-tuned model to compute the selective score.Learners with a higher interaction with the tool prove more expertise related to the design of digital systems.Using this model, more learners should be able to avoid the final exam, and perform a final assessment validation activity (FV).
The presence and authorship models also show interesting results.The presence model shows a reduction of the correlation.This is a consistent result and it is related to the fact that there are learners that do not interact in the web forum.They only access to the course to perform the assessment activities (CA and FP) and the exam (FE).If no presence is shown during the course, the selective score decreases since the learner has not proved additional knowledge.Therefore, in a hypothetical adaptive assessment system, this learner should perform the final exam (FE) instead of a final validation activity (FV).This result shows that models based on presence checking could be useful in courses where interaction among students is important in terms of discussion or collaborative activities.
The authorship model shows similar results.When security is analyzed, the detection of security or misconduct issues will always impact negatively in the selective score.In this case, plagiarism detection automatically requires the learner to do a final exam (FE).
Finally, if the progression of the correlation is analyzed within a selective model, we observe that the correlation degrades on harder constraints in all selective models.This result reflects that an excessive hard constraint disables the adaptive system since it triggers the final exam to learners that successfully has passed the final exam.However, this is a relevant result since in some type of courses can be used.For instance, in MOOCs hard constraints are commonly set in order to give the certificate of completion.
With respect the authorship checking model, we can observe that on harder constraints the number of infractions decreases.Learners with larger evidences that have acquired the sufficient knowledge to pass the course do not tend to produce infractions.

VI. CONCLUSIONS AND FUTURE WORK
In this paper, we have described a general adaptive assessment system where activities can be adapted based on relevant evidences collected from the instructional process of a learner.The system can be totally parameterizable with any profile model to build the learner profile and any selective model to adapt the assessment activities.Additionally, a simulation has been shown to see how a general assessment system can impact positively on the selection of an assessment activity.The presented contribution is an initial description of the architecture of the system without few details of the internal design of the modules.
As future work, we will focus on specifying the internal components of the system and how the parameterizable models will impact on their operation.After that, we will be ready to design the adaptive assessment system and test it.There is a long path before the whole picture can be gauged since the experimentation will be done firstly on individual subjects and, therefore, on multiple subjects.We expect that evidences collected in multiple subjects can give more information related to the presence and, even, the reputation of the learner to build a better learner profile.
Additionally, profile and selective models will be explored to find the best suited to be applied on a general adaptive assessment system.

Figure 1 .
Figure 1.Description of the evaluation models in our university.
Cognitive Evaluator: This component evaluates the knowledge and proficiency level of the learner during his learning and assessment process.The evidences collected are the ones needed for adaptive systems focused on knowledge.Information such as a score of an activity, an achievement, acquisition of a competence, amongst others, can be transformed into an indicator of the progression of the learner.• Security Evaluator: Security information is significantly relevant in VLE systems.Learner actions should be analyzed to check if any infraction has been done.The results of checking security properties (authentication, authorship, non-repudiation and integrity) are transformed into indicators to see whether the learner cheats during the learning and assessment process.Note that this information may have a high level of uncertainty in some cases.Authors in [22] describe a set of security mechanisms that can be used to collect evidences.

Figure 5 .
Figure 5. Parameterization of the system for the simulation VerilUOC allows collecting evidences from the learning and assessment process of each learner.In this case, the objective of the adaptive assessment system would be to adapt the final assessment activity to a final exam (FE) or a final validation activity (FV) based on the selective score computed by the proposed selective models.The summary of the experiment is shown in Figure 5.For the sake of simplicity, the figure only shows the parameters that have been configured for the experiment.The evidences collected in the subject are summarized next: • Learner's performance: The scores of the continuous assessment activities (CA) and the final project (FP) are used to evaluate the knowledge and competences acquisition of each learner.• Number of exercises performed within the VerilUOC system: This evidence is used to evaluate the learning progression of each learner.• Presence of the learner in the web forum: This evidence evaluates the presence of the learner in the course and the knowledge he is proving in the web forum.• Plagiarism detection: VerilUOC also checks automatically plagiarism among learners.This evidence validates authorship on assessment activities submission.Four different selective models have been used.For the sake of simplicity, each model generates a unique selective score that specifies the level of adaptation.The selective score has been normalized to values from 0% to 100% where 100% is the maximum score that could be generated by the model.• Score-based Model: This model only takes into account the score of the continuous assessment.Therefore, the selective score is the score of the continuous assessment 4 .• Knowledge-based Model: In addition to the score, this model takes into account the knowledge construction of the learner.The model defined in [9] has been used:

TABLE I .
CORRELATION ON THE DEPLOYMENT OF FINAL VALIDATION ACTIVITIES BETWEEN SELECTIVE MODELS AND FINAL PERFORMANCE