Towards a Standardized e-Assessment System: Motivations, Challenges and First Findings

Global Learning" with shared learning contents, resources, activities and goals is one of the contributions of Globalization. With the capability to use new Information and Communication Technologies (ICT) it is a bit easier to have a technology based learning systems that enable learners to share the learning resources and possibilities. As a result many Learning Management Systems (LMS) were developed with divers of platforms and approaches. Consequently, sharing learning resources and components has become a major challenge. E-assessment as a primary activity of any LMS is facing the same challenges and problems. In order to stand on this challenge people in the field of technology enhanced learning have recommended that LMS should conform to specific standards. This paper discuses this challenge, the consequences and limitations of standards in the modern learning settings and shows our first service oriented approach which aims to make our e- assessment system flexible and also to initiate the term of "Global Learning Assessment" with the possibility of sharing the e-assessment system components.


INTRODUCTION
Members of our society are affected by rapid changes in every part of our modern life.Terms such as "postindustrial society", "information society" and "knowledge society" have been used to identify and understand the extent of these changes.Knowledge has become a primary resource for production instead of capital and labor.As a result the knowledge society creates shares and uses knowledge to improve and to have a well-being of its people.Another term of "global society" with a shared knowledge is one of the aims of Globalization and using new Information and Communication Technologies (ICT).Therefore, "global learning" is needed as a primary mean of delivering this shared knowledge to the society people.As a result many open-source or even commercial Learning Management Systems (LMS) were developed.The variety of the platforms and approaches used in these LMSs makes it difficult to exchange information between them, the thing that makes some of them obsolete and dedicated for specific institutions [1].E-assessment as a main part of any e-learning system also faces the same challenge and problem.Different standards have been used to represent the e-assessment systems components.The multiplicity of such standards increases the difficulty of making those systems sharable and interoperable.
In order to have a highly quality e-assessment systems, a set of features and requirements have been identified.
One of these requirements is standards conformation while designing and implementing the system.According to [2], standards help to ensure five abilities to the elearning and e-assessment systems: • Interoperability.
Interoperability is defined by [3], as the ability of different systems to share information and services in a common file format.Reusability refers to the ability of using the learning content by different tools and platforms.Manageability is how much the system is able to keep track on the learning experience and activities, rather than the ability of tracking how learning objects are created, stored, assembled and delivered to users.Accessibility is the ability of customize, access and deliver learning contents from anywhere and anytime.Durability means that the learning content does not need any redesign or redevelopment even with new versions of the system.
Before writing this paper we have identified three main research questions.What is a standardized e-assessment system?, Why e-assessment systems must be standardconformant?and where we are in our research towards a flexible e-assessment system with regards to standards?.This paper is organized to answer those questions as follows: Standards organizations and types are discussed in section 2. Section 3 shows a set of applications scenarios for e-assessment systems.In section 4 we have discussed how to make an e-assessment system standardconform.The problems and challenges of designing a standardized e-assessment system are identified in section 5. Section 6 stresses the importance of having a service oriented architecture of our e-assessment system in order to be flexible and standard-conform.Conclusions and outlook is the content of section 7.

II. STANDARDS IN A MODERN LEARNING SETTINGS
The process of proposing educational standards starts by defining a set of specifications that describes some elearning topics such as learning objects metadata, Learner/educator information, content sequencing and services delivery.This step is done by many organizations and consortia like Dublin Core (DC) [4], The Instructional Management System Global Learning Consortium (IMS GLC) [5], The Aviation Industry CBT (Computer Based Training) Committee (AICC) [6], The Alliance of Remote Instructional Authoring and Distribution Networks for Europe (ARIADNE) [7] and the EU-funded PROmoting Multimedia access to Education and Training in EUropean Society (PROMETEUS) [8].Specifications are then tested by organizations such as Advanced Distributed Learning (ADL) [9] to be tested specifications such as, ADL Sharable Courseware Object Reference Model (SCORM) [10].The tested specifications are forwarded then to a standard committee as IEEE Learning Technology Standardization Committee (IEEE LTSC) [11].At the end standards are approved by official standards organizations as ISO and ANSI to be official standards.Standards vary according to their approval and use.There are four types of standards based on their approval [12]: • version of a combination of standards and specifications focusing on architectural aspects of an e-learning system, definitions of parts of the system and their interactions.e.g.LTSA (Learning Technology Systems Architecture) [15], SCORM (Sharable Courseware Object Reference Model) ).
In the e-learning domain, standards can be classified according to their applications into the following [16]: • Metadata Standards: a set of standards used to describe Learning objects' (LO) attributes, Such as the authors, title and languages.This description can be published with the LOs to facilitate their search and retrieval.such as, IEEE Learning Object Metadata (LOM) [17], IMS Meta-data) [18].

III. APPLICATION SCENARIOS FOR E-ASSESSMENT
In order to identify the main requirements of our eassessment system, and to figure the limitations of the available standards we will outline a set of application scenarios for e-assessment in modern learning settings.
WebSys is a software company that requires any job applicant to have a specific certificate related to their system.They are looking for a tool that can be engaged to their system with the ability to prepare tests to evaluate the new applicants.In order to handle this need and prepare factual knowledge questions based on the selected content, the e-assessment tool must have a modular design that facilitates the process of integration with the current system.Also the tool should support a flexible number of standards to facilitate the engagement process especially the ones used in the current system.
Ali is a lecturer in a university who teaches Management Information Systems for the students of the second year in the college of Management and Administration.His didactic objectives include the level of understanding of the factual knowledge by his students through a continuous assessment.To do that he decided to use an eassessment tool to deliver the tests and analyze the results through a set of continues feedback during the course.The e-assessment tool should have flexible and userfriendly interfaces to help him to generate his tests and deliver them to his students.As well as helping him to (semi-) automatically generate the tests based on the selected contents and to assess the results.Furthermore, the tool should be designed to analyze the answers of the students and provide a feedback which makes it useful for him to conduct continues assessment during his courses.
Sara is a lecturer and teaches Algebra to undergraduate students.One of her didactic objectives is to use computers to assess and assist students during here courses.She believes that when her students practice Algebra on computers and do more and more on-the-fly generated exercises they can easily pass the course.In this situation, the e-assessment system should provide her with flexible and easy to use interfaces to design here algebraic questions and save them in a database.Then, the tool itself can generate a set of exercises to the students and assess their answers based on the answers had been prepared by Sara before, or based on the algebraic engine that the tool should have.Moreover, the tool must provide a feedback to the student about her/his metacognitive knowledge and an appropriate feedback about the progress of the same student during the same course.
Jake is a teacher in a high school and he is interested in applying a set of online rubrics to assess the students' results according to a specific criteria.Regards online rubrics, the e-assessment tool should be flexible to help him to design a set of rubrics to automatically grade the students' results based on these rubrics.

IV. STANDARDIZED E-ASSESSMENT SYSTEM
Before discussing the application of standards in an eassessment system let us briefly discuss what we are thinking about our e-assessment system?.We are developing a flexible e-assessment system which includes: (a) flexible design to be used as a stand-alone service or to be easily integrated in existing systems.(b) User-friendly interfaces for both students and educators where a user interaction and online submission of solution and evaluation can be done.(c) Assessment environment for various learning and assessment settings which supports guided as well as self-directed learning.(d) Management and (semi-)automatic support over the entire assessment lifecycle (exercises creation, storage and compilation for assessments, as well as assessment performance, grading and feedback provision).(e) Rubrics design and implementation interfaces to allow the educators to design their own rubrics based on learning objectives to assess learners' performance against a set of criteria.(f) Support of various educational objectives and subjects by using various tools sets which for example enables automatic exercise generation or selection, automatic grading and feedback provision.(g) Results analysis and feedback provision (immediately or timely) of the current state of user knowledge and metacognitive skills for both educators and learners and also for adapting course activities and learning contents based on users' models.
(h) Standard-conform information and services to be easily sharable, reusable and exchangeable.This will include the tests' questions, answers and students' results, rather than any other required services.And finally, (i) Security and privacy where a secure logon of users based on pre-defined levels of access, and also users' authentication based on machine (domain users) or by usernames/passwords.For further information and for the conceptual architecture of this system you can refer to [22].
A standard-conformant e-assessment system is the system that their components are designed and implemented according to specific standards.As depicted in Fig. 1, our conceptual e-assessment system has three main components.The first one is the core e-assessment system which should be flexible to work as a stand-alone system or to be part of any other system.The other component is the interface which is used for the external communication between the core system and the other external ones.Therefore, it should support as much as possible of different standards to keep the core system flexible and modular.The last component is the external users and other systems which could be e-learning systems or e-assessment systems.
In order to have a flexible system we have to distinguish between two levels of standardization.The first level is the Internal one, where the core e-assessment system' components should be conformant to specific types of standards.Where the External level, is related to the ability of this system to interact and exchange components with other systems.This level of standardization takes place in the interface, which makes the whole system flexible and supports different types of standards.The combination of those two levels guarantees our e-assessment system to be standard-conform.Consequently, it will be flexible and interoperable.

A. The Internal Level
When we ware about a standard-conformant system we are taking into consideration that this system is designed and implemented according to specific standards.Fig. 2 shows some of our e-assessment system components and the possible standard(s) to be used in representing them.The Test Preparation Unit is responsible for the purposes of tests Authoring and Delivery.A specification such as IMS QTI is used by this unit during the test authoring.In cases of having learning objects related to the test we may use the IEEE LOM standards.The tests can be analyzed be the use of Test Analysis Unit which is based on the same type of specifications to provide a feedback (timely or immediate) to the system users (individuals or organizations).The system users are managed by the User Unit which is a standard-conform to provide some services as user personalization and modeling.Standards such as PAPI Learner or IMS LIP can be used.

B. The External Level
As we mentioned earlier our e-assessment system should be flexible and standard-conform.Therefore, we have added the interface unit in our conceptual model for this system.The interface is responsible for the external communication between our e-assessment system and the other related systems.Via this interface information such as questions/exercises and answers, users' information, list of enrolled students, courses information and learning objectives can be shared with other systems and tools.The more standards this interface supports the much more flexible our e-assessment system will be.As depicted in Fig. 3 different examples of possible standards that the interface should be flexible to support.

V. PROBLEMS AND CHALLENGES
This section discusses the problems and challenges for designing a standardized e-assessment system based on the previously discussed conceptual e-assessment system and the application scenarios in section 3.
Based on the scenarios discussed in section 3 we will show some recommendations and limitations on the available standards.In the scenario of the Company, the e-assessment system should be flexible to work as a standalone system or to be engaged with other systems such as the case in this scenario.To make this applicable the e-assessment system must have a modular design so that some modules can be integrated with other systems such as the system in this company.In this situation the problem of standards appears where the e-assessment system must support the standards used in the other system.Therefore, we recommend that the e-assessment system should support as much as possible of the available standards.
The second scenario is a traditional one where the eassessment system is applied as a standalone system to deliver and assess the students' tests and provides feedback.The limitation of standards appears again in the third scenario where a mathematical representation of the question (symbolic representation) is needed.For example, when the student is going to solve an equation we need some symbolic representation for the solution.Furthermore, a standard such as IMS QTI do not have the ability to represent the solution as a set of symbolic representation of the equations using XML.Therefore, no reference answer is available to automatically assess the student result and provide him a valuable feedback.One of the other limitations of IMS QTI specification is rubrics representation.The problem appears in the fourth scenario where online rubrics are needed to assess the students answers based on a specific criteria.
One of the most important problems and challenges of designing a standard-conform system is the lake between the features offered by the standard and the ones needed in a particular application domain [23].For example, IMS QTI (Question and Test Interoperability) is a specification that provides a questions/test description for the authoring tools.Rather than it supports the development of question/test databases that have a common schema which makes them easily sharable and interoperable.It also provides a common definition for interfaces that facilitates the creation and retrieval of tests and results [24].Even though the IMS QTI has these features it still has some difficulties in the application domain (such as, foreign languages teaching).One of these difficulties is that the IMS QTI is designed to formulate general types of questions and does not take into consideration some specific questions and test types for a particular domain [25].Crossword puzzles which is used in the domain of foreign language teaching is an example of those not supported question types by the QTI [23].According to [26] the QTI standard are not related to didactical issues and tries to be didactically neutral as possible.Another example is what authors of [27] have noted about the IEEE LOM (Learning Object metadata).They noted that IEEE LOM from a perspective of metadata don't provide enough information to support the learning process.According to [16] some developers find parts of IEEE LOM too restrictive or imprecise.And they also argue that the amount of metadata is not enough to facilitate the search and retrieval of the LOs.
Another major challenge is the problem of selecting the most appropriate standard in cases of having different types of standards for the same aspect of the Learning Management System (LMS) [16].For example IEEE PAPI Learner and IMS Learner Information Package (LIP) both of them are related to the issue of learner modeling.Even though they look similar but there are a lot of differences in the way how they model the learner.Therefore, the developer should have a good understand of the current available standards and the main requirements that helps him to choose the most appropriate standard.

VI. ABSTRACT FRAMEWORK FOR ASSESSMENT
Based on what we have discussed earlier and a step towards our e-assessment system design and implementation we have identified an Abstract Framework for Assessment (AFA).AFA is a serviceoriented approach which gives it the ability to support standards and specifications.As a result the system will be interoperable and flexible.Service-oriented architectures allow the development of modular and flexible systems [24], where the components of the system are flexible to be added, replaced or removed.As well as, new systems can be composed from a collection of suitable services.
A service-oriented framework may provide eassessment systems to easily share and exchange test between each others.Services for tests, items, results, users information…etc, can be easily implemented in the system and they are flexible to be used by other authorized services or systems.For example, students that are registered for a specific test can only attend the e-learning course in other system and vice-versa.
The services of Fig. 4 are a set of fundamental services for e-assessment systems.The services are organized in a set of layers based on the IMS GLC Abstract Framework (IAF) [28], which consists of four main layers, the "Application Layer", the "Application Layer Services", the "Common services" and the "Infrastructure Layer".The assessment services in AFA have been identified based on FREMA (Framework REference Model for Assessment) processes concept map [29].All of the services in this group are assessment services and work together in order to support the assessment process.The group of Common Services is a set of services that may be found in any assessment system or any other system such as e-learning systems.
The services should be standard-conform in order to gain the benefits of standards mentioned earlier.For example, services such as the Author service and the Deliver one can be designed based on standards or specifications like IMS QTI where the service of the Mange User can be based on IMS LIP or PAPI.

VII. CONCLUSIONS AND OUTLOOK
In this paper we tried to answer three questions."What is a standardized e-assessment system?", "Why eassessment systems must be standard-conformant?" and "where we are in our research towards a flexible eassessment system with regards to standards?".A standard-conformant (standardized) e-assessment system is the system that their components are designed and implemented according to specific standards.In order to be more clearly about this question, we have distinguished between two levels of standardization in the e-assessment system.The internal level and the external one and we have shown in some detail how and where standards could be used in both levels.Standards-conformation is the way of how to ensure that our e-assessment system will be flexible, interoperable, reusable, manageable, access able and durable.Several organizations and consortia are working on e-learning and e-assessment standards in particular.The multiplicity of these standards has made some challenges and problems to the people in the field of designing and implementing e-assessment systems.The lake between the features offered by the standard and the ones needed in a particular application domain is one of them.Another major challenge is the problem of selecting the most appropriate standard in cases of having different types of standards for the same aspect of the LMS e.g.IEEE PAPI Learner and IMS Learner Information Package.In order to have a flexible standardconform e-assessment system we have identified an Abstract Framework for Assessment (AFA).AFA is based on a service oriented architecture, where a set of fundamental services of assessment are taken into consideration.AFA consists of three layers of services and an interface layer.The services should be standardconform in order to gain the benefits of standards mentioned earlier.
Based on that, we will start an in-deep requirements analysis and evaluation of the available standards.The evaluated standards will be used in the services design and implementation of AFA.A first phase of a prototype implementation will be started soon after that.

Figure 2 .
Figure 2. Possible standards for the e-assessment system' components.

Figure 3 .
Figure 3. Possible types of standards that the interface can support.

Figure 4 .
Figure 4. Abstract Framework for Assessment [21]tion and Test Standards: Special types of standards which are used in the assessment systems to represent questions and tests.IMS QTI (Question and test Interoperability)[21]is an example of such standards.
[20]arner Information Standards: Formulates the description of the learner information and used to exchange this information between several systems, rather than its use in users modeling and personalization such as, IMS LIP (Learner Information Package)[20]and PAPI Learner (Public and Private Information).•