An Advanced eLearning Environment Developed for Engineering Learners

.


INTRODUCTION
The rapid technological progress of computers and telecommunications during the past few decades, and particularly the adoption rate of home computers and the internet, allowed for the development of educational methods that were implausible some years ago.A large portion of the research on new educational methods is focused on the development of Open Learning Environments (OLEs), especially by higher education institutions that seek to deliver their educational material with limited supervision and or remotely [1][2][3][4][5][6].However, OLEs were initially designed with theoretical education in mind, not laboratory education that is vital in engineering training [7,8].Even in conditions where the learners were expected to apply their theoretical knowledge to perform experiments and or exercises, the educational approach was based on a "one size fits all" scheme and the evaluation was based solely and quantitatively on the final result [9,10].This approach does not provide feedback to the learners and restricts the tutors, as the process followed by the learners and any of their strengths or weaknesses are opaque to the tutor.It soon became apparent that effective OLEs had to be adaptive to the individual learner and capable of providing multivariate feedback and assessment [11,12].The monitoring process that measures the progress of the learners and assesses their performance has to be continuous [13], which is crucial both for the ability of the system to adapt to the needs of the learners and for a didactic assessment; otherwise, the delivery of personalized education would be ineffective [14,15].This is especially true during laboratory courses in engineering education, where the "one size fits all" assessment approach was proven to be highly ineffective, depriving the learners from the possibilities of adaptation and customization that are critical in engineering education [16][17][18].
Furthermore, the lack of face-to-face interaction with a tutor can have a diminishing impact on the motivation of the learner, which too requires the OLE to be capable of realizing the behavior of the learner and respond appropriately [19].Despite the high adoption rate of OLEs, there have not been many studies on how the use of an OLE may affect the motivation of the learners.Motivation is a highly complex part of human behavior, especially in academia, as it is the enabler of academic success [20,21].Most of the studies were focused on the comparison of earlier web-based systems, designed for distance learning applications, displaying that the learners often felt isolated and lost motivation [22].Nevertheless, studies on recent systems and methods indicated that the motivation of the learners increased, with the eLearning tools assisting their educational capabilities [23][24][25].It has also been demonstrated that advanced OLEs may be adjusted to monitor and evaluate the motivation of the learner, even in realtime [26].
In this paper, we present the Learner Diagnosis, Assistance, Evaluation System based on Artificial Intelligence (StuDiAsE), an advanced OLE developed specifically to cater for the needs of engineering learners.StuDiAsE is based on the text comprehension theory by Denhière & Baudet [27] and dialogue theory [28], and is capable of monitoring the comprehension of the learners, assess their prior knowledge, construct individual educational profiles, provide personalized assistance, and evaluate a learner's performance both quantitatively and qualitatively through artificial intelligence [29][30][31].It can also be adjusted to monitor factors that may indicate the motivation of the learner, allowing the delivery of personalized assistance and feedback.
The structure of the paper is as following.In paragraph II, we will present the architecture of the OLE and the functions of its individual modules.In paragraph III, we show the user interface of both the learner and the tutor.In paragraph IV we provide the results of a case study that has been performed with the participation of 60 learners over four courses.

PAPER AN ADVANCED ELEARNING ENVIRONMENT DEVELOPED FOR ENGINEERING LEARNERS II. SYSTEM ARCHITECTURE
To implement the educational environment, the system is using C# and MS .netframework 3.5 technologies, while various required parts are implemented via custom web user controls and web services.The database is based on Microsoft SQL-Server 2008.The online version of the pilot system is accessible via the following link: http://pclab.et.teiath.gr/studiase/index.en.htm.
The system architecture is divided into three levels.The lowest level essentially implements the Database Access Layer of the 3-tier architecture, where necessary entities are modeled to represent the components of the system.
The upper level is the level that users interact with and essentially is the User Interface (UI).This level essentially implements the Presentation Layer of the 3-tier architecture.The upper layer is split into three sublevels: the learners sublevel, the tutors sublevel, and the administration sublevel.Learners should be able to access the system in a classroom or through the internet.A user-friendly interface helps the learners to easily and freely navigate throughout the educational materials, perform activities selected or created by their tutors, realize their own capabilities and weaknesses and improve their educational profiles.The interactive system seeks to cause them to reflect on their answers, enhance their motivation and guide them to acquire better scientific thought.Tutors can access options regarding the modification and or insertion of educational material, as well as options concerning the assessment subsystem.If a learner has already performed actions and or tests, the tutor can also view them and their results.
The middle layer includes the subsystems that are necessary to implement the logic operation of the system, designed so as to allow the independent development and use of the subsystems.This level implements the Business Logic Layer of the 3-tier architecture.The middle layer includes the five core subsystems of StuDiAsE, which are: 1.The monitoring subsystem 2. The logging subsystem 3. The profiling subsystem 4. The modeling subsystem 5.The evaluation subsystem Fig. 1 displays how these subsystems are linked to the main database and between each other.The operation of these subsystems is imperceptible by the learners, as StuDiAsE provides personalized educational material and support based on the profile and performance of the learner.The profiling, modelling and evaluation of the learners is being performed by the use of artificial intelligence and, specifically, fuzzy logic [32][33][34].Using artificial intelligence and exploiting the data logged during the educational process, StuDiAsE is capable of deriving personalized learner profiles.These profiles can then be used to assess the capabilities and weaknesses of a learner, as well as for their evaluation [35,36].The five subsystems that have been developed by the research team for the pilot application of StuDiAsE are being presented in the following paragraphs in detail.

A. Monitoring Subsystem
As its name suggests, this subsystem monitors and logs selected actions of the learners during the educational process.All recordings are being performed with the user's knowledge and consent, yet on an imperceptible for the user level, without any intervention of the educational process.Data collected during this process can be correlated with the prior knowledge (theoretical background), skills to use new technologies, attitudes and patterns during his use of the learning environment, as well as the times and points where assistance has been sought.The objective of the subsystem is the logging of sufficient data, in terms of both number and quality, which can be then used to build a profile for the learner and provide personalized material and assistance.
StuDiAsE is capable of monitoring and logging several different types of information, depending on the required application.For the means of this study, the types of information recorded were selected from a series of research studies, references and based on the experience of the research team members.Specifically, the following basic types of information are being collected: 1. Static information, such as the name and registration number of the learner.2. Dynamic information, such as the total time spent in the educational environment, the time devoted to each of the questions, the number and order of the questions that have been answered, as well as the number of questions answered correctly.3. Other information that are relevant to the test, such as the frequency and type of assistance which the learner requested, the type and frequency of errors committed by the learner and specific information derived from the navigation of learner within the learning environment and which can be used during his evaluation.

PAPER AN ADVANCED ELEARNING ENVIRONMENT DEVELOPED FOR ENGINEERING LEARNERS
When the learner begins using the educational environment, the monitoring subsystem automatically triggers and records the actions taken by the user.The learner is required to enter all the vital static information used to identify him and the logging subsystem initiates, standing by to receive the dynamic information.At the same time, it initiates the system timer, which records the time required for every action of the learner.Any response or action (help, skipping a question, etc.) by the learner is then being recorded.Upon the exit of the learner from the learning environment, the monitoring subsystem completes a report and records it to the database, filed under the static information provided by the learner.

B. Logging Subsystem
The logging subsystem operates in parallel with the monitoring subsystem, recording the answers given by the learner alongside the recording of the actions taken during the process.By recording responses to questions on a specific theme, it is possible to reveal the educational needs of the learner: the inert knowledge, misconceptions, contradictions, gaps, etc.Similarly, the recording of responses facilitates the analysis, processing and coding of the arguments of the learner, in order to form the initial cognitive profile.
The output of the logging subsystem includes information: for the particular learner, for the selected module / submodule, for the selected test, a specific number of questions and alternative answers per question, for the kind of questions / answers (multiple choice, justification, matching, fill the gap, etc.), for the type of questions (relational, transformational, teleological), for the total number of questions and the individual number of questions per question type and per subject, etc.This information is used by other subsystems of the system, which are being discussed in the following paragraphs, for the diagnostic, modeling and evaluation processes.

C. Evaluation Subsystem
The involvement of learners throughout the entire process depends on individual decisions, from answers and movements, the willingness to participate, from the compliance with instructions and encouragement offered by the system in various phases.If the learner persists on flawed or wrong responses, the artificial intelligence system should be designed to seek the minimization of conflicts and focus on trying to change the reasoning of the learner.The minimization will be possible when the learner alone removes the contradiction and thus becomes able to construct a more coherent argument (reflection) [37].
The core of the evaluation subsystem is the learner model.By using artificial intelligence techniques, it is possible to evaluate the details of the initial and final cognitive profile.This subsystem implements fuzzy logic techniques, the output of which is the cognitive profile of the learner and the inputs are listed below.
1) Recorded data of learner involvement during modeling 1. Elements indicating the engagement of the learner in the diagnostic process: informing the learner for initial cognitive profile 2. Elements indicating the engagement of the learner in the process of creating a cognitive profile and model: the number of times that the cognitive profile characterization has changed, the learner's decision to reconsider contradictory answers to questions / errors, etc.

Elements indicating the engagement of the learner in
the improvement of his cognitive model: steps leading to a change in thinking and changes in the model.
2) Recording of system navigation elements 4. Recording data on getting help 5. Recording data of moving between previous and later stages of the activity 6. Recording time intervals corresponding to engagement with each activity 7. Any other information that may be associated with this activity.
The utilization of information not related to answering questions depends largely on the type of exercise and is customizable.The rules should be based on the comparison with the corresponding figures of an expert who solved the same exercise.

D. Profiling Subsystem
The profiling subsystem is designed to extract the original cognitive profile of a learner, which represents the prior knowledge on the selected topic based on the options that have been selected by the learner.The status of the learner is then represented by particular characteristics, such as level of prior knowledge, knowledge gaps, contradictions, learning style, attitude during the study and his willingness to participate.The aim is to study the characteristics of the learner, which are important for personalization of the environment, and those that are expected to arise through interactive diagnostic feedback process.
The aim of the diagnostic process is utilizing the diversity of learning needs and abilities of learners, who will be identified based on their responses to questions (cognitive profiles) to set the main educational objective and design interactive feedback.Objective of the profiling subsystem also is the investigation and evaluation of possible ways to engage learners in the diagnostic process, which aims for the proper generation of a cognitive profile.

1) Educational strategies
The research team has developed a series of educational strategies for the extraction process of the cognitive profile.These strategies are based on research studies and references, but also in the teaching experience of the team members.Logged results of the diagnostic tests, including appropriate questions with alternative answers related to the theme, must be interpreted by the profiling system.There are three types of questions, equally numbered: a. Questions R -type (relational) b.Questions M -type (transformational) c.Questions T -type (teleological) R (Relational) text describes a document and questions which focus on simple descriptions of the part-whole relations of the system described in the document, as well as on descriptions of the processes, event and system status.Due to its short, concise sentences and detailed text, the relational text is usually better for weaker students and beginners.

# Initial Cognitive Profile Cognitive profile type Focus of personalized feedback
M (Transformative) text designates a document and questions which are focused on describing the sequence of events and the state to state transitions of the system.The transformative text assumes some basic educational background but describes the system events in detail, therefore is usually more suitable for average students.
T (Teleological) text is focused on detailed descriptions of the objectives and sub-objectives for which the system has been constructed, or of the events and mechanisms which take place within a system.The teleological text jumps to the results and conclusions without a thorough analysis of the system's events, usually assuming a strong educational background and therefore is typically preferred by advanced students.
It is educationally useable to record the number of successful responses to questions separately, R-type, M-type and T-type, but of all the questions as well.This focuses on identifying the skill of the learner to respond successfully to questions of R, M or T type, as any of their combinations. 2

) Cognitive profile structuring
The educational strategies set by the team members led to the determination of the structure of the cognitive profile and the classification of its potential cases.Thus, the possible structures of profiles presented in this study are 1) descriptive notation per answer for each question type (Rtype, M-type or T-type), with the possible classifications being high, medium or low profile and 2) numerical notation in percentages: 0%-25%, 26%-75% and 76% 100% respectively.Table 1 shows in detail the 27 possible cases cognitive profile and the corresponding abbreviations for the characterization of learner profiles.In the event that the student displays very high overall performance that would result to an R/M/T-High profile, the system will openly offer all three educational text versions to the student for selection.

3) Profile generation rules
According to the data of table 1 and the educational strategies set by the team members, a set of rules has been formulated in order for the artificial intelligence of the profiling subsystem to function.These are being summarized in table 2.

4) Subsystem operation -process of the cognitive profile generation
The profiling subsystem takes as input the output of the logging subsystem, which recorded the answers that the learner selected over a specific set of questions.Considering the rules displayed in table II and in conjunction with the data recorded by the logging system, the initial cognitive profiles are being generated.Thus, if a learner partakes a diagnostic test in order to determine his initial cognitive profile consisting of, for example, fifteen questions total (five of each type) and the logging subsystem records four or five correct answers out of the five questions for a specific type, then the learner's skill over texts of this type is considered high.Accordingly, if the learner has two or three correct answers, then his skill is considered medium, while one or zero correct answers would rate his skill as low.
For example, if the learner answers correctly five Rtype based questions, four M-type based questions and one T-type based question, then the initial profile of the learner will be set as R/M -High and the system will provide personalized text and assistance over teleological texts.Accordingly, if the learner answers correctly one R-type based questions, five M-type based questions and zero T-PAPER AN ADVANCED ELEARNING ENVIRONMENT DEVELOPED FOR ENGINEERING LEARNERS type based questions, then the system would diagnose the learner as of M-high type, offering personalized assistance primarily over texts and examples of relational type, then of teleological type.

E. Modeling Subsystem
According to the initial cognitive profile of the learner, which is generated by the profiling subsystem, the modeling subsystem begins the assembly of a learner model.This includes the initial cognitive profile and personalized feedback according to the results of the initial cognitive profile.The final cognitive profile is obtained after any additional activities have been partaken.
The feedback is given to the learner after the initial diagnosis as a personalized activity that includes text and questions with alternative answers.Additional feedback can be given in the form of assistance, suggestions/advice, didactic instructions, examples, or any combination of the above, in order to achieve the best possible learning and diagnostic result.There are 26 possible initial cognitive profile cases, which the system will provide feedback and personalized activities for, as displayed in table 1.The artificial intelligence is programmed to provide personalized feedback to learners so as to first maximize their relational text comprehension, then their transformational text comprehension and finally their teleological text comprehension.
The final cognitive profile includes any changes that may have occurred on the initial cognitive profile of the learner after the personalized feedback procedure.The learner is provided with the same exact test that has been used to generate the initial cognitive profile, assuming that the answers to the initial questions did not became known to him and or they were not included in any of the person-alized tests provided during the feedback process.Otherwise, a test with different questions but of the exact same difficulty level may be used.For example, a learner with an initial cognitive profile designated as R/M-Medium will receive additional support so as to improve on the relational text comprehension.Once the profile of the learner has changed from R/M-Medium to R-High, the next step for the system would be to provide personalized assistance based on M-type educational material, in order to improve the comprehension of the learner on texts of transformational type.If the comprehension of the learner improves until the final cognitive profile becomes R/M-High, then the system will proceed to provide assistance with teleological educational material.

III. USER INTERFACE
StuDiAsE essentially covers five user interface usage scenarios.There are the theory and laboratory sections for learners, the same sections for tutors, and a fifth section that is the administration section for personnel with access.Tutors can create dynamic activities for learners to increase their comprehension on various subject areas.They may also expand the tree of activities by adding their own.For the creation of these activities, the tutor is being led systematically by the interface.Each activity includes a properly structured text into paragraphs, which is accompanied with corresponding comprehension diagnostic questions from several categories.Fig. 2 displays the introductory page that all users will be greeted with when they enter the educational environment.From this page, the user is called to choose whether he or she wants to enter the section with the theoretical courses or the section with the laboratory courses.Once either option is selected, the user will be asked to log into PAPER AN ADVANCED ELEARNING ENVIRONMENT DEVELOPED FOR ENGINEERING LEARNERS the system.Learners and educators alike log in from this section and can create new accounts as well.All new accounts that are being created are treated as learner accounts, unless the administrator changes their access privileges.

1) Learner User Interface
The learner user interface is split into two main sections; the theory and the laboratory.Once the learner selects a section, he or she will be asked to log in.The home page of the learner's theory section user interface can be seen in Fig. 3.The learner can select a course, view completed courses and activities, as well as check his or her progress and cognitive profile when enough data has been acquired.The suggested path for a learner to follow is to take an initial diagnostic test first, with which the system will assess the initial cognitive profile of the learner, and will then offer a personalized suggestion regarding which educational text to study in order for the learner to improve his or her cognition on the subject.Afterwards, the system will propose the completion of a second diagnostic test that will assess the progress of the learner.However, if the learner does not wish to follow this path, he or she can freely choose other options, although the system may not be able to perform certain functions if the suggested has not been followed.For example, if a learner decides not to take the initial diagnostic test, the system will suggest the relational text type and will not be able to assess the level of educational improvement of the learner.
The tasks of learner profiling and personalized feedback are being performed by the profiling subsystem, using fuzzy logic AI and based on the comprehension theory of Denhière & Baudet [27,34].Note that even though the system will propose a specific type of text after the completion of the diagnostic text, the learner is free to choose from any type of text available (relational, transformative or teleological) [34].After selecting and studying a text, the learner can start an assessment activity that is based on the exact type of text he or she just studied.There may be any number and type of questions as the educator who compiled the particular section saw fit.The learner may choose to skip a question, revert to a previous question and even seek additional assistance on a specific question.The latter is being done by clicking the "?" icon to the right side of the interface, in which case a supplementary text will appear in the teal box to the right.This option is not available during the diagnostic tests.Although the pilot system presented in this paper has such options disabled for the time being, the logging and monitoring subsystems can also log information regarding the user's actions and preferences, such as the time to answer a question, the number of times that he or she reverted to previous questions and or requested help, etc.Such information can then be used by other subsystems to improve the quality of the feedback and assessments.Fig. 4 displays the user interface while the learner is taking an activity after reading an educational text.
After the learner completes the second diagnostic test, the system will present his or her assessment results.If the learner followed the suggested path, performing the initial diagnostic test, then any of the three available educational activities and finally the second diagnostic test, the system will also display his or her initial and final cognitive profiles, as well as the specific improvement on each type of educational text (Relational, Transformative or Teleological).Fig. 5 displays such an assessment, of a hypothetical learner who performed the first diagnostic test, then chose a transformative text and completed the associated activity and, finally, took the second diagnostic test as well.

2) Tutor User Interface
In a way similar as with the learner user interface, the tutor user interface is also split into the theory and the laboratory sections.Once a section is selected, the user will be asked to log in.If the username/password corresponds to an account with educator access, the user is directed to the tutor's user interface home page (Fig. 6).The home page of the educator allows the user to access the management pages of courses, activities, texts and questions.It is up to the administrator to limit the capabilities of a tutor's account if that is deemed necessary, otherwise the tutor can insert a virtually infinite amount of educative material into the system.It is also possible to give certain tutor privileges to accounts, such as the ability to manage activities and questions but not courses, allowing the creation of tutor accounts that can essentially only alter the material of their own course.
The management of the courses, activities, texts and questions is a guided process and the insertion of educational material requires only very little familiarization with computers.Higher education tutors nowadays are highly adept on the use of computers and therefore the use of a guided UI should be a seamless process with a very short learning curve [38][39][40][41].For example, the tutor selects to create a new course, the system will automatically request the input of the core educational text (Fig. 7).
The insertion of activities, texts, feedback material and questions follows a similar guided pattern.Finally, the tutor is also capable of selecting to which user accounts the course will be visible to, allowing the selection of only the learners partaking the course or the creation of learner groups.Even though the generation of the eLearning educational material and the evaluation parameters/rules can be a tedious process at first, the use of the OLE reduces the workload of the tutor considerably in the long run [42]. 3

) Administrator User Interface
The management of other users can only be performed by users whose the account has administrator access.Specifically, the administrators may: 1. See a list of the registered users, their communication data, as well as the time of their registration, their last login and last action.2. The number of the users online at the time and the course that they are partaking.3. Modify the data and the access privileges of any user.
The administrators may also manually create user accounts with any level of access and access the data recorded by the logging system.

IV. CASE STUDY
In order to assess the functionality, abilities, capabilities and shortcomings of the developed Open Learning Environment, a pilot application was arranged.The sixty participants were postgraduate students of the T.E.I. of Athens, pursuing a Master's degree on Energy Technology, a course that is being taught in the T.E.I. of Athens in collaboration with the Heriot-Watt University.Four short courses have been developed and sixty volunteers chose to participate, using the system as additional educational material.All of the students followed the path proposed by the system.They first took a diagnostic test that the profiling subsystem used to create their initial cognitive profile, and then the system provided personalized feedback to each of them.After the students completed going through (or skipped) the extra educational material provided by the system, they performed a second diagnostic test, with the system generating the final cognitive profile of each student.Fig. 7 displays a demo image of the final evaluation that a learner received.The same process was also performed by an expert in parallel, in order to identify any flaws in the diagnostic and or evaluation processes.For the means of this pilot study, only quantitative assessment rules were implemented.Qualitative data are being recorded but their inclusion in the final evaluation of the learners is disabled.The marks of the learners before and after the educational process are extracted from the logging subsystem and are examined by experts to determine how the use of the OLE affects different user groups.The marks of the sixty students that participated in this study are summarized in Table III, which displays the original mark of the student prior to using the learning environment and how much the student improved after the process.
The data of table III reveal that the use of the OLE had a positive influence on the performance of every participating learner.Depending on the course, the average score of the class increased between 14% and 16% after the use of the educational system.The improvement is more appar-PAPER AN ADVANCED ELEARNING ENVIRONMENT DEVELOPED FOR ENGINEERING LEARNERS ent on weaker learners, with the most prominent example being participants #13, 48 and 60, which managed to facilitate a score increase of 36%.However, the impact varies from learner to learner, as there have been several learners with low original marks that they did not display as much as an improvement as others.In a few cases, advanced learners that had high comprehension of the material before the use of the system did not show any improvement at all.
According to the results of the case study and as it can be seen in the following figures, the use of the system affects all learners but the impact is higher on students with lower performance and decreases for students that performed very well before the use of the OLE.
The fact that some weak learners improved significantly more than others with similar marks drove the experts to seek the cause behind this by holding interviews with some of the participants and manually checking the data recorded by the logging system.The interviews revealed that learners who displayed very significant improvements were originally lacking in motivation to study, i.e. they were finding the subject uninteresting or were procrastinating, but had no significant lack of prior knowledge.As something new and previously unexplored, the use of the OLE motivated them to study the provided texts and feedback, which they could comprehend and implement with ease.The learners that displayed low performance but still did not improve a lot during this trial study had significant knowledge gaps, either from a long history of low performance or from being inexperienced on the topic of heat transfer due to the curriculum of their undergraduate studies.
The examination of the data recorded by the logging system also revealed three more cases that are interesting.In two cases , the time in which the learner completed the test was considerably lower than the time recorded by the expert.Both of these cases were during the fourth application of the OLE (learners 3 and 32).Both of the learners had good performance and participation in other courses, indicating that they began to lose interest and or did not want to invest time on the use of the OLE.The third case is that of learner #47 during the second course, who increased his score from 84% to 100%.The time that the learner required to complete the educational process was significantly greater than that of the expert and the logs indicated that the majority of that time was spent during the answering of the questions, as well as that the answers of each question have been altered several times before the final submission.By disabling the ability of the computer to perform any other processes other than that of the OLE, the performance of the particular learner in the following courses was significantly reduced, which, in conjunction with the data recorded by the logging system, hints that the particular learner was engaged in dishonest behavior.The data presented in figures 9-12 also indicates that the disparity of the impact that the use of the OLE has on weaker learners diminishes with time.This is due to two reasons.First, the learners that were originally unmotivated to study and were intrigued by the use of the OLE were those that displayed the greatest score increases.However, after using the OLE a few times, the effect that the inclusion of the new procedure in the curriculum had on the motivation of these learners began to diminish.Second, the learners whose low scores were the result of educational gaps and lack of comprehension gradually improved, displaying higher and more constant performance improvements.The proposed system can be programmed to recognize such behaviors and attempt to motivate the students by changing the delivery method of the feedback and educational material, which will be attempted in future case studies.
iJET -Volume 10, Issue 3, 2015 PAPER AN ADVANCED ELEARNING ENVIRONMENT DEVELOPED FOR ENGINEERING LEARNERS After the completion of the current pilot study, the learners were asked to complete a questionnaire, which can be seen in table IV.As it can be seen from the results of the survey, the evaluation of most aspects of the proposed OLE is highly favorable.However, the survey also indicates that there were some concerns about the user interface, mainly regarding the use of icons/buttons and the on-screen information.In response, the research team is performing modifications on the user interface for a second trial and a quick start user guide has been composed respectively.Rightfully, the learner also expressed concerns on the inability to download learning material, as this option was disabled in the version that was used for the means of the pilot study.

Evaluation Criteria On The Functionality Of The Educational Software
The educational environment informs the student -user about the consequences of various actions and choices that may lead to impairment of the application?32% 50% 17% 2% 0% The environment allows the undoing of user actions or choices?0% 62% 25% 5% 8% The possibility of failure to complete a process due to an environment error is minimal or non-existent?58% 42% 0% 0% 0% The messages displayed to the user are simple and understandable?83% 13% 2% 2% 0% The offered assistance covers how to use the environment?23% 42% 20% 13% 2% Is the response time of basic functions of the educational environment within reasonable limits?100% 0% 0% 0% 0% Is it easy for students to use the basic functions of the educational environment?97% 3% 0% 0% 0% Protection is provided against users that do not have permission to access portions of the educational environment and data.10% 15% 25% 47% 3% It is possible to collect and transfer (download) learning material?0% 0% 0% 12% 88% V. CONCLUSION In this paper, we demonstrated the Student Diagnosis, Assistance, Evaluation System based on Artificial Intelligence (StuDiAsE), an advanced OLE based on the text comprehension theory.By using logging/recording functions and artificial intelligence rules, StuDiAsE can be used to identify, assess and improve the comprehension of students.In treating each user as a singular entity and in conjunction with proper rules, compiled by an educational expert, StuDiAsE can define the strengths, weaknesses and individual characteristics of each student, provide personalized feedback and improve the engagement of the student on the educational process.This processing helps to overcome the "one size fits all" approach that is the rule in engineering education today.
The assessment provided by StuDiAsE is multidimensional and may use both quantitative and qualitative elements.Aside from the number of correct, wrong and unanswered questions, qualitative factors can also be implemented into the assessment.These can be the choice of the additional educational material, if the user skipped the additional material entirely and proceeded to the test, the time taken to complete each activity, the number of times that the user requested assistance, etc.Even though they have not been included in the evaluation process during this pilot study, the logging of qualitative data proved to be very useful for the thorough evaluation of the learners.
iJET -Volume 10, Issue 3, 2015 PAPER AN ADVANCED ELEARNING ENVIRONMENT DEVELOPED FOR ENGINEERING LEARNERS With the correct inclusion of evaluation steps and rules using qualitative data, it is possible to affect the motivation of the students, detect plagiarism and identify other specific educational problems.
During the pilot application of StuDiAsE and with the help of sixty postgraduate learners, the research team identified multiple advantages, as well as numerous points that required further research.For learners, the proposed environment is an engaging educational tool that will automatically identify their weaknesses and seek to improve them.As the learner uses the educational tool, even if only out of curiosity, StuDiAsE generates his or her educational profile and seeks to improve his or her skills via personalized feedback.This approach proved to be particularly effective on unmotivated students, who would otherwise not study sufficiently or at all, as the scores of average and belowaverage learners were those that improved the most during this pilot course.However, that effect appears to have been temporary, as the learners ceased to display large improvements after partaking a few courses, indicating that they were losing motivation.As such, qualitative evaluation and feedback can be implemented to instill motivation of the learners, which will be explored in future case studies of the system.There are distinct advantages for tutors as well, with the most apparent being the simplicity and effectiveness of the OLE to include educational material.The time to develop the material required for each course may be significant at first but it is greatly lower than the time required for the manual assessment of the performance of each individual learner over the lifetime of the course.
Future research will be performed to assess the efficacy of qualitative evaluation, the capability of the system to include laboratory courses effectively and the possibility to replace classic taught modules partially or entirely.

Figure 1 .
Figure 1.The structure of StuDiAsE and the interconnections between its subsystems.

TABLE II .
COGNITIVE PROFILE GENERATION RULES

TABLE IV .
EVALUATION QUESTIONNAIRE FOR LEARNERS