An Experiment of Introducing ePortfolio to Evaluate Students in the Tunisian Context

We present an experiment of ePortfolio use as a tool to assess students in a module taught in a master degree class. Each student is required to deposit an abstract for each chapter in his own ePortfolio. A mark is given to each assignment and then the average of all these marks is considered for the success criteria. At the end of the semester, students are submitted a questionnaire in order to evaluate their appreciation of the ePortfolio use. The formal analysis of the questionnaire confirms our overall idea that most students appreciate the ePortfolio use. They also express their wishes to extend the ePortfolio to the other modules. The teacher's perspective for this experiment is also pre-


INTRODUCTION
According to Seldin, the electronic portfolio (ePortfolio) operates in so many levels that it is considered to be one of the most powerful learning and assessments tool to emerge in the past quarter century [1].On the other side, Penny [2] indicates that "… proponents of the ePortfolio movement have argued for well over a decade now that learners need to document what they know, reflect on their knowledge and present that knowledge to specific audiences in order to strengthen their learning , thus the ePortfolio seems to be an ideal tool.Considering these reflections together with a strong belief in the ePortfolio's importance as well as the fact that the Tunisian university should follow the progress trend of the other universities in this matter, we conducted an experiment for the use of the ePortfolio in our institute.The aim is to introduce this concept to our students, principally as a large number aim at a teaching career .This paper introduces the ePortfolio concept and its impact on assessment, then our experiment is presented together with the questionnaire results and we conclude with the lessons drawn from the whole concept .II.EPORTFOLIO AND ASSESSMENT Abrami and Barett present the learner's ePortfolio as a purposeful collection of student work that exhibits his efforts, progress and achievements in one or more areas over time [3].This collection should include:  student participation in selecting contents,  the criteria for selection,  the criteria for judging merit,  evidence of student self-reflection.
According to Cambridge "... all stakeholders in higher education should be invited to engage in dialogue, mediated by authentic and integral representation of student identity and performance in the form of ePortfolio, in order to produce institutional innovation" [4].
The ePortfolio is currently one of the electronic learning tools gaining prominence.It is largely used for building online records of one's achievement but also for other purposes related to learning such as assessment [5].According to Walvood [6] 'Assessment of student learning is the systematic gathering of information about student learning, using the time, resources, and expertise available, in order to improve the learning'.EPortfolios offer an opportunity to enhance systematic gathering of student learning, including real proof of their work [7].This potential has gained attention and the ePortfolio assessment is considered as belonging to authentic assessment and formative assessment [8], [9].It aims to break through the traditional assessment ways such as quantification assessment and paper test [9].According to Schneider, the president of the Association of American Colleges and Universities, ePortfolios offer an opportunity for unique forms of assessment for the individual student [8].

III. THE EXPERIMENT DESCRIPTION
Our experiment has been conducted at the High Institute of Management of Tunis (http://www.isg.rnu.tn) with the participation of seventy students enrolled in a master degree entitled "Sciences et Techniques de l'Informatique de Décision" (STID).The experiment has already been carried out last year in the module entitled "Elearning" which deals with the use of Information and Communication Technology (ICT) in distance learning as well as elearning 2.0 and the game-based-learning.Because of administrative constraints, our module remained quite traditional and students had to attend a three hour course per week.The first chapter of the module introduced the elearning concepts and the second presented the ePortfolio concept.We described its use throughout the world, and in order to sensitize students, we referred to the importance attributed to the ePortfolio in the developed countries and presented some examples where its use is compulsory.We proposed to those students to join an experiment using the ePortfolio as an application to the module's content and because the students themselves can be the ideal population for the extension of this concept because their majority wants teaching as a profession.We asked our students to create their own ePortfolio where they deposit their homework to be evaluated.So at the end of each chapter they have to deposit a summary file of it.We proposed to use the ePortfolio of the University of Montreal which is Edu-portfolio (http://www.edu-portfolio.org).We selected PAPER AN EXPERIMENT OF INTRODUCING EPORTFOLIO TO EVALUATE STUDENTS IN THE TUNISIAN CONTEXT it because it is a free ePortfolio for learners and educators at all levels for non-profit use in addition to its French interface, a language well mastered by our students.

A. The evaluation of the ePortfolio
To encourage students to participate, we indicated that the evaluation work will be done only through the ePortfolio and that there will be no written test as in the other modules.We defined the evaluation process at the beginning and we talked with students about the structure and the required elements.In fact we asked our students to deposit for each chapter files , one as a summary file and another to show the positive and negative elements of the chapter.Only three students deposited this last type of file for the first chapter.We may justify this by the fact that they were told that only the file containing the summary would be evaluated.From the beginning of the experiment we present the established criteria to assess the students work deposit.These criteria are based on those used by the San Francisco State University (http://eportfolio.sfsu.edu/resources/Eng_Ed_ePortfolio_rubric.doc).We find them well adapted to our experience environment and we used the same rubrics for the formative assessment as well as for the summative assessment .
In order to obtain assessment and evaluation for their homework we asked the students to inform us by e-mail after every homework deposit.Our assessment is given as a comment relevant to each deposit.In each comment we indicate areas for improvement of their work; we congratulate the student for creative work and for the positive points.We also provide formative guidance about next steps or areas to improve and we also specify the attributed mark.From the start, we informed our students that an average mark would be counted as 30% of the final module mark.We asked our students to compose their own ePortfolio not only in terms of specific goals of that module but also in relation to their learning and development as a whole [4].After the last chapter we asked our students to reorganize their ePortfolios so that they should document and reflect their learning and development.
At the beginning of this experiment most students were reluctant to use the ePortfolio, but at the end of the semester many of them expressed their desire to extend the experiment to the other modules.Although this nonformal evaluation was positive, we needed to quantify the students' opinions and during the last class session, we asked them to fill in a questionnaire so we could collect their feedback and suggestions.The purpose of the questionnaire is to determine the level of students' knowledge of the ePortfolio concept and to identify their experiment appreciation.Each student is required to answer the questionnaire file, to place it on his ePortfolio, and to inform us by mail.The questionnaire consists of fifty questions, most of them are closed and use a scale because we need to assess attitudes [11].Among the attitude scales we proposed the rating scale in which the student indicates a degree of importance.

IV. THE EXPERIMENT RESULTS
The questionnaire is divided into essentially two sections, whereby the first one is looking for students' difficulties that may be solved by the ePortfolio.The second group of questions is relevant to this experiment and how students consider the ePortfolio before and after the experiment as well as the used tool edu-portfolio.To ensure responses of all of the students (69) they were informed that the evaluation of their last homework and the overall assessment of their ePortfolio would be forwarded to them once the questionnaire responses submitted.It is only through this subterfuge that we have obtained (100%) participation.To apply statistical methods to the questionnaire, we considered questions as variables and students as observations.The software used to compute statistics is the Statistical Analysis System SAS [12] as it is available in our institute.

A. Our students and some issues requiring an ePortfolio
For each qualitative variable we compute descriptive analysis on the (69) observations.The SAS procedure FREQ computes for each variable the frequency of its modalities [12].The "Fig.1" shows an example of the result generated by the FREQ procedure applied to the variable "Did you know ePortfolio before the experiment", where 1 corresponds to the modality "Yes".The summary of the first group of variable gives us a general idea about students' difficulties that may be solved by the ePortfolio as shown by the "Fig.2".
It's a bit disappointing that (91%) of students do not have their own website, especially because they are in computer sciences field.They do not have the culture to present themselves on the web.They also rarely use the web to store their productions (2%).(83%) of the student frequently store these productions, so (67%) of them use their own computer for the storage, (28%) put their productions on "CD" and (3%) put them on "USB Key".Concerning the question "Can the (curriculum vitae) give all your accomplishments and your acquisition", (53%) of the students seem to be satisfied by the curriculum vitae.Besides, only (38%) of the students claim having some difficulties to prove their skills to a recruiter.Only a few students (34%) can not easily prove that they have prerequisites for training.
We can claim that our students don't have the culture to present themselves on the web and are not very innovative in their ICT use.In fact they don't feel the need to present themselves other than by conventional tools and this is confirmed by the fact that only (9%) have a personal web site.This classical use of the web is not due to an equipment's problem because (96%) of our students have a personal computer and (93%) of them have an "Asymmetric digital subscriber line" (ADSL) connection at home.The lack of enthusiasm with which our experiment has been accepted at the beginning may be justified by the cultural aspect as well as the classical use of internet.

B. The students' appreciation of the ePortfolio concept
The analysis of the second group of questions shows that (83%) of students were not aware of the ePortfolio concept before the start of this experiment "Fig.3".Nearly (57%) of them showed little or no enthusiasm for its use at the beginning of this semester.They justified this reluctance by the extra work it requires.Some reluctant students proposed to use facebook instead of the ePortfolio as they find them identical.
At the end of the semester, (97%) of the students approved the ePortfolio use, and (79%) agreed that their institute should extend it to the other modules.After this experiment, the concept of ePortfolio appears to be well assimilated as (86%) of students find that facebook can not substitute it.The justifications are for half of the students, due to the fact that facebook is mainly aimed at interpersonal contact.The other half find that the ePortfolio is more official, allows the management of documents and can be used in finding a job.
We are also interested in the nature of the homework they have to deposit and (93%) of students find that a summary file for each chapter is a good idea.Students were also well aware of the importance of the feedbacks given by the teacher on their ePortfolio.It is described as "very important" by (77%) of the students.On the other hand, (78%) of students appreciate that the ePortfolio evaluation would be counted in the final module mark instead of the classical-face test.
Moreover (90%) of students plan to use their ePortfolio when applying for a job, and (80%) believe that it may provide assistance in managing their documents.We are also interested in the tool's ergonomy, i.e edu-portfolio, which is appreciated by (47%) of students, but (33%) suggest some criticisms.
The quantitative variables of the questionnaire are treated separately.Students have to respond essentially with a percentage to indicate their appreciation degree of the following variables: ("FullEnglish": if the course Figure 3.The interaction between students and the ePortfolio should be taught in English, "FullEAD": if the course should be totally online, "HalEAD": if the course should be half-online (blended learning), "Nocomment": if the teacher should communicate his comments on the students homework, "IntroePort": the introduction of the ePortfolio in this course, "Encourage": if they encourage the introduction of ePortfolio in their institute, "Securit": the security of the tool edu-portfolio, "Use": the facility to use this tool, "NoteMast": an evaluation of the master's degree content).
We apply the procedure "MEANS" of SAS on these variables [12].For each variable it computes the mean, the standard deviation, etc. "Fig.4".Because these two parameters are dependent on the extreme values, literature recommends to use the coefficient of variation (CV) to interpret the variables tendencies [7].The (CV) equals the standard deviation divided by the mean (expressed as a percentage).It is a statistical measure of the dispersion of data points in a data series around the mean.According to Martin [5], this parameter is significant when its value is between 0.0% and 16.0%, it indicates that the mean's variation is small and its estimate is reliable.Unfortunately, the (CV) of our quantitative variables are between (25%) and (116%), so no significant conclusion may be presented concerning the variables' tendency.Because these last parameters are not significant, we use one method of the data analysis field which is the Principal Component Analysis (PCA) to observe trends and patterns in our data [13].

C. The Principal Component Analysis (PCA)
According to Jolliffe [13], the goal of the (PCA) is to reduce the dimensionality of a data set consisting of a certain number of interrelated variables to retain the variation present in the data set.It is realized by transforming to a new set of variables, the principal components (PCs), which are uncorrelated.Each (PC) is a linear combination of the original variables, with coefficients equals to the eigenvectors of the correlation matrix.These  In geometric terms, the j-dimensional linear subspace spanned by the first j (PC) gives the best possible fit to the original data points [13].After performing the principal components analysis, we can have a scatter plot in which the first and second (PC) (representing the largest fraction of the overall variability) are plotted on the horizontal and vertical axis respectively.We wonder which variables are responsible for the patterns seen among the observations, or which variables are influential.The distance to the origin also conveys information.The further away from the plot origin a variable lies, the stronger impact that variable has on the model, and precisely on the (PC).So when a variable has a high absolute coordinate on a factor, it contributes the most to associate a meaning to it.To interpret the PCA results we determine the variables with highest coordinates on the first factor, those with positive values give a meaning to the right side of the horizontal factor.The significance of left side of this factor is linked to the variables with the negative values.The same process is repeated for the other (PC).Using the results of variables analysis allows interpretation of observations.For example when the first component is highly determined by a certain variable on the positive side, this means that observations with large positive coordinates along this factor are characterized by this variable.Interpreting observations consist of examining their coordinates and especially their resulting graphical representation.We can have an idea about which observations are scattered and which ones are similar.We compute the SAS procedure PRINCOMP [12] on our (69) observations and our quantitative variables.One of its results is the correlation matrix which indicates that these variables are nearly non-correlated as the values of the correlation coefficients are between (0.39) and (-0.28).
The eigenvalues "Fig.5" indicate that the projection of the initial points on the two first factors (PRIN1, PRIN2) explains (36%) of the variation existing in the original data set [13].These two first factors (or PC) are then used and a meaning is attributed to each one in terms of its highest variables' coordinates "Fig.6".The results in the "Fig.6" show that the variables with the highest positive coordinates on (PRIN1) are (encourage, security, introe-Port).The variables with the negative coordinates are (Fullenglish, HalEAD).The factor (PRIN1) seems to oppose students who encourage the experiment and its extension, approving the edu-portfolio' security, to those who are more interested in changes in the module progress by transforming it to a "half" distance one and teaching it in English.In the second factor (PRIN2) the highest positive coordinates are associated to the variables (use, fulEAD).The negative coordinates correspond to the variables (Nocomment, IntroePort, Fullenglish).(PRIN2) seems to be opposing students appreciating edu-portfolio as far as the transformation of the module to a "full" distance one, to those who appreciate the experiment and are not convinced of the teacher's comment importance.These students are also favorable to teach this module in English.
The interpretation of the plot of the (PC) shows four groups represented by frames (A, B, C, D) "Fig.7".
The position in the frame (A) is due to variables (encourage, security, introePort) for the (PRIN1) and to the variables (use, fullEAD) for the (PRIN2).The frame (A) represents the students who are fervent supporters of the experiment and its extension.They also appreciate the ePortfolio as a concept as well as the used tool (eduportfolio) and the transformation of the module to a distance learning one.
The position in the frame (B) is due to the variables (Fullenglish, HalEAD) for the (PRIN1) and also to the variables (use, fullEAD) for the (PRIN2).The frame (B) represents students who appreciate the tool edu-portfolio but are essentially concerned by the transformation of the module to a distance learning one and also taught in English.
The position in the frame (C) is due to variables (encourage, security, introePort) for the (PRIN1) and to the variables (Nocomment, IntroePort, Fullenglish) for the (PRIN2).The frame (C) gathers students who support the experiment and its extension as well as those who have the intention to teach in English, and are not convinced of the teacher's comment importance.
The position in the frame (D) is due to (Fullenglish, HalEAD) for the (PRIN1) and to the variables (Nocomment, IntroePort, Fullenglish) for the (PRIN2).The frame PAPER AN EXPERIMENT OF INTRODUCING EPORTFOLIO TO EVALUATE STUDENTS IN THE TUNISIAN CONTEXT (D) gathers those who support teaching in English as well as introducing the ePortfolio and are also not convinced of the teacher's comment importance.
Most students have positive coordinates on the first factor (PRIN1) as proved by the "Fig.8" where students are sorted out according to the ascending order of their coordinates on (PRIN1)."Fig.8" represents only the beginning of this result to prove that students with negative coordinates on (PRIN1) correspond to the minority; they are only (29).Many other students (40) are then divided between the frame (A) and (C).So the student's majority are fervent supporters of the experiment, as well as the used tool.
They are also interested in the module's progress, so some of them encourage to teach it in English while others encourage to transform it to an online one.Because the first two factors (PRIN1, PRIN2) represent only (36%) of the initial inertia, we extended our analysis to the third one.With the factor (PRIN3) we have (51%) of the inertia.The interpretation of the projection on the factors (PRIN1, PRIN3) and (PRIN2, PRIN3) identifies the same students groups presented in the frames (A, B, C, D).We also used the forth factor (PRIN4), and with the precedent factors we can explain (64%) of the initial variation.The different plots' combinations of these factors highlight similar groups of students presented in the frames (A, B, C, D) with nearly the same distribution.

D. Lessons learned
Although our student were very reluctant to participate in this experience in the beginning, they ended becoming very enthusiastic about it and even asked for continuing it and extending it to the other modules.We claim that we succeed to sensitize our students to the ePortfolio concept but with great difficulties because the "classical" assessment using the paper test is easier than using the authentic assessment used in our experiment.According to Hammond and Snyder (2000) "...the authentic assessment includes opportunities for the development and examining of students' thinking and actions.This implies that authentic assessment requires to demonstrate their learning" [14].In fact we asked our students to demonstrate their learning by synthesizing what they retained from each chapter and we have to evaluate it and this is a time consuming activity.In fact, the "classical" assessment using the paper test is easier than using ePortfolio because correcting a unique paper-test during a semester for these (69) students is much simpler than correcting at least (9) deposit-work for every one of them .We have to evaluate nearly (30) ePortfolios weekly This assessment doesn't concern the same deposit-work as too few students meet the deadlines.Although we announced, at the beginning of the experiment that late deposits won't be accepted, we ended up by taking them.We did this in order to be able to continue the experiment as only (3%) deposited their homework on time.But even if deposits were on time, this experiment continues to raise the issue of the extra teaching load per hour for teachers whose aim is to enhance the ePortfolio use [15].
Because of administrative constraints and because it is a personnel initiative, our teaching load wasn't reduced and we didn't renew our experiment during this academic year.
To conduct such an experiment we recommend it is done in consultation with the institute administration be- cause the teacher's load shouldn't be evaluated on the only "effective teaching hours".Correcting the ePortfolio of each student and for each of its deposits requires more time than the correction of a written test.Besides, having the administration support is also preferable in order to motivate the students' participation.When the administration recommend the use of the ePortfolio, teachers as well as students feel themselves more engaged in the action.We also recommend to conduct such an experiment with few students (20 for example) to ensure its success.As a matter of fact we were about to give up the experiment several times and our only satisfaction was that we managed to teach our students the concept of ePortfolio to the point that they became seekers.The best thing to do is to establish a national policy to introduce the ePortfolio in our institutions but unfortunately this doesn't seem for the nearest future.

PAPER AN EXPERIMENT OF INTRODUCING EPORTFOLIO TO EVALUATE STUDENTS IN THE TUNISIAN CONTEXT
V. CONCLUSION This paper presents an experiment of applying the ePortfolio in a traditional master degree class in the Tunisian university.At the end of the semester a questionnaire was proposed to the students to collect their opinions about the experiment.The formal analysis of the questionnaire confirms our overall idea that most students appreciate the experiment.This confirms the students' feeling at the end of the semester: they enjoy the experiment.The students expressed their wishes to extend the ePortfolio use to the other modules too.The tool used to realize this experiment (edu-portfolio) is globally appreciated.Indeed, while evaluating the students' ePortfolios we find that some have even adopted the concept to extend their skills to other modules.The proposed method to assess student is also well appreciated by the students' majority.One of the revelations of the questionnaire is the students' approval of the idea of teaching this module in English.During the semester, when teachers proposed to teach this module in English, the majority of responders didn't welcome the idea One main suggestion given through the questionnaire answers is to transform this module to an "online"concept We started a project of transforming our module into a distance learning one which will be hosted at the virtual university of Tunis (http://www.uvt.rnu.tn).
Even if students are favorable to continue this experiment for the next year and to extend it to the other modules , it is unfortunately not the case for the teacher.As it still raises the issue of payment for extra teaching hours for teachers whose intend is to promote the ePortfolio use [15].Because of administrative constraints and because it is a personnel initiative, the working load for teachers hasn't been reduced and as a consequence , our experiment was not pursued during this past academic year.Therefore we recommend a close collaboration with the administration for the success of similar experiment.The introduction of the assessment by the ePortfolio requires most importantly a reconsideration of the working load per hour so as to make the experiment most efficient and profitable for both teachers and users .

Figure 1 .
Figure 1.The result of the procedure FREQ on the question "Did you know ePortfolio before the experiment"

Figure 2 .
Figure 2. The summary of the students' difficulties.
PAPER AN EXPERIMENT OF INTRODUCING EPORTFOLIO TO EVALUATE STUDENTS IN THE TUNISIAN CONTEXT

Figure 4 .
Figure 4.The results of the procedure MEANS(PC) are ordered so that the first retains most of the variation (inertia) present in all of the original variables.The eigenvectors are orthogonal, so the (PC) represent jointly perpendicular directions through the space of the original variables.The first (PC) has the largest variance of any unit-length linear combination of the observed variables.In geometric terms, the j-dimensional linear subspace spanned by the first j (PC) gives the best possible fit to the original data points[13].After performing the principal components analysis, we can have a scatter plot in which the first and second (PC) (representing the largest fraction of the overall variability) are plotted on the horizontal and vertical axis respectively.We wonder which variables are responsible for the patterns seen among the observations, or which variables are influential.The distance to the origin also conveys information.The further away from the plot origin a variable lies, the stronger impact that variable has on the model, and precisely on the (PC).So when a variable has a high absolute coordinate on a factor, it contributes the most to associate a meaning to it.To interpret the PCA results we determine the variables with highest coordinates on the first factor, those with positive values give a meaning to the right side of the horizontal factor.The significance of left side of this factor is linked to the variables with the negative values.The same process is repeated for the other (PC).Using the results of variables analysis allows interpretation of observations.For example when the first component is highly determined by a certain variable on the positive side, this means that observations with large positive coordinates along this factor are characterized by this variable.Interpreting observations consist of examining their coordinates and especially their resulting graphical representation.We can have an idea about which observations are scattered and which ones are similar.We compute the SAS procedure PRINCOMP[12] on our (69) observations and our quantitative variables.One of its results is the correlation matrix which indicates that these variables are nearly non-correlated as the values of the correlation coefficients are between (0.39) and (-0.28).The eigenvalues "Fig.5"indicate that the projection of the initial points on the two first factors (PRIN1, PRIN2) explains (36%) of the variation existing in the original data set[13].These two first factors (or PC) are then used and a meaning is attributed to each one in terms of its highest variables' coordinates "Fig.6".The results in the "Fig.6"show that the variables with the highest positive coordinates on (PRIN1) are (encourage, security, introe-Port).The variables with the negative coordinates are (Fullenglish, HalEAD).The factor (PRIN1) seems to oppose students who encourage the experiment and its extension, approving the edu-portfolio' security, to those who are more interested