A Cluster-based Personalized Item Recommended Approach on the Educational Assessment System

—Personalized item recommendation enables the educational assessment system to make deliberate efforts to perform appropriate assessment strategies that ! t the needs, purposes, preferences, and interests of individual teachers. This study presents a dynamically personalized item-recommendation approach that is based on clustering in-serve teachers with assessment compiling interest and preference characteristics to recommend available, best-fit candidate items to support teachers to construct their classroom assessment. A two-round assessment constructing activity was being adopted to collect and extract these teacher’ assessment knowledge (item selected preference behaviors), and through the designed item-recommendation mechanism to facilitate IKMAAS [1] to recommend proper items to meet different individual in-serve teachers. To evaluate the effectiveness and usability for the cluster-based personalized item-recommendation, the assessment system log analysis and the questionnaire collected from participating teachers’ perceptions were being used. The results showed the proposed item-recommendation approach based on clustered teachers’ assessment knowledge can effectively improve their educational assessment construction.


I. BACKGROUND AND MOTIVATION
Recommender Systems and related recommender technology applications can provide personalized information services by adopting knowledge discovery (such as Bayesian networks, decision trees etc.) [2] or data mining techniques from the actions and attributes of users (personality factors, behavioral factors, etc.) [3] are now considered to be the most promising way wherein many domains ranging from electronic commerce to personalized service and knowledge management, they not only enable to present specific objects in accordance with the different users' interest or need on the basis of his or her known preferences or reference to those of other users with similar characteristics [4], but also could efficiently filter out the overload of information or provide favorable information when the users need to make decisions [5]. Likewise, the applications of recommendation have gradually appeared in the area of e-learning and e-education, it seems to be as "right-hand man of learners and instructors" which recommend the best-fit educational resources that directly meet the learning needs of different learners, or help the individual instructors making better teaching decisions [6,7]. In which, these perspectives of personalized recommendation could also be designed into e-assessment or assessment systems, such as driving personalized item recommendation that advises different teachers on the appropriate assessment item selection or efficiently provides suitable item recommendation information referred to previous assessment activities. In Rafaeli's previous research, he has strongly advocated that if the development of an impeccable educational assessment system should integrate useful recommend technologies for the item selecting and in turn enhancing the effectiveness of educational assessment system to improve classroom assessment tasks. Even he also directly points out some essential issues, whether or not the researchers focus on the development of item recommendation for the nextgeneration assessment system [8]. How can item recommendations be performed most effectively? Who are the appropriate recommenders? What needs to be performed in constructing the recommendation algorithm to improve the use and acceptance of recommendations?
From the perspective of knowledge sharing, people might obtain the proper recommendation referred to the neighborhood group's preference or experience similar to themselves. The terms "groups" and "neighbours" which are defined as mutually similar data items depending on similarity are somewhat interdependent in the context of recommendation [9]. This concept is acceptable because it suggests that "neighbours" (a like-minded group) are relevant to low-risk recommendation domains, such as movies, music or jokes, whereas "friends" are more relevant to higher-risk domains, such as knowledge items used in learning and assessment [8]. Therefore, for this concept of recommendation referring from the similar neighbour group, we could apply this idea to design item recommendation for educational assessment system.
Currently, some developed assessment systems or tools were to facilitate the educational assessment construction and administration for the classroom teachers [1,10,11,12]. As is known, these systems take responsibility to help teachers to dominate the assessment arrangement and administer their assessment tasks. In one of our earlier projects [1], a well-designed, intelligent educational assessment assisted system, named the IKMAAS (Internet-based knowledge map assisted in assessment system) that was implemented to efficiently assist and enable primary school teachers in constructing their educational assessment activities at anytime and anywhere, and teachers can flexibly log in to check assessment progress. The PAPER A CLUSTER-BASED PERSONALIZED ITEM RECOMMENDED APPROACH ON THE EDUCATIONAL ASSESSMENT SYSTEM IKMAAS is as a knowledge management system that carries out the teachers' assessment knowledge acquisition and conversion, and allows it to be shared among teachers, thus assisting them in assessment tasks. In addition, the IKMAAS could automatically extract these abstract, implicit teachers' assessment knowledge, and sequentially calculate and transform to the specific item-selecting information through a knowledge map displayed that supports teachers in their assessment conducting tasks [1]. Obviously, a great amount of items-selecting information is undoubtedly preserved in the IKMAAS system, which contain diversified assessment information, including the teachers' assessment behaviors, which could be collected, stored and managed. Consequently, in the context of this study, we consider that item recommendation method motivated by referring this item-selecting information whose analyzing these similar assessment interest and preference characteristics extracted from the IKMAAS system's records, have the potential to help in-serve teachers construct their assessment tasks.
In order to carry out the assessment records extraction and analysis work smoothly before executing item recommendation through IKMAAS. It becomes possible to discover similar data distributions and patterns, group physical or abstract objects into classes of similar objects by data mining and information extraction techniques. In which, the main intention of clustering is used to discover interesting data distributions and patterns, and group physical or abstract objects into similar classes, it helps construct meaningful partitioning of a large set of objects based on a 'divide and conquer' methodology that decomposes a large-scale system into smaller components to simplify design and implementation [13,14]. The clustering is the process of grouping the data into classes, so that physical or abstract objects within a cluster have high similarity in comparison to one another, but are very dissimilar to objects in other cluster [15,16]. Currently, many of the clustering applications can be adapted to design a systematic grouping method in several e-learning settings [17,18]. For example, Ref. [19] used this technique to find clusters of students with similar learning characteristics and to promote group collaborative learning as well as to provide incremental learner diagnosis; Ref. [20] grouping students in order to give them differentiated guiding according to their skills and other characteristics; Ref. [21] explores the possibility of using a hierarchical clustering and analytical procedure to diagnosis individual and class learning and misconceptions; Ref. [22] proposed the ontology and the clustering techniques for classifying the behavior of the students which is based on the level of meaningful learning characteristics from their log activities in Moodle e-learning system; Ref. [23] use the frequency of access and the duration of sessions to establish several categories of learners by cluster analysis, which depict the differences among the cohort in terms of participation; or Ref. [18] used clustering algorithms to compare versus classification for predicting whether students will pass or fail the course on the basis of data about their forum usage.
As mentioned above, the main purpose of this study is to develop the efficient, personalized item recommendation mechanism, using the IKMAAS as the platform, to assist teachers on classroom assessment conduction. The mechanism works by using a cluster-mining method to cluster historical teacher assessment compiling records as the assessment knowledge from the item repository, and makes a personalized recommendation for the teacher to provide both the proper item selecting proportion and the best-fit test items available when they conduct their educational assessment. This personalized item recommendation approach initially appliesd an implemented clustering algorithm is employed to distinguish teacher groups who might have similar assessment selecting behaviors, and then provide item recommendation as an integrated module in a previously developed assessment system IKMAAS to assist teachers to find the proper items and carry out the assessment construction more effectively. An experimental design was adopted to understand the effectiveness of personalized item recommendation in collecting and analyzing teacher assessment knowledge of item preference behaviors to perform an educational assessment compiling and in improving teacher assessment work performance.

A. Clustering teacher's assessment knowledge
In order to perform personalized item recommendation based on clustered teacher's assessment knowledge to assist teachers in classroom assessment construction. The first and foremost mission is to proceed cluster work from the IKMAAS's item repository. Among these cluster approaches, the most well-known and commonly used partitioning method is k-means. It takes the input parameter k, and partitions a set of n objects into k clusters so that the resulting intra-cluster similarity is high but the intercluster similarity is low. Cluster similarity is measured in regard to the mean value of the objects in a cluster, which can be viewed as the cluster's centre of gravity. This algorithm allows us to specify the number of clusters (k), which given an initial set of k-means ) ( k m , the algorithm proceeds by alternating between two steps which was described below, and it is said to converge when the clustering assignments no longer change.
First step: Assign each item to the cluster with the closest mean.
Second step: Calculate the new means to be the centroid of the items in the cluster.
We adopted k-means clustering algorithm, the reason because it possess iterative refinement characteristics based on a Euclidean distance metric, and has become a popular, simple, useful, and an automatic clustering method [24,25]. In this study, we select four mainly critical assessment information regarding item-concept weight, Bloom's taxonomy, item difficulty, and item discrimination as meaningful assessment attributes which are used to find similar clusters for classifying these teachers' assessment construction behaviours. These assessment information involved with different teachers' item selecting data has previously stored in the IKMAAS's item repository. Each of the four critical assessment information is basically divided into high and low degree of the parameter, and transformed to interval-scaled variables to facili-PAPER A CLUSTER-BASED PERSONALIZED ITEM RECOMMENDED APPROACH ON THE EDUCATIONAL ASSESSMENT SYSTEM tate k-means clustering (as shown in Fig 1). Due to the major limitation of using the K-means algorithm is that the number of clusters needs to be predefined and considering the number of participants and satisfy the condition of identifying the most suitable number of clusters which is decided based on not only the smallest distance between the features in a same cluster, but also the largest distance between the features in different clusters in the K-means algorithm. The parameter exploration was applied to decide the parameters of the K-means algorithm in this study. First, we searched for 3-5 clusters for the cluster initialization which is used to decide the suitable value of parameters. Form the test result, the best fitting cluster model with four clusters was selected by taking into account the greatest distance between the final cluster centres. After doing so, we found that the K-means algorithm produces more efficient outcomes for four clusters, that means the participating teachers are divided into four groups and ensures that each group included at least two teachers, to resolve the issue of outliers and avoid each cluster that included just a single teacher. Therefore, these similar assessment item selecting preferences will be analyzed by using clustering to provide the next stage of the item recommendations.

B. Item recommendations methodology
The personalized item recommendation approach was executed to predict the active teacher's assessment preference for a particular item by comparing and clustering other teachers' similar assessment knowledge, and then give favorable item recommendation for active teachers. The underlying framework on item recommendation for the cluster-based filtering method is presented in Fig 2. The item recommended scenario assumes a set of n teachers, U= {u 1 , u 2 , … u n }, and a set of m assessment items, I= {i 1 , i 2 , … i m }. Each teacher u i has a list of items selecting I ui (where I ui ! I, and I ui could be an empty set) for which the teacher has expressed assessment preferences. The preference of a teacher u i on item i j might be an explicit numerical scale rating or an implicit measure based on teacher-compiled assessment activities. In this study, the frequency of being selected items represents the degree of assessment preference so that the items selected by the teachers in the group are an implicit measure of teachers' assessment preferences. In addition, items are grouped into distinct clusters based on similarity of assessment preferences, and similarities between item clusters are measured during the clustering process. For each cluster Cr, the average item preference score of a teacher u i is given as r nc P . Based on concept weight, item difficulty, item discrimination and Bloom's cognitive taxonomy for a specific teacher u a (where u a ! U), the recommendation decision predicts the preference of that teachertermed an active teacher-for a particular item that this teacher has not rated. In this recommendation framework, the similarity between the active teacher and another teacher is determined by the similarity of their assessment preferences both within and between item clusters. C. The mechanism of personalised item recommendation in IKMAAS Before executing the item recommendation, the teacher must initially complete the registration to create a personal assessment profile by the IKMAAS. In case the teachers construct an assessment by the IKMAAS, and the assessment constructed and item selected record portfolios are stored in the IKMAAS. All of these personal assessment portfolios are involved with various of personal assessment information included identification and login information, previous assessment knowledge, discipline, assessment frequency and some detailed assessment operating information regarding assessment conducting and item selecting records.
The mechanism of personalized item recommendation based on clustering the teachers' assessment knowledge profiles consists three modules: teacher assessment data aggregation, clustering pattern analysis and item recommendation (see Fig. 3). All modules are performed online in the IKMAAS. The module of teacher assessment data aggregation automatically transforms teachers' assessment log files and profiles in the database into the proper format. The module of pattern analysis is used to perform the k-mean cluster analysis on these assessment profiles. And the module of item recommendation is used to provide the specific teachers with proper personalized item choices by the analyzed assessment patterns. Moreover, when teachers want to construct the assessment by the IKMAAS in turn, three alternative user interfaces for item selecting were provided: manual item selection, item recommendation employing clustering assessment knowledge and the automatic item selection. Teachers can select one of three alternative user interfaces according to his or her needs to construct their assessment.
• Manual item selection: The interface of manual item selection displays all of the items in the IKMAAS database and the teacher could search or select specific items freely. • Item recommendation based on clustering assessment knowledge: The interface of item recommendation based on clustering assessment knowledge can provide appropriate candidate items according to predict his or her particular assessment preference and then searching the closed teacher's clusters. In this interface, several candidate items involved with the particular teaching concept were shown, and the teacher PAPER A CLUSTER-BASED PERSONALIZED ITEM RECOMMENDED APPROACH ON THE EDUCATIONAL ASSESSMENT SYSTEM could add into the assessment lists by accepting or rejecting the recommendations. By the way, one limitation is that the new registration teachers might obtain a poor item recommendation if his or her current assessment behaviors are not analyzed or stored into IKMAAS.
• Automatic item selection: The interface showed an item list according to the teacher identifies the number of the items desired. These items were automatically selected by personalized item recommendations from clustering similar teacher assessment knowledge.

III. EXPERIMENTAL DESIGN
To test the effectiveness of the personalized itemrecommendation mechanism, the researchers conducted a two-round experiment to compare the effects of different ways of providing item recommendation, and to evaluate the feasibility and usability of item recommendation based on clustering of teachers' assessment knowledge (as shown in Figure 4). In this two-round experiment, eighteen third-grade elementary teachers in Taiwan were invited to use the IKMAAS system for editing their assessments for the topics of "natural science and life technology" for a period of three months. Four topics of item selecting for different course was scheduled repeatedly and conducted in the two-round assessment activity. During the first item compile round, teachers construct assessments without using the personalized item recommendation. During the second item compile round, teachers using personalized item recommendations to construct assessments by the IKMAAS. After finishing the second item compile round, teachers were asked to complete a questionnaire and accept an interview about their reflection on using this item recommendation mechanism.

IV. RESULTS AND DISCUSSION
This section presents the experimental results which evaluated the item-recommendation approach based on clustering teacher assessment knowledge. A cluster analysis was performed after participating teachers finished their assessment construction in the first-round assessment activity, and then the system will be executed to calculate these similarities of teachers' assessment constructing behaviors. When the teacher goes on to construct assessment by applying item recommendation mechanism from IKMAAS in the second-round assessment activity. The system will immediately identify his/her assessment preferences and recommend appropriate candidate items from searching the closed k-nearest neighbor teacher groups with the highest similarity. Within the experiment results, the system usage logs were used to evaluate the effect on the item recommendation mechanism through comparing the teacher's behavior of assessment construction between the first and second round assessment activity. And the questionnaire and interview were used to investigate the teachers' perception and use willingness when they use an item -recommendation mechanism to made constructing assessments. The analyzed results are discussed and interpreted below respectively. Above all, With regard to system usage logs, the dispersion of the standard deviation was calculated to compare the similar dispersion on the item selecting for three different assessment records: items selecting compared among all teachers selected items, each teacher within the clustering and group clustering The analyzed result that compared three values of standard deviation between item selection for four different assessment topics among individual, the clustered groups and all teachers' group was presented in Figure 4. Blue line is used to indicate the standard difference between compared with each teacher's item selected behavior. Red line is used to indicated the standard difference between compared with each teacher within his or her clustering group and different clustering groups, and green line is presented to the standard difference between they adopted item recommendation mechanism. Here, we noticed that green line is presented the lowest rate than the other two lines, it meant that if teachers are not grouped, the degree of dispersion for recommended items will have a significant difference, the central value of group for item recommendation will not meet the needs of teachers, but compare with teachers through item recommendation by clustering teacher assessment knowledge have the smallest differences. Moreover, we founded that more than half of teachers directly chosen these items which generated out from the item recommended mechanism. This system PAPER A CLUSTER-BASED PERSONALIZED ITEM RECOMMENDED APPROACH ON THE EDUCATIONAL ASSESSMENT SYSTEM usage analyzed result could support the claim that item recommendation is beneficial to assisting teachers' assessment construction. Figure 5. Comparison of item similarity for the three item-selecting modules One questionnaire was used for gathering the teachers' perception toward the usability of the item recommendation mechanism. It mainly intended to measure seven different dimensions involved with teacher background, teacher motivation, assessment judgments, perceived usefulness, perceived ease of use, assessment attitude, intentions, and especially access teachers' perception for using item recommend mechanisms. The questionnaire consists 32 items which be measured by the five point Likert scale. These questionnaire results are be out of statistics and analysis which showed in the figure 5. In the dimension of Assessment judgment, as we can see that teachers usually consider the priority of the assessment criterion is student learning conditions (100%) and their teaching schedule (83.33%), when most of participated teachers constructed their new educational assessments; in addition, when teachers were constructing assessments, they still considered these assessment factors: Course concepts (100%), Item difficulty (94.44%), Item discrimination (55.56%), and Bloom's taxonomy (33.33%).It could be used to prove that the developed item recommendation approach considered these assessment factors matched the teachers' assessment belief. Besides, about 94.44% teachers are willing to use these similar assessment items from other teachers as well as share their assessment information with other teachers, as well as 88.89% teachers will revise these previous items when they reuse these assessment items. Overall, the teachers' responses are positive on the Perceived usefulness, Ease of use, Attitude, Intentions and Satisfaction with the item-recommendation.
In addition, we also explored teachers' opinions by interview for the usefulness and clarity of the item recommendation mechanism. Some teacher comments are as follows.
Teacher A: I could refer to the item-recommendation based on other teachers' assessment knowledge to identify or filter redundancy items; I could select the items I wanted.
Teacher H: It is convenient; I can find similar assessment knowledge teachers' assessment items by using the item -recommendation system, which reduces the time spent constructing a test.
Teacher K: Although this system helps me construct a test, I think the item-recommendation system could be improved by adding the degree to which an item is recommended. Overall, through their system usage logs and answers to the questionnaires, the results of the experiment show that the item-recommendation mechanism based on assessment knowledge of clustered teachers can effectively improve their assessment work, and most teachers approved of the efficiency and the convenience of the item recommendation and agreed that they actually reduced the stress of compiling assessments. It enhanced assessment construction and that the item-recommendation mechanism reduced the time spent constructing assessments. Although a few teachers consistently avoided items recommended by the IKMAAS due to concerns regarding the ability to control for previous item exposure.

V. CONCLUSIONS
This research proposes a personalized itemrecommendation mechanism that automatically adapts to the assessment interests and preferences of different teachers. When teachers compile their assessments, this mechanism helps the teachers by recommending the bestfit test items available in the item banks and by displaying their allocation information. The mechanism is built into an e-assessment system named IKMAAS, which was built in one of our earlier projects. It utilizes clusters of teachers' assessment knowledge to provide appropriate items and facilitate to primary school teachers' assessment construction. The mechanism performs a series of tasks. Firstly, the mechanism finds the clusters of similar teachers based on their assessment inclinations by using a clustermining method that is based on the K-means Clustering Algorithm. The inclinations of teachers (such as, their preferences for particular assessments and their interest on specific course topics) are collected as assessment knowledge by the IKMAAS. Secondly, when a teacher is PAPER A CLUSTER-BASED PERSONALIZED ITEM RECOMMENDED APPROACH ON THE EDUCATIONAL ASSESSMENT SYSTEM creating an assessment, the IKMAAS makes a personalized recommendation of the teacher regarding both the proper proportion of the number of test items to the to-betest course concepts and the best-fit test items available in the item banks for each course concept. To make these recommendations, the IKMAAS uses the assessment knowledge of the cluster in which the teacher belongs.
The effectiveness of item recommendation is evaluated by examining the system usage logs and user questionnaire responses. The results indicate that the item recommendation based on clustering teacher assessment knowledge indeed benefit to help primary school teachers on assessment construction. Based the proposed clustered item recommendation method, it can help teachers to perform precise assessments based on clustering the assessment portfolios of individual teachers collected from IKMAAS system. The inferred assessment performance can be adopted as a reference guide for teachers. The recommendation mechanism benefits all teachers to understand their current assessment status and make appropriate assessment adjustments.
Although the item recommendation served on the IKMAAS by the cluster algorithm to gathering teacher assessment information, generate the 'friends' group, and then retrieve the item recommendation. The IKMAAS was initially designed to promote assessment collaboration and knowledge-sharing among different teachers. The IKMAAS has a small-grained focus on items sharing and provide a convenient assessment construction environment, especially emphasize assessment knowledge sharing among teachers. In certain extent it allows teachers to work ''together apart'' by sharing their assessment knowledge and allows instructors to share ''craft wisdom'' in a professional online assessment culture.
Future research will investigate clustering procedures should examine larger samples of teachers, different assessment topics and obtain more extensive teacher assessment profiles. Other data-mining algorithms, in particular, clustering algorithms that do not require teachers to specify any parameters could also be examined to assist educational assessment construction.