Semi-Automatic Concept Map Generation Approach of Web-Based Kit-Build Concept Map Authoring Tool

—Apart from contributing to students' learning outcomes, learning activities with digital concept maps were useful, fun, and engaging. Kit-Build concept map is a learning framework that incorporated concept map recomposi-tion as its essential activity. Students learn through recomposing digital concept maps from a set of teacher's concept map components; hence, a teacher concept map is essential in Kit-Build. In composing a teacher concept map of Kit-Build, teachers should reflect the learning context and strategy, embody their purpose and intention, students' understanding level, and focus questions in the forms of concept maps. Automatic generation approach produces general concept maps that are perceived to be unsuitable in corresponding said reasons; thus, a semi-automatic approach becomes preferred. The Kit-Build concept mapping tool has been extended with a support function that semi-automatically generates concept maps with Concept Map Mining approach. The design of the extraction and summarization phase of the concept map generation process, which suggests the extracted concepts and proposition triples to the teachers, is presented in this study. However, the accuracy of the support system's suggestions has yet to be discovered before the tool being considered to be used in a real learning environment of EFL reading comprehension with Kit-Build and therefore investigated in this study. The result suggested that the proposed Kit-Build concept map authoring support tool is better used to refine a concept map in more detail.


Introduction
A concept map can be identified as a form of boundary objects. A boundary object is a tool, artifact, or scaffold to mediate discussion and negotiation between two or ing. Even though concept maps could be generated automatically from texts, the generated concept maps have issues regarding coverage, accuracy, readability, and suitability [27] and yet satisfying to be practically used in supporting learning EFL reading comprehension with Kit-Build. Therefore, semi-automatically generating the concept maps with human interference becomes an option to address the issue.
This study presented the design of the generation approach of a concept map authoring tool that adopted the CMM approach and text mining to generate concept maps of EFL reading comprehension texts semi-automatically. The text processing of the authoring support tool was implemented and developed using web technology, and therefore the application was generally accessible through web and mobile devices. The designed authoring support tool was semi-automatically generating the concept maps by suggesting keywords and proposition triples for the teachers to choose, modify, and incorporate the suggestions into their concept maps. However, before the tool is considered to be used practically with Kit-Build, the performance of the authoring support system has to be evaluated. Therefore, the accuracy of the suggestions yielded by the system, and how the tool is perceived, need to be discovered.
Assessing an education application could depict how the application is used and discover how it further affected the learning [28,29]. However, before evaluating the learning outcomes, several experts were involved. They used and evaluated the performance of the authoring support tool in suggesting concept map components of EFL reading comprehension texts and assisting their concept map composition activity. In order to guide the study, the following research questions were addressed: 1. With the designed extraction approach towards EFL reading comprehension learning strategy with Kit-Build, what is the accuracy of the suggested keywords and proposition triples of the authoring support tool? 2. Will the developed concept map authoring support tool perceived to be useful for assisting teachers in composing concept maps from EFL reading comprehension texts and support their teaching activities with Kit-Build concept map?
The results suggested that the authoring support tool provided good accuracy in suggesting the concepts; the proposition triple suggestions were acceptable. The tool was perceived as useful to assist teachers compose concept maps of EFL reading comprehension texts, especially in refining a concept map more detail. The design of semi-automatic generation approach is presented in the Literature Review; the methodology and the EFL learning strategy with Kit-Build concept map are presented in Methodology section; the results are discussed in the Result and Discussion section; the remaining sections conclude the results and present the limitation and future work.

Kit-build concept map
Kit-Build concept map is a learning framework that incorporated high-directed concept maps in its learning strategy. Concept map re-composition is the key activity in learning with Kit-Build concept map. Students recomposed a set of concept map components-called a kit-to represent their understanding of a particular learning topic. Using concept maps in digital form to compose and recompose concept maps, Kit-Build offered a quick and easy assessment of students' understanding by comparing students' concept maps with a respective teacher concept map [17,30]. The comparison and analysis could be easily performed with the provided authoring tool and analyzer. The general process of learning with Kit-Build is depicted in Fig. 1. As previously mentioned, students recomposed their concept map from a Kit-Build kit that is a decomposition of a teacher's respective concept map. Before students recomposed concept maps, teachers have to compose concept maps and decompose the map into a kit before conducting the actual learning activity with Kit-Build concept map. Teachers have to carefully plan and compose their concept maps beforehand because their concept maps held important concepts and ideas of a learning topic. Thus, the concept map in which be decomposed into a kit could help the students focused on important ideas represented by the kit. Extending the concept mapping activity [15] and incorporating Kit-Build into collaborative learning [31,32] could also improve their interaction during learning [21,33] and help students to understand and comprehend the learning material.

Concept's extraction and prioritization
Concept's labels in a concept map are commonly specified from a document keywords as they most likely represent the document's topics or main ideas. There are many ways to determine keywords for a text document, but most existing approaches use manual assignments by experts or defined manually based on the concept map's authors' judgment. However, with the NLP and text mining techniques, it is possible to extract keywords from a text document based on statistical approaches.
Most concept labels in a concept map are nouns or noun phrases. Therefore, it is evident that one approach to extracting concepts from a text is by capturing all of the nouns and noun-phrases tagged from NLP's annotation process. However, not all nouns or noun-phrases are essential to consider to put into a concept map. Subject to using the concept maps, a concept label should be non-trivial and essential. The relevance of words or keywords to a document can be measured by their frequency of appearance [34]. The term Term Frequency-Inverse Document Frequency (TF-IDF) is a popular algorithm in calculating the weight of terms or keywords to a document. The TF-IDF weight (wij) of a term i in a document j can be computed from the number of occurrences of the term in the document (tfij), the number of documents containing the term (dfi), and the total number of documents (N) in a corpus as in (1).
The similarity between two keywords can also be computed by finding the cosine similarity ( ( )) between vectors of two keywords ( & 000⃗ , ' 0000⃗ ) composed of n unique words from both vectors. The similarity value could be calculated based on the TF-IDF weight of the composing words in each keyword (wij, wik) as in (2).
An unsupervised method for extracting keywords from a text document, namely Rapid Automatic Keyword Extraction (RAKE), measures the importance of a keyword to the document based on its composing words' degree and frequency [35]. The degree of a word is defined as the frequency of words appear in the keyword candidate list plus its frequency of co-occurrence with other words in the candidate list. The score of a keyword (SR), which composed of n words, can be calculated as a sum of the ratio of each composing word's degree (deg(wi)) and its frequency of appearance (freq(wi)) in the candidate list as in (3). Important keywords can be selected from top T scoring keywords from the list or by setting a minimum keyword score. When working with keywords on a single document, RAKE is also more computationally efficient than a graph-based ranking approach, such as TextRank [35].

Relationship extraction
In the extraction process of propositions from a text, a CMM system should identify concepts and relationships (links). Identifying an accurate and meaningful relationship between a link and two concepts plays an essential role in forming a concept map proposition. A concept map proposition can be represented as a triple of concept-linkconcept or a set of subject-relation-object in open information extraction (Open IE). The Open IE annotation process of Stanford CoreNLP annotates and extracts triples from a text, representing a subject, relation, and relation object [36]. The Open IE annotation corresponds to the open domain relation that captures the relation phrases expressed by the combination of verb-nouns patterns [37] and a natural logic classifier [36]. The triples resulting from the syntactic relationship extraction process serve potential candidates for concept map propositions.
Relationships in a complete sentence were mostly constructed by verbs. However, many sentences use a combination of verb-noun phrases instead of a single verb to depict a meaningful relationship between subjects and objects. Therefore, identifying relationship by relying on a single verb is insufficient to identify one good and meaningful relationship. A simple regular expression pattern can be applied to a sentence's POS tags annotation sequence to identify verbs and verb-noun phrases as a potential relationship candidate for propositions in a concept map. The extraction process that uses the pattern to extract relationships from a sentence is called the syntactic relationship extraction [36]. The pattern is given by (4) V | VP | VW * P (4) where V = verb particle? adverb? W = noun | adjective | adverb | pronoun | determiner P = preposition | particle | information marker Furthermore, the pattern could reduce the number of uninformative relationships extracted from a sentence [37] and improve the extraction performance [38].
The computed distance between two keyword vectors represents the similarity level between keywords. The similarity among two keywords or phrases using cosine similarity measure can be determined by (a) calculating the keywords' TF-IDF value with (1), (b) transforming the keywords into vectors space model, and (c) computing the distance of both vectors with cosine similarity measure as in (2).

Methodology
This study focused on supporting the concept map composing activity of teacher concept map semi-automatically. Support was given in the form of recommendations of concepts and propositions extracted from English texts. The extraction process adopted the CMM approach that involved NLP and text mining techniques. In addition to presenting the design of the extraction process, the accuracy of the yielded recommendations was analyzed and evaluated.
The Kit-Build concept map authoring tool was designed to be used on tablet computers and implemented in HTML5 and Javascript technology. The support feature extended the current authoring tool and could also be run on the same platforms. Users could use the tool with the new support feature with their personal computers or tablet devices. This study extended the current Kit-Build concept map authoring tool's functionality by adding a recommendation system as an authoring support feature and enriched the way teachers composed and improved their concept map. The support function recommended keywords and proposition triples while also allowed modifications made to the suggested items.
The design and development of the authoring support function are shown in Fig. 2. Before the design and development of the support function were conducted, a prelim-inary study regarding Kit-Build concept map framework, CMM, and the Stanford CoreNLP toolkit was carried out to identify and analyze how CMM and the NLP toolkit were able to extract concepts and propositions from English reading texts. How teachers get assistance from the support function was designed following the strategy of learning EFL reading comprehension with Kit-Build concept map.

Fig. 2. Research Methodology
Review and analysis of the current Kit-Build concept map tool were carried out to identify how the authoring support function could be integrated into the current Kit-Build concept map tool, thus resulting in a general requirements specification. The support system architecture, activities, and text processing were designed according to the requirements specification. After all of the designs were implemented into the target program and prototype, several tests were carried out to ensure the system work as designed, thus yielded the expected outcomes.
The support system performance regarding support function accuracy was evaluated using several English reading comprehension materials. Several English teachers evaluated the system and composed concept maps using the support function of the authoring tool. The yielded suggestions of the support function were evaluated and classified by the teachers per their initial concept maps to measure the accuracy of suggested items. Additionally, the evaluators were given a questionnaire and requested to evaluate the tool's support function regarding their perceived usefulness in contrast to the traditional concept mapping approach with Kit-Build concept map.

EFL reading comprehension learning strategy
The concept mapping strategy that was applied to this study consisted of several steps. First, teachers created a concept map that sophisticatedly represented the reading and decomposed the map into a set of concept map components (Kit-Build kit). The map would be used in the next phase of learning with Kit-Build concept map framework, where students reconstructed the map to express their understanding regarding the reading. One sophisticating concept map should have enough relevant concepts and relationships to represent the content. The map should not be too general to represent the text with merely a small number of concepts and relationships or be overwhelmed with many complicated concepts and unnecessary relationships that might confuse the students.
During the comparison analysis phase, students' concept maps were compared with a teacher concept map. The comparison identifies their misconceptions or misunderstanding regarding the reading pointed by the different and missing parts. The teachers were then further explaining the reading and refining their concept map to a more detailed concept map that better cover the missing and different parts. In this learning strategy of EFL reading comprehension with Kit-Build concept map, teachers composed their concept map at least twice, i.e., during the initial concept map composing activity and during the refinement of their initial concept map. Therefore, the authoring support features were expected to support their composing activity in these two situations.

Support function evaluation
To evaluate the support system's performance, three English teachers were selected as expert evaluators based on their expertise in using concept maps in teaching EFL reading comprehension. They were requested to compose their initial concept maps of a sophisticated level based on Mueller's concept map rubric by using the system with support function [39]. Upon reviewing several publicly available concept map evaluation rubrics, they had agreed that Mueller's concept map rubric was compatible with the strategy of learning EFL reading comprehension with Kit-Build concept map; hence, used in this research.
Fifteen reading comprehension texts of Barron's TOEFL iBT learning materials [40] were selected to evaluate the accuracy performance of support function. During their initial concept map composing, teachers were requested to use the authoring tool's support features to get recommendations of keywords and proposition triples. They were given a tutorial and practicing to use the concept map authoring tool.
A recommendation system's accuracy can be evaluated from the positive predictive value (PPV) and the true-positive rate (TPR). In information retrieval, PPV and TPR are generally called by precision and recall, respectively. PPV considers both truepositive items (tp) and false-positive items (fp), while TPR considers both true-positive items and false-negative items (fn). PPV or precision was calculated with (5), while TPR or recall was calculated with (6).
In evaluating the suggested keywords and proposition triples' accuracy, the teachers were asked to identify and classify every keyword and triple from the suggestion list for its relevance and appropriateness to their concept maps. The support function accuracy performance of this study was measured by F-measure (F1). F-measure is commonly used to evaluate the performance of information retrieval systems such as search engines, machine learning models, and natural language processing. Both pre-cision and recall values were considered in calculating the system's accuracy performance. The F-measure is formalized in (7).
True-positive items (tp) were defined as the number of suggested items and were used as part of their concept maps, either with or without slight modifications. The modifications may include adding or renaming part of the concept's or link's label; or altering the suggested triples. On the contrary, false-positive items (fp) were defined as the number of suggested items not used in their concept maps. Part of their concept maps, which were not suggested by the support tool and had to be created manually, were identified as false-negative items (fn). True-negative items were irrelevant in this study, because the system did not suggest the items or used in the concept map.
In this study, the support system was used in two concept mapping activities. First, it was used during the initial concept map composing; and second, it was used during the refinement of teachers' concept map. As the purpose of both concept mapping activities was different, the accuracy of the suggestions might differ. Thus, suggested concepts and propositions, which were not included in teachers' initial concept maps but potentially used to refine the concept maps, were classified differently. These potential suggestions were classified as false-positive potential (fpp). Therefore, falsepositive potential items were counted as false-positive and true-positive for the initial and the refinement concept mapping, respectively. The precision value of the support function for the initial concept mapping was calculated with (8), and the precision value for the refinement concept mapping activity was calculated with (9).

4
Result and Discussion

Architecture design and support processing flow
The system was built on top of web technologies, thus accessible from most computers and mobile devices with access to the Internet. A web server served the application interface and provided the data Application Programming Interface (API) to communicate with the database. Both the NLP toolkit and Web API transferred the data in JavaScript Object Notation (JSON) format through the Hypertext Transfer Protocol (HTTP). The processing layer of the support system is depicted in Fig. 3.
The support system processing logic was integrated into the Kit-Build concept map web application. The essential part of the processing layers was the NLP toolkit and served by the Stanford CoreNLP server. The toolkit pre-processed, parsed, and annotated the text input. The Stanford CoreNLP annotated the English input text with tokenization, sentence splitting, lemmatization, part-of-speech (POS) tagging, dependency parsing, coreference resolution, and open information extraction. The POS tagging annotation identified and labeled noun and noun-phrases for keyword identification processing. Each identified noun or noun-phrase was calculated for its weight by the TF-IDF method. Similarities among keywords can be computed with the cosine similarity measure. RAKE ranked the identified keywords according to the text. The Stanford CoreNLP also yielded Open IE triples that already in a subject, relation, relation-object format. All triples, whose subject or relation-object has a strong similarity to the extracted keywords would be suggested to the teachers. However, teachers can also opt to see all of the Open IE annotated triples. The Syntactic Relationship Extraction process extracted possible relationships and triples based on the regular expression patterns as in (4). According to the review of the annotated text, some of the Open IE annotation results were unsatisfactory. Several triples resulting from the Open IE annotation were considered unnecessary, and verb-based linking words cannot be properly identified. Thus, the Syntactic Relationship Extraction (SRE) was incorporated to identify verb-based relationships of a sentence.
The system's suggestion processing flow in recommending the keywords and proposition triples and the relevant application layers that process the document text is shown in Fig. 4. The processing flow was initiated when teachers opened a text file. The processing flow was divided into two parts to optimize the processing time. The first part-the pre-processing-ran immediately after the NLP toolkit had annotated the text. The process extracted essential keywords from the text and calculated the

Concept Map Database
Learning Content Provider

Concept Map Authoring Tools and Libraries
Concept Map Canvas similarity among keywords. The second part ran during sentence navigation and processed the currently selected sentence.

Implementation result
The support system's interface and functionality were mostly implemented in web programming languages. The Web API was implemented in the PHP programming language. The cosine similarity, RAKE, SRE, and TF-IDF processors were implemented with Javascript and ran client-side. The graph visualization of the concept map used the Cytoscape.js library [41] with several plugins added.
The keywords and proposition triples were served on a sentence-by-sentence basis, which means that the given suggestions reflected a sentence selected by the teacher. The suggestions were presented to the teacher in a support dialog, as depicted in Fig.  3. Using the support dialog, teachers can navigate the text to get suggestions for another sentence. Teachers could modify the triples and put the desired keywords into the canvas from the suggested keywords and proposition triples. They can also modify the propositions, organize the concept map layout, and alter the map's visual appearance with the existing simple concept map composing tool.
As depicted in the support dialog in Fig. 5, the suggestions were divided into three sections. Each section represented the suggestions from RAKE, SRE, and Open IE. The topmost part, which consisted of three suggested proposition triples, was obtained from the SRE method. The center part, which consisted of one suggested proposition triple, was obtained from the filtered Open IE method. Lastly, three keywords were suggested from the RAKE approach. The label of a proposition triple's subject and relation object can be modified or replaced with any suggested keywords. Thus, forming a more appropriate proposition before incorporated the triples as part of the con-

Cached output
Suggested Items cept map. Keyword added to the canvas would represent a concept in the concept map. Additionally, teachers could compose a proposition triple manually.

Initial concept mapping using the authoring support tool
The system evaluation involved 15 EFL reading texts. According to the evaluation result, on average, the support tool suggested 157 keywords, 85 proposition triples of the SRE approach, and 62 proposition triples of the optimized Open IE. The SRE method extracted and recommended verb-based proposition triples according to the POS pattern of a sentence [37], while Open IE triples corresponded to open domainbased relations [36]. If all said keywords and propositions were visualized into a concept map, a relatively huge concept map would be generated, and evaluating such a concept map would be challenging. As generating proper and satisfying concept maps was difficult to achieve, the semi-automatic approach proposed was presumed a better approach than a fully-automated or manual approach, at least in this study.
On average, the EFL readings used to evaluate the support features were composed of 719 words and 34 sentences with a standard deviation of 50 words and six sentences, respectively. The description of the EFL reading comprehension text used in the evaluation of the tool, the number of sentences (ns), the number of suggested keywords (nk), the number of suggested propositions from the syntactic relationship extraction (nps), and the number of suggested propositions from the filtered open IE triples (npo) are shown in Table 1. During the initial concept mapping activities, teachers created 15 sophisticating concept maps-one concept map for each reading, resulting in concept maps composed of 35 concepts and 36 propositions on average. Composing concept maps with such numbers of concepts and propositions was not an easy task. Moreover, teachers were requested to compose many concept maps in a relatively short time. The descriptive statistics of the expert teacher's initial concept maps during authoring support system evaluation are shown in Table 2.

Support function accuracy performance measurement
Three expert teachers classified the suggestions into a statistical confusion matrix, representing instances in the predicted classes and instances in the actual classes based on their initial concept maps. Each suggestion is classified into one of truepositive, false-positive, false-negative, or true-negative class. The classifications process was also carried out to all groups, i.e., RAKE keywords, SRE propositions triples, and filtered Open IE triples.
The support system's accuracy performance can be depicted from the F-measure value, which is the harmony between precision and recall values. According to Table  3, it can be seen that the system has the lowest F-measure average performance of 42%, 14  When teachers created their initial concept maps, all false-positive potential items were regarded as false-positives items, resulting in a larger number of false-positive items, thus lowering the system's precision value. In composing the initial concept maps, they created one sophisticated concept map rather than directly composed a very detailed and huge concept map. Thus, it was expected that a lower number of true-positive items was noticed in their initial concept maps. However, in another use of the support function where they refine their concept map, false-positive potential items were potentially useful to refine the concept map and could be regarded as truepositive items; hence, improved precision.

Usefulness evaluation
A questionnaire based on the Technology Acceptance Model (TAM) questionnaire was given to the expert teachers to get their usefulness impression regarding the support tool. The questionnaire consisted of 5 questions on a 7-Likert scale, which has a Cronbach alpha value of 0.84, indicating good reliability. The questionnaire was giv-en in comparison with the manual concept mapping approach. Their responses to the questionnaire are shown in Fig. 6. According to the questionnaire result, as shown in Fig. 6, none of them gave neutral or negative feedback regarding the usefulness. They agreed that the authoring support tool would enhance the effectiveness and increase their productivity in creating concept maps from text learning materials; thus, they thought the support tool was very useful. However, two of them somewhat agreed that the authoring support feature would make them easier in composing concept maps because the way they composed their concept maps with the authoring support tool was more complex than a manual composition. Their responses regarding the support tool's usefulness were in line with the tool's usability level, as presented in [22]. Hence, improvement in usability could improve the usability of the authoring support tool. Nevertheless, it can be said the developed Kit-Build concept map authoring tool was useful in assisting teachers in composing concept maps from English texts.

Limitation and Future Work
The result of the proposed semi-automatic concept map generation showed a promising result to be used in the actual EFL learning environment with Kit-Build concept map. Thus, practical use and trial of the authoring support tool with EFL teachers for use in EFL reading comprehension learning will be the future work of this study.
The concept map generation approach does not elaborate on teachers' logic and thinking while composing their concept maps. Following the teachers' logic in determining which concepts and relationship triples are important and genuinely relevant to the topic is important could improve the suggestions' accuracy. Discovering factors that influence how teachers compose their concept maps is also one important aspect of suggesting more accurate extraction and further improving the suggestions. Specifying the context of learning and adapting the automatic generation approach to con- form to the learning context could potentially yield better relevance keywords and propositions; hence, more accurate and better components to suggest. Technically, the extraction approach applied does not elaborate coreference and semantic resolution in which they may improve how the system resolves pronouns and identify similar words in a text. Elaborating coreference resolution could improve the extraction approach. Incorporating semantic approach and additional processing to support ill-structured English sentences are several other potential future works of this study towards a better concept map generation process. Even though the extraction approach in this study was not specifically designed and limited for EFL reading comprehension use, it is also possible to use the tool to extract concept map components from another English text. However, it might yield inaccurate and unexpected results.

Conclusion
Assessing an education application could depict the application performance when it is used and also discover the drawbacks of the application. While this study does not evaluate how the concept map authoring tool could impact the learning, the prototype of the tool itself was used and evaluated by experts to depict its performance. The experts, who used Kit-Build in their teaching, also reviewed the tool regarding the usefulness of the support tool in assisting concept map composition of EFL reading comprehension texts.
The tool's functionality has been extended with an authoring support function in addition to the standard authoring features of the existing Kit-Build concept map tool. The support function semi-automatically generates concept maps with the CMM approach in assisting teachers in composing their concept maps. The assistance was given semi-automatically in the form of keyword and proposition triple recommendations. In the experiment, the recommendations were extracted from EFL reading comprehension learning material using a combination of CMM, NLP, and text mining methods. Three experts were involved in using the support function and evaluated its performance. They perceived that the developed concept map authoring tool is useful; thus, ascertaining the tool could help teachers compose concept maps from English learning material. However, some limitations in using the tool exist, and further improvement regarding the tool's usability was needed.
Even though the resulting concept maps were subject to the teacher's subjectivity, the proposed semi-automatic concept map generation approach provided by the authoring support tool could achieve an average-to-good accuracy level. Concerning the learning strategy of EFL reading comprehension of this research, accuracy level of 55.9%, 40.3%, and 44.8% could be attained from the respective RAKE keywords, open domain-based triples, and verb-based triples of the initial concept maps. Better accuracy could be achieved if potential suggestions were incorporated to refine the concept maps, yielding an improved accuracy of 95.1%, 69.3%, and 76.7% for the respective RAKE keywords, open domain-based triples, and verb-based triples. According to the accuracy measurement results, it can be concluded that the tool support feature was performed exceptionally well, especially in supporting teachers to refine and improve their concept maps through a semi-automatic concept map generation approach.