Natural Language Processing Approach to Evaluate Real-Time Flexibility of Ideas to Support Collaborative Creative Process

Abstract

Natural Language Processing (NLP) has emerged as a valuable approach to assist in solving complex challenges in educational settings. This study explores NLP techniques, particularly sentence embedding models, to evaluate the flexibility dimension of divergent thinking during an open-ended collaborative creative (cocreative) process. The methodology involved a case study in which 25 secondary education students participated. The students worked in five collaborative groups to solve a real-life challenge through a cocreative process. During this study, we focus on evaluating flexibility, defined as the shift from one semantic category to another, grounded in the semantic similarity of ideas. Initially, we measured semantic similarity with eight-sentence embedding models and experts. We also conducted a correlation analysis of the experts and sentence embedding models to choose one highly correlated model. Subsequently, we evaluated the flexibility of ideas in creative techniques using experts and the high-correlated sentence embedding model. The results disclose that among the eight applied sentence embedding models to evaluate the semantic similarity of open-ended ideas, the Universal Sentence Encoder Transformer (USE-T) is highly correlated with experts. Moreover, USE-T strongly aligns with experts’ evaluation to evaluate flexibility. These results will be valuable for designing and providing immediate feedback during cocreation, enabling AI-driven support to foster innovative solutions to real-world challenges.