Automatic Assessment of University Teachers’ Critical Thinking Levels

Abstract

The present work describes the structure of a pilot study which was addressed to test a tool developed to automatically assess Critical Thinking - CT Levels through language analysis techniques. Starting from Wikipedia database and lexical analysis procedures based on n-grams, a new approach aimed at the automatic assessment of the open-ended questions, where CT can be detected, is proposed. Automatic assessment is focused on four CT macro-indicators : basic language skills, relevance, importance and novelty. The pilot study was carried out through different workshops adapted from Crithinkedu � EU Erasmus + Project model aimed at training university teachers in the field of CT. The workshops were designed to support the development of CT teaching practices at higher education level and enhance University Teachers’ CT as well. The two-hour workshops were conducted in two higher educational institutions, the first in the U.S.A (CCRWT Berkeley College NYC, 26 university teachers) and the second in Italy (Inclusive memory project - University Roma Tre, 22 university teachers). After the two workshops, data were collected through an online questionnaire developed and adapted in the framework of the Erasmus + Crithinkedu project. The questionnaire includes both open-ended and multiple choice questions. The results present CT level shown by university teachers and which kind of pedagogical practices they intend to promote after such an experience within their courses. In addition, a comparison between the values inferred by the algorithm and those calculated by domain human expert is offered. Finally, following up activity is shown taking into consideration other sets of macro-indicators: argumentation and critical evaluation.