A Proposed Framework for Human-like Language Processing of ChatGPT in Academic Writing

Abstract

The study proposed a framework for analyzing and measuring the ChatGPT capabilities as a generic language model. This study aims to examine the capabilities of the emerging technological Artificial Intelligence tool (ChatGPT) in generating effective academic writing. The proposed framework consists of six principles (Relatedness, Adequacy, Limitation, Authenticity, Cognition, and Redundancy) related to Artificial Language Processing which would explore the accuracy and proficiency of this algorithm-generated writing. The researchers used ChatGPT to obtain some academic texts and paragraphs in different genres as responses to some textbased academic queries. A critical analysis of the content of these academic texts was conducted based on the proposed framework principles. The results show that despite ChatGPT’s exceptional capabilities, its serious defects are evident, as many issues in academic writing are raised. The major issues include information repetition, nonfactual inferences, illogical reasoning, fake references, hallucination, and lack of pragmatic interpretation. The proposed framework would be a valuable guideline for researchers and practitioners interested in analyzing and evaluating recently emerging machine languages of AI language models.