Skill Scanner

An AI-Based Recommendation System for Employers, Job Seekers and Educational Institutions

Abstract

Skills are the common ground between employers, job seekers and educational institutions which can be analyzed with the help of artificial intelligence (AI), specifically natural language processing (NLP) techniques. In this paper we explore a state-of-the-art pipeline that extracts, vectorizes, clusters, and compares skills to provide recommendations for all three players—thereby bridging the gap between employers, job seekers and educational institutions. As companies hiring data scientists report that it is increasingly difficult to find a so-called "unicorn data scientist" [1], we conduct our experiments and analysis using companies’ job postings for a data scientist position, job seekers’ CVs for that position, and a curriculum from a master's program in data science. However, our investigated methods and our final recommendation system can be applied to other job positions as well. Our best system combines Sentence-BERT [2], UMAP [3], DBSCAN [4], and K-means clustering [5]. To also evaluate feedback from potential users, we conducted a survey, in which the majority of employers’, job seekers’ and educational institutions’ representatives state that with the help of our automatic recommendations, processes related to skills are more effective, faster, fairer, more explainable, more autonomous and more supported.