Towards Data-Driven Network Intrusion Detection Systems: Features Dimensionality Reduction and Machine Learning

Abstract

Cyberattacks have increased in tandem with the exponential expansion
of computer networks and network applications throughout the world. In this study,
we evaluate and compare four features selection methods, seven classical machine
learning algorithms, and the deep learning algorithm on one million random instances
of CSE-CIC-IDS2018 big data set for network intrusions. The dataset was
preprocessed and cleaned and all learning algorithms were trained on the original
values of features. The feature selection methods highlighted the importance of
features related to forwarding direction (FWD) and two flow measures (FLOW) in
predicting the binary traffic type; benign or attack. Furthermore, the results revealed
that whether models are trained on all features or the top 30 features selected by any
of the four features selection techniques used in this experiment, there is no significant
difference in model performance. Moreover, we may be able to train ML models on
only four features and have them perform similarly to models trained on all data,
which may result in preferable models in terms of complexity, explainability, and
scale for deployment. Furthermore, by choosing four unanimity features instead of all
traffic features, training time may be reduced from 10% to 50% of the training time
on all features.