Comparison of Industrial Control System Anomaly Detection Methods

Industrial Control System (ICS) are used to produce goods that must be free of errors. Examples are medicines, medical equipment or vehicle parts. It is essential in such production environments to detect an attack which may aim to compromise goods. While Anomaly Detection (AD) is common to protect Information Technology (IT) infrastructure, it is not yet widely used to protect Operational Technology (OT) elements such as ICS and ultimately production. In this work we analyze the usefulness of different AD algorithms in the context of ICS. We aim to determine if simple statistical methods such as K-Means clustering (K-Means), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Stochastic Gradient Decent (SGD) or Support Vector Machine (SVM) are sufficient or if more advanced Machine Learning (ML) algorithms such as an Autoencoder are necessary to achieve a useful performance. Specifically, we consider real-world constraints such as limited available attack examples in training data and variations in background conditions. We use an evaluation framework called Anomaly Detection Evaluation Framework (ADEF) to model an autoclave manufacturing use case and possible attacks. Using ADEF we benchmark different AD algorithms. Our results show that simple methods perform very well, that large amount of attack examples are unnecessary and that fluctuations in environmental conditions pose a significant challenge.