Data Mining and Knowledge Discovery Computer Networks and Communications

Detecting anomalies in log files using the Damerau-Levenshtein distance metric

Gábor Horváth, András Mészáros, Kamel Charaf, Péter Szilágyi
Springer Nature · 2026-01-01 · DOI 10.1007/s10618-025-01182-8

Abstract

Abstract In recent decades, significant research effort has been put into developing solutions to support automatized or semi-automatized analysis of log files. A large number of algorithms appeared based on neural networks. In this paper we introduce a new approach to anomaly detection in log files, that does not rely on neural networks. The building blocks of our approach have been well-known in machine learning for a long time. We propose to use a weighted Damerau-Levenshtein distance metric to quantify the similarity between log sequences. We introduce a kNN-based algorithm for semi-supervised log anomaly detection, and an HDBSCAN-based solution for the unsupervised problem. For the latter, we extend the algorithm by incorporating a manual feedback mechanism, enabling human domain experts to modify sequence labels when necessary.

Read at publisher →