An online adaptive model for streaming anomaly detection based on human-machine cooperation

Anomaly detectors are used to distinguish differences between normal and abnormal data, which are usually implemented by evaluating and ranking the anomaly scores of each instance. A static unsupervised streaming anomaly detector is difficult to dynamically adjust anomaly score calculation.

To solve the problem, a research team led by Prof. Zhiwen Yu published their new research on 15 April 2023 in Frontiers of Computer Science co-published by Higher Education Press and Springer Nature.

The team proposed a human-machine interactive streaming anomaly detection method, named ISPForest, which can be adaptively updated online under the guidance of human feedback. In particular, the feedback will be used to adjust the anomaly score calculation and structure of the detector, ideally attaining more accurate anomaly scores in the future.

The experimental results demonstrated that the utility of incorporating feedback can improve the performance of anomaly detectors with a few human efforts.

In the research, they analyze the anomaly detection principle of the space partitioning forest model. To improve the original anomaly detector, they add the human feedback mechanism for the detection result during the streaming anomaly detection process. Firstly, in the light of the relationship between forest structure and anomaly score calculation, they construct regional likelihood function and instance likelihood function, respectively, to depict the consistency of the detection results and the human feedback. Then, the parameters and structures of the original anomaly detector are adjusted timely according to the gradient decrease process following the principle of maximum likelihood estimation. Finally, an uncertainty function of the detection results is designed to control the frequency of human-machine interaction.

Experimental results on the effects of feedback reveal that combining anomaly detectors with human feedback is meaningful for adapting to a dynamic environment, and the performance of the detector is improved promptly with a small increase in labor costs. Future work can consider the extension of the method and explore the time-series anomaly detection under the feedback mechanism.

###

Research Article, Published: 15 April 2023

Qingyang LI, Zhiwen YU, Huang XU, Bin GUO. Human-machine interactive streaming anomaly detection by online self-adaptive forest. Front. Comput. Sci., 2023, 17(2): 172317,  https://doi.org/10.1007/s11704-022-1270-y

About Frontiers of Computer Science (FCS)

FCS was launched in 2007. It is published bimonthly both online and in print by HEP and Springer. Prof. Zhi-Hua Zhou from Nanjing University serves as the Editor-in-Chief. It aims to provide a forum for the publication of peer-reviewed papers to promote rapid communication and exchange between computer scientists. FCS covers all major branches of computer science, including: architecture, software, artificial intelligence, theoretical computer science, networks and communication, information systems, multimedia and graphics, information security, interdisciplinary, etc. The readers may be interested in the special columns “Perspective” and “Excellent Young Scholars Forum”.

FCS is indexed by SCI(E), EI, DBLP, Scopus, etc. The latest IF is 2.669. FCS solicits the following article types: Review, Research Article, Letter.