Tensor Text-Mining Methods for Malware Identification and Detection, Malware Dynamics Characterization, and Hosts Ranking
Malware is one of the most persistent and costly cyber threats endangering reputation, confidentiality, integrity, and availability for organizations and national security. Consequently, many of the incident detection and prevention systems, and incident responders have begun to utilize machine learning as a helper in the fight against malware and other cyber threats. However, cyber defenders rely on interpretability and generalizability, yet the popular machine learning methods are black-box and often use traditional supervised solutions that do not generalize to novel malware. Therefore, there is a need to improve the existing solutions. At the same time, the majority of the prior research ignored essential evaluation criteria when reporting the results of their methods, which disables the safe reproducibility of the methods in a production environment. Tensor decomposition, on the other hand, enables interpretable unsupervised analysis of the large-scale data for the discovery of hidden patterns. Our findings, performed on real-world and large-scale experiments, show that tensor factorization-based methods yield performance results that surpasses or competes with existing supervised solutions with the added benefit of interpretability and generalizability. With the ability to analyze complex and large-scale data using tensors, we report results that reflect real-world production environments. We propose to develop new game- changing tools for malware identification and characterization that can trace malware evolution, rank the infected or malicious hosts, and streamline the work of incident response teams, malware analysts, and incident detection and prevention systems.