ESET IS JUST A LITTLE AHEAD OF THE REST
You increasingly hear about 'machine learning', or ML for short, as an important component of cybersecurity. And that is before we have even grasped what ML's true potential for cybersecurity and threat intelligence might one day be. But do try to distinguish fact from fiction, and marketing from actual action. First, a brief look under the bonnet of ESET cybersecurity and the role of ESET Machine Learning.
ESET's experts have been exploring the possibilities of ML for more than 20 years, since the first neural networks arrived in 1997. Since then, various internal projects have focused on automated security analysis, dividing this new virtual world into 'the good, the bad and the ugly' (or even grey areas such as potentially unwanted applications, or PUAs).
One of the first effective uses of ML was an automated expert system, designed specifically for large volumes of analysis. In 2006, the system helped process the growing number of malware discoveries quickly and reduced the enormous workload on engineers. Years later, its capabilities have been fully optimised, making it an indispensable part of the technology used for the initial sorting and classification of hundreds of thousands of items received every day by the global ESET LiveGrid system, through security feeds and through collaboration with other security vendors.
Another ESET Machine Learning project, running behind the scenes at ESET since 2012, places analysed items on the 'cybersecurity map' and identifies those needing more attention. Interestingly, it was this system that made an important contribution to recent WannaCryptor developments, identifying the rapidly spreading ransomware file at an early stage. Although a system for detecting EternalBlue exploits was already in place, the ML system provided additional detections that improved protection for ESET customers.
But ESET Machine Learning remains a distinctive technology, and not everything always goes to plan. Older projects focused on automatically discovering the DNA of previous malware developments to prepare for the future. These ML-based developments have since been overtaken by other, faster techniques.
Much has been learnt and considerable experience gained. All of this has led to what exists today: a mature application using machine learning technologies, both in the cloud and on endpoint systems.
ENTER AUGUR, THE ESET MACHINE LEARNING BEAST
ESET likes history. Even the name ESET comes from an Egyptian goddess, so the ML system was given a fitting name. In ancient Rome, an augur was a religious figure who interpreted natural phenomena to decide whether something had divine approval. The comparison with cybersecurity is easy to make, although ESET's Augur bases its decisions on science, mathematics and previous experience.
From a more technical perspective, three important factors contribute to this success:
- The arrival of big data and cheaper hardware has made ESET Machine Learning more accessible, whether for medical applications, mathematics or cybersecurity.
- The growing number of ESET Machine Learning algorithms and the science behind them mean broader applications and greater availability for anyone wishing to use them.
- After three decades of battling the cyber underworld and its products, ESET has a modern equivalent of the 'Library of Alexandria', filled with information about malware. This enormous database provides a well-organised view of the capabilities and DNA of everything previously examined: the ideal training ground for ESET Augur.
The rapid growth of algorithms and applications also brought the challenge of selecting only the best for actual use, since not everything works optimally in cybersecurity.
After extensive testing, the most effective approach proved to be a combination of two methods:
- Neural networks, specific 'deep learning' and plenty of 'short-term' memory
- Combined reporting from six carefully selected classification algorithms
To illustrate, take any suspicious executable file. Augur first emulates its behaviour and performs some basic DNA analyses. It then derives certain numerical characteristics, examines the processes it wants to start and reviews its DNA profile to determine the best category: clean, potentially suspicious or malicious. It is important to emphasise that unpacking compressed data, analysing behaviour and emulation are essential to ML. Without them, it is like trying to classify noise as a genre of music.
The group of classification algorithms has two possible settings:
The more aggressive version labels a file malicious when most of the six algorithms reach that conclusion. This setting is widely used by IT administrators using ESET Enterprise Inspector, as it can flag these files as suspicious and leave further handling to an experienced engineer.
The milder, more conservative version labels a file clean when at least one of the six algorithms reaches that conclusion. This is used on general-purpose systems where less technical expertise is available.
Pictures say more than words these days, so if anything is still unclear, this overview may help:
Coincidentally or otherwise, Facebook turns out to use ESET Machine Learning in a similar way to the architecture on which Augur was developed, focusing on the best combination of neural networks and classification algorithms.
What does this mean in practice? How do ESET Machine Learning's theories and approach hold up against the recent malware attacks that used the EternalBlue exploit to deploy both WannaCryptor ransomware and CoinMiner malware? Excluding ESET network detection and alerts from other ML systems, Augur immediately classified samples of both as malicious.
What could Augur do without existing knowledge of recent malware or ransomware? Even a month-old version, without recent updates, was able to classify both the malware and the ransomware as malicious using ML and training data alone. Pure technology, then.
Thirty years of progress and innovation in IT security have taught the people at ESET that some things do not happen automatically or easily, especially in cyberspace, where change is rapid and the entire playing field can shift in minutes. Machine learning, however attractively marketed, will not change that any time soon. That is why ESET does not believe ESET Machine Learning will replace skilled, experienced researchers: the very people who laid the foundations of the system and continue to drive innovation. ESET is proud that many of them work there, helping protect computer users against future threats.
Do you suspect malware in your business environment? Take a look at our Threat Hunt Services. Cyber Threat Hunting: hunt down dormant and active malware or hackers in your IT environment!