Sqrrl Blog

Mar 7, 2016 3:11:00 PM

What Is a Threat Hunting Platform: Part 1 - An Introduction

Hunting and its Obstacles

One of the major security  problems facing organizations today is that they are simply not finding hidden threats on their network in time. On average, it will take an organization 205 days before finding a malicious actor burrowed in their systems. 70% of breach notifications companies receive come from third party organizations. To find advanced threats, you need more than traditional automated security solutions; you need to be hunting.

Threat hunting is the process of proactively and iteratively searching through networks to detect and investigate advanced threats that evade existing detection tools. Hunting can radically enhance the process of finding those hidden threats and can cut the time it takes to find them from multiple hundred days to hours. But even if you want to start hunting, there are still 2 major issues that you will likely face.

First, hunting takes skill. Even security operations center (SOC) analysts that have multiple years of experience in incident response and forensics may not know how to begin a proactive detection in pursuit of a potential adversary. Hunting is hypothesis-driven, so you need to know how to ask the right questions. Hunting is also investigative, so you need to know where and how to look for the answers to those questions. Finally, hunting requires an analyst to be able to extract patterns and TTPs from an investigation and fold those findings into automated analytics and intel for your other detection solutions. There are few security analysts that have both the data science and operational cybersecurity skillsets to take on a hunt doing all these things. A platform that automates the hunt can make hunting easier so that non-hunters can begin to hunt.

Second, a hunter needs to use many  different tools and techniques, which can slow down and fragment the hunting process. For example a hunter might use Splunk for data querying and investigation, and then move to custom Python or R scripts to develop analytics for pattern recognition. A platform that brings together these different technical capabilities can make hunters significantly more productive.

In order to simplify, automate, and unify hunting, hunters require a Threat Hunting Platform (THP). In this 2-part blog series we will discuss what a THP is, and how it can radically augment an organization’s detection efforts.

What Is a Threat Hunting Platform?

A threat hunting platform is a unified solution that brings all of the important hunting processes and capabilities together within a single tool. It’s a place to collect and manage big data, detect and analyze anomalies, and visually investigate an attacker’s Tactics, Techniques, and Procedures (TTPs), all in a contextual way. A good THP helps hunters quickly identify malicious patterns of activity and assists them in turning those patterns into automated detections, protecting against future occurrences. It is also a place for hunters, incident responders, and intel team members to collaborate on joint activities.

Hunting Technology

There are four key requirements for a hunting platform:

Threat_Hunting_Platform.png

Data: Collecting as much data as possible is the critical foundation for hunting. Attacks often times stretch back many months, so in order to determine the full scope or impact of an incident a long historical record is required. The types of data that are important for a hunt, such as flow data, are often too large to fit within a traditional SIEM. Lastly, a long historical baseline of data can make analytics more powerful, as it helps dampen false positive detections. As such, THPs require multi-petabyte scalability for diverse datasets.

Analytics: Analytics are a hunter’s best friend, as they help a hunter quickly spot anomalies in large amounts of data. Analytics can range from simple statistics and standard deviations to machine learning and User and Entity Behavioral Analytics (UEBA). A THP should incorporate as many quality analytics as possible. These analytics should also be extensible and allow for analyst input, so that they can be adjusted based on the findings of a hunt.

Exploration: Analysts need to be able to quickly and easily search, explore, and pivot on datasets to hunt for threats. An easy-to-use and powerful query language is an important starting point for a THP to begin investigating datasets, but will not be enough on its own. Dynamic and contextual visualizations are required to help analysts quickly see patterns in the data. Further, analysts should be able to visually navigate the data in addition to writing custom search queries. Lastly, analysts should be able to easily pivot between underlying raw datasets and higher level abstractions of that data.

Collaboration: Hunting is a team sport, so collaboration is key. Having common threat ontologies and speaking the same language across hunters allows them to more efficiently pursue adversaries. Further, hunters should be able to easily share their findings with other team members. For less experienced analysts, collaboration with other senior analysts in a THP can help them more quickly undertake more advanced hunts.

In Part 2 we will discuss the benefits of a THP and examples of what one looks like.

Threat Hunting Platform White Paper Download

Topics: Cyber Hunting, Threat Hunting, Enterprise Security, Hunting Platform