Name: Matt Arnao
Organization: Lockheed Martin
Years hunting: 5
Favorite datasets: Network sensor and security device logs, windows events, application logs
Favorite hunting techniques: Pivoting, "over the horizon" data gathering, kill chain analysis
Who are you?
My name is Matt Arnao and I’m a cyber intelligence analyst and software engineer with the Lockheed Martin Computer Incident Response Team (LM-CIRT).
Why do you hunt and what is your experience hunting?
I’ve been with the LM-CIRT for over 5 years, most of which has been spent developing custom software and tools in support of our mission to provide computer network defense for Lockheed Martin’s global networks. I am part of a team of developers who directly support our analysts. As software engineers in LM-CIRT, we are expected to be in tune with the mission, the threats we face, and the capabilities we need to develop. Most of our developers, including myself, have spent time cross-training as analysts.
I do this because it’s what I’m passionate about. The constant arms race between defenders and attackers is enough to keep anyone busy for a lifetime. I genuinely believe this is one of the most challenging and rewarding fields to be involved in today. I landed in LM-CIRT after a couple years of rotational assignments at Lockheed Martin. I’ve been fortunate enough to work with some extremely bright and talented people who have helped shape the security field as we know it today.
How would you define Threat Hunting?
Threat Hunting is a new name for something we have been doing for a long time as part of our duties as computer network defenders. As the security field has matured over the years, specialized disciplines like Threat Hunting have emerged. I would define threat hunting as the act of identifying and tracking malicious activity in and outside of your network. The goal of the latter is to ensure you keep up with the threats during periods where you are not being targeted. Just because you’re not a target today doesn’t mean you won’t be tomorrow. You need to know what your adversaries are up to in the meantime. I would argue that everyone involved in computer network defense shares this responsibility, but having dedicated resources can help drive progress and ensure your team is well equipped.
What projects and organizations are you involved with?
Last year we open sourced our file analysis framework known as Laika BOSS, which is available on Lockheed Martin’s github page. As one of the original developers, I help shape the direction of the project. This week, my colleague Zach Rasmor is planning to demonstrate integration of Laika BOSS with Suricata IDS in his talk at SuriCon. This capability will allow Suricata users to leverage external file analysis frameworks such as Laika BOSS to analyze the files Suricata extracts from HTTP and other protocols.
What hunting techniques, tools, and datasets do you use most frequently?
Passive network visibility tools such as NSM and IDS are staples in any threat hunter’s toolkit. It’s often the easiest thing to implement, especially when going into a new and unfamiliar environment. Coupled with good intelligence, network visibility can quickly help you discover malicious activity within an environment. Unfortunately, these tools are not silver bullets and their value is waning over time. There will always be a place for these tools, but as encryption becomes more and more common, we need to look elsewhere to fill this ever widening visibility gap. Passive network visibility can also be challenging when networks (especially internet points of presence) are distributed and choke points are not clearly defined.
Endpoint visibility and protection tools are an obvious alternative. It should come as no surprise that this space has exploded with competition over the past few years. Historically, scaling these solutions up has been a challenge. Large enterprises with tens or hundreds of thousands of endpoints are often out of the question. Endpoint visibility is challenging on many levels, from collection, to post processing, to storage and retrieval. If you can overcome these challenges, there is a wealth of information beyond what you will ever get from passive network visibility. I believe cloud computing, ubiquitous encryption and other factors will continue to drive the need for better endpoint visibility and protection.
There are of course many other important tools and data sets available—including application logs, security appliance logs, open source and paid data sets. Most threat hunters will tell you their thirst for data is insatiable. I would like to mention a final less talked about option that has its own set of challenges, but also some distinct benefits. Integrating with your mail transfer agents and proxy servers for content inspection can overcome some of the challenges of passive network visibility. We recently released our milter server for Laika BOSS that can help you achieve inline inspection and blocking of malicious email before it ever reaches an end user. Although the benefits are obvious, this approach requires a great deal of trust with your leadership and business to be successful.
In terms of techniques, I would say kill chain analysis, pivoting across data, and "over the horizon" data gathering are all important. "Over the horizon" is a term we adapted from over the horizon radar which is radar used for detecting targets at long distance. We refer to data sources that let us see beyond our environment as OTH. This may be paid or open data sources that we use to track adversary activity.
What general advice do you have for new Threat Hunters?
I believe almost anyone can succeed in this field given a passion for the work, the right mindset, and foundational skills. Every successful person I have met shares these qualities. You can always tell who is passionate about the work—they are the ones who are always up to date on the latest news, tools, and trends. They are the ones who debate over the smallest details that others might ignore. They are the ones who pursue relevant interests outside of work because they are genuinely excited about them.
Analytical thinking and problem solving are two key components of a good mindset, but they are hardly unique to threat hunting. Be prepared to demonstrated these universally valued qualities to prospective employers with whatever experiences you have. Most relevant skills and knowledge can be learned over time without formal training, but here are a few broad categories that I think are really important:
- Basic programming and scripting: automate and become more efficient in your work
- Systems and network administration: understand how the systems you’re protecting actually work
- Systems and network forensics: piece together a timeline of events based on available data