Threat Hunting: Methodologies, Tools, and Tips for Success

🔹How to Protect Critical Assets Through Systematic, Proactive Threat Intelligence🔹

🔹What is Threat Hunting? What is it, Who is it for, and Why?

For many of today’s companies, cybersecurity is often an uphill battle. On the other hand, cybersecurity spending continues to grow, while new security technologies are constantly being developed. But, on the other hand, cyber-attacks continue to plague major businesses and organizations (and even recently, part of the U.S. government).

Between increasingly sophisticated hackers and a legal landscape that threatens to punish those companies that fail to adequately protect their customers’ privacy rights, it’s not hard to see why cybersecurity professionals are so concerned about the risks they face. Adding to the problem, many major cyber-attacks are allowed to continue for an extended period before the victims discover them and take action. In fact, it now takes an average of 280 days before a data breach is detected and contained, according to IBM.

If even today’s advanced cybersecurity technologies cant always keep companies safe from major cyberthreats (or even alert them properly to ongoing cyber-attacks), what can these companies do to gain the upper hand? One increasingly promising strategy is to add an extra, proactive layer of security through threat hunting.

Cyber threat hunting is an active information security strategy used by security analysts. It consists of searching iteratively through networks to detect indicators of compromise (IOCs); hacker tactics, techniques, and procedures (TTPs); and threats such as Advanced Persistent Threats (APTs) that are evading your existing security system.

Threat hunting is also seeking evidence that a threat has begun to materialize before you have any indications that the threat has become your reality. By seeking out the highest-priority potential cyberthreats rather than waiting for evidence of them to appear, threat hunting provides a proven, systematic way to defend yourself from the risks that most concern you. Armed with the early warning of cyber-attacks that threat hunting often provides, your cybersecurity team can then take action to mitigate a risk posed by threat actors.

🔹Threat Hunting is…

Threat Hunting for Effective Cybersecurity: How to Protect Critical Assets Through Systematic, Proactive Threat Intelligence

🔹Prerequisites for a Threat Hunt🔹

Information and Tools

The first piece of information you’ll need when planning a threat hunt is likely to be an inventory of your critical information assets. To make sure your threat hunt is effective and efficient, you’ll want to know what relevant data you have, where it is, who can access it, and which safeguards protect it.

The reality is that many companies and organizations carry out threat hunts without a full inventory of their information assets, but it is well worth your time to gather as much of this information up front as possible. The more complete your inventory is before you start, the faster and more complete your threat hunt will be.

Eye

🔹Details to Include in Your Inventory:

🔹Physical and logical topologies.

🔹Network device information (make, model, OS version, and configuration).

🔹Security control information (make, model, OS version, and configuration).

🔹Host information (make, model, hardware configuration, and OS version and configuration — as well as the names, versions, and configurations of any applications on that host).

🔹Pan-host/pan-infrastructure information for hypervisors, content management systems, data interchange systems, etc. (including versions, security controls, and access lists).

🔹Data flow between apps and hosts for business solutions.

🔹Access controls for all of the above.

🔹Access lists for all of the above.

🔹Locations, types, and formats of logs for all of the above.

After you’ve created this inventory (to the extent feasible), your next step is selecting your most critical data assets and ranking them in order of importance. In a large and well-funded organization, this is typically done either in a risk assessment or by a risk management program.

Which assets are most important to protect? The answer varies widely from organization to organization, based on specific needs, goals, and threats. For example, one company may be most concerned with its financial accounts while another may be more focused on protecting its intellectual property.

🔹Using Investigative Tools to Identify Threats

In addition to knowing which data assets you need to protect, developing a threat-hunting roadmap requires you to have a sense of what threats are out there that may impact your organization. You can get a snapshot of the latest and most urgent threats to watch out for by relying on a cyberthreat intelligence feed that can provide real-time updates on threats identified on the deep and dark web. This kind of feed can also be used in conjunction with auto-block rules, enabling you to automatically protect yourself against obvious threats in real-time, without relying on a threat-hunting or IT team.

If you have enough cybersecurity resources to support a threat-hunting team, then an investigative research portal is likely a worthwhile investment for you. With a solution in place, you can take a highly tailored approach to both searching for threats and setting up automatic alerts, based on your industry’s threat landscape and the most critical assets listed in your inventory.

🔹Creating Priority Intelligence Requirements (PIRs)

Once you know what your key information assets are, which of them are most critical, and what threats you need to watch out for, you’re ready for an analyst to create a roadmap of the most urgent threats to investigate. They should do this by generating a list of priority intelligence requirements (PIRs) — a set of very specific questions about potential cyberthreats that should guide your threat-hunting program. Simply put, your list of PIRs should lay out which specific risks you want to investigate and in what order.

With all of this information in hand, your team will be ready to start the six steps that make up a well-organized threat hunt.

🔹Step 1

Define Your Threat Hunt

With your list of PIRs in hand, you’re ready for an analyst to lay out your threat hunt. First, this involves articulating the purpose of the hunt. Why are you about to conduct a threat hunt, and which possible threat will you focus on? Keep in mind that each hunt focuses on one specific threat and answers one main question.

Next, the analyst defines the scope of the threat hunt. This process starts with identifying your assumptions about the hunt and laying out your hypothesis based on your threat intelligence.

At the heart of the hypothesis is a critical question: If the threat that you’re worried about happened to you, what evidence would there be? Based on the answer to this question, the analyst can (and should) generate their hypothesis.

For example, let’s say that threat X uses tools that typicall leave the registry key “gotcha” in location Z. If threat X happened, I would expect to find the key “gotcha” at location Z. I care about threat X on servers A, B, and C. Final hypothesis: If key “gotcha” is at Z on servers A, B, or C, I might be suffering from threat X.

Each of your hypotheses should answer a single yes/no question, so that the threat hunt will either confirm them or determine that there is no evidence. For complex threats, you may have multiple sub-hypotheses that you research answers to.

After your team articulates their hypothesis (and maybe sub-hypotheses) for each threat hunt, they can determine which elements of your environment to search.

The last component of defining a threat hunt is laying out its limitations. For this step, it is important to consider key questions:

🔹Questions to Consider When Defining Limitations:

🔹What timeframe will the threat hunt consider?

🔹What environments should it not consider?

🔹Do you have any relevant legal, regulatory, or contractual constraints?

🔹Do you have any technical limitations that could constrain the threat hunt?

🔹What is the deadline by which you need to have the threat hunt completed?

🔹Step 2

Equip Your Threat Hunt

To make threat hunting viable on a scalable, ongoing basis, your team will need to operate with the efficiency that comes with the right technological tools. Using the most effective digital solutions can accelerate a threat hunt by more than 20 times.

The time to make sure you have those tools in place is before you start collecting data for a threat hunt. You’ll want to consider three types of tools here: threat intelligence sources, telemetry-based technologies, and automation solutions.

When it comes to threat intel, there are a wide variety of tools that gather information in different ways from different sources. Depending on your inventory of information assets and the hypothesis (or hypotheses) driving your threat hunt, you may want to use any or all of the information sources listed below.

🔹Key Tools for Threat Hunts

🔹Solutions (including automated feeds, investigative portals, or both) offering you threat intelligence gathered from the deep and dark web.

🔹Open-source threat intel feeds.

🔹Web spiders.

🔹General-purpose search engines.

🔹Information provided by major cybersecurity vendors, such as antivirus service providers.

🔹Government-provided resources.

🔹Insights gathered from publicly available media, such as cybersecurity blogs.

Telemetry-based tools can either alert you to potential threats or provide insight into anomalies that you’re already aware of. System logs can be a rich source of information on cyberthreats, and SIEM (security information and event management) solutions offer you an automated way to sift through this data to draw conclusions.

To make the most of all the information you have access to — and to do so efficiently — it is critical to tap into the power of automation. A SOAR (security orchestration, automation, and response) solution can be a massive force multiplier here, using automated playbooks gathering information from disparate sources in threat hunting: just gathering the data. If you do not have a SOAR, then APIs and scripting solutions go a long way toward streamlining a threat hunt by automating tasks that would otherwise be time-consuming. Leading artificial intelligence engines can likewise use automation to cut down on the amount of work time needed for a threat hunt by identifying patterns and relationships for the analyst.

🔹Step 3

Finalize Your Threat Hunt Plan

Having defined the threat hunt and which tools to use, you are ready to address the rest of the questions that should be answered before starting the data collection process. These should fill in the remaining gaps in your plan.

🔹Basic Questions for Completing a Threat Hunt Plan:

🔹Who will conduct the threat hunt?

🔹When will they conduct it?

🔹How will they conduct it?

🔹Where will they conduct it?

🔹What resources will they use to conduct it (including the tools you have selected for the hunt)?

After answering these questions, you will want to clearly define your company or organization’s change control process and any legal oversight, and how these factors will affect the threat hunt. You’ll also want to lay out a schedule for the hunt.

Then comes the last step before getting into the heart of the threat hung: the review process. The idea here is to ensure that your plan is workable, unbiased, appropriate in light of your hypothesis and sub-hypotheses, and cost-effective. You should involve somebody besides the analyst who made the plan here, minimizing the chances that biases compromise the plan’s effectiveness and reliability.

🔹The Review Process Should Ensure That:

hgfghv

🔹The hypothesis and plan server your hunting objective.

🔹The most appropriate tools and resources have been selected to achieve your objective.

🔹Your result will answer the key question of the hunt.

🔹Your hunt will not disrupt your organization’s other activities.

🔹You have all the necessary approvals (including internal change, stakeholder, and legal oversight, as well as approvals from any relevant third-parties).

🔹Step 4

🔹Execute the Threat Hunt

With preparations complete, your cyberthreat analyst can start carrying out the threat hunt based on your plan.

🔹How to Execute a Threat Hunt:

A. Collect Data

How you will collect data should already be laid out in your plans, so now it’s just a matter of following through. Data collection is typically the most laborious part of executing a threat hunt, especially if there are hurdles making it difficult to access all of the systems and data that your plan calls for. In this part of the process, it is especially worthwhile to use automation to dramatically reduce the amount of work and time required.

B. Process Data

The second-most work-intensive part of the threat hunt (after data collection) is processing the data you’ve collected. This involves compiling the information so that a threat analyst will be able to examine it. Here is another great opportunity to streamline your threat hunts through automation — especially with scripting, SOAR solutions, or both. Ultimately, the success of the threat hunt depends on the quality and comprehensiveness of the data gathered and processed. The more data points you have, and the more extensive the background information at your disposal, the higher the quality of your analysis will be.

C. Analyze Data

While much of the data collection and processing can be automated, analyzing that information is still a job for a (human) threat analyst today. Expert AI systems can help with pattern associations, particularly in open-source data. SOAR and SIEM systems can be configured to help detect and block IOCs, but require frequent retuning and configuration. A professional’s expertise and capabilities can really make a powerful difference here.

🔹PROFESSIONAL RECOMMENDATION

You want your experts spending their time on hypotheses and analysis — not maintaining and curating dark-web contacts, not negotiating access to logs and configuration data with sysadmins in your environment, and not collecting data. Purchase the dark-web portals and feeds, automate the data collection and collating, and let your analyst analyze. This is how you achieve 20 times greater throughput, maximize your analyst’s productivity, minimize your spend, and make proactive threat hunting commercially viable for your team.

D. Draft a Conclusion

The last part of executing a threat hunt is answering the questions at the heart of the threat hunt and writing a report explaining your findings. There are three basic questions your report should address:

🔹What is the answer to the question defined in your PIRs? (Keep in mind: Although it’s a good idea to provide some explanation in the report, it’s important to provide a clear “yes” or “no” to the basic question(s) from your PIR).

🔹Even if you found no evidence of a cyber-attack, did you find that your organization has any vulnerabilities to cyberthreats? (If so, recommend the priority for remediation, and which stakeholders should be engaged for further discussion).

🔹Did you run across any other findings of note?

🔹Step 5

Evaluate the Threat Hunt

Evaluate your team’s performance and learn actionable lessons. This is the key to continually improving your threat-hunting team, and the time to do it is after you’ve executed a hunt and answered the key question defined in its PIR.

🔹Questions to Consider:

🔹Was the chosen hypothesis appropriate and sufficiently specific for the threat hunt? (And if not, was the hypothesis too specific or too general, and what made it a poor match for this threat hunt?)

🔹Was the scope of the threat hunt ideal? (And if not, was the scope too wide or too narrow, and why?)

🔹Was the threat intelligence you received helpful, and what would have made it even better?

🔹If you used a threat intelligence provider’s portal, was the portal sufficient? What would have made it more helpful?

🔹What other tools did you use? Were they sufficient? What would have made them more helpful?

🔹Did everyone follow your threat-hunting and associated change/notice processes? Were there any areas not addressed in your process that you had to work around? Are there any process improvements you can make for better detail, speed, accuracy, or coordination?

🔹Did staff perform as expected? Were there any issues with following processes? Any missing training? And is there any training that would enhance future performance?

🔹Did leadership have sufficient information to address leadership questions and report status throughout the effort? Did leadership communications in any way inhibit the hunt?

Finally, for each of the above, what went WELL? What did you do right? Be sure to recognize those responsible for the successful parts.

🔹Step 6

Share and Act on Your Findings

After receiving any necessary approvals on your conclusions, it is important to share this information within your company, so that improvements can be made for future threat hunts. It is also a good idea to share relevant findings (when possible, and only with the necessary approvals) with the third-party vendors you worked with on this threat hunt, such as threat intelligence vendors, so that they can better help you with future threat hunts.

Finally, you should act on the conclusion of your threat hunt. If you found evidence to support your hypothesis, then it is important to quickly hand your report over to your incident response team and initiate your incident response process.

If you did not find evidence to support your hypothesis, then it’s worth remembering that this does not necessarily prove that your hypothesis is false — it simply shows that, based on the data you gathered, you could not confidently confirm that hypothesis. If this is the case, you should report your findings internally and then move on to your next threat hunt.

🔹Your Six Steps for Effective Threat Hunting:

Six Step Process for Effective Threat Hunting

Given the cyberthreat landscape facing today’s businesses and organizations, it is not difficult to see how proactive steps such as threat hunting can help them stay safe. By searching for specific evidence of a possible cyber-attack rather than waiting for 20 hours, when done manually into a task that can be completed in about one hour.

🔹Threat Hunting is…

Threat Hunting Definition

…”the threat has become your reality.”

Threat hunting activities include:

🔹Hunting for insider threats or outside attackers — Cyber threat hunters can detect threats posed by insiders, like an employee, or outsiders, like a criminal organization.

🔹Proactively hunting for known adversaries — A known attacker is one who is listed in threat intelligence services, or whose code pattern is on the denylist of known malicious programs.

🔹Searching for hidden threats to prevent the attack from happening — Threat hunters analyze the computing environment by using constant monitoring. Using behavioral analysis, they can detect anomalies which could indicate a threat.

🔹Executing the incident response plan — When they detect a threat, hunters gather as much information as possible before executing the incident response plan to neutralize it. This is used to update the response plan and prevent similar attacks.

🔹A Three-Step Threat Hunting Framework

There are three phases in a proactive threat hunting process: an initial trigger phase, followed by an investigation, and ending with a resolution.

🔹Threat Hunting Methodologies

Intelligence-Based Hunting

Intelligence-based hunting is a reactive threat hunting technique designed to react according to input sources of intelligence. You can input intelligence such as indicators of compromise, IP addresses, hash values, and domain names.

This process can be integrated with a SIEM and threat intelligence tools, which use the intelligence to hunt for threats. Another great source of intelligence is the host or network artifacts provided by computer emergency response teams (CERTs), which allow you to export automated alerts.

You can input the information into a SIEM using Trusted Automated eXchange of Intelligence Information (TAXII) and Structured Threat Information eXpression (STIX).

🔹Hypotheses-Based Hunting

This threat hunting technique involves testing three types of hypotheses:

🔹Analytics-driven — Makes use of machine learning (ML) and user and entity behavior analytics (UEBA) to develop aggregated risk scores and formulate hypotheses.

🔹Intelligence-driven — Includes malware analysis, vulnerability scans, and intelligence reports and feeds.

🔹Situational-awareness driven — Enterprise risk assessments and crown jewel analysis (the identification of the digital assets that are critical to the company).

The large amounts of data collected means threat hunters need to automate a big part of the process using machine learning techniques and threat intelligence.

🔹Investigation Using Indicators of Attack (IoA)

The most proactive threat hunting technique is investigation using indicators of attack. The first step is to identify advanced persistent threat (APT) groups and malware attacks by leveraging global detection playbooks. This technique commonly aligns with threat frameworks such as MITRE ATT&CK.

Here are the actions that are most often involved in the process:

  1. Use IoAs and TTPs to identify threat actors.
  2. The hunter assesses the domain, environment, and attack behaviors to create a hypothesis that aligns with MITRE.
  3. After identifying a behavior, the threat hunter attempts to locate patterns by monitoring activities. The goal is locating, identifying, and then isolating the threat.

🔹Hybrid Hunting

The hybrid threat hunting technique combines all of the above methods, allowing security analysts to customize the hunt. It usually incorporates industry-based hunting with situational awareness, combined with specified hunting requirements. For example, the hunt can be customized using data about geopolitical issues. You can also use a hypothesis as the trigger, and leverage IoAs and IoCs.

🔹What Makes a Great Threat Hunter?

A threat hunter is a security analyst who uses manual or machine-assisted techniques to detect, isolate, and neutralize APTs that are not detected by automated security tools. To improve their skills, security staff may undergo threat hunting training, obtain a threat hunting certification, such as Certified Cyber Threat Hunting Professional (CCTHP) or Certified Ethical Hacker (CEH).

Threat hunters typically report to a director of information security, who ultimately reports to the chief information security officer (CISO). When working in a security operations center (SOC), threat hunters report to the SOC manager.

Some important skills for a good threat hunter are:

🔹Data analytics and reporting — Pattern, recognition, technical writing, data science, problem solving, and research

🔹Operating systems and networks knowledge — Need to know the ins and outs of organizational systems and networks

🔹Information security experience — Malware reverse engineering, adversary tracking, and endpoint security; needs to have a clear understanding of past and current TTPs used by the attackers.

🔹Programming language fluency — At least one scripting language and one compiled language is common, though modern tools are increasingly eliminating the need for using scripting language.

🔹More Tips to Improve Your Threat Hunting

Data breaches and cyber-attacks cost organizations millions of dollars every year. These tips can help your organization better detect these threats:

🔹Identify your organization’s “normal” — Threat hunters need to sift through anomalous activities and recognize the actual threats, so it is crucial to understand what the normal operational activities of the organization are. To accomplish this, the threat hunting team collaborates with key personnel both within and outside of IT to gather valuable information and insights. This enables them to decide what is a threat and what is unusual, but normal, activity. This process can be automated using a technology like UEBA, which can show normal operation conditions for an environment, and the users and machines within it.

🔹Observe, Orient, Decide, Act (OODA) — Threat hunters use this strategy, borrowed from the military, in cyber warfare. OODA stands for:

🔹Observe — Routinely collect logs from IT and security systems.

🔹Orient — Cross-check the data against existing information and analyze and look for IoAs, such as signs of command & control (C2) activity.

🔹Decide — Identify the correct course of action according to the incident status.

🔹Act — In case of an attack, execute the incident response plan. Take measures to prevent similar attacks in the future.

🔹Have appropriate and sufficient resources — A threat hunting team should have enough of the following:

🔹Personnel — A threat hunting team that includes, at minimum, one experienced cyber threat hunter

🔹Systems — A basic threat hunting infrastructure that collects and organizes security incidents and events

🔹Tools — Software designed to identify anomalies and track down attackers

🔹Threat Hunting Platforms

Threat hunters use solutions and tools to find suspicious activities. These are the three main categories:

🔹Security monitoring tools — Tools such as firewalls, antivirus, and endpoint security solutions collect security data and monitor the network.

🔹SIEM solutions — Security Information and Event Management (SIEM) solutions help manage the raw security data and provide near real-time analysis of security threats.

🔹Analytics Tools — Statistical and intelligence analysis software provides a visual report through interactive charts and graphs, making it easier to correlate entities and detect patterns.

🔹Threat Hunter Solutions

Finding a good threat hunting solution will help analysts outsmart attackers by simplifying threat detection. It should allow investigators to use point-and-click search of specific criteria including by user, asset, event, risk type, alerts, and attacker TTPs. Investigators should also be able to search through timelines for abnormal behaviors, so that they can respond faster, stopping attacks when they appear.

Key features of a solid threat hunting platform will help your organization build more effective threat hunting capabilities, and to achieve this, should, at the minimum include:

🔹Easy to use interface — Point-and-click interfaces make it simple to query data

🔹Context-aware data — Enables complex searches

🔹Behavioral threat hunting — Allows analysts to search for IoAs, which are much higher value indicators than IoCs

🔹Automatic incident timelines — Automation makes gathering evidence faster and easier than maintaining logs

🔹Provides visual aid — Represents relationships, revealing hidden correlations between data

Happy hunting, my friends, and as always, please disclose responsibly. 🔹

Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.