7

The Research Advantage

This chapter will focus on leveraging the principle of innovation to gain an advantage in a conflict. Investing in additional research, such as exploits or new log sources, can give either side a significant leg up in these conflicts. We will see throughout this chapter how complex technology stacks have left a myriad of vulnerabilities and forensic artifacts hidden in their implementations. This research can be shallow reconnaissance, such as gaining a basic understanding of the tools and techniques the opponent uses, to ensure you can detect them in your environment. Or it can be deep research, such as looking at specific applications your target uses and developing exploits for their tools. This chapter will focus on methods for gaining a clear advantage, dominant strategy, or, at the very least, finding the Nash equilibrium, or optimal strategy. This chapter may stray from the established focus of this book a little, looking at topics such as memory corruption, game hacking, and competitions such as the DEF CON CTF. This is to show how these lessons can be broadly applied to many competitions or security scenarios. This chapter will also use a few different meanings of the word exploitation, such as offensive security's use of memory exploitation, meaning memory corruption attacks, and the defensive use of evidence exploitation, meaning analyzing evidence.

In this chapter, we will cover the following topics:

• Dominant strategies in CTFs
• Offensive exploitation
• Memory corruption
• Offensive targeting
• Software supply chain attacks
• Watering hole attacks
• Phish in the middle
• F3EAD
• Clandestine exploitation
• Threat modeling
• Application research
• Data logging and analysis
• Attribution

Gaming the game

I lied at the beginning of the book when I said that these security competitions don't have a Nash equilibrium or optimal gameplay state at their highest levels. While it's true that these are vastly complex competitions, sometimes there are flaws in the competition or rule set that allow the game to be shorted or manipulated in some unintended way. In the spirit of hacking, it's important to be aware that often teams or individuals will abuse mechanics in a game scenario within the rules. I think this is worth mentioning because if a tactic can be performed within the rules, and you've checked with the competition admins, then it's worth abusing the technique even if it's considered cheap. While this sounds tacky, it is the very essence of exploiting an advantage to win. The important part is operating within the rules, not cheating, but finding some way to abuse some aspect of the game.

Sometimes, gaming the game creates a Nash equilibrium or optimal method of play. We can see this played out with older versions of the US Cyber Challenge CTF, or USCC CTF for short. The USCC CTF had a preliminary round where participants could register anonymously and then take three attempts at a static list of questions. The winners would be the participants with the most correct answers and then the fastest times. The top 30 or so finalists would be invited to a weeklong, expenses-paid training camp with SANS-like courses. Because of the free entry and vacation-like reward, this became a competitive competition when I played in it. Almost every score in the top 100 was perfect, but the top 20 of those had submission times within zero to two seconds. It was clear there was a better way to play this competition if you wanted to be in that top-performing group. The solution I arrived at was to register an anonymous account to take the test the first time and use this attempt to record all of the HTML form field values and the correct answers. Next, I used GreaseMonkey^[1], a browser plugin that lets you automate interactions on your page with JavaScript, to fill this page out with the correct answers, and submit it as soon as I would load the challenge.

This technique worked for getting me into the optimal play group, although later I learned that just submitting the pregenerated answer page to the endpoint would be even faster. In this way, there was an entire group of people playing this game at a different, optimal level compared to the normal participants. These forms of Nash equilibrium do exist in cybersecurity, often around automating techniques for a new level of speed and execution. As we will see, one of the critical aspects in setting up such automation is research and preparation for that automation.

Offensive perspective

From the offensive vantage, performing reconnaissance and research on the target is invaluable for both exploitation and post exploitation. Learning what software the target uses internally and then setting up a lab with this software can give some insight into where to hide or what can be exploited for internal movement. Exploring and exploiting this software often requires lots of testing and debugging, which, as an attacker, you will want to do safely outside of your target network. We will see how this can be valuable in both exploitation for gaining new access and hiding your activities for post exploitation. Exploitation to gain new access can open new doors and opportunities for your offensive teams. Performing additional target research and building a more believable cover will be important for protecting your operations in the long run. We will also see how exploitation can occur at various depths in different contexts, such as web exploitation versus memory corruption. The required expertise and exploitation targets can all demand different skillsets, so this reconnaissance and research can be just as helpful when putting a team together. Target research is important when planning how you will pivot through the target environment to reach your objective or goals. Understanding and planning how you will breach and navigate the target organization are critical for moving effectively toward your goal.

The world of memory corruption

In the realm of offensive security research, memory corruption attacks are an entire subworld unto themselves. This text won't act as an authority in that area, as it is a very deep area of research with many unique aspects and technologies. One book that serves as a good introduction to the basic techniques of memory corruption is Hacking: The Art of Exploitation^[2]. Following that, there are three exploit development courses on https://OpenSecurityTraining.info that I recommend if you want to continue to develop your techniques^[3]. Another amazing course comes from RET2^[4]; their online wargame-like demo is amazing, although a little pricey^[5]. A similar, yet free, version of this course is the RPISEC Modern Binary Exploitation (MBE) course^[6].

However, it's important to note that like many niche areas of infosec, there have been several reaction correspondences around these memory corruption techniques over the years. There exists a complex series of strategies, techniques, and defenses within the field of memory corruption, such as technologies like DEP, or data execution prevention, and stack cookies which are designed to make these memory corruption techniques more difficult. I also recommend the Corelan free and paid training^[7] if you are looking to get into more advanced techniques here. If you're looking to get into heap exploitation, there is a really great guide by the legendary CTF team, Shellphish, called how2heap^[8]. Exploit development can also be a lucrative field on its own, with exploit sales ranging from $1,000-$250,000, upward to $1,000,000 or more^[9], depending on the vulnerability and target.

There are likewise many competitions focused on offensive research and memory corruption, such as many CTFs and the annual Pwn2Own competition, where people compete to exploit various technology stacks for large prize pools. Pwn2Own 2021 saw more than $1.5 million in prizes for its combined categories. In past years, people have hacked into and won Tesla Model S cars at Pwn2Own^[10]. This year, 2021, participants took home more than $1 million from that prize pool with 0-day vulnerabilities such as privilege escalation from user to NT/SYSTEM or root in both Windows and Linux desktop environments^[11]. There are also attack and defense style competitions that include memory corruption as a critical aspect of the competition, such as the DEF CON CTF. The CTF changes organizers every few years, the tradition being that the competition is passed down through the winners, each adding their own unique twists and flavor to the game^[12]. The twists each year usually try to change the game to break some Nash equilibrium or dominant strategy that may exist. For example, for a long time, a dominant strategy was to monitor network traffic to see which services were being exploited, and then steal and replay the exploits against other teams. Other such twists have added different networking protocols, machine architectures, and even introduced new king of the hill challenges. In general, the CTF starts out with a Jeopardy-style qualifier round to narrow it down to around 10 teams. These 10 teams then compete in the finals, which are an attack and defense style CTF. The major differentiator with the DEF CON CTF is that the attack and defense services must be exploited and binary patched for points or remediation of issues. You get points for exploiting other services and getting flags, and you lose points for having your services exploited or going down. I have never played in the DEF CON CTF finals, so I only know about this through talking to participants, spectating in person, and reading post-mortem blogs on the game^[13]. The Plaid Parliament of Pwning, or PPP, from Carnegie Mellon University, is one of the top CTF teams in the world, having won the DEF CON CTF more than 5 times^[14]. One amazing thing from reading the PPP writeups is that you can see how much an emphasis they put on planning and preparation from an offensive perspective, a call back to our principle of planning^[15]. While PPP puts a huge focus on technical skill development and training, they also spend time writing tools to automate tasks and support the team^[15].

One of the wilder stories I've heard about the DEF CON CTF involved a team called sk3wl0fr00t, or Skewl of Root. Supposedly, one of the years, they determined the other teams were further network hops away than the score bot service, such that they could reliably tell the difference between other teams and the score bot service based on the TTL of their IP traffic. They were then able to drop all traffic from other teams but still allow the traffic from the score bot checks, based on the packet's TTL. It's an example of a dominant strategy, or another way of gaming the game, like we were looking at before. We can accomplish something like that with the following iptables rule, say if the score bot service was coming from one network hop away, and the other teams were at least two network hops away:

$ sudo iptables -A INPUT -m ttl --ttl-lt 63 -j DROP
$ sudo iptables -A OUTPUT -m ttl --ttl-lt 63 -j DROP

If the other hosts are Linux hosts with a default TTL of 64, then the preceding command would drop any host that has performed two or more network hops. Additionally, if we wanted to block any Windows machines that have a default TTL of 128, we could use a command like the following:

$ sudo iptables -A INPUT -m ttl --ttl-gt 65 -j DROP 
$ sudo iptables -A OUTPUT -m ttl --ttl-gt 65 -j DROP 

A strategy such as this gives a team a clear advantage, like a shield that repels all other team's attacks. Obviously, using such dominant strategies are things teams should consider how to accomplish and game designers should audit for and remove. These strategies are often harder to come by in real life or a balanced competition because more options exist to counter them and open the field of play up.

Targeting research and prep

Another form of research often leveraged by the offensive team is reconnaissance on the target network and organization. Red teams will often attempt to map out privileges within the network, understanding which users in the organization can access which systems. Hunting key information on individuals that are core to the hierarchy of the company is important for knowing which users to impersonate, keylog, or pivot through. Knowing which users would have access to different systems, either in their role or as a network administrator, is often an important task for an internal red team. This is where the red team will leverage tools such as Bloodhound or enumerate internal wikis to understand how to access different systems. Even knowing which members of the organization have authority, or that others would listen to, can be important for further social engineering.

Credential dump attacks are a popular new spin on this idea of user targeting or account takeover. With all of the breaches that have been happening over the past few decades, many of these credentials have leaked or been sold online. While some services such as Have I Been Pwned^[16] exist to help users understand if their credentials have been compromised, other services exist to resell these compromised credentials to researchers and security practitioners. Even more services offer hosted compromised credential checking, presumably through some hashing protocol. Legitimate security teams will use these compromised credential sets to make sure their user base doesn't have accounts that are easy to hijack. Meanwhile, offensive teams will use these compromised password lists to target various organizations, find users in those organizations, password spray for access, and even build generic password lists for cracking. Speaking of generic password lists for cracking, red teams often collect generic passwords from lists like RockYou or these exposed credential dumps: https://github.com/danielmiessler/SecLists/tree/master/Passwords/Leaked-Databases. The collection of the credential sets is important for spraying or enumerating access. Likewise, toolsets to spray the credentials in a usable way are equally important. I like using Hydra^[17] because it's so versatile, but there is also go-netscan^[18] for the same purposes.

Info gathering in or on the network can't be understated. The goal of the operation is often to act on a specific target and get out, as opposed to maintaining access indefinitely. That means once inside, a plan should be constructed with how to get to the objective as soon as possible. Learning various ingress or egress patterns to secure enclaves or more difficult networks is often a critical step in internal reconnaissance. Stuxnet is an incredible example of preparing an operation to migrate across an air-gapped network using USB devices^[19]. Learning how the target network operates can help you prepare to get in through various means or exploits. This can also help when phishing, such that the attacker can use exploits along with spear phishing, allowing them to silently exploit the target's software client rather than have to social engineer or trick the target. While memory corruption exploits can be a huge advantage in competitions such as CCDC or Pros V Joes, they often aren't required to get access, nor are they the focus of the competition, as they are in Pwn2Own and the DEF CON CTF. CCDC and Pros V Joes put more of an emphasis on network penetration and incident response as opposed to exploitation. That said, similar to the way the teams prepare for the DEF CON CTF, the CCDC Red Team will spend the months and weeks leading up to the event by preparing infrastructure and tools to use in our operations.

Target exploitation

One example of how research and non-memory corruption-based exploitation can give an edge in a competition setting is with the 2020 CPTC, or Collegiate Penetration Testing Competition finals.

In CPTC, students are presented with a semi-unique IT environment for both a regional event, and then the same environment with minor tweaks for the finals event. In between those events, vulnerabilities are patched and some things are fixed, but core applications and infrastructure are generally unchanged.

This last year, members of the RIT team found several unauthenticated 0-days in the RocketChat application that allowed them to read messages, and later glean passwords^[20]. The exploit itself is amazing, but the research in the downtime is even more impressive. RocketChat was the internal chat solution for the CPTC organization NGPew, an open-source Slack-like application^[21]. The RIT team remembered us hosting RocketChat in the corporate domain and both sloppy password policies, constantly sharing passwords in chat. In the time between the two competitions, they investigated the unauthenticated functions of the Rocket Chat application and found that some APIs could be called with either authentication (/api/v1/method.call) or no authentication (/api/v1/method.callAnon), and it was up to the function to perform additional authentication checks. One function that did not perform any additional authentication could be used to read the messages from any channel, including private conversations (livechat:loadHistory). This function could be abused to read chat in the #general channel, which would not only glean passwords, but reveal the user IDs of the users posting in #general. Next, the attacks can programmatically combine these user IDs to read messages between these various users. After this bug was discovered at the CPTC 2020 Finals event, we can see the patch diff where authentication was then added to this function^[22]. This is a great example of harnessing the principle of innovation, seeing a bunch of complexities in an application and digging into them to find an assumption or vulnerability to exploit. Anticipating the technologies in play and preparing exploits for these vulnerabilities will clearly give these teams an advantage over their competition. Something like this would be just as useful in CCDC, Pros V Joes, or any offensive situation; we saw in the last chapter how devastating getting access to an opponent's chat could be to their operations. We also see the principle of time applied to computer research, using the long downtime period, between the regional and final events to develop new exploits, techniques, and tooling to use in the competition.

Creative pivoting

In preparing any kind of advanced offensive campaign, you will want to build a general profile and knowledge base of the target. What are their primary business operations, who are the people in charge, who are the admins, and what technology stacks do they leverage? This information is critical to building any advanced social engineering attack or even pivoting throughout the network. Often, when this information is leveraged in crafting an attack to blend in it is known as the pretext. Insider or advanced information, such as an authority or a certain technology, is useful for a pretext or general cover for your operations.

When exploring different ingress options into a network, consider where the applications get their data from and how they process this data. Creative red teams may consider the libraries or stacks the target uses to process data and how they can affect that code. Two avenues to consider are how do humans access the network, and how does new code get introduced to the network. With humans accessing the network, we should remember the principle of humanity and consider who we need to exploit to get access. With code accessing the network we should consider how data is entered into the system or how the existing code is updated. Finally, when considering ways to egress out of the target network, look for normal system traffic. A creative red team will use a public CDN, file sharing service, or common network endpoint that users frequently visit, remembering to blend into the target network traffic. Spending the time to understand whether an encrypted tunnel will be a red flag or a crucial protection for your communications is worthwhile local reconnaissance. Awgh wrote a small network analysis utility called nfp, or Network Finger Printer^[23], that red teams can use to take stats on a network before choosing which protocols to egress over. You can also fall back on techniques such as stenography if there is deep packet inspection, so these won't be obstacles so much as they are low-hanging alarms we can avoid with proper recon. We will explore more egress options in the next chapter, Chapter 8, Clearing the Field.

One idea for propagating to secure environments or systems is backdooring the target's code deployment pipeline or one of their dependencies. For example, you could do this to pivot from a mobile application developer network to a target's phone through the phone application update. You could also do this to pivot into a secure network, or pivot into a CI/CD pipeline for whatever reason. Sometimes, advanced actors will learn the libraries or dependencies in use at a target company and hack the upstream developer in what is known as a software supply chain attack. In this way, they can backdoor the development library to get into the target company. Dependency collision attacks are a popular modern spin on this type of attack. This attack works by using the same name for a public package that a company is using for their internal package. The researcher Alex Birsan was able to do this with several dependencies in different languages to access over 35 different organizations. He backdoored repositories like PyPI for Python, npm for Node, and Ruby gems. Not only would the attacker use namespace collisions with the internal package names, but they would also set the version of the public package to be much higher, which would often trigger the package manager to choose it or update it automatically^[24]. Repo-diff is a tool that can help detect when a repository has been hijacked with a namespace takeover or has an existing collision (https://github.com/sonatype-nexus-community/repo-diff), something both offensive and defensive teams can leverage. At a higher security level, many teams scan their dependencies for vulnerabilities in QA or a test phase when there are updates. They may also have code that ensures the program is using the right package and namespace, such that there is a proper mitigating control for this attack.

Very similar to software supply chain attacks, watering hole attacks can be extremely effective at breaching a target organization. In a watering hole attack, a popular 3^rd party site or vendor will be breached and used as a staging ground to breach all or specific targets that visit the backdoored pages. One of the differences is that a watering hole attack may focus more on backdooring a website, whereas a software supply chain attack would often focus on the package delivery mechanism, such as apt, pip, or npm. An example of a devastating watering hole attack would be Operation Aurora in 2010^[25]. Essentially, Chinese threat actors compromised a popular website many Silicon Valley tech people browsed and used a drive-by download^[26] to exploit several targeted users. Despite browser enhancements, these watering hole attacks can still be effective today. Let's look at a recent technique and how this can be leveraged with a watering hole attack to gain access to a target network.

Samy Kamkar is a legend in the infosec scene, coming out with awesome hacks every several years^[27]. One of Samy's latest innovations, NAT Slipstreaming^[28], is a dream come true for many network scanning enthusiasts. The effect of NAT Slipstreaming is that by getting a victim to visit a website, the attacker can then get full network access to the victim's local network. At a high level, this works by abusing the SIP protocol to enable what is essentially call forwarding on the network, to be able to send traffic to any port and any machine on the network. At a low level, this uses packet fragmentation to cause the victim's router to see a proper SIP or H.323 call forwarding packet, which triggers the application-level gateway (ALG) to forward any specified TCP or UDP port. It's a brilliant exploit chain that potentially opens all-new avenues into target networks. While this was recently patched in Google Chrome by blocking the SIP ports 5060 and 5061, it was later bypassed by researchers releasing the H.323 version. Google then responded by blocking seven more ports, showing how the reaction correspondence occurs often in infosec exploits and responses. It's also interesting to see that while the reaction correspondence increases browser security in this specific case, the vulnerability at large still exists and will affect many older systems.

An extremely reliable pivoting technique that relies on social engineering is known as the phish in the middle^[29]. This technique was popularized by Lares and is wildly successful in my experience. It involves accessing the email of a trusted internal user after already breaching the company through credential spraying or other means. Once you have access to the email of the trusted employee, use their contact list and account to phish other employees. This combination can be extremely effective for spreading laterally in an organization, based on the principle of humanity or targeting users rather than vulnerable computer systems. Obviously, you will want to do a lot of supporting recon, such that you have the appropriate payloads per victim, or you know the victims you want to compromise, based on what you are trying to access, so you can exploit your victims intelligently.

That said, the phish in the middle is especially effective for spear phishing or getting access to a specific team or set of users, to then access your target environment.

As we've seen, this recon can be important to how an attacker will package their post-exploitation tooling. Blending into the target processes and applications should be a paramount concern for attackers. If there are applications that are always running, the attacker will want to backdoor or impersonate these applications. Conversely, if there are applications that have strict signing, tracked releases, error reporting, or verbose logging, attackers will probably want to avoid these applications. Attackers should consider how normal system operators perceive various applications. Understanding a system from a user's perspective can help determine how likely they are to suspect a compromise. Consider if your victim struggles with technology or is a power user, and restrict interaction with the victim to the absolute minimum affordable. This level of understanding of victim expertise can help the offense understand how deep they need to go with their post-exploitation techniques to avoid detection.

Finally, keeping notes or a wiki from year to year can be immensely helpful in a competition setting. Often, competitions will reuse infrastructure or challenges in various ways, so keeping detailed notes of the infrastructure, applications, vulnerabilities, techniques, and game mechanics at play can provide a huge advantage. This can be added to any existing knowledge base of techniques or tools; it will be useful for new players if any previously used infrastructure comes around again.

Defensive perspective

From the defensive side, we will want to gather as much information on the threats, the potential attacks, and our systems as possible. This means that we will be digging into, investigating, and reversing any forensic or attacker artifacts we may have recovered. If we don't have attacker or forensic artifacts, we can use threat modeling to spar with ourselves, and in turn, create our defensive capabilities. In our downtime, we may even investigate our host systems or applications to better understand them and any forensic sources they may offer. We will also want to add our own signal generation to any of our systems where possible, and our own analysis of our data for abuse. One method we can use for generating and disseminating our analysis is F3EAD. F3EAD is a model used in military intelligence targeting that stands for Find, Fix, Finish, Exploit, Analyze, and Disseminate. In this section, we want to focus on the intelligence aspects, or EAD part: the Exploit, Analyze, and Disseminate stages^[30]. After we have engaged the opponent, we will be left with various artifacts of their operations. Exploit in this context means reverse-engineering the attacker's tools or methods.

Analyze means turning this exploitation process into usable data in conjunction with our existing knowledge base or available data, such as arriving at the attribution of the attackers based on their tools or techniques. Finally, dissemination is making this data useful to our analysts, tools, or frameworks to continue searching for and investigating malicious activity.

Tool exploitation

One of our core goals when facing an attacker is to obtain any of the attacker's tools and perform extensive reverse engineering on these tools. Almost all tool developers make mistakes or leave forensic artifacts in some way, and deep analysis of these tools can reveal these mistakes and give the defender the advantage in detecting the adversary. One of the first questions you want to ask yourself when doing this analysis is if the tool is clandestine, some custom hacking tool, or if it's an open-source and commonly available tool. Both categories have their benefits; for example, open-source tools are often easier to analyze, whereas clandestine tools can help with aspects of attribution. Regardless of the source, a deeper understanding of the tool can help you detect and counter its usage in your environment. I've personally had tremendous success spending the time analyzing attacker tools for mistakes. Think of it as the principle of humanity applied back to attacker technologies. There are bound to be errors in their code as well. One example of this when applied to open-source attacker tools is searching for attacker-specific code words or lingo. For example, both hackers and angry developers seem to love swear words. Steve Miller of Mandiant describes searching for common swear words and misspellings to find new pieces of malware and data in his hunting^[31]. Hunting for these OPSEC mistakes or human aspects of the attacker can sometimes reveal their presence more than a technical mistake. One of my favorite mistakes to capitalize on is when an attacker hardcodes credentials into their malware or post-exploitation agent. If these credentials can be used to pivot back to the attacker infrastructure, it's like easy mode on turning the tables on the attacker and hacking back.

Threat modeling

Threat modeling is a great technique to help prepare for an attacker. In the down time, you can essentially estimate how your organization will be attacked and by whom to prepare your defenses. While there are many great books on this topic, such as Threat Modeling by Adam Shostack^[32], the core concept behind threat modeling is relatively simple. Threat modeling boils down to hypothesizing how you will get attacked and the risks associated with those attacks. Risk is the likelihood of an event multiplied by the impact of the event, which are estimates we can use to guide which threats we will flush out with threat modeling.

This is an extremely valuable activity that allows you to engage in the intelligence half of the F3EAD cycle, without having to actually be attacked. It often involves multiple stakeholders brainstorming on different threats or roadblocks that could set a project back. When done right, this can open the doors to purple teaming, or a blue team and red team working together to simulate attacks and model detections. I once heard this described as a red team painting a target for a blue team to hit or detect.

Throughout this book, we've been using Sliver as the C2 framework for our hypothetical threat model. We've examined techniques and implementations native to the tooling, looking for mistakes or tells that could help us detect an attacker using these tools. Throughout our research, we've determined a number of interesting reveals, such as how the Sliver shellcode generation leverages sRDI or how the initial DNS C2 beacon reveals the campaign name. Cobalt Strike is another popular tool to threat model around currently because it's used by so many different pentest groups, red teams, and threat actors alike. Cobalt Strike, while a legitimate security testing tool, is often pirated and used by criminal actors, in everything from APT operations to ransomware campaigns. While ideally, you want to threat model around an actor or a collection of techniques, targeting a tool or framework can catch many actors using it in a default configuration. One great idea for hunting Cobalt Strike agents comes from Twitter user inversecos^[33], whereby beacon's lateral movement techniques will, by default, use the following locations (where random.exe is a collection of 7 random alphanumeric characters):

\hostname\ADMIN$\random.exe

Hunting default values in tools, such as default registry keys or named pipes, can be a very effective strategy for detecting specific tools. Further, if you can use regex to search for patterns of names in specific locations you can catch a great deal of commodity malware that will use common, random patterns. Another amazing example of developing detections based on threat modeling is Andrew Oliveau's BeaconHunter^[34]. Andrew knew that the NCCDC red team used Cobalt Strike in some of their operations (threat research would reveal Raphael Mudge as being on the core NCCDC red team and the original author of Armitage/Cobalt Strike). Through research with Windows ETW, Andrew observed that Cobalt Strike agents start a new thread with the Wait:DelayExecution state. BeaconHunter detects this abnormal behavior and starts scoring which processes use these suspicious threads. To take this even further, the tool will also record whenever one of these processes reaches out over the network, including the IP, port, and how many times it has made this call. This is an incredibly effective tool at detecting Cobalt Strike beacons, and this threat modeling research proved useful in the NCCDC 2021 competition, where their team caught over 210 beacons.

Operating system and application research

Investigating host systems or applications can also reveal new and insightful forensic artifacts. Time spent reversing an operating system or a specific application can often reveal new log sources. For example, you can use an application, such as SysInternals' procmon, to get a detailed trace of an application on Windows, which may reveal a secret log file or temporary files that can be used to gain insight into previous executions^[35]. I will regularly use this when reversing a game or application on Windows and discover a hidden log file that makes my task much easier. I think a great example of this at an operating system (OS) level is the forensic communities' recent exploration of the ShimCache, AmCache, CIT database, and later Syscache logs. The ShimCache, or AppCompatCache, is part of the application compatibility shimming on Windows and will record a timeline of the last time things ran, including the file size and most recent modification time. This log started with the creation of AppCompat, with Windows XP, and has been around since as a great hidden forensic log source on Windows^[36]. ShimCache became the AmCache with Windows 8 and underwent some major changes. This now keeps a record of every application that has run, its path, creation, and last modified date, along with the SHA1 of the PE^[37]. Essentially, whenever a program runs, a service called AeLookupService checks whether that program should be shimmed with the application compatibility features. Then, around 12:30 every day, the task scheduler will flush that cache to a file in %WinDir%AppCompatPrograms with the timestamp and file path of the file execution. The discovery of this log source, which was kept for compatibility and performance, was a boon to forensics. The forensics community spent a lot of time reversing this operating system process and documenting this new log source^[38], which is now in many forensic presentations. David Cowen's Forensic Kitchen is where I learned of these log sources and is a great example of what an individual contributor or forensic practitioner researching these various log sources can find and contribute back to the community. It's important to understand how these logs are generated and their shortcomings, such as flushing the cache daily as opposed to recording in real time. Dave would spend a few hours a week exploring some forensic concept or log source and digging into forensic techniques hands-on, on a live stream in his lab. That's also where I learned about the Syscache hive^[39]. The Syscache started as part of AppLocker and is only available with Windows 7. Still, syscache.hve records executables run on the Desktop and may miss executables started elsewhere. These can be tremendously useful forensic sources when an application is no longer on the system or has been securely deleted from a system, such that a malicious program can't be forensically recovered. Understanding what these logs won't record is just as important as knowing which evidence they will capture.

The team that released BLUESPAWN, the University of Virginia (UVA), won the CCDC national competition for three consecutive years. It's no small coincidence that they put so much effort into researching these attacker techniques, automating their detection, and preparing them in the form of open-source software tools. While we often see the offensive community performing this kind of host-based research more, all security practitioners can get more value out of a deeper understanding of the operating systems and how they work under the hood. This also nods to the principle of innovation, these are deeply complex operating systems, so any research, or even simplifying existing research, pays dividends quickly.

Log and analyze your own data

If you control the application or system you are defending, you should create logs based on your user activity. These logs can help profile user activity and understand generally how users interact with your systems. Logging your own data for security functions or abuse can be critical to spotting fraud or abuse in your applications. You can also do this type of logging with a middleware application, such that you can read Apache2 or nginx logs to understand how users interact with your API instead of your actual application. While graphs and statistics are some of the most important tools for understanding how users are using your application, outliers and anomalous data are good for getting insight into application abuse. Looking at impossible values or strange user interactions can help reveal flaws in the application that you may not detect through source code analysis.

While game hacking strays from our established focus a little, we can use it as a solid example in place for any custom application. It is easier to visualize abuse with game hacking, but the abused application can really be anything the business supports. Ubisoft's Rainbow Six Siege was suffering a major hacking and abuse problem in their application during the period of 2016–2021. They had solutions like Battle Eye in place for cheat detection, but it was less effective over time, catching fewer hackers despite the problem still existing. Ubisoft started collecting many of their own metrics from their application and was able to use "data-based detection models for early detection and flagging of cheats." The process they describe sounds like manually hunting through data for these outliers or cheater data^[40]. I personally know that this is easily achievable with outlier logging. Reviewing high scores or high values in your application can show users who may be hacking the application. Even flagging values over a certain threshold or performed in a certain time period can help detect abuse or botting. Investigating these outliers can help reveal new abuse patterns that can be prevented in application controls or hunted further to reveal users with repeat abuse patterns.

One of my favorite game hacking blogs, secret.club, shows how games with historic abuse problems are still missing obvious hacks by simply not checking outlier data points they already collect. In their example, they automate the game RuneScape. Their botting shows that even if an attacker were to bypass the native detection systems in the client, like the heuristics engine, checking a final player dataset is consistent with other normal player data would still reveal abuse, such as showing this account largely AFK while still leveling at a fast rate (https://secret.club/2021/04/03/runescape-heuristics.html). That is to say, you can't just rely on the controls in your application to detect abuse, but sometimes you need to review the data to make sure obvious abuse isn't getting through the system in some way.

Attribution

Attribution can be a touchy subject. In some contexts, it matters a lot less, as with commodity malware or the members of another team in a competition environment. Although in some settings, understanding who is attacking you can be one of the most important first steps in stopping the attacks. I've always been a big fan of internally tracking common bad actors, relative to the context. In a competition setting, this means recoding common attacker IPs, netblocks, domains, and other IOCs for long-term analysis. In a corporate environment, this could be tracking frequent offenders to make sure there isn't an insider threat developing. In reality, the number of purposefully malicious actors in any ecosystem is relatively small. On top of that, actors who repeatedly seek out and abuse vulnerabilities, not just in an opportunistic sense, are even smaller. Keeping track of these users in a long-term manner, often thought of as attribution, can be valuable for defenders. Unless you're an intelligence firm, this activity should be aimed specifically at abusers of your applications, as opposed to keeping large lists of general threats. If you cannot ID an attacker or an offender, feel free to give them a code name or an anonymous name so you can track them until you can identify them.

Contributing to attribution, threats need to be brought to justice, or their attacks need to be exposed for a greater understanding of the offensive techniques at use. The results of an internal investigation should be submitted back to the community at large through IOCs or blogs on the lessons learned. If you can, work with law enforcement or intelligence firms to document the activities so that they can use them in building a profile or case. For anything to happen to bad actors, you need documented evidence of their actions, so recording that evidence and providing it to the appropriate tracking parties is an important part of the life cycle; IP addresses, logs, artifacts, or even screenshots can help assist attribution, but also records of the impact of damages and what the attackers have caused (such as downtime or assets stolen). While this can reveal information pertaining to the victim, often, the resulting information gained about the attacks and threats outweighs any minor information loss.

Summary

Clearly, researching in the downtime can give a strong advantage in a competition or adversarial setting. These advantages can be leveraged in many forms, from exploits used to gain access, through mapping out an organization, to better understanding an application or operating system, or even exploiting an attacker tool to learn more about it. The point is that using this time to research and automate techniques can give a clear advantage over the opposition. A deeper understanding of any of the technology at play can give one party an advantage over the other, by exploiting a feature of the technology the other party is using but don't yet know about.

In the final chapter, we will look at ending the conflict and remediating a compromise.

References

1. GreeseMonkey – A browser automator: https://en.wikipedia.org/wiki/Greasemonkey
2. Jon Erickson, Hacking: The Art of Exploitation: https://www.amazon.com/Hacking-Art-Exploitation-Jon-Erickson/dp/1593271441
3. Open Security Training Exploits1 Course: https://opensecuritytraining.info/Exploits1.html
4. RET2 Cyber Wargames: https://wargames.ret2.systems/
5. RET2 Wargames Review: https://blog.ret2.io/2018/09/11/scalable-security-education/
6. Modern Binary Exploitation (MBE): https://github.com/RPISEC/MBE
7. Corelan free exploit tutorial: https://www.corelan.be/index.php/2009/07/19/exploit-writing-tutorial-part-1-stack-based-overflows/
8. How2heap – Educational Heap Exploitation: https://github.com/shellphish/how2heap
9. Zerodium Vulnerability Purchase Program: https://www.zerodium.com/program.html
10. Winning a Tesla Model S at Pwn2Own 2019: https://www.securityweek.com/pwn2own-2019-researchers-win-tesla-after-hacking-its-browser
11. Pwn2Own 2021 Results: https://www.zerodayinitiative.com/blog/2021/4/2/pwn2own-2021-schedule-and-live-results
12. DEF CON 25, 20 years of DEF CON CTF Organizers: https://www.youtube.com/watch?v=MbIDrs-mB20
13. DEFCON 2015 CTF FINALS – Blog from DEF CON CTF 2015: https://research.kudelskisecurity.com/2015/08/25/defcon-2015-ctf-finals/
14. Welcome to the New Order: A DEF CON 2018 Retrospective: https://dttw.tech/posts/Hka91N-IQ
15. Kernel Panic: A DEF CON 2020 Retrospective: https://dttw.tech/posts/Skww4fzGP
16. Have I Been Pwned, password exposure database: https://haveibeenpwned.com/FAQs
17. Attacking SSH Over the Wire - Go Red Team! – Using Hydra to password spray: https://isc.sans.edu/forums/diary/Attacking+SSH+Over+the+Wire+Go+Red+Team/23000/
18. go-netscan – a multiprotocol credential spraying tool: https://github.com/emperorcow/go-netscan
19. Kim Zetter, Countdown to Zero Day: Stuxnet and the Launch of the World's First Digital Weapon: https://www.amazon.com/Countdown-Zero-Day-Stuxnet-Digital/dp/0770436196/
20. A RocketChat 0-Day Vulnerability Discovered as part of CPTC 2020: https://securifyinc.com/disclosures/rocketchat-unauthenticated-access-to-messages
21. RocketChat – Open-source chat solution: https://github.com/RocketChat/Rocket.Chat
22. Patch diff of RocketChat adding authentication to loadHistory: https://github.com/RocketChat/Rocket.Chat/commit/ac9d7612a8fd6eae8074bd06e5449da843065be6#diff-61e120f3236b5f0bc942992a3cf0abfd107838aa5bff8cd0a1d9fc5320a43269
23. Network Finger Printer – Go tool: https://github.com/awgh/nfp
24. Dependency Hijacking Software Supply Chain Attack Hits More Than 35 Organizations: Alex Birsan's software supply chain attack: https://blog.sonatype.com/dependency-hijacking-software-supply-chain-attack-hits-more-than-35-organizations
25. Operation Aurora – Watering hole attack on Google and Apple: https://en.wikipedia.org/wiki/Operation_Aurora
26. What is a Drive by Download: https://www.kaspersky.com/resource-center/definitions/drive-by-download
27. Samy Kamkar: https://en.wikipedia.org/wiki/Samy_Kamkar
28. NAT Slipstreaming v2.0: https://samy.pl/slipstream/
29. Phish-in-the-Middle: https://twitter.com/Lares_/status/1258075069714235392
30. Intelligence Concepts – F3EAD: https://sroberts.io/blog/2015-03-24-intelligence-concepts-f3ead/
31. Threat Hunting: https://twitter.com/stvemillertime/status/1100399116876533760
32. Adam Shostack, Threat Modeling: Designing for Security: https://www.amazon.com/Threat-Modeling-Designing-Adam-Shostack/dp/1118809998
33. Inversecos' tweet about Cobalt Strike: https://twitter.com/inversecos/status/1377415476892987395
34. BeaconHunter – Cobalt Strike detection tool: https://github.com/3lp4tr0n/beaconhunter
35. The Ultimate Guide to Procmon: https://adamtheautomator.com/procmon/
36. AmCache and ShimCache in forensic analysis: https://www.andreafortuna.org/2017/10/16/amcache-and-shimcache-in-forensic-analysis/
37. Digital Forensics – ShimCache Artifacts: https://countuponsecurity.com/2016/05/18/digital-forensics-shimcache-artifacts/
38. Blanche Lagny, 2019, Analysis of the AmCache v2: https://www.ssi.gouv.fr/uploads/2019/01/anssi-coriin_2019-analysis_amcache.pdf
39. David Cowen's Daily Blog #579: The meaning of Syscache.hve: https://www.hecfblog.com/2018/12/daily-blog-579-meaning-of-syscachehve.html
40. Ubisoft's Advanced Anti-cheat in Rainbow Six Siege: https://www.ubisoft.com/en-us/game/rainbow-six/siege/news-updates/4CpkSOfyxgYhc5a4SbBTx/dev-blog-update-on-anticheat-in-rainbow-six-siege