Physical assets

A large range of data, such as account numbers, names, services in use, and highly sensitive personal information could be located on physical assets. Throwing away old invoices, scrap papers, personnel records, or other potentially sensitive information enables attackers to use a technique known as “dumpster diving” to retrieve this sensitive data. Ensuring the proper processes and controls are in place to safeguard against the improper disposal of possibly sensitive information is a key step. It is best to keep boxes of paper waiting on being shredded under supervision or in a secure area.

Dumpster diving

By removing trash and recyclables from the business the threat is introduced of any bystander being able to rifle through dumpsters or recycling containers. Throwing away old invoices, scrap papers, personnel records, or other potentially sensitive information allows the attacker to add to his overall profile. There is a possibility that a large range of data could be located on physical assets, such as account numbers, names, services in use, and highly sensitive personal information. Ensuring the proper processes and controls are in place to safeguard against the improper disposal of possibly sensitive information is a key step. It is best to keep boxes of paper waiting on being shredded under supervision or in a secure area.

Shoulder surfing

Shoulder surfing is another technique attackers can use to steal sensitive information. Simply looking over the shoulders of someone sitting at a computer or using a smartphone enables an attacker to gain insight on software or services being used, the deployed operating systems and controls, or other sensitive information. It is best to protect against this by utilizing a monitor privacy screen, positioning computers away from areas where they can be easily viewed, and training users to be aware of their surroundings for this type of activity.

Company assets

Email addresses are one of the easier pieces of data to find on the internet. We will go over harvesting these in an automated way later on in this chapter. Lists of addresses can be used for larger phishing campaigns. While in this day and age it is close to impossible to prevent them from being on the internet in some shape or form, this can be used to your advantage. By creating “canary in a coal mine” email accounts, or honeypot accounts, suspicious activity can now be monitored and alerted on. They should be email addresses that look real and blend in with the real addresses of your organization. It is as simple as adding them somewhere on the organization’s main website so it can easily be scraped by an email harvesting tool or you can use it to create an entire fake online persona. To ensure the fake email address is not used by a legitimate customer attempting to establish contact, it could be added as white text on a white background, or some other color that would blend in.

A business opens itself to a certain amount of risk when contracting with third-party services to save costs and allow itself to focus on its speciality as opposed to wasting time on other duties. Outsourcing takes place in a variety of situations. Roles such as housekeeping, food services, safety inspectors, and other professional services can be an avenue for a good social engineer to gain physical access to buildings. Educating against tailgating and keeping up-to-date on vendor access is a must.

Technology

There are many automated scanning tools, browser plug-ins, and scripts that will allow the gathering of external-facing server details such as operating system and software versions. More information can be added to a profile by utilizing the information gathered from public-facing devices and technolgy. This can uncover vulnerable applications or devices to anyone who is permitted to access them. For example, builtwith.com (Figure 18-1) can be used as a standalone website or as a part of tools such as recon-ng or Maltego, which we will cover later on.

Metadata, which is basically data about data, can contain a wealth of information about the person or company that it originated from. A common type of metadata is EXIF. Exchangeable image file format or EXIF are the properties of documents, images, and sound files. The list of properties that can be attached to these include the username of who created it, customized comments, created date, geographic location information, and sharing/permissions, just to give some examples.

Figure 18-1. builtwith.com

Documents

Sensitive information included in documents can be obtained through several methods. One of the most popular, Google Dorking, has an entire section of the exploit-database website dedicated to new searches aimed at exfiltrating data. For example, a Google search for inurl:wp-config -intext:wp-config “‘DB_PASSWORD’” will provide a result with many different Wordpress database passwords.

Personal assets

The general population is not aware that personal social media posts can contain a great deal of information that an attacker can use for pretexting. Posts on various media, including geolocation to professional networking, give away what technologies may be being used internally. Without the proper security settings and knowledge, advanced targeting of specific employees becomes much easier for an attacker to paint a full picture of a person’s habits and personality. This also falls back on user education and its effects transferring from the workplace to home. Having an all-encompassing program can assist in this becoming less of a problem.

Online forums, whether public or private, can present their own security concerns. Employees may reach out directly to vendors to ask specific questions on technology or to answer others’ questions. This documentation could possibly list software build numbers, logfiles, error messages, or other sensitive data.

Ever since large companies have begun storing customer data, there have been criminals breaching their security infrastructure (if they had any to begin with) and publicly publishing the information for anyone to parse. In 2015 alone, CVS, Walgreens, Patreon, LassPass, Experien, UCLA, Ashley Madison, Anthem, IRS, and OPM were among the largest data breaches, releasing around 200 million customer records combined. Data mining can be performed against the released records, which would include information helpful to any OSINT campaign. Many times passwords are reused across multiple organizations and these can be leveraged for attacking other sites.

Maltego

“[Maltego provides] the aggregation of information posted all over the internet - whether it’s the current configuration of a router poised on the edge of your network or the current whereabouts of your Vice President on his international visits, Maltego can locate, aggregate and visualize this information.”²

Maltego is an incredibly powerful reconnaissance tool that many professional organizations use. Entire profiles can be built and visualized through this piece of software. It has a smooth graphical user interface that allows you to drag and drop items, and perform many queries with several clicks. Depending on the size of the organization, it may not be worth the subscription prices for Maltego. It would depend on how often you believe this information changes, and how important it is to have it in a visually pleasing format.

You can find the free version of Maltego in both Kali and on their website Paterva.com. We are using the full version of the software in the following exercise to take you through the steps of setting up a single domain using Maltego:

1. To add the domain, just drag and drop the “Domain” object from the lefthand Palette menu under Infrastructure tab and give it the domain of your choosing, as shown in Figure 18-2.

   

   Figure 18-2. Adding a new domain

2. Select the “Run All Transforms” for “DNS for Domain” to populate all subdomains, as shown in Figure 18-3.

   

   Figure 18-3. All subdomains for paterva.com

3. Choose “Select by Type” to choose all of the DNS names, as shown in Figure 18-4.

   

   Figure 18-4. Type selection

And run other transforms such as resolving to IP address, discovering what websites are running on the servers, and GeoIP location.

Figure 18-5. Selecting “All Transforms” can be an information overload, but it can be interesting to see the outcome

A comprehensive scan has the potential of giving too much information and being too noisy. Not only will it take a significant amount of time, but it will also fill the graph with an extreme amount of data that may not be beneficial. You can test this by running All Transforms against a domain.

recon-ng

“...is a full-featured Web Reconnaissance framework written in Python. Complete with independent modules, database interaction, built in convenience functions, interactive help, and command completion, Recon-ng provides a powerful environment in which open source web-based reconnaissance can be conducted quickly and thoroughly.”³

Recon-ng is a tool written by Tim Tomes (@LaNMaSteR53). It is free and open source, written in the same style as the popular hacking tool Metasploit. We will set up and run a couple example exercises that will populate a database and give us the opportunity to create a report showing the findings:

1. Load recon-ng by navigating to the correct directory (on Kali it is /usr/share/recon-ng) and running ./recon-ng. As shown in Figure 18-6, you will add a workspace. Workspaces can be used for separate projects. In this example, we will be using the workspace for one domain.

   

   Figure 18-6. Creating a workspace

2. Use the netcraft module to search for subdomains in Figure 18-7. You can load a module by using the load or use command.

   

   Figure 18-7. Subdomains found by netcraft

3. Netcraft is one of many modules that can be run. Use the command show modules to list everything available, as shown in Figure 18-8.

   

   Figure 18-8. Show modules

4. Many of the Recon-ng modules require you to have an API to use. To set these modules up to work, enter your API as shown in Figure 18-9. Many APIs are free for use and only require you to sign up for an account. The following example is shown with Builtwith. This service will crawl a website and return all of the information it can about what technologies the site has been built with, as well as what type of servers and software that it runs on.

   

   Figure 18-9. Adding an API key and running the builtwith module

5. It is also helpful to find the IP addresses that the domains and subdomains resolve to. You can do this by using the resolve module, as shown in Figure 18-10.

   

   Figure 18-10. Resolving hosts

6. The interesting_files module can be loaded (as seen in Figure 18-11) to scan potentially sensitive file locations to try and find a match. It downloads the files into a directory for later viewing. These files may give more insight into the company’s server infrastructure.

   

   Figure 18-11. Finding potentially sensitive files

7. Now you can proceed to finding personal information, such as email addresses on individuals tied to the corporation. As shown in Figure 18-12, load the whois_poc module. This module finds the domain registrar Whois contacts and adds them to the database.

   

   Figure 18-12. whois-pocs module

8. You can also run a search against public PGP servers, as shown in Figure 18-13, to capture names and email addresses. This search, however, will list every domain found with your domain it, and as you can see there are a few false positives.

   

   Figure 18-13. PGP email address search

Other helpful commands for recon-ng are:

show info

This command will return all information and settings on the particular module that is currently loaded.

show options

This will show the different options that are available to set for the module that is currently loaded.

delete hosts “row id”

If there are hosts that have shown up that you don’t wish to keep in any reports, they can be deleted per row, by comma delimitation, or if you have a range, a dash can be used.

Regon-ng also has several built-in reports that offer a variety of ways to display the information that has been gathered. The reports are also modules and can be csv, html, list, pushpin, and xml formats. Something to note is that the data from the database will need to be cleaned up if it is not required to be present in the report. Reports also have options that can be set to customize them further.

theharvester.py

“The objective of this program is to gather emails, subdomains, hosts, employee names, open ports and banners from different public sources like search engines, PGP key servers and SHODAN computer database.”⁴

The Harvester (Figure 18-14) is a great simple way to scrape the internet for email addresses that match a certain domain name.

Figure 18-14. The Harvester command line

Here we will again be using united.com and running it against all data sources by using the command ./theHarvester.py -d united.com -l 500 -b all in Figure 18-15.

Figure 18-15. The beginning of the harvester results

Websites

There are many websites that we could list for OSINT gathering. Instead we will focus on the top two that will provide the most use in a purple team scenario. These two are the Google Hacking Database (AKA GHDB & Google Dorks—Exploit-DB) and Shodan.

“Originally created by Johnny Long of Hackers for Charity, The Google Hacking Database (GHDB) is an authoritative source for querying the ever-widening reach of the Google search engine. In the GHDB, you will find search terms for files containing usernames, vulnerable servers, and even files containing passwords.”⁵

GHDB is now maintained and hosted by Offensive Security, the same group that brings you Kali (a Linux hacking distro) and a range of ethical hacking certifications.

“Shodan is the world’s first search engine for Internet-connected devices.”

Shodan was created by John Matherly (@achillean) in 2009 to crawl the web. The end goal for your environment should be to have the least amount of devices show up in Shodan as humanly possible. If anything, when any amount of your IP address range is on Shodan, it should be something that you are aware that the entire internet has access to. For instance, a listing of a hardened and patched webserver, controlled mail relay, or DNS server would make perfect sense. However, having printers, database servers, out-of-date operating systems, or embedded devices directly connected to the internet is not only bad practice, but also a perfect foothold to the inside of a network. There are entire talks and research papers on the number of devices that are connected to the internet that Shodan has indexed. Accounts can be created for free and have a limited functionality option, as well as paid options that provide access to more results and reporting options.

Knowing the IP address(es) of your organization is the first step toward finding what is currently listed, but if you’re not sure there are several options. You can use the IP addresses of the resolver that we utilized earlier in this chapter or you can look at arin.net (Figure 18-16). Using the resolver (or shodan) may return results of server IP addresses that are not owned by the company in question. They may be IP addresses of servers that are hosted somewhere like Amazon or another hosting provider. ARIN (American Registry for Internet Numbers) will display the resulting IP address network blocks that are owned by the organization.

Figure 18-16. ARIN.net search for “united airlines”

Clicking one of the organizations listed (you may have to cycle through each one in a large company to see all netblocks) and then “Related Networks” presents a page that will list IP addresses associated with that specific organization. Searching for “united airlines” yields the results shown in Figure 18-17.

Figure 18-17. Specific network blocks

This now gives a specific list of what networks to search for on Shodan. By using the search string “net:209.87.112.0/24” we find 23 results of differing services all in the United States. Figure 18-18 shows all HTTP, HTTPS, and DNS servers located in the US. From here, Shodan will link to the website itself or show more information about what is housed on that particular IP address. While Shodan doesn’t have all 65535 ports of every IP address indexed, it does have the most popular ones, and it is adding more consistently.

Figure 18-18. Shodan netblock results

You can find a comprehensive list of searches and guides using the API in the official Shodan book.

Responder

“Responder is a multi threaded tool that answers to IPv4 LLMNR (Link-local Multicast Name Resolution) and Netbios Name Service (NBT-NS) queries.”⁶

One of the first methods of attack that can be performed is using Responder by SpiderLabs. This tool allows the spoofing of LLMNR and NBT-NS responses to queries that Windows workstations broadcast by default on a network. By default, when a Windows workstation requests a name lookup, it performs the following three queries: Local Hosts File, Configured DNS Servers, and NetBIOS Broadcasts. If the first two queries return no results, the machine broadcasts an NBT-NS request, which any device on the network would be able to respond to. LLMNR queries are made after a DNS query fails to see if the device in question is on the local network. By using Responder you can answer these requests and possibly gain usernames and password hashes.

Here are some Responder command examples:

• -I is for the ethernet interface traffic is being redirected to

• -w starts the wpad proxy

• -r will allow the device to respond to netbios wredir queries and should be used with caution as it can cause problems if used in a production environment

• -f will fingerprint the host to return operating system it is running

• Just typing ./Responder.py will display full output of available commands

• ./Responder.py -I eth0 -wrf yields the results shown in Figure 18-19 on the test network

Figure 18-19. Starting up Responder

In Figure 18-20, we’ve successfully captured a username and password hash for a Microsoft local account. This information can be used in a Pass the Hash attack without having to crack/decrypt the password. If it is a poor password, it can be cracked giving the user’s full username and password.

Figure 18-20. Successful capture of credentials using Responder

Rawr

The Rapid Assessment of Web Resources (Rawr) will allow the scanning of a network, which will report on and pull screen grabs of all listening web pages. Created by Adam Byers (@al14s), it is a useful tool to combine a quick view of pages all in one place, among a variety of other useful information.

To use Rawr, follow these steps:

1. To download Rawr, type git pull link-to-git-repo.

2. To install rawr with any dependencies that you may need, type ./rawr.py -install.

3. To run with common netblocks or by following other possible inputs dictated by the documentation, type ./rawr.py 192.168.11.0/24, as shown in Figures 18-21 and 18-22.

   

   Figure 18-21. Running rawr.py

   

   Figure 18-22. Rawr raw output

4. The end result is an .html report page offering up screen captures of any responding websites, a report on security headers, and much more, as shown in Figures 18-23 and 18-24.

   

   Figure 18-23. Rawr site output

   

   Figure 18-24. Rawr report output