Chapter 1. What Is Automated Traffic?
There is a range of different definitions of what can be classed as automated traffic.
For example, Frost & Sullivan describe bot traffic as “computer programs that are used to perform specific actions in an automated fashion,” Akamai has defined it as “automated software programs that interact with websites,” and Wikipedia defines a bot as “a software application that runs automated tasks (scripts) over the Internet,” whereas Hubspot says “A bot is a type of automated technology that’s programmed to execute certain tasks without human intervention.”
For the purposes of this book I will use the following description for automated traffic, which I feel captures the essential details of what is meant by the term and removes some of the vagaries included in the other descriptions:
Automated traffic is any set of legitimate requests made to a website that is made by an automated process rather than triggered by a direct human action.
Key Characteristics of Automated Traffic
For the purposes of this book, I will have a limited definition of automated traffic; this is not to say that other types of automated traffic are not a concern, just that they are addressed elsewhere.
Web-based Systems
The automated traffic discussed in this book is targeted at web-based systems and excludes other types of traffic, such as automated emails.
Layer 7
Automated traffic operates at layer 7 of the OSI Model—in other words, it operates at the application level, making HTTP/HTTPS requests to websites and receiving responses in the same format. Anything that interacts with servers via any other means is classed as outside the scope of this book.
Legitimate Requests
Automated traffic is defined as traffic that makes legitimate requests to websites (i.e., requests formulated in the same way as those made by human users). This means that the automated traffic that is identified as negative is focused on exploiting weaknesses in business logic of systems, not exploiting security weaknesses.
Exclusions
The following types of traffic, which could be categorized as automated traffic, have been excluded from any discussion within this book. The reason for this exclusion is that they are subjects in their own right and are well catered for in other literature, with a range of well-established products and solutions in existence to mitigate the issues created.
Their exclusion from this work does not imply that they are not worthy subjects of concern for website owners. They are, in fact, very real threats that should be handled as part of any website management strategy.
DDoS (Distributed Denial of Service)
DDoS is a low-level volumetric attack, designed to overwhelm the server by the quantity of requests being made. There are a wide range of different attacks that can be made to achieve this objective, all of which aim to exploit weaknesses in networking protocols. To mitigate this, there are well-established, dedicated DDoS management tools and services that can be put in place to minimize risk from DDoS attacks.
A variation on this called application DDoS aims to make large numbers of requests for certain, known pressure points within systems, with the intention of bringing the system to its knees. This will be discussed in more depth in Chapter 4.
Security Vulnerability Exploits
These types of exploits involve attempts to make illegitimate requests to a system with the aim of exploiting weaknesses within the security of a system allowing the operator to gain control over the server or data within the application. Common examples include SQL injection and cross-site scripting.
Hackers employ constant automated scripts that execute across the internet looking for sites/servers where these vulnerabilities have not been mitigated. Well-managed servers and good application development can protect systems from these exploits, but it is also a good practice to use a web application firewall (WAF) to identify and block illegitimate requests to further minimize risk from these or future exploits.
These automated scans and attacks are a real threat and should be taken seriously by anyone who has responsibility for the security of a website.