The Internet can fail in many creative and hard-to-pinpoint ways. Even though these aren’t your fault, they’re still your problem. To detect these problems, you need external tests that can watch your site from around the world.
A synthetic monitoring solution can:
Alert you when your site is unreachable or unacceptably slow
Detect localized outages limited to a region or a carrier
Identify performance issues specific to a particular segment of your visitors, such as those using a certain web browser or a specific operating system
Baseline your performance and availability around the world
Predict whether you’ll need to use content delivery networks when entering new markets
Localize errors and slowdowns to a particular component of your infrastructure
Generate additional traffic to test the capacity of your system or evaluate a new code release before it goes into production
Test things you don’t control, like mashup content or third-party web services
Compare your performance to that of your competitors
Estimate required capacity going forward by simulating load
A web startup we know had invested heavily in bottom-up device monitoring tools. It had also deployed several web analytics tools, and employees were confident they could finally sleep well at night, knowing that a series of well-designed alerts would notify them immediately if something went wrong.
The company had also been through a dramatic reduction in its operational staff. Management, emboldened by its recent investments in monitoring, decided that the humans weren’t important now that the tools were in place.
Early one morning, one of the few remaining IT employees sent a panicked email message to the entire company to let them know that their site wasn’t working. The IT team immediately brought up its dashboard, which showed that all servers were operating properly. The dashboard suggested that the infrastructure was, if anything, unusually healthy: web, application, and database servers were faster than normal.
Unfortunately, the company was under siege. A well-orchestrated denial-of-service attack had overloaded some of the core routers linking the company’s data center to Europe, where nearly half of its customers were located.
The attack had a significant impact on revenues, costing the company a substantial amount in SLA refunds. That quarter, the company missed its revenue targets, and this ultimately resulted in delayed financing of their startup, at terms that were much less favorable for the founders and managers. Perhaps worse, management lost faith in the team’s ability to properly monitor the infrastructure. Even though the problem was resolved in only three hours, the fact that it was discovered almost by accident was unforgivable.
Testing your site from outside your own firewall means looking at the many things that can break between a visitor and your website. As you saw in Chapter 8, a web application relies on many components: DNS, routers, CDNs, load balancers, servers, third-party components, client-side scripts, and browser add-ons (Figure 9-3). A failure of any of these components will break or negatively impact the user’s experience.
You need to verify the health of all these systems, all the time, from all locations that matter to your business.
It would be expensive for you to set up and maintain dozens of servers in various countries and carriers around the world for the sole purpose of generating requests to your site. While some large organizations can afford their own testing networks, most companies rely on hosted synthetic testing services. There are several reasons for using a third-party service:
- Cost
It’s cheaper than building and running your own servers.
- No setup time
These companies offer predefined reports and tools that make it easier to set up tests and report performance.
- Viewed as impartial
As independent organizations, the results of their tests are considered impartial and trusted—something you’ll care about when you’re using a performance report to resolve a dispute with a customer.
- Competitive analysis
The testing services can show you how your site stacks up against other businesses or even competitors.
- Visibility into backbone health
Because they have many points of presence, hosted services can test connections between their own servers and report on the health of carrier-to-carrier communications. Portals such as Keynote Systems’ Internet Health Report (www.internetpulse.net) tell you which Tier 1 ISPs are currently experiencing issues. For example, in Figure 9-4, there is increased latency between Verizon and both AT&T and Sprint.
Synthetic testing services are relatively simple to use. Figure 9-5 shows how a basic service works.
You configure the test through a web user interface.
The service rolls out the test to nodes around the world.
The nodes run the test at regular intervals, recording performance.
The results are collected in reports you can view or receive by email.
If an error occurs, the nodes detect it, and may even capture a copy of the error.
Alerts are sent to a mobile device or email client.
Depending on the sophistication of the synthetic testing service, the system may perform additional diagnostics or capture the page error for review, which makes pinpointing and correcting the issue easier.
