People

As the previous chapter explained, many companies hold the view that performance issues should just be solved by developers and that performance issues are actually simply caused by poor-quality development. Managers and developers alike feel like they should be able to achieve good performance just through more time or more powerful hardware.

In reality, of course, that is true up to a point. If you are developing a website of average complexity with moderate usage and moderate data levels, you should be able to develop code that performs to an acceptable level. As soon as these factors start to ramp up , however, performance will suffer and will require special expertise to solve. This does not reflect on the competency of the developer; it means that specialized skill is required.

The analogy I would make to this would be to look at the security of a website. For a standard brochureware or low-risk site, a competent developer should be able to deliver a site with sufficient security in place. However, when moving up to a banking site, you would no longer expect the developer to implement security. Security specialists would be involved and would be looking beyond the code to the system as a whole. Security is so important to the system and so complex that only a specialist can fully understand what’s required at that level. Managers accept this because security is regarded as a first-class citizen in the development world.

Performance is exactly the same: performance issues often require such a breadth of knowledge (APM tooling, load generation tools, network setup, system interaction, concurrency effects, threading, database optimization, garbage collection, etc.) that specialists are required to solve them. To address performance, either appropriately skilled individuals must be recruited or existing people skilled up. This is the role of the performance engineer.

Performance engineers are not better than developers (indeed they are often also developers); they just have different skills.

Process

Performance is often not considered in a typical development process at all, or is done as a validation step at the end. This is not treating performance as a first-class citizen.

In this sense, performance is again like security, as well as other nonfunctional requirements (NFRs). Let’s look at how NFRs are integrated into the development process.

For security, an upfront risk assessment takes place to identify necessary security standards, and testing is done before major releases. Builds will not be released if the business is not satisfied that security standards have been met.

For user experience (UX) design, the company will typically allocated a design period up front, dedicate time to it within the development process, and allow additional testing and validation time afterward. Builds will not be released if the business is not happy with the UX.

In contrast, performance is often not considered at all. If it is, the developers do it in vague, subjective terms (“must be fast to load”), with no consideration of key issues such as platform size, data quantities and usage levels. It is then tested too late, if at all.

To be an effective performance warrior, you must start considering performance throughout the development lifecycle.  This includes things such as doing performance risk assessments at the start of a project, setting performance targets, building performance testing and performance code reviews into the development process, and failing projects if performance acceptance targets are not met. Many of these are addressed in more detail in later chapters.

Tooling

To effectively handle performance challenges, you need the right tools for the job.

A wide range of tools that can be used, from tools that come built into the systems being used (for instance, Perfmon on Windows), to open source toolsets (for instance, JMeter), free web-based tools (such as WebPagetest), and tools that you can pay a little or a lot for.

Determining the right toolset is a difficult task and will vary greatly depending on:

• The kind of performance challenge you are facing (poor performance under load, poor performance not under load, poor database performance, networking congestion, etc.)
• The platform you are working on
• The type of system you develop (website, desktop, web service, mobile app, etc.)
• The budget you have to work with
• Skillsets you have in house
• Other tools already used in house or existing licences that can be leveraged

Choosing the right tools for your situation is very important. Poor tool choices can lead to wasted time and effort when trying to get to the bottom of a problem by misdiagnosing the root cause of an issue.

It is also essential that sufficient hardware and training is provided to get the full value out of the selected tools. Performance tooling is often complex, and users need to be given time and support to get the full value from it.

Make Performance Part of the Conversation

All too often, performance flies under the radar because it is never discussed. As a performance warrior, your first step is to change that, and a few simple steps can move the discussion forward:

• Start discussing performance at planning sessions, standups, retrospectives, and other get-togethers.
• Start asking the business users what they expect from performance.
• Start asking the development team how they plan on addressing potential performance bottlenecks.
• Start asking the testers how they plan on validating performance.

Often the answers to these questions will be unsatisfactory, but at least the conversation is started.

Set Performance Targets

It is essential that everyone within the team know what levels of performance the system is aiming for and what metrics they should be considering. This subject is addressed in more detail in the next chapter.

Treat Performance Issues with the Same Importance and Severity as Functional Issues

Performance issues should fail builds. Whether informal or formal performance targets have been set, there must be the organizational policies to declare a build not fit for release on the grounds of performance.

This then will require testing for performance, not just for functionality.

Assign Someone with Responsibility for Performance Within the Project

When performance is not being considered, a good way to move things forward is to assign someone within a team who is responsible for performance on that project/product. This doesn’t necessarily mean that this person will be doing all performance target setting, testing, optimization, etc. She will just be responsible for making sure that it is done and that performance is satisfactory.

Give People What They Need To Get Expertise

Performance engineering is hard: it requires a breadth of understanding of a wide range of aspects of application development that can contribute to performance issues (clients, browsers, network, third-party systems, protocols, hardware, OS, code, databases, etc.). There are also many specialist tools that can be used to identify the cause of performance issues. All these require a specialist’s skills.

Performance testing presents a new set of challenges (what to test, how to test, how the load model should be constructed, where to test from, how to analyse the results, etc). These skills don’t lie beyond most people within development teams, but they do need time to learn and practice the skills needed and the budget to buy the tools.

Create a Culture of Performance

This sounds grandiose but doesn’t need to be. It simply means evolving your company to see the performance of its systems as a key differentiator. Good performance should be something that everyone within the company is proud of, and you should always be striving toward better performance. Often this culture will start within one team and then be driven out to the wider business.

Some simple rules to follow when thinking about how to introduce a performance culture include:

• Be realistic: focus on evolution, not revolution. Change is hard for most people.
• Take small steps: set some achievable targets and then celebrate hitting them, after which you can set harder targets.
• Put things into a relevant context: present stats that convey performance in terms that will matter to people. Page load time will be of little interest to the business, but the relationship between page load time and sales will be.
• Get buy-in from above: performance can begin as a grassroots movement within an organization, and often does; but in order to truly get the results the site needs, it eventually needs buy-in from a senior level.
• Start sending out regular reports about performance improvements and the impact they are having on the business.