Chapter 21
Performance Tuning
Squeezing extra performance out of your hardware might sound like a pointless task, given the low price of commodity upgrades today. To a certain degree, that is true; for most of us, it is cheaper to buy a new computer than to spend hours fighting to get a 5 percent speed boost. But what if the speed boost were 20 percent? How about if it were 50 percent?
The amount of benefit you can get by optimizing your system varies depending on what kinds of tasks you are running, but there is something for everyone. Over the next few pages we look at quick ways to optimize the Apache web server, both the KDE and GNOME desktop systems, both MySQL and PostgreSQL database servers, and more.
Before we get into the details, you need to understand that optimization is not an absolute term: If you optimize a system, you have improved its performance, but it is still possible it could further be increased. You are typically not interested in getting 99.999 percent performance out of a system because optimization suffers from the law of diminishing returns: Some basic changes make the biggest differences, but after that, it takes increasing amounts of work to obtain decreasing speed improvements.
Many Linux users love to tinker under the hood to increase the performance of their computers, and Linux gives you some great tools to do that. Whereas stability-loving nurturers generally tell us, “Don’t fix what’s not broken,” experiment-loving hot-rodders often say, “Fix it until it breaks.” In this section, you learn about many of the commands used to tune, or “tweak,” your file system.
Before you undertake any under-the-hood work with Linux, however, keep a few points in mind. First, perform a benchmark on your system before you begin. Linux does not offer a well-developed benchmarking application, and availability of what exists changes rapidly. You can search online for the most up-to-date information for benchmarking applications for Linux. If you are a system administrator, you might choose to create your own benchmarking tests. Second, remember to tweak only one thing at a time so you can tell what works, what does not work, and what breaks things. Some of these tweaks might not work or might lock up your machine, but if you are only implementing them one at a time, you will find it much easier to reverse a change that caused a problem.
If you aren’t measuring, you aren’t tuning. Measure what you want to improve before making changes, as you make incremental changes, and after you think you are done making changes. Don’t assume anything, ever. Only then can you be sure what you have done was/is useful.
Always have a working boot disc handy, such as the live Ubuntu CD or DVD. Remember that you are personally assuming all risks for attempting any of these tweaks. If you don’t understand what you are doing or are not confident in your ability to revert any changes discussed here, do not attempt any of the suggestions in this chapter. The default settings in Ubuntu work very well for most people and really don’t need adjusting—just as most people can use and enjoy their car just as it is. However, some people love taking apart their cars and building hot-rods; they enjoy tweaking and breaking and fixing them. This chapter is for that sort of person. If you don’t think you can fix it, don’t risk breaking it.
Storage Disk
This first set of tuning ideas all have to do with accessing and storing your data. There are often tradeoffs involved when tuning for performance, such as reducing or eliminating redundancy or journaling. For some data that is ephemeral and trivial if lost, that’s fine. For most data, you want to keep all of these intact to help ensure data persistence and integrity.
Also, with solid state drives and the speed increase seen when using them versus using hard drives, there really isn’t the compelling need to tune most of these things anymore.
Linux File Systems
Most of the options discussed in this chapter are for the ext2, ext3, or ext4 file systems. The default in Ubuntu—and most other current Linux distributions today—is ext4. It is also common to see xfs and zfs systems.
Both ext3 and ext4 are journaling file systems, which are much more reliable than other file systems for data storage. A journaling file system writes data changes first to a log in a separate part of the drive. Once journaled, the change is queued and performed in the background. This can help prevent data corruption while also keeping data in a consistent state. If there is a problem, the data can be rebuilt from the journal.
Other file system options are available. Most of them are considered legacy options or are used for compatibility with other operating systems, such as NTFS, used by Microsoft Windows.
The file system receiving the most attention currently is ZFS. ZFS comes from Sun Microsystems and the Solaris operating system (now owned by Oracle) and has been long loved in data centers. However, ZFS is proprietary and was unavailable for other operating systems—but not anymore. Well, it is available, but it is still proprietary licensed, so there is a potential risk involved for anything but personal use. This option is not recommended for casual use, but if you have large-scale storage needs and are willing to do a bit of reading and configuration, you may find it incredibly useful.
The hdparm Command
The hdparm utility can be used by root to set and tune the settings for IDE hard drives. You would do this to tune the drives for optimal performance. After previously requiring a kernel patch and installation of associated support programs, the hdparm program is now included with Ubuntu. You should only experiment with the file systems mounted read-only because some settings can damage some file systems when used improperly.
The general format of the command is this:
matthew@seymour:~$ hdparm command device
The following command runs a hard disk test:
matthew@seymour:~$ hdparm –tT /dev/hda
You must replace /dev/hda with the location of your hard disk. hdparm then runs two tests: cached reads and buffered disk reads. A good IDE hard disk should be getting 400Mbps to 500Mbps for the first test and 20Mbps to 30Mbps for the second. Note your scores and then try this command:
matthew@seymour:~$ hdparm –m16 –d1 –u1 –c1 /dev/hda
This enables various performance-enhancing settings. Now try executing the original command again. If your scores increase from the previous measurement, you should run this command:
matthew@seymour:~$ hdparm –m16 –d1 –u1 –c1 –k1 /dev/hda
The extra parameter tells hdparm to write the settings to disk so that they will be used each time you boot, ensuring optimal disk performance in the future.
The man page entry for hdparm is extensive and contains useful detailed information, but because the kernel configuration selected by Ubuntu already attempts to optimize the drives, it might be that little can be gained through tweaking. Because not all hardware combinations can be anticipated by Ubuntu or by Linux, and performance gains are always useful, you’re encouraged to try.
Tip
You can use the hdparm command to produce a disk transfer speed result as follows:
matthew@seymour:~$ hdparm -tT device
Be aware, however, that although the resulting numbers appear quantitative, they are subject to several technical qualifications beyond the scope of what is discussed and explained in this chapter. Simply put, do not accept values generated by hdparm as absolute numbers but only as a relative measure of performance.
File System Tuning
Never content to leave things alone, Linux provides several tools to adjust and customize file system settings. The belief is that hardware manufacturers and distribution creators tend to select conservative settings that will work well all the time, leaving some of the potential of your system leashed—which is why you have chosen Ubuntu Unleashed to help you.
The Linux file system designers have done an excellent job of selecting default values used for file system creation, and any version of the Linux kernel after 2.6.x contains code for the IDE subsystem that significantly improves I/O (input/output) transfer speeds over older versions, obviating much of the need for special tweaking of the file system and drive parameters if you use IDE disks. Although these values work well for most users, some server applications of Linux benefit from file system tuning. As always, observe and benchmark your changes.
Synchronizing the File System with sync
Because Linux uses buffers when writing to devices, a write will not occur until the buffer is full, until the kernel tells it to, or if you tell it to by using the sync command. Traditionally, the command is given twice, as in the following example:
matthew@seymour:~$ sync ; sync
To do it twice is overkill. Still, it can be helpful before the unmounting of certain types of media with slow write speeds (such as some USB hard drives or PCMCIA storage media)—but only because it delays the user from attempting to remove the media too soon rather than because two syncs are better than one.
The tune2fs Command
With tune2fs, you can adjust the tunable file system parameters on an ext2, ext3, or ext4 file system. A few performance-related items of note are as follows:
▸ To disable file system checking, the -c 0 option sets the maximal mount count to zero.
▸ The interval between forced checks can be adjusted with the -i option.
▸ The -m option sets the reserved blocks percentage with a lower value, freeing more space at the expense of fsck having less space to write any recovered files.
▸ Decrease the number of superblocks to save space with the -O sparse_super option. (Modern file systems use this by default.) Always run e2fsck after you change this value.
▸ More space can be freed with the -r option, which sets the number of reserved (for root) blocks.
Note that most of these uses of tune2fs free up space on the drive at the expense of the capability of fsck to recover data. Unless you really need the space and can deal with the consequences, just accept the defaults; large drives are now relatively inexpensive.
The e2fsck Command
The e2fsck utility checks an ext2/ext3/ext4 file system. Some useful arguments taken from man e2fsck are as follows:
▸ -c—Checks for bad blocks and then marks them as bad
▸ -f—Forces checking on a clean file system
▸ -v—Enters verbose mode
The badblocks Command
Although not a performance-tuning program per se, the utility badblocks checks a partition (preferably unmounted) for bad blocks. Running this command by itself is not recommended; rather, you should allow it to be called by fsck. You should use it directly only if you specify the block size accurately; don’t guess or assume anything.
The options available for badblocks are detailed in the man page. They allow for very low-level manipulation of the file system that is useful for data recovery by file system experts or for file system hacking, but they are beyond the scope of this chapter and the average user.
Disabling File Access Time
Whenever Linux reads a file, it changes the last access time (known as the atime). This is also true for your web server: If you are getting hit by 50 requests a second, your hard disk will be updating the atime 50 times a second. Do you really need to know the last time a file was accessed? If not, you can disable the atime setting for a directory by typing this:
matthew@seymour:~$ chattr –R +A /path/to/directory
The chattr command changes file system attributes, of which “don’t update atime” is one. To set that attribute, use +A and specify -R so that it is recursively set. /path/to/directory gets changed, and so do all the files and subdirectories it contains.
Kernel
As the Linux kernel developed over time, developers sought a way to fine-tune some of the kernel parameters. Before sysctl, those parameters had to be changed in the kernel configuration, and then the kernel had to be recompiled.
The sysctl command can change some parameters of a running kernel. It does this through the /proc file system, which is a “virtual window” into the running kernel. Although it might appear that a group of directories and files exist under /proc, that is only a representation of parts of the kernel. When you’re the root user (or using the sudo command), you can read values from and write values to those “files,” referred to as variables. You can display a list of the variables as shown in the following example (which presents an abbreviated list because roughly 250 items or more exist in the full list):
matthew@seymour:~$ sysctl -A net.ipv4.tcp_max_syn_backlog = 1024 net.ipv4.tcp_rfc1337 = 0 net.ipv4.tcp_stdurg = 0 net.ipv4.tcp_abort_on_overflow = 0 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_syncookies = 0 net.ipv4.tcp_fin_timeout = 60 net.ipv4.tcp_retries2 = 15 net.ipv4.tcp_retries1 = 3 net.ipv4.tcp_keepalive_intvl = 75 net.ipv4.tcp_keepalive_probes = 9 net.ipv4.tcp_keepalive_time = 7200 net.ipv4.ipfrag_time = 30
The items shown are networking parameters, and tweaking these values is beyond the scope of this book. If you want to change a value, however, you use the -w parameter:
matthew@seymour:~$ sysctl -w net.ipv4.tcp_retries 2=20
This increases the value of that particular kernel parameter.
If you find that a particular setting is useful, you can enter it into the /etc/sysctl.conf file. The format is as follows, using the earlier example:
net.ipv4.tcp_retries 2=20
Of more interest to kernel hackers than regular users, sysctl is a potentially powerful tool that continues to be developed and documented.
Tip
The kernel does a good job of balancing performance for graphical systems, so there’s not a great deal you can do to tweak your desktop to run faster.
Both GNOME and KDE are “heavyweight” desktop systems: They are all-inclusive, all-singing, and all-dancing environments that do far more than browse your file system. The drawback is that their size makes them run slowly on older systems. On the flip side, Ubuntu has other systems available in the repositories, like the Xfce and LXDE desktops, which are a great deal slimmer and faster than GNOME and KDE. If you find that GNOME and KDE are struggling just to open a file browser, Xfce or LXDE are likely to work for you.
Tuned
The Tuned Project provides a system tuning service for Linux. Tuned does many things to help you optimize your system. Tuned is a profile-based system that uses the udev device manager to monitor connected devices and also enables both static and dynamic tuning of system settings according to your chosen profile. It can drastically improve performance.
Tuned does the following:
▸ Monitors connected devices
▸ Tunes system settings using a profile you designate
▸ Supports configurations like sysctl, sysfs, and kernel boot command-line parameters
▸ Support hot-plugging of devices
▸ Can be run in no-daemon mode, which reduces functionality, which is needed in secure environments
▸ Stores its configuration in one place
▸ Allows hierarchical definitions of profiles through inheritance
▸ Supports full rollback of changes it makes
▸ Includes a set of predefined profiles for common use cases while also allowing the creation of custom profiles
To begin, install Tuned.
$ sudo apt install tuned
To view a list of available profiles for you to use, enter
$ sudo tuned-adm list
Listed profiles include short descriptions.
To show the profile currently in use, enter
$ sudo tuned-adm active
To show the recommended profile for your system, enter
$ sudo tuned-adm recommend
To change the profile currently in use, enter this, where profilename is the name of the profile you desire to switch over to using:
$ sudo tuned-adm profile profilename
To verify that the system is currently using the settings in your current profile (checking to see whether a setting has perhaps been directly modified after the Tuned profile was set), enter
$ sudo tuned-adm verify
To unload all tunings made by Tuned, enter
$ sudo tuned-adm off
The Tuned daemon is controlled by systemd using standard systemctl commands like start, stop, and restart. Tuned is incredibly easy to use and in some settings can make a big difference, so it’s worth checking out. For more information, see https://tuned-project.org/.
References
▸ www.coker.com.au/bonnie++/—The home page of Bonnie++, a disk benchmarking tool, which also contains a link to RAID benchmarking utilities and Postal, a benchmarking utility for SMTP servers
▸ www.phoronix-test-suite.com—Home of the Phoronix Test Suite, which does automated performance testing and comparisons and is a quality benchmarking software option to consider
▸ https://tuned-project.org—The official website for Tuned