Testing and Quality Assurance

Testing a kernel extension can be a challenge. Modern computer systems and devices are usually very complex, and even if your component is fairly isolated from the rest of the system, unwanted side effects happen as result of it being another cog in the machinery. Proper testing, preferably conducted by those not directly involved in the engineering process, is essential—those directly involved may have preconceived notions of how the product works and fail to discover issues an end-user would. Testing kernel extensions is very dependent on the nature and type of the extension; so, providing an exhaustive list of things to test for is impossible. The following are some general suggestions:

• Test that the driver handles going to sleep and waking up. You can stress test this easily by using the SleepX tool included as part of Xcode. The application allows you to execute a script or external program every time the computer wakes up. You can also tune sleep intervals and the number of cycles you wish to test. It is extremely important that your driver handles sleep properly, especially for mobile devices and laptops, as having the driver still active means the CPU will need to be active—this will drain the system’s battery and could even cause a laptop to overheat if it is operating with the lid closed.
• Test that the driver handles repeated loading and unloading with kextload and kextunload. You can test for reference leaks or memory allocation leaks using the ioclasscount and ioalloccount tools.
• Test applications that use the driver by performing common user tasks. For example, for an audio device, playback and capture audio with as many permutations of settings and formats as possible.
• Test on all supported platforms and operating systems, such as Macbook Pro, Mac Mini, iMac, Mac Pro or iPod, iPhone, and iPad. For Macs, be sure to test both 32-bit and 64-bit versions and all supported OS versions; e.g., Snow Leopard, Lion, etc.
• For hot–pluggable devices:
  • Test that drivers and devices continue to work correctly after repeated plugging and unplugging. You may wish to do a minimum of 100+ repetitions to be confident no issue will happen in the field.
  • Test that the driver handles the system going to sleep. The driver should also be prepared to handle the device being removed while the system was sleeping.
• Test 32-bit and 64-bit versions of a KEXT. If you support pre-Lion operating systems, you will want to provide universal binaries with support for both 32-bit and 64-bit systems.
• Test the installer package; make sure that all files are installed correctly—with the correct permissions, and in the correct location.

If a problem is discovered either during testing or reported by a customer, it is a good idea to have pre-archived symbol information and/or debug versions for every released KEXT, so you can quickly attempt to reproduce a problem and debug it. This may seem obvious to most people, but it does happen more often than you might think: someone passes on a release of a software component, such as a KEXT, without incrementing its version number. Every time a change is made and released/given to some external entity (or even the internal QA group), the version number should be unique, or you will quickly lose track and confuse everyone involved.