Testing with Valgrind

Valgrind is the most commonly used collection of open source tools for analyzing and profiling everything from the cache and heap behavior of an application to memory leaks and potential multithreading issues. It works in tandem with the underlying operating system as, depending on the tool used, it has to intercept everything from memory allocations to instructions related to multithreading and related. This is the reason why it is only fully supported under Linux on 64-bit x86_64 architectures.

Using Valgrind on other supported platforms (Linux on x86, PowerPC, ARM, S390, MIPS, and ARM, also Solaris and macOS) is definitely also an option, but the primary development target of the Valgrind project is x86_64/Linux, making it the best platform to do profiling and debugging on, even if other platforms will be targeted later on.

On the Valgrind website at http://valgrind.org/info/platforms.html, we can see a full overview of the currently supported platforms.

One very attractive property of Valgrind is that none of its tools require us to alter the source code or resulting binary in any fashion. This makes it very easy to integrate into an existing workflow, including automated testing and integration systems.

On Windows-based system, tools such as Dr. Memory (http://drmemory.org/) are available as well, which can handle at least the profiling of memory-related behavior. This particular tool also comes with Dr. Fuzz, a tool that can repeatedly call functions with varying inputs, potentially useful for integration testing.

By using an integration test such as what we looked at in the previous section, we're free to fully analyze the behavior of our code from the comfort of our PC. Since all of Valgrind's tools significantly slow down the execution of our code (10-100 times), being able to do most of the debugging and profiling on a fast system means that we can save a significant amount of time before embarking on testing on the target hardware.

Of the tools we'll likely use the most often, Memcheck, Helgrind, and DRD are useful for detecting memory allocation and multithreading issues. Once our code passes through these three tools, while using an extensive integration test that provides wide coverage of the code, we can move on to profiling and optimizing.

To profile our code, we then use Callgrind to see where our code spends the most of the time executing, followed by Massif to do profiling of heap allocations. With the information we can glean from this data, we can make changes to the code to streamline common allocation and de-allocation cases. It might also show us where it might make sense to use a cache to reuse resources instead of discarding them from memory.

Finally, we would run another cycle of MemCheck, Helgrind, and DRD to ensure that our changes didn't cause any regressions. Once we're satisfied, we move on to deploying the code on the target system and see how it performs there.

If the target system also runs Linux or other supported OSes, we can use Valgrind on there as well, to check that we didn't miss anything. Depending on the exact platform (OS and CPU architecture), we may run into limitations of the Valgrind port for that platform. These can include errors such as unhandled instruction, where the tool hasn't had a CPU instruction implemented and hence Valgrind cannot continue.

By extending the integration test to use the SBC instead of a local process, we can set up a continuous integration system whereby, in addition to the tests on a local process, we also run them on real hardware, taking into account the limitations of the real hardware platform relative to the x86_64-based Linux system used for most of the testing.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.222.115.120