The following points intend to justify why fuzz tests are popular among the software professional community:
- They are extremely simple, easy, cost-effective, and quick to set up as they are free of preconceptions about system behavior
- Their one-time setup is easy to repeat for regression (automation)
- As it is a protocol-aware test, test results lead to finding precise, descriptive, and easy-to-debug errors
- They enables you to find bugs that are impossible to find with the human eye in defined testing or approach-based testing
- They yields the best results when used in conjunction with black-box testing, beta testing, and other debugging methods
While fuzz tests bring many advantages to the table, we also need to be aware of and understand a few disadvantages, which are listed here:
- Mutation-based fuzz tests can run indefinitely (generate numerous test cases and run indefinitely) and so determining the optimal number of tests, or determining whether time they run for is long enough, is a difficult in some cases
- Tests results may report no defects, even after running numerous test cases
- Test results may report the same defects for various test cases
- It is challenging to find which test case caused the fault
- It is difficult to find the vulnerability in the event of a system crash
To conclude this section, let's list a few tools that you can take advantage of when running fuzz tests for APIs without much pain.
Open source:
Mutational fuzzing |
Fuzzing Frameworks |
Domain Specific fuzzing |
American fuzzy lop |
Sulley |
Microsoft SDL MiniFuzz File Fuzzer |
Radamsa—a flock of fuzzers |
Boofuzz |
Microsoft SDL Regex Fuzzer |
OWASP WebScarab |
BFuzz |
ABNF Fuzzer |
OWASP WSFuzzer |
- |
- |
The preceding table details open source fuzzing tools. The following list provides a few commercial tools you may want to make use of for fuzz tests:
- Codenomicon's product suite
- The Peach Fuzer Platform
- Spirent Avalanche NEXT
- Beyond Security's beSTORM product
And also, here are a few of the latest tools that readers might find exciting to run through their APIs:
Tool |
Reference/Link |
REST-ler |
https://www.microsoft.com/en-us/research/uploads/prod/2018/04/restler.pdf |
Burp |
|
Fuzzapi |
|
Fuzz-rest-api |
|
Big-list-of-naughty-strings |