Filtering with grep

When you’re dealing with program output, you’ll often want to filter the results. The grep command lets you search text for characters or phrases. You can use grep to search through program output or a file. Let’s explore grep by working with some files.

Create a file named words.txt that contains several words, each on its own line:

Now use grep to search the file for the word and:

This displays the two lines of the file that show the string you specified. You get both results because they contain the string and somewhere on the line. This is the most simple form of searching. Surrounding the search term in quotes isn’t always necessary, but it’s a good habit to get into because you can run into some strange edge cases with special characters if you don’t.

You can also tell grep to remove lines containing that text. The -v option instructs grep to only show lines that don’t contain the search pattern you specified.

grep reads the file in and processes its contents, but you’re not limited to using grep on just files. You can use it to process output from other programs, which means you can use it to filter the streams of text other programs display.

Try it out by using grep to show you all the ls commands in your history:

When you ran the command on your machine, you probably saw a lot of results, and the last result was the history | grep command. You can filter that last command out by piping the output to grep again:

If there are too many commands for you to see, you can always pipe the output to less:

grep supports searching multiple files as well. Create another file with some more words:

Then, use grep to search both files for the word blue:

This time, grep shows the word, along with the name of the file that contains the word.

The grep command only shows the exact line containing the match, but you can tell it to give you a little more context. Using the -A and -B switches, you can specify the number of lines above and below the match:

The output separates the matches clearly.

In this example, you selected the same amount of lines before and after the matched line. In cases like this, you can shorten the command by using the -C switch instead of specifying both -A and -B:

The resulting output is the same as before. The -C switch shows the “context” around the results.

Adding the -n flag will show you the line number where the match was found:

This is helpful when working with source code. You can use grep to look at your entire codebase and find phrases or keywords quickly, as grep can read directories recursively.

To demonstrate this, use grep to scan the contents of the /var/log folder for instances of your username:

You’ll see a stream of data returned, displaying events from your system logs.

All of the searches you performed so far are simple text searches, but you can use regular expressions, or regexes as well. A regex is a sequence of characters that defines a pattern for finding text.

This book doesn’t go into a ton of detail on regular expressions. However, you’ll use regular expressions a few more times throughout this book, so I’ll explain what’s going on with each one.

If you’d like more information on regular expressions, lots of online resources will help get you started, including Regex101,^[10] an interactive online tool for building and debugging regular expressions.

For now, let’s try out regular expression with grep. If you search both files for the letter b, you get all of the lines containing that word:

But if you use the regular expression ^b, which means “look for the lower-case letter b at the beginning of the line,” you only see two results: blue and blue car:

Similarly, if you use the expression e$, which means “look for any line ending with the letter e,” you see these three results:

Likewise, use the regular expression blue|apple to search for lines that contain “blue” or “apple”. To use this regular expression with grep, use the -E switch:

The -E switch lets you use extended regular expressions, which means that the characters |, ?, +, {, (, and ) are supported in the expression. These characters let you create more advanced search patterns. For example, the expression a(n|r) will look for any lines containing either “an” or “ar”:

grep is a general purpose text search tool, and while there are some other options out there like ack^[11] or ripgrep,^[12] which have additional features aimed at working with source code, you should be comfortable using grep since it’s universally available.

Next, you’ll look at how to remove characters from output.