This chapter is a guide to the command line tools you should learn for the biggest bang for the buck. Mastery of the command line is important for two reasons: first, because it can make you much faster in day-to-day tasks, even ones as simple as browsing code in a codebase and, second, because it's a way to signal to other programmers that you mean business. I wouldn't exactly call it a shibboleth—it's useful for reasons beyond helping programmers recognize our own—but good command line skills do build a little cred, and a little cred never hurt anyone.
This list is nowhere close to the end of what you should learn, but if you familiarize yourself with everything on the list, you'll be well on your way. Looking down the barrel of two pages of pages of bullets, that prospect probably seems daunting, but many of these tools are quite simple and can be learned in minutes. For each one, you should read the entire man page1 where one exists; you won't retain everything, but you just might remember something when it counts. Where there's no man page, you can find a blog post or buy a book.
Shell
I/O redirection and pipes: These will allow you to rapidly build complex ad hoc tooling by composing other tools, which you'll have to do all the time. You should at the very least know >, >>, <, |, >&, and <<.
cd: learn to back up with -.
Backgrounding with &.
Killing and suspending with ctrl-c and ctrl-z; foregrounding with fg.
Setting and reading variables; quoting and escaping therein.
Special variables: $?, !$, !!, $#, %%, $!, $_, $0 through $9 (command line arguments).
- Configuring your environment:
PATH, .bashrc, and .bash_profile
Aliases
for and while loops: I write ad hoc loops to iterate over static or dynamic lists literally every day.
read into a variable (often in a while loop).
if-then-else, [[ ]]; file and string tests like -z, -n, and their many friends.
Helpful shell config settings like (in bash) set -x, set -e, set -u, set -o pipefail.
Up arrow and ctrl-r for browsing history.
hash -r and hash -d for managing cached paths to tools.
Defining functions.
Environment variables.
PIPESTATUS.
which: see where a command comes from.
/dev/null.
String substitutions and default values: ${foo:-bar}, ${foo:=bar}, ${foo#bar}, and many more.
Information Discovery and Text Manipulation
grep: The original search tool. Emphasis on -R, -l, -L, -E (with | for "or"), and -v.
find: Identify files, including complex boolean tests.
xargs: For turning output of one command into arguments to the next; often a better option than a while loop.
ls: Get familiar with at least -R, -d, -t, -l, -1.
cat: Extract file content.
sort and uniq; when ad hoc debugging, almost nothing is more useful than <do something> | sort | uniq -c # How many of each type?
- awk: at least enough to print columns (it's extremely rich, but column printing is a great start): awk '{print $1}':
You can also try cut, but I prefer awk.
wc: How many words, characters, or lines in a body of text? my-tool | sort | uniq | wc.
less, head, and tail for quick glances a files.
sed: Complex and powerful, but you should learn enough for simple substring replacement.
Regular expressions: Learn the basics; they pay off again and again.
column: Align text into columns.
tr: fast character mappings.
Special Section for the Best Tool Ever
I've never met an engineer familiar with jq who doesn't consider it a remarkable achievement; some say, including me, that it may be the greatest tool ever written—a one-stop shop for processing json. You should spend a cozy afternoon with the man page; try to go beyond the basic usage.
Networking
ping: confirm you can reach a host/IP.
dig and host: DNS.
netstat: List sockets on your host. If nothing else, remember netstat -nap.
curl and its countless options for ad-hoc HTTP requests. If nothing else, you should be able to pass headers and change your HTTP verb without looking at the man page.
nc: Ad hoc TCP (like sending some bytes over a TCP connection).
To examine a host's IP networking: ifconfig on mac, ip on Linux.
Local and Remote Sessions
ssh: Tunnel with -L and -R, and glance at the -o options. Learn to run both interactive sessions and one-shot commands.
scp and rsync for copying files across machines (rsync is your friend when performance becomes a bottleneck).
tmux or screen: Multiple shells in a single window and sessions you can rejoin on remote machines.2
Running Processes and Host State
ps, pgrep, top, pstree, and htop for looking at running processes
lsof for looking at open files
pkill, kill, and killall for sending signals
iostat/iotop for I/O behavior
df and du for disk usage
journalctl and systemctl for logs and daemon state on Linux
strace for observing system calls on Linux
/proc for more process details on Linux
Databases
You should master the command line of whatever database you're using; I promise that you will need to introspect the DB at some point and that this will be the best way.
Git
git cherry-pick
git rebase (favor this over merging! It will make your histories cleaner)
git log
git shortlog
git tag
git show
git reset and git reset --hard
git stash
git status
git rev-parse
Miscellany
The ISO8601 standard for dates: You don't need to pass around times as Unix timestamps or with ambiguous day–month ordering. Use a standard! Ref: date --iso-8601.
md5 and sha1sum for when you need a quick checksum, especially when you want to check if two files are the same without sending their bytes over the wire!
uuidgen -v4 for when you need a random UUID.
- If you use a Mac:
pbcopy and pbpaste for using the Mac clipboard in shell pipelines.
open for opening browser windows (or other apps) from the shell.
homebrew (brew) for package management.
autojump is one of the best tips I've ever gotten, for faster command line navigation; look it up.