Chapter 5. Developing your Style Guide

The overarching theme of this book is writing idiomatic, yet readable bash code, in a consistent style, and we hope we’ve provided the tools you need to do that. Style is just another way to say, “how we write things.” Find some style guidelines, write ‘em down, and stick to ‘em. We’ve covered a number of important style considerations in this book, and there are some other guidelines we’d like to mention as well, things to keep in mind when designing systems and writing code. You can use this chapter as a starting point for your own style guide, or just adopt is as-is if you like it. Appendix A is the same material without the talking points, to use as a “cheat sheet.”

Some high level principles to keep in mind are:

  • Above all: KISS — Keep It Simple, Stupid! Complexity is the enemy of security 1, but it’s also the enemy of readability and understanding. Sure, modern systems and circumstances are complex, so try hard not to make it any worse than it already is.

  • The corollary, as Brian Kernighan famously said, is that debugging is twice as hard as writing the code in the first place, so if your code is as clever as you can make it, you are—by definition—not smart enough to debug it.

  • Try not to re-invent the wheel. Whatever you are trying to do has probably been done before, and there’s likely a tool or library for it. If that tool is already installed, just use it. No matter how how hard you try you are never going to be able to match the effort and testing that went into rsync, so just use it. If you find random code on the internet…well, think about that one for a bit.

  • Plan ahead for special cases or overrides, since they will happen. Take a page out of Linux distribution packaging systems and provide an /etc/thing/global.cfg for defaults that you can blindly overwrite, then allow for overrides in /etc/thing/config.d/ or similar, see The dropin directories section in Chapter 9.

  • If it’s not in revision control it doesn’t exist! (And sooner or later, it will be lost and then really won’t exist.)

  • Document everything. (But you don’t have to write a book about it…oh wait…) Write your code, your comments, and your docs for the new person who will join the team a year from now, when you’ve forgotten why you did it that way, so you can’t explain it to them. Document what didn’t work, and maybe why, and cross-reference things, especially things that you can predict can hurt you. (Yes, rm -rf /$undefined_variable turned out to be a really bad idea!)

  • Keep your code and documentation DRY; Don’t Repeat Yourself. As soon as you end up with more than one copy, it’s guaranteed they will get out of sync sooner or later, the only question is when. Use functions and libraries, don’t be WET (We Enjoy Typing).

Bash Idioms Style Guide is not portable

This Bash Idioms Style Guide is specifically for Bash, so it is not portable to POSIX, Bourne, Dash or other shells.! If you need to write for those shells you will need to test and tweak this guide to account for the supported syntax and feature of those shells.

Getting down into more detail, what types of things should be captured in your style guide? We’ll cover some of them in the following sections.

Readability

Readability of your code is important! Or as Python says, Readability counts. You only write it once, but you (and others) will probably read it many times. Spend the extra few seconds or minutes thinking about the poor clueless person trying to read the code next year…it’s very likely to be you. There’s a balance and a tension between abstraction (Don’t Repeat Yourself) and readability.

  • KISS (Keep it Simple, Stupid!)

  • READABILITY: don’t be “clever,” be clear

  • Good names are critical!

  • ALWAYS USE A HEADER

  • If at all possible, emit something useful for -h, --help and incorrect arguments!

    • Prefer using a “here” document (with leading TABs) to a bunch of echo lines

  • Use source (instead of .) to include config files, which should end in .cfg (or .conf or whatever your standard is)

  • If at all possible use ISO-8601 2 dates for everything

  • If at all possible keep lists of things (e.g., ip addresses, hostnames, packages to install, whatever) in alphabetical order; this prevents duplication and makes it easier to add or remove items

    • This applies to case statements, contents of variables or arrays/lists, and wherever it makes sense

  • If possible, use long arguments to commands for readability, e.g., use diff --quiet instead of diff -q, though watch out for portability to non GNU/Linux systems

    • If any options are short or obscure, add comments.

    • Also strongly consider documenting why you chose or needed the options you chose, and even options you considered but didn’t use for some reason.

    • Definitely document any options that might seem like a good idea but that actually can cause problems, especially if they are options you commonly use elsewhere.

For variable naming, watch out for generic variable names like:

`${GREP} ${pattern} ${file}`

That’s very abstracted out, and maybe reusable, but it’s also pretty context free. We’ve replaced the older `` with newer and more readable (we think) $(), but more importantly the variables are less noisy because we omitted unnecessary ${} and have meaningful names:

$($GREP "$re_process_errors" $critical_process_daily_log_file)

After a while, you’ll see that you and your team find certain (dare we say it) idiomatic ways to express concepts and operations you often face. That’s when it’s time to write a style guide, if you don’t already have one.

On the other hand, if you are submitting a patch or maintaining code, it’s best to follow the conventions of that code unless you’re going to restyle and possibly refactor the whole thing.

Comments

A lot has been written about comments: what, when, where, and so on. Following are style guidelines related to comments:

  • ALWAYS USE A HEADER

  • Write your comments for the new person on the team a year from now

  • Comment your functions

  • Do not comment on what you did. Comment on why you did, or did not do, something

    • Exception: comment on what you did when bash itself is obscure

  • Consider adding comments about external program options, especially if they are short or obscure

Add useful comments that explain why you did something and your intent! Like:

continue # to the next file in for loop

In theory it should not be necessary to explain what something does if you write clear code. But sometimes bash itself is just convoluted or obscure, e.g., variable substitution like:

PROGRAM=${0##*/}  # Basename in bash, see "Prefix Removal" in Chapter 4

Separators are often useful for delimited logical blocks of code. But don’t add a closing separator at the bottom to “box” it out, that just adds clutter and reduces your on-screen lines-of-code by 1. Whatever you do, do not build boxes with characters on the right-hand-side! It’s totally unnecessary and you’ll waste a lot of time getting it to “look right” up front. What’s worse is that it’s a strong disincentive to fixing or updating comments later, because then you have to go fix the box again.

Don’t do this:

#################################################################
# Please don't build boxes like this!                           #
#                                                               #
# They make it VERY painful to edit the comments later, because #
# now you have to worry about the closing character on the      #
# right-hand-side.  This example isn't too bad, but it gets     #
# out of hand quickly.                                          #
#################################################################

Do this:

#################################################################
# Please DO build boxes like this!
#
# This one is easier to edit, because while you still have to
# worry about text wrapping, it's just a simple leading "#" when
# do do re-wrap.

Names

Naming is important. We can’t stress that enough. The difference between $file and $critical_process_daily_log_file doesn’t seem like anything except some extra typing now, when all the details are in your head. But we guarantee if you take the extra time to really think about what you are naming and how it will read in the code, that will pay off soon, by reducing coding errors, and in the future when re-reading, debugging, and enhancing. Following are style guidelines related to names:

  • Good names are critical!

  • Global variables and constants are in UPPER case

    • Prefer not to make changes to global variables, but sometimes that’s just much simpler

    • Use readonly or declare -r for constants

  • Other variables are in lower case

  • Functions are in Mixed_Case

  • Use “_” in place of space or CamelCase (remember “-” is not allowed in variable names)

  • Use bash arrays carefully, they can be hard to read (see Chapter 4). for var in $regular_var often works as well.

  • Replace $1, $2, .. $N with readable names ASAP

    • That makes everything much more debugable and readable, but also makes it easy to have defaults and add or re-arrange arguments

  • Distinguish between types of referents, like $input_file vs. $input_dir

  • Use consistent “FIXME” and “TODO” labels, with names and ticket numbers if appropriate

Consider how easy it would be to confuse these and use (or make an off-by-one typo) the wrong one:

file1='/path/to/input'
file2='/path/to/output'

This is much more intuitive and harder to mess up:

input_file='/path/to/input'
output_file='/path/to/output'

Also, don’t leave out two characters to save typing, just spell out $filename so you don’t get it wrong later. Was it $filenm or $flname or $flnm or what?

Functions

Following are style guidelines related to fuctions:

  • ALWAYS USE A HEADER

  • Good names are critical!

  • Functions must be defined before they are used

    • Group them at the top and use two blank lines and a function separator between each function.

    • do NOT intersperse code between functions!

  • Use Camel_Case and “_” to make function names stand out from variable names.

  • Use function My_Func_Name { instead of My_Func_Name() { because it’s more clear and easier to grep -P '^s*function '.

  • Each function should have comments defining: what it does, inputs (including GLOBALS), and outputs

  • When you have useful, standalone pieces of code, make them a function.

  • Use a function any time you use the same (or substantially similar) block of code more than once. If they are very common, like logging or emailing, consider creating a “library.” that is, a single common file you can source as needed.

    • Prefix “library” functions with “_”, like _Log, or some other prefix.

  • Consider using “filler” words for readability in arguments if it makes sense, then define them as local junk1="$2" # unused filler, e.g.:

    • _Create_File_If_Needed "/path/to/$file" containing important value

  • Do use the local builtin when setting variables in functions.

    • But be aware that it will mask a return code, so declare and assign it on separate lines if using command substitution, like local my_input and then my_input="$(some-command)".

  • For any function longer than about 25 lines, close it with } # end of function MyFuncName to make it easier to track where you are in the code on your screen. For functions shorter than 25 lines this is optional but encouraged unless it gets too cluttered.

  • Don’t use a main function, it’s almost always just an unnecessary layer.

    • That said, using “main” makes sense to Python and C programmers, or if the code is also used as a library, and it may be required if you do a lot of unit testing.

  • Consider using 2 blank lines and a main separator above the main section, especially when you have a lot of functions and definitions at the top.

Also, define a single logging function in your library (e.g., _Log), and use it! Otherwise you’ll end up with a wild mix of logging functions, styles, and destinations. Ideally, as we said above, log to syslog and let the OS worry about final destination(s), log rotation, etc.

Quoting

Quoting is pretty simple, until it’s not and then you get a headache. We know how this works, and we still end up doing trial-and-error sometimes, especially when trying to create a one-liner to run as another user via sudo or on another node via ssh. Add some echo lines, and be careful. Following are style guidelines related to quoting:

  • Do put quotes around variables and strings because it makes them stand out a little and clarifies your intent

    • Unless it gets too cluttered

    • Or they need to be unquoted for expansion

  • Don’t quote integers

  • Use single quotes unless interpolation is required

  • Don’t use ${var} unless needed, it’s too cluttered

    • But that IS needed sometimes, like ${variable}_suffix or ${being_lower_cased,,}

  • Do quote command substitutions, like: var="$(command)"

  • ALWAYS quote both sides of any test statement like [[ "$foo" == bar ]]

    • FIXME-TOOLS: AsciiDoc missing single quotes again!

    • Unless you are using ~= in which case you can’t quote the regular expression!

  • Consider single quoting variables inside echo statements, like echo "cd to $DIR failed." because it’s more clear when a variable is unexpectedly undefined or empty

    • FIXME-TOOLS: AsciiDoc missing single quotes again!

    • Or echo "cd to [$DIR] failed." as you like

  • Prefer single quotes around printf formats (see Chapter 6)

Layout

Following are style guidelines related to layout:

  • Line things up! Multiple spaces almost never matter in bash (except around =), and lining up similar commands make it easier to read and to see both the similarities and differences.

  • DO NOT ALLOW TRAILING WHITE SPACE! This will later cause noise in the VCS (version control system) when removed.

  • Indent using 4 spaces, but use TAB with here-docs as needed

  • Break long lines at around 78 columns, indent line continuations 2 spaces, and break just before | or > so those parts jump out as you scan down the code

  • The code to open a block goes on 1 line, like:

    • if expression; then

    • for expression; do

  • List elements in case..esac are indented 4 spaces and closing ;; are at that same indent level, blocks for each item are also indented 4 spaces

    • One-line elements should be closed with ;; on the same line

    • Prefer lining up the ) in each element, unless it gets cluttered or out of hand

    • See the example code in Chapter 8

There’s an argument about not bothering to break lines at 70-80 columns that assumes everyone is using a wide graphical terminal and an IDE. First, depending on the team and person, that may not be true, and second even if it is, when it really breaks and you end up debugging under fire in vi on an 80x24 console, you will not appreciate 200+ column code.

Break your lines.

Break them just before the important part so when you scan down the left side of the column the continuations jump out. We also like to use half the usual indent for broken lines. We can’t do a good (bad?) long line here in this book because it will just break in odd places anyway. But we can provide a simple, and we think readable, example:

... Lots of code, indented a ways...
        /long/path/to/some/interesting/command 
          | grep    "$re_stuff_we_care_about 
          | grep -v "$re_stuff_to_ignore 
          | /path/to/email_or_log_command ...
...more code

Syntax

Following are style guidelines related to syntax:

  • Use #!/bin/bash - or #!/usr/bin/env bash when writing bash code, not #!/bin/sh

    • FIXME-TOOLS: there’s a Markdown/Asciidoctor rendering issue with the three # ! above

  • Use [email protected] unless you are really sure you need $*

  • Use == instead of = for equality, to reduce confusion with assignment

  • Use $() instead of <code>``</code> backticks/backquotes

    • FIXME-TOOLS: there’s a Markdown/Asciidoctor rendering issue with the backticks

  • Use [[ instead of [

  • Use –0— –1— and –2— –3— as needed for integer arithmetic, avoid let and expr

  • Use [[ expression ]] && block or [[ expression ]] || block when simple and readable. Do not use [[ expression ]] && block || block because that doesn’t do what you think it does, use if .. (elif ..) then .. else for that.

Other

Other guidelines:

  • For “system” scripts, log to syslog and let the OS worry about final destination(s), log rotation, etc.

  • Error messages should go to STDOUT, like echo 'A Bad Thing happened' >&2

  • Sanity check that external tools are available

  • Provide useful messages when things fail

  • Set exit codes, especially when you fail

Script Template

This is a sample skeleton or template script you can copy as a reminder and to save some typing when you create a new script.

#!/bin/bash -
# Or possibly: #!/usr/bin/env bash
# <Name>: <description>
# Original Author & date:
# Current maintainer?
# Copyright/License?
# Where this code belongs?  (Hosts, paths, etc.)
# Project/repo?
# Caveats/gotchas?
# Usage?  (Better to have `-h` and/or `--help` options!)
# $URL$  # If using SVN
ID=''    # If using SVN
#_________________________________________________________________________
PROGRAM=${0##*/}  # bash version of `basename`

# GLOBAL and constant variables are in UPPER case
LOG_DIR='/path/to/log/dir'

# Functions are in Mixed Case
###########################################################################
# Define functions

#--------------------------------------------------------------------------
# Example function
# Globals: none
# Input:   nothing
# Output:  nothing
function foo {
    local var1="$1"
    ...
} # end of function foo


#--------------------------------------------------------------------------
# Another example function
# Globals: none
# Input:   nothing
# Output:  nothing
function bar {
    local var1="$1"
    ...
} # end of function bar


###########################################################################
# Main
...

Other Style Guides

We strongly suggestion you have a style guide and use it! If you don’t like ours, and you don’t want to tweak it yourself, you can go steal one from someplace else.

Bash Linter

Using a “linter” instead of or in addition to a style guide can also be handy. To be honest, we don’t use one, but we thought we should cover it. The one we know about is shellcheck, and we have mixed feelings about the results, it’s especially picky about quoting and we don’t always agree with its suggestions either on quoting or in general, but those things are somewhat adjustable. That said, it’s still very cool, and worth checking out.

If you install through your package system remember that the version may be quite old. The Linux version is a tarball containing a single compiled binary though, so it’s easy to drop a current version in your path somewhere.

Summary

Hopefully, this book has given you a better understanding of how to read and write “bashistic” code, and this chapter has provided a place to get started on defining a style (and guide) that will work for you.

Happy bashing!

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset