There are many solutions that provide filesystem backup capabilities. These include everything from apps such as Dropbox, Box, Carbonite to hardware solutions such as Apple's Time Machine, Seagate, or network-attached storage products, to name a few. Most consumer tools provide some key automatic functionality, along with an app or website for you to manage your policies and content. Often, especially for developers, these tools don't quite do the things we need them to. However, thanks to Go's standard library (that includes packages such as ioutil and os) we have everything we need to build a backup solution that behaves exactly as we need it to.
For our final project, we will build a simple filesystem backup for our source code projects that archive specified folders and save a snapshot of them every time we make a change. The change could be when we tweak a file and save it, or if we add new files and folders, or even if we delete a file. We want to be able to go back to any point in time to retrieve old files.
Specifically in this chapter, you will learn:
- How to structure projects that consist of packages and command-line tools
- A pragmatic approach to persisting simple data across tool executions
- How the
ospackage allows you to interact with a filesystem - How to run code in an infinite timed loop, while respecting Ctrl + C
- How to use
filepath.Walkto iterate over files and folders - How to quickly determine if the contents of a directory have changed
- How to use the
archive/zippackage to zip files - How to build tools that care about a combination of command-line flags and normal arguments
We will start by listing some high-level acceptance criteria for our solution and the approach we want to take:
- The solution should create a snapshot of our files at regular intervals, as we make changes to our source code projects
- We want to control the interval at which the directories are checked for changes
- Code projects are primarily text-based, so zipping the directories to generate archives will save a lot of space
- We will build this project quickly, while keeping a close watch over where we might want to make improvements later
- Any implementation decisions we make should be easily modified if we decide to change our implementation in the future
- We will build two command-line tools, the backend daemon that does the work, and a user interaction utility that will let us list, add, and remove paths from the backup service
It is common in Go solutions to have, in a single project, both a package that allows other Go programmers to use your capabilities, and a command-line tool that allows end users to use your code.
A convention is emerging to structure the project by having the package in the main project folder, and the command-line tool inside a subfolder called cmd, or cmds if you have multiple commands. Because all packages (regardless of the directory tree) are equal in Go, you can import the main package from the subpackages, knowing you'll never need to import the commands from the main package. This may seem like an unnecessary abstraction, but is actually quite a common pattern and can be seen in the standard Go tool chain with examples such as gofmt and goimports.
For example, for our project we are going to write a package called backup, and two command-line tools: the daemon and the user interaction tool. We will structure our project in the following way:
/backup - package /backup/cmds/backup – user interaction tool /backup/cmds/backupd – worker daemon