There are many solutions that provide filesystem backup capabilities. These include everything from apps such as Dropbox, Box, and Carbonite to hardware solutions such as Apple's Time Machine, Seagate, or network-attached storage products, to name a few. Most consumer tools provide some key automatic functionality, along with an app or website for you to manage your policies and content. Often, especially for developers, these tools don't quite do the things we need them to. However, thanks to Go's standard library (which includes packages such as ioutil and os), we have everything we need to build a backup solution that behaves exactly the way we need it to.
For our next project, we will build a simple filesystem backup for our source code projects that archive specified folders and save a snapshot of them every time we make a change. The change could be when we tweak a file and save it, when we add new files and folders, or even when we delete a file. We want to be able to go back to any point in time to retrieve old files.
Specifically, in this chapter, you will learn about the following topics:
- How to structure projects that consist of packages and command-line tools
- A pragmatic approach to persisting simple data across tool executions
- How the
ospackage allows you to interact with a filesystem - How to run code in an infinite timed loop while respecting Ctrl + C
- How to use
filepath.Walkto iterate over files and folders - How to quickly determine whether the contents of a directory have changed
- How to use the
archive/zippackage to zip files - How to build tools that care about a combination of command-line flags and normal arguments
We will start by listing some high-level acceptance criteria for our solution and the approach we want to take:
- The solution should create a snapshot of our files at regular intervals as we make changes to our source code projects
- We want to control the interval at which the directories are checked for changes
- Code projects are primarily text-based, so zipping the directories to generate archives will save a lot of space
- We will build this project quickly, while keeping a close watch over where we might want to make improvements later
- Any implementation decisions we make should be easily modified if we decide to change our implementation in the future
- We will build two command-line tools: the backend daemon that does the work and a user interaction utility that will let us list, add, and remove paths from the backup service
It is common in Go solutions to have, in a single project, both a package that allows other Go programmers to use your capabilities and a command-line tool that allows end users to use your programs.
As we saw in the last chapter, a convention to structure such projects is emerging whereby we have the package in the main project project folder and the command-line tool inside a subfolder called cmd or cmds if you have multiple commands. Because all packages are equal in Go (regardless of the directory tree), you can import the package from the command subpackages, knowing you'll never need to import the commands from the project package (which is illegal as you can't have cyclical dependencies). This may seem like an unnecessary abstraction, but it is actually quite a common pattern and can be seen in the standard Go tool chain with examples such as gofmt and goimports.
For example, for our project, we are going to write a package called backup and two command-line tools: the daemon and the user interaction tool. We will structure our project in the following way:
/backup - package /backup/cmds/backup - user interaction tool /backup/cmds/backupd - worker daemon