Adding clap as a Dependency

Although there are various methods and crates for parsing command-line arguments, I will exclusively use the clap crate in this book. To get started, I need to tell Cargo that I want to download this crate and use it in my project. I can do this by adding a dependency to Cargo.toml. Edit your file to add clap version 2.33 to the [dependencies] section:

$ cat Cargo.toml
[package]
name = "echor"
version = "0.1.0"
edition = "2018"

[dependencies]
clap = "2.33" 

The crate name is not quoted. The equal sign indicates the version of the crate, which should be quoted.

The next time I try to build the program, Cargo will download the clap source code (if needed) and all of its dependencies. For instance, I can run cargo build to just build the new binary and not run it:

$ cargo build
    Updating crates.io index
   Compiling libc v0.2.93
   Compiling bitflags v1.2.1
   Compiling unicode-width v0.1.8
   Compiling vec_map v0.8.2
   Compiling strsim v0.8.0
   Compiling ansi_term v0.11.0
   Compiling textwrap v0.11.0
   Compiling atty v0.2.14
   Compiling clap v2.33.3
   Compiling echor v0.1.0 (/Users/kyclark/work/sysprog-rust/playground/echor)
    Finished dev [unoptimized + debuginfo] target(s) in 27.30s

You may be curious where these packages went. Cargo places the download source code into $HOME/.cargo, and the build artifacts go into the target/debug/deps directory. This brings up an interesting part of building Rust projects: Each program you build can use different versions of crates, and each program is built in a separate directory. If you have ever suffered through using shared modules as is common with Perl and Python, you’ll appreciate that you don’t have to worry about conflicts where one program requires some old obscure version and another requires the latest bleeding-edge version in GitHub. Python, of course, offers virtual environments to combat this problem, and other languages have similar solutions. Still, I find Rust’s approach to be quite comforting.

A consequence of Rust placing the dependencies into the target directory is that it’s now quite large. I’m already at close to 30MB for the project directory, with almost all of that living in the target directory. If you run cargo help, you will see that the clean command will remove the target directory at the expense of having to recompile again in the future. You might do to reclaim disk space if you aren’t going to work on the project for a while.

Parsing Command-Line Arguments Using clap

To learn how to use clap to parse the arguments, I need to read the documentation. I like to use Docs.rs, “an open source documentation host for crates of the Rust Programming Language.” I can follow examples from the documentation that show how to create a new App struct. Change your src/main.rs to the following:

use clap::App; 

fn main() {
    let _matches = App::new("echor") 
        .version("0.1.0") 
        .author("Ken Youens-Clark <[email protected]>") 
        .about("Rust echo") 
        .get_matches(); 
}

Import the clap::App struct.

Create a new App with the name echor.

Use semantic version numbers as described earlier.

Your name and email address so people know where to send the money.

This is a short description of the program.

Ask the App to parse the arguments.

With this code in place, I can run the program with the -h or --help flags to get a usage document. Note that I didn’t have to define this argument as clap did this for me:

$ cargo run -- -h
echor 0.1.0 
Ken Youens-Clark <[email protected]> 
Rust echo 

USAGE:
    echor

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

The app name and version number appear here.

Here is the author information.

This is the about text.

In addition to the help flags, I see that clap also automatically handles the flags -V and --version to print the program’s version:

$ cargo run -- --version
echor 0.1.0

Next, I need to define the parameters which I can do by adding Arg structs to the App.

use clap::{App, Arg}; 

fn main() {
    let matches = App::new("echor")
        .version("0.1.0")
        .author("Ken Youens-Clark <[email protected]>")
        .about("Rust echo")
        .arg( 
            Arg::with_name("text")
                .value_name("TEXT")
                .help("Input text")
                .required(true)
                .min_values(1),
        )
        .arg( 
            Arg::with_name("omit_newline")
                .help("Do not print newline")
                .takes_value(false)
                .short("n"),
        )
        .get_matches(); 

    println!("{:#?}", matches); 
}

Import both the App and Arg structs from the clap crate.

Create a new Arg with the name text. This is a required positional argument that must appear at least once and can be repeated.

Create a new Arg with the name omit_newline. This is a flag that has only the short name -n and takes no value.

Parse the arguments and return the matching elements.

Pretty-print the arguments.

If you request the usage again, you will see the new parameters:

$ cargo run -- --help
echor 0.1.0
Ken Youens-Clark <[email protected]>
Rust echo

USAGE:
    echor [FLAGS] <TEXT>...

FLAGS:
    -h, --help       Prints help information
    -n               Do not print newline 
    -V, --version    Prints version information

ARGS:
    <TEXT>...    Input text 

The -n flag to omit the newline is optional.

The required input text is one or more positional arguments.

Run the program with some arguments and inspect the structure of the arguments:

$ cargo run -- -n Hello world
ArgMatches {
    args: {
        "text": MatchedArg {
            occurs: 2,
            indices: [
                2,
                3,
            ],
            vals: [
                "Hello",
                "world",
            ],
        },
        "omit_newline": MatchedArg {
            occurs: 1,
            indices: [
                1,
            ],
            vals: [],
        },
    },
    subcommand: None,
    usage: Some(
        "USAGE:
    echor [FLAGS] <TEXT>...",
    ),
}

If you run the program with no arguments, you will get an error indicating that you failed to provide the required arguments:

$ cargo run
error: The following required arguments were not provided:
    <TEXT>...

USAGE:
    echor [FLAGS] <TEXT>...

For more information try --help

This was an error, and so you can inspect the exit value to verify that it’s not 0:

$ echo $?
1

If you try to provide any argument that isn’t defined, it will trigger an error and a nonzero exit value:

$ cargo run -- -x
error: Found argument '-x' which wasn't expected, or isn't valid in this context

USAGE:
    echor [FLAGS] <TEXT>...

For more information try --help

There’s another subtle thing happening with the output that is not at all obvious. The usage and error messages are all appearing on STDERR (pronounced standard error), which is another channel of Unix output. To see this in the bash shell, I can redirect channel 1 (STDOUT) to a file called out and channel 2 (STDERR) to a file called err:

$ cargo run 1>out 2>err

You should see no results from that command because all the output was redirected to files. The out file should be empty because there was nothing printed to STDOUT, but the err file should contain the output from Cargo and the error messages from the program:

$ cat err
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/echor`
error: The following required arguments were not provided:
    <TEXT>...

USAGE:
    echor [FLAGS] <TEXT>...

For more information try --help

So you see that another hallmark of well-behaved systems programs is to print regular output to STDOUT and error messages to STDERR. Sometimes errors are severe enough that you should halt the program, but sometimes they should just be noted in the course of running. For instance, in Chapter 3 you will write a program that processes input files, some of which will intentionally not exist or will be unreadable. I will show you how to print warnings to STDERR about these files and skip to the next argument.

Creating the Program Output

My next step is to use the values provided by the user to create the program’s output. It’s common to copy the values out of the matches into variables. To start, I want to extract the text argument. Because this Arg was defined to accept one or more values, I can use either of these functions that return multiple values:

• ArgMatches::values_of: returns Option<Values>

• ArgMatches::values_of_lossy: returns Option<Vec<String>>

To decide which to use, I have to run down a few rabbit holes to understand the following concepts:

• Option: a value that is either None or Some(T) where T is any type like a string or an integer. In the case of ArgMatches::values_of_lossy, the type T will be a vector of strings.

• Values: An iterator for getting multiple values out of an argument.

• Vec: A vector, which is a contiguous growable array type.

• String: A string of characters.

Both of the functions ArgMatches::values_of and ArgMatches::values_of_lossy will return an Option of something. Since I ultimately want to print the strings, I will use ArgMatches::values_of_lossy function to get an Option<Vec<String>>. The Option::unwrap function will take the value out of Some(T) to get at the payload T. Because the text argument is required by clap, I know it will be impossible to have None; therefore, I can safely call Option::unwrap to get the Vec<String> value:

let text = matches.values_of_lossy("text").unwrap();

The omit_newline argument is a bit easier as it’s either present or not. The type of this value will be a bool or Boolean, which is either true or false:

let omit_newline = matches.is_present("omit_newline");

Finally, I want to print the values. Because text is a vector of strings, I can use Vec::join to join all the strings on a single space into a new string to print. Inside the echor program, clap will be creating the vector. To demonstrate how Vec::join works, I’ll show you how create a vector using the vec! macro:

let text = vec!["Hello", "world"];

Vec::join will insert the given string between all the elements of the vector to create a new string. I can use println! to print the new string to STDOUT followed by a newline:

println!("{}", text.join(" "));

It’s common practice in Rust documentation to present facts using assert! to say that something is true or assert_eq! to demonstrate that one thing is equivalent to another. In the following code, I can assert that the result of text.join(" ") is equal to the string "Hello world":

assert_eq!(text.join(" "), "Hello world");

When the -n flag is present, the output should omit the newline. I will instead use the print! macro which does not add a newline, and I will choose to add either a newline or the empty string depending on the value of omit_newline. Depending on your background, you might try to write something like this:

let ending = "
"; 
if omit_newline {
    ending = ""; // This will not work 
}
print!("{}{}", text.join(" "), ending); 

Assign a default value.

Change the value if the newline should be omitted.

Use print! which will not add a newline to the output.

If I try to run this code, Rust tells me that I cannot reassign the value of ending:

$ cargo run -- Hello world
error[E0384]: cannot assign twice to immutable variable `ending`
  --> src/main.rs:27:9
   |
25 |     let ending = "
";
   |         ------
   |         |
   |         first assignment to `ending`
   |         help: make this binding mutable: `mut ending`
26 |     if omit_newline {
27 |         ending = ""; // This will not work
   |         ^^^^^^^^^^^ cannot assign twice to immutable variable

Something that really sets Rust apart from other languages is that variables are immutable by default, meaning they can’t be altered from their initial value. I’m not allowed to reassign the value of the variable ending; however, the compiler tells me to add mut to make the variable mutable:

let mut ending = "
"; 
if omit_newline {
    ending = "";
}
print!("{}{}", text.join(" "), ending);

Add mut to make this a mutable value.

There’s a much better way to write this. In Rust, if is an expression and not a statement. An expression returns a value, but a statement does not. Here’s a more Rustic way to write this:

let ending = if omit_newline { "" } else { "
" };

Since I only use ending in one place, I don’t need to assign it to a variable. Here is how I would update the main function:

fn main() {
    let matches = ...; // Same as before
    let text = matches.values_of_lossy("text").unwrap();
    let omit_newline = matches.is_present("omit_newline");
    print!("{}{}", text.join(" "), if omit_newline { "" } else { "
" });
}

With these changes, the program appears to work correctly; however, I’m not willing to stake my reputation on this. I need to, as the Russian saying goes, “Доверяй, но проверяй.”¹ This requires that I write some tests to run my program with various inputs and verify that it produces the same output as the original echo program.

Creating the Test Output Files

I can now run cargo test to verify that I have a program that runs, validates user input, and prints usage. Next, I would like to ensure that the program creates the same output as echo. To start, I need to capture the output from the original echo for various inputs so that I can compare these to the output from my program. In the 02_echor directory of my GitHub repository, you’ll find a bash script called mk-outs.sh that I used to generate the output from echo for various arguments. You can see that, even with such a simple tool, there’s still a decent amount of cyclomatic complexity, which refers to the various ways all the parameters can be combined. I need to check one or more text arguments both with and without the newline option:

$ cat mk-outs.sh
#!/usr/bin/env bash 

OUTDIR="tests/expected" 
[[ ! -d "$OUTDIR" ]] && mkdir -p "$OUTDIR" 

echo "Hello there" > $OUTDIR/hello1.txt 
echo "Hello"  "there" > $OUTDIR/hello2.txt 
echo -n "Hello  there" > $OUTDIR/hello1.n.txt 
echo -n "Hello" "there" > $OUTDIR/hello2.n.txt 

The “shebang” line tells the operating system to use the environment to execute bash for the following code.

Define a variable for the output directory.

Test if the output directory does not exist and create it if needed.

One argument with two words.

Two arguments separated by more than one space.

One argument with two spaces and no newline.

Two arguments with no newline.

If you are working on a Unix platform, you can copy this program to your project directory and run it like so:

$ bash mk-outs.sh

It’s also possible to execute the program directly, but you may need to execute chmod +x mk-outs.sh if you get a permission denied error:

$ ./mk-outs.sh

If this worked, you should now have a tests/expected directory with the following contents:

$ tree tests
tests
├── cli.rs
└── expected
    ├── hello1.n.txt
    ├── hello1.txt
    ├── hello2.n.txt
    └── hello2.txt

1 directory, 5 files

If you are working on a Windows platform, then I recommend you copy this directory structure into your project. Now you should have some test files to use in comparing the output from your program.

Comparing Program Output

The first output file was generated with the input Hello there as a single string, and the output was captured into the file tests/expected/hello1.txt. For my next test, I will run echor with this argument and compare the output to the contents of that file. Be sure add use std::fs to tests/cli.rs bring in the standard file system module, and then replace the runs function with this:

#[test]
fn hello1() {
    let outfile = "tests/expected/hello1.txt"; 
    let expected = fs::read_to_string(outfile).unwrap(); 
    let mut cmd = Command::cargo_bin("echor").unwrap(); 
    cmd.arg("Hello there").assert().success().stdout(expected); 
}

This is the output from echo generated by mk-outs.sh.

Use fs::read_to_string to read the contents of the file. This returns a Result that might contain a string if all goes well. Use the Result::unwrap method with the assumption that this will work.

Create a Command to run echor in the current crate.

Run the program with the given argument and assert it finishes successfully and that STDOUT is the expected value.

If you run cargo test, you might see output like this:

running 2 tests
test hello1 ... ok
test dies_no_args ... ok

Using the Result Type

I’ve been using the Result::unwrap method in such a way that assumes each fallible call will succeed. For example, in the hello1 function, I assumed that the output file exists and can be opened and read into a string. During my limited testing, this may be the case, but it’s dangerous to make such assumptions. I’d rather be more cautious, so I’m going to create a type alias called TestResult. This will be a specific type of Result that is either an Ok which always contains the unit type or some value that implements the std::error::Error trait:

type TestResult = Result<(), Box<dyn std::error::Error>>;

In the preceding code, Box indicates that the error will live inside a kind of pointer where the memory is dynamically allocated on the heap rather than the stack, and dyn “is used to highlight that calls to methods on the associated Trait are dynamically dispatched.” That’s really a lot of information, and I don’t blame you if your eyes glazed over. In short, I’m saying that the Ok part of TestResult will only ever hold the unit type, and the Err part can hold anything that implements the std::error::Error trait. These concepts are more thoroughly explained in Programming Rust (O’Reilly, 2021).

This changes my test code in some subtle ways. All the functions now indicate that they return a TestResult. Previously I used Result::unwrap to unpack Ok values and panic in the event of an Err, causing the test to fail. In the following code, I replace unwrap with the ? operator to either unpack an Ok value or propagate the Err value to the return type. That is, this will cause the function to return the Err variant of Option to the caller, which will in turn cause the test to fail. If all the code in a test function runs successfully, I return an Ok containing the unit type to indicate the test passes. Note that while Rust does have return to return a value early from a function, the idiom is to omit the semicolon from the last expression to implicitly return that result. Update your tests/cli.rs to this:

use assert_cmd::Command;
use predicates::prelude::*;
use std::fs;

type TestResult = Result<(), Box<dyn std::error::Error>>;

#[test]
fn dies_no_args() -> TestResult {
    let mut cmd = Command::cargo_bin("echor")?; 
    cmd.assert()
        .failure()
        .stderr(predicate::str::contains("USAGE"));
    Ok(()) 
}

#[test]
fn hello1() -> TestResult {
    let expected = fs::read_to_string("tests/expected/hello1.txt")?;
    let mut cmd = Command::cargo_bin("echor")?;
    cmd.arg("Hello there").assert().success().stdout(expected);
    Ok(())
}

Use ? instead of Result::unwrap to unpack an Ok value or propagate an Err.

Omit the final semicolon to return this value.

The next test passes two arguments, Hello and there, and expects the program to print Hello there.

#[test]
fn hello2() -> TestResult {
    let expected = fs::read_to_string("tests/expected/hello2.txt")?;
    let mut cmd = Command::cargo_bin("echor")?;
    cmd.args(vec!["Hello", "there"]) 
        .assert()
        .success()
        .stdout(expected);
    Ok(())
}

Use the Command::args method to pass a vector of arguments rather than a single string value.

I have a total of four files to check, so it behooves me to write a helper function. I’ll call it run and will pass it the argument strings along with the expected output file. Rather than use a vector for the arguments, I’m going to use a std::slice because I don’t need to grow the argument list after I’ve defined it:

fn run(args: &[&str], expected_file: &str) -> TestResult { 
    let expected = fs::read_to_string(expected_file)?; 
    Command::cargo_bin("echor")? 
        .args(args)
        .assert()
        .success()
        .stdout(expected);
    Ok(()) 
}

The args will be a slice of &str values, and the expected_file will be a &str. The return value is a TestResult.

Try to read the contents of the expected_file into a string.

Attempt to run echor in the current crate with the given arguments and assert that STDOUT is the expected value.

If all the previous code worked, return Ok containing the unit type.

Below is how I can use the helper function to run all four tests. Replace the earlier hello1 and hello2 definitions with these:

#[test]
fn hello1() -> TestResult {
    run(&["Hello there"], "tests/expected/hello1.txt") 
}

#[test]
fn hello2() -> TestResult {
    run(&["Hello", "there"], "tests/expected/hello2.txt") 
}

#[test]
fn hello1_no_newline() -> TestResult {
    run(&["Hello  there", "-n"], "tests/expected/hello1.n.txt") 
}

#[test]
fn hello2_no_newline() -> TestResult {
    run(&["-n", "Hello", "there"], "tests/expected/hello2.n.txt") 
}

A single string value as input.

Two strings as input.

A single string value as input with -n flag to omit the newline. Note that there are two spaces between the words.

Two strings as input with -n flag appearing first.

As you can see, I can write as many functions as I like in tests/cli.rs. Only those marked with #[test] are run when testing. If you run cargo test now, you should see five passing tests (in no particular order):

running 5 tests
test dies_no_args ... ok
test hello1 ... ok
test hello1_no_newline ... ok
test hello2_no_newline ... ok
test hello2 ... ok