Okay, by now we can be reasonably certain that hopper is fit for purpose, at least in terms of not crashing and producing the correct results. But, it also needs to be fast. To that end, hopper ships with the criterion (https://crates.io/crates/criterion) benchmarks. As the time of writing, criterion is a rapidly evolving library that performs statistical analysis on bench run results that Rust's built-in, nightly-only benchmarking library does not. Also, criterion is available for use on the stable channel. The target to match is standard library's MPSC, and that sets the baseline for hopper. To that end, the benchmark suite performs a comparison, living in benches/stdlib_comparison.rs in the hopper repository.

The preamble is typical:

#[macro_use]
extern crate criterion;
extern crate hopper;
extern crate tempdir;

use std::{mem, thread};
use criterion::{Bencher, Criterion};
use hopper::channel_with_explicit_capacity;
use std::sync::mpsc::channel;

Note that we've pulled in both MPSC and hopper. The function for MPSC that we'll be benching is:

fn mpsc_tst(input: MpscInput) -> () {
    let (tx, rx) = channel();

    let chunk_size = input.ith / input.total_senders;

    let mut snd_jh = Vec::new();
    for _ in 0..input.total_senders {
        let tx = tx.clone();
        let builder = thread::Builder::new();
        if let Ok(handler) = builder.spawn(move || {
            for i in 0..chunk_size {
                tx.send(i).unwrap();
            }
        }) {
            snd_jh.push(handler);
        }
    }

    let total_senders = snd_jh.len();
    let builder = thread::Builder::new();
    match builder.spawn(move || {
        let mut collected = 0;
        while collected < (chunk_size * total_senders) {
            let _ = rx.recv().unwrap();
            collected += 1;
        }
    }) {
        Ok(rcv_jh) => {
            for jh in snd_jh {
                jh.join().expect("snd join failed");
            }
            rcv_jh.join().expect("rcv join failed");
        }
        _ => {
            return;
        }
    }
}

Some sender threads get made, and a receiver exists and pulls the values from MPSC as rapidly as possible. This is not a logic check in any sense and the collected materials are immediately discarded. Like with the fuzz testing, the input to the function is structured data. MpscInput is defined as follows:

#[derive(Debug, Clone, Copy)]
struct MpscInput {
    total_senders: usize,
    ith: usize,
}

The hopper version of this function is a little longer, as there are more error states to cope with, but it's nothing we haven't seen before:

fn hopper_tst(input: HopperInput) -> () {
    let sz = mem::size_of::<u64>();
    let in_memory_bytes = sz * input.in_memory_total;
    let max_disk_bytes = sz * input.max_disk_total;
    if let Ok(dir) = tempdir::TempDir::new("hopper") {
        if let Ok((snd, mut rcv)) = channel_with_explicit_capacity(
            "tst",
            dir.path(),
            in_memory_bytes,
            max_disk_bytes,
            usize::max_value(),
        ) {
            let chunk_size = input.ith / input.total_senders;

            let mut snd_jh = Vec::new();
            for _ in 0..input.total_senders {
                let mut thr_snd = snd.clone();
                let builder = thread::Builder::new();
                if let Ok(handler) = builder.spawn(move || {
                    for i in 0..chunk_size {
                        let _ = thr_snd.send(i);
                    }
                }) {
                    snd_jh.push(handler);
                }
            }

            let total_senders = snd_jh.len();
            let builder = thread::Builder::new();
            match builder.spawn(move || {
                let mut collected = 0;
                let mut rcv_iter = rcv.iter();
                while collected < (chunk_size * total_senders) {
                    if rcv_iter.next().is_some() {
                        collected += 1;
                    }
                }
            }) {
                Ok(rcv_jh) => {
                    for jh in snd_jh {
                        jh.join().expect("snd join failed");
                    }
                    rcv_jh.join().expect("rcv join failed");
                }
                _ => {
                    return;
                }
            }
        }
    }
}

The same is true of HopperInput:

#[derive(Debug, Clone, Copy)]
struct HopperInput {
    in_memory_total: usize,
    max_disk_total: usize,
    total_senders: usize,
    ith: usize,
}

Criterion has many options for running benchmarks, but we've chosen here to run over inputs. Here's the setup for MPSC:

fn mpsc_benchmark(c: &mut Criterion) {
    c.bench_function_over_inputs(
        "mpsc_tst",
        |b: &mut Bencher, input: &MpscInput| b.iter(|| 
         mpsc_tst(*input)),
        vec![
            MpscInput {
                total_senders: 2 << 1,
                ith: 2 << 12,
            },
        ],
    );
}

To explain, we've got a function, mpsc_benchmark, that takes the mutable criterion structure, which is opaque to use but in which criterion will store run data. This structure exposes bench_function_over_inputs, which consumes a closure that we can thread our mpsc_test through. The sole input is listed in a vector. The following is a setup that does the same thing, but for hopper:

fn hopper_benchmark(c: &mut Criterion) {
    c.bench_function_over_inputs(
        "hopper_tst",
        |b: &mut Bencher, input: &HopperInput| b.iter(|| 
         hopper_tst(*input)),
        vec![
            // all in-memory
            HopperInput {
                in_memory_total: 2 << 11,
                max_disk_total: 2 << 14,
                total_senders: 2 << 1,
                ith: 2 << 11,
            },
            // swap to disk
            HopperInput {
                in_memory_total: 2 << 11,
                max_disk_total: 2 << 14,
                total_senders: 2 << 1,
                ith: 2 << 12,
            },
        ],
    );
}

Notice now that we have two inputs, one guaranteed to be all in-memory and the other guaranteed to require disk paging. The disk paging input is sized appropriately to match the MSPC run. There'd be no harm in doing an in-memory comparison for both hopper and MPSC, but your author has a preference for pessimistic benchmarks, being an optimistic sort. The final bits needed by criterion are more or less stable across all the benchmarks we'll see in the rest of this book:

criterion_group!{
    name = benches;
    config = Criterion::default().without_plots();
    targets = hopper_benchmark, mpsc_benchmark
}
criterion_main!(benches);

I encourage you to run the benchmarks yourself. We see times for hopper that are approximately three times faster for the systems we intended hopper for. That's more than fast enough.