Example 4-3 is a revision of Example 4-1 that uses a mutex to serialize access to race_list.
Note
To save space, the functions race_ioctl, race_new, race_find, and race_destroy aren’t listed here, as they haven’t been changed.
Example 4-3. race_mtx.c
#include <sys/param.h>
#include <sys/module.h>
#include <sys/kernel.h>
#include <sys/systm.h>
#include <sys/conf.h>
#include <sys/uio.h>
#include <sys/malloc.h>
#include <sys/ioccom.h>
#include <sys/queue.h>
#include <sys/lock.h>
#include <sys/mutex.h>
#include "race_ioctl.h"
MALLOC_DEFINE(M_RACE, "race", "race object");
struct race_softc {
LIST_ENTRY(race_softc) list;
int unit;
};
static LIST_HEAD(, race_softc) race_list = LIST_HEAD_INITIALIZER(&race_list);
static struct mtx race_mtx;
static struct race_softc * race_new(void);
static struct race_softc * race_find(int unit);
static void race_destroy(struct race_softc *sc);
static d_ioctl_t race_ioctl_mtx;
static d_ioctl_t race_ioctl;
static struct cdevsw race_cdevsw = {
.d_version = D_VERSION,
.d_ioctl = race_ioctl_mtx,
.d_name = RACE_NAME
};
static struct cdev *race_dev;
static int
race_ioctl_mtx(struct cdev *dev, u_long cmd, caddr_t data, int fflag,
struct thread *td)
{
int error;
mtx_lock(&race_mtx);
error = 
race_ioctl(dev, cmd, data, fflag, td);
mtx_unlock(&race_mtx);
return (error);
}
static int
race_ioctl(struct cdev *dev, u_long cmd, caddr_t data, int fflag,
struct thread *td)
{
...
}
static struct race_softc *
race_new(void)
{
...
}
static struct race_softc *
race_find(int unit)
{
...
}
static void
race_destroy(struct race_softc *sc)
{
...
}
static int
race_modevent(module_t mod __unused, int event, void *arg __unused)
{
int error = 0;
struct race_softc *sc, *sc_temp;
switch (event) {
case MOD_LOAD:
mtx_init(&race_mtx, "race config lock", NULL, 
MTX_DEF);
race_dev = make_dev(&race_cdevsw, 0, UID_ROOT, GID_WHEEL,
0600, RACE_NAME);
uprintf("Race driver loaded.
");
break;
case MOD_UNLOAD:
destroy_dev(race_dev);
mtx_lock(&race_mtx);
if (!LIST_EMPTY(&race_list)) {
LIST_FOREACH_SAFE(sc, &race_list, list, sc_temp) {
LIST_REMOVE(sc, list);
free(sc, M_RACE);
}
}
mtx_unlock(&race_mtx);
mtx_destroy(&race_mtx);
uprintf("Race driver unloaded.
");
break;
case MOD_QUIESCE:
mtx_lock(&race_mtx);
if (!LIST_EMPTY(&race_list))
error = EBUSY;
mtx_unlock(&race_mtx);
break;
default:
error = EOPNOTSUPP;
break;
}
return (error);
}
DEV_MODULE(race, race_modevent, NULL);This driver declares a mutex named race_mtx, which gets initialized as a sleep mutex in the module event handler.
Note
As you’ll see, a mutex is not the ideal solution for Example 4-1. However, for now, I just want to cover how to use mutexes.
In Example 4-1, the main source of concurrent access to race_list is the race_ioctl function. This should be obvious, because race_ioctl manages race_list.
Example 4-3 remedies the race conditions caused by race_ioctl by serializing its execution via the race_ioctl_mtx function. race_ioctl_mtx is defined as the d_ioctl function. It begins by acquiring race_mtx. Then race_ioctl is called and subsequently race_mtx is released.
As you can see, it takes just three lines (or one mutex) to serialize the execution of race_ioctl.
The race_modevent function is the module event handler for Example 4-3. Here is its function definition (again):
static int
race_modevent(module_t mod __unused, int event, void *arg __unused)
{
int error = 0;
struct race_softc *sc, *sc_temp;
switch (event) {
case MOD_LOAD:
mtx_init(&race_mtx, "race config lock", NULL, MTX_DEF);
race_dev = make_dev(&race_cdevsw, 0, UID_ROOT, GID_WHEEL,
0600, RACE_NAME);
uprintf("Race driver loaded.
");
break;
case MOD_UNLOAD:
destroy_dev(race_dev);
mtx_lock(&race_mtx);
if (!LIST_EMPTY(&race_list)) {
LIST_FOREACH_SAFE(sc, &race_list, list, sc_temp) {
LIST_REMOVE(sc, list);
free(sc, M_RACE);
}
}
mtx_unlock(&race_mtx);
mtx_destroy(&race_mtx);
uprintf("Race driver unloaded.
");
break;
case MOD_QUIESCE:
mtx_lock(&race_mtx);
if (!LIST_EMPTY(&race_list))
error = EBUSY;
mtx_unlock(&race_mtx);
break;
default:
error = EOPNOTSUPP;
break;
}
return (error);
}On module load, this function initializes race_mtx as a sleep mutex. Then it creates Example 4-3’s device node: race.
On MOD_QUIESCE, this function acquires race_mtx, confirms that race_list is empty, and then releases race_mtx.
On module unload, this function first calls destroy_dev to destroy the race device node.
Note
The destroy_dev function does not return until every d_foo function currently executing completes. Consequently, one should not hold a lock while calling destroy_dev; otherwise, you could deadlock your driver or panic your system.
Next, race_modevent confirms that race_list is still empty. See, after the execution of MOD_QUIESCE, a race_softc structure could have been added to race_list. So, race_list is checked again and every race_softc structure found is released. Once this is done, race_mtx is destroyed.
As you can see, every time race_list was accessed, mtx_lock(&race_mtx) was called first. This was necessary in order to serialize access to race_list throughout Example 4-3.
Now that we’ve examined Example 4-3, let’s give it a try:
$sudo kldload ./race_mtx.koRace driver loaded. $sudo ./race_config -a & sudo ./race_config -a &[1] 923 [2] 924 $ unit: 0 unit: 1 ... $sudo kldload ./race_mtx.koRace driver loaded. $sudo ./race_config -a & sudo kldunload race_mtx.ko &[1] 933 [2] 934 $ Race driver unloaded. race_config: open(/dev/race): No such file or directory [1]- Exit 1 sudo ./race_config -a [2]+ Done sudo kldunload race_mtx.ko
Unsurprisingly, it works. Yet using a mutex has introduced a new problem. See, the function definition for race_new contains this line:
sc = (struct race_softc *)malloc(sizeof(struct race_softc), M_RACE,
M_WAITOK | M_ZERO);Here, M_WAITOK means that it’s okay to sleep. But it’s never okay to sleep while holding a mutex. Recall that sleeping while holding a mutex causes the kernel to panic.
There are two solutions to this problem: First, change M_WAITOK to M_NOWAIT. Second, use a lock that can be held while sleeping. As the first solution changes the functionality of Example 4-1 (that is, currently, race_new never fails), let’s go with the second.