In order to apply our classifier to traffic sign recognition, we need a suitable dataset. A good choice might be the German Traffic Sign Recognition Benchmark (GTSRB), which contains more than 50,000 images of traffic signs belonging to more than 40 classes. This is a challenging dataset that was used by professionals in a classification challenge during the International Joint Conference on Neural Networks (IJCNN) 2011. The dataset can be freely obtained from http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset.
The GTSRB dataset is perfect for our purposes because it is large, organized, open source, and annotated. However, for the purpose of this book, we will limit the classification to data samples from a total of 10 classes.
Although the actual traffic sign is not necessarily a square, or centered, in each image, the dataset comes with an annotation file that specifies the bounding boxes for each sign.
A good idea before doing any sort of machine learning is usually to get a feeling of the dataset, its qualities, and its challenges. If all the exemplars of the dataset are stored in a list, X, then we can plot a few exemplars with the following script, where we pick a fixed number (sample_size) of random indices (sample_idx) and display each exemplar (X[sample_idx[sp-1]]) in a separate subplot:
sample_size = 15
sample_idx = np.random.randint(len(X), size=sample_size)
sp = 1
for r in xrange(3):
for c in xrange(5):
ax = plt.subplot(3,5,sp)
sample = X[sample_idx[sp-1]]
ax.imshow(sample.reshape((32,32)), cmap=cm.Greys_r)
ax.axis('off')
sp += 1
plt.show()The following screenshot shows some examples of this dataset:
Even from this small data sample, it is immediately clear that this is a challenging dataset for any sort of classifier. The appearances of the signs change drastically based on viewing angle (orientation), viewing distance (blurriness), and lighting conditions (shadows and bright spots). For some of these signs, such as the rightmost sign in the second row, it is difficult even for humans (at least for me) to tell the correct class label right away. Good thing we are aspiring experts of machine learning!
Luckily, the chosen dataset comes with a script for parsing the files (more information can be found at http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset#Codesnippets).
We spruce it up a bit and adjust it for our purposes. In particular, we want a function that not only loads the dataset, but also extracts a certain feature of interest (via the feature input argument), crops the sample to the hand-labeled Region of Interest (ROI) containing solely the sample (cut_roi), and automatically splits the data into a training and a test set (test_split). We also allow the specification of a random seed number (seed), and plot some samples for visual inspection (plot_samples):
import cv2
import numpy as np
import csv
from matplotlib import cm
from matplotlib import pyplot as plt
def load_data(rootpath="datasets", feature="hog", cut_roi=True,
test_split=0.2, plot_samples=False, seed=113):Although the full dataset contains more than 50,000 examples belonging to 43 classes, for the purpose of this chapter, we will limit ourselves to 10 classes. For easy access, we will hardcode the class labels to use here, but it is straightforward to include more classes (note that you will have to download the entire dataset for this):
classes = np.array([0, 4, 8, 12, 16, 20, 24, 28, 32, 36])
We then need to iterate over all the classes so as to read all the training samples (to be stored in X) and their corresponding class labels (to be stored in labels). Every class has a CSV file containing all of the annotation information for every sample in the class, which we will parse with csv.reader:
X = [] # images
labels = [] # corresponding labels
# subdirectory for class
for c in xrange(len(classes)):
prefix = rootpath + '/' + format(classes[c], '05d') + '/'
# annotations file
gt_file = open(prefix + 'GT-'+ format(classes[c], '05d')+ '.csv')
gt_reader = csv.reader(gt_file, delimiter=';')Every line of the file contains the annotation for one data sample. We skip the first line (the header) and extract the sample's filename (row[0]) so that we can read in the image:
gt_reader.next() # skip header
# loop over all images in current annotations file
for row in gt_reader:
# first column is filename
im = cv2.imread(prefix + row[0])Occasionally, the object in these samples is not perfectly cut out but is embedded in its surroundings. If the cut_roi input argument is set, we will ignore the background and cut out the object using the bounding boxes specified in the annotation file:
if cut_roi:
im = im[np.int(row[4]):np.int(row[6]),
np.int(row[3]):np.int(row[5]), :]Then, we are ready to append the image (im) and its class label (c) to our list of samples (X) and class labels (labels):
X.append(im)
labels.append(c)
gt_file.close()Often, it is desirable to perform some form of feature extraction, because raw image data is rarely the best description of the data. We will defer this job to another function, which we will discuss in detail in the next section:
if feature is not None:
X = _extract_feature(X, feature)As pointed out in the previous subsection, it is imperative to keep the samples that we use to train our classifier, separate from the samples that we use to test it. For this, we shuffle the data and split it into two separate sets such that the training set contains a fraction (1-test_split) of all samples and the rest of the samples belong to the test set:
np.random.seed(seed) np.random.shuffle(X) np.random.seed(seed) np.random.shuffle(labels) X_train = X[:int(len(X)*(1-test_split))] y_train = labels[:int(len(X)*(1-test_split))] X_test = X[int(len(X)*(1-test_split)):] y_test = labels[int(len(X)*(1-test_split)):]
Finally, we can return the extracted data:
return (X_train, y_train), (X_test, y_test)