We will now construct our very own DNN for a real-life problem: classification of flower types based on the measurements of petals. We will be working with the well-known Iris dataset for this. This dataset is stored as a comma-separated value (CSV) text file, with each line containing four different numerical values (petal measurements), followed by the flower type (here, there are three classes—Irissetosa, Irisversicolor, and Irisvirginica). We will now design a small DNN that will classify the type of iris, based on this set.

We will start by processing this file into appropriate data arrays that we can use for training and validating our DNN. Let's start by opening up our main function; we will need to translate the names of the flowers into actual classes that a DNN can output, so let's make a small dictionary that will give us a corresponding label for each class. We will also set up some empty lists to store our training data and labels:

if __name__ == '__main__':
 to_class = { 'Iris-setosa' : [1,0,0] , 'Iris-versicolor' : [0,1,0], 'Iris-virginica' : [0,0,1]}
 
 iris_data = []
 iris_labels = []

Now, let's read from the CSV file. We will use the reader function from Python's csv module, which we imported earlier:

with open('C:/Users/btuom/examples/9/iris.data', 'rb') as csvfile:
 csvreader = csv.reader(csvfile, delimiter=',')
 for row in csvreader:
  newrow = []
  if len(row) != 5:
   break
  for i in range(4):
   newrow.append(row[i])
  iris_data.append(newrow)
  iris_labels.append(to_class[row[4]])

We will now randomly shuffle the data and use two-third of these samples as training data. The remaining one-third will be used for test (validation) data:

iris_len = len(iris_data)
shuffled_index = list(range(iris_len))
np.random.shuffle(shuffled_index)
  
iris_data = np.float32(iris_data)
iris_labels = np.float32(iris_labels)
iris_data = iris_data[shuffled_index, :]
iris_labels = iris_labels[shuffled_index,:]
 
t_len = (2*iris_len) // 3
 
iris_train = iris_data[:t_len, :]
label_train = iris_labels[:t_len, :]
 
iris_test = iris_data[t_len:,:]
label_test = iris_labels[t_len:, :]

Now, finally, we can begin building our DNN! First, let's create a SequentialNetwork object. We'll set the max_batch_size to 32:

sn = SequentialNetwork( max_batch_size=32 )

Now, let's create our NN. This will consist of four dense layers (two hidden) and a softmax layer. We will increment the number of neurons in each layer until the final layer, which will only have three outputs (one for each class). This increasing amount of neurons per layer allows us to capture some of the subtleties of the data:

sn.add_layer({'type' : 'dense', 'num_inputs' : 4, 'num_outputs' : 10, 'relu': True, 'sigmoid': False, 'weights' : None, 'bias' : None} ) 
sn.add_layer({'type' : 'dense', 'num_inputs' : 10, 'num_outputs' : 15, 'relu': True, 'sigmoid': False, 'weights': None, 'bias' : None} ) 
sn.add_layer({'type' : 'dense', 'num_inputs' : 15, 'num_outputs' : 20, 'relu': True, 'sigmoid': False, 'weights': None, 'bias' : None} ) 
sn.add_layer({'type' : 'dense', 'num_inputs' : 20, 'num_outputs' : 3, 'relu': True, 'sigmoid': False, 'weights': None , 'bias': None } ) 
sn.add_layer({'type' : 'softmax'})

We will now condition our training data and begin the training with our BSGD method that we just implemented. We will train with batch_size set to 16, max_streams set to 10, the number of epochs set to 100, the delta set to 0.0001, and the training_rate set to 1—these will be admissible parameters for virtually any modern GPU. We will also time the training procedure while we're at it, which can be rather time-consuming:

ctrain, means, stds = condition_data(iris_train)

t1 = time()
sn.bsgd(training=ctrain, labels=label_train, batch_size=16, max_streams=20, epochs=100 , delta=0.0001, training_rate=1)
training_time = time() - t1

Now, our DNN is fully trained. We are ready to begin the validation process! Let's set up a Python variable called hits to count the total number of correct classifications. We will also need to condition the validation/testing data too. One more thing—we determine the class by the index corresponding to the largest value of the softmax layer of our DNN. We can check whether this gives us the correct classification by using NumPy's argmax function, like so:

hits = 0
ctest, _, _ = condition_data(iris_test, means=means, stds=stds)
for i in range(ctest.shape[0]):
 if np.argmax(sn.predict(ctest[i,:])) == np.argmax( label_test[i,:]):
  hits += 1

Now, we are ready to check how well our DNN actually works. Let's output the accuracy as well as the total training time:

print 'Percentage Correct Classifications: %s' % (float(hits ) / ctest.shape[0])
print 'Total Training Time: %s' % training_time

Now, we are done. We can now fully implement a DNN with Python and CUDA! Generally speaking, you can expect an accuracy ranging from 80%-97% for this particular problem, with a training time of 10-20 minutes on any Pascal-level GPU.