Scikit-learn's neural network application is especially interesting for parameter tuning purposes. Unfortunately, its capabilities for unsupervised neural network applications are limited. For the next subject, where we dive into more sophisticated deep learning methods, we need another library. In this chapter, we will focus on theanets. We love theanets because of its ease of use and stability; it's a very smooth and well-maintained package developed by Lief Johnson at the University of Texas. Setting up a neural network architecture works quite similarly to sklearn; namely, we instantiate a learning objective (classification or regression), specify the layers, and train it. For more information, you can visit http://theanets.readthedocs.org/en/stable/.
All you have to do is install theanets with pip:
$ pip install theanets
As theanets is built on top of Theano, you also need to have the Theano properly installed. Let's run a basic neural network model to see how theanets works. The resemblance with Scikit-learn will be obvious. Note that we use momentum for this example, and softmax is used by default in theanets so we don't have to specify it:
import climate # This package provides the reporting of iterations
from sklearn.metrics import confusion_matrix
import numpy as np
from sklearn import datasets
from sklearn.cross_validation import train_test_split
from sklearn.metrics import mean_squared_error
import theanets
import theano
import numpy as np
import matplotlib.pyplot as plt
import climate
from sklearn.cross_validation import train_test_split
import theanets
from sklearn.metrics import confusion_matrix
from sklearn import preprocessing
from sklearn.metrics import accuracy_score
from sklearn import datasets
climate.enable_default_logging()
digits = datasets.load_digits()
digits = datasets.load_digits()
X = np.asarray(digits.data, 'float32')
Y = digits.target
Y=np.array(Y, dtype=np.int32)
#X = (X - np.min(X, 0)) / (np.max(X, 0) + 0.0001) # 0-1 scaling
X_train, X_test, y_train, y_test = train_test_split(X, Y,
test_size=0.2,
random_state=0)
# Build a classifier model with 64 inputs, 1 hidden layer with 100 units and 10 outputs.
net = theanets.Classifier([64,100,10])
# Train the model using Resilient backpropagation and momentum.
net.train([X_train,y_train], algo='sgd', learning_rate=.001, momentum=0.9,patience=0,
validate_every=N,
min_improvement=0.8)
# Show confusion matrices on the training/validation splits.
print(confusion_matrix(y_test, net.predict(X_test)))
print (accuracy_score(y_test, net.predict(X_test)))
OUTPUT ]
[[27 0 0 0 0 0 0 0 0 0]
[ 0 32 0 0 0 1 0 0 0 2]
[ 0 1 34 0 0 0 0 1 0 0]
[ 0 0 0 29 0 0 0 0 0 0]
[ 0 0 0 0 29 0 0 1 0 0]
[ 0 0 0 0 0 38 0 0 0 2]
[ 0 1 0 0 0 0 43 0 0 0]
[ 0 0 0 0 1 0 0 38 0 0]
[ 0 2 1 0 0 0 0 0 36 0]
[ 0 0 0 0 0 1 0 0 0 40]]
0.961111111111