In the following steps, you will learn to build a perceptron to separate a nonlinear data:

1. Since the perceptron is a linear classifier, you can imagine that it would have trouble trying to classify data that is not linearly separable. We can test this by increasing the spread (cluster_std) of the two blobs in our toy dataset so that the two blobs start overlapping:

In [12]: X, y = make_blobs(n_samples=100, centers=2,
...      cluster_std=5.2, random_state=42)
...      y = 2 * y - 1

2. We can plot the dataset again using matplotlib's scatter function:

In [13]: plt.scatter(X[:, 0], X[:, 1], s=100, c=y);
...      plt.xlabel('x1')
...      plt.ylabel('x2')

As is evident in the following screenshot, this data is no longer linearly separable because there is no straight line that perfectly separates the two blobs:

The preceding screenshot shows an example of data that is not linearly separable. So what would happen if we applied the perceptron classifier to this dataset?

3. We can find an answer to this question by repeating the preceding steps:

In [14]: p = Perceptron(lr=0.1, n_iter=10)
...      p.fit(X, y)

4. Then we find an accuracy score of 81%:

In [15]: accuracy_score(p.predict(X), y)
Out[15]: 0.81000000000000005

5. In order to find out which data points were misclassified, we can again visualize the decision landscape using our helper function:

In [16]: plot_decision_boundary(p, X, y)
...      plt.xlabel('x1')
...      plt.ylabel('x2')

The following graph makes the limitations of the perceptron classifier evident. Being a linear classifier, it tried to separate the data using a straight line but ultimately failed. The main reason it failed is because the data was not linearly separable even though we achieved 81% accuracy. However, from the following plot, it is clear that many of the red dots lie in the blue region and vice versa. So, unlike a perceptron, we need a nonlinear algorithm that can create not a straight but a nonlinear (circular) decision boundary:

Fortunately, there are ways to make the perceptron more powerful and ultimately create nonlinear decision boundaries.