However, corner detection is not sufficient when the scale of an image changes. Multiple papers have been published describing different algorithms for feature detection and description. We will look at a combination of the Speeded Up Robust Features (SURF) detector (for more information, see https://en.wikipedia.org/wiki/Speeded_up_robust_features) and the Binary Robust Independent Elementary Features (BRIEF) descriptor. The feature detector identifies the keypoints in the image and the feature descriptor calculates the actual feature value for all of the keypoints.
The details of these algorithms are beyond the scope of this book. Advanced users can refer to the papers describing these algorithms in detail.
For more details, you can refer to the following links:
- For SURF: https://www.vision.ee.ethz.ch/~surf/eccv06.pdf
- For BRIEF: https://www.cs.ubc.ca/~lowe/525/papers/calonder_eccv10.pdf
The entire process starts with reading the image, converting it into grayscale, using the star feature detector to find the interesting points, and finally, using the BRIEF descriptor to calculate the feature value.
- Let's first read the image and convert it to grayscale:
In [23]: img = cv2.imread('data/rubic-cube.png')
In [24]: gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
- Now, we will create the feature detector and descriptor:
In [25]: star = cv2.xfeatures2d.StarDetector_create()
In [26]: brief = cv2.xfeatures2d.BriefDescriptorExtractor_create()
- Next, it's time to use the star detector to get the keypoints and pass them to the BRIEF descriptor:
In [27]: keyPoints = star.detect(gray, None)
In [28]: keyPoints, descriptors = brief.compute(img, keyPoints)
There's a catch here. At the time of writing this book, the OpenCV version 4.0 didn't have the resolved version of the cv2.drawKeypoints function. So, I have written a similar function that we can use to draw the keypoints. You don't need to worry about the steps involved in the function—it's just for your reference. If you have installed the OpenCV version specified in this book (OpenCV 4.1.0 or OpenCV 4.1.1), you can use the cv2.drawKeypoints function directly:
In [29]: def drawKeypoint (img, keypoint, color):
... draw_shift_bits = 4
... draw_multiplier = 1 << draw_shift_bits
... center = (int(round(keypoint.pt[0])),int(round(keypoint.pt[1])))
... radius = int(round(keypoint.size/2.0))
... # draw the circles around keypoints with the keypoints size
... cv2.circle(img, center, radius, color, 1, cv2.LINE_AA)
... # draw orientation of the keypoint, if it is applicable
... if keypoint.angle != -1:
... srcAngleRad = keypoint.angle * np.pi/180.0
... orient = (int(round(np.cos(srcAngleRad)*radius)),
int(round(np.sin(srcAngleRad)*radius)))
... cv2.line(img, center, (center[0]+orient[0],
center[1]+orient[1]),
color, 1, cv2.LINE_AA)
... else:
... # draw center with R=1
... radius = 1 * draw_multiplier
... cv2.circle(img, center, radius,
color, 1, cv2.LINE_AA)
... return img
In [30]: from random import randint
... def drawKeypoints(image, keypoints):
... for keypoint in keypoints:
... color = (randint(0,256),randint(0,256),randint(0,256))
... image = drawKeypoint(image, keypoint, color)
... return image
- Let's now use this function to draw the detected keypoints:
In [31]: result = drawKeypoints(img, keyPoints)
In [32]: print("Number of keypoints = {}".format(len(keyPoints)))
Out[32]: Number of keypoints = 453
In [33]: plt.figure(figsize=(18,9))
... plt.imshow(result)
And we get the following output:
It's pretty awesome, right?
As easy and fast as BRIEF is, it doesn't work well with the rotation of an image. You can try it out by rotating the image (more information at https://www.pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/) and then running BRIEF. Let's see how ORB helps us to resolve this.