Panorama is an interesting subject in application development. In OpenCV, the stitching module can easily create a panorama image from a sequence of images. One benefit of the stitching module is that the sequence of images don't have to be in order and can be in any direction. However, in the OpenCV Android SDK, the stitching module does not exist. Therefore, we must use the stitching module in the C++ interface. Luckily, Android provides the Native Development Kit (NDK) to support native development in C/C++. In this chapter, we will guide you through the steps to capture camera frames from Java and process the data in OpenCV C++ with the NDK.
In this chapter, you will learn:
- How to make a complete panorama stitching application
- How to use Java Native Interface (JNI) to use OpenCV C++ in Android Studio
- How to use the stitching module to create a panorama image
A panorama image gives the viewer a much broader field of view than a normal image and allows them to fully experience a scene. By extending the range of panorama to 360 degrees, viewers can simulate turning their head around. A panorama image can be created by stitching a sequence of overlapping images.
The following figure shows a demonstration of a panorama image captured with our application.
A panorama image captured in an horizontal direction
In order to capture a panorama image, you must capture many images of the scene at different angles, as in the following figure. For example, you take your first picture at the left side of the room. Then, you move the phone straight to a new angle to start capturing. All the images will be stitched together to create a panorama image.
Illustration showing how you pan the phone to create a panorama image
Normally, a panorama application only supports capturing images in horizontal. With the stitching module in OpenCV, we can extend the height of an image by capturing more images in both directions. The following figure shows an image that can be captured by changing the camera view in a horizontal and vertical direction.
A panorama image captured in both directions
In this chapter, we will implement a panorama application in Android using OpenCV 3.0.0. The chapter contains two main sections:
- Android section: We will implement the user interface in Android Studio. In this chapter, we only implement the panorama capture activity with two buttons, Capture and Save. When the panorama is captured, we will save it to the phone's internal storage.
- OpenCV section: We will show the process to integrate OpenCV to Android Studio with NDK/JNI and implement the code to create a panorama image from a sequence of images captured in the Android section.
In the following sections, we will show the process to create a user interface in Android Studio. If you want to review the OpenCV code, you can go to the Integrating OpenCV into the Android Studio section and come back to this later.