OpenCV is a simple and powerful programming framework for computer vision. It is preferred by both novices and experts in the field of computer vision. We can easily learn computer vision by writing OpenCV programs using Python 3 as the programming language. The Raspberry Pi family of single-board computers uses Python as its preferred development language. Using a Raspberry Pi board and Python 3 for learning OpenCV programming is one of the best approaches that we can follow to commence our wonderful journey into the amazing field of computer vision programming. In this chapter, you will become familiar with all of the important concepts that you need in order to get started with the Raspberry Pi and computer vision. By the end of this chapter, you will be able to set up the Raspbian Operating System (OS) on various Raspberry Pi board models. You will also learn how to connect the boards to the internet.
In this chapter, we will cover the following topics:
By the end of this chapter, you will be able to set up your own Raspberry Pi board.
The field of computer vision is a combination of different fields, including (but not limited to) computer science, mathematics, and electrical engineering. It includes ways to capture, process, and analyze images and videos from the real world in order to assist in decision making. Computer vision means mimicking biological (that is, human and non-human) vision. The end goal of most computer vision systems is to extract useful information from still images and videos (including prerecorded videos and live feeds) for the purpose of decision making. Biological vision systems work in a similar fashion. Additionally, unlike biological vision, computer vision can also acquire and work with images from the visual spectrum that are not visible to biological entities, for example, infrared and depth images.
Computer vision also relates to the area of extracting information from captured images and videos. A computer vision system may accept various types of data, such as images, videos, and live video streams, as inputs to further process, analyze, and extract meaningful information for the purpose of making important decisions.
The fields of artificial intelligence, machine vision, and computer vision overlap and share many topics, such as image processing, pattern recognition, and machine learning, as depicted in the following diagram:
In order to work as a researcher in the area of computer vision, you need to have a solid background and understanding of mathematics. However, to write programs for computer vision using OpenCV and Python 3, you don't need to know a lot of mathematics. Note that, in this book, you will be learning all of the mathematical and theoretical concepts required to get started with image processing and computer vision.
The typical objectives of a computer vision system could be one or more of the following:
Do not get stressed if you are unfamiliar with these key terms. We will explore and implement many of these concepts throughout our journey.
OpenCV (also known as Open Source Computer Vision) is an open source library for computer vision and machine learning. It has many functionalities for image processing and computer vision. It is a cross-platform library, and it works with many programming languages and OSes. It has a large collection of computer vision and machine learning-related functions. It also has several Graphical User Interface (GUI) and event handling features.
OpenCV is free for academic and commercial usage as it is under the Berkley Software Distribution (BSD) license. It is written with the C++ programming language. It has interfaces for most of the popular programming languages, including (but not limited to) C/C++, Python, and Java. It runs on a variety of OSes, including Windows, Android, Linux, macOS, and other Unix-like OSes. In this book, we will write computer vision-related programs with OpenCV and Python 3.
The library has more than 2,500 optimized algorithms for machine learning and computer vision tasks. It has a community of more than 47,000 computer vision professionals, and it has been downloaded more than 18 million times. OpenCV is extensively used in academics for teaching, research organizations, government organizations, and various industry segments. Reputed able organizations such as Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, and Toyota all use OpenCV.
Let's take a look at the history of OpenCV. OpenCV was originally an in-house initiative of Intel Research and was used to develop a framework to work with images and videos. It was initially supported by Willow Garage and then Itseez.
Note
You can visit the website of Willow Garage at http://www.willowgarage.com/.
In August 2012, the responsibility for further development and support for OpenCV was assumed by an independent, not-for-profit, organization, OpenCV.org. It maintains the website for OpenCV. In May 2016, Intel acquired Itseez. The following URLs have the press announcement from Intel and OpenCV.org:
Here's a brief timeline of the developments related to OpenCV:
You can find all the details, including different versions and press releases of the OpenCV library, at https://opencv.org/.
As we will be writing computer vision programs with Raspberry Pi as the platform, we will study single-board computers and Raspberry Pi in detail. We will learn how to set up the Raspbian OS on various models of a Raspberry Pi single-board computer.
A single-board computer (abbreviated to SBC) is a complete computer system on a single printed circuit board (abbreviated to PCB). The board usually has a processor(s), RAM, input/output (I/O), an Ethernet port for networking, and USB ports for interfacing with USB devices. A few single-board computers have Wi-Fi and Bluetooth, too. SBCs run OS distributions such as Ubuntu, Windows, Debian, and more. These OS distributions have specially tailored versions for use with these SBCs.
Unlike traditional computers, an SBC is not modular and its hardware cannot be upgraded because all the components (such as the CPU, RAM, GPU, and interfacing ports) are integrated on a single PCB itself. SBCs are used as low-cost computers in academia, research, and various other industries. The use of SBCs in embedded systems is quite widespread, and many individuals, research organizations, and companies have developed and released fully functional and usable products based on SBCs. Many of these products are crowdfunded. The main advantage of SBCs is onboard General-Purpose Input/Output (GPIO) pins. These pins provide functionalities such as various buses (Serial Peripheral Interface (SPI), I2C, and SMBus), digital I/O, analog input, and Pulse Width Modulation (PWM) output. Try not to get overwhelmed with all of this technical vocabulary. We will learn most of these concepts in more detail with the help of experiments. Almost all of the popular SBCs have GPIO in some form or other. Due to their small form factor and onboard GPIO, they are popular in schools, universities, training centers, boot camps, and maker spaces. They are frequently used in the areas of sensor networks and the internet of things (IoT).
To summarize, the advantages of SBCs are as follows:
However, SBCs come with their own set of disadvantages. As all the components of an SBC are on the same PCB, it can be very difficult to repair if a component is damaged due to mechanical or electrical reasons. For the same reason, we cannot even upgrade anything on an SBC. These are the only major disadvantages of SBCs.
The Microcomputer Trainer MMD-1, designed by John Titus in 1976, is the first true single-board microcomputer that was based on the Intel microprocessor, C8080A. It was called dyna-micro during the prototyping phase, and the production units were called MMD-1 (short for Mini-Micro Designer 1).
We are now going to take a look at the Raspberry Pi series in detail. However, before that, we will become acquainted with other popular SBC families.
The BeagleBoard.org Foundation is an organization based in the USA. It is a non-profit entity, and its objective is to provide education and collaboration around the design, development, testing, and use of open source hardware and software in the area of embedded systems. They have developed various SBCs named after beagles (a popular breed of domestic canine species). You can find a list of the current SBCs that they developed, which are in production, at http://beagleboard.org/boards. You can also find related products and accessories for Beagle boards at the same URL.
Their latest product, at the time of writing, is PacketBeagle (http://beagleboard.org/pocket).
The ASUS Tinkerboard is designed and manufactured by ASUS (a Taiwan-based multinational corporation). Its size, layout, and pins are compatible with second- and third-generation Raspberry Pi boards. You can find more details about all the editions of the ASUS Tinkerboard at https://www.asus.com/us/Single-Board-Computer/. The following photograph shows the top view of an ASUS Tinkerboard:
NVIDIA Jetson is a family of modules that is used for computer vision, AI, and speech processing tasks (https://developer.nvidia.com/embedded/develop/hardware). The best member for beginners to get started with is Jetson Nano. And the best place to begin is on the web page of Jetson Nano Developer Kit at https://developer.nvidia.com/embedded/jetson-nano-developer-kit. Here's a side view of the developer kit:
Intel Corporation also produces many boards that can be called SBCs. You can find details on the current generation of modules that are in production at https://software.intel.com/en-us/iot/hardware/all. We have had the privilege of working with several of the excellent Intel SBCs and modules. Many of them are discontinued, and you can find the full list and support documentation for them at https://software.intel.com/en-us/iot/hardware/discontinued. Note that you might be able to get a good deal on used and discontinued boards from Intel. They are also great for learning. For beginners of computer vision, I like to recommend Intel Up Squared Kit. You can find out more at https://software.intel.com/en-us/iot/hardware/up-squared-ai-vision-dev-kit.
Raspberry Pi is a series of low-cost and credit card-sized SBCs developed by the Raspberry Pi Foundation in the United Kingdom. The purpose of developing Raspberry Pi was to promote the teaching of basic computer skills and programming in schools, in which it has served very well. Raspberry Pi has expanded its footprint well beyond its intended purpose by gaining prominence in the embedded systems market and computer science research in academia and industrial applications.
The Raspberry Pi Foundation offers downloads for many popular OS distributions. We can use a variety of programming languages such as Python, C, C++, and Java with Raspberry Pi. You can find more information on the Raspberry Pi Foundation website (https://www.raspberrypi.org/).
The Raspberry Pi board comes in many models. Additionally, there are a lot of associated accessories with these models. You can find the current list of models under production on the products page of the Raspberry Pi Foundation (https://www.raspberrypi.org/products/). Unfortunately, the page does not have any information on the discontinued product boards of the Raspberry Pi family.
Additionally, Raspberry Pi is also available in a more flexible form that is intended for industrial and embedded applications. This is known as a compute module. The compute module also has many iterations. A compute module prototyping kit is also made available by the foundation. You can find out more about compute modules and the prototyping kit on the same Raspberry Pi products page we discussed earlier.
As we have discussed, there are many models of Raspberry Pi boards available. And while it is tempting to discuss the technical specifications in detail for all of those boards, it is difficult to achieve that in brief. In the first edition of the book, I discussed the specifications of all the available Raspberry Pi board models in detail, since the number of models was far lower and we could count them on our fingers. Since writing the second edition of this book, there are over a dozen Raspberry Pi models. Therefore, we will discuss the technical specifications of only a couple of board models of Raspberry Pi.
We will use the Raspberry Pi 4B 4 GB and Raspberry Pi Zero W with header models for our computer vision examples. However, these examples can also be run on the other board models of Raspberry Pi. This is because all the software that we use (the OS, the programming language, and the OpenCV library) is fully backward compatible.
You can find the product specifications of the Raspberry Pi 4B at https://www.raspberrypi.org/products/raspberry-pi-4-model-b/specifications/.
The following table explains the product specifications in detail:
The following diagram shows all of the important connectors and components on a Raspberry Pi board:
The following photograph shows the top view of the Raspberry Pi model 4B:
Here is a photograph of the model at an angle:
We are going to use the 4GB variant of this model.
You can find the specifications of the Raspberry Pi Zero W at https://www.raspberrypi.org/products/raspberry-pi-zero-w/.
The following table explains the specifications of this model in more detail:
You can discover where to buy Raspberry Pi boards and their accessories on the products page of the RPi website. Here's a screenshot:
You can also find Raspberry Pi boards and their accessories on Amazon. If you live in a big city, then you can find a lot of hobby electronics stores that sell Raspberry Pi boards and related items.
Many OSes have tailored distributions of OSes for Raspberry Pi boards. However, the early board models do not support all OSes. The latest model board, Raspberry Pi 4B, supports all the OSes mentioned at https://www.raspberrypi.org/downloads/.
The Raspbian OS supports all the models of the Raspberry Pi board, and it is the most recommended OS for beginners. We are going to demonstrate how to install this in the next section.
Raspbian is a free OS based on Debian, which is a popular distribution of Linux. Raspbian is optimized to Raspberry Pi hardware. You can find more information about the Raspbian project on its home page (http://raspbian.org/).
Note
Raspbian's home page mentions that it is not affiliated with the Raspberry Pi Foundation and is managed by fans of the Raspberry Pi and Debian projects.
The Raspbian web page provides a list of recommended Raspbian images at http://raspbian.org/RaspbianImages. An OS image is a file that can be written onto an SD card, and this SD card can then be used to boot the Raspberry Pi board. This is the easiest way of getting started with RPi; we will try to use it from now on. The image provided on the RPi Foundation's download page is the one most recommended by Raspbian. We will learn how to use this image to get started with RPi in the next section.
The setup is the one thing that usually deters many novice enthusiasts from getting started with SBCs. Many times, the instructions are very generic and do not cover all the cases for various types of hardware components. That is why I have dedicated an entire section to the setup of Raspbian on RPi. In this section, we will demonstrate the setup in detail with all the board models ever produced, with the exception of the compute modules.
We need the following components for the setup:
RPi 4B has a micro-HDMI output. Therefore, we need a micro-HDMI to HDMI converter. RPi Zero and RPi Zero W both have mini-HDMI outputs. So, for them, we need a mini-HDMI to HDMI converter. The following photograph shows the HDMI, mini-HDMI, and micro-HDMI ports, respectively:
We also need to plug the mini- and micro-HDMI ends to the RPi boards and the HDMI to the monitor. If you are planning to use a VGA monitor, then we will need HDMI/mini-HDMI/micro-HDMI to VGA converters depending on the board models.
Here is a photograph of an HDMI to VGA converter:
The following is a photograph of a mini-HDMI to VGA converter:
The following is a photograph of a micro-HDMI to VGA converter:
We need a Windows computer and a wired or wireless internet connection.
Finally, we require an SD card reader, as follows:
Many laptops have this (SD card reader) feature built-in. So, in that case, a separate reader is not required as we can use the built-in reader.
We will need a few more hardware components by the end of the chapter. We will discuss them when the need arises. For now, we are okay to proceed further.
To get started, we need to download all of the free software. Follow these instructions to download all the necessary software:
This will open the folder that has the ZIP file for the Raspbian OS image.
Double-click on the ZIP file and then click on the Extract button in the menu. This will extract the file. The extracted file has the img extension.
The best way of installing an OS on a microSD card is to do it manually. This allows us to prepare the SD card manually so that we have easier access to the /boot/config.txt configuration file, which must be modified, in a few cases, before booting up the RPi. We will discuss this in detail later. The default Raspbian image has only two partitions—boot and system. I recommend choosing, at a minimum, a 16 GB class 10 microSD card. Then, follow these steps:
This means that the image has been successfully written to the microSD card. Now we can use it to boot up the RPi.
a) Change #disable_overscan=1 to disable_overscan=1.
b) Change #hdmi_force_hotplug=1 to hdmi_force_hotplug=1.
c) Change #hdmi_group=1 to hdmi_group=2.
d) Change #hdmi_mode=1 to hdmi_mode=16.
e) Change #hdmi_drive=2 to hdmi_drive=2.
f) Change #config_hdmi_boost=4 to config_hdmi_boost=4.
g) Save the file.
The commented lines (that have # at the beginning) are disabled. We must enable these lines by uncommenting them. This can be done by removing # at the beginning of these commented lines.
Note
If you are using Linux or macOS, then you will find the instructions to install the Raspbian OS on your microSD card for these OSes at https://www.raspberrypi.org/documentation/installation/installing-images/.
Let's boot up our Pi for the first time with the microSD card using the following steps:
Now, our RPi board will start booting up. The green LED on the board will start blinking. Congratulations! The RPi board is booting for the first time.
Note
If your HDMI monitor is showing no signal, then power down the RPi and change #hdmi_force_hotplug=1 to hdmi_force_hotplug=1 in /boot/config.txt on the microSD card. Boot up the RPi with this changed setting and the HDMI monitor will show the signal.
Once the RPi boots up, the Raspbian desktop and a guided setup window appear, as follows:
Click on the Next button, and the following window will appear:
In the preceding window, set Country: and Language:. It will automatically select the time zone according to the country you selected. You can change that too if you wish. Click on the Next button, and the following window will appear:
You can choose to set a new password for the default pi user. If you leave it blank, then it will retain the default password. The following is the next window that appears:
Check the checkbox if there are black borders on the edges of the desktop view. The Raspbian OS will rectify it upon the next boot. The following window will appear after you click on the Next button, but only if the board model has Wi-Fi:
Choose the network that you know the credentials for, and click on the Next button. The following window will appear:
Key in your Wi-Fi password here, and click on the Next button. The following window will appear:
We can update the Raspbian OS and installed software here. We are going to learn how to do it manually in the latter part of this chapter. Click on the Skip or Next button, and the following window will appear:
We have finished most of the setup. Now, there are a few more things to do before we reboot out RPi, so click on the Later button.
Now, in the top-left corner of the desktop, you should see a Raspberry icon. It is the menu for Raspbian and functions in a similar way to the Windows logo on Microsoft Windows. Click on the logo and navigate to Preferences | Raspberry Pi Configuration:
This is the Raspberry Pi Configuration tool. It will open a window as follows, and we can change the settings of the Raspberry Pi board:
The preceding screenshot is the System tab. As of now, there is no need to change anything here. The following is the Interfaces tab:
Enable the camera, SSH, and VNC. The following is the Performance tab:
This menu has an option for overclocking and GPU memory. For the RPi 4B, overclocking is disabled. We will learn how to overclock an RPi 4B board manually in the next chapter. The Localisation tab is as follows:
You might want to change these settings as per your region of residence.
Once all these settings have been changed as per our choice, we can restart the RPi board by clicking on the Shutdown button in the Raspbian menu:
Here, we find the option to reboot the RPi. Once we reboot, and if we have chosen to retain the original password for the default user, pi, the following warning message window will appear when booting up:
And this will keep on appearing after every boot as long as we choose to retain the default password.
We can directly plug in the Ethernet cable to the RJ45 Ethernet port Pi boards. This will automatically detect the connection and connect to the internet.
Note
Make sure that DHCP (Dynamic Host Configuration Protocol) is enabled at the Wi-Fi router, the managed switch, or the internet gateway.
PRi 1 A, PRi 1 A+, RPi Zero, RPi Zero W, and RPi 3 A+ do not have Ethernet ports. However, RPi Zero W and RPi 3 A+ have built-in Wi-Fi. We can use a USB Wi-Fi dongle for the remaining models:
Plug this Wi-Fi adapter into the USB port. If the USB ports are not enough, then use a powered USB hub. For Raspberry Pi Zero, we need to use an additional USB OTG cable, as discussed earlier.
After plugging in the USB Wi-Fi adapter, we need to open lxterminal. This is the command-line utility. We can find it as a small black icon in Raspbian's taskbar and under Accessories in the Raspbian menu. Once we click on it, the following window will appear:
We can type in the Linux commands here. After typing them in, press Enter to execute the command. We have opened this so that we can manually configure the network interface of Raspbian. It is quite easy. All the network-related information is stored in the /etc/network/interfaces file. To connect to the Wi-Fi after plugging in the USB Wi-Fi dongle, we need to add a few entries to this file. First, take the backup of the original file by executing the following command:
mv /etc/network/interfaces /etc/network/interfaces.bkp
Then, we can create the interfaces file from scratch by running the following command:
sudo nano /etc/network/interfaces
The preceding command will open the network interface's file with a plain text editor known as nano. It is a simple WYSIWYG editor. Enter the following lines there:
source-directory /etc/network/interfaces.d
auto lo
iface lo inet loopback
auto wlan0
allow-hotplug wlan0
iface wlan0 inet dhcp
wpa-ssid "AshwinIon"
wpa-psk "internet1"
After entering the lines, press Ctrl + X and then press Y. In the preceding settings, substitute AshwinIon with your own SSID and internet1 with a password for the same. Then, run the following command in Command Prompt:
sudo service networking restart
This restarts the networking service and connects to the Wi-Fi. In any case (Ethernet or Wi-Fi), the RPi is assigned with a unique IP address. We can find it out by running the ifconfig command at lxterminal. The output of the command will have the Ipv4 address listed under inet.
Another way to know the IP address of the RPi is to check the active client tables in the router or the managed switch that the RPi board is connected to. The following is a screenshot of my router's active client table where we can see an entry for RPi:
Advanced Package Tool (APT) is a package management utility in Debian, Ubuntu, Raspbian, and their derivatives. APT is used to install, upgrade, and remove software. We will learn how to use it to update the OS and the software on the RPi board.
Run the following command:
sudo apt-get update
This command synchronizes the package list from the online source repository of software. Indexes of all the packages are refreshed. This updates all the repositories of the apps to all the latest update lists. This command must be executed before we execute the upgrade command.
Then, run the following command:
sudo apt-get dist-upgrade –fix-missing -y
This downloads and installs all the packages. It also removes obsolete packages. Depending on the speed of the internet, it takes some time. Finally, update the firmware by running the following command:
sudo rpi-update
This will update the firmware. Following this, the RPi board will be up to date in all aspects.
Finally, we can run the following command to shut down the RPi:
sudo shutdown -h now
And the following command reboots it:
sudo reboot
This will update the firmware. Following this, the RPi board will be up to date in all aspects.
In this chapter, we learned important terms such as computer vision, OpenCV, SBCs, and Raspberry Pi. We learned how to set up a Raspbian OS on Raspberry Pi and how to configure a Pi to access the internet. We also learned how to update a Pi.
With the completion of this chapter, you can go ahead and set up the Raspbian OS on your Raspberry Pi. Additionally, you can connect your RPi board to the internet using Wi-Fi or Ethernet. This will make you ready for the computer vision adventure that will soon follow.
In the next chapter, you will learn how to remotely access a RPi, how to overclock it, and the installation of OpenCV 4 for Python 3 on an RPi.
18.118.193.232