Project Setup with a Leiningen Template

The project template this time is named clj-opencv and is called with Leiningen using the following one-liner on the terminal or console:

This will download the new template and create a myfirstcljcv folder with approximately the following content:

The project.clj file, as you remember, holds almost all of the project metadata. This time we will use a version that is slightly updated from what you have seen in Chapter 1.

The main differences from the previous chapter are highlighted in the following, so let’s review it quickly.

As expected, the origami library has been added as a dependency in the dependencies section.

A plug-in named gorilla has also been added . This will help you run python’s notebook style code; we will cover that later on in this recipe.

The injections segment may be a bit obscure at first, but it mostly says that the loading of the native OpenCV library will be done on starting the environment, so you do not have to repeat it in all the examples, as was the problem in the first chapter.

Everything Is OK

The main namespace to run is opencv3.ok; let’s run it right now to make sure the setup is ready. This has not changed from the first chapter, and you still use the same command on the terminal or console to load code with:

After a short bit of output, you should be able to see something like

The file grey-neko.jpg would have been created in the project folder and be like the picture in Figure 2-1.

The code of the opencv3.ok namespace is written in full as follows:

You would recognize the imread, cvtColor, imwrite opencv functions used in the previous chapter, and indeed the java opencv functions are simply wrapped in Clojure.

This first code sequence flow written in the origami DSL is shown in Figure 2-2.

Webcam Check

If you have a webcam plugged in, there is another sample that starts the camera and stream in a video. The file to run this is in samplevideo.clj.

As before, you can start the sample by specifying the namespace to the lein run command.

When the command starts, you will be presented with a moving view of the coffee shop you are typing those few lines of code in, just as in Figure 2-3.

While this was just to run the examples included with the project template , you can already start writing your own experimental code in your own files and run them using the lein run command.

The Auto Plug-in Strikes Back

You will see soon why this is usually not the best way to work with origami, because this recompiles all your source files each time . This is however a technique that can be used to check that all your code compiles and runs without errors.

So here is a quick reminder on how to set up the auto plug-in solution presented in Chapter 1 for Java, Scala, and Kotlin, this time for Clojure/Origami code.

Modify the project.clj file to add the lein-auto plug-in so it matches the following code:

This is not in the project template by default because it’s probably not needed most of the time.

Once you have added this, you can run the usual auto command by prefixing the command you want to execute with auto. Here:

This will execute the main namespace and wait for the file change to compile and execute again.

And so, after modifying the main method of the ok.clj file like in the following:

You can see a new file jet-neko.jpg created and a new fun-looking cat, as in Figure 2-4.

Now while this setup with the auto plug-in is perfectly ok, let’s see how to minimize latency between your code typing and the processing output, by using a Clojure REPL .

At the REPL

We have just reviewed how to run samples and write some Origami code in a fashion similar to the setup with Java, Scala, and Kotlin, and saw again how to include and use the auto plug-in.

Better than that, Clojure comes with a Read-Eval-Print-Loop (REPL) environment, meaning you can type in lines of code, like commands, one by one, and get them executed instantly.

To start the Clojure REPL, Leiningen has a subcommand named repl, which can be started with

After a few startup lines are printed on the terminal/console:

You will then be greeted with the REPL prompt:

opencv3.ok is the main namespace of the project, and you can type in code at the prompt just like you were typing code in the opencv3/ok.clj file. For example, let’s check whether the underlying OpenCV library is loaded properly by printing its version:

The library is indeed loaded properly, and native binding is found via Leiningen’s magic.

Let’s use it right now for a kick-start. The following two lines get some functions from the utils namespace, mainly to open a frame, and then load an image and open it into that frame:

The cute cat from Figure 2-5 should now be showing up on your computer as well.

Origami encourages the notion of pipelines for image manipulation. So, to read an image, convert the color of the loaded image, and show the resulting image in a frame, you would usually pipe all the function calls one after the other, using the Clojure threading macro ->, just like in the following one-liner:

Which now converts the minicat.jpg from Figure 2-5 to its gray version as in Figure 2-6.

-> does nothing more than reorganize code so that the first invocation result goes to the input of the next line and so on. This makes for very swift and compact image-processing code.

Note that the lines execute directly, so you don’t have to wait for file changes or anything and can just get the result onscreen as you press the Enter key.

REPL from Atom

The REPL started by Leiningen is quite nice, with a bunch of other features you can discover through the documentation, but it’s hard to compete with the autocompletion provided by a standard text editor.

Using all the same project metadata from the project.clj file, the Atom editor can actually provide, via a plug-in, instant and visual completion choices.

to get the same setup on your atom editor, as shown in Figure 2-7.

The same Leiningen-based REPL can be started either by the atom menu as in Figure 2-8 or by the equivalent key shortcut.

When starting the REPL, a window named Proto-REPL opens on the right-hand side of the Atom editor. This is exactly the same REPL that you have used when executing the lein repl command directly from the terminal. So, you can type in code there too.

But the real gem of this setup is to have autocompletion and choice presented to you when typing code, as in Figure 2-9.

You can now retype the code to read and convert the color of an image directly in a file , let’s say ok.clj. Your setup should now be similar to that shown in Figure 2-10.

Once you have typed the code in, you can select code and execute the selected lines of code by using Ctrl-Alt+s (on Mac, Command-Ctrl+s).

You can also execute the code block before the cursor by using Ctrl-Alt+b (on Mac, Command-Ctrl+b) and get your shot of instant gratification.

After code evaluation, and a slight tab arrangement, you can have instant code writing on the left-hand side, and the image transformation feedback on the right-hand side, just as in Figure 2-11.

The jet-set cat is now showing in the output.jpg file, and can be updated by updating and executing code in the opened editor tab.

For example, see by yourself what happens when adding the resize! function call in the processing flow, as in the following code.

Nice. A newly resized jet-set cat is now instantly showing on your screen .

Gorilla Notebook

To complete this recipe, let’s present how to use gorilla from within an Origami project.

Gorilla is a Leiningen plug-in, where you can write and run notebooks, à la python’s jupyter.

This means you can write code alongside documentation, and even better, you can also share those notes to the outside world.

How does that work? Gorilla takes your project setup and uses it to execute the code in a background REPL. Hence, it will find the origami/opencv setup taken from the project.clj file .

It will also start a web server whose goal is to serve notes or worksheets. Worksheets are pages where you can write lines of code and execute them.

You can also write documentation in the sheet itself in the form of markdown markup, which renders to HTML.

As a result, each of the notes, or worksheets, ends up being effectively a miniblog.

The project.clj file that comes with the clj-opencv template defines a convenient leiningen alias to start gorilla via the notebook alias:

This effectively tells leiningen to convert the notebook subcommand to the following gorilla command:

Let’s try it, by using the following command on a console or terminal:

After a few seconds, the Gorilla REPL is started. You can access it already at the following location:

http://localhost:10000/worksheet.html?filename=notes/practice.clj

You will be presented with a worksheet like in Figure 2-12.

In a gorilla notebook , every block of the page is either Clojure code or markdown text. You can turn the currently highlighted block to text mode by using Alt+g, Alt+m (or Ctrl+g, Ctrl+m on Mac) where m is for markdown, as in Figure 2-13.

You can also turn back the highlighted block into code mode by using Alt+g, Alt+j (or Ctrl+g, Ctrl+j on Mac), where j is for Clojure, as in Figure 2-14.

To execute the highlighted block of code , you would use Shift+Enter, and the block turns into executed mode, as in Figure 2-15.

What that does is read from the code block, send the input to the background REPL via a websocket, retrieve the result, and print it the underlying div of the code block.

Alright; so now you know all that is needed to start typing code in the gorilla REPL. Let’s try this out right now. In a new code block of the worksheet, try to type in the following Clojure code .

And now… Shift+Enter! This should bring you close to Figure 2-16 and a new shot of instant gratification .

Remember that all of this is happening in the browser , which has three direct positive consequences.

The first one is that remote people can actually view your worksheets, and they can provide documentation directly from their own machines by connecting to the URL directly.

Second, they can also execute code directly block by block to understand the flow.

Third, the saved format of the worksheets is such that they can be used as standard namespaces and can be used through normal code-writing workflow. Conversely, it also means that standard Clojure files can be opened, and documentation can be added via the Gorilla REPL.

From now on, we won’t impose using either the Gorilla REPL or the Atom environment, or even simply typing on the REPL. Effectively, these are three different views on the same project setup.

Simply remember for now that to show a picture, the function to use is slightly different depending on whether you are in the Gorilla REPL or in a standard REPL.

In the Gorilla REPL:

In the standard REPL:

In atom , you would save the file:

OK, this time you are really all set! Time for some computer vision basics.

Creating a Mat

Remember that you need a height , a width, and a number of channels to create a mat. This is done using the new-mat function. The following snippet creates a 30×30 Mat, with one channel per pixel, each value being an integer.

If you try to display the content of the mat, either with u/mat-view (gorilla repl) or u/show (standard repl), then the memory assigned to the mat is actually left as is. See Figure 2-17.

Let’s assign a color, the same to every pixel of the Mat. This is either done when creating the Mat, or can be done with set-to, which is a call to the .setTo Java function of OpenCV.

Every pixel in the mat now has value 105 assigned to it (Figure 2-18).

To understand most of the underlying matrix concepts of OpenCV , it is usually a good idea for you to check the values of the underlying mat using .dump or simply dump.

This will be done a few times in this chapter. To use it, simply call dump on the mat you want to see the internals from.

And the expected output is shown in the following, with the mat points all set to the value of 128.

.dump calls the original OpenCV function and will print all the row and column pixel values in one string.

Creating a Colored Mat

With one channel per pixel, you can only specify the white intensity of each pixel, and thus, you can only create gray mats .

To create a colored mat, you need three channels, and by default, each channel’s value representing the intensity of red, blue, and green.

To create a 30×30 red mat, the following snippet will create an empty three-channel mat with each point in the mat set to the RGB value of [255 0 0] (yes, this is inverted, so be careful):

In a similar way, to create a blue or green mat:

If you execute all this in the gorilla REPL, each of the mats shows up, as in Figure 2-19.

Using a Submat

You will remember that we have seen how to use a submat in Chapter 1; let’s review how to use those submats using origami.

Here, we first create an RGB mat with three channels per pixel, and set all the pixels to a cyan color.

A submat can then be created using, well, the submat function and a rectangle to define the size of the submat.

This gives the following code snippet:

The resulting main mat, with yellow inside where the submat was defined, and the rest of the mat in cyan color, is shown in Figure 2-20.

Just for the kicks at this stage, see what a one-liner of origami code can do, by using hconcat!, a function that concatenates multiple mats together, and clojure.core/repeat, which creates a sequence of the same item.

The resulting pattern is shown in Figure 2-21.

At this point, you can already figure out some creative generative patterns by yourself.

Setting One Pixel Color

Setting all the colors of a mat was done using set-to. Setting one pixel to a color is done using the Java method put. The put function takes a position in the mat, and a byte array representing the RGB values of that pixel.

So, if you want to create a 3×3 mat with all its pixels to yellow, you would use the following code snippet.

Unfortunately, the 3×3 mat is a bit too small for this book, so you should type in the code yourself.

The dump function works nicely here though, and you can see the content of the yellow mat in the following:

Typing all this line by line is a bit tiring though, so this where you use Clojure code to loop over the pixel as needed.

A call to Clojure core doseq gets convenient to reduce the boilerplate.

The preceding simple doseq snippet simply loops over all the pixels of a 3×3 mat.

So, to have a bit more fun, let’s display some random red variants for each pixel of a 100×100 colored mat. This would be pretty tiresome by hand, so let’s use the doseq sequence here too.

Figure 2-22 gives one version of the executed snippet.

Piping Process and Some Generative Art

You can already see how Origami makes it quite simple and fun to integrate generative work with OpenCV mats.

This short section will also be a quick introduction to the piping process that is encouraged by Origami.

Clojure has two main constructs (called macros) named -> and ->>. They pipe results throughout consecutive function calls.

The result of the first function call is passed as a parameter to the second function, and then the result of that call to the second function is passed on the third one, and so on.

The first macro, ->, passes the result as the first parameter to the next function call.

The second macro, ->>, passes the result as the last parameter to the next function call.

For example, creating a random gray mat could be done this way:

You could do the same with a randomly colored mat as well. This time, the rand function is called three times (Figure 2-24).

Or, with the same result, but using a few more Clojure core functions:

You can see now how you could also build on those mini code flows to build different generative variations of mat. You can also combine those mats, of course, using hconcat! or vconcat! functions of OpenCV.

The following new snippet generates a sequence of 25 elements using range and then creates gray mats in the range of 0–255 by scaling the range values (Figure 2-25).

 (->> (range 25)

      (map #(new-mat 30 30 CV_8UC1 (new-scalar (double (* % 10)))))

      (hconcat!)

      (u/mat-view))

You can also smooth things up by generating a range of 255 values, and making the created mat slightly smaller, with each mat of size 2×10 (Figure 2-26).

(->> (range 255)

     (map #(new-mat 20 2 CV_8UC1 (new-scalar (double %))))

     (hconcat!)

     (u/mat-view))

Loading

imread works the exact same as its opencv equivalent; this mostly means that you simply give it a path from the filesystem, and the file is read and converted to a ready-to-be-used Mat object.

In its simplest form, loading an image can be done as in the following short code snippet:

The file path, resources/kitten.jpg, is relative to the project, or can also be a full path on the file system.

The resulting loaded Mat object is shown in Figure 2-27.

When loading an image, you can refer to Table 1-3 to specify the option used to load the image, such as grayscale, and resize at the same time.

To load in grayscale and resize the image to a quarter of its size, you could use the following snippet, written using the pipeline style you have just seen.

It loads the same picture, but the mat looks different this time , as its color has been converted, like in Figure 2-28.

Saving

The imwrite function from Origami takes from opencv’s imwrite, but reverses the order of the parameters to make the function easy to use in processing pipes.

For example, to write the previously loaded gray cat to a new file, you would use

A new file, grey-neko.png , will be created from the loaded mat object (Figure 2-29).

You can observe that the resulting file image has actually been converted from jpg to png for you, just by specifying it as the extension in the file name.

The reason that the parameter order has been changed is that, in this case, you can save images from within the pipeline code flow.

See in the following how the image is saved during the flow of transformation.

The mat will be saved in the file image grey-neko.png, and the processing will go on to the next step, here mat-view.

Showing

Origami comes with a quick way of previewing images, and streams in the form of the imshow function, from the opencv3.utils namespace.

The imshow function takes a mat as the parameter and opens a Java frame with the mat inside, as shown in Figure 2-30.

This is not all; you can pass a map when using imshow to define various settings from the background color of the frame to its size and so forth. Also, a handlers section can be added to the map, where you can define your own key shortcuts.

See an example of the configuration map for the following frame.

In the handlers section, each entry of the map is made of an ASCII key code and a function. The function takes a mat and has to return a mat. Here, you can suppose gamma! is a function changing brightness on a mat, depending on a brightness parameter.

Figure 2-31 shows the mat after pressing u.

Figure 2-32 shows the mat after pressing v.

This is not the most important section of this book, but the quick frame becomes quite handy when playing with the video streams later on in Chapter 4.

Loading from URL

While it is usually the case that the picture can be accessed from the filesystem the code is running on, many times there is a need to process a picture that is remotely hosted.

Origami provides a basic mat-from-url function that takes a URL and turns it into an OpenCV mat.

The standard way to do this in origami is shown in the following snippet:

And the resulting image is shown in Figure 2-33.

This was the only way to load a picture until recently. But then, most of the time, you would be doing something like

to resize the picture right after loading it. Now, u/mat-from-url also accepts imread parameters. So, to load the remote picture in gray, and reduce its size altogether, you can directly pass in the IMREAD_* parameter. Note that this has the side effect of creating a temporary file on the filesystem.

The same remote picture is now both smaller and loaded in black and white, as shown in Figure 2-34.

Simple Colors

Colors from the origami packages need to be required, and so when you use them, you need to update your namespace declaration at the top of the notebook.

With the namespace rgb, you can create scalars for RGB values instead of guessing them.

So, if you want to use a red color , you can get your environment to help you find and autocomplete the scalar you are looking for, as shown in Figure 2-35.

And so, using this in action, you can indeed use the following snippet to create a 20×20 mat of a red color.

Note that since rgb/red-2 is a scalar, you can dump the values for each channel by just printing it:

This is pretty nice to find color codes quickly.

The opencv3.colors.html namespace was created so that you could also use the traditional hexadecimal notation used in css. For a nice light green with a bit of blue, you could use this:

In full sample mode, and using threading ->>, this gives

which creates a small mat of a light green/blue color (Figure 2-36).

Printing the color itself gives you the assigned RGB values:

And you can indeed check that the colors match by creating the RGB scalar yourself.

This will give you a mat with the exact same RGB-based color .

Color Maps

Color maps can be understood by a simple color change, using a simple filter, which results in something similar to your favorite smartphone photo application.

There are a few default maps that can be used with OpenCV; let’s try one of them, say COLORMAP_AUTUMN, which turns the mat into a quite autumnal red.

To apply the map to a Mat, for example the cat from Figure 2-37, simply use the apply-color-map! function .

The following snippet shows how to make use of the usual imread and the apply-color-map sequentially.

The resulting cat is shown in Figure 2-38.

You can also define your own color space conversion . This is done by a matrix multiplication, which sounds geeky, but is actually simpler than it sounds.

We will take the example of rgb/yellow-2. You may not remember, so if you print it, you’ll find out that this is actually coded as, no blue, some green, and some red, which translated into RGB gives the following: [0 238 238].

Then, we define a transformation matrix made of three columns and three rows; since we are working with RGB mats, we will do this in three-channel mode .

What does this matrix do? Remember that we want to apply a color transformation for each pixel, meaning in output we want a set of RGB values for each pixel.

And so, since our Mat is all yellow, we have the following input:

And the output of each pixel is such as follows:

Or, since 255 is the maximum value for a channel:

Now in origami code, this gives the following:

Here, the mat content is shown with dump:

Then:

And the result of the transformation is shown in the following, as expected consists of a matrix of [0 119 255] values.

Make sure you execute the statements one by one to see the different RGB values in the output, along with the colored mats.

You may look around in the literature, but a nice sepia transformation would use the following matrix:

With the resulting sepia cat in Figure 2-39.

Time to go out and make your own filters!

We have seen how transform is applied to each pixel in RGB. Later on, when switching to other colorspaces, you can also remember that even though the values won’t be red, blue, green anymore, this transform! function can still be used in the same way.

Color Space

You have been working almost uniquely in the RGB color space up to now, which is the simplest one to use. In most computing cases, RGB is not the most efficient, so many other color spaces have been created in the past and are available for use. With Origami, to switch from one to the other, you usually use the function cvt-color!

What does a color space switch do?

It basically means that the three-channel values for each pixel have different meanings.

For example, red in RGB can be encoded in RGB as 0 0 238 (and its graphical representation is shown in Figure 2-40):

(-> (new-mat 1 1 CV_8UC3 rgb/red-2)

    (.dump))

; "[  0,   0, 238]"

(-> (new-mat 30 30 CV_8UC3 rgb/red-2)

    (u/mat-view))

However, when you change the color space and convert it to another namespace, say HSV , Hue-Saturation-Value, the values of the matrix are changed.

And of course, the simple display of the mat content is not really relevant anymore; as shown in Figure 2-41, it turned to yellow!!

Changing color space does not mean changing anything to the colors of the mat, but changing the way those are represented internally.

Why would you want to change colorspace?

While each colorspace has its own advantages, color space HSV is widely used due to the fact that it is easy to use ranges to identify and find shapes of a given color in a mat.

In RGB, as you remember, each value of each channel represents the intensity of red, green, or blue.

In opencv cv terms, let’s say we want to see a linear progression of red; we can increase or decrease the value of the two other channels, green and blue.

That shows the line of Figure 2-42.

But what if in a picture, we are trying to look for orange-looking shapes? Hmm… How does that orange color look in RGB again?

Yes, it starts to get slightly difficult. Let’s take a different approach and look into the HSV color space.

In that case, let’s see what happens if we draw this ourselves, with what we have learned so far.

The function hsv-mat creates a mat from a hue value.

As you can read, the code switches the color space of the mat twice, once to set the color space to HSV and set the hue, and then back to RGB so we can draw it later with the usual function imshow or mat-view.

We have seen the hue ranges from 0 to 180 in OpenCV, so let’s do a range on it and create a concatenated mat of all the small mats with hconcat.

The drawn result is shown in Figure 2-43.

First, you may notice that toward the end of the bar, the color goes back to red again. It is often considered a cylinder for that reason.

The second thing you may notice is that it is easier to just tell which color you are looking for by providing a range. 20-25 is usually used for yellow, for example.

Because it can be annoying to select red in one range, you can sometimes use the reverse RGB during the color conversion: instead of using COLOR_BGR2HSV, you can try to use COLOR_RGB2HSV (Figure 2-44).

This makes it easier to select red colors, with a hue range between 105 and 150.

Let’s try that on a red cat . It is hard to find a red cat in nature, so we will use a picture instead.

The cat is loaded with the following snippet (Figure 2-45).

(-> "resources/redcat.jpg"

    (imread IMREAD_REDUCED_COLOR_2)

    (u/mat-view))

Then, we define a range of lower red and upper red. The remaining saturation and value are set to 30 30 (sometimes 50 50) and 255 255 (sometimes 250 250), so from very dark and grayed to full-blown hue color.

Now, we use the opencv in-range function, which we will see again later in recipe 2-7, to say we want to find colors in a specified range and store the result in a mask, which is initialized as an empty mat.

Et voila: the resulting mask mat is in Figure 2-46.

We will see that finding-color technique in more detail later, but now you see why you would want to switch color space from RGB to something that is easier to work with, here again HSV.

Flipping

Alright; this one is rather easy. You just need to call flip on the image with a parameter telling how you want the flip to be done.

Note here the first-time usage of clone in the image-processing flow.

While flip! does transformation in place, thus modifying the picture that it is passed, clone creates a new mat, so that the original neko is left untouched.

And the result is shown in Figure 2-48.

Most of the Origami functions work like this. The standard version, here flip, needs an input mat and an output mat, while flip! does the conversion in place and only needs an input/output mat. Also, while flip has no return value, flip! returns the output mat so it can be used in a pipeline.

Similarly, you have already seen cvt-color, and cvt-color!, or hconcat and hconcat!, and so on.

Let’s play a bit with Clojure and use a sequence to show all the possible flips on a mat.

This time, all the flips are showing (Figure 2-49).

Rotation

The function rotate! also takes a rotation parameter and turns the image according to it.

Note again the use of clone to create an intermediate mat in the processing flow, and the result in Figure 2-50.

Note also how clone and ->> can be used to create multiple mats from a single source.

In the final step, the multiple mats are concatenated using hconcat! (Figure 2-51) or vconcat! (Figure 2-52).

Thanks to the usage of clone , the original mat is left untouched and can still be used in other processing pipelines as if it had just been freshly loaded.

Warp

The last one, as promised, is the slightly more complicated version of rotating a picture using the opencv function warp- affine along with a rotation matrix.

In this first example, we keep the zoom factor to 1 and take a rotation angle of 45 degrees.

We also make the rotation point the center of the original mat.

matrix is also a 2×3 Mat, made of Float values, as you can see if you print it out. The rotation matrix can then be passed to the warp function. Warp also takes a size to create the resulting mat with the proper dimension.

And the 45-degrees-rotated cat is shown in Figure 2-53.

Let’s now push the fun a bit more with some autogeneration techniques . Let’s create a mat that is made of the concatenation of multiple mats of rotated cats, each cat rotated with a different rotation factor.

For this purpose, let’s create a function rotate-by! , which takes an image and an angle and applies the rotation internally, using get-rotation-matrix-2-d.

Then you can use that function in a small pipeline. The pipeline takes a range of rotations between 0 and 360, and applies each angle in sequence to the original neko mat.

And the fun concatenated mats are shown in Figure 2-54.

Furthermore, let’s enhance the rotate-by! function to also use an optional zoom parameter. If the zoom factor is not specified, its value defaults to 1.

The zoom parameter is then passed to the get-rotation-matrix-2-d function.

This time, the snippet simply does a range over seven random zoom values.

And the result is shown in Figure 2-55. Also note that when the zoom value is too small, default black borders can be seen in the resulting small mat.

In the same way, many other image transformations can be done with warp-affine, by passing matrixes created with a transformation matrix using get-affine-transform, get-perspective-transform, and so forth.

Most of the transformations take a source matrix of points and a target matrix of points, and each of the opencv get-** functions creates a transformation matrix to accordingly map from one set of points to the others.

When OpenCV requires a mat of “something,” you can use the origami constructors, matrix-to-matofxxx from the util package.

Applying the transformation is done in the same way with warp-affine.

Figure 2-56 shows the result of the affine transformation .

Manual Filter

The first example is a function that sets all but one of a channel’s values to 0, in a three-channel picture . That has the effect of completely changing the color of the mat.

Notice how the function internally creates a fully sequential byte array of all the bytes of the mat. 3 is used here because we are supposing that we are working with a mat made of three channels per pixel.

The mod if statement makes it so we set all values of that channel to 0 for all pixels in the mat.

We then use a new cat picture (Figure 2-57).

And simply put our function into action. The value 0 in the parameter means that all but the blue channel will be set to 0.

And yes, the resulting picture is overly blue (Figure 2-58).

Playing with Clojure code generative capability here again, we range over the channels to create a concatenated mat of all three mats (Figure 2-59).

(def source

    (imread "resources/emilie1.jpg"))

(->> (range 0 3)

     (map #(filter-buffer! (clone source) %))

     (hconcat!)

     (u/mat-view))

Multiply

It was nice to create a filter manually to see the details of how its filters are working, but actually, OpenCV has a function called multiply that does exactly all of this already for you.

The function takes a mat, created with origami’s matrix-to-mat-of-double, to apply a multiplication to the value of each channel in a pixel.

Putting this straight into action, we use the following short snippet to turn the white cat into a mellow blue picture (Figure 2-60).

(->

  "resources/emilie1.jpg"

  (imread)

  (multiply! (u/matrix-to-mat-of-double [ [1.0 0.5 0.0]] ))

  (u/mat-view))

Luminosity

Combined with what you have learned already in chapter 2 about changing the channels, you may remember that while RGB is great at changing the intensity of a specific color channel, changing the luminosity value can be easily done in the HSV color space.

Here again, we use the multiply function of OpenCV, but this time, the color space of the mat is changed to HSV ahead of the multiplication.

Note how the matrix used with multiply only applies a 1.5 factor to the third channel of each pixel, which in the HSV color space is indeed the luminosity . A bright result is shown in Figure 2-61.

Highlight

The preceding short snippet actually gives you a nice way of highlighting an element in a mat. Say you create a submat, or you have access to it through some finding shape algorithm; you can apply the luminosity effect to highlight only that part of the whole mat.

The resulting highlight mat is shown in Figure 2-62.

Filter 2d

filter-2-d , the new OpenCV function introduced here, also performs operations on bytes. But this time, it computes the value of each pixel of the target mat, depending on the value of the src pixel and the values of the surrounding pixel.

To understand how it is possible to do absolutely nothing, let’s take an example where the multiplication keeps the value of the pixel as is, by applying a filter that multiplies the value of current’s pixel by 1, and ignoring the values of its neighbors. For this effect, the 3×3 filter matrix has a value of 1 in the center (the target pixel) and 0 for all the other ones, the surrounding neighbor pixels.

This does nothing! Great. We all want more of that. The filter-2-d function call really just keeps the image as is, as shown in Figure 2-63.

Let’s get back to matrixes and raw pixel values to understand a bit more about how things work under the hood, with an example using a simple gray matrix.

The preceding snippet, as you know by now, creates a small 100×100 gray mat (Figure 2-64).

Now, we’ll focus on a portion of that gray mat using submat and apply the filter-2-d function only on the submat.

We take a 3×3 matrix for the operation and use a 0.3 value for the main center pixel. This means that when we apply the filter, the value of the corresponding pixel in the target matrix will be 200×0.25=50.

Here, that means the entire submat will be darker than the pixels not located in the submat, as confirmed in Figure 2-65.

And if you look at the pixel values themselves on a much smaller mat, you’ll see that the value of the center pixel (the submat) has been divided by exactly 4.

What else can you do with filter-2-d? It can be used for art effects as well; you can create your own filters with your custom values. So, go ahead and experiment.

The preceding filter turns the cat image into a mat ready to receive some brushes of watercolors (Figure 2-66).

Threshold

Threshold is another filtering technique that resets values in a mat to a default, when they are originally above or below a threshold.

Uh, what did you say?

To understand how that works, let’s go back to a small mat at the pixel level again, with a simple 3×3 mat.

Here is how this works.

And the resulting matrix is

As you can see, only pixels with values greater than 150 are left to nonzero values.

You can create the complementary matrix by using THRESH_BINARY_INV, as seen in the following.

Now applying this technique to a picture makes things quite interesting by leaving only the interesting shapes of the content of the mat.

Figure 2-67 shows the resulting mat after applying the threshold to my sister’s white cat.

For reference, and for the next chapter’s adventures, there is also another method named adaptive-threshold , which computes the target value depending on the values from the surrounding pixels.

Figure 2-68 shows the result of the adaptive threshold.

Adaptive threshold is usually used in recipe 2-8 with blurring techniques that we will study very shortly.

Simple Blur and Median Blur

The flow to apply a simple blur is relatively simple. Like many other image-processing techniques, we use a kernel, a square matrix with the main pixel in the center, like 3×3 or 5×5. The kernel is the matrix in which each pixel is given a coefficient.

In its simplest form, we just need to give it a kernel size for the area to consider for the blur: the bigger the kernel area, the more blurred the resulting picture will be.

Basically, each pixel of the output is the mean of its kernel neighbors.

The result can be seen in Figure 2-77.

And the bigger the kernel, the more blurred the picture will be.

Figure 2-78 shows the result of using different kernel sizes with the blur function.

(->> (range 3 10 2)

     (map #(-> neko  clone (u/resize-by 0.5) (blur! (new-size % %))))

     (hconcat!)

     (u/mat-view))

Gaussian Blur

This type of blur gives more weight to the center of the kernel. We will see that in the next chapter, but this type of blur is actually good at removing extra noise from pictures.

The result of the gaussian blur is shown in Figure 2-79.

Bilateral Filter

Those filters are used when you want to smooth the picture, but at the same time would also like to keep the edges.

What are edges? Edges are contours that define the shapes available in a picture.

The first example shows a simple usage of this bilateral filter.

(-> neko

    clone

    (bilateral-filter! 9 9 7)

    (u/mat-view))

This second example shows an example where we want to keep the edges. Edges can be easily found with the famous opencv function canny. We will spend some more time with canny in the next chapter.

For now, let’s focus on the output and lines of Figure 2-81.

(-> neko

    clone

    (cvt-color! COLOR_BGR2GRAY)

    (bilateral-filter! 9 9 7)

    (canny! 50.0 250.0 3 true)

    (bitwise-not!)

    (u/mat-view))

The third example quickly shows why you would want to use a bilateral filter instead of a simple blur. We keep the same small processing pipeline but this time use a simple blur instead of a bilateral filter.

The output clearly highlights the problem: defining lines have disappeared, and Figure 2-82 shows a disappearing cat …

Median Blur

Median blur is a friend of simple blur.

It is worth noting that at high kernel length, or a kernel length of greater than 21, we get something more artistic.

It is less useful for shape detection, as seen in Figures 2-83 and 2-84, but still combines with other mats for creative impact, as we will see in chapter 3.

Voila! Chapter 2 has been an introduction to Origami and its ease of use: the setup, the conciseness, the processing pipelines, and the various transformations.