Changing Orientation and Pushing Notifications

Using the plyer.notification , we can push notifications to the notification bar of the Android device. This module has a function named notify that accepts a number of arguments to initialize the notification.

Listing 7-1 shows the KV file of an application with three button widgets. The first one executes the show_notification() function , which pushes the notification to the notification bar.

After deploying and running the application on an Android device and then pressing the first button, we will see the notification shown in Figure 7-1. The application icon is used as the icon of the notification. This is the least amount of code required to push a notification.

Controlling the Android Flashlight

After running the application, we will see the window shown in Figure 7-2. Remember to list CAMERA and FLASHLIGHT as items in the android.permissions property of the buildozer.spec file.

You can extend this application to create flash patterns that turn on and off the flashlight in a specified period of time.

The previous examples show how simple the Plyer library is for accessing Android features within Python code. Unfortunately, Plyer is restricted to some features and does not do everything an Android developer wants to do. There are some unsupported features, such as displaying a toast message, and there are some features that are not implemented for some platforms, such as accessing the camera and playing audio. We can use the Pyjnius library to access these missing features using Java code.

Playing Audio in Android Using Pyjnius

We can start by discussing the Java code used to play audio in Android and think about how to write it in Python using Pyjnius. The code is shown in Listing 7-8, in which there is a single class used named MediaPlayer in the android.media package. After instantiating that class and returning an instance into the mediaPlayer variable, we can call the required methods to load and play the audio file.

After completing the Java code, we next need to discuss the process for reflecting this code in Python.

This class can be reflected in Python using the autoclass() function discussed before. If the object created from that reflected class is named mediaPlayer, the methods will be called exactly as was done in the Java code.

Based on the code in Listing 7-9, we can create an application for playing, pausing, and stopping audio. The KV file of the application is shown in Listing 7-10. There is a BoxLayout with five widgets. The first two widgets are Labels and the last three are Buttons.

The first Label is updated according to the progress of playing the audio file. This is done by drawing a rectangle inside canvas.before using the Rectangle vertex instruction, which is colored red according to the Color instruction. The default size of this Label widget is 0.0, meaning it is hidden when the application starts. The size changes according to the playing progress.

The three buttons are responsible for starting, pausing, and stopping the audio. The callback functions start_audio(), pause_audio(), and stop_audio() are called when the on_release event is fired.

Figure 7-4 shows the window of the application after it runs on the Android device.

The Python file is shown in Listing 7-11. It implements the callback functions associated with the three Button widgets. The start_audio() function does the work discussed previously, from reflecting the MediaPlayer class until playing the audio file using the start() method. Notice that the instance of the MediaPlayer is set as a property inside the current object referenced using the self keyword in order to access it outside the function it is defined into. It can be accessed by self.mediaPlayer.

A class variable named prepare_audio is created to determine whether the audio file is previously loaded. It is initialized to False to mean that the file is not loaded. If its value is False, the start_audio() function loads the file and starts it. After being started, its value is set to True.

If the audio file is just paused, we do not have to reload the file again. Thus, the file will be started by calling the start() method inside the else part of the if statement. This resumes the audio file from the previous position before being paused. If the file is prepared after being paused, the file will play from the beginning.

Using the schedule_interval() function inside the kivy.clock.Clock class, a callback function named update_position() is executed every 0.1 seconds. This updates the two Label widgets inside the KV file according to the current millisecond being played from the audio file.

First, the audio file duration is returned using the getDuration() method inside the audioDuration variable. The current position, in milliseconds, is returned using the getCurrentPosition() method inside the currentPosition variable. The progress percentage is calculated by dividing the audioDuration by currentPosition. Because the values inside these variables are integers and the expected result is between 0.0 and 1.0, which is float, we have to change their data type from integer to float using the float() function. The file duration, current position, and current percentage are displayed on the second Label widget inside the KV file.

Note that displaying the progress of playing the audio file is similar to updating the life percentage of the character of the CoinTex game. After discussing the function that handles the on_release event of the first button, we can discuss the functions for the remaining two buttons.

The second Button widget pauses the played audio file using the pause() method . According to its callback function, pause_audio(), an if statement ensures that the file is playing before executing this method because it must be called for an active audio file.

The last button stops the audio by calling the stop() method . Similar to the pause_audio() callback function, the stop_audio() function has an if statement that ensures that the audio file is playing before stopping it. After calling this method, the file can be played again only after preparing the file again. Thus, we must execute the code that prepares the audio file inside the if statement of the start_audio() function.

Figure 7-5 shows the result after playing the audio file and pausing it at millisecond 5943, which corresponds to a percentage equal to 19.44% of the complete file duration. The first label width is 19.44% of the screen width.

The process requires more than reflecting the Toast class (android.widget.Toast) within Python. For example, we have to know that the text to be displayed is actually an instance of the CharSequence class . Thus, this class must be reflected in order to convert the text to its type to be suitable to the text argument. Moreover, toast is only created within the UI thread, not outside it. Thus, it can be displayed inside a Runnable instance. The developer has to do a lot of work than with the Java example.

Remember that the goal of using Pyjnius is to make life simpler for Kivy developers to build Android applications that use Java features in Python. If the process will get complicated, I do not recommend using it to write Java code. It can be used for simple tasks.

But if we cannot write Java code within Python, where do we write it? The answer is simply within a Java file.

Project Main Activity

Because the entry point for any Android application is the onCreate() method , we can discuss it. A few lines from the top of the PythonActivity class in addition to the method are shown in Listing 7-15. The first two lines import the classes named SDLActivity and ResourceManager.

We used to find a class named R.java in the Android projects to manage the resources by maintaining their IDs for example. For Kivy projects, this class is replaced by a class called ResourceManager, which is in the org.renpy.android package. This is why an instance is created from this class at the beginning of the onCreate() method .

A method named showLoadingScreen() inside the activity is called. This method just loads the presplash image inside an ImageView. Remember that the name of this image is specified inside the presplash.filename property of the buildozer.spec file. That method is shown in Listing 7-16.

The method works as follows. If the ImageView is not created inside the mImageView variable, it creates it by loading the presplash image resource. The resource identifier is returned in the presplashId variable using the getIdentifier() method inside the resourceManager instance. This method accepts the resource name and its kind and returns an integer representing the ID. This is why the presplashId variable type is integer. Note that in regular Android projects, the findViewById() method returns the IDs.

The presplash image is located in this path NewApp/src/main/res/drawable/presplash.jpg. Because the resource is an image, its kind is drawable and thus it’s added to the drawable directory. This directory is located under the res directory to indicate that it holds resources. The name of a resource is presplash, which is the resource filename without the extension.

After returning its ID, the raw image file is opened and returned to the is variable, which is an instance of the InputStream class. Then the raw data is decoded as an image using the decodeStream() method of the BitmapFactory class. The data is returned to a variable named bitmap.

After that, an instance of the ImageView class is returned to the mImageView variable and the image displayed on it is set using the setUmageBitmap() method . It accepts the bitmap variable we previously created.

In order to display the presplash image view, the image view must be added to the activity layout. In order to add a view to the layout, the addView() method is used. Using the command mLayout.addView(mImageView), the presplash image view is added to the activity layout referred to it by the mLayout variable .

After the screen finishes loading, the presplash image view is removed according to the method named removeLoadingScreen() inside the PythonActivity class. It will be discussed later in this chapter.

Note that if there is no layout created previously, the image itself is displayed on the screen as the main view using the setContentView() function.

The purpose of displaying the presplash image on the loading screen rather than starting the application directly is that files must be loaded before the application starts. Inside the project root directory, there is a folder named python-install that includes all the Python files needed to build the application.

After displaying the loading screen, the onCreate() method starts a background thread by creating an instance from the UnpackFilesTask class that extends AsyncTask. Note that a toast message is shown after running this thread while the loading screen is active.

The class header and some chunks of code from this class are shown in Listing 7-17. Note that this class is nested inside the PythonActivity class and thus no import statement is required.

Note that the goal of finishing the loading screen is not just starting the application. Another purpose is to make the UI layout managed by the SDLActivity, as it is the one responsible for preparing the application UI. The finishLoad() method prepares a layout inside the mLayout instance. We will use this method later for adding views to the layout of the main activity (PythonActivity).

Displaying Toast Messages from Python

After releasing the button, the toast is displayed according to Figure 7-6.

Adding Views to the PythonActivity Layout

By default, the presplash image is the only view displayed on the activity layout while the screen is loading. We can alter this layout to add more views. Just remember that there is only one layout for the activity and thus we have remove the newly added views once they are not required.

The place where we can add more views while the screen is loading is the showLoadingScreen() method of the PythonActivity class. We can add a single TextView within the activity layout according to this modified method shown in Listing 7-25. The TextView is added after the presplash image view that’s defined into the mImageView variable gets added to the layout. The code omits some parts and focuses on the part where the TextView is added to the layout.

An instance from the TextView class is returned into a static variable defined in the PythonActivity class named loadingTextView. The reason for not making this variable local to the method is that we need to access it later inside the removeLoadingScreen() method to remove it.

After we run the application, the text view will appear as shown in Figure 7-12.

Assuming that the text view is not removed from the layout, it will remain visible in the screen after the application starts, as shown in Figure 7-13.

In summary, a view is added to the activity layout that remains as long as the screen is loading. Once the loading process is over, the view is removed. If the view is not removed from the parent, it will still visible after the application UI defined in Python is loaded.

Assume we need to add a view to the activity layout after the loading step ends. Where in the Java code do we add this view? Let’s discuss this matter in the next section.

SDLSurface

In order to determine the suitable place within the project to add views after the files are loaded but before the application layout is visible, it is important to discuss the SDLSurface class.

The SurfaceView is an Android component in which we can draw on the application UI. The UI widgets specified in the Kivy project KV file are drawn within this SurfaceView.

SurfaceView implements an interface named SurfaceHolder.Callback, which provides a number of callback methods to help receive information about the surface. For example, when the surface is created, the callback method surfaceCreated() is called. The surfaceDestroyed() and surfaceChanged() callback methods are called when the surface is destroyed and changed, respectively. An example of changes that occur is creating an instance of this class and drawing something on the surface.

To handle hardware key presses, SurfaceView implements the second interface named View.OnKeyListener. It has a single callback method named onKey() that is called when a hardware key is pressed. To handle touch events, the View.OnTouchListener is implemented. It has a single callback method named onTouch().

The final interface implemented by the SurfaceView class is SensorEventListener. It listens to the changes in the sensor data. It has two callback methods—onAccuracyChanged() and onSensorChanged(). Both are implemented in the class but the onAccuracyChanged() is empty. The onSensorChanged() is used to monitor the device orientation.

There are two lines that are worth mentioning inside the implemented surfaceChanged() callback method of the SurfaceHolder.Callback interface. These lines create and start an instance of the Thread class into the sdlThread variable, which refers to the main SDL thread. There are two arguments specified inside the Thread class constructor, which are the target Runnable object and the thread name.

This is not a Java method but a C function. This method is the entry point for the SDL library. This method and the other C methods are found in the source file of the SDL named libSDL2.so, located in the /NewApp/libs/rmeabi-v7a directory.

By reaching this step, the C files are ready to be executed and the application will start and get ready for user interaction. We can return to our question, which is where in the Java project do we add views that appear in the application activity after the screen loading completes. Let’s follow the chain of execution to find the suitable position.

It is now clear that the nativeInit() function inside the SDLMain class is the entry point for executing the SDL library and rendering the application UI. Thus, we have to add our views before calling this function. Back in the chain, an instance of the SDLMain class is created within the surfaceChanged() method inside the SDLSurface class. Remember that the surfaceChanged() method is a callback method inside the SurfaceHolder.Callback interface that listens to changes in the SurfaceView.

Remember that the finishLoad() method is called within the onPostExecute() callback method of the UnpackFilesTask class. That is after the application loads its files. We can assume that the finishLoad() method of the SDLActivity class is the starting point where the activity layout is prepared. It is a good idea to add views to the activity layout that is visible only after the application loading step ends.

The button click action is handled by specifying the click listener using the setOnClickListener() method . This method accepts an instance of the OnClickListener() interface after implementing its onClick() method. The code inside this method is executed when the button is clicked. The button shows a toast message when clicked.

The two new views are added as children to a parent LinearLayout with horizontal orientation. Finally, the linear layout is added as a child to the activity layout.

Figure 7-14 shows the result after clicking the new button that was added in Java.

We can modify the toast message to print all child views inside the activity layout when the button is clicked. The code in Listing 7-31 modifies the onClickListener() callback method for that purpose. The number of child views inside the layout is returned using the getChildCount() method. To retrieve a specific view, the getChildAt() method accepts an index referring to the position of the view within the parent, where index 0 refers to the first child added to the parent.

Figure 7-15 shows the toast message after the button is clicked. It shows that the number of children in the layout is 2. The first one is an instance of the SDLSurface class and the second one is an instance of the LinearLayout class. The children inside the LinearLayout are also printed. The SDLSurface view contains all widgets defined in the Kivy application. In a later section, we discuss how events are handled for such widgets.

In summary, in order to add views to the layout while the screen is loading, add the code inside the showLoadingScreen() method inside the PythonActivity class. The keepAlive() method is called by SDL when the loading completes, which calls a method named removeLoadingScreen() where views can be deleted from the layout after the loading. After the project files are loaded, the onPostExecute() callback method of the UnpackFilesTask class calls a method named finishLoad() inside the SDLActivity class. It is a good place to add views to the layout after the screen is loaded.

Handling Touch Events of Kivy Widgets in Java

The activity layout contains the SDLSurface as a child view. This view holds all the widgets defined in the KV file of the Kivy application. In this section, we discuss how events are handled for these widgets within the Android Studio project.

In the KV file of the Python application, the four widgets are added to a BoxLayout . It has four child widgets (one Button, two TextInput, and one Label). Due to its vertical orientation, the widgets span the entire width of the BoxLayout but share the height equally. The height of the screen is divided by four and thus the height of each widget is 25% of the screen height (1/4)x1.0=0.25).

Because the (0, 0) point of the SDL Surface starts at the top-left corner, the Y coordinate of the region that the Button widget covers in the layout ranges from 0.75 to 1.0. SDL associates any touch event within that range with the button widget.

The Y range of the TextInput widget above the button goes from 0.5 to 0.75. Thus, SDL associates any touch event in this region to that widget. The same process repeats for the second TextInput widget (0.25 to 0.5) and the Label (0.0 to 0.25).

After changing the height of the surface to 3/4=0.75 of the screen height, the Y range of the widgets will change. For example, the Label widget Y coordinate will start at 0.0 but ends at 1/4 of the new surface height, which is 0.75, which is 0.25x0.75=0.1875. The Y coordinate of the first TextInput will start at 0.1875 and end at 0.375. Thus, the screen Y range from 0.1875 to 0.25 is now associated with the TextInput widget.

By calculating the range of the Y coordinate for each widget in the SDL surface, SDL can easily determine the destination of the touch events and take the proper action. For example, if the position of the touch event is (x=0.1, y=0.9), it is a touch on the Button widget and thus the callback function defined in the Kivy application will be called. Touch event handling for the Kivy widgets is done inside the onTouch() callback method of the SDLSurface class, which is located inside the SDLActivity Java file. The part of this method responsible for handling the touch events is shown in Listing 7-35.

This method accepts the target view and the event. The touch event might have multiple actions to be taken including pressing, releasing, motion, and more. For that reason, a switch block filters the action to make the proper decision.

First, the action of the event is returned from the event using the getActionMasked() method in the action integer variable. For example, when the touch is pressed, the returned integer is 0. When it is released, the integer is 1.

Inside an action such as ACTION_POINTER_DOWN, the x and y coordinates of the touch event are returned using the getX() and getY() methods. In order to know its relative position to the SDL surface size, they are divided by the width and height of the surface stored in the mWidth and mHeight variables. Remember that these variables are set equal to the width and height of the surface inside the surfaceChanged() callback method. Finally, the onNativeTouch() native function is called for handling the action according to the instructions written in Python. This method accepts the touch coordinates to decide which widget receives the event.

For the actions ACTION_UP, ACTION_DOWN, ACTION_POINTER_UP, and ACTION_POINTER_DOWN, the same code is executed. This is why there is no break statement for the cases associated with the actions ACTION_UP, ACTION_DOWN, and ACTION_POINTER_UP. If the action number is 1, for example, the switch block enters the case associated with ACTION_UP. Because this case does not have a break statement, the next case is executed, which is ACTION_DOWN. Also, this case does not have a break statement and thus the next case enters.

Similarly, the next case associated with ACTION_POINTER_UP does not have a break statement and thus the case associated with ACTION_POINTER_DOWN enters. Because this case includes a break statement at its end, its code is executed and the switch breaks.

Having reached this point, we have discussed the most important parts of the Java project created using Buildozer from the Kivy project. This helps us to understand how things work and modify the project to add Java components that work with the project.

Remember that one target of working on the Kivy project in Android Studio is that the complexity of implementing some actions within Python increases even with Pyjnius. In the next section, a Kivy button touch event is handled in Java.

Kivy Application

The KV file of the application is shown in Listing 7-36. There is a new widget named FileChooserIconView, which displays the filesystem of the device. The path property specifies the path in which the files and folders are displayed. It is set to the current directory. The dirselect determines whether the user can select directories or not. It is set to False to prevent the user from selecting directories. The multiselect property is set to False to avoid selecting multiple files. We just need to select a single image file. When the user makes a selection, the on_selection event is fired. It is handled within the Python file using a callback function named load_file().

After selecting the image file, the image is displayed on the Image widget by setting its source property to the selected file directory. A Label widget displays informational messages such as indicating whether the user selected an image or not.

Figure 7-18 shows the result after running the application. Note that there is no image displayed on the Image widget and thus its color is just a white region.

The function starts by accessing the widgets defined in the KV file. The file chooser is returned to the fileChooser variable. Even when disabling multi-selection, the selected file directory is returned into a list that can be accessed using the selection property. Index 0 returns the directory of the selected file as a string into the file_dir variable.

The selected file extension is compared to a list with the target extensions, which are .png and .jpg. If the list is equal to either, we are sure that the selected file is an image. Thus, the source property of the Image widget is set to the image directory.

If the extension is not .png or .jpg, a message is printed on the Label widget to inform the user of the incorrect selection.

The image is copied from its original path to the current directory using the copy() function within the shutil library. This copy throws an exception if the file already exists in the target directory. This is why it is written inside the try block. Inside the except block, the Label widget indicates that the image exists in the current directory, as shown in Figure 7-19.

Inside the Java file, we need a way to indicate the name of the image file. We can standardize the process by setting the name of the image to be processed inside the Java file to a fixed name. The selected name is processImageJavaOpenCV. Using the os.rename() function , the old and new filenames are specified. Using the join() function, the filename and its extension are concatenated. This is done within the try and except blocks.

After making sure the Python application works well, we need to build the Android project using Buildozer. The produced project will be opened in Android Studio, as is done in the previous project. Remember to set the compile SDK and build tool versions to whatever your PC requires.

Detecting the Touch Event of the Button Widget Inside Java

Remember that the all Kivy widgets are drawn inside the SDL surface defined within the SDLActivity class. When the surface is touched, the touch event position is returned within the onTouch() callback method. This position is compared against the regions of each Kivy widget. The event is associated with the widget where the touch occurred in its region. In this section, we will detect the region of the Kivy button.

Similar to the previous example within the previous section titled “Handling Touch Events of Kivy Widgets in Java,” the widgets are added within the KV file inside a BoxLayout with vertical orientation. Thus, the screen height is divided equally across all child widgets. Note that the screen and the BoxLayout can be used interchangeably because the layout fills the screen.

Because the screen has four widgets, the height for each widget is 25% of the screen height. Thus, the values of the Y coordinate of the Button widget relative to the screen height start at Y=0.75 and end at Y=1.0. Any touch event that occurs in this region of the screen is associated with the Button widget.

The modified onTouch() method that handles the button touch event is shown in Listing 7-38. If we are interested in handling the event after the touch is released, the code will be written inside the case associated with ACTION_UP.

Using an if statement, the Y coordinate of the touch position is compared against the range of Y values of the button. If the touch falls within this range, the required operations for handling the touch event are added within the if block. Currently, there is just a toast message shown. Later, after linking OpenCV with the project, some operations will be applied over the selected image.

Because there is no action associated with the widgets inside the Kivy project, we do not need to call the onNativeTouch() . It is enough to return the X and Y coordinates of the touch event to determine the target widget.

After building and running the Android project with the modified onTouch() method and then pressing the button, the result is shown in Figure 7-20. The next step is to link the project with OpenCV in order to process the image.