Thus far in this book we have used the OCI web-based console to work with OCI products and services. We saw how to set up our OCI account, create networking services, and create and configure DB Systems. If you are comfortable with the web-based console, you should consider continuing using it as every operation you need to perform is easy to find and most information can be found on a single page or a page with tabs.
However, what do you do if you want to write development operations (DevOps) procedures in the form of Bash (or similar) script files? What if you want to write those scripts in a programming language like Python? Further, what if you want to configure your PC to use only command-line access to manipulate objects in OCI? Fortunately, the answers for all these questions can be found in the command-line and application programming interfaces available for OCI.
In this chapter, we will see an overview of the capabilities of the command-line interface (CLI) and application programming interface (API) for working with OCI and MDS DB Systems.
Getting Started
One of the nice features of the OCI documentation is that Oracle often includes text that covers both web-based and CLI examples for working with OCI products and services. There are also examples where Oracle includes how to perform operations using the APIs, but those are often seen only in the API documentation.
You can use the CLI with very little effort on your part. In fact, if you have executed the examples in Chapter 7, you have already installed the CLI. Recall, we installed the CLI because MySQL Shell required it. However, what was not revealed was that the CLI is written using the Python OCI API. So, if you install the CLI, you’ve also installed the Python API. Nice!
An SSH key used for signing API requests and the public key uploaded into your OCI account.
Python 3 installed on a supported operating system.
See the CLI installation documentation at https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm for more information about installing the CLI.
Command-Line Interface (CLI)
The OCI CLI is a Python-based application that you can download and run from your PC. It represents an alternative to the web-based cloud console and is especially helpful for running repeated tasks or tasks you want to automate from your developer operations (DevOps) tools.
Interestingly, the CLI is built using the OCI APIs (SDK) and can show you what is possible for your own applications should you decide to use the API.
Install the CLI
Recall, we installed the CLI in Chapter 7 in the section entitled Install and Test the OCI CLI. If you have not installed the CLI yet, please refer to those sections to configure your PC and install the CLI.
Capabilities
The CLI is a macro-level interface for the Python API. As such, many of the operations available perform a set of tasks that would typically require several methods in the API. Fortunately, the CLI is designed with an eye for the operator. Thus, the operations are those that we think about when we work with OCI products and services.
CLI Help Example
CLI mysql Help Example
CLI mysql backup Help Example
Notice how much information is included in the help. This is the quickest way to explore the CLI and for the most part you should not need to visit the longer, more detailed online documentation. This is why we started with the web-based console. Once you master that, all these terms will become clear, and you can easily find the command and parameters you need to script any supported operation.
CLI mysql backup list Help Example
Wow, that’s a lot of information! We now know all we need to know to list the backups for a compartment or for a specific DB System (using the --db-system-id option).
Some options are required while others are optional. Also, some provide with shortcuts. For example, --compartment-id and -c are the same option.
Now that we understand the basics of the CLI, let’s look at some example operations that you can use right away with your DB Systems.
Example Uses
The following are some examples of using the CLI for common MDS operations ranging from simple listing of objects to creating objects. These are provided as examples and do not represent complete coverage of all MDS CLI commands. See the For More Information below for links to the documentation.
The parameters for the oci command must not use spaces. For example, --param1=value is valid, but --param1 = value is invalid and will lead to “invalid option” errors.
List Backups for a DB System
Listing Backups for a DB System
You may be required to enter the password for your API SSH key.
Stop/Start a DB System
Stopping a DB System
Getting a Work Request Status
Notice we see a status of SUCCEEDED. If the work request were still running, we would see the appropriate status. This is how you can track your work requests with the CLI, and it is a common mechanism.
Getting the DB System Information (stopped)
Like start/stop, you can also delete DB Systems as well as list the DB Systems in a compartment. There are many more commands available for DB Systems. Follow the links below in the For More Information section to learn more about the commands.
Ok, let’s look at one more example but this time we will use a complex operation: creating a new DB System.
Create a DB System
Creating a DB System requires a more complex set of parameters and options. In this case, we need, at a minimum, the compartment OCID, subnet OCID, shape name (not OCIDs), and availability domain name. We will add some more optional parameters, but let’s start assuming we do not know the OCIDs for these objects. Never fear, we can list them with the CLI!
List Compartments
Listing the Subnets for a Compartment
Next, we need the MySQL shapes available. Since this list could be long, we will see a paged output that will require multiple calls to get the complete list. If you’re like me, you don’t have the patience for that, so we can use the --all parameter.
Listing the MySQL Shapes
Listing Availability Domains
--admin-password <password>: The password for the MySQL administrative user.
--admin-username <name>: The username for the MySQL administrative user.
--data-storage-size-in-gbs <int>: Size of the data storage (database drive) in GBs.
--display-name <text>: A user-friendly name for the DB System.
--is-highly-available <bool>: Specifies if the DB System is highly available.
There is one more option we should discuss – the option to have the operation wait for a specific state. We do this with the --wait-for-state <state> option and one of the states (ACCEPTED, CANCELED, CANCELING, FAILED, IN_PROGRESS, SUCCEEDED). You can specify this option multiple times to wait for multiple operations. This operation asynchronously executes the operation and uses a work request to track the progress. Multiple states can be specified, returning on the first state. If timeout is reached, a return code of 2 is returned. For any other error, a return code of 1 is returned.
Create DB System Example Command
Deleting a DB System
Now that we’ve seen a few of the more common operations including listing, creating, and deleting resources, let’s look at the documentation available for the other MDS operations available in the CLI.
For More Information
Backup: Operations for working with backups including creating and listing (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/backup.html).
Channel: Operations for working with replication channels for inbound and outbound replication (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/channel.html).
Configuration: Operations on the set of MySQL server variables that configure the parameters of MySQL (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/configuration.html).
DB System: Operations on DB Systems including all features such as high availability, HeatWave, and more (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/db-system.html).
Shape: List operation to see available shapes (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/shape.html).
Version: List operation to list the MySQL server versions available (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/version.html).
Work-Request: Operations to list and get work requests for monitoring operations (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/work-request.html).
Work-Request-Error: List operation to get more information about work request errors (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/work-request.html).
Work-Request-Log-Entry: List operation to get more information from the work request log (https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/cmdref/mysql/work-request-log-entry.html).
Refer to the links above for more details about the CLI operations for these areas. You can also visit the OCI CLI main documentation at https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.15.0/oci_cli_docs/index.html to see the complete list of the OCI CLI commands.
OCI APIs
Java: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdk.htm#SDK_for_Java
Python: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/pythonsdk.htm#SDK_for_Python
TypeScript and JavaScript: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/typescriptsdk.htm#SDK_for_TypeScript_and_JavaScript
.NET: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/dotnetsdk.htm#SDK_for_NET
Go: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/gosdk.htm#SDK_for_Go
Ruby: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/rubysdk.htm#SDK_for_Ruby
Since the Python SDK is popular and the Python language is easy to learn, we will look at a few examples of using the Python SDK. Before we jump into the Python API examples, let’s discuss the MDS APIs briefly so that we understand the interfaces.
Each SDK has as part of its documentation an API reference, which is an extensive list of the APIs for each resource supported. For example, the Python SDK API reference is found at https://docs.oracle.com/en-us/iaas/tools/python/2.79.0/api/landing.html.
Now, let’s take a brief look at the MDS API.
MDS API
oci.mysql.ChannelsClient: API for working with inbound and outbound replication.
oci.mysql.DbBackupsClient: API for working with backups.
oci.mysql.DbSystemClient: API for working with DB Systems.
oci.mysql.MysqlaasClient: API for working with the MySQL client access (e.g. configuration, shapes, versions).
oci.mysql.WorkRequestsClient: API for interactive with work requests.
oci.mysql.ChannelsClientCompositeOperations: API for working with inbound and outbound replication.
oci.mysql.DbBackupsClientCompositeOperations: API for working with backups.
oci.mysql.DbSystemClientCompositeOperations: API for working with DB Systems.
oci.mysql.MysqlaasClientCompositeOperations: API for working with the MySQL client access (e.g., configuration, shapes, versions).
oci.mysql.WorkRequestsClientCompositeOperations: API for interactive with work requests.
It is often necessary to use multiple MDS API classes to achieve certain functions. Let’s get a brief tour of the MDS API in Python using a few examples.
Example Uses
Using the OCI SDKs can be a little challenging when you first begin working with them. In this section, we will see how to get started by demonstrating the basics of how the Python SDK works. But first, let’s install the Python SDK.
You must have Python 3 installed on your PC. See the Python organization at www.python.org/ for more details about installing Python on your PC.
Install the Python SDK
Before we can use the Python SDK, we must install it. Recall, we mentioned that the Python SDK is part of the OCI CLI, but it is installed in its own environment, so it is not accessible by a typical Python client. Fortunately, we can install the Python SDK alongside the OCI CLI without affecting either product. The following demonstrates how to install the Python SDK on Windows 11, but details for installing on other platforms or with other methods can be found at https://docs.oracle.com/en-us/iaas/tools/python/2.79.0/installation.html.
Installing the Python SDK
Getting Started
The basic layout of any Python SDK script begins with an import section where you import the getpass and oci modules. We need the getpass module because we must read the SSH key (called a passphrase) from the user (command-line).
This is followed by creating an instance of the oci.config class. This is an operation that reads your configuration file and prepares the SDK objects for use with your credentials. We then read and add the passphrase to the configuration file dictionary. Next, we connect to whichever API class we want to use and, if the connection is successful, we can proceed to make the API method calls.
If you have not set up your PC to use the OCI CLI, please refer to Chapter 7 and finish that setup before attempting the API examples. The configuration file is required for access to MDS via the API.
Listing 9-16 shows the basic Python script you need to access the OCI API classes. The script is named basic_api.py.
If you are new to Python programming, you can visit www.python.org/ for how to download and install Python as well as tutorials and help getting started with Python. There are also many excellent books on Python and a few websites that can help you get started.
Basic Python API Script
Now, let’s look at some of the examples we used in the CLI section to see how to accomplish the same using the Python SDK. Let’s start by listing the MySQL shapes using the oci.mysql.MysqlaasClient class.
Listing the MySQL Shapes
To list the MySQL shapes, we use the list_shapes() method from the oci.mysql.MysqlaasClient class, which takes as a parameter the compartment id, which is common among many of the API methods.
This returns a list or ShapeSummary classes, which we can use to loop through the shapes and print the name, which is a lot cleaner (and fewer lines) than a JSON output like the CLI. You can find the description of the ShapeSummary class at https://docs.oracle.com/en-us/iaas/tools/python/2.79.0/api/mysql/models/oci.mysql.models.ShapeSummary.html#oci.mysql.models.ShapeSummary.
List MySQL Shapes Script
Output of List MySQL Shapes Script
Now, let’s take the complexity up a bit. Let’s see how to control DB Systems.
Stop/Start DB System
When working with DB Systems, we will use the oci.mysql.DbSystemClient class and its methods. To stop a DB System, we use the stop_db_system() method and to start a DB System, we use the start_db_system() method. As you can see, the API methods are often quite self-explanatory.
This example is a bit more complicated because we will have to provide a DB System id (OCID) in the form of another class named stop_db_system_details() for the stop or start operation. This class uses a model to populate the data. In this case, we need the DB System OCID and the shutdown type.
We are also going to make it a bit more complex by first listing the DB Systems in our compartment and select the one with a specific name. You may be amazed at how difficult it is to remember OCIDs, but it is much easier to remember a DB System name. We’ll display a list for the user to choose the DB System to control. To list the DB Systems, we use the list_db_systems() method and pass in the compartment id, which we have saved in an environment file.
In summary, we will import the libraries we need, read the configuration file, read the SSH key passphrase from the user, read the compartment id from the environment variable, then read the DB System name from the user. Once we have that information, we can call the method to list the DB Systems in the compartment, select one that matches the name, then either stop or start it.
Control DB System Script
Output of the Control DB System Script
This script would make a good starting point for other operations you may want to perform on your DB Systems. Simply modify it to your needs!
The last example we will look at is the operation to create a new DB System. While the Python code may seem more complex, but it uses the same features of this example only with a more sophisticated Python programming style. However, there are more API methods, and the parameters are more complex.
Creating a DB System
--compartment-id <id>: The compartment id.
--shape-name <name>: The shape name.
--subnet-id <id>: The subnet OCID.
--availability-domain <name>: The availability domain name to use.
--admin-password <password>: The password for the MySQL administrative user.
--admin-username <name>: The username for the MySQL administrative user.
--data-storage-size-in-gbs <int>: Size of the data storage (database drive) in GBs.
--display-name <text>: A user-friendly name for the DB System.
--is-highly-available <bool>: Specifies if the DB System is highly available.
Recall from the CLI example, we used four CLI commands; one each to get the shape name, subnet id, and availability domain, and one to perform the create. This example using the Python API will do the same thing only in a single Python script.
Like the last example, we will display a simple user interface to allow the user to choose the items from the lists asking only for the display name and data size as prompts. We will also prompt for the MySQL administrative user and password.
Rather than go through line-by-line, let’s summarize what we will be doing. We know we must get the subnet, shape, and availability domain so we will write methods to get that information using a helper method that presents a list to the user prompting her to choose one from the list. Thus, we will get these three data items from the user.
In doing so, we will explore more API classes including the oci.core.VirtualNetworkClient, oci.mysql.MysqlaasClient, and oci.identity.IdentityClient classes calling the appropriate methods for getting a list of those resources.
The optional parameters; MySQL administrator user and password, data storage size, and display name are read from the user with prompts. We also set the HA to disabled by default.
Once we have all the information, we use the create_db_system() method of the oci.mysql.DbSystemClient class to create the DB System. This method requires the population of a model named oci.mysql.models.CreateDbSystemDetails.
Finally, since the create_db_system() method returns immediately, we pole the DB System by fetching its details with the get_db_system() method passing in the OCID for the new DB System as returned in the create_db_system_response.data.id variable. We pole the DB System until its lifecycle_state equals ACTIVE. There are other methods for doing this including some helper methods in the OCI API, but those are a bit more advanced for those getting started. Polling the lifecycle_state is an acceptable initial solution if you want the script to execute and wait until the resource is ready.
That’s about it. There are more nuances to this example especially the Python programming parts, but we leave those details to you to ponder as an exercise. There are no magical or arcane code constructs here, just normal Python code around the Python API for OCI.
Create DB Systems Script
Output from DB System Script
Take some time and experiment with this script for your own uses. You may find it helpful to getting your own automation in Python going for MDS.
But wait, there is more. Let’s add a bonus example for you to try on your own.
Delete a DB System
Your challenge is to take the control DB System script above and add the delete operation. To get started, you will need to use the delete_db_system () method of the oci.mysql.DbSystemClient class. You can find the documentation for this method and an example Python script at https://docs.oracle.com/en-us/iaas/tools/python/2.79.0/api/mysql/client/oci.mysql.DbSystemClient.html#oci.mysql.DbSystemClient.delete_db_system. You should not have to add much to the control DB System script (control_db_system.py) to accomplish this task, but it will be a good exercise for those just starting out with Python and the API.
Well, that’s our tour of the API! Although the tour of the MDS API and the Python SDK was brief, it should give you an idea of what is possible and the basis to begin researching your own Python applications to work with MDS resources.
Summary
When working with tens, hundreds, or thousands of DB Systems, performing maintenance operations on or working with the OCI web-based console may become tedious and somewhat slower than you’d like. That’s where the CLI and APIs shine; they allow you to script common operations either in a Bash (or similar) script or in a Python code file that you can execute.
Either way, you will gain greater control and greater productivity when working with many OCI products and services and especially when your OCI operations become routine. In those cases, any improvement in productivity pays dividends. If these advantages appeal to you, I encourage you to read the online documentation for those areas you want to automate (script) first so that you can get acquainted with the mechanisms used. Once you’ve mastered the basics, you can branch out to explore more complex operations.
In the next chapter, we will learn some strategies and planning for migrating existing on-prem MySQL installations to MDS.