How Can Our Application Access This Information?

There are two ways to access the information described earlier: using environment variables or volume files. Both need to be configured in the YAML file of the Pod manifest, respectively the Deployment manifest. In the environment variable approach, we just map the information into an environment variable that can then be read by our application. In the volume file approach, we store the information in a file that can then be read by our application. Let us now look at an example from the Local News application, more specifically our Quarkus-based News-Backend. A label in our Pod defines whether the Pod is running in a development or a production environment. If the Pod is labeled as dev=true, the application should run with dev profile and live coding enabled. This allows, among others, for remote debugging from our local IDE.

We can see that Quarkus has been started in development mode. If you now change Kubernetes label “dev” to false in the file snippets/chapter4/downward/news-backend-deployment.yaml and apply the changes to the Deployment with “kubectl -n news-backend-prod-dev apply -f snippets/chapter4/downward/news-backend-deployment.yaml”, the application will start with the Prod profile.

Let us summarize what we have learned about the Downward API so far. On one hand, it provides limited information about the Kubernetes environment our application is running in. On the other hand, it introduces minimal dependencies between our application and Kubernetes. The interface is just a simple environment variable or a file mapped as a volume. However, the application must also read and process the environment variables or files defined in the Pod to make use of it. Depending on how general or specific the purpose of the environment variable is, this could lead to weaker or stronger dependencies. In our example, we used an existing environment variable provided by the Quarkus framework and injected the boolean value into it. Instead, we could have defined a new NAMESPACE variable to pass in the current Namespace, for example, with the postfix prod or dev. In this example, the application would be fully aware of the fact that there is something like a Kubernetes Namespace, and the coupling increases.

Simply reading the information of the environment of one Pod is still pretty basic. What if we would like to read information or even create, modify, or delete other resources? This brings us to the next topic.

The RESTful Kubernetes API

To access the Kube-apiserver in Minikube, we use the same approach that has been introduced in Chapter 1 and run “kubectl proxy --port 8080” which makes the Kubernetes API available at localhost:8080.

Now we can retrieve all of our Deployments directly via the Kupe-apiserver. To do this, we first need the group and version which can be derived from the apiVersion attribute of our YAML resource files. The group for the Deployment resource type is apps, and the version is v1. With the rules described earlier, we will get http://localhost:8080/apis/apps/v1 which will return a list of all resources managed under the same group and version. Let us add our Namespace part and the resource type, and we will obtain the following URL: http://localhost:8080/apis/apps/v1/namespaces/localnews/deployments.

Put it into your browser, and the response will be a DeploymentList showing all of our Deployments in this Namespace. If we would like to see a particular Deployment, let’s say the one for the news-backend, we can just append its name: http://localhost:8080/apis/apps/v1/namespaces/localnews/deployments/news-backend.

We will then get the resource represented as a JSON string. If we would like to create a Deployment instead of retrieving it, we can just send a POST request to the previous URL (without the resource name news-backend because this will already be derived from the name attribute in the POST body as part of the resource YAML, respectively its JSON representation). But wait, what we just did was again driven by us and not the application itself. Quit the proxying of the Kubernetes API to our localhost with Ctrl+C in the terminal, and let us then look at how an application would ideally interact with the Kubernetes API.

Using Client Libraries to Access the API

While we could just install and use a REST client library to interact directly with the Kubernetes API as described earlier, it is much simpler to use a Kubernetes client library. There is a rich ecosystem of libraries for different languages and platforms: Java, Python, Go, JavaScript, and many more. But what would that look like in a real-world scenario? Assume we need a new feature for our Local News application to scrape additional RSS Feeds such as the ones from Al Jazeera, CNN, The New York Times, and others. We would like to create new feed scraping processes using our existing frontend application, for example, by adding a form to submit the feed URLs to be analyzed.

The most obvious approach would be to connect our JavaScript-based News-Frontend directly with the API server. However, this is not the best idea since the News-Frontend is a client application and thus runs on the end user’s machine. To implement this, we would need to expose the Kube-apiserver to our end users, and they should not necessarily know that our application runs on Kubernetes, not to mention the security risk in doing so.

A better approach would be to add the functionality to the News-Backend and expose it via its REST interface to the frontend. This scenario is depicted in Figure 4-1.

The News-Backend component is written in Java, and we decided to use the Quarkus framework to implement its functionality (as explained in Chapter 2). Although there are many different Java-based Kubernetes client libraries, we decided to use the one called Fabric8 Kubernetes Client² because it can be easily included via a Quarkus extension³ and because it provides an elegant programming model as we will see in the following.

The new controller can be called via the URI path /scraper using a POST request with a JSON array as the request body. Once called, it will create a new Kubernetes Deployment resource in the same Namespace as the backend. In order to do this, we first load an existing YAML manifest which is then used to make the Kubernetes Deployment. We have omitted the details of this method for brevity, but you can find its details in our Git repository in the folder components/news-backend/src/main/java. It makes use of the Kubernetes client which is injected via CDI⁵ (Jakarta Context and Dependency Injection) into the controller class. The deployment object is then modified, for example, the name and Namespace are changed and the RSS Feed URLs passed in via the request body are added as an environment variable SCRAPER_FEEDS_URL into the new Deployment of the Feed-Scraper. Finally, we use the deployment object to create the resource via the client. To do so, the client – as depicted in Figure 4-1 – sends a POST request to the API server with the JSON representation of the deployment object. This POST request is sent when invoking the createOrReplace method on the kubernetesClient object with the deployment object as a parameter.

You can test the API server access described earlier quite easily directly from your IDE. You only need to make sure that you can access the API server via kubectl. The Fabric8 Kubernetes Client will then reuse your existing configuration. This allows you to develop and test against the API quite easily. Please also compare the development models described in Chapter 3. Let us quickly test the scenario by running “mvnw compile quarkus:dev” from the project root folder components/news-backend (make sure that the previous proxy process to port 8080 is stopped). Then send the following curl command “curl -X POST -H ‘Content-Type: application/json’ -d ‘[“http://rss.cnn.com/rss/edition_world.rss”]’ localhost:8080/scraper”. This will create a new Feed-Scraper Deployment which you can check by running “kubectl get deployments”. Since we started the News-Backend directly from our IDE and hence outside our Kubernetes cluster, the Deployment is in the default Namespace.

Authenticate and Authorize Against the API

Now let us try this out in a Kubernetes environment. We can access the News-Backend directly via “minikube -n localnews service news-backend”. Let us send a POST request as defined in Listing 4-8.

curl -X POST -H "Content-Type: application/json" -d '["http://rss.cnn.com/rss/edition_world.rss", "https://www.nytimes.com/svc/collections/v1/publish/https://www.nytimes.com/section/world/rss.xml"]' $(echo | minikube -n localnews service news-backend --url)/scraper

Listing 4-8

Make an (Unauthorized) POST Request to the Kube-apiserver

As we can see, there seems to be an error with our request. Let us dig into the logs by running “kubectl -n localnews logs deployments/news-backend”. The results can be found in Listing 4-9.

Failure executing: GET at: https://10.96.0.1/apis/apps/v1/namespaces/ch4/deployments. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. deployments.apps is forbidden: User "system:serviceaccount:localnews:default" cannot list resource "deployments" in API group "apps" in the namespace "localnews".

Listing 4-9

Logs from the News-Backend

That is, the Kubernetes client is already connected to the API server using the internal Service IP, but access is forbidden. The News-Backend is using the default ServiceAccount which is created automatically with a Namespace and used by all Pods inside this Namespace. Hence, we need a new ServiceAccount, a Role that grants access to list and create Deployments, and a RoleBinding that binds our new Role to the ServiceAccount. Figure 4-2 shows the relationships between the resources.

Let us deploy the additional resources to grant access for the News-Backend component to the Kubernetes API by running “kubectl -n localnews apply -f snippets/chapter4/api” where a Kubernetes Role, RoleBinding, and ServiceAccount get created, and, finally, the News-Backend Deployment gets updated to reference the new ServiceAccount. Note the additional key for the ServiceAccount in the Deployment manifest in Listing 4-10.

template:

    metadata:

      labels:

        app: news-backend

    spec:

      serviceAccountName: news-backend-sa

Listing 4-10

Excerpt of News-Backend Deployment Specifying a Nondefault ServiceAccount

When we send the same POST request from Listing 4-8 to the News-Backend, we will get a 200 OK status now, and we will see the JSON representation of the newly created Deployment in the body of our response. This new Deployment of the Feed-Scraper should now be active as scraper-1, which can be checked by running “kubectl -n localnews get deployments”. Moreover, the news items from the two additional feed URLs should soon be processed, and the corresponding news should appear on the map of the frontend. Check it by running “minikube service news-frontend -n localnews”.

The approach that we’ve outlined here, the direct access from an application to the Kubernetes API, certainly introduces more dependencies between our application and Kubernetes compared to the Downward API. Still, if you carefully design the architecture of your application, you could avoid unnecessary dependencies. In the case of our Local News example, the only dependency to Kubernetes is the inclusion of the Fabric8 Kubernetes Client library. If our application would run in a non-Kubernetes environment, the worst thing that could happen is that someone calls our new scraper controller which would result in an internal server error because there is no such thing as a Kubernetes API. However, all other controllers will still work. This is not self-evident; depending on the application, there could be stronger dependencies, for example, if the application would access the Kubernetes API during startup, it would fail to start and become completely unavailable.

What still needs to be considered is the extension of the frontend to access our new backend REST API. Figure 4-3 shows a screenshot of how it looks. We skip any details on its implementation; you can just check out the source code of the News-Frontend if you’re interested. In summary, it requires some effort to implement this feature (user interface code + REST interface + Kubernetes client), although it is probably not exposed to the end user. Which feeds are scraped is configured by an administrator of the application. Couldn’t we hence find a more elegant solution based on Kubernetes-native capabilities? Yes, sure, we can, and we can even do more! Let us define our own resource type and see how that helps!

The Custom Resource Definition (CRD) – Schema for Your Custom Resource

To create our own resource, we must first define a Custom Resource Definition . It defines a type for our Custom Resources. The relationship between a Custom Resource Definition and a Custom Resource is similar to that of a class and an object in an object-oriented programming language. It defines metadata such as the name and a version for your resource type. The latter becomes quite handy when your Custom Resource Definition changes with the requirements of your application. This allows you to version the different alternatives, and hence you can manage to use multiple versions at the same time, for example, one Custom Resource could refer to the v1 of your Custom Resource Definition and another could refer to the new v2.

The schema is defined by the OpenAPI Version 3 specification⁷ which is a well-known standard for defining HTTP APIs. In this case, we define that the resource must contain a spec property with a nested urls property of type array. The items of this array must be of type string to represent the feed URLs. After applying this resource, we are now able to create FeedAnalysis resources. But we can even do more. We can just interact with our FeedAnalyses resources as with any other Kubernetes resource via kubectl. Let us again create a Namespace by running “kubectl create ns localnews” and demonstrate this by creating an instance of our CRD, more specifically the CR shown in Listing 4-13, via “kubectl -n localnews create -f snippets/chapter4/crd/my-feed-analysis.yaml” and list it with “kubectl get feedanalyses -n localnews”.

Kubectl vs. User Interface

With kubectl as a command-line interface, we can manage FeedAnalyses with ease. As with other resources, we can create, apply, get, list, modify, and delete FeedAnalyses. We do not need to put any effort into writing frontend code nor writing an additional REST backend, and thus we do not need to maintain it anyway. But there is much more than this!

The advantage of this approach is that we can specify which feed to analyze declaratively in a YAML file instead of relying on a user interface. Why is this better? It is reproducible in a consistent manner. If we click around in our user interface and add or delete news feeds, this is not reproducible; at least it is not easy to do so and involves manual steps such as analyzing the logs. With the YAML manifest, we can capture all the information in a text format which is a perfect means to be stored, side by side with our code, in our version control system. And we will see in Chapters 5 and 6 how we could automate the synchronization between manifests in a Git repository and Kubernetes with GitOps.

CRDs are a great way to extend Kubernetes and automate certain operations. But wait, didn’t we miss something? Yes, right, where is the logic behind the CRD? How can we define what happens when a new FeedAnalysis is created? Let us elaborate on this in the following.

Attempt 1 – Let the Backend Do the Job

The most obvious approach that we can follow is to reuse the functionality that we have already written in our News-Backend in the previous approach. Instead of calling the createDeployment method via a POST request, we just want to call it whenever a new FeedAnalysis resource is created. The approach is depicted in Figure 4-5.

The approach works quite well because we have already implemented the logic for creating the respective Deployments via the Kubernetes API. But as you might remember, the primary intention to choose the backend to implement this was driven by the requirement to provide a REST interface for our user interface. Let us reflect on this. Is the backend really the right component to implement the logic behind the CRD? In the end, we would like to control the Feed-Scraper, why not add the logic directly to the Feed-Scraper then?

Attempt 2 – Extending the Feed-Scraper

This route uses the Kubernetes client to watch for the creation of Custom Resources with name feedanalysis and the respective group and version. The JSON text is transformed into an object of type FeedAnalysis (similar to the one in the preceding approach), and then a new RssReader route is started. Figure 4-6 depicts the approach.

The approach described here is more lightweight than the previous one because the logic behind our CRD is placed directly in the component that contains the logic the CRD targets at. This is, however, not always the case. We could also write more comprehensive CRDs which could target the functionality of several components. Another drawback is that we now make the Feed-Scraper aware and thus dependent on Kubernetes which would not be necessary if we placed our logic somewhere else. But where could that be? How about a separate component?

Attempt 3 – Writing a Custom Controller

In this attempt, we shift our logic behind the CRD into a separate application component which is referred to as a custom controller. The good news is that we can use the same technology/logic that we have used in our previous attempt. The bad news is that we cannot start a route in the Feed-Scraper because we are now part of another process and thus another instance of the JVM. We cannot call the RssReader route from our CrdRoute in the custom controller locally. By adding an API, we are able to start the RSS reading remotely which leads to the approach depicted in Figure 4-7.

As you can see, the approach adheres quite well to the separation of concerns principle. The logic is separated, and our components such as Feed-Scraper, News-Frontend, and News-Backend stay untouched and are thus not even aware that they are running on Kubernetes. If our CRD evolves, we can release new versions of our controller, and if we decide not to run our application on top of Kubernetes, we can just deploy all components except the controller (because it is Kubernetes specific and cannot run outside).

The drawback is that it is a bit cumbersome to interact with the Feed-Scraper, and we need this extra API to connect the controller with our component. Maybe we can get rid of this as well. What if we deployed the Feed-Scraper as a Kubernetes Job instead of using a Deployment? We discussed the topic Deployment vs. Job for the Feed-Scraper already in Chapter 3 when we ran the Feed-Scraper as a Kubernetes Job to trigger some action in the Location-Extractor which ran our local machine but was connected to the News-Backend in the cluster. So let us see how the job-based approach resonates with our controller approach.

Attempt 4 – Let the Kubernetes Job Do the Job

In Attempt 1, we created a Deployment whenever a new CR of type FeedAnalysis was created. This time, we will create a Kubernetes Job which is a little bit easier since we don’t have to worry about which functionality of the component is exposed via a REST API. We can just control the lifecycle via a Kubernetes Job API. This scenario is shown in Figure 4-8.

Cleaning Up

The hands-on parts of this chapter are now over. Therefore, don’t forget to clean up to have a fresh environment ready for the next chapter. The easiest way to do it is to run a “minikube delete” and then start Minikube again with a command such as “minikube start --addons=ingress --vm=true --kubernetes-version=‘v1.22.3’ --memory=‘8g’ --cpus=‘4’ --disk-size=‘25000mb’ ”. Alternatively, it may suffice to remove the Namespace from the last example via “kubectl delete namespace attempt4” and the earlier one via “kubectl delete ns localnews”.

Everything Is a Resource – So Treat It Like That

In Kubernetes, everything is described as a resource and thus can be managed via its API. This is the main topic of this chapter, and we have already demonstrated several ways how to leverage this by the example of the Local News application. Our journey showed that the most native way to dig into this is to extend the existing Kubernetes resources such as Deployments, Services, or Jobs by application-specific resources to raise the level of abstraction and, thus, leverage the key concepts of Kubernetes. This enables administrators of our application to declaratively define what should be done instead of describing how that must be done, for example, they could just create new FeedAnalysis resources to define which feeds should be analyzed without having to define how this will be implemented.

Don’t Reinvent the Wheel – Make Use of Kubernetes’ Capabilities Whenever Possible