9

Extending Istio Data Plane

Istio provides various APIs to manage data plane traffic. There is one API called EnvoyFilter that we have not yet used. The EnvoyFilter API provides a means to customize the istio-proxy configuration generated by the Istio control plane. Using the EnvoyFilter API, you can directly use Envoy filters even if they are not directly supported by Istio APIs.

There is another API called WasmPlugins, which is another mechanism to extend the istio-proxy functionality WebAssembly (Wasm) support is becoming common for proxies such as Envoy to enable developers to build extensions.

In this chapter, we will discuss these two topics; however, the content on EnvoyFilter will be brief, as you have already learned about filters and plugins for Envoy in Chapter 3. Rather, we will focus on how to invoke Envoy plugins from Istio configurations. However, we will delve deeper into Wasm with hands-on activities as usual.

In this chapter we will be covering the following topics:

  • Why extensibility?
  • Customizing the data plane using EnvoyFilter
  • Understanding the fundamentals of Wasm
  • Extending the Istio data plane using Wasm

Technical requirements

To keep it simple, we will be using minikube to perform the hands-on exercises in this chapter. By now, you must be familiar with installing and configuring minikube, and if not, please refer to the Technical requirements section of Chapter 4.

In addition to minikube, it is good to have Go and TinyGo installed on your workstation. If you are new to Go, then follow the instructions at https://go.dev/doc/install to install it. Install TinyGo for your host OS by following the instructions at https://tinygo.org/getting-started/install/macos/. Then validate the installation by using the following command:

% tinygo version
tinygo version 0.26.0 darwin/amd64 (using go version go1.18.5 and LLVM version 14.0.0)

Why extensibility

As with any good architecture, extensibility is very important because there is no one size fits all approach to technology that can adapt to every application. Extensibility is important in Istio as it provides options to users to build corner cases and extend Istio as per their individual needs. In the early days of Istio and Envoy, the projects took different approaches to build extensibility. Istio took the approach of building a generic out-of-process extension model called Mixer (https://istio.io/v1.6/docs/reference/config/policy-and-telemetry/mixer-overview/), whereas Envoy focused on in-proxy extensions (https://www.envoyproxy.io/docs/envoy/latest/extending/extending). Mixer is now deprecated; it was a plugin-based implementation used for building extensions (also called adaptors) for various infrastructure backends. Some examples of adapters are Bluemix, AWS, Prometheus, Datadog, and SolarWinds. These adapters allowed Istio to interface with various kinds of backend systems for logging, monitoring, and telemetry, but the adapter-based extension model suffered from significant resource inefficiencies that impacted tail latencies and resource utilization. This model was also intrinsically limited and had limited application. The Envoy extension approach required users to write filters in C++, which is also Envoy’s native language. Extensions written in C++ are then packaged along with Envoy’s code base, compiled, and tested to make sure that they are working as expected. The in-proxy extension approach for Envoy imposed a constraint of writing extensions in C++ followed by a monolithic build process and the fact that you must now maintain the Envoy code base yourself. Some bigger organizations were able to manage their own copy of the Envoy code base, but most of the Envoy community found this approach impractical. So, Envoy adopted other approaches for building extensions, one being Lua-based filters and the other being Wasm extensions. In Lua-based extensions, users can write inline Lua code in an existing Envoy HTTP Lua filter. The following is an example of a Lua filter; the Lua script has been highlighted:

http_filters:
name: envoy.filters.http.lua
typed_config:
  "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
  default_source_code:
    inline_string: |
     function envoy_on_request(request_handle)
...... -- Do something on the request path.
       request_handle:headers():add("NewHeader", "XYZ")
     end
     function envoy_on_response(response_handle)
       -- Do something on the response path.
      response_handle:logInfo("Log something")
     response_handle:headers:add("response_size",response_handle:body():length())
      response_handle:headers:remove("proxy")
    end

In this example, we are using the HTTP Lua filter. The HTTP Lua filter allows Lua scripts to be run during both the request and response cycle. Envoy runs the Lua script as a coroutine; LuaJIT is used as the Lua runtime environment and is allocated per Envoy worker thread. The Lua scripts should contain the envoy_on_request and/or envoy_on_response functions, which are then executed as coroutines on the request and response cycles, respectively. You can write Lua code in these functions to perform the following during request/response processing:

  • Inspection and modification of headers, body, and trailers of request and response flows
  • Asynchronous HTTP invocation of upstream systems
  • Performing direct response and skipping further filter iteration

You can read more about Envoy HTTP Lua filters at https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/lua_filter.html?highlight=lua%20filter. This approach is great for simple logic, but when writing complex processing instructions then writing inline Lua code is not practical. Inline code cannot be easily shared with other developers or easily aligned with best practices of software programming. The other drawback is the lack of flexibility, as developers are obliged to only use Lua, which inhibits non-Lua developers from writing these extensions.

To provide extensibility to Istio, an approach that imposed fewer tradeoffs was needed. As Istio’s data plane comprises Envoy, it made sense to converge on a common approach for extensibility for Envoy and Istio. This can decouple Envoy releases from their extension ecosystem, enables Istio consumers to build data plane extensions using their languages of choice, using best-of-breed programming languages and practices, and then deploy these extensions without causing any downtime risk to their Istio deployments in production. Based on this common effort, Wasm support for Istio was introduced. In the upcoming sections, we will discuss Wasm. But before that, let’s quickly touch on Istio support for running Envoy filters in the next section.

Customizing the data plane using Envoy Filter

Istio provides an EnvoyFilter API, which provides options to modify configurations created via other Istio custom resource definitions (CRDs). Essentially one of the functions performed by Istio is translating high-level Istio CRDs into low-level Envoy configurations. Using the EnvoyFilter CRD, you can change those low-level configurations directly. This is a very powerful feature but also should be used cautiously as it has the potential to make things worse if not used correctly. Using EnvoyFilter, you can apply configurations that are not directly available in Istio CRDs and perform more advanced Envoy functions. The filter can be applied at the namespace level as well as selective workload levels identified by labels.

Let’s try to understand this further via an example.

We will pick one of the hands-on exercises we performed in Chapter 7 to route a request to hhtppbin.org. Do not forget to create the Chapter09 folder and turn on istio-injection. The following commands will deploy the httpbin Pod as described in Chapter09/01-httpbin-deployment.yaml:

kubectl apply -f Chapter09/01-httpbin-deployment.yaml
curl -H "Host:httpbin.org" http://a816bb2638a5e4a8c990ce790b47d429-1565783620.us-east-1.elb.amazonaws.com/get

Carefully check all the response fields containing all the headers passed in the request.

Using EnvoyFilter, we will add a custom header to the request before sending it to the httpbin Pod. For this example, let’s pick the ChapterName header name and set its value to ExtendingIstioDataPlane. The configuration in Chapter09/02-httpbinenvoyfilter-httpbin.yaml adds the custom header to the request.

Apply the following configuration using EnvoyFilter:

$ kubectl apply -f Chapter09/02-httpbinenvoyfilter-httpbin.yaml
envoyfilter.networking.istio.io/updateheaderhorhttpbin configured

Let’s go through Chapter09/02-httpbinenvoyfilter-httpbin.yaml in two parts:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: updateheaderforhttpbin
  namespace: chapter09
spec:
  workloadSelector:
    labels:
      app: httpbin
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        portNumber: 80
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
            subFilter:
              name: "envoy.filters.http.router"

In this part, we will create an EnvoyFilter named updateheaderforhttpbin in the chapter09 namespace, which will be applied to the workload which has the app label with a httpbin value. For that configuration, we are applying a configuration patch to all inbound traffic to the Istio sidecar aka istio-proxy aka Envoy for port 80 of the httpbin Pod. The configuration patch is applied to HTTP_FILTER and, in particular, to the HTTP router filter of the http_connection_manager network filter.

In the next part of the EnvoyFilter configuration, we apply configuration before the existing route configuration and, in particular, we are appending a Lua filter with inline code as specified in the inlineCode section. The Lua code runs during the envoy_on_request phase and adds a request header with the X-ChapterName name and the ExtendingIstioDataPlane value:

 patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.lua
        typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
          inlineCode: |
            function envoy_on_request(request_handle)
              request_handle:logInfo(" ========= XXXXX ==========");
              request_handle:headers():add("X-ChapterName", "ExtendingIstioDataPlane");
            end

Now, go ahead and test the endpoint using the following command:

% curl -H "Host:httpbin.org" http://a816bb2638a5e4a8c990ce790b47d429-1565783620.us-east-1.elb.amazonaws.com/get

You will receive the added headers in the response.

You can see the final Envoy config applied using the following commands. To find the exact name of the httpbin Pod, you can make use of proxy-status:

% istioctl proxy-status | grep httpbin
httpbin-7bffdcffd-l52sh.chapter09
Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED      NOT SENT     istiod-56fd889679-ltxg5     1.14.3

This is followed by the proxy-config details for listeners:

% istioctl proxy-config listener httpbin-7bffdcffd-l52sh.chapter09  -o json

In the output, look for envoy.lua, which is the name of the patch and the filter we applied via the config. In the output, look for filterChainMatch and for destinationPort set to 80:

"filterChainMatch": {
                    "destinationPort": 80,
                    "transportProtocol": "raw_buffer"
                },

We applied the config via EnvoyFilter:

  {
                                    "name": "envoy.lua",
                                    "typedConfig": {
                                        "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua",
                                        "inlineCode": "function envoy_on_request(request_handle)
  request_handle:logInfo(" ========= XXXXX ==========");
  request_handle:headers():add("X-ChapterName", "ExtendingIstioDataPlane");
end 
"
                                    }
                                }

Hopefully, that gave you an idea of EnvoyFilter and how the overall mechanism works. In the hands-on exercise for this chapter, another example applies the same changes but at the Ingress gateway level. You can find the example at Chapter09/03-httpbinenvoyfilter-httpbiningress.yaml. Make sure that you delete the Chapter09/02-httpbinenvoyfilter-httpbin.yaml file before applying the Ingress gateway changes.

For more details about the various configurations of EnvoyFilter, please refer to the Istio documentation at https://istio.io/latest/docs/reference/config/networking/envoy-filter/#EnvoyFilter-EnvoyConfigObjectPatch.

Important note

For cleanup, use this command: kubectl delete ns chapter09.

In the next section, we will read about Wasm fundamentals, followed by how to use Wasm to extend the Istio data plane.

Understanding the fundamentals of Wasm

Wasm is a portable binary format designed to run on virtual machines (VMs), allowing it to run on various computer hardware and digital devices, and is very actively used to improve the performance of web applications. It is a virtual instruction set architecture (ISA) for a stack machine designed to be portable, compact, and secure with a smaller binary file size to reduce download times when executed on web browsers. A modern browser’s JavaScript engines can parse and download the Wasm binary format in order of magnitude faster than JavaScript. All major browser vendors have adopted Wasm, and as per the Mozilla Foundation, Wasm code runs between 10% and 800% faster than the equivalent JavaScript code. It provides faster startup time and higher peak performance without memory bloat.

Wasm is also a preferred and practical choice for building extensions for Envoy for the following reasons:

  • Wasm extensions can be delivered at runtime without needing to restart istio-proxy. Furthermore, the extension can be loaded to istio-proxy through various means without needing any changes to istio-proxy. This allows the delivery of changes to the extension and changes to proxy behavior in the form of extensions without any outages.
  • Isolated from the host and executed in a sandbox/VM environment, Wasm communicates with the host machine via an application binary interface (ABI). Through ABIs, we can control what can and cannot be modified and what is visible to the extension.
  • Another benefit of running Wasm in a sandbox environment is the isolation and defined fault boundaries. If anything goes wrong with Wasm execution, then the scope of disruption is limited to the sandbox and won’t spread to the host process.

Figure 9.1 – An overview of Wasm

Figure 9.1 – An overview of Wasm

There are over thirty programming languages that support compilation to Wasm modules. Some examples are C, Java, Go, Rust, C++, and TypeScript. This allows most developers to build Istio extensions using the programming language of their choice.

To get familiar with Wasm, we will build a sample application using Go. The source code is available in the Chapter09/go-Wasm-example folder.

The problem statement is to build an HTML page that takes a string in lowercase and provides the output in uppercase. We assume that you have some experience working with Go and that it is installed in your hands-on environment. If you don’t want to use Go, then try implementing the example using the language of your choice:

  1. Copy the code from Chapter09/go-Wasm-example and reinitialize the Go module:
    % go mod init Bootstrap-Service-Mesh-Implementations-with-Istio/Chapter09/go-Wasm-example
% go mod tidy

First, let’s check Chapter09/go-Wasm-example/cmd/Wasm/main.go:

package main
import (
    "strings"
    "syscall/js"
)
func main() {
    done := make(chan struct{}, 0)
    js.Global().Set("WasmHash", js.FuncOf(convertToUpper))
    <-done
}
func convertToUpper(this js.Value, args []js.Value) interface{} {
    strings.ToUpper(args[0].String())
    return strings.ToUpper(args[0].String())
}

done := make(chan struct{}, 0) and <-done is a Go channel. A Go channel is used for communication between concurrent functions.

js.Global().Set("WasmHash", hash) exposes the Go hash function to JavaScript.

The convertToUpper function takes a string as an argument, which is then typecasted using the .String() function from the syscall/js package. The strings.ToUpper(args[0].String()) line converts all arguments provided by JavaScript into an uppercase string and returns it as output of the function.

  1. The next step is to compile Chapter09/go-Wasm-example/cmd/Wasm/main.go using the following command:
    % GOOS=js GOARCH=Wasm go build -o static/main.Wasm cmd/Wasm/main.go

The secret recipe here is GOOS=js GOARCH=Wasm, which tells the Go compiler to compile for JavaScript as the target host and Wasm as the target architecture. Without this, the Go compiler will compile for the target OS and architecture as per your workstation specifications. You can find more about the possible values of GOOS and GOARCH at https://gist.github.com/asukakenji/f15ba7e588ac42795f421b48b8aede63.

The command will then produce the Wasm file with the main.Wasm name in the static folder.

  1. We also need to fetch and execute Wasm in the browser. Luckily, Go makes that possible with Wasm_exec.js.

The JavaScript file can be found in the GOROOT folder. To copy it to the static directory, use the following command:

% cp "$(go env GOROOT)/misc/Wasm/Wasm_exec.js" ./static
  1. We have Wasm and JavaScript to load and execute Wasm in the browser. We need to create an HTML page and then load JavaScript from there. You will find the sample HTML page at Chapter09/go-Wasm-example/static/index.html. You will find the following snippet in the HTML to load JavaScript and instantiate Wasm:
    <script src="Wasm_exec.js"></script>
<script>
    const go = new Go();
    WebAssembly.instantiateStreaming(fetch("main.Wasm"), go.importObject).then((result) => {
        go.run(result.instance);
    });
</script>
  2. As the last step, we need a web server. You can use nginx or a sample HTTP server package with the sample code at Chapter09/go-Wasm-example/cmd/webserver/main.go. Run the server using the following command:
    % go run ./cmd/webserver/main.go
Listening on http://localhost:3000/index.html
  3. Open http://localhost:3000/index.html in a browser and test that whatever lowercase letters you type in the text box are converted to uppercase:
Figure 9.2 – Go used to create Wasm

Figure 9.2 – Go used to create Wasm

This concludes the introduction to Wasm, and I hope you have acquired a basic understanding of Wasm after reading this section. In the next section, we will learn about how Wasm helps to extend the Istio data plane.

Extending the Istio data plane using Wasm

The main goal for Wasm was to enable high-performance applications on web pages, and hence Wasm was originally designed for execution in web browsers. There is a World Wide Web Consortium (W3C) working group for Wasm, whose details are available at https://www.w3.org/Wasm/. The working group manages the Wasm specification available at https://www.w3.org/TR/Wasm-core-1/ and https://www.w3.org/TR/Wasm-core-2/. Most internet browsers have implemented the specification, and you can find details for Google Chrome at https://chromestatus.com/feature/5453022515691520. Mozilla Foundation also maintains browser compatibility at https://developer.mozilla.org/en-US/docs/WebAssembly#browser_compatibility. When it comes to supporting the execution of Wasm on layer 4 and 7 proxies, most of the effort is recent. When executing Wasm on proxies, we need a way to communicate with the host environment. Similar to how web browsers are developed, Wasm should be written once, after which it should be able to run on any proxy.

Introducing Proxy-Wasm

For Wasm to communicate with the host environment and the development of Wasm to be agnostic of the underlying host environment, there is a Proxy-Wasm specification, also known as Wasm for proxies. The specification is made up of Proxy-Wasm ABIs, which are low-level. The specification is then abstracted in high-level languages, called Proxy-Wasm software development kits (SDKs), which are developer friendly and easy to understand and integrate with high-level language implementations. Every proxy also then implements a Proxy-Wasm ABI specification in the form of the Proxy-Wasm modules.

The concepts of Proxy-Wasm can be difficult to understand. To make it easy to digest them, let’s break them down into the following sections and go through them one by one.

Proxy-Wasm ABI

ABI is a low-level interface specification that describes how Wasm communicates with the VM and host. The specification details are available at https://github.com/proxy-Wasm/spec/blob/master/abi-versions/vNEXT/README.md, and the specification itself is available at https://github.com/proxy-Wasm/spec. To understand the API, it is best to go through some of the most commonly used methods of the ABI specification to appreciate what it does:

  • _start: This function needs to be implemented on Wasm and will be called when Wasm is loaded and initialized.
  • proxy_on_vm_start: This is called when the host machine starts the Wasm VM. Wasm can use this method to retrieve any configuration details of the VM.
  • proxy_on_configure: This is called when the host environment starts the plugin, which loads Wasm. Using this method, Wasm can retrieve any plugin-related configuration.
  • proxy_on_new_connection: This is a level 4 extension that is called when a TCP connection is established between the proxy and the client.
  • proxy_on_downstream_data: This is a level 4 extension that is called for each data chunk received from the client.
  • proxy_on_downstream_close: This is a level 4 extension that is called when the connection with downstream is closed.
  • proxy_on_upstream_data: This is a level 4 extension that is called for each data chunk received from upstream.
  • proxy_on_upstream_close: This is a level 4 extension that is called when the connection with upstream is closed.
  • proxy_on_http_request_headers: This is a level 7 extension that is called when HTTP request headers are received from the client.
  • proxy_on_http_request_body: This is a level 7 extension that is called when the HTTP request body is received from the client.
  • proxy_on_http_response_headers: This is a level 7 extension that is called when HTTP response headers are received from upstream.
  • proxy_on_http_response_body: This is a level 7 extension that is called when the HTTP response body is received from upstream.
  • proxy_send_http_response: This is also a level 7 extension that is implemented in the host environment, Envoy. Using this method, Wasm can instruct Envoy to send an HTTP response without actually calling the upstream services.

This list doesn’t cover all methods in the ABI, but we hope it gave you a good understanding of what the ABI is used for. The following diagram illustrates what we covered in this section:

Figure 9.3 – Proxy-wasm ABI

Figure 9.3 – Proxy-wasm ABI

If we analyze this diagram in the context of Envoy, we arrive at the following interpretation:

  • Native extensions execute in the order specified in the configuration.
  • There is also a native extension in Envoy for loading Wasm, specified at https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/Wasm/v3/Wasm.proto. The extension is responsible for loading and asking Envoy to execute Wasm.
  • Envoy executes Wasm on a VM.
  • During execution, Wasm can interact with the request, VM, and Envoy via the Proxy-Wasm ABI, and we saw some of those interaction points earlier in the section.
  • Once Wasm completes execution, the execution flows back to other native extensions defined in the configuration file.

While ABIs are elaborate, they are also very low-level and not programmer-friendly, who usually prefer writing code in high-level programming languages. In the following section, we will read about how the Proxy-Wasm SDK can solve this problem.

Proxy-Wasm SDK

Proxy-Wasm SDK is a higher-level abstraction of the Proxy-Wasm ABI and is implemented in various programming languages. Proxy-Wasm SDK complies with the ABI so that when creating Wasm, you don’t need to know about the Proxy-Wasm ABI. At the time of writing this chapter, there are SDKs of the Proxy-Wasm API in Go with TinyGo compiler, Rust, C++, and AssemblyScript. Similar to what we did for ABIs, we will pick SDKs for one of the languages and go through it to understand the correlation between the ABI and the SDK. So, let’s go through some of the functions in the Proxy-Wasm Go SDK to get a feel of them; the SDK is available at https://pkg.go.dev/github.com/tetratelabs/proxy-Wasm-go-SDK/proxyWasm.

First, you need to understand the various types defined in the SDK, so we have provided the following list of the fundamental ones:

  • VMContext: This corresponds to each Wasm VM. For every Wasm VM, there is one and only one VMContext. VMContext has the following methods:
    • OnVMStart(vmConfigurationSize int) OnVMStartStatus: This method is called when the VM is created. From within this method, Wasm can retrieve the VM configuration.
    • NewPluginContext(contextID uint32) PluginContext: This creates a plugin context for each plugin configuration.
  • PluginContext: This corresponds to each plugin configuration in the host. Plugins are configured at HTTP or network filters for listeners. Some of the methods in PluginContext are as follows:
    • OnPluginStart(pluginConfigurationSize int) OnPluginStartStatus: This is called for all plugins configured. Once the VM has been created, Wasm can retrieve the plugin configuration using this method.
    • OnPluginDone() bool: This is called when the host deletes PluginContext. If this method returns true, it signals to the host that PluginContext can be deleted, and false means that the plugin is in a pending state and cannot yet be deleted.
    • NewTcpContext(contextID uint32) TcpContext: This method creates TCPContext, corresponding to every TCP request.
    • NewHttpContext(contextID uint32) HttpContext: This method creates HTTPContext, corresponding to every HTTP request.
  • HTTPContext: This method is created by PluginContext for every HTTP stream. The following are some of the methods available in this interface:
    • OnHttpRequestHeaders(numHeaders int, endOfStream bool) Action: This method provides access to HTTP headers as part of the request stream.
    • OnHttpRequestBody(bodySize int, endOfStream bool) Action: This method provides access to data frames of the request body. It is called multiple times for every individual data frame in the request body.
    • OnHttpResponseHeaders(numHeaders int, endOfStream bool) Action: This method provides access to response headers.
    • OnHttpResponseBody(bodySize int, endOfStream bool) Action: This method provides access to response body frames.
    • OnHttpStreamDone(): This method is called before the deletion of HTTPContext. From this method, Wasm can access all information about the request and response phases of the HTTP connection.

Among other types to read about is TCPContext. We have not covered all methods and types available in the SDK; you can find the complete list along with details at https://pkg.go.dev/github.com/tetratelabs/[email protected]/proxyWasm/types#pkg-types.

With this overview in mind, let’s write a Wasm to inject a custom header in the response of the envoydummy Pod. Please note that in the Customizing the data plane using the Envoy filter section, we used EnvoyFilter to patch Istio and applied a Lua filter with inline code to inject headers to requests bound for the httpbin Pod.

Create the chapter09-temp namespace with istio-injection disabled:

% kubectl create ns chapter09-temp
namespace/chapter09-temp created

Run envoydummy to check that it is working as expected:

% kubectl apply -f Chapter09/01-envoy-dummy.yaml
namespace/chapter09-temp created
service/envoydummy created
configmap/envoy-dummy-2 created
deployment.apps/envoydummy-2 created

Forward the ports so that you can test locally:

% kubectl port-forward svc/envoydummy 18000:80 -n chapter09-t
emp
Forwarding from 127.0.0.1:18000 -> 10000

Then, test the endpoint:

% curl  localhost:18000
V2----------Bootstrap Service Mesh Implementation with Istio----------V2%

So, we have verified that envoydummy is working. The next step is to create Wasm to inject headers into the response. You will find the source code at Chapter09/go_Wasm_example_for_envoy.

There is only one main.go file in the Go module, and the following are the key parts of the code:

The entry point in the Go module is the main method. In the main method, we are setting up the Wasm VM by calling SetVMContext. The method is described in the Entrypoint.go file at https://github.com/tetratelabs/proxy-wasm-go-sdk/tree/main/proxywasm. The following code snippet shows the main method:

func main() {
    proxyWasm.SetVMContext(&vmContext{})
}

The following method injects a header into the response headers:

func (ctx *httpHeaders) OnHttpResponseHeaders(numHeaders int, endOfStream bool) types.Action {
    if err := proxyWasm.AddHttpResponseHeader("X-ChapterName", "ExtendingEnvoy"); err != nil {
        proxyWasm.LogCritical("failed to set response header: X-ChapterName")
    }
    return types.ActionContinue
}

Also, notice AddHttpResponseHeader, which is defined at https://github.com/tetratelabs/proxy-Wasm-go-SDK/blob/v0.20.0/proxyWasm/hostcall.go#L395.

The next step is to compile the Go module for Wasm, for which we will need to use TinyGo. Please note that we cannot use the standard Go compiler due to a lack of support for the Proxy-Wasm Go SDK.

Install TinyGo for your host OS by following the instructions at https://tinygo.org/getting-started/install/macos/.

Using TinyGo, compile the Go module with Wasm using the following command:

% tinygo build -o main.Wasm -scheduler=none -target=wasi main.go

Once the Wasm file is created, we need to load the Wasm file into configmap:

% kubectl create configmap 01-Wasm --from-file=main.Wasm -n chapter09-temp
configmap/01-Wasm created

Modify the envoy.yaml file to apply Wasm filters and load Wasm from configmap:

http_filters:
              - name: envoy.filters.http.Wasm
                typed_config:
                  "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                  type_url: type.googleapis.com/envoy.extensions.filters.http.Wasm.v3.Wasm
                  value:
                    config:
                      vm_config:
                        runtime: "envoy.Wasm.runtime.v8"
                        code:
                          local:
                            filename: "/Wasm2/main.Wasm"
              - name: envoy.filters.http.router
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

We specify envoy in the config to use the v8 runtime for running Wasm. The changes are also available at Chapter09/02-envoy-dummy.yaml. Apply the changes, as shown here:

% kubectl apply -f Chapter09/02-envoy-dummy.yaml
service/envoydummy created
configmap/envoy-dummy-2 created
deployment.apps/envoydummy-2 created

Forward the port 80 to 18000:

% kubectl port-forward svc/envoydummy 18000:80 -n chapter09-temp

Test the endpoint to check whether Wasm injected the response header:

% curl -v localhost:18000
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 72
< content-type: text/plain
< x-chaptername: ExtendingEnvoy
* Connection #0 to host localhost left intact
V2----------Bootstrap Service Mesh Implementation with Istio----------V2%

Hopefully, this section gave you confidence on how to create Wasm that is compliant with Proxy-Wasm and how to apply it to Envoy. We suggest you do more hands-on exercises by looking at examples available at https://github.com/tetratelabs/proxy-Wasm-go-SDK/tree/main/examples.

Before we conclude this section, let’s also check how Wasm is compliant with the Proxy-Wasm ABI. For that, we will install the Wasm Binary Toolkit (WABT) available at https://github.com/WebAssembly/wabt. On MacOS, it is simple to install using brew:

% brew install wabt

WABT provides various methods to manipulate and introspect Wasm. One such tool, Wasm-objdump, prints information about a Wasm binary. Using the following command, you can print a list of all functions that become accessible to the host environment once Wasm has been instantiated:

% Wasm-objdump main.Wasm --section=export -x.

You will notice the output is a list of functions that are defined in the Proxy-Wasm ABI.

Important note

To do the cleanup, you can use the following command:

% kubectl delete ns chapter09-temp

That completes the section on Proxy-Wasm, and we hope you now understand how to create Proxy-Wasm-compliant Wasm using the Go SDK. In the next section, we will deploy Wasm in Istio.

Wasm with Istio

In this section, we will extend the Istio data plane using Wasm that we built in the previous section. We will be using Istio’s WasmPlugin API, and we will go into the details of this plugin once we have configured it for the httpbin application:

  1. The first step is to upload main.Wasm created in the Go module available at Chapter09/go_Wasm_example_for_envoy to an HTTPS location. You can make use of AWS S3 or something similar for that purpose; another option is to use an OCI registry such as Docker Hub. To complete this exercise, I uploaded main.Wasm to AWS S3. The HTTPS location of the S3 bucket hosting the file is https://anand-temp.s3.amazonaws.com/main.Wasm. Please note that for security reasons, the link might not be accessible to you while reading this book, but I am sure you can manage to create your own S3 buckets or Docker registry.
  2. The second step is to deploy the httpbin application, which is already available at Chapter09/01-httpbin-deployment.yaml:
    % kubectl apply -f Chapter09/01-httpbin-deployment.yaml

Check the response of the following commands and observe the headers added during the request:

% curl -H "Host:httpbin.org" http://a816bb2638a5e4a8c990ce790b47d429-1565783620.us-east-1.elb.amazonaws.com/get
  1. After this, we will apply the following changes using WasmPlugin:
    apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: addheaders
  namespace: chapter09
spec:
  selector:
    matchLabels:
      app: httpbin
  url: https://anand-temp.s3.amazonaws.com/main.Wasm
  imagePullPolicy: Always
  phase: AUTHZ

Apply WasmPlugin using the following command:

% kubectl apply -f Chapter09/01-Wasmplugin.yaml
Wasmplugin.extensions.istio.io/addheaders configured

We will read more about WasmPlugin after step 5. For now, let’s check the response headers from httpbin:

% curl --head -H "Host:httpbin.org" http://a816bb2638a5e4a8c990ce790b47d429-1565783620.us-east-1.elb.amazonaws.com/get

You will notice that, as expected, we have x-chaptername: ExtendingEnvoy in the response.

  1. Let’s create another Wasm to add a custom header to request so that we can see it in the response of the httpbin payload. There is a Wasm already created in Chapter09/go_Wasm_example_for_istio for this purpose. Notice the OnHTTPRequestHeaders function in main.go:
    func (ctx *httpHeaders) OnHttpRequestHeaders(numHeaders int, endOfStream bool) types.Action {
    if err := proxyWasm.AddHttpRequestHeader("X-Chapter", "Chapter09"); err != nil {
        proxyWasm.LogCritical("failed to set request header: X-ChapterName")
    }
    proxyWasm.LogInfof("added custom header to request")
    return types.ActionContinue
}

Compile that into Wasm and copy it to the S3 location. There is also another Istio config file available at Chapter09/02-Wasmplugin.yaml, which deploys this Wasm:

apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: addheaderstorequest
  namespace: chapter09
spec:
  selector:
    matchLabels:
      app: httpbin
  url: https://anand-temp.s3.amazonaws.com/AddRequestHeader.Wasm
  imagePullPolicy: Always
  phase: AUTHZ
  1. After applying the changes, test the endpoints, and you will find that both Wasm have executed, adding a header in the response as well as one in the request, which is reflected in the httpbin response. The following is a shortened version of the response:
    % curl -v -H "Host:httpbin.org" http://a816bb2638a5e4a8c990ce790b47d429-1565783620.us-east-1.elb.amazonaws.com/get
< HTTP/1.1 200 OK
……
< x-chaptername: ExtendingEnvoy
<
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.79.1",
…..,
    "X-Chapter": "Chapter09",
…
  },
  "origin": "10.10.10.216",
  "url": "http://httpbin.org/get"
}

In steps 3 and 4, we used WasmPlugin to apply Wasm on the Istio data plane. The following are the parameters we configured in WasmPlugin:

  • selector: Specify the resource on which the Wasm should be applied in the selector field. It can be the Istio gateway and Kubernetes Pods. You provide labels that must match the workload on whose Envoy sidecar the Wasm configuration will be applied. In the examples we implemented, we applied the app:httpbin label, which corresponds to the httpbin Pod.
  • url: This is the location where the Wasm file is available to download. We provided the HTTP location, but OCI locations are also supported. The default value is oci://, used for referencing OCI images. To reference file-based locations, use file://, which is used for referencing Wasm files present locally within the proxy container.
  • imagePullPolicy: The possible values for this are the following:
    • UNSPECIFIED_POLICY: This is the same as IfNotPresent unless the URL points to an OCI image with the latest tag. In that case, this field will default to Always.
    • Always: We will always pull the latest version of an image from the location specified in the URL.
    • IfNotPresent: Use this to pull Wasm only if the requested version is unavailable locally.
  • phase: The possible values for this are the following:
    • UNSPECIFIED_PHASE: This means the Wasm filter will be inserted at the end of the filter chain.
    • AUTHN: This inserts the plugin before the Istio authentication filters.
    • AUTHZ: This inserts the plugin between the authentication and authorization filters.
    • STATS: This inserts the plugin after the authorization filter but before the stats filter.

We have described the values we used in the example, but various fields can be configured in WasmPlugin; you can find the detailed list at https://istio.io/latest/docs/reference/config/proxy_extensions/Wasm-plugin/#WasmPlugin.

For production deployment, we definitely suggest you use the sha256 field to ensure the integrity of the Wasm modules.

Istio provides a reliable, out-of-the-box distribution mechanism for Wasm by leveraging the xDS proxy inside istio-agent and Envoy’s Extension Configuration Discovery Service (ECDS). Details about ECDS are available at https://www.envoyproxy.io/docs/envoy/latest/configuration/overview/extension.

After applying WasmPlugin, you can check the istiod logs for ECDS entries:

% kubectl logs istiod-56fd889679-ltxg5 -n istio-system

You will find log entries similar to the following:

10-18T12:02:03.075545Z     info ads  ECDS: PUSH for node:httpbin-7bffdcffd-4zrhj.chapter09 resources:1 size:305B

Istio makes an ECDS call to istio-proxy about applying the WasmPlugin. The following diagram describes the process of applying Wasm via the ECDS API:

Figure 9.4 – Distributing Wasm to the Istio data plane

Figure 9.4 – Distributing Wasm to the Istio data plane

The istio-agent deployed alongside Envoy intercepts the ECDS call from istiod. It then downloads the Wasm module, saves it locally, and updates the ECDS configuration with the path of the downloaded Wasm module. If the WASM modules are not accessible to Istio-agent, it will reject the ECDS update. You will be able to see ECDS update failure in the istiod logs.

This concludes this section, and I hope it arms you with enough knowledge to start applying Wasm to your production workload.

Summary

In this chapter, we read about Wasm and its use. We learned about how Wasm is used on the web due to its high performance, and we also familiarized ourselves with how to build Wasm using Go and use it from a web browser using JavaScript. Wasm is also becoming a popular choice on the server side, especially among network proxies such as Envoy.

To get a standardized interface for implementing Wasm for proxies, there are the Proxy-Wasm ABI specifications which are low-level specifications describing the interface between Wasm and the proxy hosting the Wasm. Wasm for Envoy needs to be Proxy-Wasm compliant, but the Proxy-Wasm ABIs are difficult to work with; the Proxy-Wasm SDKs are much easier to work with. At the time of writing this chapter, there are many programming languages in which Proxy-Wasm SDK implementations are available, of which Rust, Go, C++, and AssemblyScript are among the most popular. We made use of the Envoy Wasm filter to configure a Wasm on an Envoy HTTP filter chain. We then built a few simple Wasm examples to manipulate request and response headers and deployed them on Istio using WasmPlugin. Wasm is not the only option to extend the Istio data plane, and there is another filter called EnvoyFilter, which can be used to apply the Envoy configuration as a patch on top of the Envoy configuration created by Istiod.

The next chapter is very interesting as we will learn about how to deploy an Istio Service Mesh for non-Kubernetes workloads.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.147.80.3