Routes

To enable traffic from external clients, apps require a specific URL, known as a route. For example, developers can create a route by mapping the route myapp.shared-cf-domain.com to the app myapp.

You can construct routes via a combination of the following:

• Domain

• Host

• Port

• Context path

Route construction is explained further in a later section.

Each Cloud Foundry instance can have a single default domain and further additional domains that can be shared across organizations (Orgs) in a single Cloud Foundry instance. Routes are then based on those domains. Routes belong to a Space, and only apps in the same Space as that route can be mapped to it. A developer of one Space cannot create or use a route if it already exists in another Space. For this reason, many developers place app-name-${random-word} in their route to ensure that their app route is unique during the dev/test phase.

One app, one route, multiple app instances

You can map an individual app to either a single route or, if desired, multiple routes. Because apps can have multiple app instances (ActualLRPs), all accessed by the single route, each route has an associated array of host:port entries stored in a routing table on the GoRouter. Figure 7-1 shows that the host is the Diego Cell machine running the LRP in a container and the port corresponds to a dedicated host port for that container. The router regularly recomputes new routing tables based on the Cell IP addresses and the host-side port numbers for the containers, as illustrated in Figure 7-1. In a cloud-based distributed environment, both desired and actual state can rapidly change; thus, it is important to dynamically update routes both periodically and immediately in response to state changes.

Figure 7-1. Cell-to-container port mapping

One app, multiple routes

You can also map an individual app to multiple routes, granting multiple URLs access to that app. This capability is illustrated in Figure 7-2.

Figure 7-2. One app mapped to two different routes

Mapping more than one route to an app can be a valuable feature for establishing techniques such as blue/green deployments. You can read more about blue/green deployments in the Cloud Foundry Documentation.

Several apps, one route

In addition to being able to map all identical app instances to a single route, as depicted in Figure 7-3, you can also map independent apps to a single route. This results in the GoRouter load-balancing requests for the route across all instances of all mapped apps, as demonstrated in Figure 7-3. This feature is also important for enabling the use of blue/green and canary deployment strategies. It is also used when dealing with different apps that must work collectively with a single entry point; for example, microservices architecture (discussed shortly).

Figure 7-3. Routing mechanism allowing for several apps mapped to the same route

Hostnames

Cloud Foundry provides the option of creating a route with a hostname. A hostname is a name that can be explicitly used to specify an app, as shown in the following code:

$ cf create-route my-space shared-domain.com --hostname myapp

This creates the unique route myapp.shared-domain.com comprising the host that is prepended on to the shared domain.

At this stage, all we have done is reserved the route so that the route is not used in another Space. The app is only routable by this route after it is mapped to the route, as in the following:

$ cf mapp-route app-name domain hostname

Note that although this route is created for the Space my-space, the Space is not featured in the route name.

Routes created for shared domains must always use a hostname. Alternatively, you can create a route without a hostname. This approach creates a route for the domain itself and is permitted for private domains only. You can create private domains as follows:

$ cf create-route my-space private-domain.com

This example creates a route in the Space my-space from domain private-domain.com. After configuring your DNS, Cloud Foundry will route requests for http(s)://private-domain.com or any context path under that URL (e.g., private-domain.com/app1) to apps that are mapped to that route or context path. Any subdomain (e.g., *foo*.private-domain.com) will fail unless additional routes are specified for that subdomain and then mapped to a subsequent app.

You can use wildcard routes here as a catch-all (e.g., *.private-domain.com); for example, to serve a custom 404 page or a specific homepage.

Domains

Cloud Foundry’s use of the terms domain, shared domain, and private domain differ from their common use:

• Domains provide a namespace from which to create routes.

• Shared domains are available to users in all Orgs, and every Cloud Foundry instance requires a single default shared domain.

• Private domains allow users to create routes for privately registered domain names.

As discussed in “Hostnames”, by default apps are assigned a route with a hostname my-app, and the app domain my-app.apps.cf-domain.com, resulting in the route my-app.apps.cf-domain.com.

The presence of a domain in Cloud Foundry indicates that requests for any route created from that domain will be routed to a specific Cloud Foundry instance. This provision requires a DNS to be configured to resolve the domain name to the IP address of a load balancer that fronts traffic entering Cloud Foundry.¹

The recommended practice is to have a wildcard canonical name (CNAME) that you can use as a base domain for other subdomains. An example of a wildcard CNAME is *.cf-domain.com.

To use a subdomain of your registered domain name with apps on Cloud Foundry, configure the subdomain as a CNAME record with your DNS provider, pointing at any shared domain offered in Cloud Foundry.

When installing Cloud Foundry, it is good practice to have a subsequent system domain and one or more app domains; for example, system.cf-domain.com for your system domain, and apps.cf-domain.com for (one of) your app domain(s). Multiple app domains can be advantageous and are discussed further later.

The system domain allows Cloud Foundry to receive requests for and send communication between its internal components (like the UAA and Cloud Controller). Cloud Foundry itself can run some of its components as apps. For example, a service broker can deploy an app. The app domain guarantees that requests for routes based off that domain will go to a specific Cloud Foundry instance.

If we had only one combined system and app domain, there would be no separation of concerns. A developer could register an app domain name that infers it is a potential system component. This can cause confusion to the Cloud Foundry operator with respect to what are system apps and what are developer-deployed apps. Moreover, mapping an arbitrary app to a system component can cause fundamental system failures, as we will explore in “Scenario Five: Route Collision”. For these reasons, it is recommended that you always have at least one default system domain and default app domain per environment.

All system components should register routes that are extensions of the system domain; for example, login.system.cf-domain.com, uaa.system.cf-domain.com, doppler.system.cf-domain.com, and api.system.cf-domain.com.

Context Path Routing

Context path routing allows for routing to be based not only on the route domain name (essentially the host header), but also the path specified in the route’s URL. The GoRouter inspects the URL for additional context paths and, upon discovery, can then route requests to different apps based on that path. Here are a couple of examples:

• myapp.mycf-domain.com/foo can be mapped to the foo app.

• myapp.mycf-domain.com/bar can be mapped to the bar app.

This is important when dealing with a microservices architecture. With microservices, a single “big-A” apps can be comprised of a suite of smaller microservices apps, as shown in Figure 7-4. The smaller applications often require the same single top-level route myapp.mycf-domain.com to offer a single entry point for the user. Context path routing allows different microservices apps (e.g., foo and bar), all served by the same parent route, to provide support for different paths in the URL, based on their unique context path.

With context path-based routing, you can also independently scale up or down those portions of your big-A app that are being heavily utilized.

Figure 7-4. Routing using a single route and context paths to target a specific app

Setting Request Header Fields

When a client connects to a web server through an HTTP proxy or load balancer, it is possible to identify the originating IP address and the send protocol by setting the X-Forwarded-For and X-Forwarded-Proto request header fields, respectively. These headers must be set on the load balancer that fronts the traffic coming into Cloud Foundry. HTTP traffic passed from the GoRouter to an app will include these headers. If an app wants to behave differently based on the transport protocol used, it can inspect the headers to determine whether traffic was received over HTTP or HTTPS.

X-Forwarded-For: 203.0.56.67

X-Forwarded-Proto: https

WebSocket Upgrades

WebSockets is a protocol providing bidirectional communication over a single, long-lived TCP connection. It is commonly implemented by web clients and servers. WebSockets are initiated via HTTP as an upgrade request. The GoRouter supports WebSocket upgrades, holding the TCP connection open with the selected app instance.

Supporting WebSockets is important because the Firehose (the endpoint of all aggregated and streamed app logs) is a WebSockets endpoint that streams all event data originating from a Cloud Foundry deployment. To support WebSockets, operators must configure their load balancer to pass WebSockets requests through as opaque TCP connections. WebSockets are also vital for app log streaming, allowing developers to view their app logs.

Some load balancers are unable to support listening for both HTTP and TCP traffic on the same port. Take, for example, ELB offered by AWS. ELB can listen on a port in either HTTP(s) mode or TCP mode. To pass through a WebSocket request, ELB must be in TCP mode. However, if ELB is terminating TLS requests on 443 and appending the XFF and XFP headers, it must be in HTTP mode. Therefore, ELB cannot handle WebSockets on the same port. In this scenario, you can do the following:

• Configure your load balancer to listen for WebSocket requests on a nonstandard port (e.g., 8443) and then forward WebSocket requests to this port in TCP mode to the GoRouter on port 80 or 443. App clients must make WebSockets upgrade requests to this port.

• Add a second load balancer listening in TCP mode on standard port 80. Configure DNS with a dedicated hostname for use with WebSockets that resolves to the new load balancer serving port 80 in TCP mode.

The PROXY Protocol

As just described, WebSockets require a TCP connection; however, when using TCP mode, load balancers will not add the XFF HTTP protocol headers, so you cannot identify your clients. Another solution for client identification is to use the Proxy protocol. This protocol allows your load balancer to add the Proxy protocol header so that your apps can still identify your clients even when you use TCP mode at the load balancer.

Another scenario is to terminate TLS with a component that does not support HTTP and operates only in TCP mode. Therefore, an HTTP connection is then passed on to the GoRouter.

A point to note is that some load balancers in TCP mode will not give you HTTP multiplexing and pipelining. This could cause a performance problem unless you have a content delivery network (CDN) in front.

TLS Termination and IPSec

Although Cloud Foundry is a distributed system, conceptually we can consider it as a software appliance. It is designed to sit in a dedicated network with defined egress and ingress firewall rules. For this reason, if the load balancer sits within or on the edge of the private network, it can handle TLS decryption and then route traffic unencrypted to the GoRouter.

However, if the load balancer is not dedicated to Cloud Foundry and it is located on a general-purpose corporate network, it is possible to pass the TLS connection on to the GoRouter for decryption. To implement this, you must configure your load balancer to re-sign the request between the load balancer and the GoRouter using your wildcard certificate. You will also need to configure the GoRouter with your Cloud Foundry certificates.

There might be situations in which you require encryption directly back to the app and data layer. For these scenarios, you can use the additional IPSec BOSH add-on that provides encrypted traffic between every component machine.

Routing Table

The router uses a routing table to keep track of available apps. This table contains an up-to-date list of all the routes to the Cells and containers that are currently running ActualLRPs. As described earlier, you can map multiple routes to an app and map multiple apps to a route. The routing table keeps track of this mapping. It provides the source of truth for all routing, dynamically checking for and pruning dead routes to avoid 404 errors.

Diego uses its Route-Emitter component to consume event streams from the Diego Database—the BBS—and then pushes the route updates to the router. Additionally, the Route-Emitter performs a bulk lookup operation against its database every 20 seconds to fetch all the desired and actual routes.

The GoRouter then recomputes a new routing table based on the IP addresses of each Cell machine and the host-side port numbers for the Cell’s containers. This ensures that the routing table information is up to date in the event that an app fails.

Router and Route High Availability

GoRouters should be clustered both for resiliency and for handling a large number of concurrent client connections.

When GoRouters come on line, they send router.start messages informing Route-Emitters that they are running. Route-Emitters are monitoring desired and actual LRP events in the BBS to establish and map routes to app instances. They compute the routing table and send this table to the GoRouter via NATS at regular intervals.

This ensures that new GoRouters update their routing table and synchronize with existing GoRouters. Routes will be pruned from the routing table if an app connection goes stale. To maintain an active route, the route must be updated by default at least every two minutes.

An important implementation consideration for the GoRouter is that because it uses NATS, it must be brought online after the NATS component in order to function properly.

Router Instrumentation and Logging

Like the other Cloud Foundry components, the GoRouter provides logging through its Metron agent. In addition, a /routes endpoint returns the entire routing table as JSON. Because of the nature of the data present in /routes, the endpoint requires HTTP basic authentication credentials served on port 8080. These credentials are obtained from the deployment manifest under the router job:

    status:
      password: some_password
      port: 8080
      user: some_user

The credentials can also be obtained from the GoRouter VM at /var/vcap/jobs/gorouter/config/gorouter.yml.

Each route contains an associated array of host:port entries, which is useful for debugging:

$ curl -vvv "http://some_user:[email protected]:8080/routes"

In addition to the routing table endpoint, the GoRouter offers a healthcheck endpoint on /health:

$ curl -v "http://10.0.32.15:8080/health"

This is particularly useful when performing healthchecks from a load balancer. This endpoint does not require credentials and should be accessed at port 8080. Because load balancers typically round-robin the GoRouters, by regularly checking the GoRouter health, they can avoid sending traffic to GoRouters that are temporarily not responding.

You can configure the GoRouter logging levels in the Cloud Foundry deployment manifest. The meanings of the router’s log levels are as follows:

fatal

An error has occurred that makes the current request unserviceable; for example, the router cannot bind to its TCP port, or a Cloud Foundry component has published invalid data to the GoRouter.

warn

An unexpected state has occurred. For example, the GoRouter tried to publish data that could not be encoded as JSON.

info, debug

An expected event has occurred. For example, a new Cloud Foundry component was registered with the GoRouter, and the GoRouter has begun to prune routes for stale containers.

TCP Routing Management Plane

The TCP routing management plane (see Figure 7-5) has similar functionality to the HTTP routing management plane. There is a Route-Emitter listening to events in Diego’s BBS. For example, whenever a new app is created or an app is moved or scaled, the BBS is updated.

The emitter detects these events, constructs the routing table, and then sends this table on to the routing API. The routing API effectively replaces the need for NATS; it maintains the routing table and then makes the configuration available across a tier of TCPRouter instances. Therefore, TCPRouters receive their configuration from the routing API and not via NATS. Both the TCPRouter and the Route-Emitter receive their configuration from both periodic bulk fetches and real-time server sent events.

TCP routing introduces some complexity through additional NAT involving different ports at different tiers, as illustrated in Figure 7-6. There is a route port to which clients send requests. This port is reserved on the TCPRouter when you create a TCP route. Behind the scenes, the TCPRouter makes a translation between that route port and the app instances. Containers include app ports (that default to 8080). These ports are not directly accessible via the TCPRouter, because containers are running in a Cell providing an additional NAT for the container. Therefore, the ports made known to the TCPRouter are the Cell ports (the backend port).

Figure 7-6. TCPRouter port mappings from the load balancer through to the app

TCPRouter Configuration Steps

Here are the deployment steps required for configuring TCP routing:

1. Choose a domain name from which developers will create their TCP routes.

2. Configure DNS to point to that domain name via the load balancer.

3. Choose how many TCP routes the platform should support based on the reserved ports on the TCPRouter.

4. Configure the load balancer to listen on the port range² and then forward requests for that port range and domain to the TCPRouters.

5. Configure the TCPRouter group within Cloud Foundry with the same port range.

6. Create a domain associated with that TCPRouter group.

7. Configure a quota to entitle Orgs to create TCP routes.

Route Service Workflow

All requests arrive (1) via the external load balancer, which passes traffic on to the GoRouter (2). The GoRouter checks for a bound route service for a route, and if no service exists, it will simply pass traffic on to the appropriate LRP (3). If a route service does exist for the route, the router will then pass that traffic on to the service that is bound to the route.

Before passing the request on to the service, the router generates an encrypted short-lived message to include both the requested route and the route service GUID. The router then appends this message to the request header and forwards the request to the bound route service. After the specific route service has undertaken its work (e.g., header modification or rate limiting) the service can do one of two things:

• Respond directly to the request (e.g., serve an access-denied message if acting as an app firewall)

• Pass the traffic to the app

The route service passes traffic to the app (4) by resolving via DNS back to the load balancer (5), then to the router (6), and then to the app. The response traffic then follows that same flow backward to return a response to the client (7/8/9/10/11/12). This allows the service to do further modification on the returned response body if required. Figure 7-8 provides an overview of the architecture.

Figure 7-8. Route service workflow showing the redirection of app traffic to the route service before being directed back to the app via the load balancer (note that the return flow retraces the same path in reverse)

Route Service Use Cases

You can consider any use cases that can be on the request path as eligible for a route service. Here are some examples:

• Gateway use cases such as rate limiting, metering, and caching

• Security use cases such as authentication, authorization, auditing, fraud detection, and network sniffing

• Analytics use cases such as monetization, chargeback, and utilization

• Mobile backend as a service (MBaaS) such as push notifications and data services

In line with the rest of Cloud Foundry, the key goal of routing services is to increase app velocity. Without the developer self-service that route services provide, most organizations are left with the pain of ticketing systems and extra configuration to achieve these use-case capabilities.