Undifferentiated Heavy Lifting

Cloud Foundry is a platform for running applications and one-off tasks. The essence of Cloud Foundry is to provide companies with the speed, simplicity, and control they need to develop and deploy applications. It achieves this by undertaking many of the burdensome boilerplate responsibilities associated with delivering software. These types of responsibilities are referred to as undifferentiated heavy lifting, tasks that must be done—for example, container orchestration or application placement—but that are not directly related to the development of revenue-generating software. The following are some examples of undifferentiated heavy lifting:

• Provisioning VMs, OSs, middleware, and databases

• Application runtime configuration and memory tuning

• User management and SSO integration

• Load balancing and traffic routing

• Centralized log aggregation

• Scaling

• Security auditing

• Providing fault tolerance and resilience

• Service discovery

• Application placement and container creation and orchestration

• Blue/green deployments with the use of canaries

If you do not have a platform to abstract the underlying infrastructure and provide the aforementioned capabilities, this additional burden of responsibility remains yours. If you are spending significant time and effort building bespoke environments for shipping software, refocusing investment back into your core business will provide a huge payoff. Cloud Foundry allows enterprises to refocus effort back into the business by removing as much of the undifferentiated heavy lifting as possible.

The Cloud Operating System

As an application platform, Cloud Foundry is infrastructure-agnostic, sitting on top of your infrastructure of choice. As depicted in Figure 2-1, Cloud Foundry is effectively a cloud-based operating system that utilizes cloud-based resources, which are hidden and abstracted away from the end user. As discussed in Chapter 1, in the same way that the OS on your phone, tablet, or laptop abstracts the underlying physical compute resource, Cloud Foundry abstracts the infrastructure’s compute resource (specifically virtual storage, networking, RAM, and CPU). The net effect is that Cloud Foundry serves both as a standard and efficient way to deploy applications and services across different cloud-computing environments. Conversely, if you are stuck with directly using IaaS–specific APIs, it requires knowledge of the developer patterns and operations specific to the underlying IaaS technology, frequently resulting in applications becoming tightly coupled to the underlying infrastructure.

Figure 2-1. Cloud Foundry layers forming a cloud-based OS

Do More

Historically the long pole of application delivery, the part on the critical path that blocks progress, has been the IT department. This results in a concept I call server hugging, whereby developers hold on to (and hug) a plethora of VMs just in case they need them again someday.

Nowadays, businesses no longer need to be constrained by lengthy IT processes or organizational silos. Cloud Foundry provides a contractual promise to allow businesses to move with velocity and establish a developer–feedback loop so that they can tightly align products to user expectations. With Cloud Foundry, product managers get their business back and IT engineers can focus on more interesting issues and get to eat dinner at home.

Platforms are concerned not only with providing environments and middleware for running applications. For example, Cloud Foundry takes on the responsibility of keeping applications up and running in the face of failures within the system. It also provides security, user administration, workload scheduling, and monitoring capabilities. Onsi Fakhouri, Pivotal’s Vice President of Research and Development, famously tweeted this haiku:

Here is my source code,
run it on the cloud for me.
I do not care how!

Onsi’s quote captures the essence of Cloud Foundry’s do more capability. Cloud Foundry is about doing more on behalf of both the developer and operator so that they can focus on what really differentiates the business. This characteristic is seen all throughout the Cloud Foundry ecosystem. You can take a similar approach with BOSH, Cloud Foundry’s release-engineering system, and state, “Here are my servers, make them a Cloud Foundry. I do not care how!”

The Application as the Unit of Deployment

Traditionally, deploying application code required provisioning and deploying VMs, OSs, and middleware to create a development environment for the application to run in. After that environment was provisioned, it required patching and ongoing maintenance. New environments were then created as the application moved through the deployment pipeline.

Early incarnations of platforms centered on middleware: defining complex topology diagrams of application servers, databases, and messaging engines into which you could drop your application. When this topology diagram (or blueprint) was defined, you then specified some additional configuration such as IP addresses and ports to bring the defined topology and applications into existence. Although this was a step in the right direction, from a developer’s perspective there was still a layer of complexity that you needed to configure for each deployment.

Cloud Foundry differs from traditional provisioning and orchestration engines in a fundamental way: it uses middleware and infrastructure directly, allowing streamlined development through self-service environments. Developers can build, deploy, run, and scale applications on Cloud Foundry without having to be mindful of the specific underlying infrastructure, middleware, and container implementation.

Cloud Foundry allows the unit of deployment, i.e., what you deploy to run your application, to be isolated to just the application itself. Even though there are some benefits to encapsulating both your application and dependencies as a precomposed container image, I still believe it is more secure and more efficient to keep just the application as the unit of deployment and allow the platform to handle the remaining concerns. The trade-offs between both approaches are discussed further in Chapter 9; however, the benefit of Cloud Foundry is that it supports both approaches. Buildpacks are discussed at length in that chapter, but for now, it is enough to know that buildpacks provide the framework and runtime support for your applications. A specific buildpack is used to package your application with all of its dependencies. The resulting staged application is referred to as a droplet.

On-boarding developers is easy; they can deploy applications to Cloud Foundry using existing tool chains with little to no code modification. It enables the developer to remove the cost and complexity of configuring infrastructure for their applications. Using a self-service model, developers can deploy and scale applications without being directly locked into the IaaS layer.

Because developers no longer need to concern themselves with, for example, which application container to use, which version of Java, and which memory settings or garbage-collection (GC) policy to employ, they can simply push their applications to Cloud Foundry, and the applications run. This allows developers to focus on delivering applications that offer business value. Applications can then be bound to a wide set of backing services that are available on demand.

Using cf push Command to Deploy

Cloud Foundry provides several ways for a user to interact with it, but the principal avenue is through its CLI. The most renowned CLI command often referenced by the Cloud Foundry community is $ cf push.

You use the cf push command to deploy your application. It has demonstrably improved the deployment experience. From the time you run cf push to the point when the application is available, Cloud Foundry performs the following tasks:

• Uploads and stores application files

• Examines and stores application metadata

• Stages the application by using a buildpack to create a droplet

• Selects an appropriate execution environment in which to run the droplet

• Starts the AI and streams logs to the Loggregator

This workflow is explored in more depth in Chapter 6.

Staging

Although it is part of the cf push workflow, staging is a core Cloud Foundry concept. Cloud Foundry allows users to deploy a prebuilt Docker image or an application artifact (source code or binaries) that has not yet been containerized. When deploying an application artifact, Cloud Foundry will stage the application on a machine or VM known as a Cell, using everything required to compile and run the apps locally, including the following:

• The OS stack on which the application runs

• A buildpack containing all languages, libraries, dependencies, and runtime services the app uses

The staging process results in a droplet that the Cell can unpack, compile, and run. You can then run the resulting droplet (as in the case of a Docker image) repeatedly over several Cells. The same droplet runs the same app instances over multiple Cells without incurring the cost of staging every time a new instance is run. This ability provides deployment speed and confidence that all running instances from the same droplet are identical.

Self-Service Application Life Cycle

In most traditional scenarios, the application developer and application operator typically perform the following:

• Develop an application

• Deploy application services

• Deploy an application and connect (bind) it to application services

• Scale an application, both up and down

• Monitor an application

• Upgrade an application

This application life cycle is in play until the application is decommissioned and taken offline. Cloud Foundry simplifies the application life cycle by offering self-service capabilities to the end user. Adopting a self-service approach removes hand-offs and potentially lengthy delays between teams. For example, the ability to deploy an application, provision and bind applications to services, scale, monitor, and upgrade are all offered by a simple call to the platform.

With Cloud Foundry, as mentioned earlier, the application or task itself becomes the single unit of deployment. Developers just push their applications to Cloud Foundry, and those applications run. If developers require multiple instances of an application to be running they can use cf scale to scale the application to N number of AIs. Cloud Foundry removes the cost and complexity of configuring infrastructure and middleware per application. Using a self-service model, users can do the following:

• Deploy applications

• Provision and bind additional services, such as messaging engines, caching solutions, and databases

• Scale applications

• Monitor application health and performance

• Update applications

• Delete applications

Deploying and scaling applications are completely independent operations. This provides the flexibility to scale at will, without the cost of having to redeploy the application every time. Users can simply scale an application with a self-service call to the platform. Through commercial products such as Pivotal Cloud Foundry, you can set up autoscaling policies for dynamic scaling of applications when they meet certain configurable thresholds.

Removing the infrastructure, OS, and middleware configuration concerns from developers allows them to focus all their effort on the application instead of deploying and configuring supporting technologies. This keeps the development focus where it needs to be, on the business logic that generates revenue.

The Twelve-Factor Contract

An architectural style known as cloud-native applications has been established to describe the design of applications specifically written to run in a cloud environment. These applications avoid some of the antipatterns that were established in the client-server era, such as writing data to the local filesystem. Those antipatterns do not work as well in a cloud environment because, for example, local storage is ephemeral given that VMs can move between different hosts. The Twelve-Factor App explains the 12 principles underpinning cloud-native applications.

Platforms offer a set of contracts to the applications and services that run on them. These contracts ensure that applications are constrained to do the right thing. Twelve Factor can be thought of as the contract between an application and a cloud-native platform.

There are benefits to adhering to a contract that constrains things correctly. Twitter is a great example of a constrained platform. You can write only 140 characters, but that constraint becomes an extremely valuable feature of the platform. You can do a lot with 140 characters coupled with the rich features surrounding that contract. Similarly, platform contracts are born out of previously tried-and-tested constraints; they are enabling and make doing the right thing—good developer practices—easy for developers.

Release Engineering through BOSH

In addition to developer concerns, the platform provides responsive IT operations, with full visibility and control over the application life cycle, provisioning, deployment, upgrades, and security patches. Several other operational benefits exist, such as built-in resilience, security, centralized user management, and better insights through capabilities like aggregated metrics and logging.

Rather than integrating, operating, and maintaining numerous individual components, the platform operator deals only with the platform. Structured platforms handle all the aforementioned undifferentiated heavy lifting tasks.

The Cloud Foundry repository is structured for use with BOSH. BOSH is an open source tool chain for release-engineering, deployment, and life cycle management. Using a YAML (YAML Ain’t Markup Language) deployment manifest, BOSH creates and deploys (virtual) machines¹ on top of the targeted computing infrastructure and then deploys and runs software (in our case Cloud Foundry and supporting services) on to those created machines. Many of the benefits to operators are provided through using BOSH to deploy and manage Cloud Foundry. BOSH is often overlooked as just another component of Cloud Foundry, but it is the bedrock of Cloud Foundry and a vital piece of the ecosystem. It performs monitoring, failure recovery, and software updates with zero-to-minimal downtime. Chapter 10 discusses BOSH at length.

Rather than utilizing a bespoke integration of a variety of tools and techniques that provide solutions to individual parts of the release-engineering goal, BOSH is designed to be a single tool covering the entire set of requirements of release engineering. BOSH enables software deployments to be:

• Automated

• Reproducible

• Scalable

• Monitored with self-healing failure recovery

• Updatable with zero-to-minimal downtime

BOSH translates intent into action via repeatability by always ensuring every provisioned release is identical and repeatable. This removes the challenge of configuration drift and removes the sprawl of snowflake servers.

BOSH configures infrastructure through code. By design, BOSH tries to abstract away the differences between infrastructure platforms (IaaS or physical servers) into a generalized, cross-platform description of your deployment. This provides the benefit of being infrastructure agnostic (as far as possible).

BOSH performs monitoring, failure recovery, software updates, and patching with zero-to-minimal downtime. Without such a release-engineering tool chain, all these concerns remain the responsibility of the operations team. A lack of automation exposes the developer to unnecessary risk.

Built-In Resilience and Fault Tolerance

A key feature of Cloud Foundry is its built-in resilience. Cloud Foundry provides built-in resilience and self-healing based on control theory. Control theory is a branch of engineering and mathematics that uses feedback loops to control and modify the behavior of a dynamic system. Resiliency is about ensuring that the actual system state (the number of running applications, for example) matches the desired state at all times, even in the event of failures. Resiliency is an essential but often costly component of business continuity.

Cloud Foundry automates the recovery of failed applications, components, and processes. This self-healing removes the recovery burden from the operator, ensuring speed of recovery. Cloud Foundry, underpinned by BOSH, achieves resiliency and self-healing through:

• Restarting failed system processes

• Recreating missing or unresponsive VMs

• Deployment of new AIs if an application crashes or becomes unresponsive

• Application striping across availability zones (AZs) to enforce separation of the underlying infrastructure

• Dynamic routing and load balancing

Cloud Foundry deals with application orchestration and placement focused on even distribution across the infrastructure. The user should not need to worry about how the underlying infrastructure runs the application beyond having equal distribution across different resources (known as availability zones). The fact that multiple copies of the application are running with built-in resiliency is what matters.

Cloud Foundry provides dynamic load balancing. Application consumers use a route to access an application; each route is directly bound to one or more applications in Cloud Foundry. When running multiple instances, it balances the load across the instances, dynamically updating its routing table. Dead application routes are automatically pruned from the routing table, with new routes added when they become available.

Without these capabilities, the operations team is required to continually monitor and respond to pager alerts from failed apps and invalid routes. By replacing manual interaction with automated, self-healing software, applications and system components are restored quickly with less risk and downtime. The resiliency concern is satisfied once, for all applications running on the platform, as opposed to developing customized monitoring and restart scripts per application. The platform removes the ongoing cost and associated maintenance of bespoke resiliency solutions.

Resiliency Through Availability Zones

Finally, Cloud Foundry supports the use of availability zones (AZs). As depicted in Figure 2-2, you can use AZs to enforce separation of the underlying infrastructure. For example, when running on AWS, you can directly map Cloud Foundry AZs to different AWS AZs. When running on vCenter, you can map Cloud Foundry AZs to different vCenter Cluster and resource-pool combinations. Cloud Foundry can then deploy its components across the AZs. When you deploy multiple AIs, Cloud Foundry will distribute them evenly across the AZs. If, for example, a rack of servers fails and brings down an entire AZ, the AIs will still be up and serving traffic in the remaining AZs.

Figure 2-2. Application resiliency through Cloud Foundry AZs

Aggregated Streaming of Logs and Metrics

Cloud Foundry provides insight into both the application and the underlying platform through aggregated logging and metrics. The logging system within Cloud Foundry is known as the Loggregator. It is the inner voice of the system, telling the operator and developer what is happening. It is used to manage the performance, health, and scale of running applications and the platform itself, via the following:

• Logs provide visibility into behavior; for example, application logs can be used to trace through a specific call stack.

• Metrics provide visibility into health; for example, container metrics can include memory, CPU, and disk-per-app instance.

Insights are obtained through storing and analyzing a continuous stream of aggregated, time-ordered events from the output streams of all running processes and backing services. Application logs are aggregated and streamed to an endpoint via Cloud Foundry’s Loggregator Firehose. Logs from the Cloud Foundry system components can also be made available and processed through a separate syslog drain. Cloud Foundry produces both the application and system logs to provide a holistic view to the end user.

Figure 2-3 illustrates how application logs and syslogs are separated as streams, in part to provide isolation and security between the two independent concerns, and in part due to consumer preferences. Generally speaking, app developers do not want to wade through component logs to resolve an app-specific issue. Developers can trace the log flow from the frontend router to the application code from a single log file.

Figure 2-3. The Loggregator system architecture used for aggregating application logs and metrics

In addition to logs, metrics are gathered and streamed from system components. Operators can use metrics information to monitor an instance of Cloud Foundry. Furthermore, Cloud Foundry events show specific events such as when an application is started or stopped. The benefits of aggregated log, metric, and event streaming include the following:

• You can stream logs to a single endpoint.

• Streamed logs provide timestamped outputs per application.

• Both application logs and system-component logs are aggregated, simplifying their consumption.

• Metrics are gathered and streamed from system components.

• Operators can use metrics information to monitor an instance of Cloud Foundry.

• You can view logs from the command line or drain them into a log management service such as an ELK stack (Elasticsearch, Logstash, and Kibana), Splunk, or PCF Metrics.

• Viewing events is useful when debugging problems. For example, it is useful to be able to correlate an app instance event (like an app crash) to the container’s specific metrics (high memory prior to crash).

The cost of implementing an aggregated log and metrics-streaming solution involves bespoke engineering to orchestrate and aggregate the streaming of both syslog and application logs from every component within a distributed system into a central server. Using a platform removes the ongoing cost and associated maintenance of bespoke logging solutions.

Security

For enterprises working with cloud-based infrastructure, security is a top concern. Usually the security teams have the strongest initial objections to Cloud Foundry because it works in a way that is generally unprecedented to established enterprise security teams. However, as soon as these teams understand the strength of Cloud Foundry’s security posture, my experience is that they become one of your strongest champions.

UAA Management

Role-based access defines who can use the platform and how. Cloud Foundry uses RBAC, with each role granting permissions to a specific environment the user is targeting. All collaborators target an environment with their individual user accounts associated with a role that governs what level and type of access the user has within that environment. Cloud Foundry’s UAA is the central identity management service for both users and applications. It supports federated login, Lightweight Directory Access Protocol (LDAP), Security Assertion Markup Language (SAML), SSO, and multifactor authentication. UAA is a powerful component for strengthening your security posture for both user and application authentication. Chapter 15 looks at UAA in more detail.

Organizations and Spaces

Most developers are familiar with using VMs for development and deployment. Cloud Foundry is a virtualization layer (underpinned by containers) on top of a virtualization layer underpinned by VMs. Therefore, users do not have direct access to a specific machine or VM; rather, they simply access a logical partition of resources to deploy their apps.

To partition and allocate resources, Cloud Foundry uses logical boundaries known as Organizations (Orgs) and Spaces. Orgs contain one or more Spaces. Users can belong to any number of Orgs and/or Spaces, and users can have unique roles and permissions in each Org or Space to which they belong.

Orgs and Spaces provide the following:

• Logical separation and assignment of Cloud Foundry resources

• Isolation between different teams

• Logical isolation of development, test, staging, and production environments

For some enterprise customers with traditional silos, defining their required Orgs and Spaces can at first seem complex. Ideally, development teams should have autonomy to create and manage their own Spaces, as required. For development teams embracing microservices, the best approach is to organize teams by the big-A application—meaning a group of related (small-a) applications or services that can collectively and appropriately be grouped together, often referred to as bulkheading.

Ultimately, you know your business and how your developers work, so use these logical structures to provide meaningful working environments and pipelines for your developers.

Domains Hosts and Routes

To enable traffic from external clients, applications require a specific URL known as a route. A route is a URL comprised of a domain and an optional host as a prefix. The host in this context is the portion of the URL referring to the application or applications, such as these:

• my-app-name is the host prefix
• my-business.com is the domain
• my-app-name.my-business.com is the route

Rolling Upgrades and Blue/Green Deployments

As discussed in “Security”, both the applications running on the platform and the platform itself allow for rolling upgrades and zero-downtime deployment through a distributed consensus.

You can update applications running on the platform with zero downtime through a technique known as blue/green deployments.

Summary

This chapter walked you through the principal concepts of Cloud Foundry. For example, you should now understand the meaning of a cloud OS, and the importance of the twelve-factor contract. The primary premise of Cloud Foundry is to enable the application development-to-deployment process to be as fast as possible. Cloud Foundry, underpinned by the BOSH release-engineering tool chain, achieves this by doing more on your behalf. Cloud Foundry provides the following:

• Built-in resiliency through automated recovery and self-healing of failed applications, components, and processes

• Built-in resiliency through striping applications across different resources

• Authentication and authorization for both users and applications, with the addition of RBAC for users

• Increased security posture with the ability to rotate credentials and repave and repair components

• The ability to update the platform with zero downtime via rolling upgrades across the system

• Speed in the deployment of apps with the ability to connect to a number of services via both platform-managed service brokers and services running in your existing IT infrastructure

• Built-in management and operation of services for your application, such as metrics and log aggregation, monitoring, autoscaling, and performance management

Now that you understand both the concepts and capabilities of Cloud Foundry, you are ready to learn about the individual components that comprise a Cloud Foundry deployment.