Creating Readable Roles

Roles should be easy to identify and easy to read. Because the role specifies only the profiles to be used to build it, roles should be easily readable by people with no programming experience, regardless of the profile implementation. Let’s examine some readable role declarations in the subsections that follow.

class role::web_frontend() {
  include profile::yumrepos
  include profile::antivirus
  include profile::mysql::client
  include profile::webserver
  include profile::firewall
}

  - name: "Role data"
    path: "roles/%{foreman_data.role}.yaml"

---
classes:
  - 'profile::yumrepos'
  - 'profile::antivirus'
  - 'profile::mysql::client'
  - 'profile::webserver'
  - 'profile::firewall'

Design Roles for a Singular Use Case

There’s really only one rule for role design: it should implement a single, specific node configuration. The only things different on two nodes of the same role should be their hostname, IP address, and such forth.

Because roles are a manifestation of an organization’s specific nuances, there are no hard-and-fast rules for what a role should and shouldn’t do. However, there are certain things to look for (code smells) that imply the role isn’t scoped correctly:

The role utilizes parameter input to make implementation choices

Are you sure this isn’t two different roles?

The role implements logic or resources directly

It’s likely that the logic or resources are general enough to be within a profile used by other roles.

The role is longer than a paragraph

If the role includes many profiles, it’s likely to reuse groups of profiles that could be shared with other roles.

The role implements explicit ordering of the profiles provided

Avoid ordering relationships between the profiles contained in the roles unless they are absolutely necessary. Many profiles source common modules, creating ordering conflicts.

The same role is applied to nodes that don’t provide identical services

If a role builds nodes that provide different services, it contains deterministic, not declarative logic.

Any one of these code smells isn’t necessarily bad, but if multiple of these conditions are true, it’s very likely that the role should be broken down into profiles and rescoped.

Provide the Role for Use in Data Lookups

You can assign roles in a multitude of ways, but the most common and flexible ways to do so are the following:

• A custom fact identifies the role.

• The ENC provides the role as a node parameter.

Facter.add(:role) do
  setcode do
    if Facter.value(:hostname) =~ /^(w+)[0-9]+/
      role = $1
    else
      role = 'unknown'
    end
    role
  end
end

A Sample Service Profile

Let’s examine a sample profile for the aforementioned standalone WordPress service, as shown in Example 7-2.

class profile::wordpress (
  $wp_hostname = $facts['fqdn'],
  $tcp_port    = 80,
  $directory   = '/var/www/html',
) {
  include firewall
  include apache
  include apache::mod::php
  include mysql::client
  include mysql::server

  class { 'mysql::bindings':
    php_enable => true,
  }

  class { 'wordpress':
    install_dir => $directory,
  }

  apache::vhost { $wp_hostname:
    port    => $tcp_port,
    docroot => $directory,
  }

  firewall { '010 Allow Wordpress Access':
    action => 'accept',
    proto  => 'tcp',
    dport  => $tcp_port,
  }
}

This example follows the basic pattern of a profile. It is not concerned with basic system configuration or any other application. It is concerned only with configuration of a WordPress instance.

There are no strict ordering dependencies between modules, although a few resources establish autodependency relationships. There are one or two classes that are parse-order dependent, but they are contained entirely within the profile and are not sensitive to the structure of other modules or the order in which this profile is evaluated versus other profiles.

Providing Actionable Data in Profile Parameters

The profile in Example 7-2 passes in only the hostname, port, and docroot parameters. This is because these three parameters are the ones for which this profile has specific logic.

You should use profile parameters when the parameter affects the behavior of the profile or when the data needs to be modified before being passed to the module. Using input parameters this way increases the versatility and DRY-ness of the profile. Rather than creating multiple profiles that are basically the same, use a single profile that can adjust its behavior based on the input supplied. Used in this manner, profile parameters provide the business- or site-specific data for a bespoke configuration.

For example, Consul service discovery uses the same configuration file for both clients and servers. The difference between the two is established entirely based on data in the service. In this case, a Consul profile might pass the bootstrap_expect and join_wan parameters to the Puppet-approved KyleAnderson/consul module only when the server option (node- or role-specific data) is enabled.

Unlike component modules, it is acceptable to look up site-specific data in a profile. Profiles implement business logic, so it’s perfectly alright to have unique and site-specific logic in the profile. The guideline to create reusable code is relaxed at this level: your profiles should be closely tied to your specific technology stack and its data.

Profile-specific defaults can be provided in the profile’s data, thus keeping the business logic and business data close together.

Implementing Business Logic in Profiles

Profiles are the appropriate place for business logic. You can use this logic to act based on input parameters or node facts to build the profile appropriately for a given node type.

The following profile installs the appropriate antivirus based on the node’s OS:

class profile::antivirus {
  case $facts['kernel'] {
    'Linux': {
      include clamav
    }
    'windows': {
      include eset_nod32
    }
    default: {
      warning("No antivirus available for ${facts['kernel']}")
    }
  }
}

This profile contains the conditional logic based on the OS, allowing you to include include 'profile::antivirus' for every node, regardless of OS type.

Firewall rules might be owned by multiple teams at a large or diverse organization. That breakdown would be specific to the organization, so this is the kind of logic that belongs in a profile. Let’s assume that there are security rules provided by corporate security, site-specific rules provided by the operations team, and application-specific rules necessary to allow the application to work. Here is a fairly arbitrary example that generates a list of firewall rules from data provided by three different teams:

class profile::firewall(
  Hash $site_rules     = {},
  Hash $app_rules      = {},
  Hash $security_rules = {},
) {
  use 'stdlib'

  class { 'iptables':
    # merge rules from three teams, rightmost value wins conflict
    rules => deep_merge($site_rules, $app_rules, $security_rules),
  }
}

The number of cases for which this is actually required should be fairly minimal; in most cases careful design of your data hierarchy should avoid the need to perform a hash merge data lookup. But if it needs to get weird, a profile is the appropriate place to encapsulate that weirdness.

It’s really when you put the combination of business logic and conditional logic based around the node type that you can create very readable, DRY profiles. The following example realizes virtual yum repositories based on the OS release and the application to be deployed on the node:

class profile::yumrepos(
  String $application = 'no_match',
  Hash $repositories  = {},
) {
  # Virtualize all hiera-listed repositories
  create_resources('@yumrepo', $repositories)

  # Realize all common repos
  Yumrepo <| tag == 'global' |>

  # Realize os-specific repos
  Yumrepo <| tag == "${facts['os']['name']}${facts['os']['release']['major']}" |>

  # Realize application repos
  Yumrepo <| tag == $application |>
}

This example demonstrates how to implement the business logic of a specific repository structure while tracking the needs of different operating systems, all while utilizing the common yumrepo resource.

Defining Module Relationships in Profiles

Profiles are the place to create relationships between modules and their dependencies. By creating intermodule relationships at the profile level, your modules neither need to be concerned with how their dependencies are satisfied nor how they relate to other modules.

Remember: profiles use site-specific data to assemble working services based on business logic. There is absolutely nothing wrong with creating relationships in your profiles when needed. If the installer for a component module requires Java, the profile should include Java and order it before the component module. This allows the component module’s requirements to be satisfied differently for different use cases.

Just as with module development, it’s a good idea to keep your profiles self-contained for a singular use case. Try to avoid creating relationships between profiles in your profiles; instead, try to make your profiles as self-sufficient as possible, and use roles to establish profile relationships where necessary.

Keep in mind that in many cases, it’s not necessary to create relationships between modules. As we saw in “A Sample Service Profile”, it is often sufficient to simply add modules to your profile. Use relationships to identify dependencies, not to attempt to control the order. Attempts to specify ordering will only create unnecessary conflicts.

Creating Metaprofiles to Group Configurations

If you find that certain combinations of profiles are common, it’s both reasonable and effective to create a metaprofile that connects the various profiles. Especially if there are a few parameters that would influence the implementation of each of those profiles, a metaprofile provides a great way to group together combinations by business logic.

The example that follows utilizes business logic for a standard web server build, making use of the profiles described previously. It provides a common technology stack without stipulating the specific usage details (e.g., WordPress, static content, whatever) that a specific profile can supply.

class profile::webserver(
  Boolean $external = true,
) {
  include profile::yumrepos
  include profile::antivirus
  include profile::apache
  include profile::mail::client
  if( $external ) {
    include profile::firewall
  }
}

This metaprofile provides a reusable combination of profiles that a specific application profile or role could apply. In doing so it provides a useful building block for role composition.

Let’s next discuss the good and bad, things to do, and things to watch out for when using metaprofiles.

class profile::webserver::nginx() {
  include 'profile::webserver'
  include 'nginx'
}

Designing an Appropriate Profile Structure

How do you structure your profiles when you have a lot of service profiles that share the same basic design patterns? By breaking them down into the smallest reusable components.

class profile::tomcat::titleservice() {
  include 'profile::tomcat'

  tomcat::instance { 'titleservice':
    catalina_home => '/opt/tomcat',
    catalina_base => '/opt/titleservice/root',
  }
}

class profile::tomcat(
  Hash $instances = {}.
) {
  include 'profile::tomcat'

  $instances.each |$name, $values| {
    tomcat::instance { $name:
      catalina_home => $values['catalina_home'],
      catalina_base => $values['catalina_base'],
    }
  }
}

Validating Profiles by Using Unit Tests

Unit tests verify that each class, defined type, and resource used to build the profile are added to the catalog correctly. It validates that the Puppet catalog will contain resources to do what is expected. This primarily helps identify code mistakes and regression failures—where a profile change stops providing an expected feature.

Profiles should not reimplement all of the test cases created for the profile’s component modules. Instead, it seeks to test that each of those modules is declared correctly.

The following example tests the WordPress profile built in Example 7-2.

context 'with default parameters' do
  # Ensure no parameters are supplied for this test context
  let(:params) do
    {}
  end

  # Declare the dependency classes with the profile's default values
  it { is_expected.to
         contain_class('mysql::bindings').with('php_enable' => true) }
  it { is_expected.to
         contain_apache__vhost(facts['fqdn']).with('port' => '80') }
  it { is_expected.to
         contain_class('wordpress').with('install_dir' => '/var/www/html') }
end

This example tests that a profile would invoke the component modules correctly, without recreating any of the tests of those modules. It can do this without knowing the internal structure of the modules that comprise the profile. Unit tests validate that the catalog will be built without error and ensure the stability of your profile interfaces.

Confirming Profile Implementation with Acceptance Tests

Acceptance tests apply the profile to the node and then verify that the expected changes are made. These tests measure the state before and after the test to confirm that the real-life expected results are visible on the node after Puppet has applied the profile.

You should supply each profile with an acceptance test specific to that profile. The goal of a profile acceptance test is not to reimplement all the tests created for the component modules. Instead, it should test the combined impact of the modules working together as a whole. The acceptance test provides end-to-end validation ensuring that the profile creates the model it was designed to implement.

At a minimum, test that the service is running as expected and that it responds as expected by creating a Beaker acceptance test for the profile.

The following example tests the WordPress profile built in Example 7-2.

  # test that the profile applies idempotently
  it 'should apply a second time without changes' do
    @result = apply_manifest(manifest)
    expect(@result.exit_code).to be_zero
  end

  # Make sure Apache is running with configured port
  describe service('httpd') do
    it { is_expected.to be_running }
    it { is_expected.to be_enabled }
  end
  describe port('80') do
    it { is_expected.to be_listening }
  end

This example tests that the httpd process is actively running and listening on the specified TCP port. This validates that the profile’s parameters have been applied correctly, and it can do it without knowing the internal structure of the modules that comprise the profile.

You can refer to the documentation and guides for Beaker acceptance testing for more information.