The best way to distribute custom facts is to include them in modules, using a Puppet concept called "plug-ins in modules." This concept allows you to place your custom code inside an existing or new Puppet module and then use that module in your Puppet environment. Custom facts, custom types, providers, and functions are then distributed to any host that includes a particular module.

Modules that distribute facts are no different from other modules, and there are two popular approaches to doing so. Some people distribute facts related to a particular function in the module that they use to configure that function. For example, a fact with some Bind data in it might be distributed with the module you use to configure Bind. This clusters facts specific to a function together and allows a greater portability. Other sites include all custom facts (and other items) in a single, central module, such as a module called facts or plugins. This centralizes facts in one location for ease of management and maintenance.

Each approach has pros and cons and you should select one that suits your organization and its workflow. We personally prefer the former approach because it limits custom facts and other items to only those clients that require them, rather than all hosts. For some environments, this may be a neater approach. We're going to use this approach in this section when demonstrating managing custom facts.

So where in our modules do facts go? Let's create a simple module called bind as an example:

bind/
bind/manifests
bind/manifests/init.pp
bind/files
bind/templates
bind/lib/facter

Here we've created our standard module directory structure, but we've added another directory, lib. The lib directory contains any "plug-ins" or additional facts, types or functions we want to add to Puppet. We're going to focus on adding facts; these are stored in the lib/facter directory.

In addition to adding the lib/facter directory to modules that will distribute facts, you need to enable "plug-ins in modules" in your Puppet configuration. To do this, enable options in the [main] section of the Puppet master's puppet.conf configuration file, as you can see on the next line:

[main]
pluginsync = true

When set to true, the pluginsync setting turns on the "plug-ins in modules" capability. Now, when clients connect to the master, each client will check its modules for facts and other custom items. Puppet will take these facts and other custom items and sync them to the relevant clients, so they can then be used on these clients.

After configuring Puppet to deliver our custom facts, you should actually create some new facts! Each fact is a snippet of Ruby code wrapped in a Facter method to add the result of our Ruby code as a fact. Let's look at a simple example in Listing 10-1.

Facter.add("home") do
       setcode do
           ENV['HOME']
       end
end

In this example, our custom fact returns the value of the HOME environmental value as a fact called home, which in turn would be available in our manifests as the variable $home.

The Facter.add method allows us to specify the name of our new fact. We then use the setcode block to specify the contents of our new fact, in our case using Ruby's built-in ENV variable to access an environmental variable. Facter will set the value of our new fact using the result of the code executed inside this block.

In Listing 10-2, you can see a custom fact that reads a file to return the value of the fact.

Facter.add("timezone") do
       confine :operatingsystem => :debian
       setcode do
            File.readlines("/etc/timezone").to_a.last
       end
end

Here, we're returning the timezone of a Debian host. We've also done two interesting things. First, we've specified a confine statement. This statement restricts the execution of the fact if a particular criteria is not met. This restriction is commonly implemented by taking advantage of the values of other facts. In this case, we've specified that the value of the operatingsystem fact should be Debian for the fact to be executed. We can also use the values of other facts, for example:

confine :kernel => :linux

The previous confine is commonly used to limit the use of a particular fact to nodes with Linux-based kernels.

Second, we've used the readlines File method to read in the contents of the /etc/timezone file. The contents are returned as the fact timezone, which in turn would be available as the variable $timezone.

timezone => Australia/Melbourne

We've established how to confine the execution of a fact but we can also use other fact values to influence our fact determination, for example:

Facter.add("timezone") do
       setcode do
         if Facter.value(:operatingsystem) =~ /Debian|Ubuntu/

File.readlines("/etc/timezone").to_a.last
         else
            tz = Time.new.zone
         end
       end
end

Here, if the operating system is Debian or Ubuntu, it will return a time zone value by returning the value from the /etc/timezone file. Otherwise, the fact will use Ruby's in-built time handling to return a time zone.

You could also use a case statement to select different fact values, for example as used in the operatingsystemrelease fact shown in Listing 10-3.

Facter.add(:operatingsystemrelease) do
    confine :operatingsystem => %w{CentOS Fedora oel ovs RedHat MeeGo}
    setcode do
        case Facter.value(:operatingsystem)
        when "CentOS", "RedHat"
            releasefile = "/etc/redhat-release"
        when "Fedora"
            releasefile = "/etc/fedora-release"
        when "MeeGo"
            releasefile = "/etc/meego-release"
        when "OEL", "oel"
            releasefile = "/etc/enterprise-release"
        when "OVS", "ovs"
            releasefile = "/etc/ovs-release"
        end
        File::open(releasefile, "r") do |f|
            line = f.readline.chomp
            if line =~ /(Rawhide)$/
                "Rawhide"
            elsif line =~ /release (d[d.]*)/
                $1
            end
        end
    end
end

You can use other fact values for any purpose you like, not just for determining how to retrieve a fact. Some facts return another fact value if they cannot find a way to determine the correct value. For example, the operatingsystem fact returns the current kernel, Facter.value(:kernel), as the value of operatingsystem if Facter cannot determine the operating system it is being run on.

You can create more complex facts and even return more than one fact in your Ruby snippets, as you can see in Listing 10-4.

netname = nil
       netaddr = nil
       test = {}
       File.open("/etc/networks").each do |line|
            netname = $1 and netaddr = $2 if line 

=~ /^(w+.?w+)s+([0-9]+.[0-9]+.[0-9]+.[0-9]+)/
            if netname != nil && netaddr != nil
                 test["network_" + netname] = netaddr
                 netname = nil
                 netaddr = nil
                 end
       end
          test.each{|name,fact|
                 Facter.add(name) do
                     setcode do
                        fact
                     end
                 end
       }

This fact actually creates a series of facts, each fact taken from information collected from the /etc/networks file. This file associates network names with networks. Our snippet parses this file and adds a series of facts, one per each network in the file. So, if our file looked like:

default    0.0.0.0
loopback    127.0.0.0
link-local    169.254.0.0

Then three facts would be returned:

network_default => 0.0.0.0
network_loopback => 127.0.0.0
network_link-local => 169.254.0.0

You can take a similar approach to commands, or files, or a variety of other sources.

There is a simple process for testing your facts: Import them into Facter and use it to test them before using them in Puppet. To do this, you need to set up a testing environment. Create a directory structure to hold our test facts—we'll call ours lib/ruby/facter. Situate this structure beneath the root user's home directory. Then create an environmental variable, $RUBYLIB, that references this directory and will allow Facter to find our test facts:

# mkdir -p ~/lib/ruby/facter
# export RUBYLIB=~/lib/ruby

Then copy your fact snippets into this new directory:

# cp /var/puppet/facts/home.rb $RUBYLIB/facter

After this, you can call Facter with the name of the fact you've just created. If the required output appears, your fact is working correctly. On the following lines, we've tested our home fact and discovered it has returned the correct value:

# facter home
/root

If your fact is not working correctly, an error message you can debug will be generated.

Facts just scratch the surface of Puppet's extensibility, and adding to types, providers, and functions adds even more capability. We're going to demonstrate that in the next section.

The best way to distribute custom types, providers and functions is to include them in modules, using "plug-ins in modules," the same concept we introduced earlier this chapter to distribute custom facts. Just like custom facts, you again place your custom code into a Puppet module and use that module in your configuration. Puppet will take care of distributing your code to your Puppet masters and agents.

Again, just like custom facts, you can take two approaches to managing custom code: placing it in function-specific modules or centralizing it into a single module. We're going to demonstrate adding custom code in a single, function-specific module.

So, where in our modules does custom code go? Let's create a simple module called apache as an example:

apache/
apache/manifests
apache/manifests/init.pp
apache/files
apache/templates
apache/lib/facter

apache/lib/puppet/type
apache/lib/puppet/provider
apache/lib/puppet/parser/functions

Here we've created our standard module directory structure, but we've added another directory, lib. We saw the lib directory earlier in the chapter when we placed custom facts into its Facter subdirectory. The lib directory also contains other "plug-ins" like types, providers and functions, which we want to add to Puppet. The lib/puppet/type and lib/puppet/provider directories hold custom types and providers respectively. The last directory, lib/puppet/parser/functions, holds custom functions.

Like we did when we configured Puppet for custom facts, you need to enable "plug-ins in modules" in your Puppet configuration. To do this, enable the pluginsync option in the [main] section of the Puppet master's puppet.conf configuration file, as follows:

[main]
pluginsync = true

The pluginsync setting, when set to true, turns on the "plug-ins in modules" capability. Now, when agents connect to the master, each agent will check its modules for custom code. Puppet will take this custom code and sync it to the relevant agents. It can then be used on these agents. The only exception to this is custom functions. Functions run on the Puppet master rather than the Puppet agents, so they won't be synched down to an agent. They will only be synched if the Puppet agent is run on the Puppet master, i.e., if you are managing Puppet with Puppet.

Puppet types are used to manage individual configuration items. Puppet has a package type, a service type, a user type, and all the other types available. Each type has one or more providers. Each provider handles the management of that configuration on a different platform or tool: for example, the package type has aptitude, yum, RPM, and DMG providers (among 22 others).

We're going to show you a simple example of how to create an additional type and provider, one that manages version control systems (VCS), which we're going to call repo. In this case we're going to create the type and two providers, one for Git and one for SVN. Our type is going to allow you to create, manage and delete VCS repositories.

A Puppet type contains the characteristics of the configuration item we're describing, for example in the case of VCS management type:

Correspondingly, the Puppet providers specify the actions required to manage the state of the configuration item. Obviously, each provider has a set of similar actions that tell it how to:

custom/
custom/manifests/init.pp
custom/lib/puppet/type
custom/lib/puppet/provider

custom/lib/puppet/type/repo.rb

Puppet::Type.newtype(:repo) do
    @doc = "Manage repos"
    ensurable

    newparam(:source) do
        desc "The repo source"

        validate do |value|
            if value =~ /^git/
                resource[:provider] = :git
            else
                resource[:provider] = :svn
            end
        end

        isnamevar
    end

    newparam(:path) do
        desc "Destination path"

        validate do |value|
            unless value =~ /^/[a-z0-9]+/
                raise ArgumentError, "%s is not a valid file path" % value
            end

end
    end
end

service { "sshd":
    ensure => present,
}

newparam(:source) do
  desc "The repo source"

  validate do |value|
    if value =~ /^git/
      resource[:provider] = :git
   else
     resource[:provider] = :svn
  end
 end
   isnamevar
end

repo { "puppet":
  source => "git://github.com/puppetlabs/puppet.git",
  path => "/home/puppet",
  provider => git,
  ensure => present,
}

newparam(:path) do
  desc "Destination path"

  validate do |value|
    unless value =~ /^/[a-z0-9]+/
      raise ArgumentError, "%s is not a valid file path" % value
    end
  end
end

custom/lib/puppet/provider/repo/svn.rb

require 'fileutils'

Puppet::Type.type(:repo).provide(:svn) do
  desc "Provides Subversion support for the repo type"

  commands :svncmd => "svn"
  commands :svnadmin => "svnadmin"

  def create
    svncmd "checkout", resource[:name], resource[:path]
  end

  def destroy
    FileUtils.rm_rf resource[:path]
  end

  def exists?
    File.directory? resource[:path]
  end
end

Puppet::Type.type(:repo).provide(:svn) do

commands :svncmd => "svn"
commands :svnadmin => "svnadmin"

custom/lib/puppet/provider/repo/git.rb

require 'fileutils'
Puppet::Type.type(:repo).provide(:git) do

  desc "Provides Git support for the repo provider"

  commands :gitcmd => "git"

  def create
    gitcmd "clone", resource[:name], resource[:path]
  end

  def destroy
    FileUtils.rm_rf resource[:path]
  end

  def exists?
    File.directory? resource[:path]
  end
end

repo { "wordpress":
  source => "http://core.svn.wordpress.org/trunk/",
  path => "/var/www/wp",
  provider => svn,
  ensure => present,
}

You've just seen a very simple type and provider that uses commands to create, delete and check for the status of a resource. In addition to these kinds of types and providers, Puppet also comes with a helper that allows you to parse and edit simple configuration files. This helper is called ParsedFile.

Unfortunately, you can only manage simple files with ParsedFile, generally files with single lines of configuration like the /etc/hosts file or the example we're going to examine. This is a type that manages the /etc/shells file rather than multi-line configuration files.

To use a ParsedFile type and provider, we need to include its capabilities. Let's start with our /etc/shells management type which we're going to call shells. This file will be located in:

custom/lib/puppet/type/shells.rb.

Puppet::Type.newtype(:shells) do
     @doc = "Manage the contents of /etc/shells
     shells { "/bin/newshell":
                ensure => present,
     }"

ensurable

newparam(:shell) do
  desc "The shell to manage"
  isnamevar
end

newproperty(:target) do
  desc "Location of the shells file"
  defaultto {
    if @resource.class.defaultprovider.ancestors.include? (Puppet::Provider::ParsedFile)
      @resource.class.defaultprovider.default_target
    else
      nil
  end
  }
 end
end

newparam(:shell) do
  desc "The shell to manage"
  isnamevar
end

require 'puppet/provider/parsedfile'

shells = "/etc/shells"

Puppet::Type.type(:shells).provide(:parsed, :parent => Puppet::Provider::ParsedFile,:default_target => shells, :filetype => :flat) do

  desc "The shells provider that uses the ParsedFile class"

  text_line :comment, :match => /^#/;
  text_line :blank, :match => /^s*$/;

  record_line :parsed, :fields => %w{name}
end

custom/lib/puppet/provider/shells/parsed.rb

require 'puppet/provider/parsedfile'

text_line :comment, :match => /^#/;
text_line :blank, :match => /^s*$/;

record_line :parsed, :fields => %w{name}

shells { "/bin/anothershell":
     ensure => present,
}

In this section we're going to show you a slightly more complex type and provider used to manage HTTP authentication password files. It's a similarly ensureable type and provider, but with some more sophisticated components.

Puppet::Type.newtype(:httpauth) do
    @doc = "Manage HTTP Basic or Digest password files." +
           "    httpauth { 'user':                     " +
           "      file => '/path/to/password/file',    " +
           "      password => 'password',              " +
           "      mechanism => basic,                  " +
           "      ensure => present,                   " +
           "    }                                      "

    ensurable do
       newvalue(:present) do
           provider.create
       end

       newvalue(:absent) do
           provider.destroy
       end

       defaultto :present
    end

    newparam(:name) do
       desc "The name of the user to be managed."

       isnamevar
    end

    newparam(:file) do
       desc "The HTTP password file to be managed. If it doesn't exist it is created."
    end

    newparam(:password) do
       desc "The password in plaintext."

    end

    newparam(:realm) do
       desc "The realm - defaults to nil and mainly used for Digest authentication."

       defaultto "nil"
    end

    newparam(:mechanism) do
       desc "The authentication mechanism to use - either basic or digest. Default to basic."

newvalues(:basic, :digest)

       defaultto :basic
    end

    # Ensure a password is always specified
    validate do
       raise Puppet::Error, "You must specify a password for the user." unless @parameters.include?(:password)
    end
end

begin
    require 'webrick'
rescue
    Puppet.warning "You need WEBrick installed to manage HTTP Authentication files."
end

Puppet::Type.type(:httpauth).provide(:httpauth) do
    desc "Manage HTTP Basic and Digest authentication files"

    def create
        # Create a user in the file we opened in the mech method
        @htauth.set_passwd(resource[:realm], resource[:name], resource[:password])
        @htauth.flush

end

    def destroy
        # Delete a user in the file we opened in the mech method
        @htauth.delete_passwd(resource[:realm], resource[:name])
        @htauth.flush
    end

    def exists?
        # Check if the file exists at all
        if File.exists?(resource[:file])
            # If it does exist open the file
            mech(resource[:file])

            # Check if the user exists in the file
            cp = @htauth.get_passwd(resource[:realm], resource[:name], false)

            # Check if the current password matches the proposed password
            return check_passwd(resource[:realm], resource[:name], resource[:password], cp)
        else
            # If the file doesn't exist then create it
            File.new(resource[:file], "w")
            mech(resource[:file])
            return false
        end
    end

    # Open the password file
    def mech(file)
        if resource[:mechanism] == :digest
            @htauth = WEBrick::HTTPAuth::Htdigest.new(file)
        elsif resource[:mechanism] == :basic
            @htauth = WEBrick::HTTPAuth::Htpasswd.new(file)
        end
    end

    # Check password matches
    def check_passwd(realm, user, password, cp)
        if resource[:mechanism] == :digest
            WEBrick::HTTPAuth::DigestAuth.make_passwd(realm, user, password) == cp
        elsif resource[:mechanism] == :basic
            # Can't ask webbrick as it uses a random seed
            password.crypt(cp[0,2]) == cp
        end
    end
end

httpauth { "bob":
  file => "/etc/apache2/htpasswd.basic",
  password => "password",
  mechanism => basic,
}

Like facts, you can test your types and providers. The best way to do this is add them to a module in your development or testing environment and enable pluginsync to test them there before using them in your production environment, for example let's add our HTTPAuth type to a module called httpauth, first adding the required directories:

$ mkdir -p /etc/puppet/modules/httpauth/(manifests,files,templates,lib}
$ mkdir -p /etc/puppet/modules/httpauth/lib/{type,provider}
$ mkdir -p /etc/puppet/modules/httpauth/lib/provider/httpauth

Then copying in the type and provider to the requisite directories.

# cp type/httpauth.rb /etc/puppet/modules/lib/type/httpauth.rb
# cp provider/httpauth.rb /etc/puppet/modules/lib/provider/httpauth/httpauth.rb

When Puppet is run (and pluginsync enabled) it will find your types and providers in these directories, deploy them and make them available to be used in your Puppet manifests.

The last type of custom Puppet code we're going to look at is the function. You've seen a number of functions in this book already, for example: include, notice and template are all functions we've used. But you can extend the scope of the available functions by writing your own.

There are two types of functions: statements and rvalues. Statements perform some action, for example the fail function, and rvalues return a value, for example if you pass in a value, the function will process it and return a value. The split function is an example of an rvalue function.

We're going to write a simple function and distribute it to our agents. Like plug-ins, we can use plug-in sync to distribute functions to agents; they are stored in:

custom/lib/puppet/parser/functions

The file containing the function must be named after the function it contains; for example, the template function should be contained in the template.rb file.

Let's take a look at a simple function in Listing 10-12.

Puppet::Parser::Functions::newfunction(:sha512, :type => :rvalue, :doc => "Returns a SHA1hash value from a provided string.") do |args|

  require 'sha1'

  Digest::SHA512.hexdigest(args[0])

end

Puppet contains an existing function called sha1 that generates a SHA1 hash value from a provided string. In Listing 10-12, we've updated that function to support SHA512 instead. Let's break that function down. To create the function we call the Puppet::Parser::Functions::newfunction method and pass it some values. First, we name the function, in our case sha512. We then specify the type of function it is, here rvalue, for a function that returns a value. If we don't specify the type at all then Puppet assumes the function is a statement. Lastly, we specify a :doc string to document the function.

The newfunction block takes the incoming argument and we process it, first adding in support for working with SHA hashes by requiring the sha1 library, and then passing the argument to the hexdigest method. As this is an rvalue function, it will return the created hash as the result of the function.

We mentioned earlier that functions run on the Puppet master. This means we only have access to the resources and data available on the master, but this does include some quite useful information, most importantly fact data. You can look up and use the value of facts in your functions using the lookupvar function, like so:

lookupvar('fqdn')

Replace fqdn with the name of the fact whose value you wish to look up.

You can see how easy it is to create some very powerful functions in only a few lines of code. We recommend having a look at the existing functions (most of which are very succinct) as a way to get started on your first functions. Some of the common functions include tools to manipulate paths, regular expressions and substitutions, and functions to retrieve data from external sources. There are numerous examples (many on Github or searchable via Google) of functions that you can copy or adapt for your environment.

After you've created your function you should test that it works correctly. There are a couple of ways you can do this. Some basic testing of the function can be performed by executing the function file with Ruby, like so:

$ ruby –rpuppet sha512.rb

This loads the Puppet library (Puppet must be installed on the host) and then runs the file containing the function we created in Listing 10-12. This will allow us to determine whether the file parses without error. It does not tell us if the function performed correctly.

We can also use the Ruby IRB (Interactive Ruby Shell) to confirm our function is properly defined, like so:

$ irb
irb> require 'puppet'
=> true
irb> require '/tmp/sha512.rb'
=> true
irb> Puppet::Parser::Functions.function(:sha512)
=> "function_sha512"

Here we've launched irb and then required Puppet and our new function. We then confirm that Puppet can see the new function and that it parses as a correct function.

The best way to test a function is to use it in a manifest, and the easiest way to do that is to add your functions to Puppet's libdir and run a stand-alone manifest. Assuming Puppet is installed, first find your libdir:

$ sudo puppet –configprint | grep 'libdir'
/var/lib/puppet/lib

Then create a directory to hold our functions:

$ sudo mkdir -p /var/lib/puppet/lib/puppet/parser/functions

Then copy in our function:

$ sudo cp sha512.rb /var/lib/puppet/lib/puppet/parser/functions

Then a manifest to execute our new function:

$ cat /tmp/sha.pp
$hash = sha512("test")
notify { $hash: }

And finally run the function:

$ puppet /tmp/sha.pp
notice:
ee26b0dd4af7e749aa1a8ee3c10ae9923f618980772e473f8819a5d4940e0db27ac185f8a0e1d5f84f88bc887fd67b143732c304cc5fa9ad8e6f57f50028a8ff
notice:
/Stage[main]//Notify[ee26b0dd4af7e749aa1a8ee3c10ae9923f618980772e473f8819a5d4940e0db27ac185f8a0e1d5f84f88bc887fd67b143732c304cc5fa9ad8e6f57f50028a8ff]/message: defined 'message' as'ee26b0dd4af7e749aa1a8ee3c10ae9923f618980772e473f8819a5d4940e0db27ac185f8a0e1d5f84f88bc887fd67b143732c304cc5fa9ad8e6f57f50028a8ff'

We can see that our notify resource returned a 512-bit hash generated by our sha512 function.