Transferring files with Puppet is something that is best done within modules. When you define a file resource, you can either use content => "something" or you can push a file from the Puppet master using source. For example, using our judy database, we can have judy::config with the following file definition:

class judy::config {
  file {'/etc/judy/judy.conf':
    source => 'puppet:///modules/judy/judy.conf'
  }
}

Now, Puppet will search for this file in the [modulepath]/judy/files directory. It is also possible to add full paths and have your module mimic the filesystem. Hence, the previous source line will be changed to source => 'puppet:///modules/judy/etc/judy/judy.conf', and the file will be found at [modulepath]/judy/files/etc/judy/judy.conf.

The puppet:/// URI source line mentioned earlier has three backslashes; optionally, the name of a puppetserver may appear between the second and third backslash. If this field is left blank, the puppetserver that performs the catalog compilation will be used to retrieve the file. You can alternatively specify the server using source => 'puppet://puppetfile.example.com/modules/judy/judy.conf'.

Templates are searched in a similar fashion. In this example, to specify the template in judy/templates, you will use content =>template('judy/template.erb') to have Puppet look for the template in your modules' templates directory. For example, another config file for judy can be defined, as follows:

file {'/etc/judy/judyadm.conf':
  content => template('judy/judyadm.conf.erb')
}

Puppet will look for the 'judy/judyadm.conf.erb' file at [modulepath]/judy/templates/judyadm.conf.erb. We haven't covered the Embedded Ruby (ERB) templates up to this point; templates are files that are parsed according to the ERB syntax rules. If you need to distribute a file where you need to change some settings based on variables, then a template can help. The ERB syntax is covered in detail at http://docs.puppetlabs.com/guides/templating.html. Puppet 4 (and Puppet 3 with the future parser enabled) supports EPP templates as well. EPP templates are Embedded Puppet templates that use Puppet language syntax rather than Ruby.

Modules can also include custom facts, as we've already seen in this chapter. Using the lib subdirectory, it is possible to modify both Facter and Puppet. In the next section, we will discuss module implementations in a large organization before writing custom modules.

Modules must begin with a lowercase letter and only contain lowercase letters, numbers, and the underscore (_) symbol. No other characters should be used. While writing modules that will be shared across the organization, use names that are obvious and won't interfere with other groups' modules or modules from the Forge. A good rule of thumb is to insert your corporation's name at the beginning of the module name and, possibly, your group name.

While designing modules, each module should have a specific purpose and not pull in manifests from other modules and each one of them should be autonomous. Classes should be used within the module to organize functionality. For instance, a module named example_foo installs a package and configures a service. Now, separating these two functions and their supporting resources into two classes, example_foo::pkg and example_foo::svc, will make it easier to find the code you need to work on, when you need to modify these different components. In addition, when you have all the service accounts and groups in another file, it makes it easier to find them, as well.

To start with a simple example, we will use Puppet's module command to generate empty module files with comments. The module name will be example_phpmyadmin, and the generate command expects the generated argument to be [our username]-[module name]; thus, using our sample developer, samdev, the argument will be samdev-example_phpmyadmin, as shown here:

[samdev@stand ~]$ cd control/dist/
[samdev@standdist]$ puppet module generate samdev-example_phpmyadmin
We need to create a metadata.json file for this module.  Please answer the
following questions; if the question is not applicable to this module, feel free
to leave it blank.

Puppet uses Semantic Versioning (semver.org) to version modules.
What version is this module?  [0.1.0]
--> 0.0.1

Who wrote this module?  [samdev]
-->

What license does this module code fall under?  [Apache-2.0]
-->

How would you describe this module in a single sentence?
--> An Example Module to install PHPMyAdmin

Where is this module's source code repository?
--> https://github.com/uphillian

Where can others go to learn more about this module?  [https://github.com/uphillian]
-->

Where can others go to file issues about this module?  [https://github.com/uphillian/issues]
-->

----------------------------------------
{
  "name": "samdev-example_phpmyadmin",
"version": "0.0.1",
  "author": "samdev",
  "summary": "An Example Module to install PHPMyAdmin",
  "license": "Apache-2.0",
  "source": "https://github.com/uphillian",
  "project_page": "https://github.com/uphillian",
  "issues_url": "https://github.com/uphillian/issues",
  "dependencies": [
    {"name":"puppetlabs-stdlib","version_requirement":">= 1.0.0"}
  ]
}
----------------------------------------

About to generate this metadata; continue? [n/Y]
-->y

Notice: Generating module at /home/samdev/control/dist/example_phpmyadmin...
Notice: Populating templates...
Finished; module generated in example_phpmyadmin.
example_phpmyadmin/manifests
example_phpmyadmin/manifests/init.pp
example_phpmyadmin/spec
example_phpmyadmin/spec/classes
example_phpmyadmin/spec/classes/init_spec.rb
example_phpmyadmin/spec/spec_helper.rb
example_phpmyadmin/tests
example_phpmyadmin/tests/init.pp
example_phpmyadmin/Gemfile
example_phpmyadmin/Rakefile
example_phpmyadmin/README.md
example_phpmyadmin/metadata.json

class example_phpmyadmin {
  include example_phpmyadmin::pkg
  include example_phpmyadmin::svc
}

class example_phpmyadmin::pkg {
  package {'httpd':
    ensure => 'installed',
    alias  => 'apache'
  }
}

class example_phpmyadmin::svc {
  service {'httpd':
    ensure => 'running',
    enable => true
  }
}

[samdev@standdist]$ puppet module generate samdev-example_phpldapadmin
We need to create a metadata.json file for this module.  Please answer the
following questions; if the question is not applicable to this module, feel free
to leave it blank.
…
Notice: Generating module at /home/samdev/control/dist/example_phpldapadmin...
Notice: Populating templates...
Finished; module generated in example_phpldapadmin.
example_phpldapadmin/manifests
example_phpldapadmin/manifests/init.pp
example_phpldapadmin/spec
example_phpldapadmin/spec/classes
example_phpldapadmin/spec/classes/init_spec.rb
example_phpldapadmin/spec/spec_helper.rb
example_phpldapadmin/tests
example_phpldapadmin/tests/init.pp
example_phpldapadmin/Gemfile
example_phpldapadmin/Rakefile
example_phpldapadmin/README.md
example_phpldapadmin/metadata.json

class example_phpldapadmin {
  include example_phpldapadmin::pkg
  include example_phpldapadmin::svc
}

class example_phpldapadmin::pkg {
  package {'httpd':
    ensure => 'installed',
    alias  => 'apache'
  }
}

class example_phpldapadmin::svc {
  service {'httpd':
    ensure => 'running',
    enable => true
  }
}

[root@client ~]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Package[httpd] is already declared in file /etc/puppetlabs/code/environments/production/dist/example_phpmyadmin/manifests/pkg.pp:2; cannot redeclare at /etc/puppetlabs/code/environments/production/dist/example_phpldapadmin/manifests/pkg.pp:2 at /etc/puppetlabs/code/environments/production/dist/example_phpldapadmin/manifests/pkg.pp:2:3 on node client.example.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

A resource in Puppet can only be defined once per node. What this means is that if our module defines the httpd package, no other module can define httpd. There are several ways to deal with this problem and we will work through two different solutions.

The first solution is the more difficult option—use virtual resources to define the package and then realize the package in each place you need. Virtual resources are similar to a placeholder for a resource; you define the resource but you don't use it. This means that Puppet master knows about the Puppet definition when you virtualize it, but it doesn't include the resource in the catalog at that point. Resources are included when you realize them later; the idea being that you can virtualize the resources multiple times and not have them interfere with each other. Working through our example, we will use the @ (at) symbol to virtualize our package and service resources. To use this model, it's helpful to create a container for the resources you are going to virtualize. In this case, we'll make modules for example_packages and example_services using Puppet module's generate command again.

The init.pp file for example_packages will contain the following:

class example_packages {
  @package {'httpd':
    ensure => 'installed',
    alias  => 'apache',
  }
}

The init.pp file for example_services will contain the following:

class example_services {
  @service {'httpd':
    ensure  =>'running',
    enable  => true,
    require => Package['httpd'],
  }
}

These two classes define the package and service for httpd as virtual. We then need to include these classes in our example_phpmyadmin and example_phpldapadmin classes. The modified example_phpmyadmin::pkg class will now be, as follows:

class example_phpmyadmin::pkg {
  include example_packages
  realize(Package['httpd'])
}

And the example_phpmyadmin::svc class will now be the following:

class example_phpmyadmin::svc {
  include example_services
  realize(Service['httpd'])
}

We will modify the example_phpldapadmin class in the same way and then attempt another Puppet run on client (which still has example_phpldapadmin and example_phpmyadmin classes), as shown here:

[root@client ~]# puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for client.example.com
Info: Applying configuration version '1443928369'
Notice: /Stage[main]/Example_packages/Package[httpd]/ensure: created
Notice: /Stage[main]/Example_services/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Example_services/Service[httpd]: Unscheduling refresh on Service[httpd]
Notice: Applied catalog in 11.17 seconds

For this solution to work, you need to migrate the resources that may be used by multiple modules to your top-level resource module and include the resource module wherever you need to realize the resource.

In addition to the realize function, used previously, a collector exists for virtual resources. A collector is a kind of glob that can be applied to virtual resources to realize resources based on a tag. A tag in Puppet is just a meta attribute of a resource that can be used for searching later. Tags are only used by collectors (for both virtual and exported resources, the exported resources will be explored in a later chapter) and they do not affect the resource.

To use a collector in the previous example, we will have to define a tag in the virtual resources, for the httpd package this will be, as follows:

class example_packages {
  @package {'httpd':
    ensure => 'installed',
    alias  => 'apache',
    tag    => 'apache',
  }
}

And then to realize the package using the collector, we will use the following code:

class example_phpldapadmin::pkg {
  include example_packages
  Package <| tag == 'apache' |>
}

The second solution will be to move the resource definitions into their own class and include that class whenever you need to realize the resource. This is considered to be a more appropriate way of solving the problem. Using the virtual resources described previously splits the definition of the package away from its use area.

For the previous example, instead of a class for all package resources, we will create one specifically for Apache and include that wherever we need to use Apache. We'll create the example_apache module monolithically with a single class for the package and the service, as shown in the following code:

class example_apache {
  package {'httpd':
    ensure => 'installed',
    alias  => 'apache'
  }
  service {'httpd':
    ensure  => 'running',
    enable  => true,
    require=> Package['httpd'],
  }
}

Now, in example_phpldapadmin::pkg and example_phpldapadmin::svc, we only need to include example_apache. This is because we can include a class any number of times in a catalog compilation without error. So, both our example_phpldapadmin::pkg and example_phpldapadmin::svc classes are going to receive definitions for the package and service of httpd; however, this doesn't matter, as they only get included once in the catalog, as shown in the following code:

class example_phpldapadmin::pkg {
  include example_apache
}

Both these methods solve the issue of using a resource in multiple packages. The rule is that a resource can only be defined once per catalog, but you should think of that rule as once per organization so that your modules won't interfere with those of another group within your organization.