As we build MCollective clients and agents we will be utilizing a set of libraries which comprise the SimpleRPC Framework. These libraries give us useful tools and handlers to simplify the tasks of communicating in the MCollective ecosystem. The SimpleRPC Framework provides conventions and standards that make it easy to work with plugins provided by others as well.

The framework isn’t law. You can easily peak beneath the hood using standard Ruby commands. However it is rarely necessary, and everything we do in this book can be done by someone with little Ruby experience.

In the SimpleRPC Framework data is passed back and forth in hashes like so:

mc.say_goodbye( :msg => "Goodbye, and thanks for all the fish.", :sender => "Dolphins" )

There are a few simple rules for passing data back and forth: . Parameters must always be in a hash . Parameter names can be anything you want except :process_results as this has special meaning to the agent and client. . Parameter values can be any type your Authorization plugin (if applicable) understands: string, array, hash, boolean, etc. The default Authorization plugin understands all Ruby types.

Thankfully, these are the only rules you must know up front. Let’s get jump straight in and build an agent.

Let’s create a baseline agent to get started with. If you’re a Douglas Adams fan, you might know where I’m going with this one. This is easy to understand even if you haven’t read Hitchhiker’s Guide to the Galaxy. (Spoiler alert: The book opens with the dolphins leaving Earth.)

$ mco plugin generate agent thanks actions=say_goodbye
Created plugin directory : thanks
Created DDL file : thanks/agent/thanks.ddl
Created Agent file : thanks/agent/thanks.rb

$ cd thanks/agent
$ $EDITOR thanks.rb

You’ll find a blank module in the ruby file. Let’s go ahead and flesh this out a bit. Enter in all the bolded sections below:

module MCollective
  module Agent
    class Thanks<RPC::Agent
      action "say_goodbye" do
        validate :person, String
                person = request.data[:person]

        # This will set statuscode and statusmsg fields in the reply
        reply.fail "Who should I say goodbye to?", 1 unless person != ''
        return unless reply.statuscode == 0

        delicacy = 'fish'

        reply[:message]
          = sprintf( "Goodbye %s, and thanks for all the %s!
", person, delicacy )
      end
    end
  end
end

Here we have created an agent with a single action: say_goodbye.

In the sections below we’ll go through each part of this code and discuss what it does, and how we might have done it differently.

The data from the client is always stored in a Ruby Hash object accessible from request.data{}. The keys of the hash are of the Ruby type Symbol.

As you saw in our module above we received data from the client. You’ll want to use these validators in your code. You always validate your input, right? Here are some of the built-in validators provided by MCollective. You place these in code as shown below, using a colon before the field name to provide the Symbol that is the key in the request object.

validate :message, /^[a-zA-Z]+/
validate :message, String
validate :ipaddr, :ipv4address
validate :ipaddr, :ipv6address
validate :enable, :bool
validate :mode, ["all", "packages"]
validate :commmand, :shellsafe  # always do this before using data on the command line

You cannot pass other variables, even Symbols, into these validators. The code looks directly at the request object. validate can only be used to check input received in the request.

The validate call will throw an exception if the test fails. You should not put a rescue to catch the exception—the SimpleRPC framework will catch these and handle them.

You can create your own input validator plugins for more complex data types. We go over this in Creating Other Types of Plugins below.

As we inherited from RPC::Agent at the start of our class, we are given a reply object in which to send back our response. This object is an instance of MCollective::RPC::Reply.

In most cases you’ll want to set reply.statuscode to 0 for Success or 1 for Failure, but there is a complete table of valid response codes in the table of Results and Exceptions coming up.

The following function used in the agent above set both the statuscode and statusmsg fields with a single function. If the statuscode was not success, it would immediately return failure.

reply.fail "Who should I say goodbye to?", 1 unless person != ''
return unless reply.statuscode == 0

If we had known positively that the request had failed, we could use the following variant to bail immediately and raise an Exception:

reply.fail! "I can't find my towel.", 1

For a successful reply we want to set the statuscode at the data of the reply appropriately:

  reply[:message] = "Goodbye, and thanks for all the fish!
"
  reply[:statuscode] = 0
end

In the same directory as your agent file, you’ll find a very necessary component: thanks.ddl which provides the Data Definition Language for your plugin. DDLs are required documentation for RPC systems, as they document how the agent uses input data and returns output. The client application also uses the DDL to validate the input before submitting the request.

For our example plugin, we’re going to use the following DDL:

metadata :name        => "Thanks",
         :description => "Agent to say thanks, then grab a towel",
         :author      => "Dolphins",
         :license     => "Taken",
         :version     => "1.0",
         :url         => "http://en.wikipedia.org/wiki/The_Hitchhiker's_Guide_to_the_Galaxy",
         :timeout     => 10    # how long before killing off the request

requires :mcollective => "2.5.0"

action "say_goodbye", :description => "Says Goodbye" do
  display :always   # could be :ok or :failed

  input :person,
        :prompt      => "Person's Name",
        :description => "The name of the person we are saying goodbye to.",
        :type        => :string, # could be :number, :integer, :float, :list, or :boolean
        #:list       => ["value1","value2"]  # only for type = :list
        :validation  => '^[a-zA-Zs]+$',     # only for type = :string
        :maxlength   => 20,                  # only for type = :string
        :optional    => false,
        :default     => "Arthur"

 output :message,
        :description => "The response",
        :display_as  => "Message",
        :default     => "Goodbye, fish, thanks!"
end

The vast majority of this is obvious and easy to read. You’ll need to have an action block for each action, an input block for each input desired, an output block for each value returned. And yeah, it’s a lot of typing. The good news is that this DDL is used for data validation by the client application, meaning that you aren’t required to do simple input validation in the client.

Both validation and maxlength are required when using type string.

What if we wanted the plugin to get input from configuration files, rather than from the client?

Within the agent we could replace this line with the following call:

  [blue line-through]#delicacy = 'fish'#
  # Retrieve a configuration parameter
  delicacy = @config.pluginconf.fetch("thanks.delicacy", 'fish')

You can provide configuration data to your plugin by setting key/value pairs in one of the following two files:

/etc/mcollective/server.cfg. 

plugin.thanks.delicacy = Root Beer

/etc/mcollective/plugin.d/thanks.cfg. 

delicacy = Peanuts

Either of these files would provide the delicacy config option that the Dolphins were thanking us for.

You can make up key names to read from, but both the key name and the value are strings. Due to the way the files are parsed if you repeat a key name later in the file, the first value will be lost. In my testing the plugin-specific configuration file always won out over an entry in the main server.cfg file.

There are references on the Puppet Labs website to a plugins.d directory. This is a mistype or obsolete. The correct directory name is plugin.d.

If you are on a platform supporter by the Plugin Packager, you can easily build a package containing your agent. This is how it looks on a RedHat system:

$ cd path/to/thanks
$ mco plugin package
Building packages for mcollective-thanks plugin.
Completed building all packages for mcollective-thanks plugin.
$ ls -1 *.rpm
mcollective-thanks-1.0-1.src.rpm
mcollective-thanks-agent-1.0-1.noarch.rpm
mcollective-thanks-common-1.0-1.noarch.rpm

If you are on a platform not yet supported by the Plugin Packager, you’ll need to follow the instructions in Installing from Source. For most platforms, running the following commands on a server will do the job: (adjust for the location of server.cfg and libdir of course)

$ cd path/to/thanks/agent
$ grep libdir /etc/mcollective/server.cfg
/usr/libexec/mcollective
$ sudo cp -i thanks.rb thanks.ddl /usr/libexec/mcollective/mcollective/agent/
$ sudo service mcollective restart

Now that you have build and installed the Agent, you can invoke the agent directly from the mco command line using direct RPC calls:

$ mco rpc thanks say_goodbye person="Arthur"
Determining the amount of hosts matching filter for 2 seconds .... 1

 geode                          : OK
     "Goodbye Arthur, and thanks for all the fish!"

Now we should test that data validation worked. Our validation setting only allowed letters and spaces, no numbers. Let’s see what happens when we give it invalid input.

$ mco rpc thanks say_goodbye person="Jack0" -I geode
 * [ ============================================================> ] 1 / 1

geode                                    Invalid Request Data
   Cannot validate input person: value should match ^[A-Za-zs]+$

Finished processing 1 / 1 hosts in 85.02 ms

What about if we don’t supply a value at all?

$ mco rpc thanks say_goodbye -I geode
 * [ ============================================================> ] 1 / 1

geode
    Message: Goodbye Arthur, and thanks for all the fish!

Finished processing 1 / 1 hosts in 85.02 ms

Why did it accept that? Didn’t we say that person was required input? This is due to two things:

If you edit the DDL and remove the default value you’ll get this instead:

$ mco rpc thanks say_goodbye -I geode
The rpc application failed to run, use -v for full error backtrace
  details: Action say_goodbye needs a person argument

You can call any external script, written in any programming language (including bash) which is capable of writing out the results to a file in JSON format.

action "python_script" do
  implemented_by "/lovely/little/python/script.py"
end

The script will be given two types of input:

The script should write the reply as a JSON hash into the reply file. The return code of your script should be one of the standard result codes from the table of Results_and_Exceptions below.

If you do not specify a full path to the script, it will look for the script in the agent’s plugin directory. This makes it easy to bundle your scripts in with your agent plugins.

action "bundled_script" do
  implemented_by "bundled_script.py"
end

Location:
  $libdir/agent/agentname/bundled_script.py
 

MCollective provides a function run which makes it easy to execute command line scripts and access their STDOUT and STDERR. It’s significantly smarter and more useful than Ruby’s basic system() call, and plays well with the mcollectived. The simplest form of usage is to run a command and dump the output back in the response:

hostinput = shellescape(request[:hostname])
reply[:status] = run(
  "grep " + hostinput + " /etc/hosts",
  :stdout => :out,
  :stderr => :err
)
return reply

The run command has extensive options to set the working directory, alter the environment, and to remain trailing whitespace. Shown below is a much more extensive example. This example command will retrieve details from the local keystore file created in Trusted TLS Servers.

output = []
errors = ""
rcode = run(
  "sudo keytool -list -keystore keystore.jks",
  :stdout => output, :stderr => errors,
  :chomp => true, :cwd => "/etc/mcollective/ssl",
  :environment => {"MCOLLECTIVE_SSL_PUBLIC" => "/etc/mcollective/ssl"}
)
...from here we can process the output[] array...

As output is an array, each line of STDOUT will create a new member of the array. Each line of STDERR will be appended to the string errors.

Your agent may want to access configuration information known to the server already. For example, you may want to access a Fact known to the server. You won’t need to know how facts are supplied to mcollectived, nor acquire them yourself. You can simply access them in your current namespace:

# Access the OS family
os_type = Facts['osfamily']

# Util library provides a library with comparisons
#  supports: '>=', '>=', '>', '>', '!=', '==', and '=~'
if Util.has_fact('osfamily', 'Debian', '==') {
  if Util.has_fact('kernelmajversion', '3.1', '>=') {
    # do things appropriate for modern Debian kernels
  }
}

Likewise the Agents plugins provides useful helper methods for determining which agents are installed:

# I'd like a list of all agents
array = Agents.agentlist

# These are the same
if Agents.include?("puppet") {
if Util.has_agent?("puppet") {

And finally you can use another Util helper to find out if the Puppet manifest for a server includes a class:

if Util.has_cf_class?("webserver") {
  # do spidery things
}

The classes and functions within your plugin should always exit with a response code listed in the table below so that the appropriate Exception Handler can process the result.

When calling functions or classes you will receive either an Exceptions or a result code. Here is a simple way of handling an error from the agent we built above:

mc.goodbye(:person => "Arthur") do |resp, simpleresp|
   begin
      printrpc simpleresp
   rescue RPCError => e
      puts "Your request resulted in error: #{e}"
   end
end

When you are debugging issues with your agent you will find it useful to have logs from your agent. As a matter of practice I sprinkle debugging statements throughout my code to ease the discovery process later. You can call any of the standard logging levels using the Log class.

Log.debug("You passed me input:" + )
Log.notice("This value " + input + " isn't valid for the Goodbye function.")
Log.fatal("I blew up!")

Unfortunately there’s no easy command to generate a template for us, so we’ll just have to do this ourselves. This application will be thanks/application/thanks.rb. Assuming you are still inside the thanks/agent/ directory from earlier:

$ mkdir ../application
$ cd ../application
$ $EDITOR thanks.rb

Now let’s populate the file like so: (you can find this file in the source code supplied with the book)

class MCollective::Application::Thanks<MCollective::Application
  description "Sends a thanks message."
  usage "mco thanks [OPTIONS]"

  # This options parser updates the help page
  option :person,
         :description => "The person the dolphins say Goodbye to.",
         :arguments   => ["-p NAME", "--person NAME"],
         :type        => String,
         :require     => true

  # another hook where we could throw exceptions if the input isn't valid
  def validate_configuration(configuration)
    # this shouldn't happen since the option is mandatory above
    raise "Need to supply a person to get a reply." 
      unless configuration.include?(:person)
  end

  # Now we enter main processing
  def main
   client = rpcclient("thanks")
    printrpc client.say_goodbye(
      :person => configuration[:person],
      :options => options
    )

    # Exit using halt and it will pass on the appropriate exit code
    printrpcstats
    halt client.stats
  end
end

You may want to disable some of the standard command line options. If you put one of the following lines inside the class they will disable the relevant input options.

   exclude_argument_sections "rpc"              # disables direct rpc calls
   exclude_argument_sections "common", "filter" # limits filtering and discovery

The application will always have the –help, –verbose and –config options no matter what you disable.

Your client can be passed filters with the normal command line clients, or it can define filters itself based on other input. Following are some examples of defining filters for a request that you could put inside the main block of the client code.

Servers named web followed by a digit:

client.identity_filter "/webd/"

Servers running Debian linux or its derivatives (e.g. Ubuntu):

client.fact_filter "osfamily=Debian"

Servers with the puppet class Apache defined:

client.class_filter /apache/

Servers with less than 4 processor cores:

client.fact_filter "processorcount", "4", "<"

Reset all filters:

client.reset_filter

If you change filters, you may want to reset so that discovery is re-run:

client.class_filter /apache/
client.reset
client.fact_filter "osfamily=Debian"

If your script already knows which nodes it wants, you can disable discovery. This obviously won’t work in combination with any filters.

client.discover(:nodes => ["host1", "host2"])

You may also want to limit how many servers execute the command or how many do it concurrently, without making the CLI user specify this. Here’s an example that will set limit the targets to 30% of those who match the filter:

client.limit_targets = "30%"
client.limit_method = :random

This will set limit the targets to an absolute 20 servers which match the filter:

client.limit_targets = "20"
client.limit_method = :first

This will set batch control to that only 5 process any request from this client at one time:

client.batch_size = 5
client.batch_sleep_time = 5

This will disable batch control for a single request:

client = rpcclient("thanks")
printrpc client.say_goodbye(:person => configuration[:person], :batch_size => 0)

The application should exit using the halt handler like so:

halt client.stats

This halt handler will output result codes according to the following table:

When calling functions or classes you will receive either an Exceptions or a result code. Here is a simple way of handling an error from the agent we built above:

client.say_goodbye(:person => "Arthur") do |resp, simpleresp|
   begin
      printrpc simpleresp
   rescue RPCError => e
      puts "Your request resulted in error: #{e}"
   end
   halt client.stats
end

If you are on a platform supporter by the Plugin Packager, you can easily build the packages containing both your agent and the client. This is how it looks on a RedHat system:

$ cd path/to/thanks
$ mco plugin package
Building packages for mcollective-thanks plugin.
Completed building all packages for mcollective-thanks plugin.
$ ls -1 *.rpm
mcollective-thanks-1.0-1.src.rpm
mcollective-thanks-agent-1.0-1.noarch.rpm
mcollective-thanks-client-1.0-1.noarch.rpm
mcollective-thanks-common-1.0-1.noarch.rpm

If you are on a platform not yet supported by the Plugin Packager, you’ll need to follow the instructions in Installing from Source. For most platforms, installing the client involves running the following commands: (adjust for the location of client.cfg and libdir of course)

$ cd path/to/thanks
$ grep libdir /etc/mcollective/client.cfg
/usr/libexec/mcollective
$ sudo cp -i agent/thanks.ddl /usr/libexec/mcollective/mcollective/agent/
$ sudo cp -i application/thanks.rb /usr/libexec/mcollective/mcollective/application/

And finally we can use the client we have made:

$ mco help thanks

Sends a thanks message.
Usage: mco thanks [OPTIONS]

Application Options
    -p, --person NAME                The person the dolphins say Goodbye to.

all the standard options

$ mco thanks --person=Arthur -I geode

 * [ ==================================================> ] 1 / 1

geode
   Message: Goodbye Arthur, and thanks for all the fish!

Finished processing 1 / 1 hosts in 60.99 ms

Notice that Authorization plugins don’t have their own directory. As they are shared by many applications, they should be installed in the util directory. (Check puppet bug MCO-86 which proposes to give them their own directory.)

If you create a Facts plugin it should be named myagent_facts. After installing it on the server, alter the server configuration to the name of your fact plugin in lowercase with the _facts suffix trimmed off.

server.cfg. 

factsource = myagent
fact_cache_time = 300

The fact_cache_time parameter allows you to tune how often the facts are retrieved from the plugin. If the cost of retrieving the facts is high, you may want to tune this considerably higher. Tuning this lower than 5 minutes is not generally recommended.