Chapter 17. Creating Custom Objects with Meta-programming
In Chapter 13 we looked at the two GoF factory patterns. Both of these patterns try to solve one of the fundamental problems of object-oriented programming: How do you get hold of the right object to solve the problem at hand? How do you get the right parser for your input data? How do you get the right adapter for the database that you need to talk to? Where do you come up with just the security object that you need to deal with this version of the specification as opposed to that one?
With the factory patterns, the solution was to reduce the problem of getting hold of the right object to the problem of getting hold of the right class; pick the right class and that class will produce the right object for you. This emphasis on picking the right class makes perfect sense when you are dealing with a statically typed language, where the behavior of any given object is completely determined by its class and classes don’t change while the application is running. Under those rules, picking the right class is the only game in town.
But as we have already seen, those static rules do not apply in Ruby. Ruby allows you to modify an existing class, modify the behavior of an object independently of its class, and even evaluate strings as Ruby code, right there at runtime. In this chapter, we will look at the Meta-programming pattern, which suggests that we take advantage of these very dynamic behaviors as an alternative way to access the objects that we need. With this pattern, we adopt the perspective that, because in Ruby, classes, methods and the code inside methods are all just programming constructs of one kind or another, a good way to get to the objects that we need is to manipulate those constructs in exactly the same way that we manipulate integers and strings. If this sounds a little scary, it shouldn’t: While meta-programming certainly takes a different tack in producing the right object, at its heart this pattern focuses on leveraging the flexibility of Ruby—the same flexibility that I have been going on and on about throughout this book.
Let’s start our look at meta-programming[1] by taking yet another run at producing the denizens of our wildlife habitat simulator.
Custom-Tailored Objects, Method by Method
Imagine that we are back in the Chapter 13 jungle, trying to populate it with various plants and animals. The approach we took in Chapter 13 was to use one of the factories to pick the right class for our flora and fauna. But what if we want more flexibility? What if instead of asking for some specific type of organism from a fixed list of possibilities, we want to specify the properties of the organism and get one tailored to our needs? We could, for example, have a method to produce plant life for our habitat, where this method takes some parameters that describe the kind of plant that we are looking for. The punch line is that instead of trying to pick the right class, we could just manufacture the object that we need on the spot:
def new_plant(stem_type, leaf_type)
plant = Object.new
if stem_type == :fleshy
def plant.stem
'fleshy'
end
else
def plant.stem
'woody'
end
end
if leaf_type == :broad
def plant.leaf
'broad'
end
else
def plant.leaf
'needle'
end
end
plant
end
The preceding code creates a plain old Object instance and then proceeds to tailor this object according to the specifications supplied by the caller. Depending on which options the caller passed it, new_plant will add one variant or another of the leaf and stem methods. The resulting object is more or less unique—most of its functionality does not come from its class (it is just an Object, after all) but rather from its singleton methods. In effect, the object that comes out of new_plant is made to order.
Using the new_plant method is very simple. Just specify the kind of plant you want:
plant1 = new_plant(:fleshy, :broad)
plant2 = new_plant(:woody, :needle)
puts "Plant 1's stem: #{plant1.stem} leaf: #{plant1.leaf}"
puts "Plant 2's stem: #{plant2.stem} leaf: #{plant2.leaf}"
And that's what you will get:
Plant 1's stem: fleshy leaf: broad
Plant 2's stem: woody leaf: needle
Of course, there really is no rule that says you need to start your customizations with a plain-vanilla instance of Object. In real life, you will likely want to start with an instance of a class that provides some base level of functionality and then tweak the methods from there.
This custom-tailoring technique is particularly useful when you have lots of orthogonal features that you need to assemble into a single object. By simply manufacturing an object to specification, you can avoid creating a whole host of classes with names like WoodyStemmedNeedleLeafFloweringPlant and VinyStemmedBroadLeafNonfloweringPlant.
Custom Objects, Module by Module
If you would rather not deal with your custom objects method by method, you can always handle this task module by module. Perhaps you have separate modules for plant- and meat-eating animals:
module Carnivore
def diet
'meat'
end
def teeth
'sharp'
end
end
module Herbivore
def diet
'plant'
end
def teeth
'flat'
end
end
You could also have a second set of modules for animals that are usually up and about during the day like people (well, most people) versus those animals that prowl the night:
module Nocturnal
def sleep_time
'day'
end
def awake_time
'night'
end
end
module Diurnal
def sleep_time
'night'
end
def awake_time
'day'
end
end
Because your methods are now bunched up in nice module groupings, the code to manufacture the new objects is a little bit less tedious:
def new_animal(diet, awake)
animal = Object.new
if diet == :meat
animal.extend(Carnivore)
else
animal.extend(Herbivore)
end
if awake == :day
animal.extend(Diurnal)
else
animal.extend(Nocturnal)
end
animal
end
The extend method in this code has exactly the same effect as including the module in the normal way—extend is just a bit more convenient when we are modifying an object on the fly.
No matter whether you tailor your objects one method at a time or in module-sized chunks, the ultimate effect is to create a customized object, uniquely made to order for the requirements of the moment.
Conjuring Up Brand-New Methods
Now suppose you receive yet another requirement for your habitat simulator: Your customers would like you to model various populations of plants and animals. For example, they want to be able to group together all of the living things that live in a given area, to group together all of the tigers and trees that share a given section of a jungle or a set of nearby jungles. Oh, and while you are at it, could you please add some code to keep track of the biological classifications of all these creatures, so that we will know that this tiger is of species P. tigris, which is part of the genera Panthera, which is part of the family Felidae, and so on up to the kingdom Animalia?
On the surface, what you have here are two separate programming problems: organize the organisms by geographic population on the one hand, and organize them by biological classification on the other hand. The two problems do seem very similar—both have that Composite pattern look to them—but it does seem like you will need to sit down and write some code to handle the population problem and some different code to handle the classification problem. Right? Well, maybe not. Perhaps we can extract out the common aspects of these two problems and implement a single software facility to solve both problems in one go.
Sometimes the best way to approach a task like this one is to imagine what we want the end result to look like and then work backward to an implementation. Ideally, we want to be able to announce in our Frog or Tiger class[2] that instances of this class are part of a geographic population or part of a biological classification—or both. Something like this:
class Tiger < CompositeBase
member_of(:population)
member_of(:classification)
# Lots of code omitted . . .
end
class Tree < CompositeBase
member_of(:population)
member_of(:classification)
# Lots of code omitted . . .
end
What we are trying to say here is that instances of both the Tiger and Tree classes are the leaf nodes in two different composites—one that tracks geographic populations and another that models biological classifications.
We also need to be able to announce that the classes representing species and geographic populations are actually composites:
class Jungle < CompositeBase
composite_of(:population)
# Lots of code omitted . . .
end
class Species < CompositeBase
composite_of(:classification)
# Lots of code omitted . . .
end
Ideally, instances of our new tiger, tree, jungle, and species classes would be very easy to use. For example, we should be able to create a tiger and add it to the population of some jungle:
tony_tiger = Tiger.new('tony')
se_jungle = Jungle.new('southeastern jungle tigers')
se_jungle.add_sub_population(tony_tiger)
Once we have done that, we should be able to get the parent population of our tiger:
tony_tiger.parent_population # Should be the southeastern jungle
Finally, we should be able to do exactly the same kind of things with the biological classifications:
species = Species.new('P. tigris')
species.add_sub_classification(tony_tiger)
tony_tiger.parent_classification # Should be P. tigris
The CompositeBase class, which implements all of this magic, is shown below:
class CompositeBase
attr_reader :name
def initialize(name)
@name = name
end
def self.member_of(composite_name)
code = %Q{
attr_accessor :parent_#{composite_name}
}
class_eval(code)
end
def self.composite_of(composite_name)
member_of composite_name
code = %Q{
def sub_#{composite_name}s
@sub_#{composite_name}s = [] unless @sub_#{composite_name}s
@sub_#{composite_name}s
end
def add_sub_#{composite_name}(child)
return if sub_#{composite_name}s.include?(child)
sub_#{composite_name}s << child
child.parent_#{composite_name} = self
end
def delete_sub_#{composite_name}(child)
return unless sub_#{composite_name}s.include?(child)
sub_#{composite_name}s.delete(child)
child.parent_#{composite_name} = nil
end
}
class_eval(code)
end
end
Let’s analyze this class one bit at a time. CompositeBase starts out innocently enough: The first thing that it does is to define a very pedestrian name instance variable and an initialize method to set it. It's the second method, the member_of class method, where things start to get interesting:
def self.member_of(composite_name)
code = %Q{
attr_accessor :parent_#{composite_name}
}
class_eval(code)
end
The member_of method takes the name of the composite relationship and uses it to cook up a fragment of Ruby code. If you call member_of with an argument of :population (as our Tiger class does), member_of will generate a string that looks like this:
attr_accessor :parent_population
The member_of method then uses the class_eval method to evaluate the string as Ruby code. The class_eval method is similar to the eval method that we have seen before, the difference being that class_eval evaluates its string in the context of the class instead of the current context.[3] You have probably guessed that the net effect of all of this is to add the getter and setter methods for the parent_population instance variable to the class—which is exactly what you need if your class is to be a member of (or, more precisely, a leaf node in) a composite.
The next method in CompositeBase, composite_of, simply does more of the same—this time adding the methods appropriate for a composite object. Thus, if you call the composite_of class method from one of your classes, your class will end up with three new methods: a method to add a subitem to the composite, a method to remove a subitem, and a method to return an array holding all the subitems. Because we construct all of these methods by generating a string and then class_eval-ing the string, it is easy to insert the name of the composite into the method names. Thus, when we call member_of(:population), the methods that are actually created are add_sub_population, delete_sub_population, and sub_populations.
The key point to keep in mind about this example is that subclasses of CompositeBase do not automatically inherit any Composite pattern behavior. Instead, they inherit the member_of and composite_of class methods, which, if invoked, will add the composite methods to the subclass.
An Object's Gaze Turns Inward
A question that comes up with adding functionality the way we did with the CompositeBase class is this: How do you know if any given object is part of a composite or not? More generally, if you are meta-programming new functionality into your classes on the fly, how can you tell what any given instance can do?
You can tell by simply asking the instance. Ruby objects come equipped with a very complete set of reflection features—that is, methods that will tell you all kinds of things about an object, such as the methods that it has and its instance variables. For example, one way to determine whether an object is part of a composite as defined by CompositeBase is to look at its list of public methods:
def member_of_composite?(object, composite_name)
public_methods = object.public_methods
public_methods.include?("parent_#{composite_name}")
end
Alternatively, you can just use the respond_to? method:
def member_of_composite?(object, composite_name)
object.respond_to?("parent_#{composite_name}")
end
Reflection features like public_methods and respond_to? are handy anytime but become real assets as you dive deeper and deeper into meta-programming, when what your objects can do depends more on their history than on their class.
Using and Abusing Meta-programming
More than any other pattern in this book, the Meta-programming pattern is a very sharp tool that should be taken out of your toolkit only when needed. The key facet of meta-programming is that you are writing programs that augment or change themselves as they run. The more meta-programming that you use, the less your running program will resemble the code sitting there in your source files. This is, of course, the whole point—but it is also the danger. It is hard enough to debug ordinary code, but harder still to debug the ephemeral stuff generated by meta-programming. Thus, while a good set of unit tests is vital in getting garden-variety programs to work, such tests are absolutely mandatory for systems that use a lot of meta-programming.
A key danger with this pattern is the unexpected interaction between features. Think about the chaos that would have ensued in our habitat example if the Species class already had defined a parent_classification method when it called composite_of(:classification):
class Species < CompositeBase
# This method is about to be lost!
def parent_classification
# . . .
end
# And there it goes . . .
composite_of(:classification)
end
Sometimes you can avoid these unexpected train wrecks by adding a little meta-defensive code to your meta-programming:
class CompositeBase
# . . .
def self.member_of(composite_name)
attr_name = "parent_#{composite_name}"
raise 'Method redefinition' if instance_methods.include?(attr_name)
code = %Q{
attr_accessor :#{attr_name}
}
class_eval(code)
end
# ...
end
This version of CompositeBase will throw an exception if the parent_<composite_name> method is already defined. That approach is not ideal but is probably better than just silently obliterating an existing method.
Meta-programming in the Wild
Finding examples of meta-programming in the Ruby code base is a little like trying to find dirty clothes in my son's bedroom—you just have to glance around. Take the ubiquitous attr_accessor and its friends attr_reader and attr_writer. Recall that in Chapter 2, we talked about how all Ruby instance variables are private and that to allow the outside world to get at an instance variable you need to supply getter and setter methods:
class BankAccount
def initialize(opening_balance)
@balance = opening_balance
end
def balance
@balance
end
def balance=(new_balance)
@balance = new_balance
end
end
Chapter 2 also conveyed the good news that you did not actually need to write all of those silly methods. Instead, you could just insert the appropriate attr_reader, addr_writer, or the combination punch of attr_accessor:
class BankAccount
attr_accessor :balance
def initialize(opening_balance)
@balance = opening_balance
end
end
It turns out that attr_accessor and its reader and writer friends are not special Ruby language keywords. Rather, they are just ordinary class methods[4] along the lines of the member_of and composite_of methods that we built in this chapter.
It's actually easy to write our own version of attr_reader. Given that the name attr_reader is already taken, we will call our method readable_attribute:
class Object
def self.readable_attribute(name)
code = %Q{
def #{name}
@#{name}
end
}
class_eval(code)
end
end
Once we have readable_attribute in hand, we can use it just like attr_reader:
class BankAccount
readable_attribute :balance
def initialize(balance)
@balance = balance
end
end
We might also recall the Forwardable module, which we used to help build decorators. The Forwardable module makes creating those boring delegating methods a snap. For example, if we had a Car class with a separate Engine class, we might say this:
class Engine
def start_engine
# Start the engine...
end
def stop_engine
# Stop the engine
end
end
class Car
extend Forwardable
def_delegators :@engine, :start_engine, :stop_engine
def initialize
@engine = Engine.new
end
end
The line beginning with def_delegators creates two methods, start_engine and stop_engine, each of which delegates to the object referenced by @engine. The Forwardable module creates these methods using the same class_eval technique that we looked at in this chapter.
And then there is Rails. There is so much meta-programming going on in Rails that it is hard to know where to start. Perhaps the most notable example is in the way that you define the relationships between tables in ActiveRecord. In ActiveRecord, there is one class for each database table. If we were modeling a wildlife habitat in ActiveRecord, for example, we might have a table—and therefore a class—for the individual animals. We might also model a complete physical description of each animal in a separate table. Of course, the records in the animal and description tables would bear a one-to-one relationship with each other. ActiveRecord lets you express these kinds of relationships very neatly:
class Animal < ActiveRecord::Base
has_one :description
end
class Description < ActiveRecord::Base
belongs_to :animal
end
You can also express all of the other common database table relationships. For instance, each species includes many individual animals:
class Species < ActiveRecord::Base
has_many :animals
end
But each animal belongs to a single species:
class Animal < ActiveRecord::Base
has_one :description
belongs_to :species
end
The effect of all of this "having many" and "belonging to" kerfuffle is to add the code needed to maintain and support the various database relationships to the Animal, Description, and Species classes. Once we have the relationships that we defined above, we can ask an instance of Animal for its corresponding description object simply by saying animal.description, or we can get all of the animals that are members of a given species with something like species.animals. All of this courtesy of some ActiveRecord meta-programming.
Wrapping Up
In this chapter, we took a look at meta-programming, the idea that sometimes the easiest way to get to the code that you need is not to write it at your keyboard but rather to conjure it up programmatically, at runtime. Using the dynamic features of Ruby, we can start with a simple object and add individual methods or even whole modules full of methods to it. Also, using class_eval, we can generate completely new methods at runtime. Finally, we can take advantage of Ruby’s reflection facilities, which allow a program to examine its own structure—to look at what is there—before it changes things.
In real life, meta-programming is one of the key underpinnings of the Domain-Specific Language pattern (discussed in Chapter 16). While you can build a DSL with little or no meta-programming—which is pretty much what we did in Chapter 16—meta-programming is frequently a key ingredient in building DSLs that are both powerful and easy to use.
In the next chapter, we will round out our examination of design patterns in Ruby by looking at another pattern that fits well into the meta-programming lifestyle: Convention Over Configuration.