Chapter 15: Implementing a Case Study
In this chapter, we will work on implementing a case study by applying the metaprogramming concepts that we have learned so far. For this case study, we will be using the Automobile. (1987). UCI Machine Learning Repository dataset.
In this chapter, we will be looking at the following main topics:
- Explaining the case study
- Defining base classes
- Developing a code generator library
- Generating code
- Designing an execution framework
By the end of this chapter, you should have an understanding of how to use the existing methods of the ast library in Python to enable your application to generate its own code.
Technical requirements
The code examples shared in this chapter are available on GitHub at: https://github.com/PacktPublishing/Metaprogramming-with-Python/tree/main/Chapter15.
Explaining the case study
In this section, we will be looking at the details of the case study before we start implementing it. Let’s consider a car agency, ABC Car Agency, that focuses on sales of new and used cars from multiple brands. This agency would like to build an application that produces customized catalogs for each car displaying the various specifications and features of the car.
We will look at the details available to develop and build the application by applying the concepts that we have learned throughout this book. There are 205 different cars that need to be cataloged and the data used to build this case study is taken from the following dataset: Automobile. (1987). UCI Machine Learning Repository.
There are many ways to develop an application that can solve this problem. We are going to look at how to develop a reusable application that uses metaprogramming.
A high-level view of the automobile data is as follows:
Figure 15.1 – The Automobile. (1987). UCI Machine Learning Repository dataset
For this case study, we are not going to perform any detailed data processing using the automobile dataset. Instead, we will be using the data available in this dataset to create various components for the application’s development. The flow of design for this example will start with developing a code generator library, followed by creating a code generator framework. We will then generate the ABC Car Agency library and, finally, create an execution framework. All of these processes will be explained in detail in this section.
The Python scripts that will be developed for this case study will be as follows:
Figure 15.2 – Python scripts for the ABC Car Agency case study
The car sales application will be developed by defining the following classes:
- CarSpecs
- CarMake with its subclasses
- CarCatalogue
- BodyStyle with its subclasses
- SaleType with its subclasses
Each of these classes will be explained in this section.
The overall structure of classes for this application is going to look as follows:
Figure 15.3 – Overview of the car sales application
With this understood, we will look further at the base classes for the application.
Defining base classes
We will now start building the code required for the case study.
Let’s start by developing a metaclass named CarSpecs. This class will have the following structure:
- The __new__ of the CarSpecs class will perform the following tasks:
- If the attribute of the input class is an integer, then add the attribute name in title case as feature, the value in string format as info, and type as numeric.
- If the attribute of the input class is a string, then add the attribute name in title case as feature, the value in string format as info, and type as varchar.
- If the attribute of the input class is a Boolean, then add the attribute name title case as a feature, the value in string format as info, and type as Boolean.
- If not, the actual attribute will be returned as such.
Let’s now look at the definition of CarSpecs:
from abc import ABC, abstractmethod
class CarSpecs(type):
def __new__(classitself, classname, baseclasses, attributes):
newattributes = {}
for attribute, value in attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif type(value)==int or type(value)==float:
newattributes[attribute] = {}
newattributes[attribute]['feature'] = attribute.title().replace('_', ' ')
newattributes[attribute]['info'] = str(value)
newattributes[attribute]['type'] = 'NUMERIC'
elif type(value)==str:
newattributes[attribute] = {}
newattributes[attribute]['feature'] = attribute.title().replace('_', ' ')
newattributes[attribute]['info'] = value.title()
newattributes[attribute]['type'] = 'VARCHAR'
elif type(value)==bool:
newattributes[attribute] = {}
newattributes[attribute]['feature'] = attribute.title().replace('_', ' ')
newattributes[attribute]['info'] = value.title()
newattributes[attribute]['type'] = 'BOOLEAN'
else:
newattributes[attribute] = value
return type.__new__(classitself, classname, baseclasses, newattributes)
- The next class in this example will be CarCatalogue with two abstract methods to define the color and print the catalog:
class CarCatalogue(metaclass = CarSpecs):
@abstractmethod
def define_color(self):
pass
@abstractmethod
def print_catalogue(self):
pass
- The next class will be the parent class or superclass that captures the specifications of the car:
class CarMake(metaclass = CarSpecs):
@abstractmethod
def define_spec(self):
pass
- Let’s create another superclass named BodyStyle, which will capture the body style and engine features of the car:
class BodyStyle(metaclass = CarSpecs):
@abstractmethod
def body_style_features(self):
pass
- The next class for this case study will be SaleType, in which we will add an abstract method to calculate the price of the car:
class SaleType(metaclass = CarSpecs):
@abstractmethod
def calculate_price(self):
pass
- This class will be a subclass of SaleType for calculating the price of new cars:
class New(SaleType, CarCatalogue, metaclass = CarSpecs):
def calculate_price(self, classname):
car = classname()
price = float(car.price['info'])
return price
- The next class will be another subclass of SaleType for calculating the price of resale cars:
class Resale(SaleType, CarCatalogue, metaclass = CarSpecs):
def calculate_price(self, classname, years):
car = classname()
depreciation = years * 0.15
price = float(car.price['info']) * (1 - depreciation)
return price
These are the main classes for which we will be creating templates that will be used to generate code in the next section.
Developing a code generator library
In this section, let’s look at developing a code generator that will be used to generate code for all the base classes – CarSpecs, CarMake, CarCatalogue, BodyStyle, and SaleType. The detailed steps are as follows:
- Let’s create a file named codegenerator.py and start by defining a class named CodeGenerator:
class CodeGenerator:
- Let’s define a method that imports the ast library and adds a meta_template attribute that has the string format of the CarSpecs class as a value. The meta_template attribute is further parsed and unparsed into class code:
def generate_meta(self):
ast = __import__('ast')
meta_template = '''
from abc import ABC, abstractmethod, ABCMeta
class CarSpecs(type, metaclass = ABCMeta):
def __new__(classitself, classname, baseclasses, attributes):
newattributes = {}
for attribute, value in attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif type(value)==int or type(value)==float:
newattributes[attribute] = {}
newattributes[attribute]['feature'] = attribute.title().replace('_', ' ')
newattributes[attribute]['info'] = str(value)
newattributes[attribute]['type'] = 'NUMERIC'
elif type(value)==str:
newattributes[attribute] = {}
newattributes[attribute]['feature'] = attribute.title().replace('_', ' ')
newattributes[attribute]['info'] = value.title()
newattributes[attribute]['type'] = 'VARCHAR'
elif type(value)==bool:
newattributes[attribute] = {}
newattributes[attribute]['feature'] = attribute.title().replace('_', ' ')
newattributes[attribute]['info'] = value.title()
newattributes[attribute]['type'] = 'BOOLEAN'
else:
newattributes[attribute] = value
return type.__new__(classitself, classname, baseclasses, newattributes)
'''
meta_tree = ast.parse(meta_template)
print(ast.unparse(meta_tree))
print(' ')
- Let’s now define another method named generate_car_catalogue and add the class template for CarCatalogue:
def generate_car_catalogue(self):
ast = __import__('ast')
catalogue_template = '''
class CarCatalogue(metaclass = CarSpecs):
@abstractmethod
def define_color(self):
pass
@abstractmethod
def print_catalogue(self):
pass
'''
catalogue_tree = ast.parse(catalogue_template)
print(ast.unparse(catalogue_tree))
print(' ')
- The next step is to define a method named generate_carmake_code and add the code template for the CarMake class:
def generate_carmake_code(self):
ast = __import__('ast')
carmake_template = '''
class CarMake(metaclass = CarSpecs):
@abstractmethod
def define_spec(self):
pass
'''
carmake_tree = ast.parse(carmake_template)
print(ast.unparse(carmake_tree))
print(' ')
- In the next code block, we will define another method named generate_bodystyle_parent and add the code template for the BodyStyle class:
def generate_bodystyle_parent(self):
ast = __import__('ast')
bodystyle_parent_template = '''
class BodyStyle(metaclass = CarSpecs):
@abstractmethod
def body_style_features(self):
pass
'''
bodystyle_parent_tree = ast.parse(bodystyle_parent_template)
print(ast.unparse(bodystyle_parent_tree))
print(' ')
- Let’s further define the generate_salestype_code method, which generates the class code for the SaleType class:
def generate_salestype_code(self):
ast = __import__('ast')
saletype_template = '''
class SaleType(metaclass = CarSpecs):
@abstractmethod
def calculate_price(self):
pass
'''
salestype_tree = ast.parse(saletype_template)
print(ast.unparse(salestype_tree))
print(' ')
- In this step, let’s define the generate_newsale_code method to generate code for the New class:
def generate_newsale_code(self):
ast = __import__('ast')
newsale_template = '''
class New(SaleType, CarCatalogue, metaclass = CarSpecs):
def calculate_price(self, classname):
car = classname()
price = float(car.price['info'])
return price
'''
newsale_tree = ast.parse(newsale_template)
print(ast.unparse(newsale_tree))
print(' ')
- Let’s further define the generate_resale_code method, which generates the code for the Resale class and has the method for calculating the resale price of the car:
def generate_resale_code(self):
ast = __import__('ast')
resale_template = '''
class Resale(SaleType, CarCatalogue, metaclass = CarSpecs):
def calculate_price(self, classname, years):
car = classname()
depreciation = years * 0.15
price = float(car.price['info']) * (1 - depreciation)
return price
'''
resale_tree = ast.parse(resale_template)
print(ast.unparse(resale_tree))
print(' ')
- In this step, we will define a generate_car_code method; it inherits the CarMake class, defines the color and specifications for individual car brands, and prints the catalog:
def generate_car_code(self, classname, carspecs):
self.classname = classname
self.carspecs = carspecs
ast = __import__('ast')
car_template = '''
class '''+self.classname+'''(CarMake, CarCatalogue, metaclass = CarSpecs):
fuel_type = '''+"'"+self.carspecs['fuel_type']+"'"+'''
aspiration = '''+"'"+self.carspecs['aspiration']+"'"+'''
num_of_door = '''+"'"+self.carspecs['num_of_door']+"'"+'''
drive_wheels = '''+"'"+self.carspecs['drive_wheels']+"'"+'''
wheel_base = '''+"'"+self.carspecs['wheel_base']+"'"+'''
length = '''+"'"+self.carspecs['length']+"'"+'''
width = '''+"'"+self.carspecs['width']+"'"+'''
height = '''+"'"+self.carspecs['height']+"'"+'''
curb_weight = '''+"'"+self.carspecs['curb_weight']+"'"+'''
fuel_system = '''+"'"+self.carspecs['fuel_system']+"'"+'''
city_mpg = '''+"'"+self.carspecs['city_mpg']+"'"+'''
highway_mpg = '''+"'"+self.carspecs['highway_mpg']+"'"+'''
price = '''+"'"+self.carspecs['price']+"'"+'''
def define_color(self):
BOLD = '33[5m'
BLUE = ' 33[94m'
return BOLD + BLUE
def define_spec(self):
specs = [self.fuel_type, self.aspiration, self.num_of_door, self.drive_wheels,
self.wheel_base, self.length, self.width, self.height, self.curb_weight,
self.fuel_system, self.city_mpg, self.highway_mpg]
return specs
def print_catalogue(self):
for i in self.define_spec():
print(self.define_color() + i['feature'], ": ", self.define_color() + i['info'])
'''
car_tree = ast.parse(car_template)
print(ast.unparse(car_tree))
print(' ')
- The last method of this code generator is generate_bodystyle_code, which generates class code for different body styles, such as Sedan and Hatchback, defines the color and features for an individual car body style, and prints the catalog:
def generate_bodystyle_code(self, classname, carfeatures):
self.classname = classname
self.carfeatures = carfeatures
ast = __import__('ast')
bodystyle_template = '''
class '''+self.classname+'''(BodyStyle, CarCatalogue, metaclass = CarSpecs):
engine_location = '''+"'"+self.carfeatures['engine_location']+"'"+'''
engine_type = '''+"'"+self.carfeatures['engine_type']+"'"+'''
num_of_cylinders = '''+"'"+self.carfeatures['num_of_cylinders']+"'"+'''
engine_size = '''+"'"+self.carfeatures['engine_size']+"'"+'''
bore = '''+"'"+self.carfeatures['bore']+"'"+'''
stroke = '''+"'"+self.carfeatures['stroke']+"'"+'''
compression_ratio = '''+"'"+self.carfeatures['compression_ratio']+"'"+'''
horse_power = '''+"'"+self.carfeatures['horse_power']+"'"+'''
peak_rpm = '''+"'"+self.carfeatures['peak_rpm']+"'"+'''
def body_style_features(self):
features = [self.engine_location, self.engine_type, self.num_of_cylinders, self.engine_size,
self.bore, self.stroke, self.compression_ratio, self.horse_power, self.peak_rpm]
return features
def define_color(self):
BOLD = '33[5m'
RED = ' 33[31m'
return BOLD + RED
def print_catalogue(self):
for i in self.body_style_features():
print(self.define_color() + i['feature'], ": ", self.define_color() + i['info'])
'''
bodystyle_tree = ast.parse(bodystyle_template)
print(ast.unparse(bodystyle_tree))
print(' ')
With these methods, we are all set to generate the code required for the ABC Car Agency’s catalog.
Now, let’s proceed further to develop a code generation framework that generates the hundreds of classes required for our application.
Generating code
In this section, we are going to make use of codegenerator.py to generate the base classes and its corresponding subclasses, which maintain and print various catalogs for the ABC Car Agency, as follows:
- To begin with, let’s start using the automobile data to generate the base classes required for this application. For the base data preparation, let’s import the pandas library, which helps with processing data:
import pandas as pd
- Let’s load the data and make a copy of it. For this application, we need a unique set of car brands and another unique set of car body styles:
auto = pd.read_csv("automobile.csv")
auto_truncated = auto.copy(deep=True)
auto_truncated.drop_duplicates(subset = ['make','body-style'], inplace = True)
auto_truncated.reset_index(inplace = True, drop = True)
auto_truncated['make'] = auto_truncated['make'].apply(lambda x: x.title().replace('-',''))
auto_truncated.reset_index(inplace = True)
auto_truncated['index'] = auto_truncated['index'].astype('str')
auto_truncated['make'] = auto_truncated['make'] + auto_truncated['index']
auto_truncated['body-style'] = auto_truncated['body-style'].apply(lambda x: x.title().replace('-',''))
auto_truncated['body-style'] = auto_truncated['body-style'] + auto_truncated['index']
Once the basic data has been processed, let’s create two DataFrames that will be used to generate multiple classes using the code generator:
auto_specs = auto_truncated[['make', 'fuel-type', 'aspiration', 'num-of-doors', 'drive-wheels', 'wheel-base', 'length', 'width', 'height', 'curb-weight', 'fuel-system', 'city-mpg', 'highway-mpg', 'price']].copy(deep = True)
auto_specs.columns = ['classname', 'fuel_type', 'aspiration', 'num_of_door', 'drive_wheels', 'wheel_base', 'length', 'width', 'height', 'curb_weight', 'fuel_system', 'city_mpg', 'highway_mpg', 'price' ]
for col in auto_specs.columns:
auto_specs[col] = auto_specs[col].astype('str')
auto_features = auto_truncated[['body-style', 'engine-location', 'engine-type', 'num-of-cylinders', 'engine-size', 'bore', 'stroke', 'compression-ratio', 'horsepower', 'peak-rpm']].copy(deep = True)
auto_features.columns = ['classname', 'engine_location', 'engine_type', 'num_of_cylinders', 'engine_size', 'bore', 'stroke', 'compression_ratio', 'horse_power', 'peak_rpm']
for col in auto_features.columns:
auto_features[col] = auto_features[col].astype('str')
- After processing the data into the format that we need to provide as input to the code generator, the sample data for specifications will be as follows:
Figure 15.4 – Sample specifications
- The sample data for features will be as follows:
Figure 15.5 – Sample features
- Now that the base data required to generate code is ready, we can start importing the code generator:
from codegenerator import CodeGenerator
codegen = CodeGenerator()
- In this step, let’s now define a function that generates the library by calling the code to generate each base class in a pipeline followed by generating multiple subclasses for CarMake and BodyStyle:
def generatelib():
codegen.generate_meta()
codegen.generate_car_catalogue()
codegen.generate_carmake_code()
codegen.generate_bodystyle_parent()
codegen.generate_salestype_code()
codegen.generate_newsale_code()
codegen.generate_resale_code()
for index, row in auto_specs.iterrows():
carspecs = dict(row)
classname = carspecs['classname']
del carspecs['classname']
codegen.generate_car_code(classname = classname, carspecs = carspecs)
for index, row in auto_features.iterrows():
carfeatures = dict(row)
classname = carfeatures['classname']
del carfeatures['classname']
codegen.generate_bodystyle_code(classname = classname, carfeatures = carfeatures)
- Open a Python file named abccaragencylib.py and call a generatelib function to write the code generated for all the required classes:
from contextlib import redirect_stdout
with open('abccaragencylib.py', 'w') as code:
with redirect_stdout(code):
generatelib()
code.close()
- An example class autogenerated and written into abccaragencylib.py is represented in the following screenshot:
Figure 15.6 – An autogenerated class code for a car brand
We have not autogenerated the code required for this example. We will now look at designing an execution framework.
Designing an execution framework
In this section, let’s look at the last process of designing the ABC Car Agency application where we will actually run the code generated throughout this case study:
- Let’s start by loading the autogenerated library:
import abccaragencylib as carsales
- At this stage, we will follow a sequence of steps by implementing a façade design pattern so that we can print the specifications and features for different types of cars:
class Queue:
def __init__(self, makeclass, styleclass, age):
self.makeclass = makeclass
self.styleclass = styleclass
self.make = self.makeclass()
self.style = self.styleclass()
self.new = carsales.New()
self.resale = carsales.Resale()
self.age = age
def pipeline(self):
print('*********ABC Car Agency - Catalogue***********')
self.make.print_catalogue()
print(' ')
self.style.print_catalogue()
print(' ')
print('New Car Price : ' + str(self.new.calculate_price(self.makeclass)))
print('Resale Price : ' + str(self.resale.calculate_price(self.makeclass, self.age)))
- Let’s define a method to run the façade pattern:
def run_facade(makeclass, styleclass, age):
queue = Queue(makeclass, styleclass, age)
queue.pipeline()
- In this step, we will run one combination of a car brand with a car body style to generate a catalog:
run_facade(carsales.AlfaRomero1, carsales.Hatchback28, 3)
The output is as follows:
*********ABC Car Agency - Catalogue***********
Fuel Type : Gas
Aspiration : Std
Num Of Door : Two
Drive Wheels : Rwd
Wheel Base : 94.5
Length : 171.2
Width : 65.5
Height : 52.4
Curb Weight : 2823
Fuel System : Mpfi
City Mpg : 19
Highway Mpg : 26
Engine Location : Front
Engine Type : Ohc
Num Of Cylinders : Four
Engine Size : 97
Bore : 3.15
Stroke : 3.29
Compression Ratio : 9.4
Horse Power : 69
Peak Rpm : 5200
New Car Price : 16500.0
Resale Price : 9075.0
There are 56 unique subclasses generated for CarMake and 56 unique subclasses generated for BodyStyle. We can use various combinations of CarMake and BodyStyle to print catalogs for this application.
The output generated is as follows:
*********ABC Car Agency - Catalogue***********
Fuel Type : Gas
Aspiration : Std
Num Of Door : Two
Drive Wheels : Fwd
Wheel Base : 93.7
Length : 157.3
Width : 64.4
Height : 50.8
Curb Weight : 1918
Fuel System : 2Bbl
City Mpg : 37
Highway Mpg : 41
Engine Location : Front
Engine Type : Dohc
Num Of Cylinders : Six
Engine Size : 258
Bore : 3.63
Stroke : 4.17
Compression Ratio : 8.1
Horse Power : 176
Peak Rpm : 4750
New Car Price : 5389.0
Resale Price : 1347.25
This is the step-by-step process of developing an application by applying metaprogramming methodologies in Python.
Summary
In this chapter, we have learned how to develop an application by applying various techniques of metaprogramming. We started by explaining the case study, and we defined the base classes required for this case study.
We also learned how to develop a code generator and how to generate code using it. We also designed a framework that could be used to execute or test the code generated for the application in this case study.
In the next chapter, we will be looking at some of the best practices that can be followed while designing an application with Python and metaprogramming.