34. PROXY and GATEWAY: Managing Third-Party APIs
I am endeavoring, ma’am, to construct a mnemonic circuit using stone knives and bearskins.
—Spock
There are many barriers in software systems. When we move data from our program into the database, we are crossing the database barrier. When we send a message from one computer to another, we are crossing the network barrier.
Crossing these barriers can be complicated. If we aren’t careful, our software will be more about the barriers than about the problem to be solved. The PROXY pattern helps us cross such barriers while keeping the program centered on the problem to be solved.
Proxy
Imagine that we are writing a shopping cart system for a Web site. Such a system might have objects for the customer, the order (the cart), and the products in the order. Figure 34-1 shows a possible structure. This structure is simplistic but will serve for our purposes.
Figure 34-1. Simple shopping cart object model
If we consider the problem of adding a new item to an order, we might come up with the code in Listing 34-1. The AddItem method of class Order simply creates a new Item holding the appropriate Product and quantity and then adds that Item to its internal ArrayList of Items.
Listing 34-1. Adding an item to the Object model
public class Order
{
private ArrayList items = new ArrayList();
public void AddItem(Product p, int qty);
{
Item item = new Item(p, qty);
items.Add(item);
}
}
Now imagine that these objects represent data kept in a relational database. Figure 34-2 shows the tables and keys that might represent the objects. To find the orders for a given customer, you find all orders that have the customer’s cusid. To find all the items in a given order, you find the items that have the order’s orderId. To find the products referenced by the items, you use the product’s sku.
Figure 34-2. Shopping cart relational data model
If we want to add an item row for a particular order, we’d use something like Listing 34-2. This code makes ADO.NET calls to directly manipulate the relational data model.
Listing 34-2. Adding an item to the relational model
public class AddItemTransaction : Transaction
{
public void AddItem(int orderId, string sku, int qty)
{
string sql = "insert into items values(" +
orderId + "," + sku + "," + qty + ")";
SqlCommand command = new SqlCommand(sql, connection);
command.ExecuteNonQuery();
}
}
These two code snippets are very different, but they perform the same logical function. They both connect an item to an order. The first ignores the existence of a database, and the second glories in it.
Clearly, the shopping cart program is all about orders, items, and products. Unfortunately, if we use the code in Listing 34-2, we make it about SQL statements, database connections, and piecing together query strings. This is a significant violation of SRP and possibly CCP. Listing 34-2 mixes two concepts that change for different reasons. It mixes the concept of the items and orders with the concept of relational schemas and SQL. If either concept must change for any reason, the other concept will be affected. Listing 34-2 also violates DIP, since the policy of the program depends on the details of the storage mechanism.
The PROXY pattern is a way to cure these ills. To explore this, let’s set up a test program that demonstrates the behavior of creating an order and calculating the total price. The salient part of this program is shown in Listing 34-3.
The simple code that passes this test is shown in Listings 34-4 through 34-6. The code makes use of the simple object model in Figure 34-1. It does not assume that there is a database anywhere.
Test program creates order and verifies calculation of price.
[Test]
public void TestOrderPrice()
{
Order o = new Order("Bob");
Product toothpaste = new Product("Toothpaste", 129);
o.AddItem(toothpaste, 1);
Assert.AreEqual(129, o.Total);
Product mouthwash = new Product("Mouthwash", 342);
o.AddItem(mouthwash, 2);
Assert.AreEqual(813, o.Total);
}
public class Order
{
private ArrayList items = new ArrayList();
public Order(string cusid)
{
}
public void AddItem(Product p, int qty)
{
Item item = new Item(p,qty);
items.Add(item);
}
public int Total
{
get
{
int total = 0;
foreach(Item item in items)
{
Product p = item.Product;
int qty = item.Quantity;
total += p.Price * qty;
}
return total;
}
}
}
public class Product
{
private int price;
public Product(string name, int price)
{
this.price = price;
}
public int Price
{
get { return price; }
}
}
public class Item
{
private Product product;
private int quantity;
public Item(Product p, int qty)
{
product = p;
quantity = qty;
}
public Product Product
{
get { return product; }
}
public int Quantity
{
get { return quantity; }
}
}
Figures 34-3 and 34-4 show how the PROXY pattern works. Each object to be proxied is split into three parts. The first is an interface that declares all the methods that clients will want to invoke. The second is an implementation that implements those methods without knowledge of the database. The third is the proxy that knows about the database.
Figure 34-3. PROXY static model
Figure 34-4. PROXY dynamic model
Consider the Product class. We have proxied it by replacing it with an interface. This interface has all the same methods that Product has. The ProductImplementation class implements the interface almost exactly as before. The ProductDBProxy implements all the methods of Product to fetch the product from the database, create an instance of ProductImplementation, and then delegate the message to it.
The sequence diagram in Figure 34-4 shows how this works. The client sends the Price message to what it thinks is a Product but what is in fact a ProductDBProxy. The ProductDBProxy fetches the ProductImplementation from the database and then delegates the Price property to it.
Neither the client nor the ProductImplementation knows that this has happened. The database has been inserted into the application without either party knowing about it. That’s the beauty of the PROXY pattern. In theory, it can be inserted in between two collaborating objects without their having to know about it. Thus, it can be used to cross a barrier, such as a database or a network, without either participant knowing about it.
In reality, using proxies is nontrivial. To get an idea what some of the problems are, let’s try to add the PROXY pattern to the simple shopping cart application.
Implementing PROXY
The simplest Proxy to create is for the Product class. For our purposes, the product table represents a simple dictionary. It will be loaded in one place with all the products. There is no other manipulation of this table, and that makes the proxy relatively trivial.
To get started, we need a simple database utility that stores and retrieves product data. The proxy will use this interface to manipulate the database. Listing 34-7 shows the test program for what I have in mind. Listings 34-8 and 34-9 make that test pass.
[TestFixture]
public class DBTest
{
[SetUp]
public void SetUp()
{
DB.Init();
}
[TearDown]
public void TearDown()
{
DB.Close();
}
[Test]
public void StoreProduct()
{
ProductData storedProduct = new ProductData();
storedProduct.name = "MyProduct";
storedProduct.price = 1234;
storedProduct.sku = "999";
DB.Store(storedProduct);
ProductData retrievedProduct =
DB.GetProductData("999");
DB.DeleteProductData("999");
Assert.AreEqual(storedProduct, retrievedProduct);
}
}
public class ProductData
{
private string name;
private int price;
private string sku;
public ProductData(string name,
int price, string sku)
{
this.name = name;
this.price = price;
this.sku = sku;
}
public ProductData() {}
public override bool Equals(object o)
{
ProductData pd = (ProductData)o;
return name.Equals(pd.name) &&
sku.Equals(pd.sku) &&
price==pd.price;
}
public override int GetHashCode()
{
return name.GetHashCode() ^
sku.GetHashCode() ^
price.GetHashCode();
}
}
public class Db
{
private static SqlConnection connection;
public static void Init()
{
string connectionString =
"Initial Catalog=QuickyMart;" +
"Data Source=marvin;" +
"user id=sa;password=abc;";
connection = new SqlConnection(connectionString);
connection.Open();
}
public static void Store(ProductData pd)
{
SqlCommand command = BuildInsertionCommand(pd);
command.ExecuteNonQuery();
}
private static SqlCommand
BuildInsertionCommand(ProductData pd)
{
string sql =
"INSERT INTO Products VALUES (@sku, @name, @price)";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@sku", pd.sku);
command.Parameters.Add("@name", pd.name);
command.Parameters.Add("@price", pd.price);
return command;
}
public static ProductData GetProductData(string sku)
{
SqlCommand command = BuildProductQueryCommand(sku);
IDataReader reader = ExecuteQueryStatement(command);
ProductData pd = ExtractProductDataFromReader(reader);
reader.Close();
return pd;
}
private static
SqlCommand BuildProductQueryCommand(string sku)
{
string sql = "SELECT * FROM Products WHERE sku = @sku";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@sku", sku);
return command;
}
private static ProductData
ExtractProductDataFromReader(IDataReader reader)
{
ProductData pd = new ProductData();
pd.Sku = reader["sku"].ToString();
pd.Name = reader["name"].ToString();
pd.Price = Convert.ToInt32(reader["price"]);
return pd;
}
public static void DeleteProductData(string sku)
{
BuildProductDeleteStatement(sku).ExecuteNonQuery();
}
private static SqlCommand
BuildProductDeleteStatement(string sku)
{
string sql = "DELETE from Products WHERE sku = @sku";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@sku", sku);
return command;
}
private static IDataReader
ExecuteQueryStatement(SqlCommand command)
{
IDataReader reader = command.ExecuteReader();
reader.Read();
return reader;
}
public static void Close()
{
connection.Close();
}
}
The next step in implementing the proxy is to write a test that shows how it works. This test adds a product to the database, creates a ProductProxy with the sku of the stored product, and attempts to use the accessors of Product to acquire the data from the proxy. See Listing 34-10.
[TestFixture]
public class ProxyTest
{
[SetUp]
public void SetUp()
{
Db.Init();
ProductData pd = new ProductData();
pd.sku = "ProxyTest1";
pd.name = "ProxyTestName1";
pd.price = 456;
Db.Store(pd);
}
[TearDown]
public void TearDown()
{
Db.DeleteProductData("ProxyTest1");
Db.Close();
}
[Test]
public void ProductProxy()
{
Product p = new ProductProxy("ProxyTest1");
Assert.AreEqual(456, p.Price);
Assert.AreEqual("ProxyTestName1", p.Name);
Assert.AreEqual("ProxyTest1", p.Sku);
}
}
In order to make this work, we have to separate the interface of Product from its implementation. So I changed Product to an interface and created ProductImp to implement it (see Listings 34-11 and 34-12). This forced me to make changes to TestShoppingCart (not shown) to use ProductImp instead of Product.
public interface Product
{
int Price {get;}
string Name {get;}
string Sku {get;}
}
public class ProductImpl : Product
{
private int price;
private string name;
private string sku;
public ProductImpl(string sku, string name, int price)
{
this.price = price;
this.name = name;
this.sku = sku;
}
public int Price
{
get { return price; }
}
public string Name
{
get { return name; }
}
public string Sku
{
get { return sku; }
}
}
public class ProductProxy : Product
{
private string sku;
public ProductProxy(string sku)
{
this.sku = sku;
}
public int Price
{
get
{
ProductData pd = Db.GetProductData(sku);
return pd.price;
}
}
public string Name
{
get
{
ProductData pd = Db.GetProductData(sku);
return pd.name;
}
}
public string Sku
{
get { return sku; }
}
}
The implementation of this proxy is trivial. In fact, it doesn’t quite match the canonical form of the pattern shown in Figures 34-3 and 34-4. This was an unexpected surprise. My intent was to implement the PROXY pattern. But when the implementation finally materialized, the canonical pattern made no sense.
The canonical pattern would have had ProductProxy create a ProductImp in every method. It would then have delegated that method or property to the ProductImp, as follows:
public int Price
{
get
{
ProductData pd = Db.GetProductData(sku);
ProductImpl p =
new ProductImpl(pd.Name, pd.Sku, pd.Price);
return pd.Price;
}
}
The creation of the ProductImp is a complete waste of programmer and computer resources. The ProductProxy already has the data that the ProductImp accessors would return. So there is no need to create, and then delegate to, the ProductImp. This is yet another example of how the code may lead you away from the patterns and models you expected.
Note that the Sku property of ProductProxy in Listing 34-13 takes this theme one step further. It doesn’t even bother to hit the database for the sku. Why should it? It already has the sku.
You might be thinking that the implementation of ProductProxy is very inefficient. It hits the database for each accessor. Wouldn’t it be better if it cached the ProductData item in order to avoid hitting the database?
This change is trivial, but the only thing driving us to do it is our fear. At this point, we have no data to suggest that this program has a performance problem. Besides, we know that the database engine too is doing some caching. So it’s not clear what building our own cache would buy us. We should wait until we see indications of a performance problem before we invent trouble for ourselves.
Our next step is to create the proxy for Order. Each Order instance contains many Item instances. In the relational schema (Figure 34-2), this relationship is captured within the Item table. Each row of the Item table contains the key of the Order that contains it. In the object model, however, the relationship is implemented by an ArrayList within Order (see Listing 34-4). Somehow, the proxy is going to have to translate between the two forms.
We begin by posing a test case that the proxy must pass. This test adds a few dummy products to the database, obtains proxies to those products, and uses them to invoke AddItem on an OrderProxy. Finally, the test asks the OrderProxy for the total price (see Listing 34-14). The intent of this test case is to show that an OrderProxy behaves just like an Order but obtains its data from the database instead of from in-memory objects.
[Test]
public void OrderProxyTotal()
{
Db.Store(new ProductData("Wheaties", 349, "wheaties"));
Db.Store(new ProductData("Crest", 258, "crest"));
ProductProxy wheaties = new ProductProxy("wheaties");
ProductProxy crest = new ProductProxy("crest");
OrderData od = Db.NewOrder("testOrderProxy");
OrderProxy order = new OrderProxy(od.orderId);
order.AddItem(crest, 1);
order.AddItem(wheaties, 2);
Assert.AreEqual(956, order.Total);
}
In order to make this test case work, we have to implement a few new classes and methods. The first we’ll tackle is the NewOrder method of Db. This method appears to return an instance of something called an OrderData. OrderData is just like ProductData: a simple data structure that represents a row of the Order database table. The method is shown in Listing 34-15.
public class OrderData
{
public string customerId;
public int orderId;
public OrderData() {}
public OrderData(int orderId, string customerId)
{
this.orderId = orderId;
this.customerId = customerId;
}
}
Don’t be offended by the use of public data members. This is not an object in the true sense. It is simply a container for data. It has no interesting behavior that needs to be encapsulated. Making the data variables private, and providing getters and setters would be a waste of time. I could have used a struct instead of a class, but I want the OrderData to be passed by reference rather than by value.
Now we need to write the NewOrder function of Db. Note that when we call it in Listing 34-14, we provide the ID of the owning customer but do not provide the orderId. Each Order needs an orderId to act as its key. What’s more, in the relational schema, each Item refers to this orderId as a way to show its connection to the Order. Clearly, the orderId must be unique. How does it get created? Let’s write a test to show our intent. See Listing 34-16.
[Test]
public void OrderKeyGeneration()
{
OrderData o1 = Db.NewOrder("Bob");
OrderData o2 = Db.NewOrder("Bill");
int firstOrderId = o1.orderId;
int secondOrderId = o2.orderId;
Assert.AreEqual(firstOrderId + 1, secondOrderId);
}
This test shows that we expect the orderId to somehow automatically increment every time a new Order is created. This is easily implemented by allowing SqlServer to generate the next orderId; we can get the value by calling the database method scope_identity(). See Listing 34-17.
public static OrderData NewOrder(string customerId)
{
string sql = "INSERT INTO Orders(cusId) VALUES(@cusId); " +
"SELECT scope_identity()";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@cusId", customerId);
int newOrderId = Convert.ToInt32(command.ExecuteScalar());
return new OrderData(newOrderId, customerId);
}
Now we can start to write OrderProxy. As with Product, we need to split Order into an interface and an implementation. So Order becomes the interface, and OrderImp becomes the implementation. See Listings 34-18 and 34-19.
public interface Order
{
string CustomerId { get; }
void AddItem(Product p, int quantity);
int Total { get; }
}
public class OrderImp : Order
{
private ArrayList items = new ArrayList();
private string customerId;
public OrderImp(string cusid)
{
customerId = cusid;
}
public string CustomerId
{
get { return customerId; }
}
public void AddItem(Product p, int qty)
{
Item item = new Item(p, qty);
items.Add(item);
}
public int Total
{
get
{
int total = 0;
foreach(Item item in items)
{
Product p = item.Product;
int qty = item.Quantity;
total += p.Price * qty;
}
return total;
}
}
}
How do I implement AddItem in the proxy? Clearly, the proxy cannot delegate to OrderImp.AddItem! Rather, the proxy is going to have to insert an Item row in the database. On the other hand, I really want to delegate OrderProxy.Total to OrderImp.Total, because I want the business rules—the policy for creating totals—to be encapsulated in OrderImp. The whole point of building proxies is to separate database implementation from business rules.
In order to delegate the Total property, the proxy is going to have to build the complete Order object along with all its contained Items. Thus, in OrderProxy.Total, we are going to have to read in all the items from the database, call AddItem on an empty OrderImp for each item we find, and then call Total on that OrderImp. Thus, the OrderProxy implementation ought to look something like Listing 34-20.
This implies the existence of an ItemData class and a few Db functions for manipulating ItemData rows. These are shown in Listings 34-21 through 34-23.
public class OrderProxy : Order
{
private int orderId;
public OrderProxy(int orderId)
{
this.orderId = orderId;
}
public int Total
{
get
{
OrderImp imp = new OrderImp(CustomerId);
ItemData[] itemDataArray = Db.GetItemsForOrder(orderId);
foreach(ItemData item in itemDataArray)
imp.AddItem(new ProductProxy(item.sku), item.qty);
return imp.Total;
}
}
public string CustomerId
{
get
{
OrderData od = Db.GetOrderData(orderId);
return od.customerId;
}
}
public void AddItem(Product p, int quantity)
{
ItemData id =
new ItemData(orderId, quantity, p.Sku);
Db.Store(id);
}
public int OrderId
{
get { return orderId; }
}
}
public class ItemData
{
public int orderId;
public int qty;
public string sku = "junk";
public ItemData() {}
public ItemData(int orderId, int qty, string sku)
{
this.orderId = orderId;
this.qty = qty;
this.sku = sku;
}
public override bool Equals(Object o)
{
if(o is ItemData)
{
ItemData id = o as ItemData;
return orderId == id.orderId &&
qty == id.qty &&
sku.Equals(id.sku);
}
return false;
}
}
[Test]
public void StoreItem()
{
ItemData storedItem = new ItemData(1, 3, "sku");
Db.Store(storedItem);
ItemData[] retrievedItems = Db.GetItemsForOrder(1);
Assert.AreEqual(1, retrievedItems.Length);
Assert.AreEqual(storedItem, retrievedItems[0]);
}
[Test]
public void NoItems()
{
ItemData[] id = Db.GetItemsForOrder(42);
Assert.AreEqual(0, id.Length);
}
public static void Store(ItemData id)
{
SqlCommand command = BuildItemInsersionStatement(id);
command.ExecuteNonQuery();
}
private static SqlCommand
BuildItemInsersionStatement(ItemData id)
{
string sql = "INSERT INTO Items(orderId,quantity,sku) " +
"VALUES (@orderID, @quantity, @sku)";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@orderId", id.orderId);
command.Parameters.Add("@quantity", id.qty);
command.Parameters.Add("@sku", id.sku);
return command;
}
public static ItemData[] GetItemsForOrder(int orderId)
{
SqlCommand command =
BuildItemsForOrderQueryStatement(orderId);
IDataReader reader = command.ExecuteReader();
ItemData[] id = ExtractItemDataFromResultSet(reader);
reader.Close();
return id;
}
private static SqlCommand
BuildItemsForOrderQueryStatement(int orderId)
{
string sql = "SELECT * FROM Items " +
"WHERE orderid = @orderId";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@orderId", orderId);
return command;
}
private static ItemData[]
ExtractItemDataFromResultSet(IDataReader reader)
{
ArrayList items = new ArrayList();
while (reader.Read())
{
int orderId = Convert.ToInt32(reader["orderId"]);
int quantity = Convert.ToInt32(reader["quantity"]);
string sku = reader["sku"].ToString();
ItemData id = new ItemData(orderId, quantity, sku);
items.Add(id);
}
return (ItemData[]) items.ToArray(typeof (ItemData));
}
public static OrderData GetOrderData(int orderId)
{
string sql = "SELECT cusid FROM orders " +
"WHERE orderid = @orderId";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@orderId", orderId);
IDataReader reader = command.ExecuteReader();
OrderData od = null;
if (reader.Read())
od = new OrderData(orderId, reader["cusid"].ToString());
reader.Close();
return od;
}
public static void Clear()
{
ExecuteSql("DELETE FROM Items");
ExecuteSql("DELETE FROM Orders");
ExecuteSql("DELETE FROM Products");
}
private static void ExecuteSql(string sql)
{
SqlCommand command = new SqlCommand(sql, connection);
command.ExecuteNonQuery();
}
Summary
This example should have dispelled any false illusions about the elegance and simplicity of using proxies. Proxies are not trivial to use. The simple delegation model implied by the canonical pattern seldom materializes so neatly. Rather, we find ourselves short circuiting the delegation for trivial getters and setters. For methods that manage 1:N relationships, we find ourselves delaying the delegation and moving it into other methods, just as the delegation for AddItem was moved into Total. Finally, we face the specter of caching.
We didn’t do any caching in this example. The tests all run in less than a second, so there was no need to worry overmuch about performance. But in a real application, the issue of performance, and the need for intelligent caching, is likely to arise. I do not suggest that you automatically implement a caching strategy because you fear that performance will otherwise be too slow. Indeed, I have found that adding caching too early is a good way to decrease performance. Rather, if you fear that performance may be a problem, I recommend that you conduct some experiments to prove that it will be a problem. Once proven, and only once proven, you should start considering how to speed things up.
For all the troublesome nature of proxies, they have one powerful benefit: the separation of concerns. In our example, the business rules and the database have been completely separated. OrderImp has no dependence on the database. If we want to change the database schema or the database engine, we can do so without affecting Order, OrderImp, or any of the other business domain classes.
If separation of business rules from database implementation is critically important, PROXY can be a good pattern to use. For that matter, PROXY can be used to separate business rules from any kind of implementation issue. It can be used to keep the business rules from being polluted by such things as COM, CORBA, EJB, and so on. It is a way to keep the business rule assets of your project separate from the implementation mechanisms that are currently in vogue.
Databases, Middleware, and Other Third-Party Interfaces
Third-party APIs are a fact of life for software engineers. We buy database engines, middleware engines, class libraries, threading libraries, and so on. Initially, we use these APIs by making direct calls to them from our application code (see Figure 34-5).
Figure 34-5. Initial relationship between an application and a third-party API
Over time, however, we find that our application code becomes more and more polluted with such API calls. In a database application, for example, we may find more and more SQL strings littering the code that also contains the business rules.
This becomes a problem when the third-party API changes. For databases, it also becomes a problem when the schema changes. As new versions of the API or schema are released, more and more of the application code has to be reworked to align with those changes.
Eventually, the developers decide that they must insulate themselves from these changes. So they invent a layer that separates the application business rules from the third-party API (see Figure 34-6). They concentrate into this layer all the code that uses the third-party API and all the concepts that related to the API rather than to the business rules of the application.
Figure 34-6. Introducing an insulation layer
Such layers, such as ADO.NET can sometimes be purchased. They separate the application code from the database engine. Of course, they are also third-party APIs in and of themselves, and therefore the application may need to be insulated even from them.
Note that there is a transitive dependency from the Application to the API. In some applications, that indirect dependence is still enough to cause problems. ADO.NET, for example, does not insulate the application from the details of the schema.
In order to attain even better insulation, we need to invert the dependency between the application and the layer (see Figure 34-7). This keeps the application from knowing anything about the third-party API, either directly or indirectly. In the case of a database, it keeps the application from direct knowledge of the schema. In the case of a middleware engine, it keeps the application from knowing anything about the data types used by that middleware processor.
Figure 34-7. Inverting the dependency between the application and the layer
This arrangement of dependencies is precisely what the PROXY pattern achieves. The application does not depend on the proxies at all. Rather the proxies depend on the application, and on the API. This concentrates all knowledge of the mapping between the application and the API into the proxies.
This concentration of knowledge means that the proxies are nightmares. Whenever the API changes, the proxies change. Whenever the application changes, the proxies change. The proxies can become very difficult to deal with.
It’s good to know where your nightmares live. Without the proxies, the nightmares would be spread throughout the application code.
Most applications don’t need proxies. Proxies are a heavyweight solution. When I see proxy solutions in use, my recommendation in most cases is to take them out and use something simpler. But sometimes, the intense separation between the application and the API afforded by proxies is beneficial. Those cases are almost always in very large systems that undergo frequent schema and/or API thrashing, or in systems that can ride on top of many different database engines or middleware engines.
Figure 34-8. How the Proxy inverts the dependency between the application and the layer
Table Data Gateway
PROXY is a difficult pattern to use and is overkill for most applications. I would not use it unless convinced that I needed absolute separation between the business rules and the database schema. Normally, the kind of absolute separation afforded by PROXY is not necessary, and some coupling between the business rules and the schema can be tolerated. TABLE DATA GATEWAY (TDG) is a pattern that usually achieves sufficient separation, without the cost of PROXY. Also known as data access object (DAO), this pattern uses a specialized FACADE for each type of object we wish to store in the database (see Figure 34-9).
Figure 34-9. TABLE DATA GATEWAY pattern
OrderGateway (Listing 34-24) is an interface that the application uses to access the persistence layer for Order objects. This interface has the method Insert for persisting new Orders and a method Find for retrieving already persisted Orders.
DbOrderGateway (Listing 34-25) implements the OrderGateway and moves Order instances between the object model and the relational database. It has a connection to an SqlServer instance and uses the same schema used previously in the PROXY example.1
Listing 34-24. OrderGateway.cs
public interface OrderGateway
{
void Insert(Order order);
Order Find(int id);
}
Listing 34-25. DbOrderGateway.cs
public class DbOrderGateway : OrderGateway
{
private readonly ProductGateway productGateway;
private readonly SqlConnection connection;
public DbOrderGateway(SqlConnection connection,
ProductGateway productGateway)
{
this.connection = connection;
this.productGateway = productGateway;
}
public void Insert(Order order)
{
string sql = "insert into Orders (cusId) values (@cusId)" +
"; select scope_identity()";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@cusId", order.CustomerId);
int id = Convert.ToInt32(command.ExecuteScalar());
order.Id = id;
InsertItems(order);
}
public Order Find(int id)
{
string sql = "select * from Orders where orderId = @id";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@id", id);
IDataReader reader = command.ExecuteReader();
Order order = null;
if(reader.Read())
{
string customerId = reader["cusId"].ToString();
order = new Order(customerId);
order.Id = id;
}
reader.Close();
if(order != null)
LoadItems(order);
return order;
}
private void LoadItems(Order order)
{
string sql =
"select * from Items where orderId = @orderId";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@orderId", order.Id);
IDataReader reader = command.ExecuteReader();
while(reader.Read())
{
string sku = reader["sku"].ToString();
int quantity = Convert.ToInt32(reader["quantity"]);
Product product = productGateway.Find(sku);
order.AddItem(product, quantity);
}
}
private void InsertItems(Order order)
{
string sql = "insert into Items (orderId, quantity, sku)" +
"values (@orderId, @quantity, @sku)";
foreach(Item item in order.Items)
{
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@orderId", order.Id);
command.Parameters.Add("@quantity", item.Quantity);
command.Parameters.Add("@sku", item.Product.Sku);
command.ExecuteNonQuery();
}
}
}
The other implementation of OrderGateway is InMemoryOrderGateway (Listing 34-26). InMemoryOrderGateway will save and retrieve Orders just like the DbOrderGateway, but stores the data in memory, using a Hashtable. Persisting data in memory seems rather silly, since all will be lost when the application exits. However, as we’ll see later, doing so is invaluable when it comes to testing.
Listing 34-26. InMemoryOrderGateway.cs
public class InMemoryOrderGateway : OrderGateway
{
private static int nextId = 1;
private Hashtable orders = new Hashtable();
public void Insert(Order order)
{
orders[nextId++] = order;
}
public Order Find(int id)
{
return orders[id] as Order;
}
}
We also have a ProductGateway interface (Listing 34-27) along with its DB implementation (Listing 34-28) and its in-memory implementation (Listing 34-29). Although we could also have a ItemGateway to acess data in our Items table, it’s not neccessary. The application is not interested in Items outside the context of an Order, so the DbOrderGateway deals with both the Orders table and Items table of our schema.
Listing 34-27. ProductGateway.cs
public interface ProductGateway
{
void Insert(Product product);
Product Find(string sku);
}
Listing 34-28. DbProductGateway.cs
public class DbProductGateway : ProductGateway
{
private readonly SqlConnection connection;
public DbProductGateway(SqlConnection connection)
{
this.connection = connection;
}
public void Insert(Product product)
{
string sql = "insert into Products (sku, name, price)" +
" values (@sku, @name, @price)";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@sku", product.Sku);
command.Parameters.Add("@name", product.Name);
command.Parameters.Add("@price", product.Price);
command.ExecuteNonQuery();
}
public Product Find(string sku)
{
string sql = "select * from Products where sku = @sku";
SqlCommand command = new SqlCommand(sql, connection);
command.Parameters.Add("@sku", sku);
IDataReader reader = command.ExecuteReader();
Product product = null;
if(reader.Read())
{
string name = reader["name"].ToString();
int price = Convert.ToInt32(reader["price"]);
product = new Product(name, sku, price);
}
reader.Close();
return product;
}
}
Listing 34-29. InMemoryProductGateway.cs
public class InMemoryProductGateway : ProductGateway
{
private Hashtable products = new Hashtable();
public void Insert(Product product)
{
products[product.Sku] = product;
}
public Product Find(string sku)
{
return products[sku] as Product;
}
}
The Product (Listing 34-30), Order (Listing 34-31), and Item (Listing 34-32) classes are simple data transfer objects (DTO) that conform to the original object model.
public class Product
{
private readonly string name;
private readonly string sku;
private int price;
public Product(string name, string sku, int price)
{
this.name = name;
this.sku = sku;
this.price = price;
}
public int Price
{
get { return price; }
}
public string Name
{
get { return name; }
}
public string Sku
{
get { return sku; }
}
}
public class Order
{
private readonly string cusid;
private ArrayList items = new ArrayList();
private int id;
public Order(string cusid)
{
this.cusid = cusid;
}
public string CustomerId
{
get { return cusid; }
}
public int Id
{
get { return id; }
set { id = value; }
}
public int ItemCount
{
get { return items.Count; }
}
public int QuantityOf(Product product)
{
foreach(Item item in items)
{
if(item.Product.Sku.Equals(product.Sku))
return item.Quantity;
}
return 0;
}
public void AddItem(Product p, int qty)
{
Item item = new Item(p,qty);
items.Add(item);
}
public ArrayList Items
{
get { return items; }
}
public int Total
{
get
{
int total = 0;
foreach(Item item in items)
{
Product p = item.Product;
int qty = item.Quantity;
total += p.Price * qty;
}
return total;
}
}
}
public class Item
{
private Product product;
private int quantity;
public Item(Product p, int qty)
{
product = p;
quantity = qty;
}
public Product Product
{
get { return product; }
}
public int Quantity
{
get { return quantity; }
}
}
Testing and In-Memory TDGs
Anyone who has practiced test-driven development will attest to the fact that tests add up quickly. Before you know it, you’ll have hundreds of tests. The time it takes to execute all the tests grows every day. Many of these tests will involve the persistence layer; if the real database is being used for each, you might as well take a coffee break every time you run the test suite. Hitting the database hundreds of times can be time consuming. This is where the InMemoryOrderGateway comes in handy. Since it stores data in memory, the overhead of external persistence is sidestepped.
Using the InMemoryGateway objects when testing saves a significant amout of time when executing tests. It also allows you to forget about configuration and database details, simplifying the test code. Furthermore, you don’t have to cleanup or restore an in-memory database at the end of a test; you can simply release it to the garbage collector.
InMemoryGateway objects also come in handy for acceptance testing. Once you’ve got the InMemoryGateway classes, it’s possible to run the whole application without the persistent database. I’ve found this to be handy on more than one occasion. You’ll see that the InMemoryOrderGateway has very little code and that what code it does have is trivial.
Of course, some of your unit tests and acceptance tests should use the persistent versions of the gateways. You do have to make sure that your system works with the real database. However, most of your tests can be redirected to the in-memory gateways.
With all the benefits of in-memory gateways, it makes a lot of sense to write and use them where appropriate. Indeed, when I use the TABLE DATA GATEWAY pattern, I start by writing the InMemoryGateway implementation and hold off on writing the DbGateway classes. It’s quite possible to build a great deal of the application by using only the InMemoryGateway classes. The application code doesn’t know that it’s not really using the database. This means that it’s not important to worry about which database tools you’re going to use or what the schema will look like until much later. In fact, the DbGateways can be one of the last components to be implemented.
Testing the DB Gateways
Listings 34-34 and 34-35 show the unit tests for the DBProductGateway and the DBOrderGateway. The structure of these tests is interesting because they share a common abstract base class: AbstractDBGatewayTest.
Note that the constructor for DbOrderGateway requires an instance of ProductGateway. Note also that the InMemoryProductGateway rather than the DbProductGateway is being used in the tests. The code works fine despite this trick, and we save a few roundtrips to the database when we run the tests.
Listing 34-33. AbstractDbGatewayTest.cs
public class AbstractDbGatewayTest
{
protected SqlConnection connection;
protected DbProductGateway gateway;
protected IDataReader reader;
protected void ExecuteSql(string sql)
{
SqlCommand command =
new SqlCommand(sql, connection);
command.ExecuteNonQuery();
}
protected void OpenConnection()
{
string connectionString =
"Initial Catalog=QuickyMart;" +
"Data Source=marvin;" +
"user id=sa;password=abc;";
connection = new SqlConnection(connectionString);
this.connection.Open();
}
protected void Close()
{
if(reader != null)
reader.Close();
if(connection != null)
connection.Close();
}
}
Listing 34-34. DbProductGatewayTest.cs
[TestFixture]
public class DbProductGatewayTest : AbstractDbGatewayTest
{
private DbProductGateway gateway;
[SetUp]
public void SetUp()
{
OpenConnection();
gateway = new DbProductGateway(connection);
ExecuteSql("delete from Products");
}
[TearDown]
public void TearDown()
{
Close();
}
[Test]
public void Insert()
{
Product product = new Product("Peanut Butter", "pb", 3);
gateway.Insert(product);
SqlCommand command =
new SqlCommand("select * from Products", connection);
reader = command.ExecuteReader();
Assert.IsTrue(reader.Read());
Assert.AreEqual("pb", reader["sku"]);
Assert.AreEqual("Peanut Butter", reader["name"]);
Assert.AreEqual(3, reader["price"]);
Assert.IsFalse(reader.Read());
}
[Test]
public void Find()
{
Product pb = new Product("Peanut Butter", "pb", 3);
Product jam = new Product("Strawberry Jam", "jam", 2);
gateway.Insert(pb);
gateway.Insert(jam);
Assert.IsNull(gateway.Find("bad sku"));
Product foundPb = gateway.Find(pb.Sku);
CheckThatProductsMatch(pb, foundPb);
Product foundJam = gateway.Find(jam.Sku);
CheckThatProductsMatch(jam, foundJam);
}
private static void CheckThatProductsMatch(Product pb, Product
pb2)
{
Assert.AreEqual(pb.Name, pb2.Name);
Assert.AreEqual(pb.Sku, pb2.Sku);
Assert.AreEqual(pb.Price, pb2.Price);
}
}
Listing 34-35. DbOrderGatewayTest.cs
[TestFixture]
public class DbOrderGatewayTest : AbstractDbGatewayTest
{
private DbOrderGateway gateway;
private Product pizza;
private Product beer;
[SetUp]
public void SetUp()
{
OpenConnection();
pizza = new Product("Pizza", "pizza", 15);
beer = new Product("Beer", "beer", 2);
ProductGateway productGateway =
new InMemoryProductGateway();
productGateway.Insert(pizza);
productGateway.Insert(beer);
gateway = new DbOrderGateway(connection, productGateway);
ExecuteSql("delete from Orders");
ExecuteSql("delete from Items");
}
[TearDown]
public void TearDown()
{
Close();
}
[Test]
public void Find()
{
string sql = "insert into Orders (cusId) " +
"values ('Snoopy'), select scope_identity()";
SqlCommand command = new SqlCommand(sql, connection);
int orderId = Convert.ToInt32(command.ExecuteScalar());
ExecuteSql(String.Format("insert into Items (orderId, " +
"quantity, sku) values ({0}, 1, 'pizza')", orderId));
ExecuteSql(String.Format("insert into Items (orderId, " +
"quantity, sku) values ({0}, 6, 'beer')", orderId));
Order order = gateway.Find(orderId);
Assert.AreEqual("Snoopy", order.CustomerId);
Assert.AreEqual(2, order.ItemCount);
Assert.AreEqual(1, order.QuantityOf(pizza));
Assert.AreEqual(6, order.QuantityOf(beer));
}
[Test]
public void Insert()
{
Order order = new Order("Snoopy");
order.AddItem(pizza, 1);
order.AddItem(beer, 6);
gateway.Insert(order);
Assert.IsTrue(order.Id != -1);
Order foundOrder = gateway.Find(order.Id);
Assert.AreEqual("Snoopy", foundOrder.CustomerId);
Assert.AreEqual(2, foundOrder.ItemCount);
Assert.AreEqual(1, foundOrder.QuantityOf(pizza));
Assert.AreEqual(6, foundOrder.QuantityOf(beer));
}
}
Using Other Patterns with Databases
Four other patterns that can be used with databases are EXTENSION OBJECT, VISITOR, DECORATOR, and FACADE.2
- Extension Object: Imagine an extension object that knows how to write the extended object on a database. In order to write such an object, you would ask it for an extension object that matched the
Databasekey, cast it to aDatabaseWriterExtension, and then invoke thewritefunction:
Product p = /* some function that returns a Product */
ExtensionObject e = p.GetExtension("Database");
if (e != null)
{
DatabaseWriterExtension dwe = (DatabaseWriterExtension) e;
e.Write();
} - Visitor: Imagine a visitor hierarchy that knows how to write the visited object on a database. You would write an object on the database by creating the appropriate type of visitor and then calling
Accepton the object to be written:
Product p = /* some function that returns a Product */
DatabaseWriterVisitor dwv = new DatabaseWritierVisitor();
p.Accept(dwv); - Decorator: There are two ways to use a decorator to implement databases. You can decorate a business object and give it read and write methods, or you can decorate a data object that knows how to read and write itself and give it business rules. The latter approach is not uncommon when using object-oriented databases. The business rules are kept out of the OODB schema and added in with decorators.
- Facade: This is my favorite starting point; indeed, TABLE DATA GATEWAY is simply a special case of FACADE. On the down side, FACADE does not completely decouple the business rule objects from the database. Figure 34-10 shows the structure. The
DatabaseFacadeclass simply provides methods for reading and writing all the necessary objects. This couples the objects to theDatabaseFacadeand vice versa. The objects know about the facade because they are often the ones that call the read and write functions. The facade knows about the objects because it must use their accessors and mutators to implement the read and write functions.
This coupling can cause problems in larger applications, but in smaller apps or in apps that are just starting to grow, it’s a pretty effective technique. If you start using a facade and then later decide to change to one of the other patterns to reduce coupling, the facade is pretty easy to refactor.
Conclusion
It is very tempting to anticipate the need for PROXY long before the need exists. This is almost never a good idea. I recommend that you start with TABLE DATA GATEWAY or some other kind of FACADE and then refactor as necessary. You’ll save yourself time and trouble if you do.
Bibliography
[Fowler03] Martin Fowler, Patterns of Enterprise Application Architecture, Addison-Wesley, 2003.
[GOF95] Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software, Addison-Wesley, 1995.
[Martin97] Robert C. Martin, “Design Patterns for Dealing with Dual Inheritance Hierarchies,” C++ Report, April 1997.