In addition to the Profiles
database is the CustomerProfile
Python class. This object
performs XML input and output against the database, and exposes methods
for use by the XML Switch. The basic profile actions allow you to
retrieve, insert, update, and delete profiles. Arguments to these
methods are XML versions of profile data. By making the customer profile
packets in XML, it’s easy for other applications to generate and consume
the packets without any concern for the structure of the database, or
even how to access it directly. In fact, a CustomerProfile
can easily become the payload
of a SOAP message. A profile packet appears as:
<CustomerProfile id="555-99JKK39"> <firstname>John</firstname> <lastname>Doolittle</lastname> <address1>396 Evergreen Terrace</address1> <address2/> <city>Springfield</city> <state>WA</state> <zip>98072</zip> </CustomerProfile>
Note that the address2
element
exists, even though it is empty. The DTD for such a document appears
as:
<!ELEMENT firstname (#PCDATA)> <!ELEMENT lastname (#PCDATA)> <!ELEMENT address1 (#PCDATA)> <!ELEMENT address2 (#PCDATA)> <!ELEMENT city (#PCDATA)> <!ELEMENT state (#PCDATA)> <!ELEMENT zip (#PCDATA)> <!ATTLIST CustomerProfile id CDATA #REQUIRED> <!ELEMENT CustomerProfile (firstname, lastname, address1, address2, city, state, zip)>
An instance of the document using the DTD needs to have the declaration within the document as well:
<?xml version="1.0"?>
<!DOCTYPE CustomerProfile SYSTEM "CustomerProfile.dtd">
<CustomerProfile id="555-99JKK39">
<firstname>John</firstname>
In order to keep things within the scope of this chapter, DTD
enforcement is not a part of the CustomerProfile
Python class, although the DTD
rides along with the document and may be utilized at a later date. When
embedding CustomerProfile
elements
within an XML message, the prolog is stripped out, and only the CustomerProfile
element is inserted into the
XML message.
The CustomerProfile
class supports four distinct operations. These operations allow for
retrieval, insertion, updates, and deletes. This class is used by the
XML switch to manage the insertion and retrieval of CustomerProfile
information at runtime in
the distributed system. All communication to and from this class takes
the form of XML—this enables greater flexibility in how the data is
stored on the backend. This also alleviates the burden of requiring
distributed applications to connect directly to the database and
understand the structure of its tables. In this scenario, distributed
applications only need to understand structure of a CustomerProfile
document.
getProfile(
id
)
This method accepts a customer profile ID and returns the
corresponding information in a well-formed, valid CustomerProfile
document in the form
of a string.
getProfileAsDom(
id
)
This method is identical to getProfile
, except that the return
value is not a string of XML, but rather a DOM instance. The DOM
can then be used for further manipulation.
insertProfile(
strXML
)
The insertProfile
method takes a valid, well-formed CustomerProfile
document as a string
and inserts it into the database.
updateProfile(
strXML
)
Similar to insertProfile
, this method takes a
fresh XML CustomerProfile
chunk and updates the existing record in the database based on
the customer ID. Under the covers, it performs a delete and
insert respectively.
deleteProfile(
id
)
This method takes a customer ID as a parameter, and deletes the corresponding record from the database.
With the exception of the getProfile
and getProfileAsDom
methods, these methods
return either 1
or 0
(true
or false
) to the caller, enabling
them to be used as arguments to if
statements.
The CustomerProfile
class for retrieving profiles exposes two methods: getProfile
and getProfileAsDom
. Both methods take a
customerId
as an argument. In a
simple test case (in the file runcp.py), you
could use the methods as follows:
from CustomerProfile import CustomerProfile from xml.dom.ext import PrettyPrint cp = CustomerProfile( ) print "String of XML:" print cp.getProfile("234-E838839") print "Or retrieve a DOM:" dom = cp.getProfileAsDom("234-E838839") PrettyPrint(dom)
This assumes that you have populated a record in the database
with a customerId
of 234-E838839
. The result of running this code
is the output of two identical XML representations:
G:9780596001285c10> python runcp.py String of XML: <?xml version='1.0' encoding='UTF-8'?> <!DOCTYPE CustomerProfile SYSTEM "CustomerProfile.dtd"> <CustomerProfile id='234-E838839'> <firstname>John</firstname> <lastname>Smith</lastname> <address1>123 Evergreen Terrace</address1> <address2> </address2> <city>Podunk</city> <state>WA</state> <zip>98072</zip> </CustomerProfile> Or retrieve a DOM: <?xml version='1.0' encoding='UTF-8'?> <!DOCTYPE CustomerProfile SYSTEM "CustomerProfile.dtd"> <CustomerProfile id='234-E838839'> <firstname>John</firstname> <lastname>Smith</lastname> <address1>123 Evergreen Terrace</address1> <address2> </address2> <city>Podunk</city> <state>WA</state> <zip>98072</zip> </CustomerProfile>
Whether you would like a raw XML string or a DOM really depends on what you want to do with the record after you obtain it. If passing it back to another application, it’s wise to send the string, or make the string a piece of another document such as a SOAP packet (extracting the prolog declarations, of course). If you wish to manipulate the result, and perhaps insert it into the database once again, a DOM may be more convenient to work with.
The code for getProfile
is straightforward. First, some
simple validation is performed on the parameters, and then a
database connection is prepared with a simple SQL statement:
def getProfile(self, strId, dom=0): """ getProfile - returns an XML profile chunk based on the supplied id. Returns None if not found. """ if not strId: return None # generate connection conn = odbc.odbc(CONNECTION_STRING) cmd = conn.cursor( ) cmd.execute("select * from customer where " + "customerId = '" + strId + "'") conn.close( )
In this code, that strId
is
inspected before anything else occurs, and the database command is
of a simple select
*
variety. Of note is the third default
parameter, dom=0
, present in the
method definition. This is flipped on by getProfileAsDom
to require that getProfile
return a DOM instance instead
of a string.
Next, the record is retrieved, and an XML document is
prepared using the DOMImplementation
class.
# get data record prof_fields = cmd.fetchone( ) if prof_fields is None: return None # generate XML from fields # generate CustomerProfile doctype doctype = implementation.createDocumentType( "CustomerProfile", "", "CustomerProfile.dtd") # generate new document with doctype newdoc = implementation.createDocument( "", "CustomerProfile", doctype) rootelem = newdoc.documentElement # create root element id attribute rootelem.setAttribute("id", prof_fields[CUSTOMER_ID])
In this code, the DOMImplementation
class is used to build
an XML document. In the beginning of the snippet, prof_fields
is populated with the raw list
of values from the database using the fetchone
method, which returns one row.
The indexes into prof_fields
are
given as named constants defined in the CustomerProfile
module; the definitions
are listed in the complete source listing for the module in Example 10-2, later in this
chapter. Next, a document type object is created, citing the
CustomerProfile.dtd
file created
earlier. This object is then used in the call to implementation.createDocument
to generate
an empty XML document element.
At this point, the simplicity of using the DOMImplementation
to build your XML
document (as opposed to manually constructing a string of XML, which
is done later) is illustrated when we construct the elements in a
simple loop:
# create list with field values fields = ["firstname", prof_fields[FIRSTNAME], "lastname", prof_fields[LASTNAME], "address1", prof_fields[ADDRESS1], "address2", prof_fields[ADDRESS2], "city", prof_fields[CITY], "state", prof_fields[STATE], "zip", prof_fields[ZIP], ] # loop through list adding elements and element text for pos in range(0, len(fields), 2): # create the element thisElement = newdoc.createElement(fields[pos]) # check for empty values and convert to soft nulls if fields[pos + 1] is None: fields[pos + 1] = "" thisElement.appendChild(newdoc.createTextNode(fields[pos+1])) rootelem.appendChild(thisElement)
What is interesting about this code is that you place all of the database values in a list, and then iterate the list creating and appending elements as you go. Use a list to hold the data values instead of a dictionary, because you need to create elements in the XML document in a specific order or it won’t comply with the DTD. (Python dictionaries return their keys in an unpredictable order, and certainly do not maintain the order in which you inserted them).
Using a list over a dictionary poses no big feat, as the
Python range
function is used to
hop through the list in steps of two, allowing the code to use the
current list member as the element name, and the next list member as
the element’s character data content.
With the XML in hand, a decision is made whether to return a
string of XML or a DOM, based on how the method was called
(remember, getProfileAsDom
calls
getProfile
with additional
parameters):
# return DOM or String based on how we were called... if dom: return newdoc else: # return string strXML = StringIO.StringIO( ) PrettyPrint(newdoc, strXML) return strXML.getvalue( )
At this point, the caller’s request has been satisfied. If a
string is returned, the StringIO
class is used in conjunction with the PrettyPrint
method to write the XML into
the string as if it were a file. Using a DOMImplementation
to create your document,
as opposed to manually constructing one with a string, offers
several benefits. First, it removes tricky details such as preparing
a DTD and encoding declarations from inside string assignment
statements. It’s far easier to maintain the code if you can
programmatically alter the encoding or Document Type, without
resorting to editing XML by hand inside string assignment
statements. Second, greater flexibility is enabled if you need to
change the structure of the document. It’s easier in the long run to
manipulate nodes and their position in the document with a DOM than
to parse and manipulate a text string. However, there are many times
when you may want to concatenate a string of XML together and create
a fresh DOM (in this case, you still wind up with a programmatically
accessible DOM representation of the XML).
Using getProfileAsDom
works
the same as getProfile
, but a DOM
instance is returned to you instead of a string of XML.
def getProfileAsDom(self, strId): """ This method calls getProfile with the dom option set on, which causes getProfile to return its created DOM object, and not an XML string """ return self.getProfile(strId, 1)
This approach was taken because it’s often easier to provide intuitive convenience functions as opposed to loading down a method with conditional parameters that a user must learn.
The insertProfile
method is the other workhorse method of the CustomerProfile
class. Whereas getProfile
builds a DOM, insertProfile
deconstructs a DOM to place
its values in the database. Deleting profiles is relatively easy as
just a single SQL statement is sent straight to the database with a
supplied customer ID.
The insertProfile
method is
used to add new XML CustomerProfile
documents to the database. This interface prevents client applications
from having to connect to the database or understand the table
structure. Arguably, sharing the structure of XML documents among
distributed applications is easier than sharing database structure and
potentially having to require support for proprietary data
types.
Use insertProfile
with a
string of XML such as the following:
cp = CustomerProfile( )
cp.insertProfile("<string of XML>")
The method returns true
on
success, and false
on failure.
Additionally, exceptions may be propagated if they occur in handling
code.
The code for insertProfile
is simple as well, but touches on some intricate DOM manipulation.
It starts off, as does getProfile
, with some simple validation
and the retrieval of a DOM instance (as opposed to a DOMImplementation
in getProfile
):
def insertProfile(self, strXML): """ insertProfile takes an XML chunk as a string, parses down its fields, and inserts it into the profile database. Raises an exception if XML is not well-formed, or customer Id is missing, or if SQL execution fails. Returns 1 on success, 0 on failure. """ if not strXML: return 0 # Begin parsing XML try: doc = FromXml(strXML) except: print "Error parsing document." return 0 # Normalize white space and retrieve root element doc.normalize( ) elem = doc.documentElement
Probably the most important things in the preceding code are
the calls to FromXml
and the call
to the normalize
method of the
instantiated document object. The FromXml
method is part of the Sax2
reader package, and allows for the
construction of a DOM object from a raw string of XML data.
The call to the doc.normalize
method is important as well.
The structure of a CustomerProfile
XML chunk is quite simple.
The character data residing in the elements is short, and needs no
peripheral whitespace. That is, if a web form or GUI has placed
carriage returns inside the elements, they can be safely eliminated.
This step is critical to how the elements are processed inside
insertProfile
. Without the
normalization call, it’s possible that the text contained in the
element may be contained in multiple nodes, and the firstChild
attribute provides only the
first of these; the call to normalize
ensures that all adjacent text
nodes are collapsed into a single node.
Next, the values are extracted out of the fresh XML DOM and used to populate a SQL statement.
# Extract values from XML packet customerId = elem.getAttributeNS(None, 'id') firstname = self.extractNodeValue(elem, "firstname") lastname = self.extractNodeValue(elem, "lastname") address1 = self.extractNodeValue(elem, "address1") address2 = self.extractNodeValue(elem, "address2") city = self.extractNodeValue(elem, "city") state = self.extractNodeValue(elem, "state") zip = self.extractNodeValue(elem, "zip") # prepare SQL statement strSQL = ("insert into Customer values (" "'" + firstname + "', " "'" + lastname + "', " "'" + address1 + "', " "'" + address2 + "', " "'" + city + "', " "'" + state + "', " "'" + zip + "', " "'" + customerId + "')")
Here, the customerId
attribute is extracted from the root element, and then a series of
calls to the extractNodeValue
helper method are issued. This small method (shown next) takes the
current element and attempts to extract a target’s text value
beneath itself. Since this step is repeated for every element you
need, it’s easier to relegate it to an internal function rather than
duplicate the code in each method that uses it. The preparation of
the SQL statement takes the results of the calls to extractNodeValue
and assembles a SQL
insert
statement. The work of
extractNodeValue
is shown as
follows:
def extractNodeValue(self, elem, elemName): """ Internal method to parse UNIQUE elements for text value or substitute empty strings. """ e = elem.getElementsByTagName(elemName)[0].firstChild if e is not None: return e.nodeValue else: return ""
This method attempts to extract a child element from the element it is given, and also tests for its character data content. If not available, an empty string is returned.
Now that the SQL statement is prepared, it can be sent to the database:
# generate connection conn = odbc.odbc(CONNECTION_STRING) cmd = conn.cursor( ) # execute SQL statement if not cmd.execute(strSQL): return 0 conn.close( ) return 1
If communication with the database proceeds as expected, the
method returns a positive 1
to
the caller.
The code to delete a profile from the database is
easy, and relies mainly on taking the supplied customer ID and using
it as a parameter in a SQL delete
statement:
def deleteProfile(self, strId): """ deleteProfile accepts a customer profile ID and deletes the corresponding record in the database. Returns 1 on success, 0 on failure. """ if not strId: return 0 # generate database connection conn = odbc.odbc(CONNECTION_STRING) cmd = conn.cursor( ) ok = cmd.execute("delete from customer where customerId = '" + strId + "'"): conn.close( ) return ok and 1 or 0
The result of the SQL operation is indicated to the caller by
either a 1
or 0
return value.
The process of updating a profile using the CustomerProfile
class is simple. When
calling updateProfile
, you supply a
new chunk of XML data. The customerId
of this XML chunk must match an
existing ID in the database. If so, the old record is deleted, and the
new one is inserted.
As mentioned earlier, the updateProfile
method uses insertProfile
and deleteProfile
internally, but is exposed to
make the CustomerProfile
class
easier to use.
In order to extract the customerId
from the supplied chunk of XML, a
DOM object is briefly instantiated to parse the data:
def updateProfile(self, strXML): """ This convenience function accepts a new customer profile XML packet. It extracts the customer ID and then calls deleteProfile and insertProfile using the new XML. The return value for the insert is propagated back to the caller, unless the delte fails, in which case it is propagated back and insert is never called. """ # parse document for customer Id try: doc = FromXml(strXML) customerId = doc.documentElement.getAttributeNS(None,'id') except: print "Error parsing document." return 0
As the preceding code shows, FromXml
is used once again to convert the
string-based XML data into a DOM object. The customerId
is then extracted using a call to
getAttributeNS
. Since the reader.Sax2
package is used, you must use a
namespace-oriented DOM method as opposed to the normal getAttribute
method (this is a debated bug
in the implementation that will hopefully be removed by the time of this
printing, at which point getAttribute
will work as well. To
participate in the lively commentary, join the XML-SIG at http://www.python.org). With the customerId
in hand, you can utilize the
existing insert and delete methods:
# attempt to delete and insert based on customerId if self.deleteProfile(customerId): return self.insertProfile(strXML) else: return 1
Here returns from these functions are propagated back to the caller as the return value of a call to this function.
The CustomerProfile
class is quite a workhorse. It allows customer information to be
stored on the network as XML. Any distributed application that can
access the CustomerProfile
class
can utilize its functionality without knowing anything about the
underlying database or storage medium. Additionally, it’s possible to
route XML to an application hosting the CustomerProfile
class to perform an update.
In this best-case scenario, the calling application need not even know
of the CustomerProfile
class, but
instead just construct the appropriate SOAP or XML message format, and
submit it to the network. We perform this operation later in this
chapter after building the XML switch, in Section 10.6.5 and Section 10.8.5.
The complete listing of the CustomerProfile
class is shown in Example 10-2.
""" CustomerProfile.py """ import dbi import odbc import StringIO from xml.dom import implementation from xml.dom.ext import PrettyPrint from xml.dom.ext.reader.Sax2 import FromXml # define some global members for the class CONNECTION_STRING = "Profiles/webuser/w3bus3r" FIRSTNAME = 0 LASTNAME = 1 ADDRESS1 = 2 ADDRESS2 = 3 CITY = 4 STATE = 5 ZIP = 6 CUSTOMER_ID = 7 class CustomerProfile: """ CustomerProfile - manages the storage and retrieval of XML customer profiles from a relational database. """ def getProfileAsDom(self, strId): """ This method calls getProfile with the dom option set on, which causes getProfile to return its created DOM object, and not an XML string """ return self.getProfile(strId, 1) def getProfile(self, strId, dom=0): """ getProfile - returns an XML profile chunk based on the supplied id. Returns None if not found. """ if not strId: return None # generate connection conn = odbc.odbc(CONNECTION_STRING) cmd = conn.cursor( ) cmd.execute("select * from customer where " + "customerId = '" + strId + "'") conn.close( ) # get data record prof_fields = cmd.fetchone( ) if prof_fields is None: return None # generate XML from fields # generate CustomerProfile doctype doctype = implementation.createDocumentType( "CustomerProfile", "", "CustomerProfile.dtd") # generate new document with doctype newdoc = implementation.createDocument( "", "CustomerProfile", doctype) rootelem = newdoc.documentElement # create root element id attribute rootelem.setAttribute("id",prof_fields[CUSTOMER_ID] ) # create list with field values fields = ["firstname", prof_fields[FIRSTNAME], "lastname", prof_fields[LASTNAME], "address1", prof_fields[ADDRESS1], "address2", prof_fields[ADDRESS2], "city", prof_fields[CITY], "state", prof_fields[STATE], "zip", prof_fields[ZIP], ] # loop through list adding elements and element text for pos in range(0, len(fields), 2): # create the element thisElement = newdoc.createElement(fields[pos]) # check for empty values and convert to soft nulls if fields[pos + 1] is None: fields[pos + 1] = "" thisElement.appendChild(newdoc.createTextNode(fields[pos+1])) rootelem.appendChild(thisElement) # return DOM or String based on how we were called... if dom: return newdoc else: # return string strXML = StringIO.StringIO( ) PrettyPrint(newdoc, strXML) return strXML.getvalue( ) def insertProfile(self, strXML): """ insertProfile takes an XML chunk as a string, parses down its fields, and inserts it into the profile database. Raises an exception if XML is not well-formed, or customer Id is missing, or if SQL execution fails. Returns 1 on success, 0 on failure. """ if not strXML: raise Exception("XML String not provided.") # Beign parsing XML try: doc = FromXml(strXML) except: print "Error parsing document." return 0 # Normalize whitespace and retrive root element doc.normalize( ) elem = doc.documentElement # Extract values from XML packet customerId = elem.getAttributeNS(None, 'id') firstname = self.extractNodeValue(elem, "firstname") lastname = self.extractNodeValue(elem, "lastname") address1 = self.extractNodeValue(elem, "address1") address2 = self.extractNodeValue(elem, "address2") city = self.extractNodeValue(elem, "city") state = self.extractNodeValue(elem, "state") zip = self.extractNodeValue(elem, "zip") # prepare SQL statement strSQL = ("insert into Customer values (" "'" + firstname + "', " "'" + lastname + "', " "'" + address1 + "', " "'" + address2 + "', " "'" + city + "', " "'" + state + "', " "'" + zip + "', " "'" + customerId + "')") # create connection conn = odbc.odbc(CONNECTION_STRING) cmd = conn.cursor( ) # execute SQL statement if not cmd.execute(strSQL): raise Exception("SQL Exec failed.") conn.close( ) return 1 def extractNodeValue(self, elem, elemName): """ Internal method to parse UNIQUE elements for text value or substitute empty strings. """ e = elem.getElementsByTagName(elemName)[0].firstChild if e is None: return "" else: return e.nodeValue def updateProfile(self, strXML): """ This convenience function accepts a new customer profile XML packet. It extracts the customer ID and then calls deleteProfile and insertProfile using the new XML. The return value for the insert is propagated back to the caller, unless the delte fails, in which case it is propagated back and insert is never called. """ # parse document for customer Id try: doc = FromXml(strXML) customerId = doc.documentElement.getAttributeNS(None,'id') except: print "Error parsing document." return 0 # attempt to delete and insert based on customerId if self.deleteProfile(customerId): return self.insertProfile(strXML) else: return 1 def deleteProfile(self, strId): """ deleteProfile accepts a customer profile ID and deletes the corresponding record in the database. Returns 1 on success, 0 on failure. """ if not strId: return 0 # generate database connection conn = odbc.odbc(CONNECTION_STRING) cmd = conn.cursor( ) ok = cmd.execute("delete from customer where customerId = '" + strId + "'"): conn.close( ) return ok and 1 or 0
One thing of note in CustomerProfile.py is the use of constants defined at the top of the file as field markers in the database record set. By defining constants, it’s far easier to work with the different fields in the record set array.
3.133.122.127