Chapter 12. Common Python 3 Libraries

We've covered the principles of object-oriented programming and we've applied them to Python. We've looked at the ins and outs of object-oriented design, and the higher-level design patterns that make up good programs. We've seen Python's tendency to simplify object-oriented solutions. We even know how to test our Python programs. Yet, are we able to do the common tasks of day-to-day programming?

Yes, we know Python's syntax, and we could in theory write a web framework or database engine from scratch. Python's true power, however, lies in the work other people have done before us. In the examples throughout this book, we've seen many of the Python standard library's modules at work. Yet, we haven't really covered many of the most common tasks facing Python programmers today. We've completely bypassed graphical applications and their widgets, input boxes and buttons: one of the most common interfaces users see today. And we haven't touched on web backend development: Python's current most prevalent use.

These are complex topics, and we'll see an introduction to each of them here. We'll be focusing on Python libraries that are available for Python 3 as I write this. Many popular libraries, such as Django or wxPython, are currently only compatible with older versions of Python, so they'll be passed over.

In this chapter we'll be covering:

  • Database libraries and object-relational managers
  • Point-and-click graphical applications
  • CherryPy for web applications
  • Working with XML

Database access

Talking to databases is a very common task in Python, especially in the web development world. Unfortunately, not many libraries for database access have been ported to Python 3 in a mature state. We'll be looking at a few of the available database solutions.

Python comes with built-in support for SQLite 3. We looked at some examples of it in earlier chapters. SQLite is not suitable for multi-user, multi-threaded access, but it's perfect for storing configuration or local data. It simply stores all the data in a single file and allows us to access that data using SQL syntax. All we need to do to use it is import sqlite3 and read the help file. Here's a short example to get you started:

	import sqlite3
	connection = sqlite3.connect("mydb.db")
	connection.execute(
			"CREATE TABLE IF NOT EXISTS "
			"pet (type, breed, gender, name)")
	connection.execute("INSERT INTO pet VALUES("
			"'dog', 'spaniel', 'female', 'Esme')")
	connection.execute("INSERT INTO pet VALUES("
			"'cat', 'persian', 'male', 'Oscar')")
	results = connection.execute("SELECT breed, name"
			" from pet where type='dog'")
	for result in results:
		print(result[1])
	connection.close()

This code first connects to a local file named mydb.db (it creates the file if it doesn't exist) and runs some SQL query to put a simple table in the database. Then it queries the same relation and prints one of the results.

Results are returned as iterable sequences of tuples. Each tuple represents a matching row in the query results. The order of the values in each result tuple is the same as the order of the values in the query. The name is the second column in the query (the first is type), so we print result[1] to print the name of the queried pet.

The Python API SQLite uses conforms to a database API specification known as DBAPI2. This API is a standard, designed to make it easier for code to interact with different types of databases in the same way. There are similar APIs for other databases such as PostgreSQL, MySQL and Oracle, among many others, but at this time, very few of them are mature on Python 3.

Any database API that follows the DBAPI2 specification will have a connect function (which may take different arguments for different database connections) that returns a Connection object. Queries are executed on the connection using an execute method. Often, additional methods to make querying easier or return named tuples as results are provided; however, none of this is required by the DBAPI2 specification.

However, DBAPI2 is fairly low-level and difficult to work with. In object-oriented programming, it is very common to use an Object-Relational Manager, or ORM to interact with databases. ORMs allow us to use the familiar abstraction called objects that we've been working with throughout this book, while connecting their attributes to the relational database paradigm. One of the most popular ORMs in Python is SQLAlchemy, and it was also the first to be ported to Python 3.

Introducing SQLAlchemy

SQLAlchemy can be downloaded from http://www.sqlalchemy.org/. Only the 0.6 version and higher is supported on Python 3, and at the time of writing, the only underlying databases that are supported are SQLite and PostgresSQL.

This isn't a huge deal, as SQLAlchemy provides an abstraction over database APIs, it is (theoretically) possible to write SQLAlchemy code that works on one database system and later use the exact same (or only slightly modified) code on another one. So if you're looking for MySQL support, you could begin to write your code to use SQLAlchemy with SQLite as a backend first, and port it to MySQL when that backend is eventually supported.

SQLAlchemy is a very large and robust library; it allows us to do almost anything imaginable with a database. We'll only be able to touch on the basics in this section.

The idea behind SQLAlchemy, and ORMs in general is to interact with objects that automatically modify and update database tables in the background. SQLAlchemy provides multiple ways to map objects to tables; we'll be using the modern inheritance-based solution. If you need to connect to a legacy database, SQLAlchemy provides an alternative method that allows arbitrary object classes to be explicitly mapped to database tables, but we won't have room to cover that here.

The first thing we need to do is connect to a database. The sqlalchemy.create_engine function provides a single point of access for connecting to a database. It takes a huge number of arguments to customize or tune access. The most important one is a string URL defining the kind of backend database to be connected to, the specific database backend to make the connection, the name of the database, the host the database system is running on, and a username and/or password to authenticate with. The basic form of the URL resembles a web URL: driver://user:password@host/dbname.

If we want to use a simple SQLite database, which does not require username, password, or host; we can simply specify the filename for the database, as we'll see in the next example.

Then we'll need to create a class that allows objects to store their data in the database, while optionally supplying behavior as methods on the object. Each instance of the object will be stored in a separate row in the database, identified by its primary key (it is usually a good idea to make the primary key a single integer identifier, but SQLAlchemy does not require this).

Each table in the database is normally represented by a separate class, and special attributes on each class map to table columns. When we access these attributes on an object, we get database values, and when we update and save the object, the database is modified. Here's a simple example for our pets database:

	import sqlalchemy as sqa
	from sqlalchemy.ext.declarative import declarative_base
	
	Base = declarative_base()

	class Pet(Base):
		__tablename__ = "pet"
		id = sqa.Column(sqa.Integer, primary_key=True)
		type = sqa.Column(sqa.String(16))
		breed = sqa.Column(sqa.String(32))
		gender = sqa.Column(sqa.Enum("male", "female"))
		name = sqa.Column(sqa.String(64))
		
	engine = sqa.create_engine('sqlite:///mydata.db')
	Base.metadata.create_all(engine)

SQLAlchemy first asks us to set up a Base class by calling a function called declarative_base. This function returns a class, which we are able to extend in our declaration. The subclass needs a special attribute named __tablename__ to specify the name of the table in the database.

This is followed by several column declarations. We add Column objects whose first argument is a type object (example Integer or String), and subsequent arguments depend on the type. All of these type objects are provided in the sqlalchemy package. I generally import this package with the alias sqa to make it easier to reference the many classes in the package. Some people suggest using from sqlalchemy import * syntax, so all the objects are available, but as we discussed in Chapter 2, this can make code very confusing to maintain.

After defining one or more mapped classes that extend the Base object, we connect to a specific database (in this case, an SQLite file) using the create_engine function. The Base.metadata.create_all call ensures that all the tables associated with that Base class exist. It would typically issue some sort of CREATE TABLE call to the underlying database.

Adding and querying objects

We can create instances of our table objects just like a normal object. The default constructor on the Base class accepts no arguments. It can often be useful to add an __init__ method to our subclass that initializes some or all of the variables on the object. We can also add any other arbitrary methods to the class that we like. Here's how we might instantiate a new pet object and set some values:

	pet = Pet()
	pet.id = 1
	pet.type = "dog"
	pet.breed = "spaniel"
	pet.gender = "female"
	pet.name = "Esme"

This object can be used like any other Python object, but the object is not yet connected to the database in any way. Before we can associate the object with a database table row, we need to create an SQLAlchemy Session object. Sessions are like staging areas between objects and the database. We can add multiple objects to the session, as well as use the session to record changes, deletions, and other database operations. When we're ready for this collection of changes to be saved to the database, we can commit() them, or, if something goes wrong, we can call session.rollback() to make all the changes disappear.

Here's how we can add our new pet to the database and save it:


	Session = sqa.orm.sessionmaker(bind=engine)
	session = Session()

	session.add(pet)
	session.commit()

First we have to get a special Session class by calling the sessionmaker function; this function needs to know which engine to connect to. Then whenever we want a session, we instantiate the resulting class. Each session is partially independent of the others until the changes are committed. Underneath, they basically rely on database transactions, so similar rules apply, and the rules may vary depending on the underlying database.

We can also use session objects to query the database. SQLAlchemy queries are written in a combination of Python functions and raw SQL syntax. We use the session.query() method to get a Query object. This method accepts arguments representing the tables or columns to be queried. Then methods on that object can be cascaded to get a set of results. These methods include:

  • all(), which returns all items in the table.
  • first(), which returns the first item.
  • one(), which returns the only item. If no items or multiple items are found, it raises an exception.
  • get(primary_key), which accepts a primary key value and returns the object matching that key.
  • group_by(), order_by(), and having(), which add the related SQL clauses to the query.
  • filter_by(), which uses keyword arguments to query the session.
  • filter(), which uses more advanced SQL expressions (which we will discuss shortly) to query.

The filter_by method allows us to search for items using keyword arguments. For example, we can say:

	session.query(Pet).filter_by(name="Esme").one()

This filter_by argument tries to match a name to a specific string. This returns a new query object, on which we call the one() method to get a single value (since there's only one value in our example database, and it matches our criterion, it will return that result). If we'd called all() instead, it would have returned a list of items containing, in this case, only one item.

SQL Expression Language

Unlike filter_by, which accepts keyword arguments, the filter method accepts values in SQLAlchemy's SQL Expression Language. This is a much more powerful form of querying that applies different operators to column objects. It is an interesting application of overloading the operator special methods.

For example, if we use session.query(Pet).filter(Pet.name=="Esme") the expression inside the filter query does NOT do a typical equality comparison that evaluates to a Boolean value. Instead, it constructs a proper SQL clause that the filter method will use to query the database. This is done by overriding the __eq__ method on the Pet.name column object. So we need to explicitly state the Pet.name object for equality comparison. We can't specify name as if it was a keyword argument; that would cause an error.

SQL Expression Language allows many related operators to be used to construct queries. Some of the more common ones are:

  • != to specify inequality
  • < for less than comparisons
  • > for greater than comparisons
  • <= for less than or equal
  • >= for greater than or equal
  • & to combine clauses using an AND query
  • | to combine clauses using an OR query
  • ~ to negate a query using NOT

The SQLAlchemy Expression Language allows almost any SQL statement to be constructed using Python, including creating joins, and aggregate clauses, and using SQL functions. However, we have a lot of topics to cover, so you'll have to look elsewhere to discover how to use them. Entire books have been written on SQL, SQLAlchemy, and databases in Python, so this brief introduction can do nothing more than spark your interest.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.16.51.157