Chapter 5. Hashes and Databases: Putting data in its place
Arrays aren’t the only show in town when it comes to data.
Programming languages come with other data-arranging goodies too, and our chosen tool, Python, is no exception. In this chapter, you’ll associate values with names using a data structure commonly called the hash (better known as dictionary to Python-folk). And when it comes to working with stored data, you’ll read data from an external database system as well as from regular text-based files. All the world’s awash with data, so turn the page and start applying your ever-expanding programming skills to some cool data-processing tasks.
Who won the surfing contest?
In the previous chapter, you worked out the top three scores, but they’re not much use without the names of the surfers that achieved those scores. There will no be surfing for you until you’ve finished the program.
Here’s the code so far:
You were to rewrite your current program to use two arrays: one to keep track of the scores, the other to keep track of the surfer names.
Load this!
Don’t forget to download results.txt from the Head First Programming website before continuing.
With the results.txt file saved to the same directory as your program, enter this code into IDLE and see what happens. Remember to save your program, then press F5 to run it.
Those results look a little strange. Zack is only a novice surfer but, according to the results from your program, Zack has the highest score. It looks like the association between the surfer names and their scores is somehow lost... and if you think about it, this is exactly what’s happening.
The two arrays are independent of each other: one contains scores, the other names. When the data is in the file, the surfer name and the scores are associated with each other because they appear on the same line. However, once the split occurs and the data is in the arrays, the association is severed. Sorting one array has no effect on the ordering of the other. No wonder your results are a little off the wall.
How do you fix this?
Associate the name with the score
Using two arrays just won’t cut it. You need some other data structure to hold your data in such a way that the association between the surfers’ name and their score is maintained.
You need a different data structure. But which one?
The Scholar’s Corner
Data Structure A standard method of organizing a collection of data items in your computer’s memory. You’ve already met one of the classic data structures: the array.
Match the data structure names on the left with their descriptions on the right. We’ve already done one for you. Which one do you think you’ll need to use for the surfer data?
You were asked to match the data structure names on the left with their descriptions on the right. You were also to identify which one you thought you might need to use for the surfer data.
Use a hash
You need to use a data structure that maintains the association between the surfer score and the surfer name, which is exactly what a hash gives you. There are lots of surfers with lots of scores, and you need to maintain the association between the two pieces of information.
Let’s take a look at how hashes work.
Associate a key with a value using a hash
You add data to an existing hash by describing the association between the key and the value. Here’s how to associate a surfers’ name with their score:
Iterate hash data with for
Let’s add some additional rows of data to your hash:
Once you have a hash created, you can use the trusty for loop to iterate over each of the rows:
Another hash method, called items(), returns each key-value pair in turn, and can be used with the for loop, too:
Whichever method you use to iterate over your hash’s data is up to you, because using items() or keys() produces the same output.
Rearrange the code magnets at the bottom of the page to complete the program shown below. Rather than using two arrays, this program stores the results from the surfing contest in a hash:
_________________________________________________________________________
result_f = open("results.txt")
for line in result_f:
(name, score) = line.split()
_________________________________________________________________________
result_f.close()
print("The top scores were:")
_________________________________________________________________________
_________________________________________________________________________
The data isn’t sorted
Your program now associates surfers and their scores, but it displays the data from the hash in some sort of random ordering. You need to somehow sort the data in the hash to find out who actually won the contest.
Python hashes don’t have a sort() method...
In Python, hashes are optimized for speedy insertions and even speedier look-ups (searches). As a consequence, the good folks that created Python were less interested in providing a method to sort a hash, so they didn’t bother.
...but there is a function called sorted()
Obviously, there’s a need to sort a hash so, again, the good folks that created Python decided to provide a really smart built-in function that has the ability to sort any data structure, and they called their function sorted(). Here’s how to use the sorted() function with your hash:
That’s one small change to one line at the bottom of your program. So, let’s go ahead and make that change. Now that you are sorting the keys of the hash (which represent the surfer’s scores), it should be clear why the scores were used as the key when adding data into the hash: you need to sort the scores, not the surfer names, so the scores need to be on the left side of the hash (because that’s what the built-in sorted() function works with).
When data gets complex
Hot on the heels of your success with the local surfing club, you’ve just been contacted by the Regional Surfing Association (RSA) and they want you to write a new program for them! RSA’s offering a brand-new, state-of-the-art, epoxy resin surf board as payment... once the program’s working to their satisfaction, of course.
You’ve been wanting to try out an epoxy board for ages. The trouble is, they’re sooooo expensive and you’re a poor surfer. The waves will have to wait (yet again). But, the thoughts of surfing an epoxy board... now that’s worth waiting for.
So what does RSA’s data look like?
Here’s one surfer’s data from the file, assigned to a variable called line:
line = "101;Johnny 'wave-boy' Jones;USA;8.32;Fish;21"
You were to grab your pencil and write some code to process this line and display it on screen like this:
ID: 101 Name: Johnny 'wave-boy' Jones Country: USA Average: 8.32 Board type: Fish Age: 21
Return a data structure from a function
Processing one line of surfer data was pretty straightforward. But now you have to work with all the lines of data in the file. Your program has to make the data available quickly so that a request to display the details of a particular surfer can be performed as soon as possible.
You already know enough to write a function that takes the surfer ID as a parameter, searches the file one line at a time for a matching ID, and then returns the found data to the caller:
There are really only two choices for how you return data from this function. Pass back the surfer’s data either:
As a string
or
As a hash
But which? Returning a string requires the calling code to further process the data to extract the information it needs, which (although possible) gets messy, because the calling code is then required to cut up the string using split(). This is something best left to the function, because it hides the complexity of manipulating the data from the calling code. Returning a hash allows the calling code to simply pick out the information it needs without too much fuss and without any further processing.
Return a hash from the function to keep the calling code simple.
Rearrange the code magnets to complete the function required. The following functionality is implemented:
Accept a single parameter (the surfer ID).
Process the data file one line at a time, creating a hash from the line on each iteration.
Compare the parameter against the ID read from the file.
If the IDs match, return the hash to the caller.
If no match is found, return an empty hash to the caller.
def find_details(id2find):
____________________________________________________________
for each_line in surfers_f:
____________________________________________________________
____________________________________________________________
____________________________________________________________
surfers_f.close()
____________________________________________________________
surfers_f.close()
____________________________________________________________
You were to rearrange the code magnets to complete the function required. The following functionality is implemented:
Accept a single parameter (the surfer ID).
Process the data file one line at a time, creating a hash from the line on each iteration.
Compare the parameter against the ID read from the file.
If the IDs match, return the hash to the caller.
If no match is found, return an empty hash to the caller.
Here’s some tester code that we’ve cooked up especially for you. This code will let you test your function:
Here’s your new board!
The RSA folks are delighted with your work.
Your program really hits the mark. The RSA folks can display the data from their tightly packed data file in a way that makes it easy for them to read and work with. It’s fast, too.
Your use of a hash within the function was an inspired choice. The calling code only needs to be aware that a hash is being returned from the function to work with it effectively. And, as you’ve seen, returning a data structure (like a hash) is as easy as returning any other variable from a function.
Word of your programming skills is spreading far and wide.
Meanwhile, down at the studio...
Head First TVN is an up-and-coming sports network specializing in everything and anything to do with water. They are covering the National Surfing Championship and want their TV presenters to be able to use your program to access each surfer’s details in much the same way that RSA did. There’s just one small kink in their plans: TVN has all their data in a database system, not in a file.
The code remains the same; it’s the function that changes
Your program expects the find_details() function to return a hash representing the surfer’s details. Rather than the function searching the file for the data, it needs to search the TVN database, convert what the database provides to a hash, and then return the hash to the calling code.
All you need to know is which database system TVN is using and how to access it from your function.
Let’s base your code on TVN’s code.
You were to rewrite your function to retrieve the data it needs from the TVN database. You were to use the ready bake code as provided by TVN’s technical people. You were to bear in mind that your program expects the data in the hash returned from the find_details() function to be a collection of strings.
Change your program in IDLE to use the new version of the function (which now talks to TVN’s database). Save your program under a new name and press F5 to run it.
Load this!
Download surfersDB.sdb from the Head First Programming website before running this test drive.
Now, as well as working with data in files, you can extract data from a database system. Your program fully integrates with TVN’s SQLite3 technology.
Send off that code to TVN, and sit back and wait for the money to roll in...
TVN’s data is on the money!
With the surfing data now displayed directly from the TVN database, the sports writers no longer need to worry about all those tattered and torn competition entry forms. The data is right there on screen when they need it. They couldn’t be happier. Your program has made their day.
Your Programming Toolbox
You’ve got Chapter 5 under your belt. Let’s look back at what you’ve learned in this chapter:
Programming Tools
* hash - a data structure that associates a name with a value
* s[‘age’] - retrieve the value associated with the ‘age’ name in a hash called ‘s’
* returning a data structure from a function
* database system - a technology, like SQLite3, that can store large quantities of data in a very efficient way
Python Tools
* {} - an empty hash
* s[‘wind’] = “off shore” - sets that value associated with “wind” in the “s” hash to the value “off shore”
* s.keys() - provide a list of keys for the hash called ‘s’
* s.items() - provide a list of keys AND values for the hash called ‘s’
* line.split(“,”) - split the string contained within the ‘line’ variable at every occurrence of a comma
* sorted() - a built-in function that can sort most data structures