8

Modules, Packages, and File Operations

Learning Objectives

By the end of this chapter, you will be able to:

• Describe what Python modules are and create your own
• Describe what Python packages are and create your own
• Work with the built-in Python modules
• Describe the file object in Python
• Read and write to Python files
• Work with structured data in Python files

This lesson introduces Python modules and packages. We also cover file operations available to us in Python.

Introduction

In the previous chapter, we have covered object-oriented programming in depth. We have covered important OOP concepts such as classes, methods, and inheritance. In this chapter, we will take a look at modules and file operations.

When you are working on any project, for example, a Word document, you may have a folder with the name of your project, and inside it you have the project files themselves. This helps you know which files are associated with which project. Next time you want to look at the project files, you won't have to search all over your computer for them. You can simply go to the project folder and find the files you need.

The concept of arranging work into files and folders also applies when programming in Python. You can arrange your code into pieces called modules, which makes it easier to group related functionality together. Once you have created a module, it becomes very easy to refer to that collection of functionalities again and also share and reuse the functionality defined in that module.

Specifically, a module in Python is any file with a .py extension. A Python module can contain any valid Python code you want, including classes, functions, or just variable definitions. Once you have created a module, you can then go ahead and import it elsewhere to use whatever resources are defined in the module.

A module can also be a directory containing Python files. Adding an __init__.py file inside a directory will tell the Python interpreter that the indicated directory is a module and that it will be registered in Python's module namespace. Note that it is no longer a requirement to have an __init__.py file, but it is a good practice to have one in case you need to run some custom code during module initialization.

Following the same trend, a package is therefore defined as a collection of modules. Packages are a good way of separating your modules from other people's modules to avoid name clashes.

Arranging your code into modules and packages makes it extremely easy for other developers to work with your code. They can easily see what resources are defined in each package or module and import them as needed.

Real-life applications will need to read input from files or write output to files at one point or another. Every time you open a document, be it a PDF file, a JPEG image, or even a .py file in your application, some code is running behind the scenes to process that file and output the data to you.

Imagine a payroll system. The system may be implemented in a way where user data is input into an Excel sheet containing the name of the person and the hours worked in a specific time period. The program will then read this data and calculate the amount of wages each person is owed and output a new file with the name of the person and their wages.

Does this sound interesting? This is just an example of what you can do once you can read input from a file. Files are useful for aggregating huge amounts of data that might be too cumbersome to enter through the keyboard line by line.

Imagine if the aforementioned payroll system needed input to be entered on the keyboard by hand. This would make payroll processing, for whatever company that used it, very cumbersome and prone to mistakes.

On the other hand, a file could contain thousands of lines, and the execution of the payroll would be faster because the input is not blocked by data entry. In fact, we shall build a rudimentary version of such a payroll system at the end of this chapter once you have a deeper understanding of how to work with files in Python.

Defining Modules

Following the definition of a module that was given earlier, you can now see that you have in fact been working with modules all along. Any valid .py file that you have created in this book is more or less a valid module.

In this section, though, we are going to be a bit more deliberate in how we create modules so that you can see how to define and import resources.

To recap, a valid Python module is any .py file containing valid Python code. This code could be variable definitions, functions, classes, methods, and so on. We are going to practice with a simple module that contains just one function.

Let's go ahead and create our first module.

Exercise 46: Creating Modules

In this exercise, we will create a module named calculator:

1. Create a file named calculator.py. This is where our module resources are going to live.
2. Inside the file, add the following simple function code in order to add and return the sum of two numbers:

   def add(x, y):

   return x + y

   Remember, the def keyword is used to define functions in Python. In our example, we are defining a function called add that takes two parameters, x and y, which will be both integers. The function body simply returns the sum of the two numbers by using the + operator.

3. Next, run the Python interpreter using the Python command on your terminal in the same folder you created the calculator module in.

   You can now see and use the function you defined by importing the calculator module:

   >>> import calculator

   >>> calculator.add(8, 9)

   17

Module names should follow normal Python variable naming conventions.
These include the following:

• They should follow snake_case (lower_case_with_underscore). Some good module names would be module and another_module. The following are examples of bad module names: Amodule and AnotherModule.
• Names should be descriptive. This means that the module name should reflect the purpose of the resources defined inside it; for example, a module containing mathematical functions can be named math, but not string.
• Only use underscores if it improves readability.

  Note

  These rules and other such rules are defined in Python Enhancement Proposal 8 (PEP8), which is the go-to guide for formatting your Python code. You can take a look at it at https://www.python.org/dev/peps/pep-0008/.

When importing a module, the code in the module is executed exactly once, even if you have another statement in the code importing the exact same module elsewhere. Upon import, Python compiles the module's code, executes the code, and then populates the module's namespace with the resource names defined in the module. This makes the resources that have been imported accessible in the scope in which they have been imported.

Imports and Import Statements

There are several ways we can import and use the resources defined in a module. Taking the calculator module we defined previously, you have already seen one way to go about it, which is importing the whole module by using the import keyword and then calling a resource inside it by using the dot (.) notation, like so:

>>> import calculator

>>> calculator.add(8, 9)

17

Another way of accessing a module's resources is to use the from…import… syntax:

>>> from calculator import *

>>> add(8, 9)

17

Note that the * in the preceding example tells Python to import everything in the module named calculator. We can then use any resource defined in the calculator module by referring to the resource name (in our case, the add function) directly. You should note that using * is not a good practice. You should always strive to import exactly what you need.

What if you want to only import a few resources from the module? In that case, you can name them directly, as shown here:

>>> from calculator import add

>>> add(8, 9)

17

Another useful thing you can do is name your imports by using the as keyword:

>>> from calculator import add as a

>>> a(8, 9)

17

>>> import calculator as calc

>>> calc.add(8,9)

17

This can come in handy when you have a module or resource with a long name that you will be referring to in many places.

You can also group many imports by using brackets for easier readability:

>>> from random import (choice, choices)

Exercise 47: Importing Modules

In this exercise, we will practice how to import modules and familiarize ourselves with what successful imports look like and how non-successful imports will fail:

1. Open the Python interpreter on your machine.
2. Importing an existing library, such as string:

   >>> import string

   A successful import should not produce any output.

3. Next, import an undefined library, such as packt:

   >>> import packt

   Traceback (most recent call last):

   File "<input>", line 1, in <module>

   import packt

   ModuleNotFoundError: No module named 'packt'

   >>>

   You should get a ModuleNotFoundError. We will cover errors in the next chapter.

Modules and Packages

In this section, we will turn our focus toward the standard Python modules and Python packages.

The Module Search Path

When you import any module, Python will first check whether there is a built-in module with the specified name. An example of a built-in module is the string module.

If no built-in module is found, the interpreter will look for a file with the name of the module and the .py extension in the directories given by the sys.path variable. This variable is simply a list of strings which specifies where to search for modules.

How this variable is built is beyond the scope of this book. However, it is partly dependent on your defined PYTHONPATH and can be modified if necessary. You can read more about it in the Python documentation at https://docs.python.org/3/library/sys.html#sys.path.

For your interest, though, you can inspect the sys.path in your current environment by running the following commands on your terminal:

>>> import sys

>>> sys.path

['', '/usr/local/bin', '/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/

Versions/3.6/lib/python3.6', '/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages']

>>>

Standard Python Modules

Python ships with a lot of ready-made modules and packages. These modules and packages are grouped into libraries and provided for use in the standard library.

An example is the math library, which provides utility functions for math operations.

Here are a couple of common modules that you are likely to interact with and what they do:

Figure 8.1: Common Python modules

You can import each module and use the dir() function to see a list of the methods implemented in each:

>>> import sys

>>> dir(sys)

['__displayhook__', '__doc__', '__excepthook__', '__interactivehook__', '__loader__', '__name__', '__package__', '__spec__', '__stderr__', '__stdin__', '__stdout__', '_clear_type_cache', '_current_frames', '_debugmallocstats', '_getframe', '_git', '_home', '_xoptions', 'abiflags', 'api_version', 'argv', 'base_exec_prefix', 'base_prefix', 'builtin_module_names', 'byteorder', 'call_tracing', 'callstats', 'copyright', 'displayhook', 'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix', 'executable', 'exit', 'flags', 'float_info','float_repr_style', 'get_asyncgen_hooks', 'get_coroutine_wrapper', 'getallocatedblocks', 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags', 'getfilesystemencodeerrors', 'getfilesystemencoding', 'getprofile', 'getrecursionlimit', 'getrefcount', 'getsizeof', 'getswitchinterval', 'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info', 'intern', 'is_finalizing', 'last_traceback', 'last_type', 'last_value', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', 'path_hooks', 'path_importer_cache', 'platform', 'prefix',

'set_asyncgen_hooks', 'set_coroutine_wrapper', 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit', 'setswitchinterval', 'settrace', 'stderr', 'stdin', 'stdout', 'thread_info', 'version', 'version_info', 'warnoptions']

>>>

You can also use the help() function to find more information about what a specific module does. You can use it in the same way as dir(). The help function shows the content of documentation strings (docstrings) defined on the resource.

For example, we can add a docstring to our previous calculator function so that future users can find out what the function does:

def add(x, y):

"""Return the sum of x and y."""

return x + y

You can then use help() on it:

>>> help(add)

This will output the following:

Help on function add in module calculator:

add(x, y)

Return the sum of x and y.

/var/folders/lb/k8ws21qn1x5gxq9zbrlqxp9w0000gn/T/tmphhbdd2y_ (END)

This is a best practice, and you should always strive to ensure that all your resources have docstring comments.

For a more detailed look at the built-in Python modules, you can check out the Python Module Index at https://docs.python.org/3/py-modindex.html.

Activity 34: Inspecting Modules

In this activity, we will try and use built-in Python functions to inspect modules.
The steps are as follows:

1. Open the Python interpreter on your machine.
2. Import the itertools library.
3. Use the relevant built-in function to inspect the resources defined in it.
4. Use the relevant built-in function to read about how to use the itertools library.

   Note

   Solution for this activity can be found at page 293.

Packages

Packages are collections of modules. Each package must contain an __init__.py in the root of its directory, which indicates to the Python interpreter that the directory is a module.

The file doesn't need to contain anything, but you can use it for some added functionality (for example, to specify exactly what can and cannot be imported from the package).

When importing a package, the code in the __init__.py file is executed by the Python interpreter to build the package and the package namespace.

A sample package could have a file structure such as this:

mytools

|

├── init.py

├── calculator.py

├── tokenizer.py

In our example, the mytools package has two modules, the calculator and tokenizer modules.

Sample imports would look like this:

# import both modules

from mytools import calculator, tokenizer

# import one of the modules

from mytools import calculator

# import specific resources from a module

from mytools.calculator import add, another_function

Absolute Imports

An absolute import is an import that uses the full path from a project to the module being imported. An example of this is importing a built-in library:

import string

Relative Imports

A relative import is one that uses the relative path from the current directory to the directory where a module is. Imports of the following nature are relative:

from . import calculator

The preceding statement means import calculator from the current directory, which is represented by the dot (.).

Note

Always prefer absolute imports where possible.

Activity 35: Listing the Resources Defined in a Package or Module

You are implementing a program at work and you need to use a package that you have never interacted with before. You find that you need to write a script to list all resources that are available in that package.

Our aim here is to list the resources defined in a package or module.

Define a function called package_enumerator, which will take a package or module name and print out the names of all the resources defined in the package or module:

1. Define a function called package_enumerator.
2. Inside the function, use a for statement to list the package's resources.

A sample output is as follows:

>>> package_enumerator(string)

Formatter

Template

_ChainMap

_TemplateMetaclass

__all__

__builtins__

__cached__

__doc__

__file__

__loader__

__name__

__package__

__spec__

_re

_string

ascii_letters

ascii_lowercase

ascii_uppercase

capwords

digits

hexdigits

octdigits

printable

punctuation

whitespace

Note

Solution for this activity can be found at page 294.

Activity 36: Using Resources in a Module

To attempt this activity, you should have completed the previous chapters of this book. Our aim here is to practice using the resources defined in a module to implement a solution to a problem.

Write a function called random_number_generator whose purpose is to generate random integers between 0 and 1,000.

The function should use the resources defined in the built-in random module. It should take one argument, l, which is an integer, and return a list containing l random numbers.

For example, calling the function with 3 returns a list of three random numbers:

random_number_generator(3) -> [54,13,2]

The steps are as follows:

1. Import the random module.
2. Define a function named random_number_generator.
3. Inside the function, use a looping statement and the resources from our imported module to generate random numbers.

Here is the output when l is 5:

>>> [3, 618, 298, 168, 701]

Note

Solution for this activity can be found at page 295.

File Operations

Files could be of many types, for example, text files with .txt extensions, data files in a .csv extension, or even executable files with .exe extensions. Not all file types are immediately readable. Some file types are proprietary and require specialized software to read them. An example of a proprietary file type is the PSD file that's generated by Adobe Photoshop.

We will be working with non-proprietary extensions which are easy to work with, such as .txt, .csv, and .json files for the purposes of this chapter. The skills that you will learn about will be transferrable to any other file you need to read with Python in the future.

The file Object

The file object is the default and easiest way to manipulate files in Python. It includes a couple of methods and attributes which make it easier for developers to read from, and write to, files in the filesystem.

There are two major file object types that are recognized in Python:

• Binary file objects: These can read and write byte-like objects.
• Text file objects: These can read and write strings objects.

The open() function, which we will be looking at later, is the easiest way to create a file object. Depending on the mode passed to the open() function, you will get back either a binary or text file object. We will be specifically working with text file objects.

The file Object Methods

The file object has several methods to make it easy to work with the underlying file. They include the following:

• file.read(): This method loads the entire file into memory.
• file.readline(): This method reads a single line from the file into memory.
• file.readlines(): This method reads all of the lines of a file into a list.
• file.write(): This method writes output to the file.
• file.seek(): This method is used to move the file object position to a certain location in the file.

Reading and Writing to Files

In the previous section, we mentioned that to create a file object, you can use the built-in open() function with a filename and mode parameter. But how exactly does that work? Let's talk a bit more about the open() method now.

The open() Method

The open() method is a built-in function in Python that you can use to read a file. It takes two arguments, the filename and a mode, for example, open(name_of_file, mode), and returns a file object that you can manipulate.

The mode passed to the function determines what kind of file object you will get back and what you will be able to do with it. Some available modes are as follows:

Figure 8.2: Available modes with open()

The preceding table lists the most important modes you will encounter. Some additional specialized modes are as follows:

Figure 8.3: Specialized modes with open()

Note

We will not be working with the binary modes for now, but feel free to play around with them in your own time. A good place to start is http://www.diveintopython3.net/files.html#binary.

Exercise 48: Creating, Reading, and Writing to Files

In this exercise, we will learn how to create, read, and write to files.

1. First, we will focus on creating a file. To create a file, open your terminal and start the Python interpreter.
2. Then, use the open() method to create a file:

   f = open('myfile.txt', 'w')

   You should not see any output after running the preceding command. What it does is simple; we open a file called myfile.txt in the write mode. Since that file does not exist, a blank file with that name will be created on your filesystem. The file object that's created is assigned to the variable f, which you can then use to manipulate the file.

3. Next, write something to the file, like so:

   >>> f.write("Hello, world ")

   13

   We used the write() method on the file object to write the string "Hello, world" and a new line character to the file. You can see that 13 is the output. The write() method returns the number of characters written. The contents of myfile.txt is as follows:

   Hello, world

4. Write something else again:

   >>> f.write("Hello, world again")

   18

5. Now close the file. Remember to always do this after you have finished working with a file:

   f.close()

6. Check the contents of myfile.txt; you should see the two strings we wrote:

   Hello, world

   Hello, world again

7. To add more content to the file, open it in append mode:

   >>> f = open('myfile.txt', 'a')

   >>> f.write("More content")

   12

   >>> f.close()

   myfile.txt should now have the following content:

   Hello, world

   Hello, world againMore content

   The last two lines are joined because we did not write any newline character between them to the file.

8. Now, let's read a file. Use the r mode to read the file:

   >>> f = open('myfile.txt', 'r')

   >>> f.read()

   'Hello, world Hello, world againMore content'

   You cannot write in this mode:

   >>> f.write("Some content")

   Traceback (most recent call last):

   File "<input>", line 1, in <module>

   f.write("Some content")

   io.UnsupportedOperation: not writable

   >>> f.close()

   Similarly, you cannot read in write (w) or append (a) modes:

   >>> f = open('myfile.txt', 'a')

   >>> f.read()

   Traceback (most recent call last):

   File "<input>", line 1, in <module>

   f.read()

   io.UnsupportedOperation: not readable

   >>> f.close()

   >>> f = open('myfile.txt', 'w')

   >>> f.read()

   Traceback (most recent call last):

   File "<input>", line 1, in <module>

   f.read()

   io.UnsupportedOperation: not readable

   >>> f.close()

9. If you need to combine both reading and writing, use the r+ mode:

   >>> f = open('myfile.txt', 'r+')

   >>> f.read()

   ''

   >>> f.write("Some content")

   13

   Note that the r+ mode appends to the existing file when writing, so it is safe to use on existing files.

The with Context Manager

Now that you understand the different file reading and writing modes, let's talk about automating some of the processes around working with files—more specifically, auto-closing files after you have finished using them. Remember, we need to close files after using them so that they can be removed from memory and so the memory is then freed up for other processes.

Python comes with a handy context manager for working with files, which helps in writing neater code and freeing resources automatically when the code exits so that you do not have to. The syntax is as follows:

with open("myfile.txt", "r+") as f:

content = f.read()

print(content)

This code will read the specified file for us, print the content, and exit while automatically closing the file.

You can also process each line individually (this can be done with or without the context manager, so long as you have a file object):

with open("myfile.txt") as f:

for line in f:

print(line)

Activity 37: Performing File Operations

In this activity, we will create a file and add data to it.

The steps are as follows:

1. Use the with context manager or the open() function to create a file object.
2. Write I love Python to the file.

   Remember to close the file object when you're only using open().

   Note

   Solution for this activity can be found at page 295.

Handling Structured Data

Now that you have a good handle on reading and writing to files, let's talk about how to deal with more structured data. In real-world applications, you will most likely have to read data in a structured format.

Do you remember the payroll application we talked about in the beginning of this chapter? Such data could be represented in a CSV file. A CSV file is a file with comma-separated values, usually arranged in columns.

Such data can then be easily read into a spreadsheet application, such as Excel, and manipulated there. Python provides a utility to work with CSV files, which we will cover in this topic.

Here is an example of a CSV file:

Name,City

"Nash, Colorado B.",Milton Keynes

"Herrera, Chase E.",Gentbrugge

"Hubbard, Leilani I.",Bremen

"Vinson, Marsden H.",Lakeland County

"Macias, Lawrence E.",Noisy-le-Grand

…

…

"Phelps, Amity V.",Morena

"Woods, Jaden V.",Portland

"Hyde, Duncan P.",Schellebelle

"Hendricks, Yoshio V.",Sperlinga

"Delgado, Emma T.",Reyhanlı

"Oneil, Orson B.",Rotello

"Sims, Noah C.",Selkirk

CSV files are popular because they have some advantages over files such as spreadsheets; for example:

• They are human readable and easy to parse.
• They are smaller, compact, and faster to work with.
• CSV files are easy to generate and have a standard format.

Another form of structured data is JSON. JSON is very useful, especially on the internet, for easily transferring data in the form of key-value pairs. As shown in the following sample, JSON data looks like this:

{

"data": [

{

"name": "Rodriguez, Evangeline U.",

"city": "Lobbes"

},

{

"name": "Herman, Leandra P.",

"city": "Tramonti di Sopra"

}

]

}

Other formats used to transfer data over the internet include:

• XML (you can learn more about XML at https://www.w3schools.com/xml/)
• Protocol buffers (you can learn more about protocol buffers at https://developers.google.com/protocol-buffers/).

Working with CSV Data

Python includes the very handy csv module to help us work with CSV files. It makes it very easy to read the tabular data defined in a CSV file and operate on it by creating reader or writer objects.

For our examples, we are going to be using a dummy CSV file that we generated from Mockaroo (https://mockaroo.com/). Our download file is called MOCK_DATA.csv and is available in the accompanying code bundle.

If you want different mock data, you can go to the site and generate a CSV file, as illustrated in the following screenshot. This can easily be done on the home page without creating an account:

Figure 8.4: Mockaroo home page

Make sure to change the format at the bottom to CSV and choose Windows (CRLF) as the option for Line Ending. Everything else can remain as is. You can then click on Download Data.

Reading a CSV File

Reading a CSV file requires us to import the csv module and create a reader object:

import csv

with open('MOCK_DATA.csv', 'r') as f:

mock_data_reader = csv.reader(f)

line_count = 1

for row in mock_data_reader:

if line_count > 1: # skipping line 1 which is the header row

print(row)

line_count += 1:

As you can see, we use the standard file reading open() function within a context manager with the read mode. We then create a reader object to help us read the CSV. The reader object returns each row as a list of strings.

Running the preceding code, you should see an output like the following:

['10', 'Sherwin', 'Lydall', '[email protected]', 'Male', '9.153.155.182']

['11', 'Liz', 'Linthead', '[email protected]', 'Female', '232.213.9.139']

['12', 'Connie', 'Moreby', '[email protected]', 'Male', '145.75.39.34']

['13', 'Christie', 'Gerasch', '[email protected]', 'Male', '195.178.145.92']

['14', 'Ellen', 'Bocke', '[email protected]', 'Female', '94.217.132.103']

['15', 'Cecile', 'Maginn', '[email protected]', 'Female', '253.72.149.37']

['16', 'Beverly', 'Hendrich', '[email protected]', 'Female', '224.43.75.87']

Since the first row contains field names, we can extract those from the output.

In that way, you can isolate the column names from your data and process each separately in code:

['id', 'first_name', 'last_name', 'email', 'gender', 'ip_address', '']

['1', 'Skipton', 'Mattiacci', '[email protected]', 'Male', '196.4.235.14', '']

['2', 'Keith', 'Rosenfelder', '[email protected]', 'Male', '26.145.67.148', '']

['3', 'Helaina', 'Ind', '[email protected]', 'Female', '32.201.244.49', '']

['4', 'Joelly', 'Milesap', '[email protected]', 'Female', '74.199.54.89', '']

['5', 'Portie', 'MacCoughan', '[email protected]', 'Male', '61.150.82.117', '']

['6', 'Fletcher', 'Reynold', '[email protected]', 'Male', '254.202.181.187', '']

['7', 'Arnaldo', 'Batch', '[email protected]', 'Male', '137.231.46.255', '']

['8', 'Galven', 'Turban', '[email protected]', 'Male', '180.52.139.226', '']

['9', 'Cele', 'La Vigne', '[email protected]', 'Female', '125.202.213.224', '']

Once you have your data in the reader object, you can proceed to manipulate it as you want, just like any other list of strings.

Writing to the CSV File

Writing to a CSV file is done by using the writer object. The writer object accepts a list of items to write for each row:

import csv

with open('example.csv', 'w') as f:

example_data_writer = csv.writer(f)

example_data_writer.writerow(['name', 'age'])

example_data_writer.writerow(['Steven', 25])

Running this code will create a file called example.csv with the following contents:

name,age

Steven,25

Writing a dict to the CSV File

Instead of writing rows using lists, you can write dictionaries to CSV files. To do that, you would have to use a DictWriter object instead of the usual writer object, even though their behaviors are pretty similar. However, you will also have to define a list of fieldnames which will specify the order in which values will be written to the file. Here is an example:

import csv

with open('people.csv', 'w') as f:

fields = ['name', 'age']

people_writer = csv.DictWriter(f, fieldnames=fields)

people_writer.writeheader() # writes the fields as the first row

people_writer.writerow({'name': 'Santa Claus', 'age': 1000})

This will create a file called people.csv with the following contents:

name,age

Santa Claus,1000

Activity 38: Working with Files

Suppose you are an HR manager and you need to create a file containing wages for your employees. We will read in a CSV file with the employee names and the hours worked, and then output another CSV file with their wages calculated.

The steps are as follows:

1. Write a script to read in a CSV file of the following format:
   

   Figure 8.5: Input file format

2. Output a new CSV file of the following format:

Figure 8.6: Output file format

Here, wages are calculated using the formula hours_worked * 15.

Note

Solution for this activity can be found at page 295.

Working with JSON Data

JSON stands for JavaScript Object Notation. It's a data format that was built for exchanging data in a human-readable way. Most APIs you will interact with on the internet today will be using JSON as the data-exchange format. It is therefore important that we talk about it a little before we move on.

Python includes a json module with a few functions to assist in parsing JSON and converting some Python objects, for example, dictionaries into JSON objects.

There are two critical methods from the json module that you need to know about to use JSON in Python.

json.dumps()

The json.dumps() method is used for JSON encoding, for example, converting dictionaries into JSON objects.

Here is an example:

import json

sample = {

"name": "Bert Bertie",

"age": 24

}

sample_json = json.dumps(sample)

print(sample_json)

print(type(sample_json))

This code defines a dictionary called sample and then encodes it to JSON using json.dumps(). The dumps function will return a JSON string of the object. Here is the output:

'{"name": "Bert Bertie", "age": 24}'

<class 'str'>

You can see from the preceding output that the sample dict was encoded into a string JSON object.

json.loads()

If you want to decode the JSON object, you can use json.loads() to do so. We are going to add some code in our original code to illustrate this:

original_sample = json.loads(sample_json)

print(original_sample)

print(type(original_sample))

This will output the following additional lines:

{'name': 'Bert Bertie', 'age': 24}

<class 'dict'>

As you can see, the json.loads() function directly reverses what the json.dumps() function does – or rather, it creates Python objects out of JSON objects.

Note

We will not go into too much detail about working with JSON. However, feel free to read more on encoding and decoding JSON objects in the official Python documentation at https://docs.python.org/3.6/library/json.html.

Summary

Congratulations! You now have a good understanding of modules and packages and how they function in Python. You can now go ahead and use the structuring methods you have learned to build even larger applications.

Large frameworks written in Python, such as Django and Flask, use the same principles of modules and packages that you have learned here. For reference, check out the Flask project source code on GitHub to see how files are arranged in a real-life project. You will notice that the same concepts that you just learned about are in use.

As we said in the introduction, working with data in files is a fact of any developer's life. Python, as a general-purpose scripting language, comes with a lot of built-in tools to help you read, manipulate, and write to files very easily in your program. You should now have a good grasp of how to use these tools to not only manipulate text files but also to work with more complex data such as CSV files. In our final chapter activity, you had the chance to put those skills to good use by calculating wages for the employees of a fictional company.

In our next and final chapter, we will cover error handling in detail. We will also look at the built-in exception classes and create our own custom exceptions.