As well as being a good technique for writing programs for your own use, modular programming is also an excellent way of writing programs that can be used by other programmers. In this chapter, we will look at how to design and implement modules and packages that can be shared and reused in other programs. In particular, we will:
- See how modules and packages can be used as a way of sharing the code that you write
- See how writing a module for reuse differs from writing a module for use as part of just one program
- Discover what makes a module suitable for reuse
- Look at examples of successful reusable modules
- Design a package to be reusable
- Implement a reusable package
Let's start by taking a look at how you can use modules and packages to share your code with other people.
Whenever you write some Python source code, the code you create will perform a task of some sort. Maybe your code analyzes some data, stores some information into a file, or prompts the user to choose an item from a list. It doesn't matter what your code is—ultimately, your code does something.
Often, this something is very specific. For example, you might have a function that calculates compound interest, generates a Venn diagram, or displays a warning message to the user. Once you've written this code, you can then use it wherever you want in your own program. This is the simply abstraction pattern that was described in the previous chapter: you separate what you want to do from how you do it.
Once you've written your function, you can then call it whenever you want to perform that task. For example, you can call your display_warning() function whenever you want to display a warning to the user, without worrying about the details of how the warning is displayed.
However, this hypothetical display_warning() function isn't just useful in the program you are currently writing. Other programs may want to perform the same task—both programs that you write in the future and programs that other people may write. Rather than reinvent the wheel each time, it often makes sense to reuse your code.
To reuse your code, you have to share it. Sometimes, you might share your code with yourself so that you can use it within a different program. At other times, you might share your code with other developers so that they can use it within their own programs.
Of course, you don't just share code with others for philanthropic reasons. In a larger organization, you are often required to share code to improve the productivity of your peers. Even if you work by yourself, you will benefit by using code other people have shared and, by sharing your own code, other people can help find bugs and fix problems that you're not able to fix yourself.
Regardless of whether you share your code with yourself (in other projects) or with others (within your organization or in the wider development community), the basic process is the same. There are three main ways in which you can share your code:
- You can create a code snippet that is then copied and pasted into the new program. The code snippet might be stored in an application called a
Code Snippet Manageror a folder of text files, or even published as part of a blog. - You can place the code you want to share into a module or package, and then import this module or package into the new program. The module or package can be physically copied into the new program's source code, it can be placed in your Python installation's
site-packagesdirectory, or you can modifysys.pathto include the directory where the module or package can be found. - Alternatively, you can turn your code into a standalone program, and then call this program from other code using
os.system().
While all these options work, not all of them are ideal. Let's take a closer look at each one:
- Code snippets are great for short pieces of code that form just part of a function. They're terrible, however, at keeping track of where that code ends up. Because you've copied and pasted the code into the middle of a new program, it is very easy to modify it as there's no easy way of distinguishing the pasted code from the rest of the program you've written. Also, if the original snippet needs to be modified, for example, to fix a bug, you'll have to find where you've used the snippet in your program and update it to match. All of this is rather messy and prone to errors.
- The technique of importing modules or packages has the advantage of working well with larger chunks of code. The code you are sharing can include multiple functions and even be split across multiple source files using a Python package. You are also much less likely to accidentally modify an imported module as the source code is stored in a separate file.
If you have copied the source module or package across to your new program, then you will need to manually update it if the original is changed. This is not ideal, but since you're replacing whole files, this isn't too difficult. On the other hand, if your new program uses a module stored elsewhere, then there's nothing to update—any changes made to the original module will immediately apply to any programs which use that module.
- Finally, organizing your code into a standalone program means that your new program has to execute it. This can be done in the following way:
status = os.system("python other_program.py <params>") if status != 0: print("The other_program failed!")As you can see, it is possible to run another Python program, wait for it to finish, and then check the returned status code to ensure that the program ran successfully. You can also pass parameters to the running program if you wish. However, the information you can pass to the program and receive back is extremely limited. This approach will work, for example, if you have a program that parses an XML file and saves a summary of this file into a different file on disk, but you can't directly pass Python data structures to another program for processing, and you can't receive Python data structures back again.
As you can see, snippets, module/package imports, and standalone programs form a kind of continuum: snippets are very small and fine-grained, module and package imports support larger chunks of code while still being easy to use and update, and standalone programs are large but limited in the way you can interact with them.
Of these three, using module and package imports to share code appears to hit the sweet spot: they can be used for large amounts of code, are easy to use and interact with, and are trivially easy to update when necessary. This makes modules and packages the ideal mechanism for sharing your Python source code—both with yourself, for use in future projects, and with other people.