Collections of Tcl commands are kept in libraries and organized into packages. Tcl automatically loads libraries as an application uses their commands. Tcl commands discussed are: package, pkg_mkIndex, auto_mkindex, unknown, and tcl_findLibrary.
Libraries group useful sets of Tcl procedures so that they can be used by multiple applications. For example, you could use any of the code examples that come with this book by creating a script library and then directing your application to check in that library for missing procedures. One way to structure a large application is to have a short main script and a library of support scripts. The advantage of this approach is that not all the Tcl code needs to be loaded to start the application. Applications start up quickly, and as new features are accessed, the code that implements them is loaded automatically.
The Tcl package facility supports version numbers and has a provide/require model of use. Typically, each file in a library provides one package with a particular version number. Packages also work with shared object libraries that implement Tcl commands in compiled code, which are described in Chapter 47. A package can be provided by a combination of script files and object files. Applications specify which packages they require and the libraries are loaded automatically. The package facility is an alternative to the auto loading scheme used in earlier versions of Tcl. You can use either mechanism, and this chapter describes them both.
If you create a package you may wish to use the namespace facility to avoid conflicts between procedures and global variables used in different packages. Namespaces are the topic of Chapter 14. Before Tcl 8.0 you had to use your own conventions to avoid conflicts. This chapter explains a simple coding convention for large Tcl programs. I use this convention in exmh, a mail user interface that has grown from about 2,000 to over 35,000 lines of Tcl code. A majority of the code has been contributed by the exmh user community. Such growth might not have been possible without coding conventions.
The package facility assumes that Tcl libraries are kept in well-known directories. The list of well-known directories is kept in the auto_path Tcl variable. This is initialized by tclsh and wish to include the Tcl script library directory, the Tk script library directory (for wish), and the parent directory of the Tcl script library directory. For example, on my Macintosh auto_path is a list of these three directories:
Disk:System Folder:Extensions:Tool Command Language:tcl8.4 Disk:System Folder:Extensions:Tool Command Language Disk:System Folder:Extensions:Tool Command Language:tk8.4
On my Windows 95 machine the auto_path lists these directories:
c:Program FilesTcllibTcl8.4 c:Program FilesTcllib c:Program FilesTcllibTk8.4
On my UNIX workstation the auto_path lists these directories:
/usr/local/tcl/lib/tcl8.4 /usr/local/tcl/lib /usr/local/tcl/lib/tk8.4
The package facility searches these directories and their subdirectories for packages. The easiest way to manage your own packages is to create a directory at the same level as the Tcl library:
/usr/local/tcl/lib/welchbook
Packages in this location, for example, will be found automatically because the auto_path list includes /usr/local/tcl/lib. You can also add directories to the auto_path explicitly:
lappend auto_path directory
One trick I often use is to put the directory containing the main script into the auto_path. The following command sets this up:
lappend auto_path [file dirname [info script]]
If your code is split into bin and lib directories, then scripts in the bin directory can add the adjacent lib directory to their auto_path with this command:
lappend auto_path
[file join [file dirname [info script]] ../lib]
Each script file in a library declares what package it implements with the package provide command:
package provide name version
The name identifies the package, and the version has a major.minor format. The convention is that the minor version number can change and the package implementation will still be compatible. If the package changes in an incompatible way, then the major version number should change. For example, Chapter 17 defines several procedures that use the HTTP network protocol. These include http::geturl, http::wait, and http::cleanup. The file that contains the procedures starts with this command:
package provide http 2.4
Case is significant in package names. In particular, the package that comes with Tcl is named http — all lowercase.
More than one file can contribute to the same package simply by specifying the same name and version. In addition, different versions of the same package can be kept in the same directory but in different files.
An application specifies the packages it needs with the package require command:
package require name ?version? ?-exact?
If the version is left off, then the highest available version is loaded. Otherwise the highest version with the same major number is loaded. For example, if the client requires version 1.1, version 1.2 could be loaded if it exists, but versions 1.0 and 2.0 would not be loaded. You can restrict the package to a specific version with the -exact flag. If no matching version can be found, then the package require command raises an error.
The package require command depends on an index to record which files implement which packages. The index must be maintained by you, your project librarian, or your system administrator when packages change. The index is created by the pkg_mkIndex command, which puts the index into a pkgIndex.tcl file in each library directory. The pkg_mkIndex command takes the name of a directory and one or more glob patterns that specify files within that directory. File name patterns are described on page 122. The syntax is:
pkg_mkIndex ?options? directory pattern ?pattern ...?
For example:
The pkg_mkIndex command sources or loads all the files matched by the pattern, detects what packages they provide, and computes the index. You should be aware of this behavior because it works well only for libraries. If the pkg_mkIndex command hangs or starts random applications, it is because it sourced an application file instead of a library file.
The package index, pkgIndex.tcl, is sourced in response to a package require command. The index instructs the package loading mechanism how to define the package. By default, source or load commands are specified so that packages are defined immediately as a side effect of package require. This is called direct loading. However, the original package index system used a deferred loading scheme layered on the auto_load mechanism and the unknown command hook, which is described on page 178. If you want deferred loading, use the -lazy option to pkg_mkIndex. The default behavior of pkg_mkIndex switched from -lazy to -direct in Tcl 8.3. The pkg_mkIndex options are summarized in Table 12-1.
Table 12-1. Options to the pkg_mkIndex command
Generates an index with | |
Generates an index that populates the | |
Dynamically loads packages that match | |
Displays the name of each file processed and any errors that occur. |
The files in a library can be either script files that define Tcl procedures or binary files in shared library format that define Tcl commands in compiled code (i.e., a Dynamic Link Library (DLL)). Chapter 47 describes how to implement Tcl commands in C. There is a C API to the package facility that you use to declare the package name for your commands. This is shown in Example 47-1 on page 698. Chapter 37 also describes the Tcl load command that is used instead of source to link in shared libraries. The pkg_mkIndex command also handles shared libraries:
pkg_mkIndex directory *.tcl *.so *.shlib *.dll
In this example, .so, .shlib, and .dll are file suffixes for shared libraries on UNIX, Macintosh, and Windows systems, respectively. You can have packages that have some of their commands implemented in C, and some implemented as Tcl procedures. The script files and the shared library must simply declare that they implement the same package. The pkg_mkIndex procedure will detect this and set up the auto_index, so some commands are defined by sourcing scripts, and some are defined by loading shared libraries.
If your file servers support more than one machine architecture, such as Solaris and Linux systems, you probably keep the shared library files in machine-specific directories. In this case the auto_path should also list the machine-specific directory so that the shared libraries there can be loaded automatically. If your system administrator configured the Tcl installation properly, this should already be set up. If not, or you have your shared libraries in a nonstandard place, you must append the location to the auto_path variable.
The basic structure of package loading works like this:
An application does a
package requirecommand. If the package is already loaded, the command just returns the version number of the already loaded package. If is not loaded, the following steps occur.The package facility checks to see if it knows about the package. If it does, then it runs the Tcl scripts registered with the
package ifneededcommand. These commands either load the package or set it up to be loaded automatically when its commands are first used.If the package is unknown, the
tclPkgUnknownprocedure is called to find it. Actually, you can specify what procedure to call to do the lookup with thepackage unknowncommand, but the standard one istclPkgUnknown.The
tclPkgUnknownprocedure looks through theauto_pathdirectories and their subdirectories forpkgIndex.tclfiles. It sources those to build an internal database of packages and version information. ThepkgIndex.tclfiles contain calls topackage ifneededthat specify what to do to define the package. You can use thepkg_mkIndexcommand to create yourpkgIndex.tclfiles, or you can create them by hand.In the case of deferred package loading, the
tclPkgSetupprocedure defines theauto_indexarray to contain the correctsourceorloadcommands to define each command in the package. Automatic loading and theauto_indexarray are described in more detail later.
As you can see, there are several levels of processing involved in finding packages. The system is flexible enough that you can change the way packages are located and how packages are loaded. The -lazy scenario is complicated because it uses the delayed loading of source code that is described in the next section. Using the -direct flag to pkg_mkIndex simplifies the situation. In any case, it all boils down to three key steps:
Use
pkg_mkIndexto maintain your index files. Decide at this time whether or not to use direct or lazy package loading.Put the appropriate
package requireandpackage providecommands in your code.Ensure that your library directories, or their parent directories, are listed in the
auto_pathvariable.
The package command has several operations that are used primarily by the pkg_mkIndex procedure and the automatic loading facility. These operations are summarized in Table 12-2.
Table 12-2. The package command
Deletes registration information for package. | |
| Queries or sets the command used to set up automatic loading of a package. |
Returns the set of registered packages. | |
Declares that a script file defines commands for | |
Equivalent to | |
Declares that a script uses | |
| Queries or sets the |
Compares version | |
| Returns which versions of the package are registered. |
| Returns 1 if |
You can create libraries without using the package command. The basic idea is that a directory has a library of script files, and an index of the Tcl commands defined in the library is kept in a tclIndex file. The drawback is that versions are not supported and you may need to adjust the auto_path to list your library directory. The main advantage of this approach is that this mechanism has been part of Tcl since the earliest versions. If you currently maintain a library using tclIndex files, it will still work.
You must generate the index that records what procedures are defined in the library. The auto_mkindex procedure creates the index, which is stored in a file named tclIndex that is kept in the script library directory. (Watch out for the difference in capitalization between auto_mkindex and pkg_mkIndex!) Suppose all the examples from this book are in the directory /usr/local/tcl/welchbook. You can make the examples into a script library by creating the tclIndex file:
auto_mkindex /usr/local/tcl/welchbook *.tcl
You will need to update the tclIndex file if you add procedures or change any of their names. A conservative approach to this is shown in the next example. It is conservative because it re-creates the index if anything in the library has changed since the tclIndex file was last generated, whether or not the change added or removed a Tcl procedure.
Example 12-1. Maintaining a tclIndex file
proc Library_UpdateIndex { libdir } {
set index [file join $libdir tclIndex]
if {![file exists $index]} {
set doit 1
} else {
set age [file mtime $index]
set doit 0
# Changes to directory may mean files were deleted
if {[file mtime $libdir] > $age} {
set doit 1
} else {
# Check each file for modification
foreach file [glob [file join $libdir *.tcl]] {
if {[file mtime $file] > $age} {
set doit 1
break
}
}
}
}
if { $doit } {
auto_mkindex $libdir *.tcl
}
}
The auto_path variable contains a list of directories to search for unknown commands. To continue our example, you can make the procedures in the book examples available by putting this command at the beginning of your scripts:
This has no effect if you have not created the tclIndex file. If you want to be extra careful, you can call Library_UpdateIndex. This will update the index if you add new things to the library.
lappend auto_path /usr/local/tcl/welchbook Library_UpdateIndex /usr/local/tcl/welchbook
This will not work if there is no tclIndex file at all because Tcl won't be able to find the implementation of Library_UpdateIndex. Once the tclIndex has been created for the first time, then this will ensure that any new procedures added to the library will be installed into tclIndex. In practice, if you want this sort of automatic update, it is wise to include something like the Library_UpdateIndex procedure directly into your application as opposed to loading it from the library it is supposed to be maintaining.
The unknown command implements automatic loading of Tcl commands. Whenever the Tcl interpreter encounters a command that it does not know about, it calls the unknown command with the name of the missing command. The unknown command is implemented in Tcl, so you are free to provide your own mechanism to handle unknown commands. This chapter describes the behavior of the default implementation of unknown, which can be found in the init.tcl file in the Tcl library. The info library command returns the location of the library.
The unknown command uses an array named auto_index. One element of the array is defined for each procedure that can be automatically loaded. The auto_index array is initialized by the package mechanism or by tclIndex files. The value of an auto_index element is a command that defines the procedure. Typical commands are:
The $dir gets substituted with the name of the directory that contains the library file, so the result is a source or load command that defines the missing Tcl command. The substitution is done with eval, so you could initialize auto_index with any commands at all. Example 12-2 is a simplified version of the code that reads the tclIndex file.
Example 12-2. Loading a tclIndex file
# This is a simplified part of the auto_load_index procedure.
# Go through auto_path from back to front.
set i [expr [llength $auto_path]-1]
for {} {$i >= 0} {incr i -1} {
set dir [lindex $auto_path $i]
if [catch {open [file join $dir tclIndex]} f] {
# No index
continue
}
# eval the file as a script. Because eval is
# used instead of source, an extra round of
# substitutions is performed and $dir gets expanded
# The real code checks for errors here.
eval [read $f]
close $f
}
If you do not want the unknown procedure to try and load procedures, you can set the auto_noload variable to disable the mechanism:
Auto loading is quite fast. I use it regularly on applications both large and small. A large application will start faster if you only need to load the code necessary to start it up. As you access more features of your application, the code will load automatically. Even a small application benefits from auto loading because it encourages you to keep commonly used code in procedure libraries.
The unknown command provides a few other conveniences. These are used only when you are typing commands directly. They are disabled once execution enters a procedure or if the Tcl shell is not being used interactively. The convenience features are automatic execution of programs, command history, and command abbreviation. These options are tried, in order, if a command implementation cannot be loaded from a script library.
The unknown procedure implements a second feature: automatic execution of external programs. This makes a Tcl shell behave more like other UNIX shells that are used to execute programs. The search for external programs is done using the standard PATH environment variable that is used by other shells to find programs. If you want to disable the feature all together, set the auto_noexec variable:
The history facility described in Chapter 13 is implemented by the unknown procedure.
Tcl searches for its script library directory when it starts up. In early versions of Tcl you had to compile in the correct location, set a Windows registry value, or set the TCL_LIBRARY environment variable to the correct location. Recent versions of Tcl use a standard searching scheme to locate the script library. The search understands the standard installation and build environments for Tcl, and it should eliminate the need to use the TCL_LIBRARY environment variable. On Windows the search for the library used to depend on registry values, but this has also been discontinued in favor of a standard search. In summary, "it should just work." However, this section explains how Tcl finds its script library so that you can troubleshoot problems.
The default library location is defined when you configure the source distribution, which is explained on page 732. At this time an initial value for the auto_path variable is defined. (This default value appears in tcl_pkgPath, but changing this variable has no effect once Tcl has started. I just pretend tcl_pkgPath does not exist.) These values are just hints; Tcl may use other directories depending on what it finds in the file system.
When Tcl starts up, it searches for a directory that contains its init.tcl startup script. You can short-circuit the search by defining the TCL_LIBRARY environment variable. If this is defined, Tcl uses it only for its script library directory. However, you should not need to define this with normal installations of Tcl 8.0.5 or later. In my environment I'm often using several different versions of Tcl for various applications and testing purposes, so setting TCL_LIBRARY is never correct for all possibilities. If I find myself setting this environment variable, I know something is wrong with my Tcl installations!
The standard search starts with the default value that is compiled into Tcl (e.g., /usr/local/lib/tcl8.4.) After that, the following directories are examined for an init.tcl file. These example values assume Tcl version 8.4 and patch level 8.4.1:
../lib/tcl8.4 ../../lib/tcl8.4 ../library ../../tcl8.4.1/library ../../../tcl8.4.1/library
The first two directories correspond to the standard installation directories, while the last three correspond to the standard build environment for Tcl or Tk. The first directory in the list that contains a valid init.tcl file becomes the Tcl script library. This directory location is saved in the tcl_library global variable, and it is also returned by the info library command.
The primary thing defined by init.tcl is the implementation of the unknown procedure. It also initializes auto_path to contain $tcl_library and the parent directory of $tcl_library. There may be additional directories added to auto_path depending on the compiled in value of tcl_pkgPath.
A generalization of this search is implemented by tcl_findLibrary. This procedure is designed for use by extensions like Tk and [incr Tcl]. Of course, Tcl cannot use tcl_findLibrary itself because it is defined in init.tcl!
The tcl_findLibrary procedure searches relative to the location of the main program (e.g., tclsh or wish) and assumes a standard installation or a standard build environment. It also supports an override by an environment variable, and it takes care of sourcing an initialization script. The usage of tcl_findLibrary is:
tcl_findLibrary base version patch script enVar varName
The base is the prefix of the script library directory name. The version is the main version number (e.g., "8.0"). The patch is the full patch level (e.g., "8.0.3"). The script is the initialization script to source from the directory. The enVar names an environment variable that can be used to override the default search path. The varName is the name of a variable to set to name of the directory found by tcl_findLibrary. A side effect of tcl_findLibrary is to source the script from the directory. An example call is:
tcl_findLibrary tk 8.0 8.0.3 tk.tcl TK_LIBRARY tk_library
This call first checks to see whether TK_LIBRARY is defined in the environment. If so, it uses its value. Otherwise, it searches the following directories for a file named tk.tcl. It sources the script and sets the tk_library variable to the directory containing that file. The search is relative to the value returned by info nameofexecutable:
../lib/tk8.0 ../../lib/tk8.0 ../library ../../tk8.0.3/library ../../../tk8.0.3/library
Tk also adds $tk_library to the end of auto_path, so the other script files in that directory are available to the application:
lappend auto_path $tk_library
If you supply a package, you need to follow some simple coding conventions to make your library easier to use by other programmers. You can use the namespace facility introduced in Tcl 8.0. You can also use conventions to avoid name conflicts with other library packages and the main application. This section describes the conventions I developed before namespaces were added to Tcl.
The first convention is to choose an identifying prefix for the procedures in your package. For example, the preferences package in Chapter 45 uses Pref as its prefix. All the procedures provided by the library begin with Pref. This convention is extended to distinguish between private and exported procedures. An exported procedure has an underscore after its prefix, and it is acceptable to call this procedure from the main application or other library packages. Examples include Pref_Add, Pref_Init, and Pref_Dialog. A private procedure is meant for use only by the other procedures in the same package. Its name does not have the underscore. Examples include PrefDialogItem and PrefXres.
This naming convention precludes casual names like doit, setup, layout, and so on. Without using namespaces, there is no way to hide procedure names, so you must maintain the naming convention for all procedures in a package.
You should use the same prefix on the global variables used by your package. You can alter the capitalization; just keep the same prefix. I capitalize procedure names and use lowercase letters for variables. By sticking with the same prefix you identify what variables belong to the package and you avoid conflict with other packages.
In general, I try to use a single global or namespaced array for a package (namespaces are discussed in Chapter 14). The array provides a convenient place to collect a set of related variables, much as a struct is used in C. For example, the preferences package uses the pref array to hold all its state information. It is also a good idea to keep the use of the array private. It is better coding practice to provide exported procedures than to let other modules access your data structures directly. This makes it easier to change the implementation of your package without affecting its clients. When choosing a namespace name, try to make it significant to your application.
If you do need to export a few key variables from your module, use the underscore convention to distinguish exported variables. If you need more than one global variable, just stick with the prefix convention to avoid conflicts, or provide accessor functions instead.
John Ousterhout has published two programming style guides, one for C programming known as The Engineering Manual and one for Tcl scripts known as The Style Guide. These describe details about file structure as well as naming conventions for modules, procedures, and variables. The Tcl Style Guide conventions use Tcl namespaces to separate packages. Namespaces automatically provide a way to avoid conflict between procedure names. Namespaces also support collections of variables without having to use arrays for grouping.
You can find these style guides on the CD-ROM and also in ftp://ftp.tcl.tk/pub/tcl/doc. The Engineering Manual is distributed as a compressed tar file, engManual.tar.Z, that contains sample files as well as the main document. The Style Guide is distributed as styleGuide.ps (or .pdf).