Over the last several chapters, the examples kept all of the code in a single directory. While this was convenient for illustration purposes, this will not work in practice. Furthermore, the examples often broke from established directory and package patterns found (and even enforced) in certain languages (e.g., Java, Go, etc.). In this chapter, we will correct the organizational shortcomings of prior chapters and demonstrate the facilities Bazel provides for working within a hierarchical directory structure.
Note
The directory structure we create here will be used throughout the rest of the book.
Setup
If you would like, you can confirm that all is working as expected by executing a build or run command for a target from the last chapter. If you do, also fire off a clean command; though this is strictly not necessary, it will help with keeping your top-level directory for the reorganization task.
Separating the Protocol Buffers
There are many ways to reorganize your code (e.g., language, client vs. server, etc.); beyond Bazel’s natural inclination toward a monorepo, however, this chapter does not offer any kind of strong opinion on this matter (i.e., do what makes sense for your project).
However, one obvious separation that we can do here is to pull out the files at build targets that encompass the various BUILD targets and definitions for the Protocol Buffers. These are referenced by both the client and the server and are their own language, so they can be easily pulled out from the current conglomerate directory.
Protobuf-only BUILD file
Save this to proto/BUILD.
Now, remove the corresponding targets (e.g., src:transmission_object_proto and so on) from src/BUILD. This will temporarily render the other build targets within that BUILD file unbuildable, but this will be fixed shortly.
Referencing Build Targets Outside of the Current Package
The previous examples had taken advantage of their colocation within the same directory/package to easily specify the dependencies among BUILD targets. Since everything was in the same location, each example’s build targets could specify local build targets (or would refer to external dependencies).
Updating the src/BUILD dependencies
Save the changes to src/BUILD.
One feature of Bazel is that, beyond the allowance for local dependencies, all dependency references are absolute paths with respect to a particular WORKSPACE. For the preceding examples, when we specified the dependencies on the newly created protobuf targets, this was done with respect to the root of the WORKSPACE (which is indicated by //). This is an important point: dependencies are not specified using paths relative to the current BUILD file. Although this may seem like an onerous requirement, it stems from the general Bazel theme of making everything explicit.
As seen previously, we can also refer to dependencies that are pulled into the WORKSPACE (e.g., our Go rules) by using the @ symbol prior to the name of the dependency; this informs Bazel of a dependency that is nonlocal to the WORKSPACE.
We will correct this problem within the next section.
Target Visibility
Many object-oriented languages (e.g., Java, C++, Objective-C, etc.) have a concept of visibility into an object’s member variables and functions. Typically, this is framed in the concepts of interface (i.e., the public-facing API) and implementation (i.e., the code that is used to actually perform the work). This division is hammered in countless coding books to achieve a separation of concerns. In theory, this gives the implementation the ability to change without affecting clients of said functionality; they all conform to the same interface.
In many languages, visibility need not be a binary choice; it is possible to specify visibility to some particular characteristic (e.g., members only visible to subclasses of a parent class, members only visible to the same Java package, etc.). This provides flexibility in terms of what pieces should be visible and to whom.
Bazel contains a powerful mechanism for target visibility, enabling the architect of the project to determine what should be visible and to whom. Notably, this is a language-agnostic feature ; any build target can take advantage of this particular Bazel feature, regardless of whether the language itself implements a form of member visibility. This enables us to determine which portions of our code should be considered valid for “public” consumption (i.e., the interface) and which should be retained privately (i.e., the implementation).
Throughout the course of this book, we have ignored the notion of visibility. This was enabled by virtue of all our code existing within the same directory and BUILD file (i.e., within the same Bazel package); all targets within a given package are automatically visible to one another. Additionally, without any additional specification, all targets within a given package are, by default, invisible to any external targets. That is, unless we actually make an explicit declaration regarding its visibility, a given target cannot be depended upon outside of its own package.
Although this might seem like an onerous requirement, it is actually one of the most powerful aspects of Bazel: as the author of your own code, you get to decide how best to structure it for building and how best to structure it for clients to use. These two need not be the same thing.
Since we do want clients external to a given package to make use of our code, we will need to specify the visibility. There are two major ways to accomplish this: (1) the package level and (2) the target level. We will explore both ways.
Package Visibility
As mentioned earlier, the default is that all targets within a given package are unavailable as dependencies by other targets which are external to the package. The simplest approach we can take is to make all targets visible.
Making all build targets visible
Save to proto/BUILD.
Now that we have updated the visibility of the dependencies, our build works as expected (src:echo_server should also work, but that is left as an exercise to the reader).
Note
Once again, an astute reader will note that we have include yet-another-new function(package) into our BUILD file. The package function exists to apply the same metadata to all targets within a given package. In this case, we are only using it to make modifications to the visibility.
Path-Specific Visibility
The prior section’s solution solved the immediate problem of getting the echo_client and echo_server targets to build. However, the solution of making every target within the package visible is heavy-handed to say the least. This kind of “all-or-nothing” approach doesn’t really lend itself to good code organization; in the limit it is only slightly better than putting everything into the same place.
Fortunately, we can do better. Bazel provides the ability to explicitly specify paths for target visibility. In this case, let’s restrict access to only the src package.
Restricting the visibility to a specific package
Save the proto/BUILD file.
Reconfirm that you are able to build both src:echo_client and src:echo_server. Now you have reduced the visibility to only targets that are strictly within the src package.
Note
Although the //src:__pkg__ specification will allow access to any proto target from within the src package, this specification does not automatically include any subpackages of src. That is, if you had a package such as //src/client, then the proto targets would not be visible to the targets within //src/client.
This can easily be addressed by modifying the visibility specification from __pkg__ to __subpackages__. This indicates that a given dependency should be visible both to a particular package and any subpackages therein.
Individual Target Visibility
In the last section, we specified the visibility at the package level. While this is always a good starting point, it still echoes the earlier “all-or-nothing” problem; we are still making statements about the visibility across all of the targets within a given package. Yet again, Bazel comes to the rescue.
Individual targets can declare their visibility; that is, each individual target can specify what other packages may depend upon it. This includes having an individual target make the blanket statement of visibility:public.
Specifying visibility at the build target level
Here, we removed the package level directive to make every target visible to all of src. Instead, we only made the transmission_object_java_proto target (required only by echo_client) visible to the src package. The transmission_object_go_proto (required by echo_server) is once again invisible.
Fixing the visibility for transmission_object_go_proto
Having fixed the dependency visibility, all builds are now happy again.
Mixing Package and Target Visibilities
Having demonstrated two different ways (i.e., both package and individual targets) to express visibility, it is important to also mention how they interact with each other.
Simply put, package level visibility specifications act as a default value for all targets within a package. Individual visibility specifications act as the final value for the visibility of that particular target. That is, there is no attempt to merge the values between the package and individual specifications; it is always a replacement operation.
Although this might seem draconian, an important thing to remember is that Bazel seeks to make things explicit; complicated implicit merging among visibility rules does not serve this purpose. Yes, this might entail some extra typing when trying to create some very particular rules; however, being explicit wins out over momentary convenience.
Bazel provides a mitigating strategy to this verbosity through the construct of a package_group. A package_group allows you to assign metadata (e.g., visibility rules) across a set of packages. This provides a nice middle ground between assigning visibility to individual targets and requiring a package-wide visibility policy.
Separating the Client and Server Code
Having separated our protobuf code into its own package, we will also separate out the client and server code into their own packages.
Separating the Echo Server Code
Creating the server/echo_server/BUILD file
Save that file to server/echo_server/BUILD. Trying to build this will simply reintroduce the proto visibility issues we saw earlier. Let’s first update the visibility rules for the necessary target.
Updating the visibility package for transmission_object_go_proto
Eliding the Build Target
While it might be tempting to cry foul at this point in time for Bazel, this convenience is providing a very powerful convention: if a target has an identical name to the package wherein it is contained, then it can be considered the “public”-facing target (i.e., the interface) for that package. This normalizes and simplifies expectations from external packages about which target(s) one should depend upon.
Notably, this convention is not a requirement of Bazel, but its existence is extremely powerful and can help reduce cognitive load when creating and analyzing build dependency trees.
Separating the Echo Client Code
Creating the client/echo_client/command_line/BUILD file
Save the changes down to the client/echo_client/command_line/BUILD file. Once again, we will need to update the appropriate proto/BUILD target visibility; otherwise, our echo_client target will once again fail to build.
Updating the visibility for transmission_object_java_proto
Save the file to proto/BUILD.
Note
You might notice that we have created a slightly different specification for the visibility for transmission_object_java_proto vs. its Go counterpart. In particular, while the Go version was very specifically targeted toward the echo_server package, the Java version has a wider set of potential packages (i.e., everything under the echo_client package). This is again done in anticipation of upcoming chapters.
Cleaning Up
Since we did not change any actual code, just move it around and create/reconfigure BUILD files and targets; all functionality should work as before. Verification is left as an exercise to the reader.
Final Word
Over the course of the last chapter, you employed the tools of Bazel to reorganize the code into a (more) scalable development structure. Although still toy examples, there should be enough content to begin to use the constructs of Bazel to craft your code structure into something easy to understand, scalable, and controllable.
Although what was done within this chapter represents one particular organization, it should not be considered canonical by any means. For example, the location and visibility of the Protocol Buffer and derived language-specific targets may be changed (e.g., bring the language-specific proto targets closer to their actually usage). Another example would be changing to a language-centric directory/package structure.
There is no “right” answer; there are trade-offs to each possibility. Regardless, Bazel flexibly supports the type of code organization that best suits your needs while providing tools that aid in maintaining this structure over time.