Now that we have concrete definitions of changes and modules in each of our three systems, we can finally begin to investigate how developers work with these modules.
We begin our exploration with a simple question: are most changes to the code constrained to a single module? Figure 21-5 shows a histogram of the number of modules modified per change for each system, and Table 21-4 presents some summary statistics.
When making these code changes, how many modules does a developer consult? We can answer this question quantitatively for Mylyn given the availability of task context data, but not for the other two systems.
Figure 21-6 shows the number of modules that were examined by a developer for each change. The mean number of examined modules is 2.365, with a median of 2. The mean of examined modules is higher than the mean number of modules that are actually modified (1.634), which suggests that developers occasionally consult modules that they do not end up actually changing.
If we take a closer look at the modules that are examined but not modified in each change, we find two of Mylyn’s modules stand out. In 13% of changes, a developer examined the Tasks module without changing it, and in 8% of changes the same holds for the Bugzilla module. We might infer from this that the Tasks and Bugzilla modules are less cohesive than the other modules or are coupled to many other modules in the system. We will revisit this hypothesis later, when we look at how modules emerge from a developer’s interaction with the system.
How does the number of modules examined affect the time required to complete a task? We hypothesize that when developers examine more modules, the time required for task completion increases. This hypothesis is indeed supported by our data, which show a weak correlation between the number of modules examined in a task context and the amount of time spent working on that task (Pearson’s r=0.20, p < 10‒15).
Does the number of extra modules examined vary between developers? In particular, do more experienced developers need to consult fewer modules when making changes to the system? We can use the number of closed bug reports assigned to each developer as a heuristic for the developer’s experience with the system. If we plot this against the number of modules examined but not modified in each change, we get the plot shown in Figure 21-7.
Figure 21-7. Developer experience versus number of modules examined but not modified in each change. Experience is measured by the number of bugs closed by the developer prior to making the change.
We observe that there seems to be little differentiation on the left side of the graph, with both new and experienced developers frequently consulting only a few extra modules when making changes. However, the story is quite different on the right side of the graph, where we see that most of the developers who consulted many extra modules had little experience.
All of our analysis so far has started from the assumption that the separation of each system into modules based on the filesystem structure is reasonable. We now approach these systems from a different perspective: can we infer modularity from the developers’ interaction patterns? Do modules emerge as a developer works with the system that differ from those explicitly set out in the design? We have already shown that changes to Evolution almost always touch only a single module (86.6% of changes), so we focus on Firefox and Mylyn for this part of our analysis.
We begin with a simple count of the number of times each pair of modules is changed together. For Firefox, the pairs that occur in at least 1% of commits are:
browser/base, browser/themes browser/places, toolkit/places layout/base, layout/generic browser/themes, toolkit/themes toolkit/locales, toolkit/mozapps browser/base, browser/locales toolkit/content, toolkit/themes
There are a few observations we can make from this list. First, the “base” module inside each top-level directory seems to be tightly coupled to the other modules within the same top-level directory. This seems reasonable from a design perspective. Second, modules with the same name in different top-level directories change together: for example, browser/places and toolkit/places often change together, as do browser/themes and toolkit/themes. Here, the naming suggests that the system architects realized there were multiple ways in which the system could be sliced-and-diced, indicating possible crosscutting concerns, and that choosing one separation into modules would cause other cohesive parts of the system to be split up across multiple modules. In particular, choosing to divide the browser and toolkit components of Firefox has caused the places and themes components to become split.
The most interesting interactions are those that do not match either of these two explanations. From the previous list, we see that toolkit/locales and toolkit/mozapps seem to be related, as are toolkit/content and toolkit/themes. Further investigation into the details of the changes that modified each of these pairs of modules may reveal system functionality that could be extracted into a more cohesive module.
If we apply the same approach to Mylyn, we find the following pairs:
tasks, bugzilla tasks, context tasks, jira tasks, help tasks, commons tasks, trac
A quick glance at this list reveals an obvious pattern: changes to the tasks module often involve changes to other modules in the system as well. Recall from our earlier analysis that in 13% of changes, more than for any other module, a developer examined the tasks module without ultimately changing it. Combined with this new data, there is evidence that the Mylyn tasks module is coupled to much of the rest of the system.
To explore the relationship between the tasks module and the rest of the system, we used association rule mining, specifically frequent pattern mining [Agrawal et al. 1993] on the Mylyn task context data to discover rules about how developers examine the system modules when working on a change.[28] In this context, association rule mining yields rules of the form “if a developer examines module X, he is likely to also examine module Y as part of the same change.” Each rule has an associated confidence value that measures the percentage of the dataset over which the rule holds. Our data mining produced the following rules for the task context data:
doc -> tasks (100%) help -> tasks (100%) trac -> tasks (84%) team -> tasks (82%) bugzilla -> tasks (81%) jira -> tasks (81%)
These were the only rules found with a confidence level of at least 80%. We see that from the perspective of a developer working on the system, any consultation of the six modules listed as antecedents in the preceding rules (doc, help, etc.) is very likely to also include consultation of the tasks module as part of working on the same change.
Having established that “all roads lead to tasks” within Mylyn, we can dive deeper to see whether any relationships emerge at the class level as developers interact with the system. First, we consider the perspective of a developer examining the system by once again mining the task context data, which yields these rules at a confidence level of at least 60%:
NewBugzillaTaskEditor -> AbstractRepositoryTaskEditor (88%) BugzillaTaskEditor -> AbstractRepositoryTaskEditor (85%) NewBugzillaTaskEditor -> BugzillaTaskEditor (79%) IBugzillaConstants -> BugzillaClient (71%) JiraRepositoryConnector -> AbstractRepositoryConnector (63%) BugzillaRepositorySettingsPage -> AbstractRepositorySettingsPage (62%) SynchronizeTaskJob -> RepositorySynchronizationManager (61%)
The most common type of rule identified by this analysis relates a concrete implementation to its corresponding abstract base class. It seems reasonable that developers would navigate between an abstract class and its concrete subclasses while exploring the system, so these relationships are not unexpected.
If we instead mine the change data, looking only at classes that were actually modified rather than those that were consulted by the developer, we get a very different set of rules (with a minimum confidence of 70%):
ITaskListExternalizer -> BugzillaTaskExternalizer (87%) NewAttachmentWizard -> AbstractRepositoryTaskEditor (81%) AbstractJavaRelationshipProvider -> XmlReferencesProvider (80%) ActiveHierarchyView -> ActiveSearchView (76%) PdeStructureBridge -> AntStructureBridge (75%) RepositoryTaskAttribute -> AbstractRepositoryTaskEditor (74%) ContextRetrieveAction -> ContextAttachAction (74%) TaskCategory -> TaskListView (71%) ContextAttachAction -> ContextRetrieveAction (71%)
These rules present a different view of the system. For example, we observe there is
a close relationship between the ActiveHierarchyView
and ActiveSearchView, and similarly between the
ContextRetrieveAction and ContextAttachAction. These views and actions, respectively, may contain
common functionality that could be extracted to another class. Some of the relationships
are unexpected. For example, it is unclear why PdeStructureBridge and AntStructureBridge are so closely related, as they are very different
implementations of a common superclass.
This data shows that the modules of Mylyn from the perspective of a developer making changes to the project may be quite different from the modules built into the design through the separation of code into different classes and Java packages. Where these two views of modularity differ, developers may need to consider and reason about more of the system than desired, even when making only small logical changes.
[28] Association rules were generated using the Apriori algorithm as implemented in ARtool (http://www.cs.umb.edu/~laur/ARtool).