Overview

One of the primary purposes of an issue tracker is to collect organized data, and custom fields let you personalize that data. If you need to track how many angels can dance on the head of a pin, JIRA allows you to create a custom field for that. (It would be a number field, presumably, though number field values in JIRA have an upper limit of 100 trillion.) The appeal of creating custom fields is understandable, and it makes sense that as an organization grows and adopts JIRA for varied purposes, the number of custom fields grows along with it.

The problem is that custom fields are like glazed doughnuts—they always seem like a good idea at the time, but if you continue thinking that way in perpetuity then you’re sure to witness a reduction in speed. The number of custom fields in JIRA is one of the most significant factors in the performance of a JIRA instance. You can see the details of Atlassian’s performance testing of JIRA 6.4 in their enterprise documentation at https://confluence.atlassian.com/display/ENTERPRISE/Scaling+JIRA.

There are certainly other factors that can affect performance, such as the number of issues, number of concurrent users, and behavior (or lack thereof) of add-ons. But for most administrators, many of these factors are difficult to directly control. Managing the number of custom fields in your instance is the most straightforward way to control the performance of JIRA.

In addition to improving performance, custom field management can help reduce the administrative overhead involved in updating and maintaining custom fields in JIRA. Restricting the overall number of fields is only part of this process. Care and thought should be put into the selection of custom field types and how those fields are configured. This will pay dividends for your users in terms of reporting and user experience.

What’s the Big Deal with Custom Fields?

When users think of the fields associated with an issue, they think of the fields that are actually displayed on the screen for that issue. JIRA thinks differently. When a new custom field is created in JIRA, by default it is added to every JIRA project and issue type, even if it never actually appears on a screen. This means that JIRA has to make a decision about that custom field for most issue operations such as create, edit, and search, even when the field has no value. The part of JIRA most impacted by all this additional work is the Lucene issue index.

The Lucene index is your database’s butler. It maintains its own copy of all of JIRA’s issue information in a format that is optimized for search operations. When you submit a JQL search it is the Lucene index, not the database, that handles the request. This significantly reduces the load on the database and is the reason the database is rarely a performance bottleneck for JIRA, and that most JIRA searches are surprisingly fast. You can delete the Lucene index and rebuild it from the information in the database (though it may take a while in larger JIRA instances, so be careful).

To understand the impact of custom fields on the Lucene index it helps to visualize the Lucene index as a spreadsheet where issues are along the y axis and custom fields are along the x axis, as shown in Figure 6-1. Scaling along either direction (by adding an issue or a custom field) increases the complexity of searching for issues. Keep in mind that the Lucene index has to be kept synchronized with the data. This means that whenever the data is updated, the index needs to be updated as well. This is why creating, editing, and searching issues all impact the index. As more custom fields are added to JIRA, the time it takes to complete any of these operations increases.

Figure 6-1. Visualizing information in the Lucene issues index

If you are administering a large JIRA instance, it is important to understand this relationship between custom fields, the Lucene index, and performance. This can prove helpful in troubleshooting, especially if you notice a lot of activity reading from and writing to the disk where the index lives. Note that Atlassian recommends using a fast, local disk for the Lucene index files. More commonly, you should be exercising this knowledge to guide your decisions about how you set up and configure custom fields.

If you are administering a small JIRA instance, then you are unlikely to notice any performance benefits from reducing the number of custom fields below a few hundred. Small JIRA instances often experience no real performance issues at all. However, as I mentioned in the introduction, there’s more to managing custom fields than just performance benefits.

Choosing Custom Field Types

JIRA ships with 22 custom field types.¹ The following list includes the other 20 types along with discussion about when to use each one. Add-ons can provide you with many other custom field types, see their documentation for more information about those.

What is a Custom Field Context?

A custom field context controls the scope, default value, and certain type-specific features for a custom field in JIRA. Before addressing the details of contexts, though, I’ll address the matter of what this feature is called.

The custom field context is known by several names in different places in JIRA. The two most common names are custom field context and custom field configuration scheme. Other names are a combination of the first, e.g., configuration scheme context. I have a strong preference for the term custom field context. Custom field configuration scheme is too easily confused with field configuration scheme, which is one of the seven schemes of JIRA discussed in “Field Configuration Schemes”. For this reason, I’ll use the term custom field context, or simply context, throughout this chapter.

To access custom field contexts, go to Administration → Issues → Custom fields, click on the custom field name, and then click on Configure under the cog item on the right. You should see a screen similar to the one shown in Figure 6-2. (Edit is for changing a field name, description or searcher, but not for contexts.)

Figure 6-2. An example custom field context

The context is where you can usually set the default value for any custom field. This is useful for setting list options to the most common value or for adding a template or help text to a text field.

For option fields, the context allows you to edit the option list, whether this is for renaming options, disabling or deleting options, or adding new options to the list of options. Deleting an option that has been used in JIRA constitutes data loss, so make such a change with care and read “Deleting Contexts” carefully beforehand.

Single user picker fields can optionally have user filtering applied at their context. This allows you to restrict the set of valid users for such a field by groups and project roles. For example you could restrict the options for a QA Lead single user picker custom field to only accept members of a Quality Assurance project role.

Limiting Contexts

The default context created for all new custom fields is a global context. This means that upon creation, the field is potentially applicable to all projects and issue types in JIRA. By editing the configuration of the context, you can limit which projects and issue types it will actually appear in. An issue whose project and issue type are outside the scope of a field’s context will not display that field, even if it is on the correct screen and remains shown by the field configuration.

The ability to control context is useful when you need to add a field to a commonly used screen for only a few projects or issue types.

For example, let’s say that you would like to categorize bug fixes with a custom field named Bug Category that has options like Unhandled Exception, UI Error, Logic Error, and so on. The logical time to set this field would be during the resolution of the issue, but the field is only meaningful for issues of type Bug.

In this scenario, let’s assume that the project you want to modify has several issue types that are using the same screens and workflow. Without using contexts, you would have to take the following steps to implement the requirements for the new field:

1. Copy the transition screen being used to set the resolution in the workflow.

2. Add the Bug Category field to the new transition screen.

3. Copy the workflow to a new workflow for Bug issues.

4. Edit the new workflow to use the new transition screen.

5. Update the workflow scheme for the project to use the new workflow for Bug issues.

The need to fork the workflow means that you now have another workflow to maintain for this JIRA project. If your users need to be able to view or edit this field, you would need to add new screens and screen schemes, further complicating the configuration.

The solution using contexts is much more elegant. You simply limit the context to the Bug issue type only, then add the field to all screens where it is wanted. The field will then show only for the Bug issue type.

Another advantage of using custom field contexts is that they can actually improve JIRA’s performance. Part of this benefit comes from eliminating the schemes that you would have otherwise needed. But the actual narrowing of the context has a direct impact on the Lucene index. When a field’s context is limited, the Lucene index marks that field as unsearchable for everything outside of its context. This can improve search times because JIRA is able to skip the processing of irrelevant fields.

Adding Contexts

The most glorious aspect of custom field contexts is that you can have multiple contexts for a single custom field. For a given custom field you could create a different context for each project and set each one to have a different default value. By far though, this feature is most useful for allowing option fields to list different options for each context (project and issue type). Because of this flexibility, it is recommended to give custom fields as generic a name as possible. This will allow you to more easily reuse custom fields across JIRA, while minimizing the number of total custom fields.

Figure 6-3. The TRAIN project is using the default global context that lists Horse options as breeds

Figure 6-4. The DEMO project is using the same custom field but with a different context that lists Horse options by their names

There are limits to how contexts can be used. Each JIRA project can only have a single context, and each field can only have a single global context. A project-specific context will override a global context. This effectively means that you can never have different option values for different issue types within the same project.

Dangers of Custom Field Contexts

Making changes to JIRA’s seven schemes is like being guided across a crosswalk by a giant fuzzy teddy bear. Sure, there is some inherent danger, but the path is clear and you have a friend looking out for you. By contrast, making changes to custom field contexts is like being guided across a tightrope by a grizzly bear. As long as you are careful, everything will be fine, but things sure can go sour quickly.

Screens and Field Configurations and Custom Field Contexts, Oh My!

So far in this book we’ve seen three different ways to control fields in JIRA. Table 6-1 contains a review of what screens (“Issue Type Screen Schemes (ITSS)”), field configurations (“Field Configuration Schemes”), and custom field contexts do.

Table 6-1 shows what is unique about each feature. If you need a field to be displayed in an issue, it must be on a screen. To make a field required, set its renderer, or change its description from the default, then you must use a field configuration. And custom field contexts are the only way to set and change default fields values or option field values.

In fact, the only overlapping functionality between the three features is that they can all be used to hide fields. On this matter, there is no clear winner among them. Each has its own advantages and disadvantages for hiding fields. Deciding between them is largely a matter of figuring out which one would require the fewest number of configurations to enforce the required behavior.

For instance, screens have the best granularity of the three options, as you can use a different screen for create, edit, view, and transition operations all within the same issue. So if you need a field to show on an issue only at certain times, you should use screens to control the field’s presence. I also recommend using separate screens when they serve separate purposes. While you can use elaborate field configurations and contexts to enable you to reuse a screen in highly disparate issue types, at some point it becomes easier to maintain two separate screens than a multitude of strange field configurations.

Field configurations are the only way to hide system fields. If you need to hide one of the system fields that stubbornly refuses to disappear from the view screen (e.g., the Labels field) then you must use a field configuration. Field configurations are also the only feature where hiding a field is the active option. That is, hiding a field using screens or contexts is passive—they hide fields by exclusion. This means that ideally, field configurations should be used only to hide a field that rarely needs to be hidden, as that will minimize the number of overall configurations needed (as well as the effort to maintain those configurations).

Using custom field contexts to hide fields does have performance benefits, but those benefits are slight. More emphasis should be put on using contexts appropriately than trying to eke every last performance benefit out of JIRA. The question then becomes a matter of how often the field will be used. If the field is unique to only a few projects or issue types, then contexts are a good choice for hiding the field. In this scenario, you will only rarely need to modify the context when new projects and issue types are added, as the field will generally be inapplicable to most of them.

In summary, you should create a unique set of screens for each broad type of work that your organization is tracking in JIRA. This is also discussed in Chapter 4.

• If there are unique situations that need a field to be hidden for a minority of projects, use a field configuration to hide it and avoid creating a duplicate set of screens.

• If a field is only valid for a few projects or issue types, then limit its scope with a custom field context.

Where Is My Field?

With all of the different ways to hide fields in JIRA, JIRA administrators are commonly asked Where is my field?. This may seem like an opportunity to show off your prowess at differentiating between screens, field configurations, and contexts, but resist the urge to invoke your inner Sherlock Holmes. JIRA has a feature exactly for this question and you’ll save yourself a tremendous amount of time by using it. This feature is appropriately named Where is my field? and is documented at https://confluence.atlassian.com/display/JIRA/JIRA+Admin+Helper#JIRAAdminHelper-PermissionHelper.

The Where is my field? feature (Figure 6-5) is available to administrators on any issue’s create, edit, or view screen, and also via the Configure Fields button. It is excellent for quickly assessing the reason that a field is absent in that particular scenario. In addition to determining if a field has been left off a screen, is hidden by a field configuration, or is excluded from a context, it can also detect whether a field is missing from the view screen simply because its value has not been set.

One scenario that the Where is my field? feature will not detect is when a field has been hidden by the create or edit screen’s Configure Fields option. This option allows the user to select which optional fields are displayed on a screen and is intended to allow people to streamline their create screens. It’s not uncommon for users and administrators alike to forget they have hidden fields this way.

If you have gone through all of the above checks and the field is still missing, then you have my permission to get out your tweed and your pipe, call up Watson, and set about investigating. You have a case to crack! At this point you should probably check what add-ons are installed and whether any custom JavaScript code has been embedded in the field description in your field configuration.

Figure 6-5. In this case the Horse field does not appear on the View Issue screen because it has no value. Note that Project and issue type scope refers to the field’s context.

Minimizing the Number of Custom Fields

This chapter started off by stressing the importance of minimizing the number of custom fields use in JIRA. This section presents several strategies that can be used to do that. The section after this one describes how to correct the problem of already having too many custom fields.

Use generic field names

Making your custom field names generic enough to be reused for different purposes is critical for minimizing the number of custom fields. Several other strategies discussed here assume that you have used a generic name to begin with. The primary idea here is that you are more likely to be able to reuse an existing field rather than create a new one if you have used generic names for your fields. For example, a field named Document Link can be applied more widely than one named Link to Insurance Form.

Use a new context instead of a new custom field

For any new custom field request, you should check to see if an existing field will fit the need. All that is required is that the field have an appropriate name and be of the correct field type. Remember that option field values can be made completely different for each JIRA project using custom field contexts.

Add field details using a description in a field configuration

Your users may be disappointed with generic field names. In this case, the best strategy to appease them is to add detailed information about the field using a field configuration. Field descriptions in field configurations override the original custom field’s description and give you the opportunity to impress the details of the field that are lost through the use of a generic name. To use the Document Link field example from earlier, if the intention of this field is to hold a link to a specific type of document then it is good practice to add a helpful description (such as Provide a link to your Insurance Form) to the field description in the relevant field configuration.

Rely heavily upon the Description system field

Multiline text fields are exceptional for catching a lot of information in a single field. However, the Description system field that comes with JIRA is typically sufficient. If the purpose of a proposed multiline text field is to simply contain more information about the issue, the prudent thing to do is to simply roll that functionality into the Description field.

Use the default value for multiline text fields to create templates

One argument in favor of using custom multiline text fields is that they can be given default values, whereas system text fields like Description cannot. This allows you to create a template by embedding the template as the default value for the field. For example, a good bug report will contain the steps to reproduce the bug, the actual result, and the expected result. Rather than create three separate fields, you can create one multiline text field with three headers (Steps to Reproduce, Actual Result, and Expected Result) in the default value.

Don’t try to recreate long surveys in JIRA

Surveys and questionnaires often have long lists of what may appear to be custom fields. Creating a custom field for each question for a short, generic survey is fine, but naming a field something like Describe your biggest concern about the new coffee machine is not very reusable. Adding thirty custom fields for a single survey has too great of an impact on the instance as a whole to justify using custom fields for survey questions.

Use JIRA links instead of URL fields

Unless your URL link is intended to be a required field, use the link operation to add web links to an issue, rather than capturing that information in a URL field.

Challenge all requests for new fields

When I administer a JIRA instance, I require a meeting with every user who requests a new custom field. This allows me to interpret their requirements for the field and explain why I am such a stickler about adding new fields. Cooperative requesters will then work with you to figure out if an existing field can be repurposed or if the field is needed at all. Do not be overly stingy though. If a user has a valid request for a new field, give it to them, because that’s what custom fields were made for.

Verify that each field has a purpose

Ensure that the information captured by each field serves a justifiable purpose. If the data is not used to work on the issue, or for reporting, then it is not needed. It’s worth checking with a JQL query how custom fields are actually being used on a quarterly basis.

Establish and share a governance policy

Create a clear and definitive set of regulations that establish how new objects such as custom fields and configurations are added to JIRA. A governance policy conveys that you have a well–thought-out plan with regard to how JIRA is managed. It is also much more difficult to argue with a document than with a person.

Get executive backing

You can exercise every bit of advice listed above but it will all be for naught if you can’t enforce it. Share your governance policy with relevant managers and executives at your organization and impress upon them your need for their support in keeping JIRA running smoothly. Be ready to substantiate your policy with facts.

Reducing the Number of Custom Fields

Controlling the number of custom fields proactively is one thing, but what if you already have more than a few hundred custom fields in your instance? The truth is that there is no easy path to reducing the number of custom fields once they are already in place and in use. It is a time-consuming process and requires that a great deal of care be taken to ensure that no critical data is lost. But if you are already experiencing significant performance issues due to more than around 600 custom fields, this is still an efficient way to improve performance.

To start with, you should be looking for fields that are used so rarely that they need not exist, or fields that could potentially be merged with another field. To determine how widely a field is used, the Issue Statistics gadget on a JIRA dashboard can prove very useful. Edit the gadget and group by the custom field in question. Then little-used fields will either have very few non-empty values or very few non-default values. Your saved filter can also order by the Updated system field to see when an issue with a value in the field was last updated. If it was more than a year ago, then maybe that field is no longer being used?

One way to find candidates for merging fields is to create a list of fields for each custom field type in a spreadsheet outside of JIRA. This process will initially be tedious (the REST resource rest/api/latest/field.json may help) but will pay dividends as you can limit your scope for matches to only the relevant field type. This will also give you a place to make notes about which fields to merge after you contact the relevant JIRA project leads.

Merging fields can be dangerous. Make sure you document your process well and are cautious to not lose data. You also need to announce these changes to your users so that they can update any filters, dashboards, and agile boards as needed after the merge.

The most important step of all is to implement policies to control custom field growth after you have completed your field reduction.

Dos and Don’ts

This summary section contains some tips about managing custom fields that did not fit into any of the earlier sections, and some that bear repeating. The tips are included here as a list of Dos and Don’ts.