Overview

Atlassian has released two new versions of JIRA each year for some years now. Recent releases and their dates are listed at https://confluence.atlassian.com/display/JIRA/JIRA+Release+Summary and include:

• JIRA 6.0—May 2013
• JIRA 6.1—September 2013
• JIRA 6.2—February 2014
• JIRA 6.3—July 2014
• JIRA 6.4—March 2015
• JIRA 7.0—End of 2015

Major versions such as 6.3 and 6.4 are supported for two years after the first release of that version. For example, JIRA 6.4 was released on March 17th, 2015, so it will be supported by Atlassian until March 17th, 2017. This is a change from an earlier policy that supported versions until two years after their final patch release. The current end-of-life details for JIRA and other Atlassian products are available at https://confluence.atlassian.com/display/Support/Atlassian+Support+End+of+Life+Policy.

All of this means that upgrading JIRA on a yearly basis is a common practice. The process of upgrading JIRA is complex enough to warrant having a chapter dedicated to it, especially since doing it once per year is just long enough to mislay your notes from the last time you did the same job. The Atlassian documentation about upgrading JIRA starts at https://confluence.atlassian.com/display/JIRA/Upgrading+JIRA and lists a number of different approaches. This chapter describes an upgrade procedure that corresponds most closely to Upgrading JIRA Manually and can be used over and over—though of course, it also depends on the specific release notes for each JIRA version, which are published at http://confluence.atlassian.com/display/JIRA/Production+Releases.

Preparing for an Upgrade

The very first thing to do is to work out which JIRA version you’re allowed to upgrade to. JIRA licenses permit you to upgrade to any newer version released during a period when you are covered by a support contract. The first year of support comes bundled with the initial purchase. So if you bought JIRA on July 9th, 2014 then you are allowed to upgrade to any version released before July 9th, 2015, which includes JIRA 6.4. However, if the next version (7.0) happens to be released in October 2015, then you’ll have to renew your support contract to be able to upgrade to it. You can find your support details at Administration→System→License.

The next thing to do is to make sure that your backups are working. If you don’t have a full Bare Metal Recovery procedure to test them with, then at least check that recent backups of the data and attachments are being created and contain some reasonable-looking data.

The hardest part of JIRA upgrades used to be working out whether all third-party add-ons were compatible with the new version of JIRA. You had to look up each add-on listed in Administration→Add-ons→Manage add-ons at the Atlassian Marketplace (marketplace.atlassian.com, formerly plugins.atlassian.com) by hand. If your JIRA instance is behind a firewall and cannot connect to the Marketplace, then you may still have to do this. But most JIRA administrators can now simply use the JIRA Upgrade Check feature at Administration→Add-ons→Manage add-ons to get the same information.

You may ask what can I do if I’m using an add-on that isn’t compatible with the new version of JIRA? Check if there is already an open issue at wherever that add-on tracks issues about compatibility (the add-on’s own issue tracker, not jira.atlassian.com), and create a support request if there isn’t one already there. If the add-on is supported, you can also try contacting the add-on vendor. Some add-ons take months after a new JIRA release to catch up, which is one reason why some people choose to wait for the first dot release (i.e., 7.0.1 instead of 7.0).

Many instances of JIRA are integrated with many other applications as well as email and LDAP. There may be scripts that access JIRA remotely to retrieve or modify information for other systems. These integrations should all be tested with an upgraded development JIRA instance before upgrading a production JIRA. You may need to check the JIRA access logs to find out where scripts that regularly access JIRA remotely are coming from.

The amount of effort a JIRA upgrade takes usually corresponds to the number of issues (size of data), the number of add-ons and the number of other systems that are integrated with JIRA.

You may also want to start planning possible dates for the upgrade with your users and other groups. Most upgrades I’ve done recently seemed to require about two hours of JIRA downtime. However in-place database upgrades (see the section “Upgrade Approaches”) and improved installers have somewhat reduced this time for large JIRA instances. For large instances of JIRA the upgrade time is nearer to six hours, with most of the time spent on a full reindex.

Important JIRA Locations

There are at least four locations or services involved in every working JIRA instance. These are:

jira_app

The JIRA Install directory; this is where JIRA was unpacked and is named something like atlassian-jira-core-7.0.0-standalone or atlassian-jira-6.4.4-standalone. On Linux, I usually create a soft link named jira_app to the install directory from its parent directory to make future upgrades easier, like this:

lrwxr-xr-x jira jira 40 May  7 08:00 
            jira_app@ -> atlassian-jira-core-7.0.0-standalone

jira_data

The JIRA Home directory; this is where JIRA stores any data that isn’t in the database. This includes files that are added as attachments to issues, files for add-ons, project avatars, and caches of the database data in the Lucene index files. This is the directory name that is configured in the jira-application.properties file under the install directory.

This directory can be named anything you wish, but I usually name it something like jira_data_700 if it is being used for a JIRA 7.0.0 installation. On Linux, I usually create a soft link named jira_data to the JIRA home directory at the same level. Don’t put the JIRA data directory under the install directory or an installer-driven upgrade will not work.

Database

The database used by JIRA contains both the issues added by JIRA users, the JIRA configuration that is changed by JIRA administrators and add-on data. The two major sets of files that are not kept in the database are attachments and add-on jar file, which are kept in jira_data. Avatars (custom project icons) are also not in the database.

The database configuration is stored in a file named dbconfig.xml in the jira_data directory. If you are using the non-production evaluation H2 database, it is stored in the jira_data/database directory.

User Directory

User directories are where JIRA keeps its lists of users. By default, JIRA uses an internal user directory which is stored in the database. If Active Directory, Crowd, or some other separate directory service is used, then the upgraded JIRA needs to use that too—otherwise no one will be able to log in, including JIRA administrators (see “Adding Users”). It’s always a good idea to keep one user with JIRA system administration rights in the JIRA internal user directory for this reason.

Upgrade Approaches

The upgrade procedure as described in “A General Upgrade Procedure” uses a new database for the new instance of JIRA. Leaving the database used by the current JIRA instance untouched makes abandoning an upgrade easy. However, using a new database does mean that the data has to be migrated either by a Database Administrator using a database tool to copy it, or using an XML backup file which can be very large and somewhat slow to generate for large JIRA instances. You may also want to disable auto-export for large JIRA instances as documented.

The alternative is to upgrade the database in place. This means that you can reuse the same database for the new instance. The database schema, including the database indexes, will be modified by the new instance of JIRA when it starts up. Of course, you should back up at the database level before that occurs, and upgrading a development JIRA instance first is still a good idea.

Simple Upgrades

If you aren’t changing your machine or database in any way and you’re using Linux or Windows, then you can simply follow the steps listed at https://confluence.atlassian.com/display/JIRA/Upgrading+JIRA+Using+a+Rapid+Upgrade+Method and download the appropriate installer.

The installer will prompt you for the necessary information and also save your inputs so you can rerun an upgrade as necessary. One difference is that JIRA is shipped with a Java Runtime Environment (JRE) that it uses by default. Setting the JAVA_HOME environment variable no longer changes which Java VM is used; you have to edit bin/permgen.sh, which is where the installer sets JAVA_HOME.

A General Upgrade Procedure

This section describes a general procedure for upgrades that is applicable to all versions of JIRA and to changing the server, database, or even OS. Since no procedure can fit every upgrade precisely, please use it only as a basis for your own customized upgrade procedure. Briefly, the procedure looks like this:

• Install and configure the new, upgraded JIRA instance ready for data.

• Take an XML backup of the current JIRA instance and shut it down.

• Import the XML backup into the upgraded JIRA instance and test it.

• Declare the upgrade complete.

The four instances of JIRA referred to in this upgrade procedure are:

• The current production JIRA (e.g., version 6.2)

• The upgraded production JIRA (e.g., version 7.0), usually on the same server as the current production JIRA—upgrading JIRA and moving JIRA to a new server are two steps that are best done separately

• The current development JIRA (e.g., version 6.2)

• The upgraded development JIRA (e.g., version 7.0), often on the same server as the current development JIRA

The upgrade steps follow:

1. As usual before changing anything, create a backup. Then start taking your own notes about each of the upcoming steps. Record what you did and any unexpected results.

2. Unpack the new version of JIRA on the development server, making sure the files are owned by the service account that will run JIRA. This location is the new jira_app directory for this instance. Any existing jira_app soft link will eventually refer to this directory when the upgrade is complete.

3. Create a new JIRA Home directory named jira_data_70 for the new development JIRA instance. Any existing jira_data soft link will eventually refer to this directory.

4. Create a new, empty database for the new development JIRA instance. Make sure that the database uses the correct character set, collation order, table type, and so forth (or at least the same ones as the existing JIRA database).

5. Configure the new development JIRA files. Some of the files in jira_app that are likely to be changed include:

   atlassian-jira/WEB-INF/classes/jira-application.properties

   Used to set jira.home to the absolute path of jira_data.

   bin/setenv.sh

   Used to change the JVM memory settings and to uncomment DISABLE_NOTIFICATIONS so that the development instance doesn’t send or retrieve email. Note that you can still use the Send a Test Email feature even with the rest of email disabled. JIRA on Windows uses the Registry to store the JVM settings when running as a service.

   lib/*.jar

   You should check that the JDBC driver jar file for your database is available here. Recent versions of JIRA ship with the drivers for PostgreSQL and Oracle but not MySQL.

6. If the current JIRA has a jira_data/jira-config.properties file, then make sure this is copied over to the new JIRA. This file contains advanced configuration changes that can’t be made in the UI, as documented at https://confluence.atlassian.com/display/JIRA/Advanced+JIRA+Configuration. If no such changes have been made, then this file will not exist.

   Copy or modify any other files such as custom icons for issue types, priorities, resolutions and statuses, or a logo image file. You can find a list of files that have been changed in the current production JIRA under Administration→System→System info, Modifications. If you added any custom event templates (see the section “Workflows and Events”) or changed your email templates, you’ll also need to merge them over to the upgraded instance.

   Tip

   It’s about now that I usually start to wish I had used version control or a tool such as puppet or chef to record the changes I made to the current JIRA when I last upgraded it. It’s not too late to start!.

7. If you want to, you can install any updated add-on files that you’ve downloaded manually at this stage. Having the add-ons in place will reduce the number of error log messages about missing add-ons when you import data into the new instance.

   Add-on .jar files go in jira_data/plugins/installed-plugins (you’ll have to create this directory). If an add-on’s installation involves more than just copying a .jar file, I usually postpone installing it until after this step. Some add-ons use .obr files that have to be uploaded not preinstalled (these are bundles of .jar files).

8. Export the data from the current production instance as a compressed XML backup file (Administration→System→Backup system). Use a filename that includes the JIRA version, date and time. If your JIRA instance is too large for this, then use a copy of the production database.

   There is a short period here where JIRA is still active but an XML backup is running. Any changes made by users during this period will be lost when the upgrade is complete, so make sure you warn people not to use JIRA after a certain time.

   You may ask Why can’t I just make my current JIRA read-only? You could do this, but then the port used by JIRA is still in use, and the new instance can’t use it. Also, there’s no really convenient way to allow users to log in to a read-only instance, which is necessary for issue security.

9. Make sure any automated script and network monitoring systems are aware of the upcoming outage and then shut down the production JIRA instance. This is when the clock starts timing for just how long the upgrade inconveniences your JIRA users.

10. Create a new directory called jira_data/data/import in the new development instance.

11. Copy the XML backup file into the import directory, or create a soft link to it there.

12. Start up the new JIRA development instance and browse to the start page. You shouldn’t be prompted to log in since you’re using a new database and a new jira_data directory. Choose the option to let you set up JIRA yourself, not the option for a demonstration instance.

    Caution

    If you are prompted to log in then you’ve probably just done an in-place database upgrade (see the section “Upgrade Approaches”) and you can skip the next step of importing the XML backup. However, if you stop the upgrade after this point, you will need to restore the database from a database backup.

13. Click on the link to import your existing data and give the name of the XML backup file in the import directory. Disable outgoing mail if given the choice. Then click Restore.

    If you need to update the license for this version of JIRA, you will be prompted for a new license key.

    You may also get a message about an import failing because the paths for the index directory or attachments directory wasn’t found. This means that these directories were at locations in the current JIRA instance that don’t exist on the upgraded instance. In that case, just click the retry with default paths link.

    You can follow the progress of the import in jira_app/logs/catalina.out or in jira_data/log/atlassian-jira.log. It may take some minutes to parse the XML, store the generic entities to the new database, create the Lucene indexes, apply database schema changes, and reindex for each schema change. The longest part of all this is usually the reindexing for every schema change and can take some hours for large instances.

14. Copy the attachments over to the new jira_data/data/attachments directory. This can also take some time if there are many large attachments. Check for other directories with any content in jira_data/data and copy them too.

15. Once JIRA is available again, you will want to test it as described in the next section, “Testing an Upgrade”.

16. After giving users some time to test the upgraded development instance, repeat all these steps for the production JIRA instance.

    Tip

    You can set a banner across the top of all of JIRA’s pages using Administration→System→Announcement banner. This is a great way to warn users of an upcoming JIRA outage. For extra notice, you can use the existing CSS by surrounding your banner text with div elements:

    <div class="infoBox">
    Your announcement text goes here.
    </div>

    but be careful since invalid HTML here can make JIRA unusable until you clear it in the database.

17. After the upgrade, you will need to change the name of the database that should be backed up. You may also need to change the JIRA Home directory location in the backup scripts. Any startup scripts to start JIRA after a server reboot should be checked too. If you have used soft links for jira_data and jira_app, then fewer things should have to change.

    Update any local Bare Metal Recovery documents and other administration notes, take a breath, and you’re done!

Testing an Upgrade

The first and easiest test is whether user authentication is working—can you log in?

Start at the Administration→System→System info page and look for errors or outdated settings. Does the Base URL need to change? Do the locations for Attachments, Indexing, and the Backup Service at Administration→Services look correct?

If the upgraded JIRA is a development instance, then you can compare it to the production JIRA directly in a browser. Compare the Administration→System→System info pages for unexpected differences. Pick three issues at random and check that their data, including some attachments, is the same. You can also print all this out to use when testing the production JIRA after it is upgraded. Remember that some filter results will look different if they have any kind of date clause in them.

The most likely area for trouble with an upgrade is with third-party add-ons. Check the list of add-ons for disabled add-ons or individual add-on modules.

Check that all the known integrations of other applications with JIRA are working as expected. Check that the scripts that access JIRA remotely are also still working.

Another useful test is to count the number of custom fields in the development instance and in the production instance. Disabled custom field from add-ons may be responsible for any difference in numbers.

Check that all the custom fields that used to be searchable in the Issue Navigator are still searchable. If not, then a custom field searcher add-on may not be working.

If the upgraded JIRA is the production JIRA, then check that attachments can be downloaded from issues and that email can be sent and received.

The most likely place to notice that something is wrong is in the directory jira_app/logs/catalina.out. Searching in this file for ERROR and WARN will usually suggest things to investigate further. To me, a good upgrade is one that results in a minimal number of such warning messages in log files, and no log messages that repeat every minute and cause the log file to become bloated. I also like the Log Scanner, available at Administration→System→Atlassian support tools, which checks the JIRA log for known issues using text patterns. It does raise some false positives, but can be useful for a preliminary error check and provides links to the appropriate JIRA Knowledge Base articles for each error.

Troubleshooting an Upgrade

Most JIRA upgrades go just fine, but sometimes what was intended to be just a quick upgrade turns into a Bad Day. You’ve got frustrated users and only a few cryptic error messages to go on. What now?

If this is the upgraded production instance, then you should probably abandon the upgrade and let people use the current production JIRA instance again. To do that, simply stop the upgraded instance of JIRA, change any soft links back to where they used to point, and restart the current instance of JIRA.

Then check whether it was really a problem at all. Some messages in JIRA log files can appear alarming, but aren’t in fact all that serious. For example, if you haven’t installed an add-on that provides a custom field type then you will see an error like:

2011-04-16 21:54:23,847 main ERROR [jira.issue.managers.
      DefaultCustomFieldManager] Could not load custom field type plugin with 
      key ... Is the plugin present and enabled?

JIRA preserves the data in any custom fields that use such custom field types, but doesn’t display the custom field at Administration→Issues→Custom fields. This means that you can’t delete such fields and that you will see the error message every time you restart JIRA. Old services and listeners can generate similar error messages in log files.

More troublesome are messages that keep occurring, filling up a log file and making it harder to use when real problems occur. These are always worth tracking down—or as a last resort, reducing the log level for just that area in log4j.properties.

Other approaches to use when troubleshooting an upgrade are:

• Log into JIRA using the simplest URL possible. If you have forwarding set up with Apache or NGINX, then disable that temporarily and use the default JIRA URL with http and port 8080.

• If you can’t authenticate using Active Directory, then try logging in as some user that is defined in JIRA’s internal directory service. The original JIRA administrator account may even still work for you.

• Check your https settings very carefully and only apply them after checking that everything else is working.

If all else fails, contact Atlassian Support or an Atlassian Expert to ask for help.

Further Reading

The official Atlassian support policy for JIRA can be found at http://confluence.atlassian.com/display/Support/Atlassian+Support+End+of+Life+Policy.

Logging in JIRA is configured at Administration→System→Logging and profiling. More detailed documentation about the underlying log4j logging framework can be found at http://logging.apache.org/log4j/1.2/manual.html.

There is an existing Suggestion to be able to make JIRA read-only at http://jira.atlassian.com/browse/JRA-1924.