So far we have been making changes in the packaging org for the FormulaForce application. This has worked well enough, as you're the sole developer in this case. However, when you add more developers and teams other considerations comes into play, mainly traceability of code changes and also the practical implications of conflicts when more than one developer is working in a development org at a time.
This chapter sees the packaging org take on more of a warehouse-only role, accessed only when there is a need to create a beta or release package and no longer for day-to-day development as we have been doing. This principle can become quite important to you and your customers if they desire that you demonstrate compliance and auditability with respect to controls around your software development process and life cycle.
As the code base grows and the number of contributions to it at any given time increases, it's important to monitor the quality of the code more frequently. Finding out that your Apex tests are failing prior to an upload is frustrating and costly when you're trying to release the software to your customers. Continuous Integration works with a Source Control system to validate the integrity and quality (through your tests) of your application on an automated basis, often triggered by developers committing changes to Source Control.
In this chapter, you will learn the following:
- Developer workflow
- Developing with Source Control
- Hooking up to Continuous Integration
- Releasing from Source Control
- Automated regression testing
A development workflow describes the day-to-day activities of a developer in order to obtain the latest build, make changes, and submit them back to a repository that maintains the current overall state of the source code. Having a solid well-defined developer workflow, supporting tools, and infrastructure is critical to the efficiency of any development team.
The first thing a developer will want to know is where the source code is, basically a source control system, such as Git. The next would be how to get it into a form so that they can start executing it. Salesforce provides a toolkit known as the Migration Toolkit that helps write scripts to deploy to a Salesforce org using the popular developer scripting language known as Ant. We will be building such a script in this chapter. Such scripts are also used by Continuous Integration build servers described later in this chapter.
The Migration Toolkit also provides a means to extract from Salesforce org source that has all the components needed to represent your application, not just the Apex and Visualforce source files, but XML files describing Custom Objects, fields, tabs, and practically any other component that can be packaged. We will use this facility to extract the current state of our packaging org in order to initially populate a Git repository with all the component files needed to maintain further changes to the source code outside of the packaging org that we have been developing within so far.
At the time of writing, Salesforce announced Salesforce DX. It's a new developer experience focused on enterprise developer needs. The goal is to provide additional tools and APIs that better facilitate source control-based development with a full development life cycle:
- Scratch orgs: These are special types of orgs that can be created and deleted on demand. Thus it is not necessary to clean an org (think of this as your workspace in the cloud). You can also provide JSON configuration file to set various configurations for the org, such as multi-currency and even namespace (despite your packaging org also using the same).
- Heroku flow: The Heroku command line has been extended with additional facilities, such as the creation of Scratch orgs and the ability to synchronize the contents of your local folder structure with that of an attached org.
Details are limited to what was announced at the annual Salesforce Dreamforce event. The preceding points are aimed at giving you an idea of some of the things they announced and that will start to arrive over the next few years.
Meanwhile the following sub-sections provide you with a means to achieve source control development with today's tools, both from Salesforce and the community. It is also likely that some of the approaches here will still be needed as Salesforce DX continues to mature over several releases.
As mentioned previously, it can rapidly become impractical to have a number of developers within one org. They might accidently overwrite each other's changes, make Custom Object or setup changes that others might not want to see, and consume daily API limits for that org more rapidly, eventually blocking everyone from making changes.
When it comes to the packaging org there is another more important reason, and that is the ability to know exactly what changes have been applied to your application code between one version and the next. This is something you and your customers want to have confidence in being able to know at any given time.
There is also the ability to be able to control the release of features or rollback changes, which becomes harder if those changes have already been made in the packaging org. For these reasons, it is typically a good idea to limit access to your packaging org to those responsible for making releases of your packages, thus treating your packaging org conceptually as a warehouse only.
Tip
Sandbox orgs are often attributed to development of custom solutions within an Enterprise organization, where the sandbox has been initialized with a copy of the production org. This can include data, installed packages, and other customizations. At the time of writing, sandbox orgs are available from Partner Portal-provisioned Enterprise Edition orgs (maximum of five). A sandbox can be used as an alternative to a developer org. One advantage is the ability to configure the parent org to a default blank state and then leverage the sandbox refresh process to effectively clean the org ready for a new build deployment. Note that the upcoming Salesforce DX scratch orgs are similar to sandbox orgs in some of these aspects. Note that if you are using Field Sets, sandbox orgs do not currently allow the definition of Available Fields.
In the Required and recommended organizations section of Chapter 1, Building, Publishing, and Supporting Your Application, I recommended that you create a developer org. If you have not done already, visit the Partner Portal or https://developer.salesforce.com/ and obtain a new developer org (or DE org for short) or Salesforce DX scratch org if you have that facility available to you.
Once you have this, you might want to take some time to further clean out any sample packages, Apex code, or configuration that Salesforce has put in such orgs, as these components will be merged with your own components later in the developer workflow process and you would want to avoid committing them to Source Control, or worse, getting them added to your package. Salesforce DX scratch orgs are empty by design.
Cleaning DE orgs is not something that Salesforce natively supports, yet most developers are familiar with the ability to clean their working environment (their DE orgs in the case of Force.com development) before taking new builds from Source Control. This avoids unwanted classes, objects, and fields building up during the course of development, for example when implementation strategies change or functionality is redefined, causing previously committed components to be renamed or deleted. This can leak back into your code base accidentally. Fortunately, there is an open source solution that leverages the Migration Toolkit to clean a DE org, which we will explore later. Salesforce DX scratch orgs have the same problem, but can more easily be deleted and replaced with a new org.
Tip
Due to the overhead of creating and managing installation of packages. Typically, DE orgs are also used by test engineers and other members of the team to test or preview features being developed. That being said, it can be worth considering the creation of a beta package from a stable build of your code at various intervals throughout your release development cycles, for additional sanity and smoke testing ahead of your final upload process. The uploading and installation of packages can be automated via the Tooling and Metadata APIs.
In this chapter, we are going to extract our application's components from the packaging org as files that we can later upload into a GitHub repository to allow tracking and auditability of changes in a more robust way than is permitted within a Salesforce org. We will then use this repository to populate the DE org created, make some changes, and commit those changes back to the repository. This is known as developer workflow.
There are many options when it comes to choosing an appropriate Source Control system to store your application code within. In this book, we will use GitHub as a remote Source Control store and local Source Control repository.
The aim here is not to prescribe which tools to use but to give a general walk-through of a Force.com developer flow that you can adapt and explore different options within later. The following diagram illustrates the developer workflow that we will be walking through in this chapter:
The following points relate to the preceding diagram:
- Choosing a Source Control product and server is an important decision in which you should consider security, accessibility, and durability. The source code to your application needs to be securely managed for only authorized personnel, which is managed ideally through clear processes in your business when employees arrive and leave. Where you choose to store your Source Control server also adds further importance to this. If for accessibility reasons you choose to use one of the many cloud-based Source Control hosting services or indeed manage your own, for example via Amazon EC2. Perform durability tests from each location, ensuring that the server operates efficiently (developers hate waiting!). Finally, make sure that you have a backup and recovery process that is defined and tested, focusing on minimizing the downtime of your developers should the worst happen. The next section in this chapter will walk through populating a GitHub repository from the contents of our packaging org.
- Populating a local repository with files stored in your Source Control server is the initial activity that most developers will perform. As described previously, this should be as efficient as possible. If you are using a cloud-based server, the amount of bandwidth needed to synchronize the remote and local repository is important. Some Source Control products offer more efficient implementations in this regard than others. Git is especially tailored for this kind of scenario, giving the ability to clone the whole or specific branches of the repository locally, complete with file history, followed by incrementally updating it as desired. Most common Source Control products offer excellent choices for GUI as well as the standard command-line interfaces. In this chapter we will use GitHub's GUI to clone the repository to a local folder ready for the next step in the developer flow.
- The Apache Ant open source tool offers a good OS platform-neutral solution for making a standard build (deploy) script and other custom development tools you might want to build.
- In its most basic form, an Ant build script simply leverages the Salesforce Migration Tool Ant task,
sf:deploy, to deploy the contents of the local repository to a designated developer org. However, as we will discuss in a later section, it also forms a part of your Continuous Integration solution. - Once deployed to a developer org, the tools used to develop on Force.com range from using combinations of the Setup menu, Schema Builder (often overlooked), Developer Console, Force.com IDE (Eclipse) to other third-party IDEs that have emerged.
- The key requirement is that once the developer is ready to submit to Source Control, there is an ability to synchronize the component files stored locally with the contents of the developer org. Most desktop based IDEs provide a means to do this, though you can also leverage an Ant build script via the
sf:retrieveAnt Task supplied with the Migration Toolkit. I will cover how I do this later in this chapter. - Once files have been synchronized from the developer org, the developer can use standard Source Control commit processes to review their changes and commit changes to the Source Control server (Git requires developers to commit to their local repository first). We will use the GitHub GUI for this. Later in this chapter, we will see how this action can be used to automatically trigger a build that confirms that the integration of the developer's changes with the current state of Source Control has not broken anything, by automatically executing all Apex unit tests.
Tip
More advanced developer flows can also utilize Continuous Integration to pre-validate commits before ultimately allowing developer changes (made to so-called feature branches) to be propagated or merged with the main branch. One such advanced flow is known as GitFlow, describing a staged processing for code flowing from developer/feature branches to release branches. Search via Google for this term to read about this more advanced developer flow.
Note that it is only after step 4 that a developer can actually start being productive, so it's worth ensuring that you check regularly with your developers that this part of the developer flow is working well for them. Likewise, step 5 is where most of their time is spent. Try to allow as much freedom for developers as possible to choose the tools that make each of them the most efficient but still work within the boundaries of your agreed development flow.
Tip
Debugging still remains one of the most time-consuming aspects of developing on Force.com. Make sure that all developers are fully up-to-date with all the latest tools and approaches from Salesforce as well as your chosen IDE environment. Making effective use of the debug log filters can also save a lot of time sorting through a large debug log, as debug logs are capped at 2 MB. Often, developers resort to System.debug to gain the insight they need. Do ensure that these are removed before Security Review. Though genuine supporting debug entries can be discussed and permitted (it is recommended that you utilize the appropriate debug levels), ad hoc debug entries should be removed as a part of general practice anyway. Salesforce's interactive Apex Debugger is available through the Force.com IDE leveraging the Eclipse debugger UI. This can be enabled through conversations with Salesforce on certain orgs.