Lightning App Builder

The Lightning App Builder is a way to create UIs by assembling Lightning Components. From a metadata point of view, the final result of using the App Builder is a metadata item of type Flexipage. This is an XML representation of the page that includes the page layout, the components on the page, component properties, and some other metadata such as platform actions available for that page. This metadata is stored in a single file and is easy to version and deploy.

Community Builder

Community Builder is similar to Lightning App Builder in that it is used to create and configure user interfaces with drag-and-drop elements. But it adds considerable sophistication in that it can be used to create entire Communities—multipage web sites—and to define the overarching structure of the site as well as individual pages. Community Builder is a massive area of growth and focus for Salesforce, and they’re working on a unified “Experience” model that can also encompass the web site builders associated with Marketing Cloud and Commerce Cloud.

One massive disadvantage of Community pages is that until recently they didn’t have human-readable metadata contents. That’s changing with the release of the ExperienceBundle metadata type, although that’s still in Developer Preview as of this writing. In marked contrast to most Salesforce metadata, ExperienceBundles are stored as JSON files. Salesforce began in 1998 and is mostly written in Java, the most promising programming language from that time. Salesforce also relies heavily on the data storage and exchange formats which were popular at that time, XML and SOAP, which are very well supported by Java. JSON is a newer and simpler standard that has grown in popularity alongside JavaScript, since it allows for smaller and more readable files, which can be parsed natively by JavaScript.

Although it’s currently possible to deploy entire communities using the Site, Network, and SiteDotCom metadata types, the Site is stored as a binary file which changes each time it’s retrieved, and so isn’t suitable for version control or continuous delivery. As ExperienceBundles become widely adopted, it should become possible to selectively deploy aspects of a Community without redeploying the entire thing. Until then, teams must either deploy the entire Site or manually migrate Community configuration from one Salesforce org to another.

Process and Flow Builders

Process Builder is used to create business logic that is triggered by an event such as a record update. Processes are necessarily linear sequences of steps that are performed one after another, although there are an increasing number of available options.

Flow Builder is used to create more complex logic and can also be used to create user interfaces and to define data flow across screens. Processes are built on the same underlying technology as Flows, and they are both retrieved using the metadata type Flow.

For many releases, managing Flow metadata also required managing metadata for FlowDefinition. Each Flow or Process version was stored as a separate Flow file, and a corresponding FlowDefinition file specified which version of the flow was active. Since Winter ’19, only the active Flow version is retrieved, FlowDefinitions are not needed, and Flows can be deployed as active under certain circumstances. Since Flow versions update the same filename, you can keep a clear history of changes in version control.

Flows and Processes are one of a small number of metadata types where Salesforce manages version numbers internally and allows versions to be updated or rolled back from the Setup UI.

This is a very powerful capability for many reasons. First, Flows are asynchronous and may be waiting for user input for many minutes. Flows store the user’s state in a FlowInterview object, along with which version of the Flow is being used. Even if a different version is activated while a user is still progressing through that Flow, they will be able to complete the process using the same Flow version they started with.

Second, this allows for versions of a Flow to be deployed without being immediately released. As explained in Chapter 10: Releasing to Users, the ability to deploy without releasing is a key capability that enables continuous delivery. While Flow versions do not allow you to release to only a subset of users, they certainly enable you to roll out or roll back a flow version on your own schedule, regardless of when they were deployed to an org.

Finally, Flow versions provide a form of declarative version control. The use of traditional version control is still far from ubiquitous on Salesforce. Declarative developers building Processes and Flows are grappling with complex logic and UIs that might take days or weeks to build. As with code-based development, it’s invaluable to be able to take a snapshot of your work and begin experimenting with a new approach, confident that you can always roll back to a previous version. By dispensing with FlowDefinition metadata, Salesforce is making it easier to manage Flows using traditional version control like Git while still preserving the declarative versioning of Flows.

There’s one final sense in which Flows and Processes are unique among declarative metadata. Salesforce has begun to enforce code coverage requirements on these, just as they do with Apex code. As we’ll discuss in Chapter 8: Quality and Testing, Apex tests are not limited to only validating Apex code. They can validate any form of backend business logic in Salesforce, including validation and workflow rules, Processes, and autolaunched Flows (those which don’t require a UI).

The only case when code coverage is required for Processes and Flows is when they are deployed as Active. By default, they are deployed in a Draft status and have to be manually activated. You can enable the ability to deploy these as active from Setup ➤ Process Automation Settings ➤ Deploy processes and flows as active. If you choose this option, then you must ensure that you have Apex tests that exercise these flows. For example, if a Process is triggered any time you create or update an Opportunity with a particular record type, you must have an Apex test that performs such an action.

This is a fairly soft requirement, in that not every logical execution scenario has to be tested. So it doesn’t require tests that are as detailed as those which test Apex classes or triggers, where 75% of all the lines have to be covered. But you have to at least touch those Flows and Processes.

Managing the Schema

“Schema” is a fancy name for the data structures in a relational database. In the simplest terms, it defines what data you want to store and how that data is related to other data. In a traditional database, you define spreadsheet-like tables to hold each category of data, in which each “row” is a record, each “column” is a field, and some special columns can be “lookup relationships” to rows in other tables. In Salesforce, tables are referred to as “Objects” (like the Account Object) because the definition of that object includes far more than just the data structure.

Creating a new schema object or field is typically done using multistep wizards, and this is often the first configuration activity that Salesforce admins and devs will learn. For the impatient, Salesforce provides a Schema Builder that allows you to create objects and fields quickly while bypassing nonessential steps in the wizard.

Although this process is relatively quick and easy, it can have big impacts on the rest of the org. UI, reporting, and business logic are built around displaying, analyzing, and manipulating data, and so the schema is fundamental to most other customizations.

When people first begin to set up version control and continuous delivery for Salesforce, it’s tempting to only track code. Code is certainly the most important candidate for version control, but code in Salesforce is largely built for retrieving and manipulating database objects, and so without tracking the schema you don’t have a complete picture stored in version control. If you attempt to automatically deploy code to later environments, your deployments will fail unless the necessary objects and fields are already in place.

For these reasons, you should manage the schema in your testing and production environments using version control and a continuous delivery process. You should not allow any manual changes to the schema in those environments, unless you have a high tolerance for confusion and inefficiency. You can and should modify objects and fields in a development environment and deploy from there along with the rest of your configuration.

As mentioned, there’s a lot more to Salesforce objects than just a data table. This is reflected in the object metadata format, which includes fields, validation rules, compact layouts, and record types among other components. It’s for this reason that critical objects in your data model like Accounts can easily have an object metadata file that grows to tens or hundreds of thousands of lines. Version control tools like Git excel at managing and merging line-by-line changes to code files. But the default diff tool included with Git gets extremely confused when comparing repetitive blocks of XML metadata, making such large files very hard for teams to manage and collaborate on.

The Salesforce DX source format “decomposes” each object file into a folder, with files for each child component organized into subfolders by metadata type. Breaking large XML metadata into many files makes it straightforward for teams to manage and deploy the schema collaboratively.

Changing the Schema

Most schema changes lend themselves well to being propagated from dev to test to production. There is, however, one caveat and two big exceptions. The caveat is that some seemingly innocent schema changes can cause data loss or other problems. The two exceptions where schema changes can’t (or shouldn’t) be propagated automatically are when changing the object/field names and when changing field types.

Bulk Database Operations

In addition to field type changes, there may be other bulk database operations you need to perform. Many of my colleagues at Appirio specialized in transforming and migrating data from legacy systems into Salesforce, and for this there are specialized tools and skills you might need. The Salesforce data management applications Odaseva and OwnBackup both provide a wealth of tools that you might use to perform large-scale data operations. The free Salesforce Data Loader and MuleSoft’s dataloader.io are excellent no-frills options to help with this process, especially in combination with clever use of spreadsheets for data transformations.

As mentioned earlier, the use of anonymous Apex and batch Apex are both excellent methods to perform bulk database operations, although you should approach this with due caution. Anonymous Apex allows you to build an Apex script so you can query, transform, and create/update/delete data as needed. Such scripts should generally be stored in version control (so you don’t forget what you did), but are generally run through the developer console or the Salesforce CLI. Test such scripts very carefully. Anonymous Apex allows you to transform up to 10,000 records at a time (the DML limit). Combined with a query that is smart enough to know which records have not been processed, you can script or manually trigger iterations of this code until the job is done.

Batch Apex handles the iterations for you and is designed to process large volumes of data over time. Batch Apex is an Apex class that uses the Database.Batchable interface to define start, execute, and finish methods that iterate across your database 200 records at a time. This can be used either for one-time or ongoing processing as needed.

Data Backup

Unless you’re unusually paranoid, you don’t need to back up your data out of fear that Salesforce will lose it. They have excellent redundant backup systems that distribute the backup across multiple data centers. What you need to worry about is the actions of your users and yourself. Remember those bulk database operations I just mentioned? Remember how I said you should test them extremely well? I think you’re getting the picture.

While unlikely, you could be subject to malicious users or hackers who intentionally delete data. More likely, unwitting users or admins might make sweeping changes that cause data loss or corruption. And it’s also possible for developers to introduce batch Apex, processes, triggers, workflows, or integrations that cause sweeping and undesired changes.

For all of these reasons, data backup is important. Your free option is the monthly export service accessible in Setup ➤ Data ➤ Data Export. That will generate a one-time or scheduled downloadable backup. You should practice and ensure you have the skills on hand to restore some or all of that data in an emergency. Your concierge options are data management services such as AutoRABIT Vault, OwnBackup, or Odaseva who can provide data backup and recovery tools and services.

Configuration Data Management

One of the weakest links in the continuous delivery systems I have built and seen for Salesforce is the management of configuration data. Salesforce release managers and release management tools generally specialize in migrating configuration metadata. The Salesforce Metadata API provides a mechanism for retrieving and deploying this metadata. While there are some complexities in this process, Salesforce actually takes care of most of the complexity for you. When you do a deployment, Salesforce checks the validity of your code and configuration, ensures referential integrity and idempotency, assesses differences, sequences the metadata loads, and more. There’s an enormous amount of intelligence being leveraged behind the scenes.

If you need to migrate complex configuration data, you will need to address those problems of data sequencing and referential integrity on your own. The tools and techniques for this are unfamiliar to most Salesforce release managers and so have been an afterthought for many teams.

Unfortunately, large managed packages such as Rootstock ERP, nCino, Vlocity, FinancialForce, and Salesforce CPQ require extremely complex configuration data to operate. If you use any of these packages, it’s important for your team to establish tools and expertise to facilitate migrating this data, so that you can use your sandboxes to develop this configuration as well. The reason to manage configuration data in your release management process is twofold: first, that configuration data can determine logical pathways that actually affect the behavior of your applications, and second it allows you to develop and test changes to this data before migrating it to production.

This whole process is clearly complicated, which is why I believe this is still a nascent idea among most Salesforce development teams. But the same factors that have driven teams to seek a deployment workflow for metadata will increasingly apply to data in complex orgs. Configuration data changes can be too risky to make directly in production orgs, so they should first be done in development orgs. Those changes sometimes need to be complex, spanning multiple objects, which leads to the need to manage and migrate complex data. This is tedious and error prone for an individual to migrate; therefore an automated process is important. Therefore building this capability is important to enable our teams to not just configure but to configure safely and efficiently using a migration process to separate development from deployment.

Infrastructure Security

Infrastructure security refers to securing the underlying servers, networks, and data stores for an organization. This is a massive area of concern for most IT teams, a massive area of risk and liability for most companies, and one of the important areas that Salesforce handles for you. http://trust.salesforce.com can provide you plenty of information on Salesforce’s infrastructure security protocols, certifications, and track record, but needless to say, they do an excellent job of handling this for you.

Login and Identity

Login and identity is handled by Salesforce through a variety of mechanisms. Salesforce provides a wide range of Identity and Access Management (IAM) options such as single sign-on (SSO), various types of OAuth, API access, multifactor authentication (MFA), and so on. Whereas infrastructure security is opaque to Salesforce customers, IAM is highly configurable on Salesforce, and customers take joint responsibility to ensure that this is handled correctly. Salesforce does allow you to track and deploy certain aspects of your IAM configuration between environments, and it can be helpful to manage this in your code repository along with other types of org metadata.

Some aspects of login and identity security are also handled by Profiles, and Login Flows can also affect this. Profiles can be used to specify login IP ranges and times on a per-user basis. Login Flows provide sophisticated authentication options such as calling out to third-party authentication services and assigning temporary session-based permissions.

For teams that are just getting started tracking their Salesforce code and configuration in version control, it may seem like overkill to track this kind of metadata in the repository. In fact in some cases, this metadata needs to vary slightly for different environments. For example, Connected Apps in sandboxes might have different endpoints that point to sandbox versions of other systems. But it is precisely because this configuration is so important that it makes sense to track and deploy it from version control. Many changes to login and identity should be tested in a sandbox first, and in any case, it’s extremely helpful to see a history of changes. An errant admin could readily break integrations, but having version control provides you the ability to roll back and monitor changes. Automating the deployment of some of these metadata types requires that you have the ability to dynamically substitute values as part of your deployment process. This is a native capability of tools like Copado, but we provide an overview of how you can build this yourself in “Managing Org Differences” in Chapter 9: Deploying.

Admin Access

Admin access refers to ensuring that only the appropriate people have permission to make certain changes in your org. Remember that everything in Salesforce is defined by configuration and that even custom code on Salesforce is a form of configuration. You can think of admin access as configuration security, or “what you can/can’t do in the Salesforce Setup UI.”

Salesforce provides a built-in System Administrator profile which possesses “God Mode” privileges in your org. Needless to say, you should be very selective in who is given this profile. Most users would be assigned the out-of-the-box “Standard User” profile, which typically provides the ability to use Salesforce and see data, but not to change configuration.

Admin access and user access are the two areas where admins have the most ability to tune Salesforce to meet their organizational needs. Most organizations will eventually find the need to make a copy of the System Administrator profile and the Standard User profile and begin to customize these according to the needs of their organization.

The traditional method of providing access to control configuration has been through the use of Profiles. But as mentioned earlier, Profiles are a nightmare to manage in version control and typically don’t provide the right level of granularity that teams need. Permission Sets should be your preferred method to manage permissions. You should keep profile permissions extremely thin and instead use permission sets to determine both “admin”- and “user”-level access. If possible, limit your use of profiles to defining things like login IP ranges which can’t be defined in Permission Sets. More details on this are provided in Chapter 11: Keeping the Lights On.

User Access

Whereas admin access determines who can do what in the Salesforce Setup UI, user access refers to what end users can access or modify inside the Salesforce application itself. Data security defines which users have create, read, update, or delete (CRUD) permissions on Salesforce data objects. User access also defines who can use particular Apex classes, applications, Visualforce pages, and standard Salesforce applications like Service Cloud and Knowledge. Permissions in Salesforce are always additive, meaning that the default is for users to have no permissions, but they gain permissions as a result of their profile, user settings, permission sets, and permission set licenses.

As mentioned, all of the admin and user access privileges are defined using profiles or (preferably) permission sets. Permission sets can and should be tracked in version control and deployed as part of your release management process. This provides history tracking on changes to these and ensures that permissions can be tested thoroughly before being deployed to production.

One of the most common failure modes for custom Salesforce applications is for developers to not deploy appropriate permissions along with other metadata. Developers can and should have System Administrator privileges in their own development environment. Such unobstructed access allows them to build capabilities without restriction. But it also makes it easy for developers to forget that end users will need explicit permissions to use what the developer has built.

Developers and testers should use the “Login As” capability to log in to their development and testing environments as one of the target users to ensure those users will have access. If a developer creates a custom application, a new object, with new fields, a tab for that object, a Visualforce page, and an Apex controller for that page, users will need permissions for each of those components. Thus as the complexity of an org increases, so too does the complexity of user permissions.

If you use profiles to manage permissions and you want to give all profiles access to this custom application, then you will need to assign those permissions to every profile. Permission sets thus help you follow the DRY maxim, “don’t repeat yourself.” You can define permissions for this new custom application in a single permission set and then assign that permission set to all users in the org.

Profiles and Permission Sets are both metadata and can be deployed between environments. But which users are assigned those profiles and permission sets cannot be deployed in the same way. This means that when deploying a new permission set to production, you will either need to manually assign this to all users or preferably write a bit of anonymous Apex to query all User IDs and create PermissionSetAssignment records for each user to assign the new permission set to them.

Salesforce is working on the concept of “Permission Set Groups,” which is currently in Pilot. This provides a much more elegant alternative. You attach multiple permission sets to a permission set group and then assign a large group of users to that single Permission Set Group. Permission set groups provide a simple way to bulk assign permissions consistently to a group of users. They can be updated by adding or removing permission sets to a group, and those permission sets can be used across multiple permission set groups.

Permission Set Groups are not accessible via the Metadata API in the pilot, and so you would still need to add any new permission sets to the group manually. If this is eventually made accessible through the Metadata API, then developers will have the ability to create new permission sets in a development environment, add them to the appropriate permission set group, and deploy those permissions all the way to users in production.

A new type of Permission Set called a “Muting Permission Set” is also in pilot. Muting Permission Sets can be added to a Permission Set Group and provide the ability to “Mute” permissions. To my knowledge, this is the first example of Salesforce enabling a negative permission on the platform. The explicit purpose of this is to inhibit permissions that would otherwise be given to members of the group by other Permission Sets in that same group. It might be possible to use this for security purposes to ensure that members of the group are never given powerful admin privileges such as “Modify All Data,” but this use case is not mentioned in the documentation.

Apex

Apex is a strongly typed language, similar to Java, that provides native access to some underlying Salesforce capabilities. The most obvious native capability in Apex is the ability to work with Salesforce data as first-class Objects. So, for example, you can perform a query on Accounts and then perform actions on that data without having to explicitly define an “Account” object. Although Apex compiles down to Java behind the scenes, its most notable limitation is that you can’t use third-party Java libraries or some advanced features of that language.

Salesforce runs Apex on the core platform, and Apex can’t be compiled or run outside of a Salesforce org. The limitations on the language are largely to ensure that it can run safely in a multitenant environment without causing excess load or accessing things that would compromise the security of the platform.

There are actually two kinds of Apex metadata: triggers and classes. Triggers are a concept borrowed from other relational databases and allow custom code to be run as part of a database transaction (insert, update, delete, or undelete). Apex classes are more flexible than triggers, since they allow the use of methods, interfaces, global variables, and so on. Triggers were the first form of custom code allowed on the platform and have a very strictly defined format that does not allow the use of methods or some other capabilities of Apex classes.

Salesforce executes Apex in the context of a transaction. To ensure that no single transaction goes out of control and soaks up excessive system resources, Salesforce strictly enforces governor limits on heap size, CPU time, number of SOQL and DML statements, and so on. These governor limits are uncomfortable for developers accustomed to running code on their own servers, but are a design constraint that encourages code to be performant and balances freedom with the needs of other users on the platform.

Each Apex class or trigger runs using a specified version of the Salesforce API. This wise design decision allows Salesforce to make breaking changes in a new version of Apex without impacting code that was written previously. When downloaded, this API version is shown in a “sidecar file” that has the same name as the class or trigger with an added -meta.xml suffix. As a best practice, classes and triggers should periodically be reviewed and upgraded to the latest API version to ensure optimal runtime performance. In most cases, updating the API version is a trivial change, but it’s possible to face compilation errors, and this is a good reason to write good quality Apex unit tests to ensure behavior remains unchanged even if you update the API version.

It is essential to avoid hardcoding IDs in Apex code or anywhere else. Record IDs are typically different in each environment. Instead, ensure your logic can dynamically identify the proper data to operate against and not fail.

Visualforce

Visualforce allows you to create completely custom user interfaces, using an HTML-like syntax to display information and invite user input and actions. Visualforce also excels in giving access to Salesforce’s internal data model through the use of Standard Controllers or Controller Extensions. A Visualforce page that uses a Standard Controller for Account allows you to create, access, and update Accounts without any other backend code. This page can override the standard record detail or record edit page in Salesforce, thus providing an alternative user interface for working with any Salesforce object.

There are many other capabilities of Visualforce, but it’s fallen out of favor since it’s relatively slow compared to Lightning-based pages. Visualforce pages have to be rendered on Salesforce, and their state is transferred back and forth from the browser each time you perform an action or move to a new page. There are also strict limitations to the size of this Viewstate that make them unsuitable for editing large amounts of data.

Scripting and Anonymous Apex

There are other ways to execute Apex such as creating custom REST and SOAP services. But one form of Apex worth mentioning here is “anonymous Apex,” which is not stored on the server but can nevertheless be run on demand.

Among the many uses of anonymous Apex is the ability to automate predeployment and postdeployment steps in the process of building and delivering an application. Anonymous Apex has the ability to query, transform, and update data, and this includes system-level data such as permission set assignments.

As mentioned earlier, governor limits make anonymous Apex unsuitable to perform massive or long-running data transformations such as migrating data between fields on more than 10,000 records at a time. For this purpose, you can use batch Apex. But there are many practical one-time activities that you can perform using anonymous Apex, such as creating or removing scheduled jobs, triggering batch Apex to run, modifying User Role assignments, and so forth.

Although anonymous Apex is not persisted inside of Salesforce, you can and should save such scripts in your code repositories. This allows for reuse and also provides a clear audit trail should you or someone else need to review the changes that were made.

Lightning Web Components

Most people who were using the Internet in the late 1990s and early 2000s were aware of the “Browser Wars,” when Internet Explorer competed with Firefox and eventually Chrome for market dominance. But only web developers are familiar with the “Framework Wars” that pitted Angular against React and a hundred other frameworks to provide a more robust way to build applications on the Web.

Web Components are a standards-based alternative to custom JavaScript frameworks that are natively supported by modern web browsers. Unlike custom frameworks like Angular and React, Web Components provide native execution that allows them to run quickly and a consistent syntax that allows developers to reuse their skills more easily. Salesforce was a participant in developing the open standard for Web Components, and Lightning Web Components are an implementation of the standard that is optimized for working with Salesforce.

Lightning Web Components were announced in late 2018, but Salesforce had been quietly rewriting their entire Lightning user interface in LWC for a year or more. This enabled much faster performance and gave Salesforce the confidence that it was possible for customers to mix Lightning Components and Lightning Web Components together on the same page. Salesforce themselves had been refactoring the application in that way for over a year!

Lightning Aura Components

The original Lightning Component framework was based on an open source project called Aura, inspired by AngularJS. Like other code built on custom frameworks, Lightning Aura Components require the browser to do more work, since they have to be compiled into native JavaScript, adding significant amounts of execution overhead.

The vision for Lightning Aura Components is powerful and inspiring, since it allows organizations to build custom UI components using the same technology used to create Salesforce’s own UI. Salesforce Classic is a fundamentally different technology than Visualforce. And Visualforce pages actually run in a separate domain to ensure transaction security and prevent custom Visualforce code from scraping data that it should not have access to.

By creating Lightning Aura Components, Salesforce opened the door to a long-term vision in which developers could seamlessly mix custom components with built-in components, even down to overriding a single field. That vision has not yet been fully realized, and Salesforce is still making incremental improvements to balance flexibility with security. But the Salesforce UI is far more responsive and performant today than before Lightning was rolled out.

JavaScript Remoting, S-Controls, and API Access

There are other client-side coding options on Salesforce that predate Lightning Components. Three official options are JavaScript Remoting, Visualforce Remote Objects, and S-controls. JavaScript Remoting is a technique that allows Visualforce pages to host complex JavaScript applications that can store state on the client side while still sending and receiving data from Salesforce using an Apex controller. Visualforce Remote Objects allow you to create a JavaScript representation of Salesforce objects so you can create, retrieve, and update Salesforce data using client-side JavaScript. S-controls were deprecated many years ago, but were Salesforce’s first foray into allowing custom coding. They allow you to create custom HTML and JavaScript in an iFrame, which can access Salesforce data using the standard Salesforce API.

Salesforce’s API is also what enables a variety of other third-party custom code solutions such as Restforce for Ruby and Simple Salesforce for Python. These prebuilt libraries provide convenient wrappers around the REST API that allow teams familiar with those languages to work with Salesforce data and trigger Salesforce operations. The most significant such library is JSForce for JavaScript. JSForce is now the engine used by the Salesforce CLI to communicate with Salesforce and is also at the heart of many other JavaScript, TypeScript, and Node.js libraries for Salesforce.