Asynchronous execution contexts

Salesforce is committed to ensuring that the interactive user experience (browser or mobile response times) of your applications and that of its own is as optimal as possible. In a multitenant environment, it uses the governors as one way to manage this. The other approach is to provide a means for you to move the processing of code from the foreground (interactive) into the background (async mode). This section of the chapter discusses the design considerations, implementation options, and benefits available in this context.

As the code running in the background is, by definition, not holding up the response times of the pages the users are using, Salesforce can and does throttle when and how often async processes execute, depending on the load on the servers at the time. For this reason, it currently does not provide an SLA on exactly when an async piece of code will run or guarantee exactly when the Apex Schedule code will run at the allotted time. This aspect of the platform can at times be a source of frustration for end users, waiting for critical business processes to complete. However, there are some design, implementation, user interface, and messaging considerations that can make a big difference in terms of managing their expectations.

The good news is that the platform rewards you for running your code in async by giving you increased governors in areas such as Heap, SOQL queries, and CPU governor. You can find the full details in their documentation (attempting to document them in a book is a nonstarter as they are constantly being improved and removed in some cases).

Understanding Execut ion Governors and Limits: http://www.salesforce.com/us/developer/docs/apexcode/Content/apex_gov_limits.htm.

General async design considerations

The platform provides some straightforward ways in which you can execute code in async. What is often overlooked though is error handling and general user experience considerations. This section presents some points to consider functionally when moving a task or process into async or the background to use a more end user-facing term:

  • Type of background work: I categorize work performed in the background in two ways, application and end user:
    • Application is something that the application requires to support future end user tasks, such as periodic recalculation, major data manipulation, or the on boarding of data from an external service, for example.
    • End user is something that is related (typically initiated) to an end user activity that just happens to require more platform resources than are available in the interactive (or foreground) context. The latter can be much harder to accept for business analysts and end users, who are constantly demanding real-time response times. If you do find that this is your only route, focus hard on the user interaction and messaging aspects as follows.
  • User experience: A well-crafted user experience around the execution of background work can make all the difference when the platform is choosing to queue or slow the processing of work. Here are some considerations to keep in mind:
    • Starting the background work: Decide how the user starts the background process. Is it explicitly, via a button or indirectly, via a scheduled job or as a side effect of performing another task? Starting the background work explicitly from a user interface button gives you the opportunity to set expectations. A critical one being whether the work is queued or it has actually started; if they didn't know this, they might be inclined to attempt to start the work again, thus resulting in potentially two jobs processing in parallel! If you decide to start the process indirectly, make sure that you focus on error handling and recovery to ensure that the user has a way of being notified of any important failures at a later point in time.
    • Progress bar indicators: These can be used to allow the user to monitor the progress of the job from start to finish. Providing messages during the progress of the work can also help the user understand what is going on.
    • Monitoring: Ensuring that the user can continue to monitor the progress between sessions or other tasks can also be important. Often, users might close tabs or get logged out and so consider providing a means to revisit the place where they started the job and have the user interface recognize that the job is still being processed, allowing the user to resume monitoring of the job.
  • Messaging and Logging: Think about the ways in which your background process communicates and notifies the end user of the work it has done and any errors that occurred. While e-mails can be the default choice, consider how such e-mails can get lost easily, look for alternative notification and logging approaches that are more aligned with the platform and the application environment. Here are some considerations to keep in mind:
    • Logging: Ideally, persist log information within the application through an associated log (such as a logging Custom Object) with other Custom Objects that are associated with records being processed by the job, such that the end users can leverage related lists to see log entries contextually.
    • Unhandled exceptions: They are caught by the platform and displayed on the Apex Jobs page. The problem with this page is that it's not very user-friendly and also lacks the context as to what the job was actually doing with which records (the Apex class name might not be enough to determine this alone). Try to handle exceptions, routing them via your end user-facing messaging. Some Apex exceptions cannot be handled and will still require admins or support consultants to check for these. Your interactive monitoring solutions can also query AsyncApexJob for such messages.
    • Notifications: Notifications that a job has completed or failed with an error can be sent via an e-mail; however, also consider creating a child record as part of the logging alternative. A user-related task or a chatter post can also work quite well. The benefit of options that leverage information stored in the org is that these can also be a place to display Custom Buttons to restart the process.
  • Concurrency: Consider what would happen if two or more users attempted to run your jobs at the same time or if the data being processed overlaps between two jobs. Note that this scenario can also occur when scheduled jobs execute. You might want to store the Apex job ID during the processing of the job against an appropriate record to effectively lock it from being processed by other jobs until it is cleared.
  • Error recovery: Consider whether the user needs to rerun the job processing that has only failed records or whether the job can simply be designed in a way which is re-entrant, which means that it checks whether executions of a previous job might have already been made over the same data and cleans up or resets as it goes along.

Running Apex in the asynchronous mode

There are three ways to implement the asynchronous code in Force.com; all run with the same default extended governors compared to the interactive execution context.

As stated previously, there is no guarantee when the code will be executed. Salesforce monitors the frequency and length of each execution, adjusting a queue that takes into consideration performance over the entire instance and not just the subscriber org. It also caps the number of async jobs over a rolling 24-hour period. See the Understanding Execution Governors and Limits section for more details. Salesforce provides a detailed white paper describing the queuing.

Asynchronous Processing in Force.com: https://developer.salesforce.com/page/Asynchronous_Processing_in_Force_com.

Tip

Apex Scheduler can be used to execute code in an async context via its execute method. This also provides enhanced governors. Depending on the processing involved, you might choose to start a Batch Apex job from the execute method. Otherwise, if you want a one-off, scheduled execution, you can use the Database.scheduleBatch method.

@future

Consider the following implementation guidelines when utilizing @future jobs:

  • Avoid flooding the async queue with many @future jobs (consider Batch Apex).
  • Methods must execute quickly or risk platform throttling, which will delay the execution.
  • It is easier to implement as they require only a method annotation to be added.
  • Typically, the time taken to dequeue and execute the work is less than Batch Apex.
  • You can express, via a method annotation, an increase in a specific governor to customize the governor limits to your needs.
  • Parameter types are limited to simple types and lists.
  • Be careful when passing in list parameters by ensuring that you volume test with large lists.
  • There is no way to track execution as the Apex Job ID is not returned.
  • Executing from an Apex Trigger should be considered carefully; @future methods should be bulkified, and if there is a possibility of bulk data loading, there is a greater chance of hitting the daily per user limit for @future jobs.
  • You cannot start another @future, Batch Apex or Queueables job. This is particularly an issue if your packaged code can be invoked by code (such as that written by developer X) in a subscriber org. If they have their own @future methods indirectly invoking your use of this feature, an exception will occur. As such, try to avoid using these features from a packaged Apex Trigger context.

Tip

I would recommend Queueables over @future since they have additional flexibility in terms of the type of information that can be passed to them and they are more easily tracked once submitted.

Queueables

Consider the following implementation guidelines when utilizing Queueables:

  • Avoid flooding the async queue with many jobs (consider Batch Apex).
  • Methods must execute quickly or risk platform throttling, which will delay the execution.
  • Implement the interface Queueables and the execute method. Then, use the System.enqueueJob method to start your job:
    public void execute(QueueableContext context)
  • The object implementing Queueables can contain serializable state that represents the work to be undertaken. This can be complex lists, maps, or your own Apex types.
  • Typically, the time taken to dequeue and execute the work is less than Batch Apex.
  • Be careful while processing lists, ensuring that you volume test with large volumes of lists to ensure that the job can complete with maximum data.
  • You can track execution as the Apex Job ID is returned.
  • Executing from an Apex Trigger should be considered carefully. Use of Queueables should be bulkified, and if there is a possibility of bulk data loading, there is a greater chance of hitting the daily per user limit for @future jobs.
  • You can start up to 50 queueables jobs in a sync context.
  • You can start only one queueable job within an existing queueable job; this is known as chaining. The depth of the chain is unlimited, though the platform will slow the gaps between jobs as the depth grows. Also Developer Edition orgs are limited to a depth of 5.
  • If code attempts to issue more than 1 queueable job within an existing queueable context, an error occurs. This can be particularly an issue if your packaged code can be invoked by code (such as that written by developer X) in a subscriber org. If they have their own async methods indirectly invoking your use of this feature, an exception will occur. As such, try to avoid using these features from a packaged Apex Trigger context.
  • It is possible to run more queueables jobs concurrently than Batch Apex. The exact limit is not documented. However be sure to avoid flooding the system, by putting in place logic that controls how chaining and concurrency are used.

Batch Apex

In our scenario, the Race Data records loaded into Salesforce do not by default associate themselves with the applicable Contestant records. As noted earlier, the Contestant record has a Race Data Id field that contains an Apex-calculated unique value made up from the Year, Race Name, and Driver Id fields concatenated together.

The following Batch Apex job is designed to process all the Race Data records, calculate the compound Contestant Race Data Id field, and update the records with the associated Contestant record relationship. The following example highlights a number of other things in terms of the Selector and Service layer patterns usage within this context:

  • The RaceDataSelector class is used to provide the QueryLocator method needed, keeping the SOQL logic (albeit simple in this case) encapsulated in the Selector layer.
  • The RaceService.processData method (added to support this job) is passed in the IDs of the records rather than the records passed to the execute method in the scope parameter. This general approach avoids any concern over the record data being old, as Batch Apex serialized all record data at the start of the job.
  • As per the standard practice, the Service layer does not handle exceptions; these are thrown to the caller and it handles them appropriately. In this case, a new LogService class is used to capture exceptions thrown and log them to a Custom Object, utilizing the Apex job ID as a means to differentiate log entries from others jobs. Note that the use of LogService is for illustration only; it is not implemented in the sample code for this chapter.
  • In the finish method, new NotificationService encapsulates logic, which is able to notify end users (perhaps via e-mail, tasks, or Chatter as considered earlier) of the result of the job, passing on the Apex job ID, such that it can leverage the information stored in the log Custom Object and/or ApexAsyncJob to form an appropriate message to the user. Again, this is for illustration only.

The following code shows the implementation of the Batch Apex job to process the race data records:

public class ProcessRaceDataJob implements Database.Batchable<SObject> {
  public Database.QueryLocator start(Database.BatchableContext ctx) {
    return 
      RaceDataSelector.newInstance().selectAllQueryLocator();
  }

  public void execute(Database.BatchableContext ctx, List<RaceData__c> scope) {
    try {
      Set<Id> raceDataIds = new Map<Id, SObject>(scope).keySet();
      RaceService.processData(raceDataIds);
    }
    catch (Exception e) {
      LogService.log(ctx.getJobId(), e);
    }
  }

  public void finish(Database.BatchableContext ctx) { 
    NotificationService.notify(
      ctx.getJobId(), 'Process Race Data'),
  }
}

Note that even the code to start the Apex job is implemented in a Service method; this allows for certain pre-checks for concurrency and configuration to be encapsulated. To run this job, a list view Custom Button has been placed on the Race Data object in order to start the process. The controller calls the Service method to start the job as follows:

Id jobId = RaceService.runProcessDataJob();

Consider the following implementation guidelines when utilizing Batch Apex:

  • A maximum of 5 active Apex Jobs can run in the org.
  • Apex Flex Queue (available under Setup) can queue up to 100 jobs. The platform automatically removes items from this queue when jobs complete. You can also reorder items in the queue through the UI or via an API. Jobs in the queue have a special holding status.
  • An exception is thrown if 100 jobs in an org are already enqueued or executing.
  • It is more complex to implement Batch Apex, which requires three methods.
  • Typically, the time taken to dequeue and execute can be several minutes or hours.
  • Typically, the time taken to execute (depending on the number of items) can be hours.
  • Up to 50 million database records or 50 thousand iterator items can be processed.
  • Records or iterator items returned from the start method are cached by the platform and passed back via the execute method as opposed to being the latest records at that time from the database. Depending on how long the job takes to execute, such information might have changed on the database in the meantime.
  • Consider re-reading record data in each execute method for the latest record state.
  • You can track Batch Apex progress via its Apex job ID.
  • Executing from an Apex Trigger is not advised from a bulkification perspective, especially if you are expecting users to utilize bulk data load tools.
  • You can start another Batch Apex job from another (from the finish method).
  • Calibrate the default scope size (the number of records passed by the platform to each execute method call) to the optimum use of governors and throughput.
  • Provide a subscriber org configuration (through a protected Custom Setting or Custom Metadata) to change the default scope size of 200.

Tip

Note that the ability to start a Batch Apex job from another can be a very powerful feature in terms of processing large amounts of data through various stages or implementing a clean-up process. This is often referred to as Batch Apex Chaining. However, keep in mind that there is no guarantee the second job will start successfully, so ensure you have considered error handling and recovery situations carefully.

Performance of Batch Apex jobs

As stated previously, the platform ultimately controls when and how long your job can take. However, there are a few tips that you can use to improve things:

  • Getting off to a quick start: Ensure that your start method returns as quickly as possible by leveraging indexes.
  • Calibrating: Calibrate the scope size of the records passed to the execute method. This will be a balance between the maximum number of records that can be passed to each execute (which is currently 2000) and the governors required to process the records. The preceding example uses 2000 as the operation itself is fairly inexpensive. This means that only 50 chunks are required to execute the job (given the test volume data created in this chapter), which in general, allows the job to execute faster as each burst of activity gets more done at a time. Utilize the Limits class with some debug statements when volume testing to determine how much of the governors your code is using, and if you can afford to do so, increase the scope size applied. Note that if you get this wrong (as in the data complexity in the subscribers orgs is more than you calibrated for your test data), it can be adjusted further in the subscriber org via a protected Custom Setting.
  • Frequency of Batch Apex jobs: If the use of your application results in the submission of more Batch Apex jobs more frequently, perhaps because they are executed from an Apex Trigger (not recommended) or a user interface feature used by many users, your application will quickly flood the org's Batch Apex queue and hit the concurrent limit of 5, leaving no room for your Batch Apex jobs or those from other applications. Consider carefully how often you need to submit a job and whether you can queue work up within your application to be consumed by a Batch Apex Scheduled job at a later time. The Summer'14 Batch Apex Flex Queue will help reduce this problem.

Using external references in Apex DML

The RaceService.processData service method demonstrates a great use of external ID fields when performing DML to associate records. Note that in the following code, it has not been necessary to query the Contestant__c records to determine the applicable record ID to place in the RaceData__c.Contestants__c field. Instead the Contestants__r relationship field is used to store an instance of the Contestant__c object with only the external ID field populated, as it is also unique and it can be used in this way instead of the ID.

public void processData(Set<Id> raceDataIds)
{
  fflib_SObjectUnitOfWork uow =Application.UnitOfWork.newInstance();
  for(RaceData__c raceData : 
       (List<RaceData__c>)
          Application.Selector.selectById(raceDataIds))
  {
    raceData.Contestant__r = 
      new Contestant__c(
        RaceDataId__c = 
          Contestants.makeRaceDataId(
            raceData.Year__c, 
            raceData.RaceName__c, 
            raceData.DriverId__c));
    uow.registerDirty(raceData);
  }
  uow.commitWork();
}

If the lookup by external ID fails during the DML update (performed by the Unit Of Work in the preceding case), a DML exception is thrown, for example:

Foreign key external ID: 2014-suzuka-98 not found for field RaceDataId__c in entity Contestant__c

Tip

This approach of relating records without using or looking up the Salesforce ID explicitly is also available in the Salesforce SOAP and REST APIs as well as through the various Data Loader tools. However, in a data loader scenario, the season, race, and driver ID values will have to be pre-concatenated within the CSVfile.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.118.198.81