This phase of the Salesforce Platform Governance Method tackles the data architecture and management aspects of your project’s Salesforce solution.
This is a more detailed view of the approach your team has taken to construct its application rather than the high-level view taken in Phase A. As such, this phase of governance is more applicable to a project that is more mature in its solution, even if that solution has not yet been built.
Overview
Phase B focuses on the data aspects of your application. It should be governed by team members who specialize in Salesforce’s data model and have an understanding of how data models can be integrated with the Salesforce platform and external systems.
As the governing team, you are looking to see whether the data aspects of the application being proposed by a project within your organization will be fit for the purpose at hand.
The Salesforce Platform Governance Method: Phase B
“Data management (DM) consists of the practices, architectural techniques, and tools for achieving consistent access to and delivery of data across the spectrum of data subject areas and data structure types in the enterprise, to meet the data consumption requirements of all applications and business processes.”
—Gartner1
As previously mentioned, dealing with data is complex. You need to understand the way the data is going to be used by your application so that the correct solution or solutions can be implemented to give your end users the best experience. Any end user sitting around waiting for reports to run or receiving errors that queries have timed out is not going to be satisfied.
It is therefore imperative that your data governance is not left to chance, and that you have properly challenged your projects to think not only about finding a solution to today’s business challenge but also about longer-term challenges.
The Salesforce platform protects you from many potential issues that you might face if you were to develop an application using a traditional programming language and database, mainly by hiding the design burden on the database level, such as tuning, maintenance, and security. However, the platform can only do so much given it does not know how you are going to use the application within your business.
As the team responsible for protecting your company’s Salesforce organization, you will need to explore as much as possible the data design and architecture of your project’s application.
Design & Optimization
Phase B: data architecture & management - design & optimization
Governing the design and optimization of the data architecture is an important aspect of the governance process. Your team will be responsible for making sure that whatever a project produces will fit into the larger usage of the Salesforce platform. You need to be concerned with the overall performance of your Salesforce instance, but also the long-term maintenance and manageability of it.
Govern the data model, paying attention to modeling decisions made, such as the usage of lookup versus master–detail relationships.
Given that there are basically three types of relationship available within the data design space, you will be looking to understand whether the appropriate type has been used and assessing the impact of the choice. For example, an object can only have two master–detail relationships, so you will need to make sure that this relationship is absolutely necessary, especially for standard or enterprise objects.
Additionally, data visibility will need to be reviewed to ensure that the correct outcomes are reached in terms of data access controls. This is important if the child relationship must have different security and access than the parent object.
Recognizing custom objects as duplications of standard objects is important. It is easy to identify custom objects, especially when talking with the business stakeholders. Determining whether the custom object and support functionality could be delivered by a standard configuration on the Salesforce platform will reap numerous cost and maintenance benefits. Your main goal should be to deliver a business benefit as quickly as possible, while reducing the need to create bespoke configurations.
Where possible, avoid the duplication of data, especially important data that is only really useful if it is accurate. For example, a telephone number should be an attribute on an object it relates to, such as a customer, rather than on the order the customer places. That way, every order will refer to the same customer telephone number. These are simple examples, and it is not always easy to recognize these issues at the early design phases. Your governance board should be experienced in these types of issues.
Govern the use of external data (external objects).
External data or objects refer to record data that is stored outside your Salesforce organization and is exposed to your Salesforce organization as a form of custom object. This provides access to record data on demand without needing to have that data stored within your Salesforce organization. However, from a governance perspective, there are some limitations that will need to be considered to determine the suitability of using external data.
As an example, cross-org can be a great solution, sharing data from one Salesforce organization to another, but when using external objects, you should remember that each call-out counts toward the API usage limits of the Salesforce organization providing the data.
As with most limit considerations, if you are governing multiple projects with multiple applications, you will need to take a holistic view on consumption and fair distribution of resources.
Govern the use of picklist fields versus custom objects.
Careful consideration should be given to the complexity of the data model, especially the granularity of the objects created or the normalization. It is easy to become carried away in identifying objects, but the question should be whether a picklist field is more relevant than an object and its relationships.
Keep in mind the need to report on data and produce dashboards and analytics. These things become more complicated the more normalized the data model.
Govern the use of multiple data sources, focusing on clear decisions regarding system of record and single source of truth.
When data comes from multiple sources, it must be clear regarding the flow of change and where the master data resides. If this does not happen, you can become very confused as to what system has the most up-to-date view of a record.
As the governance body, you will want to know how data arrived in and goes out of your Salesforce organization. It will no doubt have security implications as well as functional issues if your data is out of date or keeps being overwritten.
This is a common problem and has been encountered numerous times. There are so many organizations that do not know what system owns the source of truth when it comes to data.
You will need to make sure that transaction boundaries are adhered to. If a transaction fails within the Salesforce platform due to a validation rule or a trigger that fails, you will not want to update the external system. Keeping the integrity of data in this scenario is critical and should be explored by the governance team.
Given the complexities around data integrity, it may be wise to publish some design principles within your organization that detail what (as an organization) you are willing to accept projects doing. You may only allow one-way integrations with external data so that the system that owns the data will not allow incoming updates, therefore enforcing its position as a system of record.
Govern the use of custom metadata types versus custom settings.
As the governance body, you will need to look at how a project is aiming to store its configuration. There are pros and cons when comparing custom metadata with custom settings, some of which will really only be relevant in the context of your application. However, this might be a good candidate for having a design principle in place that can guide each project (and ultimately the applications being built) as to which way to go.
If hierarchical settings are required by your project, then custom settings might be the best way to go. However, custom settings are not deployable to sandboxes, unlike custom metadata. Again, if you are using custom settings to store web-service information, this may be a good idea as you will not want your production settings being copied to a sandbox.
However, custom metadata have many additional features and offer more flexibility from a permissions perspective.
Govern the usage of large data volumes, considering application performance, query, search, indexing, reporting, testing, sharing, and administrative functions.
Understanding the scale of your project’s application is key so that a good picture can be built of the expected data values. Your project may already have accepted the need for large data volumes and be expecting to implement Salesforce’s big objects. However, if this is not the case you need to know what to expect from a data generation perspective so that later issues are not uncovered and left to your administrators to resolve after your project team has disbanded.
Salesforce uses a relational database management system (RDBMS) at the infrastructure level, but due to the way in which Salesforce’s multitenant architecture uses that database solution it is not possible for the RDBMS to effectively optimize any queries it receives from the Salesforce platform. Given this scenario, Salesforce has implemented its own query optimizer, which helps the RDBMS’ optimizer produce effective query performance.
Where the capacity model provided by your project indicates a large data volume or where there is an initial data load requirement by your project you may want your project to provide evidence that any queries it has developed are efficient. Your project should provide evidence that it has understood the selectivity of any filter conditions used by its application with SOQL (Salesforce Object Query Language) queries, list views, or reports.
If your filter condition involves a custom field, you may have to work with Salesforce’s customer support to create a custom index on the field your filter uses. Not all fields can have an index, such as non-deterministic formula fields.
Where your project is frequently querying the same fields and using the same joins in reports, for example, you may want your project to investigate the usage of skinny tables. Again, you will have to work with Salesforce’s customer support teams to have this set up, but it may make a big difference in performance for reports.
Consider your project’s use of indexes. It may be necessary to create a custom index, but bear in mind the types of field that can be indexed. Flagging a field as an external ID causes an index to be created on that field, but this can only be applied to certain field types, such as email, number, text, and auto number. For other field types, you will need to contact Salesforce’s customer support.
If search is a big win for your business, then you may need your project to consider divisions. Divisions allow you to partition data—for example, by region. Again, this is a feature that must be discussed with Salesforce’s customer support.
Given the complexity of using large data volumes and the impact it can have on performance and other areas, it is important to make sure the project you are governing takes time to think about the longer term. If you need to engage Salesforce’s customer support for help, it will take time, so the earlier you understand any potential longer-term issues the better the outcome for your business.
Govern the application for performance, scalability, and maintainability of the data model.
As mentioned previously, there are a number of places where the scale of data can create a problem in the longer term, so plans must be in place to avoid disaster. As the team governing the applications that are targeted for the Salesforce platform, you may look to the project to provide evidence that first they have considered the scale at which the application will be used—the number of end users, for example.
The project team should then be able to create a model that gives some indication as to the data that those end users are expected to generate over a time period of twelve or twenty-four months, or even longer. Given that information, the project team should be able to create a test scenario that proves that their data model design works efficiently at the scales they expect to see in production.
Govern the use of AppExchange packages to maintain data quality.
As with all AppExchange packages, you will want to ensure they meet your organization’s security and commercial policies. If those are met, then from a governance perspective you need to understand what is happening to the data and how confident the project team is that they have fully understood the third-party package.
The levels of governance you apply could be determined by the types of data and quantities that the AppExchange package will have access to. If the package is restricted to data that is only relevant to the project, it is an easy situation to deal with, as opposed to when the package is accessing data used by all applications within your organization.
However, it is commonplace to use a package to take care of duplication recognition. Even though the Salesforce platform offers some features in this area, there are other solutions that provide a broader set of functionalities.
Are any data changes well documented and clearly defined so that all projects understand why and how their data is changed by a third-party package?
Data residency needs to be considered; does anything happen to the data that takes it off of the Salesforce platform temporarily?
Can data changes be reversed if things go wrong, or if the business is unhappy with the changes made?
Are there sufficient support arrangements in place for the third-party package?
The key here is that you are not trying to block the use of third-party packages to help increase benefits to your business and reduce the effort exerted by your project teams to deliver functionality, but rather are just double-checking that due diligence has taken place.
Data Movement
Phase B: data architecture & management - data movement
At some point during the lifecycle of an application, your project or support team will more than likely need to move data into or out of the Salesforce platform.
It is important that any data movement is governed to ensure that data remains secure throughout its lifecycle and that the business is clear regarding any data archiving policies that you may have within your organization.
Data migration. This is typically part of moving data from a legacy system into the Salesforce platform. This may happen during the early phases of a project because of the work involved in extracting, transforming, and loading the data into Salesforce and subsequent testing.
Archiving. This is typically part of an information lifecycle management policy. Your organization may require data to be archived onto a separate platform after a predefined period has passed, or if the data in question has not been accessed for some time. All archiving solutions should also deliver a recovery solution so that the business can recover data from the archive in the future if required. Bear in mind when governing these types of solutions that the project will need to take into account data structure changes over time, which may invalidate the data structure in the archive.
Data backup and recovery. Although the Salesforce platform is secure and robust, most organizations insist on a separate backup solution. Don’t forget the restore solution too.
Data insurance. Not all organizations want to feel tied into the Salesforce platform, and will therefore store their data on an alternative platform as well.
Off-platform analytics and data mining. It is not unusual for the business to want to mine the data they have been capturing with the Salesforce platform. This generally leads to data’s being extracted from the Salesforce platform and loaded into a separate database, sometimes referred to as a data lake.
Govern the usage for data movement (data loading).
From a governance perspective, the reasons behind moving data into or out of the Salesforce platform are down to the business requirements. However, it is your responsibility to ensure that the project has properly considered this area, because at some point it will rise to the surface, and more often than not it ends up being the problem of the platform support team.
Challenge the project team to be confident that they have considered the long-term requirements of data movement. Then discuss with the team the options they are going to use, as well as the tools, and make sure they understand the movement’s impact, if any, on the performance of the Salesforce platform while the data movement is under way.
Remember, if making large changes to data that has a significant number of sharing rules, the Salesforce platform may require a significant period to revaluate the sharing.
If you are bulk-loading data, or making significant changes to a large data set, consider the impact on any sharing rules you have implemented.
Method
Design and Optimization
Data Movement
Approach
This phase focuses on the data arena and requires the data model that will have been produced by your project and will later be provided as an input into the subsequent phase. Additionally, data volumes will be needed, predictive or otherwise, as well as the configuration and code elements where used.
For the initial pass, the data volumes will be predictive. However, proof should be provided that the application has been tested to support the data volumes—where those volumes are in excess of 20 million records, for example. Subsequent passes through this phase can use actual figures from the production environment so the data volumes can create a prediction model for future volumes.
Any SOQL and SOSL should be reviewed with the aim of determining the use of appropriate indexing. You may want to investigate the use of automation tools such as static source code analysis and comparing the usage of fields in queries against the indexes created as part of the application’s configuration.
Data movement requires the application owner to detail any requirements and the solution to moving data into and out of the Salesforce platform.
Inputs
Application data model
Performance & scale test exit reports
Application configuration & source (or the repository in the example of using a configuration management tool)
Custom settings used and created
Data unload and load details
Predictive or actual data volumes
It is up to your organization to determine the formality of the artifacts previously mentioned. Some organizations will look for formal notation to be used in the data model, while others will be happy with a high-level slide deck. This governance method should not force projects to undertake more work just for the sake of governance, but rather the governance process should utilize the standard artifacts your organization expects any project to produce.
Steps
The steps for governing the data architecture phase should help the project team understand how their application uses data and how that might affect the Salesforce platform overall. The governance team should feel confident that over the lifespan of the application, there should be no fundamental issues with the applications used.
Design and Optimization
- 1.
Govern the data model, paying attention to modeling decisions, such as the usage of lookup versus master–detail relationships
- 2.
Govern the use of external data (external objects)
- 3.
Govern the use of picklist fields versus custom objects
- 4.
Govern the use of multiple data sources, focusing on clear decisions regarding system of record and single source of truth
- 5.
Govern the use of custom metadata types versus custom settings
- 6.
Govern the usage of large data volumes, considering application performance, query, search, indexing, reporting, testing, sharing, and administrative functions
- 7.
Govern the application for performance, scalability, and maintainability for the data model
- 8.
Govern the use of AppExchange packages to maintain data quality
Data Movement
- 1.
Govern data migration (data loading) strategy and solution
- 2.Govern the data archiving strategy and solution provided by the project, including
- a.
tools selected, and
- b.
archived data storage and retrieval solution.
Outputs
Not Applicable – Project team has no intention of utilizing platform feature.
Remediate – The governance team requests that the project team remediates their design to accommodate the issues raised during the governance review.
Pass – The input provided has passed this governance phase.
Review – The input cannot be objectively measured, and therefore a subjective view has been made, which will lead to a discussion with the development team to reach consensus. Although undesirable, this could be a consequence of unclear standards/policies.
Scenario
Govern the data model, paying attention to modeling decisions made, such as the usage of lookup versus master–detail relationships.
Govern the use of picklist fields versus custom objects.
You will remember our fictitious company Bright Sound Guitars Ltd. introduced earlier. Having embarked on their first project for the Salesforce platform, they are ready to review their data architecture and management strategy.
The folks at Bright Sound Guitars understand that getting the data model right is important, as it’s going to be the foundation of their application, and once it is in use, fundamentally changing it will be hard, time consuming, and expensive.
The project’s architect, Hanna Snyder, has readied the data model; she is going to present three model types. First will be the conceptual data model . She feels that this will help the governance team to grasp quickly the business concepts she needs her data model to support. Then she’s going to show her logical model to demonstrate to the governance team how she has clarified optional and mandatory relationships and added more detail regarding the types of data each entity will store. Lastly, she is going to show the physical model, which will demonstrate how her data model will be represented on the Salesforce platform.
Phase B: data architecture & management - conceptual data model
Hanna has used a typical entity relationship diagram (ERD) to represent her conceptual model. This works perfectly for the Bright Sound Guitars’ governance team, as they are familiar with this notation.
It is clear to the governance team that Hanna’s application is going to store information regarding the construction of guitars, and then provide a relationship between a customer ordering a guitar and owning one.
Of course, Bright Sound Guitars would love their customers to enjoy the bright sound of many of their guitars, so the model supports a customer’s ordering and owning multiple guitars.
Phase B: data architecture & management - logical data model
Hanna has put more detail into this model, clearly showing the types of data or fields each entity is going to store. Hanna has also highlighted mandatory and optional relationships using the ERD notation, as well as refined the cardinality between entities.
The Bright Sound Guitars’ governance team is now starting to build up a good picture of the application’s data model that Hanna is putting together. At this point they are starting to have some concerns but want to see how this data model will be represented on the Salesforce platform. After all, there can be a big difference between models because they are aimed at different stakeholders. Hanna would have had to initially build her models with help from the business, and this means she needs them to feel confident that she has understood their requirements.
Phase B: data architecture & management - physical data model
Hanna has put together a great physical data model, which is much more detailed that the others, something she probably would not share with the business, but something her Salesforce developers would love to see.
Hanna’s model clearly shows the master–detail and lookup relationships. This will make the governance process a lot easier for the governance team and reduce the amount of time Hanna needs to spend explaining her solution.
- 1.
When questioned on the use of Customer, Order, and Order Items objects, Hanna clearly states that these would be custom objects implemented on the Salesforce platform. The governance team is concerned that Hanna will recreate functionality that is already available on the platform. The governance team explains that Bright Sounds Guitars has a number of projects in the pipeline, Hanna’s being the first, and wants to take advantage of as many standard features as possible. This allows Bright Sound Guitars to take advantage of features and functionality that Salesforce delivers in the releases three times each year. Hanna accepts this feedback and will review her models to take advantage of existing standard objects.
- 2.
The governance team asks more questions around the Guitar object. One member of the governance team asks Hanna whether the Asset standard object might be a good fit. Hanna isn’t sure of the exact nature of the Asset object, but accepts that this could be a good candidate, especially given the relationships with Products and PriceBooks. She feels that perhaps there are more standard objects within the platform that she could take advantage of, which would accelerate her development time, while potentially offering more functionality to her business stakeholders. The asset-tracking feature sounds like a great feature her business stakeholders could use, especially for product support issues and warranty expiration.
- 3.
Lastly, the governance team raises a number of concerns regarding the number of objects being used to construct a guitar. Although from a business perspective these all make sense, the team wonders if picklists could be a better option. Additionally, Hanna chose to use picklists where the governance team feels an object would work better. One example of this would be the manufacturer, which was used as a picklist in several objects. The governance team feels that the use of the Account object to represent the manufacturers could be a better use of the platform’s capability, especially as Bright Sound Guitars would like to track their own service requests with their suppliers for any warranty work on supplied items.
Overall, the governance team is happy with the work Hanna has produced, and given that this is her first iteration through the governance process, everyone feels like a great outcome was reached. The governance team provides a “remediate” rating against the criteria they assessed the project against. Hanna will now return to her team with the feedback and make the necessary changes. She will then present back to the governance team with her updated data models.
Summary
Phase B is focused on the data design of a project’s application. This is a foundation of any application, but depending on the longevity and scale of your application it is not always relevant to take such a detailed look.
The scenario provided gave a simplistic look at how the governance process might work in your organization. Governance should be seen as a collaborative exercise to make sure that you are taking the best approach, given what you know for your organization and the business requirements.
Bear in mind that design is subjective; there is not always a right and a wrong. However, you should attempt to steer your design based on principles you define, such as reusing as much of the standard functionality as possible. This is highlighted in the scenario, where the governance team has a principle to use as many standard objects as possible. Given the size of Bright Sound Guitars and the fact that IT systems are not their specialty, they chose to keep the amount of bespoke work to a minimum and focus their efforts on delivering business benefits quickly.
Depending on the complexity of your application, you may find that all of the steps laid out in this phase will need to be completed. However, make sure you take a look with the project team to make sure you are not spending time on reviewing irrelevant areas.
The main takeaway from this phase is that a little time spent up front checking a few key principles can avoid a lot of heartache later on in the usage of your application.