Integrating with Other Records Repositories
The scariest moment is always just before you start.
—Stephen King
Managing content and records within the SharePoint platform works well because Microsoft designed all the different capability areas to integrate with each other. But what happens when you have to integrate SharePoint with external records repositories? In this chapter, I discuss options for non-SharePoint records systems and how to integrate those systems with your SharePoint records management solution, particularly for physical records, document-imaging solutions, and other digital records repository systems. I also provide you with an overview of the extension capabilities available for SharePoint records management.
After reading this chapter, you will know how to
- Describe options for external records systems.
- Plan a document imaging solution.
- Integrate with external records repositories.
- Understand extensions for SharePoint records management.
Understanding Requirements for Integration
SharePoint is a broad product. It does a lot of things really well, but this breadth comes at the cost of it being less specialized in certain areas. Although the gap is narrowing with each release of SharePoint, competing products that specialize in a narrow functional area may make a stronger match to specific functional requirements. As a result, your organization might select SharePoint for a range of its capabilities, and yet select another product that is more specialized in a particular area, even though it may overlap some of the functionality SharePoint provides.
Alternatively, your organization might have an old system already in place before you ever deployed SharePoint, a system that many other applications, departments, and processes all depend on. The legacy system might have so many dependencies that migrating the records and consolidating on a single platform with SharePoint is not practical to achieve given your budget, timeline, or the politics involved.
Whatever the situation and motivating drivers behind maintaining a legacy system, you can integrate SharePoint with these external systems (at least to some degree, depending on the system and the level of integration the system’s vendor exposes). Typically, the integration depends on a SharePoint connector, a component the vendor supplies to integrate SharePoint with the third-party repository, as shown in Figure 16-1.
Figure 16-1. Example of an external records repository
Integrating multiple enterprise applications introduces complexity and dependencies. I find that establishing the roles and responsibilities between the systems offers the best way to manage the complexities and ensure coverage. This documents what is expected and from whom, making it clear what tasks or decisions need to occur and by what role.
Tip I discuss establishing roles and responsibilities, among other governance topics, in my first book, Practical SharePoint 2013 Governance (Apress, 2013). You can find out more about this book and access an online preview of it on the Apress web site at www.apress.com/9781430248873.
You might need to integrate with a variety of types of external systems, but the two popular types are document imaging solutions and external repositories. Document imaging systems input records into SharePoint by scanning a physical document. External repositories receive and preserve records from SharePoint and other systems. First, I discuss document imaging, and then in a later section, I return to discuss integrating with external repositories.
Planning a Document Imaging Solution
As far as technology has come in recent years, many organizations still retain large amounts of physical records. If your organization needs to work with physical paper documents as part of its process, then you might consider digitizing the document as part of its record declaration process. You might not always want to maintain a copy of a physical record. This is solved by scanning and capturing an image of those documents.
SharePoint itself does not have document-scanning features built in; instead, Microsoft relies on partners and independent software vendors to extend SharePoint and provide document imaging solutions. The implementation of each solution depends on the particular vendor, but in a general sense, they all perform the key task of scanning a physical document and then saving a digital image of it in a SharePoint library. Figure 16-2 illustrates the primary document imaging process.
Figure 16-2. Capturing scanned images of documents in a SharePoint library
As SharePoint does not have a document-imaging component, you must implement a document scanning solution as a third-party product that works with SharePoint. There are several on the market from a number of different vendors. Some are specialized to work with a vendor’s scanning hardware, while software products are more hardware independent.
To plan your document imaging solution, start with your scanning hardware. If you do not have scanning hardware, you should investigate the different products on the market to determine which best fits your organization’s requirements and available budget. You should also consider whether it has a component to work with SharePoint by interfacing with the SharePoint Application Programming Interface (API).
Once you have the scanning hardware and software to capture the image to a SharePoint library, follow the configuration instructions provided by the vendor you choose.
Note The implementation and configuration of the imaging solution is vendor-specific.
Document imaging follows a consistent process, and it mostly revolves around handling and then destroying the physical document. The following steps summarize the key points involved with a document imaging solution.
- Receive the physical document.
- Scan the physical document, capturing a document image and any relevant metadata.
- Save the document image and metadata to the SharePoint document library.
- Shred the physical document.
The physical document consumes the most effort in the entire process, from handling to scanning to shredding the document. In many cases, this is simply required. However, where possible, you might be able to optimize this process by having users scan and shred their own documents. This would simplify the document imaging process, or at least delegate its burden to the user, which can be beneficial in several applications, such as the expense report I discuss in the sidebar.
USERS SCANNING THEIR OWN DOCUMENTS
I have seen a number of different expense reporting processes, both from the different companies I have worked for and from the different customers I have advised. Two common elements are the itemized expense report and the expense receipt. Where they differ is how the expense workflow processes the receipt through the process.
At some point during the workflow, the receipts need to be digitized. During the first years of digital expense reporting, I noticed the receipts were often sent to a central scanning facility, which is one document imaging solution. Recently, however, I have been noticing more and more expense processes begin with the user photographing his or her expense receipt and submitting the expense through a smartphone app.
With users submitting documents already in a digital format, the entire workflow processes a digital file. This process speeds up the entire workflow because all the information is contained together as a package as soon as a user enters it into a system and initiates the process; it does not have to wait on an imaging step.
Interfacing with External Records Repositories
SharePoint cannot operate effectively as an island of information. Its value and the richness of its experience depend on connecting with your organization’s records repository. This benefit is realized by surfacing the relevant information from the records repository in context in SharePoint, such as within the search results, and archiving SharePoint content to preserve as a record in the official records management system.
One important integration requirement is to include an external records repository’s content in the SharePoint search results. As I mentioned in Chapter 9, a strong user experience involves providing users with an enterprise search portal where they can search across all content repositories, SharePoint and otherwise. Including archival content and preserved records in the search results gives users a single known location to find information in your organization.
You can interface your SharePoint search implementation with an external records repository by configuring a search connector. Some application vendors provide a SharePoint search connector you can install and configure to enable SharePoint to crawl the content. Alternatively, you can use a generic search connector in some cases or develop your own connector.
Note To learn more about the search connector framework in SharePoint 2013, please see the MSDN article at http://msdn.microsoft.com/ee556429.
There are several ways to input data into SharePoint, from users uploading files to workflows submitting files or data. But one technology still persists as a handy and functional way to transfer data: e-mail. Sending a document over e-mail is an easy way to share a copy with a colleague, and the same is true for interfacing with SharePoint.
You can configure a library to receive e-mail messages at a designated e-mail address. This enables users to send a document to a SharePoint library from Outlook. However, you do not have to limit this capability to users; any process that can send an SMTP e-mail message can forward and submit a document to a SharePoint document library.
Note To learn more about configuring incoming e-mail for a SharePoint 2013 farm, please see the TechNet article at http://technet.microsoft.com/cc262947.
Another type of system integration includes providing a connection that SharePoint can use to submit content to the external system. The vendors of enterprise applications often provide these integration connections. However, if your records repository does not provide a component to bridge a connection with SharePoint, then you can extend SharePoint by developing your own connection.
Extending SharePoint 2013 Records Management
Records management in SharePoint continues to evolve and mature. The product has included records management features for a few versions now, and the product team has made strong investments with enhancing the features with each iteration. Nevertheless, there may be things you want to do or ways you want SharePoint to manage records that the product does not do. In these cases, you can either develop your own solution or license one from an independent software vendor.
Microsoft provides an extensible application programming interface (API) with SharePoint 2013, enabling you to extend functionality in SharePoint. Although this is not a book on development or customizations, I did want to point out that the product team made an API available for developers to extend and customize functionality on top of the SharePoint platform.
Note Please see the MSDN article at http://msdn.microsoft.com/jj193038 to learn more about the SharePoint 2013 API reference libraries.
Rather than develop your own components to extend SharePoint functionality, you can purchase third-party products. Companies have developed packaged solutions for a number of different aspects of SharePoint, including records management. You can find products that specialize in a variety of solutions, including the following areas:
- Workflows and electronic forms
- Metadata management
- Document imaging
- Auditing and reporting
- Content replication
- Retention and disposition
- Records reporting and dashboards
- Policy management
I have worked with a number of different products, each product with its own benefits. In the sidebar, I provide an example of one product extending SharePoint specifically to add records management functionality. There are several other firms that license apps or components to extend the default functionality in SharePoint, depending on your needs.
EXTENDING SHAREPOINT WITH COLLABWARE
I am most familiar with the records management product from Collabware because the company is from my hometown of Vancouver. My friend Graham Sibley started the company and its product with the belief that SharePoint is a capable records management platform, but that there were some gaps that a third-party product could fill to improve the overall records management experience.
A few of the key features of the Collabware CLM product and the gaps it fills include:
- Rule-based autocategorization and metadata autopopulation
- Automatic record declaration
- Records management dashboard
- Centralized compliance policy management
- Content surfacing and querying
To learn more about Collabware and to get some ideas about how you can extend SharePoint records management, please see the company web site at www.collabware.com.
My point is that although SharePoint provides a rich environment for your organization’s entire content life cycle, including records management, it also exposes an extensible application programming interface (API) for vendors or internal developers to develop products that extend SharePoint’s functionality.
Wrapping Up
SharePoint might not be your organization’s only content repository, and this might be particularly true if you have a legacy records management product deployed. Many third-party products provide connectors to enable you to integrate SharePoint with the repository. SharePoint also exposes an application programming interface (API) that you can use to develop an integration solution.
In this chapter, I discussed planning considerations for integrating with external systems, including capturing scans of physical documents and developing custom interfaces. This integration fits in the content life cycle I discussed in Chapter 1, preserving the records in a different repository. Throughout this book, I have discussed different aspects of the content life cycle, from content creation to discovery, and finally, to managing official records. All these different aspects of enterprise content management (ECM) in SharePoint should help get you started with designing your ECM solution.