8. Predictive Analytics in the Cloud

James Taylor

“Innovation happens at the intersection of two or more different, yet similar, groups. Where one technology meets another, one discipline meets another, one department meets another.”

—Valdis Krebs, Founder and Chief Scientist, orgnet.com

Predictive analytics are increasingly the focus of many organizations’ efforts to improve business performance. At the same time, the cloud is fast becoming an important option for purchasing and deploying software. Public, private, and hybrid clouds are all evolving rapidly and are here to stay. So what’s happening at the intersection of these two technologies?

Over 200 professionals recently participated in a research study and survey of predictive analytics in the cloud conducted by Decision Management Solutions and Smart Data Collective.¹ Respondents came from organizations of all sizes. Over half were from organizations with fewer than 500 employees. Most of the remainder came from organizations with more than 2,000 employees. More than a quarter of respondents were executive management. Twenty percent of respondents identified themselves as IT professionals, 30% as business, and 40% as analytics.

The results show that early adopters are breaking away, and many kinds of cloud-based predictive analytic solutions have potential. Although industries vary in their maturity, the use of cloud-based predictive analytics to improve an organization’s focus on customers is particularly powerful. Early adopters look likely to build a sustained competitive advantage.

“What’s most impressive? It’s the amount of money (millions of dollars) that can be returned to the company’s bottom line using good predictive analytics.”

—Survey respondent

Business Solutions Focus

Most potential buyers of predictive analytics in the cloud are not specifically looking for “cloud” solutions. Years of successful industry adoptions of predictive analytics and growing awareness are resulting in more demand for analytically based solutions. Yet many organizations are not looking for “predictive analytic” solutions either. The vast majority of organizations seek a solution to a specific business challenge. Predictive analytics can help them address the challenges they face. A cloud-based approach can make these solutions faster to deploy, more cost-effective, and more collaborative. Whatever deployment method they adopt, the primary driver is a need for a solution to a business problem.

Organizations realize that technology is not enough, that they also need best practices and industry- or solution-specific implementation. Few solutions are purely software-based; most involve configuration and specialization to work for a specific organization. This requires domain expertise as well as technical know-how. Moreover, embedding predictive analytics often involves significant business change, starting with a willingness to experiment.

This business solution focus goes back to some of the earliest “cloud-based” predictive analytic solutions. Nearly 30 years ago, credit card processors offered hosted and shared applications for fraud detection and credit risk management. A key driver of the adoption of these packages was a desire on the part of banks and credit card issuers to get access to advanced analytic solutions as a packaged offering. This driver remains front and center decades later.

“Tools must be easy for my business teams to use and understand the results; they aren’t sophisticated modelers!”

—Survey respondent

Five Key Opportunities

The research revealed five common deployment patterns for cloud-based predictive analytics, each offering opportunities for organizations. These five areas include complete solutions, ways to use the cloud to push analytics into existing solutions, and ways to use the cloud to more effectively build predictive analytic models.

Prepackaged Cloud-Based “Decisions as a Service” Solutions

These are cloud-based or software as a service (SaaS) offerings that provide predictive analytics for decision-making as a core feature. Examples include cloud-based applications offering next-best action, offer selection, fraud detection, or instant credit decisions.

They are domain-specific packaged applications that make or enable specific decisions that can be described in business terms. Predictive analytic models are embedded within a solution framework so that the customer receives better decisions, not simply predictions.

For example, a multichannel cross-sell application decides which products to offer customers in different channels, and when. This is based on analytic models that predict how likely it is that the customer in question will buy each product and on rules and policies regarding how and when the products are sold.

These predictive models may be built automatically by software embedded in the solution or built by the solution provider directly. Customers do not have to build their own models, but the models may be built using the customer’s own data. However, some of these models are built using data pooled from many organizations, so multiple customers of the solution have the same predictive analytic models. For instance, applications for credit card fraud detection may use scores developed from credit card transactions across multiple card issuers to predict how likely a particular transaction is to be fraudulent.

Predictive Analytics for Software as a Service

These cloud-based solutions inject predictive analytics into other software that is cloud-based or delivered as SaaS. Examples include embedding customer churn predictions in SaaS CRM solutions or delivering risk predictions into cloud-based dashboards.

Many SaaS applications don’t include predictive analytics. A cloud-based predictive analytics solution may be the most effective way to embed more-advanced analytics into these operational systems. Predictive analytics or scores are delivered using the cloud to improve the accuracy of decisions already being made by the SaaS application.

For example, a credit risk score could be delivered to a SaaS CRM solution and then used by a customer routing script to route customers with low credit scores to an agent who specializes in helping those with poor credit. The predictive analytic models in question could be developed by the customer, the solution provider, or a third party. They also could be based on pooled data, as discussed in the preceding section. The models could be built automatically using software or built using an existing analytic infrastructure. Regardless, the focus is on making those predictions available to SaaS or cloud-based applications.

Predictive Analytics for Legacy Systems

These cloud-based solutions inject predictive analytics into in-house systems and multichannel environments. Examples include embedding risk scores into a legacy underwriting application or using cloud-based deployment as a bridge to deploy propensity-to-buy models across multiple customer-facing systems.

Most of a typical organization’s legacy systems do not use predictive analytics to drive their behavior. In addition, many predictive analytic models are built by organizations and then are not deployed because there is no efficient way to do so. These undeployed models represent lost opportunity. With the pervasiveness of cloud-based solutions and the ease with which applications can be connected to the cloud, a cloud-based predictive analytic deployment approach may significantly increase the effective use of predictive analytic models, especially in legacy applications.

The characteristics of cloud-based services to deliver predictive analytics to on-premises applications are very similar to those embedding predictive analytics in SaaS systems. The target systems may include applications of suppliers and other business partners. For example, a product’s predicted target price might need to be distributed to multiple channel partners who each have their own systems. Predictive analytic models with this deployment are more likely to be the organization’s own, built in-house or in the cloud.

Modeling with the Data Cloud

This is using cloud-based predictive analytic solutions to respond to the increasing amount of relevant data available in the cloud rather than on-premises. Examples include building predictive analytic models using customer purchase and behavior data stored in a SaaS CRM system, as well as third-party data available from a cloud-based web service.

An increasing number of the data sources that an organization needs to use to build predictive analytic models are available in the cloud. Where previously organizations had on-premises solutions that contained all their customer, sales transaction, human resources, marketing, and web data, now this data is often stored in SaaS and cloud-based solutions. In addition, social media and other unstructured data are often available only through the cloud. The increasingly widespread adoption of big-data technology is driven in part by a need to access and analyze the large volumes of new data available in the cloud. Pooled data supplied by members of a business consortium is also likely to be collected in the cloud.

Modeling with the data cloud pulls all the data available in SaaS applications as well as third-party web services into a cloud-based data management and modeling environment. It pushes predictive analytic modeling to the cloud next to this data so that an organization’s whole analytic team can access it, and build models against it, from anywhere.

Elastic Compute Power for Modeling

This is using cloud technology to provide predictive analytics solutions that can scale elastically to meet demand. Examples include assigning extra resources dynamically when optimization or other demanding algorithms are being used to build or run sophisticated predictive analytic models against large datasets.

When companies are building and using predictive analytic models, the amount of computing power needed varies widely during the process. Building predictive analytic models in the cloud offers potentially infinite scaling because clouds (private or public) can deliver elastic computing power. This makes it easy to add and provision new hardware as needed for modeling activities rather than requiring a predefined amount of hardware to be purchased, provisioned, and configured.

For instance, when large datasets must be analyzed or when complex simulations are required to produce predictive analytic models, the team needs a lot more processing power than when they are analyzing results or investigating the data. This scalability is increasingly common in the tools used to build predictive analytic models, but it is by no means pervasive. It may require significant development effort to parallelize and distribute algorithms.

The State of the Market

The market for cloud-based predictive analytic solutions is clearly growing, and early adopters have seen some positive results. Our survey showed a matching increase in confidence, the value of a focus on decision management, and the continuing strength of “traditional” structured data in predictive analytic modeling.

Early Adopters, Competitive Advantage

Survey respondents who have the most experience with predictive analytics are moving more aggressively and with greater confidence. These organizations were

• More likely to have plans to adopt more cloud-based predictive analytic solutions

• Much less likely to have performance or privacy concerns about the solutions

• More likely to embed predictive analytics in operational systems, a driver of positive ROI

• More likely to take advantage of big data from the cloud

Respondents already deploying at least one cloud-based predictive analytics solution are much more likely to adopt solutions going forward. These early adopters contrast with those not yet using cloud-based predictive analytics, who see themselves as much less likely to adopt solutions of any kind. This was true even of packaged solutions, the most preferred option for those who have not yet adopted any solution.

Not only are those already adopting cloud-based predictive analytic solutions getting positive results, but these results make them more likely to accelerate and broaden their adoption of these solutions. Those hesitating about adopting them run the risk that they will be left behind, watching early adopters establish a lead that grows with time.

Decision Management Increases the Value of Analytics

Decision management was clearly an important element for successful analytics adopters, especially as they embedded predictive analytics in operational systems. Survey respondents reporting transformational impact from predictive analytics were much more likely to integrate predictive analytics into operations. As shown in Figure 8.1, the initial impact often comes from occasional use of predictive analytics. But more impact is reported as predictive analytics are used in more operational decision-making.

Figure 8.1. The impact of predictive analytics.

Continued Strength of Traditional Data Sources

Survey results showed that structured data from cloud sources was the most important source for building predictive analytic models in the cloud. That was followed by pooled data (structured data from multiple companies pooled for analysis) and structured data uploaded from on-premises solutions to the cloud. Despite all the hype around the unstructured data component of “big data,” it seems that structured data still rules in predictive analytics. In addition, those with the most experience in building predictive analytics in the cloud were very positive about the value of both static and batch data. These experienced users were less excited about moving to real-time data than those who lacked experience. Real-time data, it seems, is widely expected to produce better results by those with limited experience. Those who have successfully built and deployed models seem to know that this is not necessarily true.

Taken together, this implies that organizations can get started with predictive analytics and get positive results from using predictive analytic models, even if those models are built only from structured data in a batch environment.

Pros and Cons

Like all things, cloud-based predictive analytic solutions have clear pros and cons. Pros include time to value, pervasiveness, agility, scalability, and data access. Cloud-based predictive analytic solutions have a much faster time to value than alternatives, and the pervasiveness of the cloud is a major factor in this value. Because cloud-based solutions focus on simple, standardized interfaces, they are easy to deploy and adapt. As organizations increase their consumption of predictive analytic models, the value of scalability offered by the cloud will only grow, especially as new sources of data emerge in the cloud.

Against these clear advantages are some cons, such as concerns about privacy and security, regulatory issues, bandwidth for moving data to the cloud, and increased complexity. Keeping the data used in building predictive analytics private and secure is an ongoing challenge, in the cloud or out, and many regulators are uncomfortable with data in the cloud. Even when these challenges are overcome, some organizations find that moving data to the cloud is a challenge due to the fairly narrow “pipes” available at the edge of the Internet. Finally, because cloud-based solutions are still somewhat new and unfamiliar, that creates potential complexity.

Our study found that organizations that have had positive results with cloud-based predictive analytics worry less about data security and privacy, about complexity, and about latency and responsiveness. Familiarity results in a slight decrease in the severity of these concerns. That said, these concerns are particularly strong in industries where the core data required for predictive analytics is regulated data. It is also clear that some of the variation in cloud choices (public, private, or hybrid) is driven by these concerns. Private clouds, for instance, are preferred where the data involved is sensitive or where responsiveness is critical.

Adopting Cloud-Based Predictive Analytics

“There is really no debate anymore on whether to add or not to add analytics to the information technology and business activities within an organization. Instead the debate centers on how to make the best use of the myriad of analytical opportunities that are out there.”

—Jane Griffin, Principal, Deloitte Consulting LLP, and Tom Davenport, IIA Research Director and Senior Advisor, Deloitte Analytics

The basic value proposition of predictive analytics in the cloud is clear: Organizations can make predictive analytics more scalable, more pervasive, and easier to deploy using cloud technologies. As more organizations seek competitive advantage through analytics, they need the ability to rapidly make analytics pervasive and to tightly integrate analytics into their business strategy and day-to-day operations. For many of the challenges organizations face on their journey toward becoming analytic competitors, cloud-based solutions have much to offer.

Before adopting cloud-based predictive analytic solutions, organizations should understand where they fall on the maturity curve: just getting started with predictive analytics, with some experience but not yet widespread use of predictive analytics, or using predictive analytics regularly and looking for ways to be even more effective.

The different kinds of solutions available under the umbrella of predictive analytics in the cloud enable organizations at every level to adopt cloud-based predictive analytics. They can use cloud-based solutions to jump-start their adoption of predictive analytics, speed and support expansion of use, or refine an already sophisticated approach. The different solutions also enable different parts of an organization to progress differently. More sophisticated or experienced departments can have different adoption strategies than those with no prior predictive analytics experience.

The research and survey results make it clear that organizations should make cloud-based predictive analytics part of both their development approach and deployment architecture. Cloud-based predictive analytics make it easier to adopt new data sources, especially cloud-based big data. The cloud improves the effectiveness of scarce modeling experts by making the power they need available on demand. The pervasiveness of the cloud and the simplicity of its interfaces make it a compelling platform for analytic models, helping to put predictive analytics to work throughout an organization’s operational systems, processes, and decisions.

Endnote

1. The survey was conducted in 2011. The research study was sponsored by Clario Analytics, FICO, Opera Solutions, Predixion Software, SAS, Teradata, and Toovio. Full details of the research study are available at http://smartdatacollective.com/predictive-analytics-cloud.