Data, information, facts—whatever term you want to use, collecting and analyzing data have played a crucial part in humankind's ability to survive and to thrive since the dawn of consciousness. The earliest humans shared with each other what they knew of the world from their brains, those powerful catalogers of data in their skulls: hunt now, not later; eat this, not that; sleep here, not there.

Data is how we understand our world, and data has the capability to take us far beyond the surface impressions that our senses give us. Even though the world may appear flat to the eye, the ancient Greeks determined that the earth was round. In 240 BC, Eratosthenes used the different angles of shadows in two locations at high noon on the summer solstice to calculate the planet's circumference with remarkable accuracy—to within 1.6 percent.

Much of the mathematics, geometry, and other information compiled and shared by the likes of Eratosthenes essentially disappeared as the Dark Ages descended after the fall of Rome. But with Johannes Gutenberg's invention of the printing press in 1440—as statistician and writer Nate Silver points out in his book The Signal and the Noise—the amount of information available to societies again began to grow. Printed content enabled data to grow exponentially.

With his mind soaking up an expanding ocean of data created by these newly printed books, a sixteenth-century Roman Catholic church administrator named Nicolaus Copernicus wrote his own book, De Revolutionibus Orbium Coelestium, which used mathematical calculations and observations—data—to prove the idea that the earth revolved around the sun. This notion wasn't widely accepted in a time ruled by the Catholic Church, which was vigorously opposed to the idea that heaven was mutable and that the earth wasn't the center of it. Copernicus didn't allow his book to be published while he was alive, fearing a backlash from the Church he served. Despite the Church's longtime opposition, the data—and the truth—were eventually published.

The advent of computers has allowed data to grow at an even more mind-boggling rate. IBM's “Big Data at the Speed of Business” website says that we create 2.5 quintillion bytes of data every day, which means that 90 percent of the data in the world has been created in the past two years. The sheer amount of data and our growing ability to process it has led to the coining of the term big data.

The increasing ability of computers to process and store data was predicted by Gordon Moore, the cofounder of Intel, in the mid-1960s, and is at the heart of the rise of Silicon Valley as a global economic force. Moore formulated what is known as Moore's law, which holds that the number of transistors on a computer chip will double approximately every two years.

The result of this law's fulfillment is that the ability to process and store data becomes faster, easier, and cheaper. Progress, as evidenced by products such as smartphones and concepts such as cloud computing, happens quickly in the technology sector.

The fulfillment of Moore's law has created what's known as big data. In a narrow sense, big data is the incredibly fast analysis (enabled by increased processing speeds and cheaper storage) of massive sets of unstructured data to find previously unavailable insights. In a larger sense, big data is the lattice of computers, mobile phones, and other digital devices that create streams of data that organizations can analyze to gain actionable insights.

Another Moore, Geoffrey, has built his philosophy of marketing technology, which he outlines in books such as Crossing the Chasm (HarperBusiness, 1991) and Inside the Tornado (HarperBusiness, 1995), around Moore's law. “We have this incredible information processing engine that has just gotten more and more and more productive, so network, bandwidth, and storage keep having this exponential reduction in cost and expansion of scale,” Geoffrey Moore said. “Pretty soon the next generation comes along, and they just design from a completely different set of assumptions.”

In the past, paradigm shifts used to takes decades. “Now it feels like a single decade is kind of like the unit of a paradigm's life,” Geoffrey Moore said.

The amazing rise of companies like Google shows the power of big data and its ability to transform not only the world of business, but the world as a whole. While big data has its skeptics, who say that big data is a fad that cannot possibly deliver on its overblown promise, the more likely reality is that the value of big data is, in fact, being underestimated. Big data—particularly for businesses and especially for marketing departments—is poised to have a profound and far-reaching impact on commerce and shareholder value. As it did for Eratosthenes, as it did for Copernicus, and as it may be doing for your company today, data will reveal the underlying truth of the world for those willing to work to see it.

Evidence that big data is much more than hype is undeniable. Big data has impacted everything from sports to politics. Case in point: even as Mitt Romney was climbing the national polls after triumphing in his first debate with President Obama during the 2012 election campaigns, Nate Silver predicted an Obama victory. Silver, a big data practitioner in baseball before he moved on to politics, stuck to his guns—and to his data. He relied on the data he blended from analyzing myriad polls. In the end, the Republicans used data to predict the result they wanted, while Silver looked more deeply into the data to predict the result that actually happened—down to the specific electoral vote count and a victory for Barack Obama.

While Nate Silver used data to accurately predict the election outcome, Dan Siroker, now the CEO of Optimizely, used data to make that outcome actually happen.

Siroker was a Google employee when he saw then-candidate Obama speak to executives at the company in 2007. Obama spoke about bringing Silicon Valley's digital and data expertise to government. Siroker was impressed. “I decided after the talk to fly to Chicago two weeks later, signed up as a volunteer, and eventually turned that into a job as the director of analytics for the Obama campaign,” he said.

At Google, Siroker was an advocate of A/B testing—a process that pits different variables in landing pages, e-mail subject lines, or display ads against each other to determine which are the most effective. He brought this expertise to the Obama campaign. “I was tasked with figuring out how to use data to help make better decisions,” Siroker said, “and it naturally led to website optimization and A/B testing.”

He said the Obama campaign had to experiment by taking advantage of data and technology, because it had no choice. “They were third behind John Edwards and Hillary Clinton,” he said. “They were forced to say, ‘If we do the same thing every other campaign does, we'll end up like how everyone else thinks we're going to end up—which is third.’ And so they said, ‘Take risks.’”

In a blog post for Optimizely, Siroker explained how a series of A/B tests, which certainly don't seem so risky in retrospect, helped the 2008 Obama campaign raise an additional $60 million. On the campaign's website splash page, Siroker and his team tested six main visuals (three videos and three photographs) and four different calls to action (CTAs) (“join us now,” “sign up now,” “sign up,” and “learn more”). The campaign tested a matrix of 24 combinations—all the potential permutations of images and CTAs.

Siroker wrote in the blog post that his team was convinced that a short inspirational video would win. The campaign tested each combination, judging them on the number of visitors who supplied e-mail addresses. The test analyzed the results of more than 300,000 visitors, which meant that each of the 24 permutations was viewed by about 12,500 people on average.

The results? The combination of the “learn more” CTA and a photo of the candidate with his wife and children posted the best performance. That combination resulted in 11.6 percent of visitors sharing their e-mail addresses compared with just 8.26 percent as the average. That meant the winning combination delivered a 40.6 percent improvement over the other combinations.

In the post, Siroker does the math. Because more than 10 million people ultimately saw the splash page, the winning combination delivered about 2.88 million more e-mail addresses. That led to 288,000 more volunteers, and—because each e-mail address averaged $21 in contributions—an additional $60 million for the campaign.

And what about the team's pretest favorite video? In his blog post, Siroker wrote, “Before we ran the experiment, the campaign staff heavily favored ‘Sam's Video.’ Had we not run this experiment, we would have very likely used that video on the splash page. That would have been a huge mistake since it turns out that all of the videos did worse than all of the images.”

This experiment was just one of a myriad conducted in the Obama campaign's digital laboratory. “The Obama campaign is a great example of how they used data to win and the big influence that data had was on our ability to find campaign volunteers, do fund-raising, get out the vote, all of those things that were conversion events,” Siroker said. “…We showed you can use data to help increase your conversion rate in an experiment, and that fundamentally was the key to the Obama campaign in 2008.”

Siroker took the lessons learned from the Obama campaign and poured them into Optimizely, a company that helps marketers optimize websites and other digital marketing tactics. “Up until today, most marketers have spent a lot of time on acquisition—getting people to come to the website,” he said. “Not everybody who shows up to your website turns into a customer, so it's about optimization. How do we get all those people—who we're spending a ton of money to get to show up—how do we turn them into customers?”

The arena where big data is having the largest impact today and where businesses may see the largest impact is the marketing department. In the past, long before companies like Optimizely, even great marketers—true believers like John Wanamaker—had a hard time proving that marketing worked. Wanamaker, a department store pioneer, reputedly said, “Half the money I spend on advertising is wasted; the trouble is I don't know which half.”

Those words have lived on long after Wanamaker passed away in 1922. As a testament to how hard it has been to measure success in advertising, that remark has had a long shelf life. For decades, marketers used Wanamaker's words to shrug their shoulders as they justified spending on tactics they believed were working—even though they usually couldn't provide proof.

But the rise of big data is making that statement as dead as Mr. Wanamaker himself.

A recent television ad from software company Adobe promoting its Marketing Cloud shows how important data can be to the marketing team—and how damaging the wrong data can be to the entire company. The ad is a perfect representation of how the data-driven marketing department is at the center of the enterprise these days. The commercial, titled “Click, Baby, Click!,” opens in the dark, dingy office of a fictional company, Encyclopedia Atlantica. Two guys wearing neckties report a surge in web activity. They inform the boss, “Clicks are off the charts!” He, in turn, calls an overseas supplier, telling him, “Yoshi. It's Walt. We're back!” Set to Edvard Grieg's short anthem “In the Hall of the Mountain King,” the phone call sends printing presses into motion, causes tractor trailers and container ships to be loaded with Encyclopedia Atlanticas, forces more trees to be cut down, and leads to wood pulp futures taking a big jump.

Then comes the kicker. The ad closes by revealing the real reason that the Encyclopedia Atlantica website was seeing all those clicks: a toddler is playing with a tablet and pressing the “buy now” button on the encyclopedia website—over and over and over again.

Adobe finishes the ad by asking viewers: “Do you know what your marketing is doing? We can help.” Aside from making a great case for the Adobe Marketing Cloud, this TV spot is a commentary on how the data rolling into the marketing department influences—for better or for worse—the rest of the enterprise. Executives make decisions based on this customer data, and these decisions determine how resources will be allocated throughout the company—and perhaps throughout an industry. With so much riding on the data, it had better be right.

In this digital age, leading marketers are embracing software platforms that deliver cascades of data. The software platforms that marketers are making use of include marketing automation systems, customer relationship management systems, data management platforms, and analytics tools to help make sense of what is happening. The most effective strategy is for companies to tie together the elements of this software, which together are known as the marketing stack. This allows the marketing team to see a complete picture—a 360-degree view—of how prospects and customers are behaving. With this insight about the target market, not only can the marketing team serve relevant messages to the right people at the right time, but it can also anticipate their needs and perhaps even create the products their customer base didn't even know it wanted.

Beyond the marketing department, data about the customer also flows into companies via e-commerce platforms, customer service call centers, and billing and payment records. The corporations that will benefit the most from their data are the ones that will bring all this information into a central repository. This centralized data repository should be managed by the marketing team, since they have more insight into the customers than another department would. How customers discover products or services, make purchasing decisions, and share their experiences—commonly referred to as the buyer journey—has changed dramatically. In the days before the Internet, potential clients researching products had little choice but to pick up a phone and call a sales representative to get more information about what they wanted to buy. Or, in the retail sector, they simply entered the store. In this new paradigm, it is not the sales department or the salesperson that is closest to the customer, as it was in the past.

Now it is the marketing department that has the clearest insight into the customer. In a 2012 blog post, Forrester Research analyst Lori Wizdo wrote that a potential customer can be 90 percent through the buyer journey before contacting a vendor. Prospects can research in solitude online via Google, by consulting LinkedIn groups, by browsing product reviews online, and by anonymously visiting vendor websites.

Through new software and platforms, marketing teams can see what Steve Woods, former Eloqua CTO and now the CTO of Nudge, calls the customer's “digital body language.” By knowing what part of the website a prospect has visited, what e-mail newsletters she has signed up for, and what white papers she has downloaded, the marketing department understands where that prospect is in her buyer journey.

All of this points to the marketing department owning the customer life cycle and the customer relationship in the digital age. This fact will lead to the growing responsibility of the marketing department. Marketing has not always been thought of so highly. It has been derided as “the toy department,” a part of the company that often had to beg the financial department for money to create branding ads. Using marketing automation tools, marketers are better able to identify which specific marketing tactics are generating return on investment.

Meagen Eisenberg, vice president of customer marketing at DocuSign, was recently asked if the Wanamaker remark about advertising and its effectiveness still had relevance. “I definitely think the quote is obsolete,” Eisenberg said in a 2013 Digital Marketing Remix webinar. “When it comes to online marketing, I feel confident that the metrics, tracking, and technology we have today can prove what spend is working and what spend is not.”

Marketing automation platforms such as Eloqua now enable marketers to quickly assess whether their branding and nurturing programs are driving conversions and generating revenue.

Use of a well-informed marketing stack is making the marketing department a more effective part of the business. Because of this increasing importance of software to the marketing department, Gartner analyst Laura McLellan projected in 2012 that the chief marketing officer (CMO) will spend more on information technology (IT) than the chief information officer (CIO) by 2017. Many CMOs, such as Motorola Solutions' Eduardo Conrado, already oversee both marketing and IT. This dual role of the marketing department reflects how central marketing data is becoming to the financial health of businesses.

Not only will the role of marketing become more critical for corporations, but former CMOs will be front and center in the next crop of great CEOs. This trend is already taking shape: Royal Dutch Shell, Audi, Mercedes-Benz, and others have all recently named CMOs as their chief executives. The movement of CMO to CEO is inevitable since other executives don't have the same amount of power to understand and solve customer problems, create brand loyalty, or move shareholder value today as quickly or as effectively as the CMO.

Marketing will no longer be defined by John Wanamaker's rather helpless-sounding quote. Instead, management guru Peter F. Drucker said words that we think are now more appropriate: “Business has only two functions: marketing and innovation.”

We would add, “…and both of them will be led by the CMO.”

In the marketing department and elsewhere, every corporation in the world is using big data to some degree. The winners will create cultures that embrace big data, employing data scientists to analyze data and draw conclusions that may contradict the company's assumptions, and to take action that takes advantage of the truth that the data reveals. The losers will use data to reinforce their own erroneous conclusions. This is not speculation. It is happening right now.

BlackBerry, which had what appeared to be an unassailable market share in smartphones, particularly among enterprise customers, initially shrugged off the launch of the Apple iPhone. But BlackBerry was ignoring the data. BlackBerry described the iPhone as a niche product aimed at consumers, but the revolutionary phone from Apple was making huge inroads not only with consumers but with BlackBerry's core business customers. Nonetheless, BlackBerry thought its enterprise dominance made it safe. But by the time it became widely known that the iPhone, with its touch screen and excellent Internet access, was a serious competitor for business customers, BlackBerry was steamrolled by Apple's momentum, even losing its place in the enterprise to Apple and Google's Android. While many say the iPhone was a product created by Steve Jobs's magical intuition, Apple understood—from up-to-the-minute data being gathered by its proprietary retail stores and from its success with the iPod—that consumers would pay for a product that would combine their iPod with mobile phones and with the Internet access of their laptops.

In the end, both BlackBerry and Apple had access to data about the marketplace. One company, however, didn't have the culture to take advantage of the data. The other did, and it was the company that triumphed.

Of course, it's easy to say in retrospect that the data pointed to Apple's success and BlackBerry's demise. So we'll point to an industry that is ignoring the data that points to a systematic erosion in its business: the cable and satellite TV sector. Two “I Want Media” tweets, released seconds apart on Twitter, reveal the level of denial. The first tweet: “Report: Pay TV loses 113,000 Customers in Quarter.” The second tweet: “Dish Says Too Early for Web to Challenge Pay TV.”

Where are those customers going? They're going to YouTube, and they're going to Netflix, Hulu, Roku, and other alternatives. And if Dish and other pay TV companies are in denial and not addressing the problems the data is telling them quite directly, they are in for the same fate that befell BlackBerry.

The companies that create a culture that has intense focus on the customer through data, that values analyzing data, that is open to the truths data analysis reveals, and that has the guts to act on those conclusions will be the companies that prevail. The benefits of big data are available to any company, of any size, in any industry. Establishing a system that gathers and analyzes the data being generated by customers will deliver insights and reveal opportunities that you can't realize in any other way. History shows that competitive advantage and outsized shareholder value will follow.