The financial sector is also sometimes called the BFSI sector; that is, banking, financial services, and insurance.
- Banking includes retail, corporate, business, investment (including capital markets), cards, and other core banking services
- Financial services include brokering, payment channels, mutual funds, asset management, and other services
- Insurance covers life and general insurance
Financial organizations have been actively using big data platforms for the last few years and their key objectives are:
- Complying with regulatory requirements
- Better risk analytics
- Understanding customer behavior and improving services
- Understanding transaction patterns and monetizing using cross-selling of products
Now I will define a few use cases within the financial services industry with real tangible business benefits. In subsequent chapters, I will pick up some of theses use cases in a little more detail.
Archiving data on HDFS is one of the basic use cases for Hadoop in financial organizations and is a quick win. It is likely to provide a very high return on investment. The data is archived on Hadoop and is still available to query (although not in real time), which is far more efficient than archiving on tape and far less expensive than keeping it on databases. Some of the use cases are:
- Migrate expensive and inefficient legacy mainframe data and load jobs to the Hadoop platform
- Migrate expensive older transaction data from high-end expensive databases to Hadoop HDFS
- Migrate unstructured legal, compliance, and onboarding documents to Hadoop HDFS
Financial organizations must comply with regulatory requirements. In order to meet these requirements, the use of traditional data processing platforms is becoming increasingly expensive and unsustainable.
A couple of such use cases are:
- Checking customer names against a sanctions blacklist is very complicated due to the same or similar names. It is even more complicated when financial organizations have different names or aliases across different systems. With Hadoop, we can apply complex fuzzy matching on name and contact information across massive data sets at a much lower cost.
- The BCBS239 regulation states that financial organizations must be able to aggregate risk exposures across the whole group quickly and accurately. With Hadoop, financial organizations can consolidate and aggregate data on a single platform in the most efficient and cost-effective way.
Fraud is estimated to cost the financial industry billions of US dollars per year. Financial organizations have invested in Hadoop platforms to identify fraudulent transactions by picking up unusual behavior patterns.
Complex algorithms that need to be run on large volumes of transaction data to identify outliers are now possible on the Hadoop platform at a much lower expense.
Stock market tick data is real-time data and generated on a massive scale. Live data streams can be processed using real-time streaming technology on the Hadoop infrastructure for quick trading decisions, and older tick data can be used for trending and forecasting using batch Hadoop tools.
Financial organizations must be able to measure risk exposures for each customer and effectively aggregate it across entire business divisions. They should be able to score the credit risk for each customer using internal rules. They need to build risk models with intensive calculation on the underlying massive data.
All these risk management requirements have two things in common—massive data and intensive calculation. Hadoop can handle both, given its inexpensive commodity hardware and parallel execution of jobs.
Once the customer data has been consolidated from a variety of sources on a Hadoop platform, it is possible to analyze data and:
- Predict mortgage defaults
- Predict spending for retail customers
- Analyze patterns that lead to customers leaving and customer dissatisfaction
Sentiment analysis is one of the best use cases to test the power of unstructured data analysis using Hadoop. Here are a few use cases:
- Analyze all e-mail text and call recordings from customers, which indicates whether they feel positive or negative about the products offered to them
- Analyze Facebook and Twitter comments to make buy or sell recommendations—analyze the market sentiments on which sectors or organizations will be a better buy for stock investments
- Analyze Facebook and Twitter comments to assess the feedback on new products
Big data has the potential to create new non-traditional income streams for financial organizations. As financial organizations store all the payment details of their retailers, they know exactly where, when, and how their customers are spending money. By analyzing this information, financial organizations can develop deep insight into customer intelligence and spending patterns, which they will be able to monetize. A few such possibilities include:
- Partner with a retailer to understand where the retailer's customers live, where and when they buy, what they buy, and how much they spend. This information will be used to recommend a sales strategy.
- Partner with a retailer to recommend discount offers to loyalty cardholders who use their loyalty cards in the vicinity of the retailer's stores.