The overall strategy of Big Data implementation in large organizations depends on the particular needs of the organization. Today, there are hundreds of options to choose from between Big Data, data science, machine learning and, of late, AI providers.
As such, there are two main considerations while implementing Big Data in large organizations:
- Technical: The selection of the proper software and hardware stack
- Operational: Management of the organizational data, creating a formal data governance strategy, and creating an adequate data management framework
Apart from these, hiring the right talent and possibly creating well-defined roles for the company's Big Data/data science implementations are additional but equally essential tasks.
Some key questions in the creation of such a strategy include:
- Is the software/hardware licensing based on size or cores? If it is based on the size of data and my data size increases, what will be my 3 year/5 year cost?
- Does the solution have enterprise support?
- Do we need to hire external resources?
- What business questions will the new capabilities answer?
- Have we done short and long-term cost-benefit analysis?
- What are the present unmet needs of the organization that the new solutions can answer?
- Is the solution scalable enough to meet my potential future needs?
Lastly, as Big Data/data science is constantly evolving, the long-term scalability and adaptability of the solution needs to be properly evaluated. The cloud-based option should be considered in light of the fact that it provides an efficient medium to access and use new and emerging solutions in an easy and affordable manner.