Chapter 2. What Is Agility?
Agility is the power of moving quickly and easily. An agile data system can be defined as a nimble system that can ingest any source data in days or weeks, not months or years. Organizational agility is a huge competitive advantage that positively affects the ability to make strategic decisions and serve users or customers. We can all see the advantages of agility on a daily basis. Whether we’re buying a house, extending a job offer, or working on implementing a centralized data hub for our organization, agility is often the difference between success or failure. With agility comes the freedom to make strategic decisions without having to consider the costs of major disruptions.
Agility and Data
Data is a fluid thing, oftentimes changing at such a rapid pass that we struggle to keep up. As an organization, we must deal with the many different aspects of data, such as the following:
-
Constantly changing content
-
Processing new data formats and schemas
-
Exposing data based on evolving requirements and use cases
-
Synchronization of data across disparate systems
Being Agile
In technology, we toss the word “agile” around quite a bit. For example, we have agile databases, agile software development, agile architecture, and the list goes on. Because of this, we might have some preconceived notions of what agility is. But, in terms of data, what does it mean to be agile, and what might the consequences of not being agile be?
When we fail to be agile
We have all been in scenarios in which we are using software or IT systems that are brittle, inefficient, or even deficient and wonder to ourselves, “Why are we doing things like this?” Most often, the answer goes something like this: “We’ve spent a lot of time and money on System XYZ, and we can’t afford to fix it, so we’re going to keep using System XYZ!” This type of scenario demonstrates why agility in technology and, specifically, data-related systems is critical. If we were only able to develop systems that were as dynamic and agile as our day-to-day processes and problems, we could avoid situations like this.
When we are agile
When data systems are created with agility as a core requirement, you increase your likelihood of experiencing the following:
-
Overall lower cost of ownership
-
Reduced costs associated with system maintenance
-
Increased ability to adapt to changing environments
-
Increased ability to serve users what they need, when they need it
-
Increased ability to meet the overall needs of an organization
-
Increased “shelf life” of the IT system
The multi-model database ensures that our data systems will start off and remain agile as system requirements are added. Data agility and the multi-model database go hand in hand, providing the ability to bring in data sources and models as is and providing mechanisms to query them in a single backend.
Dangers of Inflexible IT Systems
Inflexibility is a breeding ground for complacency and failure. As organizations recognize the cost of updating inflexible IT systems, they will usually do one of two things:
-
Come up with hacky workarounds that ultimately lead to downstream issues like increased complexity, data quality, and governance issues.
-
Accept the shortcomings of the IT system and become complacent and stagnant.
Neither of these approaches is good in the long run, and taking one of them will lead to failure.
As an IT system becomes more complex, it’s inflexibility becomes more obvious and generates more resistance to change. If an IT system cannot easily adapt to changes in use cases and requirements, you might as well shelve it even before implementation begins.
What You Lose by Not Being Agile
Your organization’s data is a treasure trove of potential knowledge. More often than not, an organization’s data will never reach its true potential. For example, as new sources and formats of data come into an inflexible IT system, they become lost in the noise. With no means of processing and accessing these new sources of data (at least not without developing the functionality to process and access it), they are left untouched and unknown to all potential beneficiaries. By not being agile, you end up losing that ability to quickly bring in new data to be processed and accessed. With that, you also lose any potential benefit that data has to offer.
When Agile Creates New Problems
An agile data system means that many different types of data must be handled. Data from disparate sources with different structures will need to be accepted and stored. Being able to bring in many different types of data, as a multi-model database allows, is only half of the data agility battle. As we bring in all of this disparate data, issues such as data quality, governance, data duplication, and stale data must be addressed. If we look at Hadoop, for example, it was embraced for its ability to mix many different data sources. What Hadoop does not address, however, is how you manage and search that data afterward. The multi-model database takes data agility a step further than Hadoop and addresses both centralizing multiple disparate data sources and managing and searching the data. Having multiple data sources in a single backend, as opposed to integrating many databases with many backends, the multi-model database makes it easier to handle these types of data issues.