Relational, Graph, NoSQL, Columnar, InMemory, Warehouse, and Analytical databases are all part of the database administrator’s job. The database platforms continue to grow and develop. Even with many relatively new databases now part of the development and production environments, the relational databases still hold a large market share and purpose.
With the collection of even more data on a daily basis and new uses in companies for data-driven results, the various databases serve in the processes and system. Data growth is one constant for the future. With all of the social media channels, smartphones, and Internet of Things (IOT), data is predicted by several different sources to grow to over 100 zetabytes in the next 5 years. This is just an increase from just under 10 zetabytes in 2015. Not to mention that there are advances in technology that can even drive this higher. Artificial Intelligence and plenty of IOT sources might even make this more than predicted.
Quote
“The Data Warehouse is far from dead, it is the core of any business reporting system, but it must adapt faster. We generate significant data through our business applications which feed the traditional data warehouse. Businesses will adapt their data warehouse architecture and finally see a widespread use of Hadoop for landing data.” Ian Abramson, Predictions, IOUG Select.
The Data Warehouse is far from dead, as it is the core of any business reporting system, but it must adapt faster. We generate significant data through our business applications that feed the traditional data warehouse; 2017 will see businesses adapt their data warehouse architecture and finally see a widespread use of Hadoop for landing data.
Can you even how much data is going to be in your data center or cloud databases in a few years? The database professional does have their work cut out for them to maintain these very large environments. The data and information business is going to be booming and bringing in significant dollars to the industry. The business of data is not going away anytime soon, and there will be a role for the guardians of the data.
In Figure 3-1, the chart shows the growth of data coming from mobile as forecast by Cisco Virtual Networking Index. The data is planned to grow to 49 Exabytes per month from currently around 11 Exabytes per month. This could be the result of more video and pictures, but it is data that will be coming from devices that can find itself in databases.
Figure 3-1 Forecast Monthly Mobile Data Traffic . Source: Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2016-2021
Additional interesting information coming out of this forecast from Cisco VNI is the number of smart and non-smart devices and connections. The number of non-smart device connections is decreasing even though they continue to grow in the forecast. However, they will still be growing year over year by significant amounts. Figure 3-2 shows the chart with this information about the number of smart devices and non-smart devices. With the increase in devices and connections, the data will continue to growth. Through online shopping, photos, and other social media, the data growth is supporting the importance of the database.
Figure 3-2 Forecast Monthly Mobile Data Traffic Source: Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2016-2021
Technology
As seen with the development of the smart devices influencing the forecasts in the mobile devices, other technology is affecting how we manage databases. Virtualization changes how we look at resources and servers. Provisioning of new resources is more transparent and available with growth or peaks of services. Server technology continues to advance and using it with virtualization allows for using these new technologies.
Storage
Oracle Exadata engineered systems provide servers that are specialized for Oracle database workloads and configurations. These engineered systems are a good example how the technology is keeping up with the database, high availability options, and features. The performance with the storage and configurations allows for the databases to run as they are designed. Memory options and parameters are working for the In-Memory database of 12cR2 and other configurations bring in the new features as they also enhance with each release of the servers, either with upgrades or new components. This actually limits some of the intervention from the DBA and the server hardware. Supporting these engineered systems is a task that a Database Machine Administrator (DMA) would carry out. We will discuss the DMA in Chapter 4.
With the amount of storage, the hardware is critical to make sure to be staying up from the latest trends and technology to improve speeds of disk operations . Solid State Disks (SSD) are going to be what is being used along with flash. The spinning disks are going to be a thing of the past, and new innovations will allow for more data in memory and faster data access.
Data
The type of data is changed too. With Augmented Reality (AR), Virtual Reality (VR), and Artificial Intelligence (AI) videos and pictures, larger datasets are being accessed and stored. With AR and VR traffic, data access per month will be reaching 160 Pedabytes up from 16 PB, according to the Cisco VNI Forecasts. AI might be the unknown as many are just touching the surface with the ability to make decisions on very, very large datasets. It can surpass the forecasts if there are new opportunities in this area.
Data collected from wearables, sensors, and other IOT devices are going to collect data and expand the amount of data crossing through the data center. Even if you think about how much data you currently use on your mobile devices each month and how that has changed over the past couple of years, just imagine how quickly that will change in the next few years. It will definitely be well over the 4GB mark, unless you have teenagers, which make it soar above 10GB very quickly.
The devices are advancing with the wearables and what is being collected, costs for points of entry decreasing, and the technology for other devices are going to play a big factor on how quickly the amount of data grows and changes the forecasts. Technology in the data center will need to be enhanced at speeds to keep up with the demand. The life cycle of devices is changing from what it was several years back to be five years, down to three, and probably now between two and three years. The technology and devices are keeping up with the business and end users.
Technology is driving both sides of the equation with devices that collect the data, and the servers, storage, network and other components to support the business with the data that flow through. Databases are going to store the necessary data for business processes and day-to-day information as they also need a way to process data and the workflows from Big Data streams to integrate with the systems.
I might be getting ahead of myself in talking about the data streams and integration as this is a place where DBAs can transfer into, but the technology of the devices and support compounds the data growth and the opportunities for the DBAs. This is going to be considered as several technology game changes for all kinds of information that is available.
Disruptors
Disruptors are innovations that potentially create new markets and new value. Therefore, they are disrupting the current markets and values in the market. It is not just something shiny that grabs our attention for a moment but it is something that disrupts, interrupts, and moves us in a different direction because of that advancement or innovation that can’t be ignored.
Technology
Technology is a big disruptor and will continue to be going forward. We just finished talking about how technical devices and how they are going to continue to provide more data and information. These are a disruption to the database environments as there are capacity plans that are extremely different now than they were a couple of years ago and will be very different in the future. Accounting for the new data market, the IOT data collection is going to add to business value as long as we incorporate how we are handling the data disturbance and manage these changes.
Social Networking
Social Networking is a disruptor as businesses are using these forms of communication as a marketing outlet, or information gathering opportunity. This is now becoming part of the plans in the business, but they might not have been included before. With the younger generation using this as their way of communication, this will also disrupt the business and how they not only market to them, but how they also prepare for them to join the workforce.
Security
Security is a disruptor in that it causes current progress on projects to stop and deal with the situation at hand. It might be a new zero-day exploit or a new security patch that has been released and should be dealt with as soon as possible. Breaches cause disruption in projects as they might require an assessment if the right controls are in place and how the environment is handling the possibility of a breach and reducing the risk. Innovations on the hacker side of the equation is not a positive outcome for the market but it will definitely cause a disruption in how it is being addressed and the risk involved. Much more will be covered in Chapter 6 on security, but understand that it is going to be a disruptor now and in the future as the security landscape is going to be one that is always changing.
Virtual Reality, Augmented Reality, and Artificial Intelligence
Mentioned in the Technology Section, VR, AR, and AI are all disruptors as there is constant innovation happening here. Robots that can now perform tasks to provide products more efficiently to the market add value. AI is starting to open new doors on how devices and behave and react, possibly even creating new environments that we might have dreamed about or only have seen in Sci-Fi movies and TV shows. We actually look at some of the movies and think, could that actually happen? Is that real? Self-driving cars are already out there being tested, but what is the next technology disruptor coming our way? It is definitely fun to think about how we might be working in a few years. Already there are plenty of companies that opt for working at home possibilities because of cost savings and technology to support this type of environment. It would be amazing to see VR play a part in going into the office in a way to speak with our coworkers and avoiding the commute.
Why are disruptors even more important now to database environments and the transformation of DBAs? Databases are normally considered a sustaining technology. Yes, it has changes and upgrades, but these are incremental improvements and efficiencies that normally take place. They are not always the innovations that are changing the direction, markets, or value. However, as described above, these innovations in technology and other areas are creating the interruptions to normal business and need to be addressed sooner rather than later. As DBAs we need to evaluate these disruptions in our database environment. Is it going to add more value? Do I need to change the way I am doing things to address it? These honest questions are going to be more than just a simple change. It will be changing and causing disruptions of our current projects and processes if we look at all of the innovations in technology and embrace those in the database systems.
We will see even more disruptors and many of these will influence the database arena. It will change how and what data is collected and the growth. These systems need to adapt and incorporate as needed by the business. This is just the beginning of some DBA transformations as more will happen with new disruptors.
Many Platforms
DBAs support multiple platforms of databases. There are many enterprise-level databases that include types, like relational, key-value and column stores, and graph databases, each with a purpose. The market share of these databases has changed slightly over the past 10 years, but relational databases still make up about 89% of the enterprise market share. Figure 3-3 shows the market share of each of the database platforms based on Gartner and other market research from 2016.
Figure 3-3 Enterprise Database Market Share
It is not just Oracle, but MSSQL, MySQL, DB2, MongoDB, PostgreSQL, Hadoop, neo4j, and other platforms that will be found in the enterprise supported by DBAs. New databases are developing as part of cloud services, and the data that is being collected by devices. For example, Amazon Cloud database, SimpleDB, recently emerged as part of the cloud services. DBAs are needed to administer the various databases, and help plan which workloads are appropriate for the type of database. Administration changes for the different platforms and integration between the platforms will pose different issues and challenges.
Future needs of databases are not just based on data needs but can be on application development needs. According to a StackOverflow survey in 2017, developers have ranked databases in order of preference to work with and MySQL is the most popular database. SQL Server, PostgreSQL, and MongoDB are next on the list. This is not to say that the developers have the ultimate choice for the databases, but the reasons they are popular are for ease of use and to work without too many blockers. A self-service model would influence a preference among other factors.
Database platform no longer matters to the DBAs, as they are responsible for managing and administering several different platforms. Understanding the various databases types and when best to utilize them will provide the input needed for design decisions.
Database as a Service
Database as a Service (DBaaS) standardizes the database installation, configuration, and monitoring. DBaaS puts in the needed configuration with security, compliance, and management tools in place with self-service provisioning. It makes the environment scalable with managing many databases as one because of the standardization. DBaaS has database containers that are deployed for the application users. It keeps production and development environments in sync with the same containers and provides services to manage and do tasks and development as needed. DBAs are needed to provide input into the containers for the company to meet policies and expected compliance, but as the containers are constructed and implemented, the DBAs become more focused in the application space to provide management of user objects, performance tuning, and application development.
The containers are developed in virtualized, Docker, Ansible, and other containers. This is a growing trend to use these tools and as this provides a consistent deployment of the database software, database, and configurations. It is a measured service that allows for resource pooling for providing for growth and scalability. Figure 3-4 shows the design of this model, using Application Program Interfaces (APIs) for providing actions that can be performed against the containers. APIs support the on-demand service of the databases to run without additional interaction. Since the databases are deployed and created the same way, it is simple to measure and understand usage. This supports other important parts for database environments such as capacity planning and monitoring.
Figure 3-4 Design and Advantages of DBaaS
APIs are going to be an important part of any database in the future. This is the way to interact with databases in DBaaS and cloud environments. It is a standardized way to get data in and out of the database and perform actions and management items. As we will discuss about the Cloud DBA, there are ways to work on developing the APIs or using the APIs to integrate to other environments. The APIs make it so that the databases can be managed as one with the consistent access to all of the environment.
DBaaS can be considered a private cloud environment as it fits the same components of delivery and offers the service to deploy database containers. The future of the database will have databases in containers to deploy either in the enterprise data center or in the cloud. According to a 2016 IOUG Survey on Database Cloud, the growth rate of DBaaS adoption is to triple in the next couple of years. The survey focused on how DBaaS is becoming mainstream and the Oracle Multitenent is providing the required structures to use the containers. DBaaS and Cloud database will probably become interchangeable going forward, understanding that there are several services available that will be part of the offerings that are in public and private cloud environments.
Databases in the Cloud
The major enterprise database companies are investing in the area of the cloud. Companies find value in using the cloud. It might be using the software and applications that are provided, or even the infrastructure to support public-facing applications. It is also possible that cloud supports development environments to cut costs in the test and lab areas. As seen in Figure 3-5, the RightScale 2016 State of the Cloud Report confirms that most companies are already invested in the cloud, both public and private.
Figure 3-5 Use of Public and Private Cloud in the Enterprise. Source: RightScale 2016 State of the Cloud Report
Understanding that database containers (DBaaS) will support the significant growth of cloud environments is vital to the transitioning DBA. Amazon with the SimpleDB has invested in the space of storing data in the cloud, Microsoft and IBM are doing the same with offerings in the cloud and in NoSQL databases. Oracle is building on its cloud service. Flexibility and cost seems to be what they are talking about as Larry Ellison, CTO at Oracle, announced recently that Oracle offers an on-demand service from applications to infrastructure including database in between. The same feature of the Oracle database 12cR2 includes InMemory, Security, and using the Oracle engineered machines on the back end. He talks about simplicity and how it is easy to manage the consumption of the service. The autonomous database that will be completely self-driving is coming to the Oracle Cloud and so will automation of the database with less interaction from the administrator or others. Talk about a disruptor for the DBA and how we manage databases. This is where the transition of the DBA to different areas or even cloud DBA will be the direction. We will cover the Cloud DBA in Chapter 6. Here it is important to understand that with the database providers’ investment in the cloud and companies shifting workloads there, the cloud is here to stay. Figure 3-6 shows the transition starting in 2011 and predicting future growth in the IOUG Cloud Survey.
Figure 3-6 Shift into the Cloud. Source: Databaseas a Service Enters the Enterprise Maistream, 2016 IOUG Survey on Database Cloud, April 2016
The cloud offers Infrastructure as a Service (IaaS) , Platform as a Service (PaaS) , Software as a Service (SaaS) , and I am sure there will be more services to come. It does appear that there will be less for the DBA to do in the database with the containers and automation and with initialization of the containers and which service needs to be implemented. There are several areas to consider and design questions to ask for migrations and management in the cloud.
Figure 3-7 Cloud Providers. Source: Synergy Research Group
The competition of the cloud providers will continue as they innovate and create services that will support customers and their business needs. Figure 3-7 clearly has AWS growth rates far ahead of the others, but each one does have various offerings that should be considered. The database transition to the cloud will continue and DBAs need to be ready to administer these databases with APIs and other tools provided. It is not going to be the same tasks, and looking at the data integrations or other aspects will be a direction to consider for future opportunities.
Conclusion
The future state of the database brings more data than even imagined 20 years ago when large databases were 50 GB in size. Cloud is the future direction and we already need to be planning for our roles. We will continue to have complex environments in dealing with several database platforms, but we can navigate the knowledge and skills that we have DBAs to embrace the future of the database as it is also transitioning.