At this point, you have now created a backup and performed a restore of the example database. You have also covered the different methods to back up and restore the database. However, there is no real plan for regular maintenance and detection and reporting of problems in the database strategy. Any jobs for backup of the database or transaction log that I have demonstrated so far are held as single units of work called steps. Not only that, but also there is nothing in place that will look after the data and indexes held within the database to ensure that they are still functioning correctly and that the data are still stored in the optimal fashion. Without a process that runs regularly, you would need to perform all of this by hand regularly and check the results each time. What a waste of time, and boring to boot!
This chapter will demonstrate building an automatic maintenance plan and then checking on the plan after it has run to ensure that all has gone well with it. This plan will perform regular backups and checks on the database and keep it in optimum health. This section will then show you how to set up the ability to e-mail results.
To do this, you will use the Database Maintenance Plan Wizard, which will help you build an automated job to monitor corruption within the database, optimize how the data are stored, and back up both the database and the transaction logs. Finally, the wizard will generate a schedule for the job to process at regular intervals. You will also see how to set up and configure SQL Server Database Mail so that errors can be mailed to a team monitoring the servers.
Some areas of this chapter, like the backup screens, are straightforward, as they were covered earlier in Chapter 7; however, this now brings the whole maintenance of the database into one job.
Now that the database is up and built and the tables are there, it really is time to start considering a whole database maintenance plan before data are entered. This will cover securing your data, ensuring performance, and rectifying database corruption through to ensuring that your database is performing as you expect it to be. Even though corruption is rare in SQL Server, it can be caused when SQL Server loses power abruptly, for example, or through hardware issues such as a motherboard failure or someone removing the network cable.
There are many areas of building a maintenance plan, and this section covers a lot of them. One or two areas are only touched on, as they are quite advanced and will not be covered in this book. You will still need a little background so that you can see how crucial this area is, and you can move on to those more advanced areas a bit later on.
A single maintenance plan can be built for one database or several databases, but I recommend one plan per database. A single plan can be set up for system databases and all user databases by selecting those options at the start of the Database Maintenance Plan Wizard. I also recommend one plan for each system database as well. The logic behind these recommendations is that each user database will have its own needs, its own overnight routines, and even its own people for callout when things go wrong, even if there is some sort of logical link between databases. Even if you are a one-man band, each user database should still have a maintenance plan. Therefore, in keeping with this, only the ApressFinancial
database will be selected.
Once the plan has been built, it will be stored within SQL Server, but will have been built as a SQL Server Integration Services (SSIS) job. This is a technique within SQL Server for running several tasks in sequence with conditions, which also has the ability to work with errors that occur. SSIS could (and does) take up a whole book in itself, but building the plan and seeing what is generated will demonstrate the very basics of what it can achieve.
Maintenance plans use extended stored procedures, which are disabled by default. This means that until extended stored procedures are enabled, it is not possible to build a maintenance plan. To demonstrate how you can enable this option via T-SQL, you will use a system stored procedure in the first step to do this.
There are two methods to building a database maintenance plan: you can either use a wizard or select options from a multitabbed screen.
There are a number of tasks that can be included in a maintenance plan:
Let's take a moment to look at options that are likely to be the most useful aside from the backups—the reorganization and rebuild index options. If fragmentation is low—below 30%, typically—then you should perform a reorganization. If fragmentation is greater, then it is best to reindex. Indexes exist on tables and views and become more fragmented than whole pages of data on a high-data modification system. Like rebuilding indexes, reorganization of indexes should be completed on a regular basis, probably weekly; however, if your batch window allows you to perform this more frequently, then look to do so. When rebuilding indexes, you can define with a certain amount of free space to allow for increases and mid-index insertions on each index page. This is a bit like inserting lines of text in a book. If you think you are going to do this, then leaving gaps at the end of the page allows for these rows to be added. Failure to leave enough means shuffling data from that page through to the end of the book.
A maintenance plan can run within a SQL Server Agent job either as an ad hoc process or on a scheduled basis. Depending on your overall application design, or even on your company standards, it may be that you have dependencies on when a database maintenance plan can run. To expand on this to clarify my point, it could be that you have processes in the overall application that load data into your database. In ApressFinancial
, this could be further details about shares, such as details of new stocks or stock actions. In corporations, it could be data from other systems, such as the orders system informing the inventory system a delivery of raw materials has been made. These processes would generate some sort of triggering action to indicate that they were complete and the database could start its maintenance. Over the years, I have used software called AutoSys, owned by Computer Associates, which schedules these jobs, and when a specific process completes I use this completion to start my maintenance task. Therefore, depending on how the whole application has been designed, a scheduled maintenance plan may be the correct method or an ad hoc/on-demand plan should be generated. It is also very easy to switch between these two, so it is not a decision that you are stuck with.
If a plan is scheduled, it should be timed for an appropriate time for the application. This could be at a quiet time of your application or at specific times within any batch processing, but, depending on what you select, a maintenance plan can be quite labor-intensive, such as rebuilding indexes and reorganizing your data. You could set up a plan to run before batch processing, at the end, or even at both ends of the schedule. Depending on the actions in the plan, it is also possible to alter the scheduling to occur when the CPU is idle. This option is ideal for heavily used, large-volume, high-performance databases where no fixed time can be defined to run the schedule, or even if you wish the process to run multiple times in a day when the server can execute the plan. One scheduling option that can be very useful is to run when SQL Server Agent starts. If a server automatically reboots, as part of a scheduled weekly reboot, or if the reboot occurs due to a Microsoft patching installation, then you may want to have a plan that takes a backup of the database, or a plan that verifies the integrity of the database post-reboot. You would create a maintenance plan to complete those tasks and use the SQL Server Agent schedule option to accomplish this.
The main option of interest when rebuilding indexes is Sort Results in tempdb
. You could be low on disk space because when you built your database, you set it to grow no larger than a specific size. Couple this with a situation in which your indexes are so fragmented that they take up more space than they will post-defragmentation. This could be because you have a large number of gaps due to deletions of rows within the index or modifications on a clustered index causing rows to be moved. When rebuilding indexes, this would by default be completed within the database the rebuild is for. The “old” indexes are kept until the new indexes are built. If there is not enough space to store them, it would not be possible to rebuild the index, unless you physically increased the size of the database. This is not a simple process. Therefore, by using the option to rebuild the new indexes within tempdb
, you do not need to increase your database size. Also, tempdb
will not be used as intensively. Therefore, it might also be faster to rebuild your indexes within that database. This is an option you may use a great deal.
A maintenance task can also be synchronous or asynchronous where you have different tasks running at the same time. This has the potential to make the maintenance plan complete in less time, but you have to be careful that the right tasks run in the right sequence.
In the following exercise, I am going to demonstrate building a scheduled maintenance plan via a wizard.
TRY IT OUT: CREATING A DATABASE MAINTENANCE PLAN
You now have successfully built a maintenance plan. You may have several plans, one for a weekend, one for pre-batch, etc., all dependent on how you want to maintain your database. The important point is that you backup your database.
The next step is to execute the plan.
Now that a plan has been built, the next step will be to execute the plan. For a maintenance plan to execute, SQL Server Agent needs to be running. It is likely at this point in the book that it is not, as it will be defined to start manually rather than automatically. This default exists for security reasons so that jobs cannot be built, deployed, and then executed on your server without your prior knowledge. You may find that some corporations keep SQL Server Agent stopped until required. It is possible to start SQL Server Agent within Management Studio for ad hoc starts of the process, or you can start the service from Administrative Tools within the Control Panel. It is also possible to start SQL Server Agent via a command prompt, therefore allowing you to put the start and stop into scheduling software.
In the following exercise, you will see how to start SQL Server Agent and then execute a maintenance plan manually.
TRY IT OUT: EXECUTING A DATABASE MAINTENANCE PLAN
It would be better to be notified about jobs that have failed rather than having to monitor each job as it processes manually. One of the ways to do this would be for the maintenance job to mail out results of its work. It is possible in SQL Server to use SMTP mail to send out notifications from your server. This means that you don't need to install a MAPI client, such as Microsoft Outlook, because SMTP processing is contained within a Windows install. The fact that you don't need another product on the server has benefits in terms of making your server more stable. To supplement stability, SMTP runs in a process outside SQL Server. Even if there are SMTP issues, SQL Server should not be affected if the SMTP process stops for any reason.
On top of this, SQL Server comes with built-in system stored procedures and functionality to use SMTP to send out mail. These features are collectively called Database Mail.
The use of Database Mail has many positives for error notification, but it is not the total solution for monitoring SQL Server or even the underlying server. The most extreme problem is that the server crashes, reboots, uses 100% processing power, and runs out of memory, among other server-specific problems, all of which would stop an e-mail being sent. Other tools need to be deployed on the server that are either third-party-based or where you can write your own using .NET, Java, etc. The very basic premise of these tools is that they “heartbeat” at regular intervals, in other words, transmit at regular intervals notifications to a central separate server that the Windows server is alive and performing well. The tool will have performed the necessary health checks about the server and notify via the heartbeat if it notices an issue arising to the central server. The central server looks out for problems, and then it notifies the necessary person, who can then react to the issue. This tool is generally standardized company-wide and not on a per-server basis.
This section will cover only how to set up and test Database Mail and will not cover the monitoring of the Windows server. You will see how to implement Database Mail within the maintenance task created in the preceding section.
TRY IT OUT: SETTING UP DATABASE MAIL
At this point, you have created a plan using the wizard, and you now have a working Database Mail account. In this section, you will apply the account to the maintenance plan and send the mail when the plan runs.
It is possible to modify the plan and add or remove steps, change the description, or add further subplans. The maintenance plan uses SQL Server Integration Services, which is commonly known as SSIS, to perform its duties. SSIS uses steps shown in a flowchart to demonstrate which items are being dealt with. SSIS is outside the scope of this book, so you won't be looking any further at modifying the plan by drilling into it to see the steps. Instead, you will alter only the reporting method. However, SSIS is an interesting area, especially if you want to move to the analysis of data via Business Intelligence and working with data in cubes. SSIS is used heavily in this area to load the initial data into a SQL Server database, which is then used to build a cube of data that is then “sliced and diced” to look at the data in many different ways. But if you are interested in using SSIS to work with your data, then I recommend the following books:
TRY IT OUT: SETTING UP DATABASE MAIL
Having a positive, regularly run maintenance plan is required to ensure your database is working correctly and performing as well as it has been designed to do. Ensuring that errors are found and dealt with by checking the database integrity, that statistics are as up to date as possible, making sure that indexes are reorganized and ensuring that errors are logged and correctly reported will avoid problems occurring, as errors may occur at the worst possible time in the life of the corporation.
As part of this maintenance plan, it is also necessary to be notified of problems so that you're aware of them and can deal with them quickly and efficiently. A database mail account is perfect for this, as it can notify you of problems in a process out of SQL Server.
18.224.59.145