Chapter 7. I/O and Persistence

IN THIS CHAPTER

• Introduction to Streams
• Introduction to Basic File I/O
• Using Asynchronous File I/O
• Working with Isolated Storage

In this day and age of enterprise applications and smart clients that get their data from large relational databases like SQL Server and Oracle, it is often easy to overlook the fact that we still use the underlying file system for something other than storing shortcuts and documents.

We use files for numerous tasks, including storing configuration settings, images, icons, textual data (such as comma-delimited files from other sources like mainframes), file-based databases such as Microsoft Access, Excel spreadsheets, and even XML data.

This chapter shows you the basics of working with files using the .NET Framework, including how to work with streams, the basic unit of input/output (I/O) used throughout the entire framework. Finally, you’ll see some more advanced file I/O techniques such as asynchronous I/O and the use of isolated storage to provide a secure, isolated location for your application’s data.

Introduction to Streams

A stream is an object that essentially connects data between two endpoints with a narrow access window. Streams allow you to write and read small portions of data at a time, providing for an extremely efficient means of access. For example, using a stream, you can read data in small portions from a file that is several hundred megabytes in size without actually consuming several hundred megabytes of memory. The same is true of writing to streams. You can place small amounts of data on a stream without having to have all of the data in memory at any given time.

In addition to the performance benefit, streams also provide a unified model for reading and writing data, regardless of the format or location of the underlying data. For example, you can access data from a disk file using a stream and you can access data from a relational database or from a web service in a stream as well.

Streams can also be connected through a process referred to as composition. By composing streams, you can attach various types of reader and writer classes to the end of the stream to make data access easier. As you will see in Chapter 15, “Cryptography and Data Protection,” you can even attach specially encrypted streams so that data is encrypted as soon as it is placed on the stream. Streams can also be used for network communication in addition to file I/O.

Using Memory Streams

Streams are the basic unit of I/O in the .NET Framework and you will find them used everywhere. Before getting into working with physical files on disk, this section will illustrate the basics of opening streams and reading and writing from streams using the MemoryStream class as an example. When you know how to manipulate a MemoryStream, you will find that you will be able to use all of the other types of streams exposed by .NET Framework classes with little difficulty.

As mentioned earlier, streams provide a narrow window of access. This often causes developers trouble. For example, when you write to a stream, the Position of the pointer in the stream advances. When you read from a stream, the read always starts from the current pointer position within the stream. A common source of problems when reading from streams is not setting the pointer position properly with the Seek method.

Before taking a look at the MemoryStream sample code, take a look at Tables 7.1 and 7.2, which list the methods and properties that belong to all Stream classes, regardless of the underlying data store.

Table 7.1 Stream Properties

Table 7.2 Stream Methods

The code shown in Listing 7.1 illustrates how to instantiate a stream, as well as how to read and write information from that stream.

Listing 7.1 MemoryStream Sample

Before running this code, see if you can predict the output. The code that retrieves the first four bytes of the stream should be fairly obvious; the output will be the word Mary.

Next, the code seeks to a position five bytes before the end of the stream and then grabs the next four bytes, producing the word “lamb.”

The next few lines of code are typically where a lot of developers get confused. When you write to a stream, you overwrite whatever bytes might lie underneath. So, when writing “really” to the stream, you don’t get “Mary had a really little lamb.” as one might expect. Rather, your stream contains “Mary had a really lamb.". If your goal is truly to insert data at a certain point in the stream, and have the remainder of the stream remain intact, you will need to do it the “old-fashioned way.” This involves using two streams and copying the old into the new, making sure to insert the new data in the right place in the new stream.

Introduction to Basic File I/O

In the .NET Framework, file I/O is accomplished using streams. Although some classes may make certain aspects of reading and writing to text files easier, those classes still make use of streams for the final read and write operations on operating system files. This section shows you how to create, append to, read from, and query information about files. You will also see how the framework provides utility classes for common file operations, such as the StreamWriter class that is used in conjunction with the File class to make working with plain text files easy.

Creating and Appending Files

When working with files using the Stream pattern, the majority of the work is done in the constructor for the FileStream class. This constructor allows you to specify the filename (or a classic Win32 file handle), as well as access modes, sharing modes, and much more. Using the constructor, you indicate whether you want to create a new file or open an existing file, or open an existing file for appending.

The following few lines of code create a new file and write some text to it:

Note that you need to close the FileStream in order for the contents of the Stream to be written to disk. As mentioned in the description for the Stream class, the Close method releases all resources and flushes the remaining contents of the buffer to whatever underlying media backs the stream, such as a disk file.

You can use a different option in the constructor to obtain a FileStream for the same file, but this time you can use the Stream to append additional data to the file:

The preceding code uses the FileMode.Append enumeration item to indicate how the file should be opened. Table 7.3 contains a description of each of the possible file modes.

Table 7.3 File Mode Enumeration Values

Now that you have seen the low-level way of creating basic files, whether they are binary files or text files, let’s take a look at a quicker and easier way of working with text files.

You can replace the preceding code where you have to work with arrays of bytes with the following code, making the code easier to read and simpler to write:

As you can see, the code is a lot simpler than the previous examples. The reason this chapter started off showing you how to work with arrays of bytes is that this knowledge will help you if you need to work with files that don’t contain simple text, such as image files or binary files containing fixed data structures.

Reading from Existing Files

Reading from files using streams works just like all other I/O that has been discussed in this chapter up to this point. You obtain a reference to the file either using the FileStream class constructor or using the File class.

When you have a reference to the file from which you want to read, you can read that data using the stream’s Read or ReadByte methods.

The following code uses a FileStream class to open an existing file, read an array of bytes from it, and display the resulting array of bytes as an ASCII string:

And now take a look at the same effect using the OpenText method of the File class to create an instance of the StreamReader class:

Using Directories and the File System

In all the previous examples, the filename of the file with which we were working was known and fixed. This isn’t exactly a good model of reality. In most commercial applications that deal with files, you need to do things like check to make sure that a file exists, or check to see if a specific directory exists. If the directory doesn’t exist when you need to create the file, you need to create the directory as well.

To help developers deal with this, the System.IO namespace provides the following classes:

• File—A static class that provides methods for testing for the existence of a file, copying files, deleting files, moving files, as well as the opening of files and, as shown in the preceding example, the creation of streams from files.
• FileInfo—An instance class that provides information about a specific file. Also contains instance methods for performing copy, move, delete and related operations.
• Directory—Static class that provides methods for determining the existence of directories, creating directories, and much more.
• DirectoryInfo—Provides instance methods for operating on a specific directory, including renaming, obtaining the list of files within the directory, and so on.
• Path—A utility class for parsing and building path strings.

For more information on each of these individual classes, you can refer to the MSDN documentation online at http://msdn.microsoft.com. The example in Listing 7.2 illustrates how to test and see if a file exists, and if it does, create a subdirectory, copy the file there, and then display some operating system-level information about the file itself.

Listing 7.2 File Manipulation and Query

The core of the code starts with the instantiation of a new FileInfo object, from which you can perform many different operations (such as a copy) as well as query detailed information about the file itself.

Developers who are familiar with file-level programming in previous versions of C# may start to get a little giddy when they notice that you can now obtain an ACL (Access Control List) starting with the FileInfo object. Previously, obtaining the list of access rights associated with a file was a painful and tedious task that involved invoking the Win32 API using a technique (discussed in Chapter 13, “COM and Windows Interoperability”) called “Platform Invoke.”

The output of the preceding code looks as follows:

Using Asynchronous File I/O

By default, when you read data from a file or write data to a file, the operation is synchronous. A synchronous operation is one where the code will attempt to complete the operation before continuing on to the next line of code. So, if it takes your code 20 minutes to perform a read operation of an entire file, the next line of code in your application will have to wait 20 minutes before executing.

This kind of waiting behavior is usually frowned upon by end users: No one wants to sit and twiddle their thumbs while an application locks everything up to complete an incredibly long operation.

This chapter does not go into too much detail on how to accomplish multithreaded programming, as more will be discussed on that topic later in the book. However, you will see that asynchronous file I/O is not only possible, but easy if you plan ahead. Take a look at the sample in Listing 7.3.

Listing 7.3 Asynchronous File I/O

A fairly large quote.txt file was created for this example. If you run it with that file, you should see that the message “Waiting for file read to finish...” appears several times before the operation is complete. This illustrates that the while loop immediately after the BeginRead method began executing even before the entire file had been read. This gives your application the ability to continue processing and providing the user with a rich experience while the data is being loaded in the background.

The concepts behind the AsyncCallback class and the IAsyncResult interface are covered more in depth throughout this book as the topics become more complex and you explore event-based programming and using multithreaded techniques to improve your application.

Working with Isolated Storage

Isolated storage is the solution to a unique problem that has arisen recently due to the increased security of users’ desktops as well as the security enforced by the Common Language Runtime itself. All managed applications run within a security “sandbox” and this sandbox often prevents an application from creating files or accessing the user’s hard disk at all. This security is highly desired, especially for applications obtained through the Internet to prevent malicious code from having unrestrained access to the user’s hard disk.

However, for every malicious application for which the security is absolutely required, there are many more innocent applications that just want to use the disk storage to store things like user preferences. User preferences can be anything from the last position of a recently opened window to highly complex workflow and business rule options specific to the user. Also, components written by the same vendor may need to access shared data but might lack the security clearance necessary to access such data.

Isolated storage provides an area in which applications can store private information, such as user preferences, that is under the complete control of the end user. This gives the application the ability to read and write from a safe location to store information and gives the user the peace of mind of knowing that even if an application can save data, it can’t do damage to any files other than the ones created by the application.

When you open an isolated storage location, your code must specify a scope for this store. This allows you to separate stores by user, by application, by domain, and even allows you to store the data within a user’s roaming profile.

The code in Listing 7.4 shows an example of an application that stores and retrieves the user’s favorite color. This preference will follow the user in his roaming profile if applicable.

Listing 7.4 Example of Isolated Storage

The first time you run the preceding application, it will indicate that the user’s current favorite color is "(none)". The next time you start the application, it will show you the color that you entered on the previous run. The data is being stored in the isolated storage area, an area that won’t allow the application to do damage to the hard disk or important files.

When you store data in isolated storage on Windows XP for a user, it will be located in <drive>Documents and Settings<User>Application Data for a roaming-enabled profile, and <drive>Documents and Settings<User>Local SettingsApplication Data for a non-roaming-enabled profile.

Summary

When most people think of reading and writing application data, they almost instinctively think about large relational databases such as SQL Server or Oracle, or flat-file databases like Microsoft Access. Even when surrounded by all of that database power, applications still need to read and write data from regular files on disk.

This chapter introduced you to the concept of streams, the basic unit of I/O used throughout the .NET Framework. You can use streams to access data for virtually any underlying source, even if that source is entirely in memory as in the case of the MemoryStream class.

Finally, this chapter showed you how to work with the operating system itself to manipulate directories and files, as well as obtain detailed information about files on disk. In addition, you saw how to use isolated storage to allow your application to store user data in a way that is easy to use and doesn’t violate CLR security policy for untrusted applications.

Whether you are creating large or small applications, Windows or web, you will undoubtedly find that the information in this chapter on streams and basic I/O is useful and applicable.