Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. | ||
| --Brian Kernigan | ||
A common task of network applications is the downloading of files off a network or Internet server. Traditionally, the developer would have to implement a TCPIP socket layer that implemented a subset of the HTTP protocol to retrieve these files. As with many other common tasks, Microsoft has provided this functionality in the .NET framework with the HttpWebRequest and HttpWebResponse classes. These classes provide both synchronous and asynchronous approaches to interacting with universal resource identifiers (URI). The synchronous approach is very straightforward, and requires little instruction on usage. The asynchronous approach, however, can be tricky to implement and use.
In this chapter I will discuss the asynchronous functionality of the HttpWebRequest and HttpWebResponse classes, and present a reusable solution to download files asynchronously off a network or Internet server.
The .NET framework provides the abstract class WebRequest, which is the request and response model for accessing data from the Internet. This model is protocol-agnostic, specialized by classes inheriting from the abstract class. There are a variety of specialized descendents of WebRequest, like FileWebRequest for handling file:// paths, but this chapter will focus on the HTTP protocol using HttpWebRequest.
The HTTP protocol is the primary transport mechanism for communicating with Internet resources. A developer may use this mechanism to download application updates and configuration information that constantly changes or even to post messages to a dynamic environment like an ASP.NET application. The HttpWebRequest class implements the WebRequest class, providing a specialized request class to communicate over the HTTP protocol. This class enables an application to interact directly with servers using HTTP.
Server resources are identified by uniform resource identifiers (URI), and the .NET framework provides the Uri class, which defines the properties and methods for handling uniform resource identifiers, such as comparing, combing, and parsing. Requests are sent from an application to a URI, such as a zipped fie or web page. The requests are sent using HttpWebRequest to the remote server, using the HTTP protocol as the transport mechanism to access the resource.
Note
If an error occurs with a request, a WebException is thrown that contains details about why the request failed. The Status property is of type WebRequestStatus, and if the value is WebRequestStatus.ProtocolError, the response returned from the server is contained in the WebException.Response property.
The remainder of the chapter will cover the construction of an asynchronous wrapper around HttpWebRequest and HttpWebResponse.
The first component of our asynchronous wrapper is the request object, which serves as the public interface between the application and the rest of the wrapper. This wrapper executes the core system and fires a download complete event when the file has finished downloading. There is also a progress update event that you can subscribe to in order to display download progress to the users.
public class AsyncFileDownloadRequest
{
public event AsyncDownloadCompleteHandler DownloadComplete;
public event AsyncDownloadProgressHandler ProgressUpdate;
private Uri address = null;
public Uri Address
{
get { return address; }
set { address = value; }
}
public void Initiate()
{
Thread thread = new Thread(new ThreadStart(InitiateThread));
thread.Start();
}
private void InitiateThread()
{
if (DownloadComplete != null && address != null)
{
AsyncFileDownloadSystem system = new AsyncFileDownloadSystem();
byte[] data = system.DownloadFile(address, ProgressUpdate);
DownloadComplete(data);
}
}
}The asynchronous mechanism provided by the HttpWebRequest object relies on a chain of successively executed methods that process data in chunks. In order to associate the data with the asynchronous mechanism, we need to build a simple state object that will be used to store information and data related to state. The asynchronous mechanism allows us to pass an arbitrary object between methods, so the following class will be used as a container to pass within the asynchronous model.
internal class AsyncFileDownloadState
{
public AsyncDownloadProgressHandler ProgressUpdate;
private const int bufferSize = 1024;
private WebRequest request = null;
private Stream responseStream;
private bool fixedSizeBuffer = true;
private byte[] processBuffer;
private byte[] staticBuffer;
private List<byte> dynamicBuffer;
private int dataLength = -1;
private int bytesRead = 0;
public WebRequest Request
{
get { return request; }
set { request = value; }
}
public Stream ResponseStream
{
get { return responseStream; }
set { responseStream = value; }
}
public bool FixedSizeBuffer
{
get { return fixedSizeBuffer; }
set { fixedSizeBuffer = value; }
}
public byte[] ProcessBuffer
{
get { return processBuffer; }
set { processBuffer = value; }
}
public byte[] StaticBuffer
{
get { return staticBuffer; }
set { staticBuffer = value; }
}
public List<byte> DynamicBuffer
{
get { return dynamicBuffer; }
set { dynamicBuffer = value; }
}
public int DataLength
{
get { return dataLength; }
set { dataLength = value; }
}
public int BytesRead
{
get { return bytesRead; }
set { bytesRead = value; }
}
public AsyncFileDownloadState()
{
processBuffer = new byte[bufferSize];
}
}The solution presented in this chapter encapsulates a lot of the implementation details of HttpWebRequest and asynchronous communication into a wrapper class. There are a number of ways to implement an asynchronous model; some are extremely simple, while some are complex and robust. The solution for this chapter sits somewhere between those extremes.
The following code describes the core system that handles asynchronous web requests and responses.
public delegate void AsyncDownloadCompleteHandler(byte[] data);
public delegate void AsyncDownloadProgressHandler(int bytesRead,
int dataLength);
internal class AsyncFileDownloadSystem
{The following is a constant that describes the temporary buffer size of incoming data when the content length is unknown.
private const int bufferSize = 1024;
The following object is used to signal that the download is complete. ManualResetEvent allows threads to communicate with each other; it is typically used when one thread must be completed before others can proceed.
public ManualResetEvent completeEvent = new ManualResetEvent(false);
The following method downloads the data of a file pointed to by the address string. The specified callback is used to report progress status.
public byte[] DownloadFile(string address,
AsyncDownloadProgressHandler callback)
{
Uri addressUri = new Uri(address);
return DownloadFile(addressUri, callback);
}The following method downloads the data of a file pointed to by the Uri. The specified callback is used to report progress status.
public byte[] DownloadFile(Uri address, AsyncDownloadProgressHandler callback)
{Set the complete event state to un-signaled.
completeEvent.Reset();
Create a new HttpWebRequest object by passing in the address to the resource. Passing in a file:// address will result in a FileWebRequest object being created, working transparently with the existing code because of the protocol-agnostic model used.
WebRequest request = WebRequest.Create(address);
Instantiate a new asynchronous state object and reference the request for later use.
AsyncFileDownloadState state = new AsyncFileDownloadState();
state.Request = request;Set the progress update callback on the state object.
state.ProgressUpdate += callback;
Launch an asynchronous request to access a web resource.
IAsyncResult result =
request.BeginGetResponse(new AsyncCallback(ResponseCallback),
state) as IAsyncResult;Wait for the complete event to be set so that the data is not returned until the callback finishes.
completeEvent.WaitOne();
if (state.FixedSizeBuffer)
{
return state.StaticBuffer;
}
else
{
return state.DynamicBuffer.ToArray();
}
}The following callback is used when an asynchronous response occurs.
private void ResponseCallback(IAsyncResult asyncResult)
{Pull the asynchronous state object out of the result object, and then retrieve the request.
AsyncFileDownloadState state =
(AsyncFileDownloadState)asyncResult.AsyncState;
WebRequest request = state.Request;Complete the asynchronous response and get the response object.
WebResponse response = request.EndGetResponse(asyncResult);
As an optimization, we check to see whether the data length of the requested resource is known, so that an appropriate storage buffer can be used. With a known length, a static buffer can be used, whereas a dynamic buffer is used when the length is unknown, resulting in decreased performance.
if (response.ContentLength != -1)
{
state.DataLength = Convert.ToInt32(response.Content Length);
state.StaticBuffer = new byte[state.DataLength];
}
else
{
state.FixedSizeBuffer = false;
state.DynamicBuffer = new List<byte>(bufferSize);
}Retrieve the response object for the request so that data can now be read from the stream.
Stream responseStream = response.GetResponseStream();
state.ResponseStream = responseStream;Begin reading stream data asynchronously.
IAsyncResult readResult
= responseStream.BeginRead(state.ProcessBuffer,
0,
bufferSize,
new AsyncCallback(ReadCallback),
state);
}The following callback is used when an asynchronous data read occurs.
private void ReadCallback(IAsyncResult asyncResult)
{Pull the asynchronous state object out of the result object.
AsyncFileDownloadState state =
(AsyncFileDownloadState)asyncResult.AsyncState;Retrieve the ResponseStream from the state object that was set in the ResponseCallback.
Stream responseStream = state.ResponseStream;
Check if there is any more data to read.
int bytesRead = responseStream.EndRead(asyncResult);
if (bytesRead > 0)
{Copy the temporary data buffer into the appropriate data buffer that holds the final data.
if (state.FixedSizeBuffer)
{
Array.Copy(state.ProcessBuffer,
0,
state.StaticBuffer,
state.BytesRead,
bytesRead);
}
else
{
byte[] data = new byte[bytesRead];
Array.Copy(state.ProcessBuffer, 0, data, 0, bytesRead);
state.DynamicBuffer.AddRange(data);
}
state.BytesRead += bytesRead;Notify any callbacks attached to the progress update event.
if (state.ProgressUpdate != null)
state.ProgressUpdate(state.BytesRead, state.DataLength);Call another data read cycle until there are no more bytes to read.
IAsyncResult readResult
= responseStream.BeginRead(state.ProcessBuffer,
0,
bufferSize,
new AsyncCallback(ReadCallback),
state);
}
else
{All the data has been downloaded, so we can now close the response stream and signal the complete event.
responseStream.Close();
completeEvent.Set();
}
}
}Using the asynchronous download system is very easy. Simply specify the resource address, and bind the two callbacks to handle progress updates and the download complete event. After which, call the Initiate() method to begin downloading.
The following code shows this.
AsyncFileDownloadRequest request = new AsyncFileDownloadRequest();
// Specify the address of the file to download
request.Address = new Uri("http://yourdomain/thefile");
// Assign a delegate to handle download complete events
request.DownloadComplete
+= new AsyncDownloadCompleteHandler(DownloadCompleteCallback);
// Assign a delegate to handle progress update events
request.ProgressUpdate
+= new AsyncDownloadProgressHandler(DownloadProgressCallback);
// Request the file
request.Initiate();This chapter discussed the HttpWebRequest and HttpWebResponse classes, and then presented a reusable solution using these classes to asynchronously download files off a network or Internet server.
The Companion Web site has the full source code to the download system, including a simple demo that shows how to use the system. Figure 29.1 shows the main interface of the provided example.
As mentioned earlier, there are a number of ways that files can be downloaded asynchronously from a network location, but the solution presented in this chapter fully utilizes the built-in functionality of the .NET Class Framework.