Network Architecture

Figure 16-1 illustrates the .NET networking types and the communication layers in which they reside. Most types reside in the transport layer or application layer. The transport layer defines basic protocols for sending and receiving bytes (TCP and UDP); the application layer defines higher-level protocols designed for specific applications, such as retrieving web pages (HTTP), transferring files (FTP), sending mail (SMTP), and converting between domain names and IP addresses (DNS).

Figure 16-1. Network architecture

It’s usually most convenient to program at the application layer; however, there are a couple of reasons why you might want to work directly at the transport layer. One is if you need an application protocol not provided in .NET, such as POP3 for retrieving mail. Another is if you want to invent a custom protocol for a special application such as a peer-to-peer client.

Of the application protocols, HTTP is special in its applicability to general-purpose communication. Its basic mode of operation—“give me the web page with this URL”—adapts nicely to “get me the result of calling this endpoint with these arguments.” (In addition to the “get” verb, there is “put,” “post,” and “delete,” allowing for REST-based services.)

HTTP also has a rich set of features that are useful in multitier business applications and service-oriented architectures, such as protocols for authentication and encryption, message chunking, extensible headers and cookies, and the ability to have many server applications share a single port and IP address. For these reasons, HTTP is well supported in .NET—both directly, as described in this chapter, and at a higher level, through such technologies as Web API and ASP.NET Core.

.NET Core provides client-side support for FTP, the popular internet protocol for sending and receiving files. Server-side support comes in the form of IIS or Unix-based server software.

As the preceding discussion makes clear, networking is a field that is awash in acronyms. We list the most common in Table 16-1.

Addresses and Ports

For communication to work, a computer or device requires an address. The internet uses two addressing systems:

IPv4

Currently the dominant addressing system; IPv4 addresses are 32 bits wide. When string-formatted, IPv4 addresses are written as four dot-separated decimals (e.g., 101.102.103.104). An address can be unique in the world—or unique within a particular subnet (such as on a corporate network).

IPv6

The newer 128-bit addressing system. Addresses are string-formatted in hexadecimal with a colon separator (e.g., [3EA0:FFFF:198A:E4A3:4FF2:54fA:41BC:8D31]). .NET requires that you add square brackets around the address.

The IPAddress class in the System.Net namespace represents an address in either protocol. It has a constructor accepting a byte array, and a static Parse method accepting a correctly formatted string:

IPAddress a1 = new IPAddress (new byte[] { 101, 102, 103, 104 });
IPAddress a2 = IPAddress.Parse ("101.102.103.104");
Console.WriteLine (a1.Equals (a2));                     // True
Console.WriteLine (a1.AddressFamily);                   // InterNetwork

IPAddress a3 = IPAddress.Parse
  ("[3EA0:FFFF:198A:E4A3:4FF2:54fA:41BC:8D31]");
Console.WriteLine (a3.AddressFamily);   // InterNetworkV6

The TCP and UDP protocols break out each IP address into 65,535 ports, allowing a computer on a single address to run multiple applications, each on its own port. Many applications have standard default port assignments; for instance, HTTP uses port 80; SMTP uses port 25.

An IP address and port combination is represented in .NET by the IPEndPoint class:

IPAddress a = IPAddress.Parse ("101.102.103.104");
IPEndPoint ep = new IPEndPoint (a, 222);           // Port 222
Console.WriteLine (ep.ToString());                 // 101.102.103.104:222

URIs

A URI is a specially formatted string that describes a resource on the internet or a LAN, such as a web page, file, or email address. Examples include http://www.ietf.org, ftp://myisp/doc.txt, and [email protected]. The exact formatting is defined by the Internet Engineering Task Force (IETF).

A URI can be broken up into a series of elements—typically, scheme, authority, and path. The Uri class in the System namespace performs just this division, exposing a property for each element, as illustrated in Figure 16-2.

Figure 16-2. URI properties

You can construct a Uri object by passing any of the following strings into its constructor:

• A URI string, such as http://www.ebay.com or file://janespc/sharedpics/dolphin.jpg

• An absolute path to a file on your hard disk, such as c:myfilesdata.xlsx or, on Unix, /tmp/myfiles/data.xlsx

• A UNC path to a file on the LAN, such as \janespcsharedpicsdolphin.jpg

File and UNC paths are automatically converted to URIs: the “file:” protocol is added, and backslashes are converted to forward slashes. The Uri constructors also perform some basic cleanup on your string before creating the Uri, including converting the scheme and hostname to lowercase and removing default and blank port numbers. If you supply a URI string without the scheme, such as “www.test.com,” a UriFormatException is thrown.

Uri has an IsLoopback property, which indicates whether the Uri references the local host (IP address 127.0.0.1), and an IsFile property, which indicates whether the Uri references a local or UNC (IsUnc) path (IsUnc reports false for a Samba share mounted in a Linux filesystem). If IsFile returns true, the LocalPath property returns a version of AbsolutePath that is friendly to the local OS (with slashes or backslashes as appropriate to the OS), on which you can call File.Open.

Instances of Uri have read-only properties. To modify an existing Uri, instantiate a UriBuilder object—this has writable properties and can be converted back via its Uri property.

Uri also provides methods for comparing and subtracting paths:

Uri info = new Uri ("http://www.domain.com:80/info/");
Uri page = new Uri ("http://www.domain.com/info/page.html");

Console.WriteLine (info.Host);     // www.domain.com
Console.WriteLine (info.Port);     // 80
Console.WriteLine (page.Port);     // 80  (Uri knows the default HTTP port)

Console.WriteLine (info.IsBaseOf (page));         // True
Uri relative = info.MakeRelativeUri (page);
Console.WriteLine (relative.IsAbsoluteUri);       // False
Console.WriteLine (relative.ToString());          // page.html

A relative Uri, such as page.html in this example, will throw an exception if you call almost any property or method other than IsAbsoluteUri and ToString(). You can directly instantiate a relative Uri, as follows:

Uri u = new Uri ("page.html", UriKind.Relative);

The Uri class also provides static helper methods such as EscapeUriString(), which converts a string to a valid URL by converting all characters with an ASCII value greater than 127 to hexadecimal representation. The CheckHostName() and CheckSchemeName() methods accept a string and check whether it is syntactically valid for the given property (although they do not attempt to determine whether a host or URI exists).

Client-Side Classes

WebRequest and WebResponse are common base classes for managing both HTTP and FTP client-side activity as well as the “file:” protocol. They encapsulate the “request/response” model that these protocols all share: the client makes a request and then awaits a response from a server.

WebClient is a convenient façade class that does the work of calling WebRequest and WebResponse, saving you some coding. WebClient gives you a choice of dealing in strings, byte arrays, files, or streams; WebRequest and WebResponse support just streams. Unfortunately, you cannot rely entirely on WebClient, because it doesn’t support some features (such as cookies).

HttpClient is a newer API for working with HTTP and is designed to work well with web APIs, REST-based services, and custom authentication schemes. In .NET Framework, HttpClient relied on WebRequest and WebResponse, but from .NET Core 3, it handles HTTP itself.

For simply downloading/uploading a file, string, or byte array, both WebClient and HttpClient are suitable. Both have asynchronous methods, although only WebClient offers progress reporting.

public void   DownloadFile   (string address, string fileName);
public string DownloadString (string address);
public byte[] DownloadData   (string address);
public Stream OpenRead       (string address);

public byte[] UploadFile  (string address, string fileName);
public byte[] UploadFile  (string address, string method, string fileName);
public string UploadString(string address, string data);
public string UploadString(string address, string method, string data);
public byte[] UploadData  (string address, byte[] data);
public byte[] UploadData  (string address, string method, byte[] data);
public byte[] UploadValues(string address, NameValueCollection data);
public byte[] UploadValues(string address, string method,
                                            NameValueCollection data);
public Stream OpenWrite    (string address);
public Stream OpenWrite    (string address, string method);

WebClient wc = new WebClient { Proxy = null };
wc.DownloadFile ("http://www.albahari.com/nutshell/code.aspx", "code.htm");

await wc.DownloadFileTaskAsync ("http://oreilly.com", "webpage.htm");

var wc = new WebClient();

wc.DownloadProgressChanged += (sender, args) => 
  Console.WriteLine (args.ProgressPercentage + "% complete");

Task.Delay (5000).ContinueWith (ant => wc.CancelAsync());
  
await wc.DownloadFileTaskAsync ("http://oreilly.com", "webpage.htm");

WebRequest req = WebRequest.Create
                ("http://www.albahari.com/nutshell/code.html");
req.Proxy = null;
using (WebResponse res = req.GetResponse())
using (Stream rs = res.GetResponseStream())
using (FileStream fs = File.Create ("code.html"))
  rs.CopyTo (fs);

WebRequest req = WebRequest.Create
                ("http://www.albahari.com/nutshell/code.html");
req.Proxy = null;
using (WebResponse res = await req.GetResponseAsync())
using (Stream rs = res.GetResponseStream())
using (FileStream fs = File.Create ("code.html"))
  await rs.CopyToAsync (fs);

string html = await new HttpClient().GetStringAsync ("http://linqpad.net");

var client = new HttpClient();
var task1 = client.GetStringAsync ("http://www.linqpad.net");
var task2 = client.GetStringAsync ("http://www.albahari.com");
Console.WriteLine (await task1);
Console.WriteLine (await task2);

var handler = new HttpClientHandler { UseProxy = false };
var client = new HttpClient (handler);
...

var client = new HttpClient();
// The GetAsync method also accepts a CancellationToken.
HttpResponseMessage response = await client.GetAsync ("http://...");
response.EnsureSuccessStatusCode();
string html = await response.Content.ReadAsStringAsync();

using (var fileStream = File.Create ("linqpad.html"))
  await response.Content.CopyToAsync (fileStream);

var client = new HttpClient();
var request = new HttpRequestMessage (HttpMethod.Get, "http://...");
HttpResponseMessage response = await client.SendAsync (request);
response.EnsureSuccessStatusCode();
...

var client = new HttpClient (new HttpClientHandler { UseProxy = false });
var request = new HttpRequestMessage (
  HttpMethod.Post, "http://www.albahari.com/EchoPost.aspx");
request.Content = new StringContent ("This is a test");
HttpResponseMessage response = await client.SendAsync (request);
response.EnsureSuccessStatusCode();
Console.WriteLine (await response.Content.ReadAsStringAsync());

public abstract class HttpMessageHandler : IDisposable
{
  protected internal abstract Task<HttpResponseMessage> SendAsync
    (HttpRequestMessage request, CancellationToken cancellationToken);

  public void Dispose();
  protected virtual void Dispose (bool disposing);
}

class MockHandler : HttpMessageHandler
{
  Func <HttpRequestMessage, HttpResponseMessage> _responseGenerator;
    
  public MockHandler
    (Func <HttpRequestMessage, HttpResponseMessage> responseGenerator)
  {
    _responseGenerator = responseGenerator;
  }
    
  protected override Task <HttpResponseMessage> SendAsync
    (HttpRequestMessage request, CancellationToken cancellationToken)
  {
    cancellationToken.ThrowIfCancellationRequested();
    var response = _responseGenerator (request);
    response.RequestMessage = request;
    return Task.FromResult (response);
  }
}

var mocker = new MockHandler (request => 
  new HttpResponseMessage (HttpStatusCode.OK)
  {
    Content = new StringContent ("You asked for " + request.RequestUri)
  });

var client = new HttpClient (mocker); 
var response = await client.GetAsync ("http://www.linqpad.net");
string result = await response.Content.ReadAsStringAsync();
Assert.AreEqual ("You asked for http://www.linqpad.net/", result);

class LoggingHandler : DelegatingHandler 
{
  public LoggingHandler (HttpMessageHandler nextHandler)
  {
     InnerHandler = nextHandler;
  }
    
  protected async override Task <HttpResponseMessage> SendAsync
    (HttpRequestMessage request, CancellationToken cancellationToken)
  {
    Console.WriteLine ("Requesting: " + request.RequestUri);
    var response = await base.SendAsync (request, cancellationToken);
    Console.WriteLine ("Got response: " + response.StatusCode);
    return response;
  }
}

// Create a WebProxy with the proxy's IP address and port. You can
// optionally set Credentials if the proxy needs a username/password.

WebProxy p = new WebProxy ("192.178.10.49", 808);
p.Credentials = new NetworkCredential ("username", "password");
// or:
p.Credentials = new NetworkCredential ("username", "password", "domain");

WebClient wc = new WebClient();
wc.Proxy = p;
  ...

// Same procedure with a WebRequest object:
WebRequest req = WebRequest.Create ("...");
req.Proxy = p;

WebProxy p = new WebProxy ("192.178.10.49", 808);
p.Credentials = new NetworkCredential ("username", "password", "domain");

var handler = new HttpClientHandler { Proxy = p };
var client = new HttpClient (handler);
...

WebRequest.DefaultWebProxy = myWebProxy;

WebRequest.DefaultWebProxy = null;

WebClient wc = new WebClient { Proxy = null };
wc.BaseAddress = "ftp://ftp.myserver.com";

// Authenticate, then upload and download a file to the FTP server.
// The same approach also works for HTTP and HTTPS.

string username = "myuser";
string password = "mypassword";
wc.Credentials = new NetworkCredential (username, password);

wc.DownloadFile ("guestbook.txt", "guestbook.txt");

string data = "Hello from " + Environment.UserName + "!
";
File.AppendAllText ("guestbook.txt", data);

wc.UploadFile ("guestbook.txt", "guestbook.txt");

var handler = new HttpClientHandler();
handler.Credentials = new NetworkCredential (username, password);
var client = new HttpClient (handler);
...

HTTP/1.1 401 Unauthorized
Content-Length: 83
Content-Type: text/html
Server: Microsoft-IIS/6.0
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="exchange.somedomain.com"
X-Powered-By: ASP.NET
Date: Sat, 05 Aug 2006 12:37:23 GMT

Authorization: Negotiate TlRMTVNTUAAABAAAt5II2gjACDArAAACAwACACgAAAAQ
ATmKAAAAD0lVDRdPUksHUq9VUA==

CredentialCache cache = new CredentialCache();
Uri prefix = new Uri ("http://exchange.somedomain.com");
cache.Add (prefix, "Digest",  new NetworkCredential ("joe", "passwd"));
cache.Add (prefix, "Negotiate", new NetworkCredential ("joe", "passwd"));

WebClient wc = new WebClient();
wc.Credentials = cache;
...

Basic, Digest, NTLM, Kerberos, Negotiate

cache.Add (prefix, "Negotiate", CredentialCache.DefaultNetworkCredentials);

var client = new HttpClient();
client.DefaultRequestHeaders.Authorization = 
  new AuthenticationHeaderValue ("Basic",
    Convert.ToBase64String (Encoding.UTF8.GetBytes ("username:password")));
...

var client = new HttpClient();
var response = await client.GetAsync ("http://linqpad.net/foo");
HttpStatusCode responseStatus = response.StatusCode;

WebClient wc = new WebClient { Proxy = null };
try
{
  string s = wc.DownloadString ("http://www.albahari.com/notthere");
}
catch (WebException ex)
{
  if (ex.Status == WebExceptionStatus.NameResolutionFailure)
    Console.WriteLine ("Bad domain name");
  else if (ex.Status == WebExceptionStatus.ProtocolError)
  {
    HttpWebResponse response = (HttpWebResponse) ex.Response;
    Console.WriteLine (response.StatusDescription);      // "Not Found"
    if (response.StatusCode == HttpStatusCode.NotFound)
      Console.WriteLine ("Not there!");                  // "Not there!"
  }
  else throw;
}

Working with HTTP

This section describes HTTP-specific request and response features of WebClient, HttpWebRequest/HttpWebResponse, and the HttpClient class.

WebClient wc = new WebClient { Proxy = null };
wc.Headers.Add ("CustomHeader", "JustPlaying/1.0");
wc.DownloadString ("http://www.oreilly.com");

foreach (string name in wc.ResponseHeaders.Keys)
  Console.WriteLine (name + "=" + wc.ResponseHeaders [name]);

Age=51
X-Cache=HIT from oregano.bp
X-Cache-Lookup=HIT from oregano.bp:3128
Connection=keep-alive
Accept-Ranges=bytes
Content-Length=95433
Content-Type=text/html
...

var client = new HttpClient (handler);

client.DefaultRequestHeaders.UserAgent.Add (
  new ProductInfoHeaderValue ("VisualStudio", "2015"));

client.DefaultRequestHeaders.Add ("CustomHeader", "VisualStudio/2015");

?key1=value1&key2=value2&key3=value3...

WebClient wc = new WebClient { Proxy = null };
wc.QueryString.Add ("q", "WebClient");     // Search for "WebClient"
wc.QueryString.Add ("hl", "fr");           // Display page in French
wc.DownloadFile ("http://www.google.com/search", "results.html");

string requestURI = "http://www.google.com/search?q=WebClient&hl=fr";

string search = Uri.EscapeDataString ("(WebClient OR HttpClient)");
string language = Uri.EscapeDataString ("fr");
string requestURI = "http://www.google.com/search?q=" + search +
                    "&hl=" + language;

http://www.google.com/search?q=(WebClient%20OR%20HttpClient)&hl=fr

WebClient wc = new WebClient { Proxy = null };

var data = new System.Collections.Specialized.NameValueCollection();
data.Add ("Name", "Joe Albahari");
data.Add ("Company", "O'Reilly");

byte[] result = wc.UploadValues ("http://www.albahari.com/EchoPost.aspx",
                                 "POST", data);

Console.WriteLine (Encoding.UTF8.GetString (result));

var req = WebRequest.Create ("http://www.albahari.com/EchoPost.aspx");
req.Proxy = null;
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";

string reqString = "Name=Joe+Albahari&Company=O'Reilly";
byte[] reqData = Encoding.UTF8.GetBytes (reqString);
req.ContentLength = reqData.Length;

using (Stream reqStream = req.GetRequestStream())
  reqStream.Write (reqData, 0, reqData.Length);

using (WebResponse res = req.GetResponse())
using (Stream resSteam = res.GetResponseStream())
using (StreamReader sr = new StreamReader (resSteam))
  Console.WriteLine (sr.ReadToEnd());

string uri = "http://www.albahari.com/EchoPost.aspx";
var client = new HttpClient();
var dict = new Dictionary<string,string> 
{
    { "Name", "Joe Albahari" },
    { "Company", "O'Reilly" }
};
var values = new FormUrlEncodedContent (dict);
var response = await client.PostAsync (uri, values);
response.EnsureSuccessStatusCode();
Console.WriteLine (await response.Content.ReadAsStringAsync());

var cc = new CookieContainer();

var request = (HttpWebRequest) WebRequest.Create ("http://www.google.com");
request.Proxy = null;
request.CookieContainer = cc;
using (var response = (HttpWebResponse) request.GetResponse())
{
  foreach (Cookie c in response.Cookies)
  {
    Console.WriteLine (" Name:   " + c.Name);
    Console.WriteLine (" Value:  " + c.Value);
    Console.WriteLine (" Path:   " + c.Path);
    Console.WriteLine (" Domain: " + c.Domain);
  }
  // Read response stream...
}

 Name:   PREF
 Value:  ID=6b10df1da493a9c4:TM=1179025486:LM=1179025486:S=EJCZri0aWEHlk4tt
 Path:   /
 Domain: .google.com

var cc = new CookieContainer();
var handler = new HttpClientHandler();
handler.CookieContainer = cc;
var client = new HttpClient (handler);
...

Cookie c = new Cookie ("PREF",
                       "ID=6b10df1da493a9c4:TM=1179...",
                       "/",
                       ".google.com");
freshCookieContainer.Add (c);

Writing an HTTP Server

You can write your own .NET HTTP server with the HttpListener class. The following is a simple server that listens on port 51111, waits for a single client request, and then returns a one-line reply:

using var server = new SimpleHttpServer();

// Make a client request:
Console.WriteLine (new WebClient().DownloadString
  ("http://localhost:51111/MyApp/Request.txt"));

class SimpleHttpServer : IDisposable
{
  readonly HttpListener listener = new HttpListener();
  
  public SimpleHttpServer() => ListenAsync();  
  async void ListenAsync()
  {
    listener.Prefixes.Add ("http://localhost:51111/MyApp/");  // Listen on
    listener.Start();                                         // port 51111

    // Await a client request:
    HttpListenerContext context = await listener.GetContextAsync();

    // Respond to the request:
    string msg = "You asked for: " + context.Request.RawUrl;
    context.Response.ContentLength64 = Encoding.UTF8.GetByteCount (msg);
    context.Response.StatusCode = (int)HttpStatusCode.OK;

    using (Stream s = context.Response.OutputStream)
    using (StreamWriter writer = new StreamWriter (s))
      await writer.WriteAsync (msg);
  }

  public void Dispose() => listener.Close();
}

OUTPUT: You asked for: /MyApp/Request.txt

On Windows, HttpListener does not internally use .NET Socket objects; it instead calls the Windows HTTP Server API. This allows many applications on a computer to listen on the same IP address and port—as long as each registers different address prefixes. In our example, we registered the prefix http://localhost/myapp, so another application would be free to listen on the same IP and port on another prefix, such as http://localhost/anotherapp. This is of value because opening new ports on corporate firewalls can be politically arduous.

HttpListener waits for the next client request when you call GetContext, returning an object with Request and Response properties. Each is analogous to a WebRequest and WebResponse object, but from the server’s perspective. You can read and write headers and cookies, for instance, to the request and response objects, much as you would at the client end.

You can choose how fully to support features of the HTTP protocol, based on your anticipated client audience. At a bare minimum, you should set the content length and status code on each request.

Here’s a very simple web page server, written asynchronously:

using System;
using System.IO;
using System.Net;
using System.Text;
using System.Threading.Tasks;

class WebServer
{
  HttpListener _listener;
  string _baseFolder;      // Your web page folder.

  public WebServer (string uriPrefix, string baseFolder)
  {
    _listener = new HttpListener();
    _listener.Prefixes.Add (uriPrefix);
    _baseFolder = baseFolder;
  }

  public async void Start()
  {
    _listener.Start();
    while (true)
      try 
      {
        var context = await _listener.GetContextAsync();
        Task.Run (() => ProcessRequestAsync (context));
      }
      catch (HttpListenerException)     { break; }   // Listener stopped.
      catch (InvalidOperationException) { break; }   // Listener stopped.
  }

  public void Stop() => _listener.Stop();

  async void ProcessRequestAsync (HttpListenerContext context)
  {
    try
    {
      string filename = Path.GetFileName (context.Request.RawUrl);
      string path = Path.Combine (_baseFolder, filename);
      byte[] msg;
      if (!File.Exists (path))
      {
        Console.WriteLine ("Resource not found: " + path);
        context.Response.StatusCode = (int) HttpStatusCode.NotFound;
        msg = Encoding.UTF8.GetBytes ("Sorry, that page does not exist");
      }
      else
      {
        context.Response.StatusCode = (int) HttpStatusCode.OK;
        msg = File.ReadAllBytes (path);
      }
      context.Response.ContentLength64 = msg.Length;
      using (Stream s = context.Response.OutputStream)
        await s.WriteAsync (msg, 0, msg.Length);
    }
    catch (Exception ex) { Console.WriteLine ("Request error: " + ex); }
  }
}

The following code sets things in motion:

// Listen on port 51111, serving files in d:webroot:
var server = new WebServer ("http://localhost:51111/", @"d:webroot");
try
{
  server.Start();
  Console.WriteLine ("Server running... press Enter to stop");
  Console.ReadLine();
}
finally { server.Stop(); }

You can test this at the client end with any web browser; the URI in this case will be http://localhost:51111/ plus the name of the web page.

Our use of asynchronous functions makes this server scalable and efficient. Starting this from a user interface (UI) thread, however, would hinder scalability because for each request, execution would bounce back to the UI thread after each await. Incurring such overhead is particularly pointless given that we don’t have shared state, so in a UI scenario we’d get off the UI thread, either like this

Task.Run (Start);

or by calling ConfigureAwait(false) after calling GetContextAsync.

Note that we used Task.Run to call ProcessRequestAsync even though the method was already asynchronous. This allows the caller to process another request immediately rather than having to first wait out the synchronous phase of the method (up until the first await).

Using FTP

For simple FTP upload and download operations, you can use WebClient, as we did previously:

WebClient wc = new WebClient { Proxy = null };
wc.Credentials = new NetworkCredential ("myuser", "mypassword");
wc.BaseAddress = "ftp://ftp.myserver.com";
wc.UploadString ("tempfile.txt", "hello!");
Console.WriteLine (wc.DownloadString ("tempfile.txt"));   // hello!

There’s more to FTP, however, than just uploading and downloading files. The protocol also defines a set of commands or “methods,” which are exposed as string constants in WebRequestMethods.Ftp:

To run one of these commands, you assign its string constant to the web request’s Method property and then call GetResponse(). Here’s how to get a directory listing:

var req = (FtpWebRequest) WebRequest.Create ("ftp://ftp.myserver.com");
req.Proxy = null;
req.Credentials = new NetworkCredential ("myuser", "mypassword");
req.Method = WebRequestMethods.Ftp.ListDirectory;

using (WebResponse resp = req.GetResponse())
using (StreamReader reader = new StreamReader (resp.GetResponseStream()))
  Console.WriteLine (reader.ReadToEnd());

RESULT:
.
..
guestbook.txt
tempfile.txt
test.doc

In the case of getting a directory listing, we needed to read the response stream to get the result. Most other commands, however, don’t require this step. For instance, to get the result of the GetFileSize command, just query the response’s Content​Length property:

var req = (FtpWebRequest) WebRequest.Create (
                          "ftp://ftp.myserver.com/tempfile.txt");
req.Proxy = null;
req.Credentials = new NetworkCredential ("myuser", "mypassword");

req.Method = WebRequestMethods.Ftp.GetFileSize;

using (WebResponse resp = req.GetResponse())
  Console.WriteLine (resp.ContentLength);            // 6

The GetDateTimestamp command works in a similar way except that you query the response’s LastModified property. This requires that you cast to FtpWebResponse:

...
req.Method = WebRequestMethods.Ftp.GetDateTimestamp;

using (var resp = (FtpWebResponse) req.GetResponse() )
  Console.WriteLine (resp.LastModified);

To use the Rename command, you must populate the request’s RenameTo property with the new filename (without a directory prefix). For example, to rename a file in the incoming directory from tempfile.txt to deleteme.txt:

var req = (FtpWebRequest) WebRequest.Create (
                          "ftp://ftp.myserver.com/tempfile.txt");
req.Proxy = null;
req.Credentials = new NetworkCredential ("myuser", "mypassword");

req.Method = WebRequestMethods.Ftp.Rename;
req.RenameTo = "deleteme.txt";

req.GetResponse().Close();        // Perform the rename

Here’s how to delete a file:

var req = (FtpWebRequest) WebRequest.Create (
                          "ftp://ftp.myserver.com/deleteme.txt");
req.Proxy = null;
req.Credentials = new NetworkCredential ("myuser", "mypassword");

req.Method = WebRequestMethods.Ftp.DeleteFile;

req.GetResponse().Close();        // Perform the deletion

catch (WebException ex)
{
   if (ex.Status == WebExceptionStatus.ProtocolError)
   {
     // Obtain more detail on error:
     var response = (FtpWebResponse) ex.Response;
     FtpStatusCode errorCode = response.StatusCode;
     string errorMessage = response.StatusDescription;
     ...
   }
   ...
 }

Using DNS

The static Dns class encapsulates the DNS, which converts between a raw IP address, such as 66.135.192.87, and a human-friendly domain name, such as ebay.com.

The GetHostAddresses method converts from domain name to IP address (or addresses):

foreach (IPAddress a in Dns.GetHostAddresses ("albahari.com"))
  Console.WriteLine (a.ToString());     // 205.210.42.167

The GetHostEntry method goes the other way around, converting from address to domain name:

IPHostEntry entry = Dns.GetHostEntry ("205.210.42.167");
Console.WriteLine (entry.HostName);                    // albahari.com

GetHostEntry also accepts an IPAddress object, so you can specify an IP address as a byte array:

IPAddress address = new IPAddress (new byte[] { 205, 210, 42, 167 });
IPHostEntry entry = Dns.GetHostEntry (address);
Console.WriteLine (entry.HostName);                    // albahari.com

Domain names are automatically resolved to IP addresses when you use a class such as WebRequest or TcpClient. However, if you plan to make many network requests to the same address over the life of an application, you can sometimes improve performance by first using Dns to explicitly convert the domain name into an IP address, and then communicating directly with the IP address from that point on. This prevents repeated round-tripping to resolve the same domain name, and it can be of benefit when dealing at the transport layer (via TcpClient, UdpClient, or Socket).

The DNS class also provides awaitable task-based asynchronous methods:

foreach (IPAddress a in await Dns.GetHostAddressesAsync ("albahari.com"))
  Console.WriteLine (a.ToString());

Sending Mail with SmtpClient

The SmtpClient class in the System.Net.Mail namespace allows you to send mail messages through the ubiquitous Simple Mail Transfer Protocol, or SMTP. To send a simple text message, instantiate SmtpClient, set its Host property to your SMTP server address, and then call Send:

SmtpClient client = new SmtpClient();
client.Host = "mail.myserver.com";
client.Send ("[email protected]", "[email protected]", "subject", "body");

Constructing a MailMessage object exposes further options, including the ability to add attachments:

SmtpClient client = new SmtpClient();
client.Host = "mail.myisp.net";
MailMessage mm = new MailMessage();

mm.Sender = new MailAddress ("[email protected]", "Kay");
mm.From   = new MailAddress ("[email protected]", "Kay");
mm.To.Add  (new MailAddress ("[email protected]", "Bob"));
mm.CC.Add  (new MailAddress ("[email protected]", "Dan"));
mm.Subject = "Hello!";
mm.Body = "Hi there. Here's the photo!";
mm.IsBodyHtml = false;
mm.Priority = MailPriority.High;

Attachment a = new Attachment ("photo.jpg",
                               System.Net.Mime.MediaTypeNames.Image.Jpeg);
mm.Attachments.Add (a);
client.Send (mm);

To frustrate spammers, most SMTP servers on the internet will accept connections only from authenticated connections and require communication over SSL:

var client = new SmtpClient ("smtp.myisp.com", 587)
{
  Credentials = new NetworkCredential ("[email protected]", "MySecurePass"),
  EnableSsl = true
};
client.Send ("[email protected]", "[email protected]", "Subject", "Body");
Console.WriteLine ("Sent");

By changing the DeliveryMethod property, you can instruct the SmtpClient to instead use IIS to send mail messages or simply to write each message to an .eml file in a specified directory. This can be useful during development:

SmtpClient client = new SmtpClient();
client.DeliveryMethod = SmtpDeliveryMethod.SpecifiedPickupDirectory;
client.PickupDirectoryLocation = @"c:mail";

Using TCP

TCP and UDP constitute the transport layer protocols on top of which most internet—and LAN—services are built. HTTP (version 2 and below), FTP, and SMTP use TCP; DNS and HTTP version 3 use UDP. TCP is connection-oriented and includes reliability mechanisms; UDP is connectionless, has a lower overhead, and supports broadcasting. BitTorrent uses UDP, as does Voice over IP (VoIP).

The transport layer offers greater flexibility—and potentially improved performance—over the higher layers, but it requires that you handle such tasks as authentication and encryption yourself.

With TCP in .NET, you have a choice of either the easier-to-use TcpClient and TcpListener façade classes, or the feature-rich Socket class. (In fact, you can mix and match, because TcpClient exposes the underlying Socket object through the Client property.) The Socket class exposes more configuration options and allows direct access to the network layer (IP) and non-internet-based protocols such as Novell’s SPX/IPX.

(TCP and UDP communication is also possible via WinRT types: see “TCP in UWP”.)

As with other protocols, TCP differentiates a client and server: the client initiates a request, while the server waits for a request. Here’s the basic structure for a synchronous TCP client request:

using (TcpClient client = new TcpClient())
{
  client.Connect ("address", port);
  using (NetworkStream n = client.GetStream())
  {
    // Read and write to the network stream...
  }
}

TcpClient’s Connect method blocks until a connection is established (ConnectAsync is the asynchronous equivalent). The NetworkStream then provides a means of two-way communication, for both transmitting and receiving bytes of data from a server.

A simple TCP server looks like this:

TcpListener listener = new TcpListener (<ip address>, port);
listener.Start();

while (keepProcessingRequests)
  using (TcpClient c = listener.AcceptTcpClient())
  using (NetworkStream n = c.GetStream())
  {
    // Read and write to the network stream...
  }

listener.Stop();

TcpListener requires the local IP address on which to listen (a computer with two network cards, for instance, can have two addresses). You can use IPAddress.Any to instruct it to listen on all (or the only) local IP addresses. AcceptTcpClient blocks until a client request is received (again, there’s also an asynchronous version), at which point we call GetStream, just as on the client side.

When working at the transport layer, you need to decide on a protocol for who talks when and for how long—rather like with a walkie-talkie. If both parties talk or listen at the same time, communication breaks down!

Let’s invent a protocol in which the client speaks first, saying “Hello,” and then the server responds by saying “Hello right back!” Here’s the code:

using System;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Threading;

new Thread (Server).Start();       // Run server method concurrently.
Thread.Sleep (500);                // Give server time to start.
Client();

void Client()
{
  using (TcpClient client = new TcpClient ("localhost", 51111))
  using (NetworkStream n = client.GetStream())
  {
    BinaryWriter w = new BinaryWriter (n);
    w.Write ("Hello");
    w.Flush();
    Console.WriteLine (new BinaryReader (n).ReadString());
  }
}

void Server()     // Handles a single client request, then exits.
{
  TcpListener listener = new TcpListener (IPAddress.Any, 51111);
  listener.Start();
  using (TcpClient c = listener.AcceptTcpClient())
  using (NetworkStream n = c.GetStream())
  {
    string msg = new BinaryReader (n).ReadString();
    BinaryWriter w = new BinaryWriter (n);
    w.Write (msg + " right back!");
    w.Flush();                      // Must call Flush because we're not
  }                                 // disposing the writer.
  listener.Stop();
}

// OUTPUT: Hello right back!

In this example, we’re using the localhost loopback to run the client and server on the same machine. We’ve arbitrarily chosen a port in the unallocated range (above 49152) and used a BinaryWriter and BinaryReader to encode the text messages. We’ve avoided closing or disposing the readers and writers in order to keep the underlying NetworkStream open until our conversation completes.

BinaryReader and BinaryWriter might seem like odd choices for reading and writing strings. However, they have a major advantage over StreamReader and StreamWriter: they prefix strings with an integer indicating the length, so a BinaryReader always knows exactly how many bytes to read. If you call StreamReader.ReadToEnd you might block indefinitely—because a NetworkStream doesn’t have an end! As long as the connection is open, the network stream can never be sure that the client isn’t going to send more data.

async void RunServerAsync ()
{
  var listener = new TcpListener (IPAddress.Any, 51111);
  listener.Start ();
  try
  {
    while (true)
      Accept (await listener.AcceptTcpClientAsync ());
  }
  finally { listener.Stop(); }
}

async Task Accept (TcpClient client)
{
  await Task.Yield ();
  try
  {
    using (client)
    using (NetworkStream n = client.GetStream ())
    {
      byte[] data = new byte [5000];
      
      int bytesRead = 0; int chunkSize = 1;
      while (bytesRead < data.Length && chunkSize > 0)
        bytesRead += chunkSize =
          await n.ReadAsync (data, bytesRead, data.Length - bytesRead);
      
      Array.Reverse (data);   // Reverse the byte sequence
      await n.WriteAsync (data, 0, data.Length);
    }
  }
  catch (Exception ex) { Console.WriteLine (ex.Message); }
}

Receiving POP3 Mail with TCP

.NET provides no application-layer support for POP3, so you need to write at the TCP layer in order to receive mail from a POP3 server. Fortunately, this is a simple protocol; a POP3 conversation goes like this:

Each command and response is terminated by a new line (CR + LF) except for the multiline LIST and RETR commands, which are terminated by a single dot on a separate line. Because we can’t use StreamReader with NetworkStream, we can start by writing a helper method to read a line of text in a nonbuffered fashion:

string ReadLine (Stream s)
{
  List<byte> lineBuffer = new List<byte>();
  while (true)
  {
    int b = s.ReadByte();
    if (b == 10 || b < 0) break;
    if (b != 13) lineBuffer.Add ((byte)b);
  }
  return Encoding.UTF8.GetString (lineBuffer.ToArray());
}

We also need a helper method to send a command. Because we always expect to receive a response starting with +OK, we can read and validate the response at the same time:

void SendCommand (Stream stream, string line)
{
  byte[] data = Encoding.UTF8.GetBytes (line + "
");
  stream.Write (data, 0, data.Length);
  string response = ReadLine (stream);
  if (!response.StartsWith ("+OK"))
    throw new Exception ("POP Error: " + response);
}

With these methods written, the job of retrieving mail is easy. We establish a TCP connection on port 110 (the default POP3 port) and then start talking to the server. In this example, we write each mail message to a randomly named file with an .eml extension before deleting the message off the server:

using (TcpClient client = new TcpClient ("mail.isp.com", 110))
using (NetworkStream n = client.GetStream())
{
  ReadLine (n);                             // Read the welcome message.
  SendCommand (n, "USER username");
  SendCommand (n, "PASS password");
  SendCommand (n, "LIST");                  // Retrieve message IDs
  List<int> messageIDs = new List<int>();
  while (true)
  {
    string line = ReadLine (n);             // e.g.,  "1 1876"
    if (line == ".") break;
    messageIDs.Add (int.Parse (line.Split (' ')[0] ));   // Message ID
  }

  foreach (int id in messageIDs)         // Retrieve each message.
  {
    SendCommand (n, "RETR " + id);
    string randomFile = Guid.NewGuid().ToString() + ".eml";
    using (StreamWriter writer = File.CreateText (randomFile))
      while (true)
      {
        string line = ReadLine (n);      // Read next line of message.
        if (line == ".") break;          // Single dot = end of message.
        if (line == "..") line = ".";    // "Escape out" double dot.
        writer.WriteLine (line);         // Write to output file.
      }
    SendCommand (n, "DELE " + id);       // Delete message off server.
  }
  SendCommand (n, "QUIT");
}

TCP in UWP

In UWP applications, TCP functionality is exposed through WinRT types in the Windows.Networking.Sockets namespace. As with the .NET implementation, there are two primary classes to handle server and client roles, StreamSocketListener and StreamSocket.

The following method starts a server on port 51111, and waits for a client to connect. It then reads a single message comprising a length-prefixed string:

async void Server()
{
  var listener = new StreamSocketListener();
  listener.ConnectionReceived += async (sender, args) =>
  {
    using (StreamSocket socket = args.Socket)
    {
      var reader = new DataReader (socket.InputStream);
      await reader.LoadAsync (4);
      uint length = reader.ReadUInt32();
      await reader.LoadAsync (length);
      Debug.WriteLine (reader.ReadString (length));
    }
    listener.Dispose();   // Close listener after one message.
  };
  await listener.BindServiceNameAsync ("51111");
}

In this example, we used a WinRT type called DataReader (in Windows.Networking) to read from the input stream, rather than converting to a .NET Stream object and using a BinaryReader. DataReader is rather like BinaryReader except that it supports asynchrony. The LoadAsync method asynchronously reads a specified number of bytes into an internal buffer, which then allows you to call methods such as ReadUInt32 or ReadString. The idea is that if you wanted to, say, read 1,000 integers in a row, you’d first call LoadAsync with a value of 4000 and then ReadInt32 1,000 times in a loop. This prevents the overhead of calling asynchronous operations in a loop (because each asynchronous operation incurs a small overhead).

The StreamSocket object that we obtained from awaiting AcceptAsync has separate input and output streams. So, to write a message back, we’d use the socket’s OutputStream. We can illustrate the use of OutputStream and DataWriter with the corresponding client code:

async void Client()
{
  using (var socket = new StreamSocket())
  {
    await socket.ConnectAsync (new HostName ("localhost"), "51111",
                              SocketProtectionLevel.PlainSocket);
    var writer = new DataWriter (socket.OutputStream);
    string message = "Hello!";
    uint length = (uint) Encoding.UTF8.GetByteCount (message);
    writer.WriteUInt32 (length);
    writer.WriteString (message);
    await writer.StoreAsync();
  }
}

We start by directly instantiating a StreamSocket and then call ConnectAsync with the host name and port. (You can pass either a DNS name or an IP address string into HostName’s constructor.) By specifying SocketProtectionLevel.Ssl, you can request SSL encryption (if configured on the server).

Again, we used a WinRT DataWriter rather than a .NET BinaryWriter and wrote the length of the string (measured in bytes rather than characters), followed by the string itself, which is UTF-8 encoded. Finally, we called StoreAsync, which writes the buffer to the backing stream, and closed the socket.