Networking Overview

Several types of distributed applications exist; they’re generally classified into either client-server applications , in which clients make requests to a central server, or peer-to-peer applications, in which computers exchange data among themselves. In this chapter, you’ll focus on building client-server applications because these applications are currently more common. Whichever type of distributed application you want to build, the way computers exchange data is controlled by a protocol. A protocol is a standard that defines the rules for communication over a network.

Building a network-enabled application is generally considered one of the most challenging tasks a programmer can undertake, with good reason. When building a network application, you must consider three important requirements:

• Scalability: The application must remain responsive when used by many users concurrently; typically this means you must perform extensive testing and profiling of your server code to ensure that it performs when a high load is placed on it.

• Fault tolerance: Networks are inherently unreliable, and you shouldn’t write code that assumes that the network will always be there. If you do, your applications will be frustrating to end users. Every application should go to great lengths to ensure communication failures are handled smoothly, which means giving the user appropriate feedback, displaying error messages, and perhaps offering diagnostic or retry facilities. Do not let your application crash because of a network failure. You should also consider data consistency (that is, can you be sure that all updates necessary to keep data consistent reached the target computer or store?). Using transactions and a relational database as a data store can help with this. Depending on the type of application, you might also want to consider building an offline mode where the user can access locally stored data, and network requests are queued up until the network comes back online. A good example of this kind of facility is the offline mode that most email clients offer.

• Security: Security should be a concern for every application you write, but it becomes a hugely important issue in network programming. This is because, when you expose your application to a network, you open it up to attack from any other user of the network; therefore, if you expose your application to the Internet, you might be opening it up to thousands or even millions of potential attackers. Typically you need to think about whether data traveling across the network needs to be secured, whether signed to guarantee it has not been tampered with or encrypted to guarantee only the appropriate people can read it. You also need to ensure that the people connecting to your application are who they say they are, and that they are authorized to do what they are requesting to do.

Fortunately, modern programmers don’t have to tackle these problems on their own; network protocols can help you tackle these problems. For example, if it is important that no one else on the network can read the data you are sending, you should not attempt to encrypt the data yourself. Instead, you should use a network protocol that offers this facility. These protocols are exposed through components from libraries that implement them for you. The type of protocol, and the library used, is dictated by the requirements of your applications. Some protocols offer encryption and authentication, and others don’t. Some are suitable for client-server applications, and others are suitable for peer-to-peer applications. You’ll look at the following components and libraries, along with the protocols they implement, in this chapter:

• HTTP/HTTPS requestssupport requests from web pages to servers, typically only for client-server applications.

• Web servicesexpose applications so other applications can request services, typically used only for client-server applications.

Despite the inherent challenge of distributed programming, it is generally worth the effort because it enables you to access interesting data and share the results of your programs with others. By the end of this chapter, you will be able to access data stored in Google spreadsheets, and use two different web-development paradigms to write a simple URL shortening service.

Using HTTP

The Web uses Hypertext Transfer Protocol (HTTP) to communicate, typically with web browsers, but you might want to make web requests from a script or a program for several reasons. For example, you might use this to aggregate site content through RSS or Atom feeds.

To make an HTTP request, you use the static method Create from the System.Net.WebRequest class . This creates a WebRequest object that represents a request to the uniform resource locator (URL, an address used to address a resource on a network uniquely) that was passed to the Create method. You then use the GetResponse method to get the server’s response to your request, which is represented by the System.Net.WebResponse class.

Listing 10-1 illustrates how to call an RSS on the British Broadcasting Corporation’s web site. The core of the example is the function getUrlAsXml, which does the work of retrieving the data from the URL and loading the data into an XmlDocument. The rest of the example illustrates the kind of post-processing you might want to do on the data; in this case, it displays the title of each item on the console, allowing users to choose which item to display. You can run the example in F# Interactive.

Listing 10-1. Using HTTP

#if INTERACTIVE
#r "System.Xml.dll"
#else
module Rss
#endif

open System
open System.Diagnostics
open System.Net
open System.Xml

/// makes a http request to the given url
let getUrlAsXml (url: string) =
    let request = WebRequest.Create(url)
    let response = request.GetResponse()
    let stream = response.GetResponseStream()
    let xml = new XmlDocument()
    xml.Load(stream)
    xml

/// the url we interested in
let url = "http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/sci/tech/rss.xml"

/// main application function
let main() =
    // read the rss fead
    let xml = getUrlAsXml url

    // write out the tiles of all the news items
    let nodes = xml.SelectNodes("/rss/channel/item/title")
    for i in 0 .. (nodes.Count - 1) do
        printf "%i. %s
" (i + 1) (nodes.[i].InnerText)

    // read the number the user wants from the console
    let item = int(Console.ReadLine())

    // find the new url
    let newUrl =
        let xpath = sprintf "/rss/channel/item[%i]/link" item
        let node = xml.SelectSingleNode(xpath)
        node.InnerText

    // start the url using the shell, this automatically opens
    // the default browser
    let procStart = new ProcessStartInfo(UseShellExecute = true,
                                         FileName = newUrl)
    let proc = new Process(StartInfo = procStart)
    proc.Start() |> ignore

do main()                                                                      

The results of this example at the time of writing were as follows (your results will vary):

1. Korean leader for UN climate panel
2. Neutrino 'flip' wins physics Nobel
3. Homo naledi was 'jack of all trades'
4. Plant uses raindrops to eat ants
5. Wild mammals 'return to Chernobyl'
6. New rat discovered in Indonesia
7. Cacti facing extinction, study warns
8. Nobel Prize for anti-parasite drugs
9. Chile creates new marine reserves
10. Mammal species outlived the dinosaurs
11. UN battle looms over climate costs
12. Supernova 'stream' in lab's sights
13. Nasa photo archive posted online
14. Ceres' spots remain mysterious
15. Additive promises crash-safe fuel
16. Charon moon seen in super detail
17. Africa's farming 'needs young blood'
18. VIDEO: Air pollution - the 'invisible hazard'

Type one of the numbers and you should be taken straight to the news story in your browser.

Using HTTP with Google Spreadsheets

Because of its simplicity and its platform independence, exposing data by HTTP and XML is one of the most popular ways to expose data publicly across the Internet. You can access a surprising amount of data using only HTTP and some JSON or XML processing. A useful application of this is accessing Google Spreadsheets published by their owners. You can see how to access Google Spreadsheets in Listing 10-2.

Listing 10-2. Using HTTP to Access Google Spreadsheets

#if INTERACTIVE
#r "System.Xml.dll"
#else
module GoogleSheets
#endif

open System
open System.IO
open System.Net
open System.Xml
open System.Xml.XPath

// some namespace information for the XML
let namespaces =
    [ "at", "http://www.w3.org/2005/Atom";
      "openSearch", "http://a9.com/-/spec/opensearchrss/1.0/";
      "gsx", "http://schemas.google.com/spreadsheets/2006/extended" ]

// read the XML and process it into a matrix of strings
let queryGoogleSpreadSheet (xdoc: XmlDocument) xpath columnNames =
    let nav = xdoc.CreateNavigator()
    let mngr = new XmlNamespaceManager(new NameTable())
    do List.iter (fun (prefix, url) ->
                     mngr.AddNamespace(prefix, url)) namespaces
    let xpath = nav.Compile(xpath)
    do xpath.SetContext(mngr)
    let iter = nav.Select(xpath)
    seq { for x in iter ->
            let x  = x :?> XPathNavigator
            let getValue nodename =
                let node = x.SelectSingleNode(nodename, mngr)
                node.Value
            Seq.map getValue columnNames }                                                                      

// read the spreadsheet from its web address
let getGoogleSpreadSheet (url: string) columnNames =
    let req = WebRequest.Create(url)
    use resp = req.GetResponse()
    use stream = resp.GetResponseStream()
    let xdoc = new XmlDocument()
    xdoc.Load(stream)
    queryGoogleSpreadSheet xdoc "/at:feed/at:entry" columnNames

// a location to hold the information we're interested in
type Location =
    { Country: string;
      NameValuesList: seq<string * option<float>> }

// creates a location from the row names
let createLocation names row  =
    let country = Seq.head row
    let row = Seq.skip 1 row
    let tryParse s =
        let success,res = Double.TryParse s
        if success then Some res else None
    let values = Seq.map tryParse row
    { Country = country;
      NameValuesList = Seq.zip names values }
// get the data and process it into records
let getDataAndProcess url colNames =
    // get the names of the columns we want
    let cols = Seq.map fst colNames
    // get the data
    let data = getGoogleSpreadSheet url cols

    // get the readable names of the columns
    let names = Seq.skip 1 (Seq.map snd colNames)
    // create strongly typed records from the data
    Seq.map (createLocation names) data

// function to create a spreadsheets                               URL from its key
let makeUrl = sprintf "http://spreadsheets.google.com/feeds/list/%s/od6/public/values"

let main() =
    // the key of the spreadsheet we're interested in
    let sheetKey = "phNtm3LmDZEP61UU2eSN1YA"
    // list of column names we're interested in
    let cols =
        [ "gsx:location", "";
          "gsx:hospitalbedsper10000population",
            "Hospital beds per 1000";
          "gsx:nursingandmidwiferypersonneldensityper10000population",
            "Nursing and Midwifery Personnel per 1000" ];
    // get the data
    let data = getDataAndProcess (makeUrl sheetKey) cols
    // print the data
    Seq.iter (printfn "%A") data

do main()

When you run the code from Listing 10-2, which you can do from F# Interactive, you get the following results:

...
{Country = "Sweden";
 NameValuesList =
  seq
    [("Hospital beds per 1000", null);
     ("Nursing and Midwifery Personnel per 1000", Some 109.0)];}
{Country = "Switzerland";
 NameValuesList =
  seq
    [("Hospital beds per 1000", Some 57.0);
     ("Nursing and Midwifery Personnel per 1000", Some 110.0)];}
...

The important thing to notice about this example is that the method you use to retrieve the data changes little; at the core of the example, you find the same few lines of code for making the HTTP request and retrieving an XML document from it:

let req = WebRequest.Create(url)
use resp = req.GetResponse()
use stream = resp.GetResponseStream()
let xdoc = new XmlDocument()
xdoc.Load(stream)

Most of the rest of the example treats the XML data that’s returned.

Using Suave.io

Now that you understand the basics of WebRequest, let’s look at the other end of the pipeline: what to do if you want to write a service that responds to web requests and serves out responses. Traditionally in .NET we’ve used products such as ASP.NET and WCF to provide frameworks and templates for writing such services. Those are still great options if you are comfortable with at least some of the layers of your application being written in C#, and if you require the huge range of features (various security options, rich UIs, and so forth) that they offer. But the downside of these options is that they are quite heavyweight. Even a new “empty” project contains a great deal of code and many library dependencies. A great alternative, if you want to write a lightweight, all-F# solution, is an open source framework called Suave.io.

Let’s use Suave.io to write a URL shortening service similar to tinyurl.com. The basic principle is that users can paste a long URL into the service and receive a much shorter URL in return. The shorter URL is much more email- and messaging-friendly. When someone else follows the URL, the shortening service resolves it back into the original URL and redirects them to the original location. You can write a URL-shortening service in a few lines of code using Suave.io, provided you accept some limitations :

• Your service won’t be persistent. When the service is restarted, the mapping from shortened to full URLs will be lost. (You might find it an interesting exercise to add a persistence layer yourself.)

• Your service won’t allow the user to specify her own text for the shortened URL. It’ll always be a random string.

• Your service will only run on localhost. This is really just a matter of what host name is prepended to the shortened URL and could easily be generalized.

Start by creating a new F# console project, and use NuGet (or your favorite package manager) to add the package Suave.io:

PM> Install-Package Suave

Copy the code from Listing 10-3 into Program.fs, replacing its existing contents.

Listing 10-3. A URL Shortener

open System
open System.Collections.Concurrent
open Suave
open Suave.Http
open Suave.Web
open Suave.Filters
open Suave.Operators

// Initialize a .NET random number generator:
let random = Random()

// Generate a random string of lower-case letters
// of a specified length.  WARNING: this has the
// potential to generate rude words!
let randomString len =
   Array.init len (fun _ -> random.Next(26) + 97 |> char)
   |> String

// A dictionary of mappings from the long to the shortened URLs:
let longToShort = ConcurrentDictionary<string, string>()
// A dictionary of mappings from the shortened to the long URLs:
let shortToLong = ConcurrentDictionary<string, string>()

// Shorten a URL, and as a side-effect store the short->long
// and long->short mappings:
let shorten (long : string) =
   longToShort.GetOrAdd(long, fun _ ->
      let short = randomString 5
      shortToLong.[short] <- long
      short)

// Try resolve a shortened URL to the long version:
let tryResolve (short : string) =
   match shortToLong.TryGetValue short with
   | true, long -> Some long
   | false, _-> None

// Create a Suave app which can add and resolve URLs:
let app =
   choose
      [
         POST >=> choose
            [ path "/add" >=> request (fun req ->
               match (req.formData "url") with
               | Choice1Of2 long ->
                  let short = shorten long
                  // You may need to amend the port number:
                  let url = sprintf "localhost:8083/go/%s" short
                  Successful.CREATED url
               | _ ->
                  RequestErrors.BAD_REQUEST "Url not supplied")
            ]
         GET >=> choose
            [ pathScan "/go/%s" (fun short ->
               match tryResolve short with
               | Some long ->
                  Redirection.MOVED_PERMANENTLY (sprintf "http://%s" long)
               | None ->
                  RequestErrors.NOT_FOUND "Url not found")
            ]
      ]

[<EntryPoint>]
let main args =
   // Start the service:
   startWebServer defaultConfig app
   0                                          

Before we analyze the code, try it out so that you are clear on what the code is trying to achieve. Start by running the project. You should see a console window with text something like this:

[I] 2015-10-21T08:12:26.6241031Z: listener started in 49.036 ms with binding 127.0.0.1:8083 [Suave.Tcp.tcpIpServer]

Take a moment to check that the port number shown (in this case 8083) is the same as the one you have in this line of code:

let url = sprintf "localhost:8083/go/%s" short

If it isn’t, stop the project, edit the code to match what was shown in the console window, and run it again.

Next, you are going to need a means of sending a POST message to your new server, containing a URL that you want to be shortened. There are a number of tools to achieve this; a common one on Windows .NET is Fiddler, which you can download free from www.telerik.com/fiddler . Linux users often use a tool called Curl (go to http://curl.haxx.se/download.html ). Run Fiddler (or the tool of your choice) and set it up to send a POST to localhost:8083/add, with a request body that specifies a value of www.bbc.co.uk (or whatever URL you prefer) for a variable called url. Figure 10-1 shows what the Compose tab in Fiddler should look like once you’ve done this.

Figure 10-1. Setting up Fiddler to send a POST

Once again, you may have to adjust the port number to match the one shown in the console window for your running service. When you’ve got these values entered, press “Execute.” This will post the request body url= www.bbc.co.uk to your service, and your service will then generate a shortened version, store it away, and return the shortened version back to the client. How can you see the shortened version? You need to look through the Fiddler history list (the left-hand window in the default layout) to find an entry that looks like the one highlighted in Figure 10-2.

Figure 10-2. Finding a response in Fiddler

Double-click the entry and locate the TextView tab. Here you should see the shortened version that your service returned (Figure 10-3). Note that it may not actually be shorter, depending on the length of the URL you originally sent.

Figure 10-3. Viewing a response in Fiddler

The last few characters (qtkpz) are the random short URL; the rest is just the server and port, and the route go within your service.

Be aware that there is a slight possibility of the random string generator generating offensive words. A good way to mitigate this simply is to exclude vowels from the character set used, and perhaps allow numbers instead.

Now it’s time to test that your shortened URL actually works. With your project still running, paste the entire generated URL (e.g. localhost:8083/go/qtkpz) into a browser, and you should be redirected to the original site. Try adding a few more URLs via Fiddler, and then navigating to them in your browser via the shortened versions. Also check what happens if you try to add the same URL more than once.

As you can see, your miniature URL shortener works quite nicely. How did you achieve so much with so little code? First, there is some simple utility code: you bind a .NET random number generator and use it in a function that will generate a random string of a fixed length:

// Initialize a .NET random number generator:
let random = Random()

// Generate a random string of lower-case letters
// of a specified length.  WARNING: this has the
// potential to generate rude words!
let randomString len =
   Array.init len (fun _ -> random.Next(26) + 97 |> char)
   |> String

Next, you declare a couple of ConcurrentDictionary instances :

// A dictionary of mappings from the long to the shortened URLs:
let longToShort = ConcurrentDictionary<string, string>()
// A dictionary of mappings from the shortened to the long URLs:
let shortToLong = ConcurrentDictionary<string, string>()

These instances are how you store a bidirectional mapping between the original and shortened version of each URL. You need two dictionaries to ensure that the program can look up rapidly in both directions. (This is quite a useful coding pattern when you need bidirectional access.) You use concurrent dictionaries because requests to your server may overlap; the plain .NET Dictionary is not thread safe and would cause exceptions as soon as the load on your service got heavy.

Now we get to the real heart of your service:

// Shorten a URL, and as a side-effect store the short->long
// and long->short mappings:
let shorten (long : string) =
   longToShort.GetOrAdd(long, fun _ ->
      let short = randomString 5
      shortToLong.[short] <- long
      short)

This is the code that takes the original long URL, creates a shorter alias for it, stores the original and shortened versions in your two dictionaries, and returns the shortened version to the caller. It’s quite a dense piece of code so we are going to walk though it very carefully. You start by calling longToShort.GetOrAdd, which is a method of the ConcurrentDictionary class, one which may seem very strange if you are only familiar with classic dictionaries. The reason it’s a little convoluted is that ConcurrentDictionary wants to make sure that it has control of the entire process of checking whether a value is already a dictionary, and adding it if it is not. If it did not have this level of control, there would be a chance of another thread coming in and doing something to the dictionary contents between the already-exists check and the adding of the value. GetOrAdd takes two arguments. The first is the dictionary key that you want to add to the dictionary (if it is not already there). The second is a function whose job is to provide a value that corresponds to the key. This function will only be called by ConcurrentDictionaryif the key from the first argument isn’t already in the dictionary.

You specify a body for the value-generating function using a classic F# lambda expression (fun _ ->). Note that the function takes an argument that can be the input key, but as you already know the value of that in this context, you ignore that parameter using an underscore (_).

Now let’s look at the body of your lambda function. Remember, the aim here is to provide a value that corresponds to the given key: the value returned from this function will be added to the dictionary along with the key–but only if the key does not already exist. You achieve this by calling your randomString function and binding the result as short:

let short = randomString 5

At this point, your random string has no inherent association with the long version; it’s just a random string. You make the association from short to long by adding it to the shortToLong dictionary :

shortToLong.[short] <- long

How do you make the association in the other direction? You do so by simply returning the value short from the lambda function. This works because the whole purpose of this lambda function is to generate a value to be associated with long when called by the longToShort dictionary’s GetOrAdd method.

Incidentally, there is a weakness in this code: because you don’t check if a given random string has been used before, there is a non-zero chance that you could end up trying to use the same shortened URL for two different long URLs. The later entry would overwrite the earlier one. The chance of this is very low for the second entry you add, but increases each time as existing values accumulate. This might not be acceptable in production! You could mitigate it somewhat by extending the character set used in the random string, using a longer random string, or, ideally, by checking if the random string already exists. We have not done this here because to do so in a fully thread-safe way would make the example rather long.

Now you have the function to create a mapping, so you need one to go the other way: to take a shortened URL and return the original long URL supplied by the caller. As you have no control over what input is sent (it might not be a valid shortened URL) you will do this using the try... idiom, where you either return Some(value) on success or None on failure. Here is the code:

// Try to resolve a shortened URL to the long version:
let tryResolve (short : string) =
   match shortToLong.TryGetValue short with
   | true, long -> Some long
   | false, _-> None

Here you are making a call to the shortToLong dictionary’s TryGetValue method, which will look up the value short in the dictionary and return a tuple of true and the value if it exists, or false and null if it does not. (In C#, TryGetValue places the value, if found, in a by-reference argument and returns a Boolean; the F# compiler uses some magic to transform the call so that the success-Boolean and the by-reference argument are instead returned as a tuple.) You can then pattern match on the returned tuple to translate it into Some(value) or None.

That’s all the business logic you need; the rest is just a matter of binding that logic to the HTTP handling that is needed to make the application work as a web service. This where you need to make use of Suave.io. Suave.io apps consist of nested sets of calls to the Suave.io function choose. The choose function takes a single argument: a list of WebPart instances (a WebPart being a function that can map from an HTTP context to another HTTP context). Each of these WebPart instances may or may not match the incoming value; the first one that actually does match is returned. This continues in a recursive way until the app can return a response. A custom operator (>=>) is defined to help you bind pairs of async workflows together.

So at the outer level here,

choose
   [
      POST >=> choose
     ...
      GET >=> choose
   ]
     ...

you are effectively saying “if the incoming request uses the HTTP verb POST, go down the first branch, and if the incoming request uses the HTTP verb GET, go down the second branch.” Zooming in on the first branch here,

... choose
[ path "/add" >=> request (fun req ->
   match (req.formData "url") with
   | Choice1Of2 long ->
      let short = shorten long
      // You may need to amend the port number:
      let url = sprintf "localhost:8083/go/%s" short
      Successful.CREATED url
   | _ ->
      RequestErrors.BAD_REQUEST "Url not supplied")
]

you are saying “if the path of the request is /add, check that there is a value labelled url in the posted form data, and if there is, attempt to shorten it using the business logic defined above.” If the shortening succeeds, you return the HTTP response CREATED (201) together with the shortened URL.

In the second branch here,

... choose
[ pathScan "/go/%s" (fun short ->
   match tryResolve short with
   | Some long ->
      Redirection.MOVED_PERMANENTLY (sprintf "http://%s" long)
   | None ->
      RequestErrors.NOT_FOUND "Url not found")
]

you are trying to resolve a shortened URL, and if successful, you return a MOVED PERMANENTLY (301) message, which will cause the requesting browser to move on to the lengthened URL.

Suave.io’s choose idiom can be more than a little bewildering at first, but you can normally get things working by editing examples, such as the one above, to fit your requirements, until you get more of a handle on the underlying concepts. Suave.io pays off handsomely in situations where you want to put together a simple, non-blocking web service from scratch with the minimum of code. For example, it is extremely useful in cases where you need to embed web-serving features into a product that is not essentially web-based. It is also very useful when you want to write simple fake servers to form part of an automated testing infrastructure.

Creating Web Services

Suave.io is a great, F#-friendly way of creating an HTML-based service. But what if you want to create a more conventional web service based on Microsoft’s .NET Web API framework? (The following section is necessarily Microsoft .NET-specific because of the Visual Studio template that it uses.)

Let’s say you want the same URL shortener that you created above, but this time in the form of an ASP.NET Web API 2.0 project. Microsoft does not currently supply Visual Studio templates for creating ASP.NET MVC web sites or services in F#; but fortunately the open source community has stepped in to fill the breach. To benefit from their efforts, start by going the Visual Studio gallery web site ( https://visualstudiogallery.msdn.microsoft.com ) and searching for “F# MVC.” In the results you should find a package called “F# Web Application templates (MVC 5 and Web API 2.2)” by Ryan Riley and Daniel Mohl. Download and run the .vsix file.

Run Visual Studio and create a new Project. Under Installed ➤ Templates ➤ Visual F# ➤ ASPNET you should find a template called “F# ASP.NET MVC 5 and Web API 2” (Figure 10-4).

Figure 10-4. Selecting the “F# ASP.NET MVC 5/Web API 2” template

Select this, and in the same dialog give your project a name. Click OK. You will be presented with a dialog that allows you to choose from a number of related project types (Figure 10-5).

Figure 10-5. Selecting a project type

For this exercise, select “Web API 2.2”. Note that there are also options to create MVC web sites that also contain Web API services. You may find these useful for more ambitious projects.

Click OK, and you should see a fully working, but almost content-free, project. Before you change anything, check that the project compiles and runs, by simply clicking Visual Studio’s Run button. A short list of cars should appear in your browser. This has been rendered by a simple JavaScript program (main.js) that calls the service that your project provides. Try getting “under the hood” by going to your browser and editing the URL so that the API endpoint’s URL is called directly. The URL should look something like this, although the port number is likely to differ:

http://localhost:48213/api/cars

You should see the same content–a very short list of cars–but this time rendered in an unformatted form. Depending on your browser, you may see the car list in XML or in JSON.

Now you need to change the project to transform it into a Web API-based version of your URL shortener. Start by adding a new F# library project to the solution. Call the new project Shorten. Once you’ve added it, right-click it in Solution Explorer, select Properties, and change the Target Framework value to .NET 4.5.1 and ensure that the F# Target Runtime value is set to F# 4.0. Then select the web project, and change these same two properties for that project to the same values: .NET 4.5.1 and F# 4.0.

Now rename the Library1.fs file in that project to Shortener.fs and add code from Listing 10-4, replacing the existing content.

Listing 10-4. A URL Shortener

module Shortener

open System
open System.Collections.Concurrent

// Initialize a .NET random number generator:
let private random = Random()

// Generate a random string of lower-case letters
// of a specified length.  WARNING: this has the
// potential to generate rude words!
let private randomString len =
   Array.init len (fun _ -> random.Next(26) + 97 |> char)
   |> String

// A dictionary of mappings from the long to the shortened URLs:
let private longToShort = ConcurrentDictionary<string, string>()
// A dictionary of mappings from the shortened to the long URLs:
let private shortToLong = ConcurrentDictionary<string, string>()

// Shorten a URL, and as a side-effect store the short->long
// and long->short mappings:
let Shorten (long : string) =
   longToShort.GetOrAdd(long, fun _ ->
      let short = randomString 5
      shortToLong.[short] <- long
      short)

// Try to resolve a shortened URL to the long version:
let TryResolve (short : string) =
   match shortToLong.TryGetValue short with
   | true, long -> Some long
   | false, _-> None                                          

You’ll notice that this code is pretty much identical to the code you used in the previous, Suave-based example.

Now you need to integrate your URL-shortening library with the web service code created by the template. Start by adding a reference to your new Shorten library in the original Web project: in Solution Explorer, right-click the References node of the web project and select Add Reference. Find and tick the Shorten item in the Solution tab. Now find the file called CarsController.fs in the web project, and rename it ShortenerController.fs. In this file, remove everything except the initial “open” statements, and then add your own controller code from Listing 10-5.

Listing 10-5. Controller for the URL Shortener

[<RoutePrefix("api")>]
type ShortenerValuesController() =
    inherit ApiController()

    [<Route("add")>]
    [<HttpPost>]
    member x.Add(long : string) : IHttpActionResult =
        let short = Shortener.Shorten(long)
        // You may need to update the port number:
        let url = sprintf "localhost:48213/api/go/%s" short
        x.Ok(url) :> _

As before, you may need to change the port number. Here you are creating your own ASP.NET Web API controller. Let’s go through the code line by line.

You start by defining a controller:

[<RoutePrefix("api")>]
type ShortenerController() =
    inherit ApiController()

This inherits from ApiController, which is a standard ASP.NET class that provides various methods useful in the construction of Web APIs. The RoutePrefix attribute means that any handlers within this controller can be reached using a URL that starts with api (after the hostname section, obviously).

Here you define a handler for a route called add:

[<Route("add")>]
[<HttpPost>]
member x.Add(long : string) : IHttpActionResult =

The handler will only respond to the HTTP method POST, and accepts one argument called long, which the caller will use to send in the long version of the URL they want to be shortened. The hander returns an IHttpActionResult, which enables you to return an HTTP response message that contains a response code (e.g. 200 OK) and a message (such as an error description if there is a problem).

let short = Shortener.Shorten(long)
// You may need to update the port number:
let url = sprintf "localhost:48213/api/go/%s" short

These lines call your shortening service, and embed the shortened version in a URL which the caller can later use to resolve the shortened version and navigate to the original site.

The last line is a little obscure:

x.Ok(url) :> _

The first part, x.Ok(url), is where you return the shortened URL to the caller, along with an OK (200) return code that tells the caller that the transaction went successfully. The final part, :> _, results from F#’s requirement that types are cast explicitly to interfaces when an interface type is expected. The Ok method of ApiController returns something that implements IHttpActionResult, but it isn’t of itself an IHttpActionResult, so you must cast it using the :> operator. The good news is that F# has enough information (from the defined return type of the Add method) to infer that what you need to cast to is indeed an IHttpActionResult, so that you can use the _ symbol instead of the whole interface name.

Now you need to add a route that will let the caller resolve a previously shortened URL. See Listing 10-6 for the code.

Listing 10-6. Adding a Go Route for the URL Shortener

[<Route("go/{short}")>]
[<HttpGet>]
member x.Go(short : string) : IHttpActionResult =
   match Shortener.TryResolve short with
   | Some long ->
       let url = sprintf "http://%s" long
       x.Redirect(url) :> _
   | None ->
       x.NotFound() :> _

Here you use the Route attribute to say that this handler will respond to the route go followed by a value called short, which will be the shortened URL that the caller wants to resolve. This handler will only respond to GET verbs. In the body of the handler, you call your TryResolve method, and if it succeeds, you return a redirect response (code 302), sending the caller to the full version of the URL. If the shortened URL couldn’t be resolved, you return a “not found” response, code 404. On both the return branches you again have to cast the returned value to IHttpActionResult using :> _.

It’s time to try out your Web API-based URL shortener. Make sure your Web API project is set as the startup project and run it. You should get a more or less empty page in your browser. Fire up Fiddler or your favorite web debugging tool and use it to send a post to the api/add route, as shown in Figure 10-6.

Figure 10-6. Using Fiddler to send a post with a query string

You should get the shortened URL in your response, similar to Figure 10-7.

Figure 10-7. Viewing a response in Fiddler

Copy this path back into your browser address bar and hit Enter. Voila! You’re redirected to the original site.

Summary

This chapter covered some useful, F#-friendly options for creating distributed applications. You looked at System.Net.WebRequest, which allows you to request data using HTTP, and you used this to access data in published Google spreadsheets. You saw how to serve HTTP requests in two different ways: using Suave.io, which allows a highly idiomatic and concise approach, and using Web API via an open source Visual Studio template, which developers coming from an ASP .NET MVC background may find more familiar. What you saw here only scratches the surface of what is possible using F# and the Internet, so we encourage you to go out and explore F#’s rich, largely open source network ecosystem.