Instead of applying successive map transformations to extract XML, in Scala, we can use a powerful construct that represents the silver bullet of iterations called for comprehension or for expression. Unlike the for loops found in Java and used for iterating, for comprehension returns a result. They are specified as follows:

for (sequence) yield expression

In the preceding code, sequence can contain the following components:

A for comprehension construct is much easier to visualize with a few concrete examples:

scala> for {
         elem <- List(1,2,3,4,5)
       } yield "T" + elem
res3: List[String] = List(T1, T2, T3, T4, T5)

We have transformed List[Int] into List[String] using only one generator. Using two generators is illustrated in the following code:

scala> for {
         word <- List("Hello","Scala")
         char <- word
       } yield char.isLower
res4: List[Boolean] = List(false, true, true, true, true, false, true, true, true, true)

We can add a filter on any generator. For instance, if we want to retain only the uppercase characters of every word, we can write as follows:

scala> for {
         word <- List("Hello","Scala")
         char <- word if char.isUpper
       } yield char
res5: List[Char] = List(H, S)

In the following example, we illustrate how to add a local variable definition:

scala> for {
         word <- List("Hello","Scala")
         char <- word
         lowerChar = char.toLower
       } yield lowerChar
res6: List[Char] = List(h, e, l, l, o, s, c, a, l, a)

Going back to our HTTP Groupon service, we can now extract names of cities using for comprehension as follows:

scala> def extractCityNames(xml: scala.xml.Elem) = 
         for {
           elem <- xml \ "division"
           name <- elem  "name"
         } yield name.text
extractCityNames: (xml: scala.xml.Elem)scala.collection.immutable.Seq[String]
scala> val cityNames = extractCityNames(citiesAsXML())
cityNames: scala.collection.immutable.Seq[String] = List(Abbotsford, Abilene, TX, Akron / Canton, Albany / Capital Region, Albuquerque, Allentown / Reading, Amarillo, Anchorage...

To be able to query the second part of the API to retrieve special discount deals for a specific area, we also need the latitude and longitude information from the queried cities. Let's do that by returning a tuple including three elements, the first one being the name, the second being the latitude, and the third being the longitude:

scala> def extractCityLocations(xml: scala.xml.Elem) = 
         for {
           elem<- xml \ "division"
           name <- elem  "name"
           latitude <- elem  "lat"
           longitude <- elem  "lng"
         } yield (name.text,latitude.text,longitude.text)
extractCityLocations: (xml: scala.xml.Elem)scala.collection.immutable.Seq[(String, String, String)]
scala> val cityLocations = extractCityLocations(citiesAsXML())
cityLocations: scala.collection.immutable.Seq[(String, String, String)] = List((Abbotsford,49.0568,-122.285), (Abilene, TX,32.4487,-99.7331), (Akron / Canton,41.0814,-81.519), (Albany / Capital Region,42.6526,-73.7562)...

Out of the list of returned cities, we might be interested in just one for now. Let's retrieve only the location for Honolulu using the following command:

scala> val (honolulu,lat,lng) = cityLocations find (_._1 == "Honolulu") getOrElse("Honolulu","21","-157")
honolulu: String = Honolulu
lat: String = 21.3069
lng: String = -157.858

The find method in the preceding code takes a predicate as a parameter. As its return type is an Option value, we can retrieve its content by invoking getOrElse where we can write a default value in case the find method does not return any match.

An alternative representation could be done using pattern matching, briefly described in Chapter 1, Programming Interactively within Your Project, as follows:

scala> val honolulu =
         cityLocations find { case( city, _, _ ) => 
           city == "Honolulu"
         }
honolulu: Option[(String, String, String)] = Some((Honolulu,21.3069,-157.858))

The regular syntax of pattern matching normally uses the match keyword before all the case alternatives, so here it is a simplified notation where the match keyword is implicit. The underscore (_) as well as the city variable given in case are wildcards in the pattern matching. We could have given these underscores variable names but it is not necessary as we are not using them in the predicate (that is, city == "Honolulu").

Let's now create a request to query for all the deals that match a particular geographic area:

scala> val dealsByGeoArea =   url("http://api.groupon.com/v2/deals.xml?client_id=<your client_id>")
dealsByGeoArea: dispatch.Req = Req(<function1>)

An alternative to handle data as tuples is to define case classes to encapsulate elements in a convenient and reusable way. We can, therefore, define a Deal class and rewrite our previous for comprehension statement returning the Deal instances instead of tuples:

scala> case class Deal(title:String = "",dealUrl:String = "", tag:String = "")
defined class Deal

scala> def extractDeals(xml: scala.xml.Elem) = 
         for {
           deal <- xml \ "deal"
           title = (deal \ "title").text
           dealUrl = (deal \ "dealUrl").text
           tag = (deal \ "tag"  "name").text
         } yield Deal(title, dealUrl, tag)
extractDeals: (xml: scala.xml.Elem)scala.collection.immutable.Seq[Deal]

As we did previously for retrieving cities, we can now retrieve deals via HTTP GET and parse XML this time for the particular city of Honolulu, knowing its latitude and longitude, as follows:

scala> val dealsInHonolulu = 
  Http(dealsByGeoArea <<? Map("lat"->lat,"lng"->lng) OK as.xml.Elem)
dealsInHonolulu: dispatch.Future[scala.xml.Elem] = scala.concurrent.impl.Promise$DefaultPromise@a1f0cb1

The <<? operator means that we attach input parameters of a GET method to the dealsByGeoArea request. The Map object contains the parameters. It is equivalent to the normal representation of HTTP GET where we put the input parameters as key/value pairs in the URL (that is, request_url?param1=value1;param2=value2). This is in contrast with the << operator, which would have specified a POST request. Creating a structured sequence of Deal instances out of the raw XML produced by the dealsInHonolulu() service call can be written as follows:

scala> val deals = extractDeals(dealsInHonolulu())
deals: scala.collection.immutable.Seq[Deal] = List(Deal(Laundry Folding StylesExam with Posture Analysis and One or Three Adjustments at Cassandra Peterson Chiropractic (Up to 85% Off)One initial consultation, one exam, one posture analysis, and one adjustmentOne initial consultation, one exam, one posture analysis, and three adjustments,http://www.groupon.com/deals/cassandra-peterson-chiropractic,Beauty & Spas), Deal(Laundry Folding Styles1.5-Hour Whale-Watching Sunset Tour for an Adult or Child from Island Water Sports Hawaii (50% Off) A 1.5-hour whale watching sunset tour for one childA 1.5-hour whale watching sunset tour for one adult,http://www.groupon.com/deals/island-water-sports-hawaii-18,Arts and EntertainmentOutdoor Pursuits), Deal(Dog or Horse?$25 for Take-Home Teeth-Whit...

Sorting the list of deals by their category is only a matter of applying a groupBy method on the collection as follows:

scala> val sortedDeals = deals groupBy(_.tag)
sortedDeals: scala.collection.immutable.Map[String,scala.collection.immutable.Seq[Deal]] = Map("" -> List(Deal(SkeleCopSix Bottles of 3 Wine Men 2009 Merlot with Shipping Included6 Bottles of Premium Red Wine,http://www.groupon.com/deals/gg-3-wine-men-2009-merlot-package,), Deal(Famous...

Notice how the groupBy method is a very convenient way of applying the Map part of a MapReduce job operating on a collection, in our case creating a Map object where keys are the tags or categories of the Groupon deals and values are a list of the deals that belong to the specific category. A possible tiny Reduce operation on the Map object can, for example, consist of counting the number of deals for each category, using the mapValues method that transforms the values of this (key,value) store:

scala> val nbOfDealsPerTag = sortedDeals mapValues(_.size)
nbOfDealsPerTag: scala.collection.immutable.Map[String,Int] = Map("" -> 2, Arts and EntertainmentOutdoor Pursuits -> 1, Beauty & Spas ->3, Food & DrinkCandy Stores -> 1, ShoppingGifts & Giving -> 1, ShoppingFraming -> 1, EducationSpecialty Schools -> 1, Tickets -> 1, Services -> 1, TravelTravel AgenciesEurope, Asia, Africa, & Oceania -> 1)

The example we went through only explores the surface of what we can do with HTTP tools such as dispatch and much more is described in their documentation. The direct interaction with the REPL greatly enhances the learning curve of such APIs.

There are several excellent alternatives of lightweight frameworks for dealing with HTTP interaction, and in the case of dispatch, we have only looked at the client side of things. Lightweight REST APIs can, therefore, be constructed by frameworks such as Unfiltered, Finagle, Scalatra, or Spray to name a few. Spray is currently being architected again to become the HTTP layer of the Play framework (on top of Akka); technologies we are going to cover later on in this book.