Now that we are familiar with the JSON and XML formats, we can start using them to handle HTTP requests and responses in the context of a Play project.
To exhibit these behaviors, we are going to call an online web service, the iTunes media library, which is available and documented at http://www.apple.com/itunes/affiliates/resources/documentation/itunes-store-web-service-search-api.html.
It returns JSON messages on search invocations. We can, for instance, call the API with the following URL and parameters:
https://itunes.apple.com/search?term=angry+birds&country=se&entity=software
The term parameter filters every item in the library that has to do with Angry Birds and the entity parameter retains only software items. We also apply an additional filter to query only the Swedish App Store.
Note
If you don't have it already in your build.sbt file, you may need to add the dispatch dependency at this point, the same way we did while working with HTTP in Chapter 3, Understanding the Scala Ecosystem:
libraryDependencies += "net.databinder.dispatch" %% "dispatch-core" % "0.11.0"
scala> import dispatch._ import dispatch._ scala> import Defaults._ import Defaults._ scala> val request = url("https://itunes.apple.com/search") request: dispatch.Req = Req(<function1>)
Parameters that will be part of our GET method call can be expressed as (key,value) tuples in a Scala Map:
scala> val params = Map("term" -> "angry birds", "country" -> "se", "entity" -> "software") params: scala.collection.immutable.Map[String,String] = Map(term -> angry birds, country -> se, entity -> software) scala> val result = Http( request <<? params OK as.String).either result: dispatch.Future[Either[Throwable,String]] = scala.concurrent.impl.Promise$DefaultPromise@7a707f7c
The type of result in this case is Future[Either[Throwable,String]], which means we can extract a successful invocation as well as a failed execution by pattern matching as follows:
scala> val response = result() match { case Right(content)=> "Answer: "+ content case Left(StatusCode(404))=> "404 Not Found" case Left(x) => x.printStackTrace() } response: Any = "Answer: { "resultCount":50, "results": [ {"kind":"software", "features":["gameCenter"], "supportedDevices":["iPhone5s", "iPad23G", "iPadThirdGen", "iPodTouchThirdGen", "iPadFourthGen4G", "iPhone4S", "iPad3G", "iPhone5", "iPadWifi", "iPhone5c", "iPad2Wifi", "iPadMini", "iPadThirdGen4G", "iPodTouchourthGen", "iPhone4", "iPadFourthGen", "iPhone-3GS", "iPodTouchFifthGen", "iPadMini4G"], "isGameCenterEnabled":true, "artistViewUrl":"https://itunes.apple.com/se/artist/rovio-entertainment-ltd/id298910979?uo=4", "artworkUrl60":"http://a336.phobos.apple.com/us/r30/Purple2/v4/6c/20/98/6c2098f0-f572-46bb-f7bd-e4528fe31db8/Icon.png", "screenshotUrls":["http://a2.mzstatic.com/eu/r30/Purple/v4/c0/eb/59/c0eb597b-a3d6-c9af-32a7-f107994a595c/screen1136x1136.jpeg", "http://a4.mzst...
Whenever you need to integrate your services with external systems that you do not own or that are not available until you deploy them in production, it can be cumbersome to test the interaction of messages that are sent and received. An efficient way to avoid calling a real service is to replace it with mock messages, that is, hardcoded responses that will cut short the real interaction, especially if you need to run your tests as part of an automated process (for instance, daily as a Jenkins job). Returning a plain JSON message from within a Play controller is very straightforward, as the following example illustrates:
package controllers
import play.api.mvc._
import play.api.libs.json._
import views._
object MockMarketplaceController extends Controller {
case class AppStoreSearch(artistName: String, artistLinkUrl: String)
implicit val appStoreSearchFormat = Json.format[AppStoreSearch]
def mockSearch() = Action {
val result = List(AppStoreSearch("Van Gogh", " http://www.vangoghmuseum.nl/"), AppStoreSearch("Monet", " http://www.claudemonetgallery.org "))
Ok(Json.toJson(result))
}
}The Json.format[. . .] declaration that involves Reads, Writes, and Format will be explained later on in this section when we invoke web services, so we can skip discussing that part for the moment.
To try out this controller, you can either create a new Play project, or, as we did before, just add this controller to the application we generated out of an existing database in the last section of Chapter 6, Database Access and the Future of ORM. You also need to add a route to the route file under conf/ as follows:
GET /mocksearch controllers.MockMarketplaceController.mockSearch
Once the app is running, accessing the http://localhost:9000/mocksearch URL in a browser will return the following mock JSON message:
Another convenient way to obtain a JSON test message that you can use to mock a response is to use the online service found at http://json-generator.appspot.com. It consists of a JSON generator that we can use as it is by simply clicking on the Generate button. By default, it will generate a JSON sample including random data in the panel to the right of the browser window, but adhering to the structure defined in the panel to the left, as illustrated in the following screenshot:
You can click on the Copy to clipboard button and paste the resulting mock message directly into the response of the Play controller.
In the previous section, to quickly experiment with the App Store search API, we have used the dispatch library; we have already introduced this library in Chapter 3, Understanding the Scala Ecosystem. Play provides its own HTTP library to be able to interact with other online web services. It is also built on top of the Java AsyncHttpClient library (https://github.com/AsyncHttpClient/async-http-client), as dispatch is.
Before we dive into invoking REST web services from Play controllers, let's experiment a little bit with Play web services from the REPL. In a terminal window, either create a new Play project or go to the root directory of the one we have used in the previous sections. Once you get a Scala prompt after having typed the > play console command, enter the following commands:
scala> import play.api.libs.ws._ import play.api.libs.ws._ scala> import scala.concurrent.Future import scala.concurrent.Future
Since we are going to invoke a web service asynchronously, we need an execution context to handle the Future placeholder:
scala> implicit val context = scala.concurrent.ExecutionContext.Implicits.global context: scala.concurrent.ExecutionContextExecutor = scala.concurrent.impl.ExecutionContextImpl@44d8bd53
We can now define a service URL that needs to be called. Here, we will take a simple web service that returns the geographic location of a site given as a parameter, according to the following signature:
http://freegeoip.net/{format}/{site}The format parameter can either be json or xml, and the site will be a reference to a website:
scala> val url = "http://freegeoip.net/json/www.google.com" url: String = http://freegeoip.net/json/www.google.com scala> val futureResult: Future[String] = WS.url(url).get().map { response => (response.json "region_name").as[String] } futureResult: scala.concurrent.Future[String] = scala.concurrent.impl.Promise$DefaultPromise@e4bc0ba scala> futureResult.onComplete(println) Success(California)
As we saw earlier in Chapter 3, Understanding the Scala Ecosystem, when working with the dispatch library, a Future is a placeholder that contains the result of an asynchronous computation and can be in two states, either completed or not. Here, we want to print the result once it is available.
We have only extracted the region_name item from the response; the whole JSON document is as follows:
{
"ip":"173.194.64.106",
"country_code":"US",
"country_name":"United States",
"region_code":"CA",
"region_name":"California",
"city":"Mountain View",
"zipcode":"94043",
"latitude":37.4192,
"longitude":-122.0574,
"metro_code":"807",
"areacode":"650"
}We can encapsulate part of the response if we want to by creating a case class as follows:
scala> case class Location(latitude:Double, longitude:Double, region:String, country:String) defined class Location
The play-json library includes support to read/write JSON structures via Reads/Writes/Format combinators based on JsPath so that validation can be made on the fly. If you are interested in all the details behind the use of these combinators, you may want to read through the blog at http://mandubian.com/2012/09/08/unveiling-play-2-dot-1-json-api-part1-jspath-reads-combinators/.
scala> import play.api.libs.json._ import play.api.libs.json._ scala> import play.api.libs.functional.syntax._ import play.api.libs.functional.syntax._ scala> implicit val locationReads: Reads[Location] = ( (__ "latitude").read[Double] and (__ "longitude").read[Double] and (__ "region_name").read[String] and (__ "country").read[String] )(Location.apply _) locationReads: play.api.libs.json.Reads[Location] = play.api.libs.json.Reads$$anon$8@4a13875b locationReads: play.api.libs.json.Reads[Location] = play.api.libs.json.Reads$$anon$8@5430c881
Now, invoking the validate method on the JSON response will verify that the data we receive is well-formed and with acceptable values.
scala> val futureResult: Future[JsResult[Location]] = WS.url(url).get().map { response => response.json.validate[Location] } futureResult: scala.concurrent.Future[play.api.libs.json.JsResult[Location]] = scala.concurrent.impl.Promise$DefaultPromise@3168c842 scala> futureResult.onComplete(println) Success(JsError(List((/country,List(ValidationError(error.path.missing,WrappedArray()))))))
The previous JsError object illustrates a validation that failed; it detected that the country element is not found in the response. In fact, the correct spelling is country_name instead of country, which we can correct in our locationReads declaration. This time validation goes through and what we get as a response is a JsSuccess object containing the latitude and longitude information as we expect it:
scala> implicit val locationReads: Reads[Location] = ( (__ "latitude").read[Double] and (__ "longitude").read[Double] and (__ "region_name").read[String] and (__ "country_name").read[String] )(Location.apply _) locationReads: play.api.libs.json.Reads[Location] = play.api.libs.json.Reads$$anon$8@70aab9ed scala> val futureResult: Future[JsResult[Location]] = WS.url(url).get().map { response => response.json.validate[Location] } futureResult: scala.concurrent.Future[play.api.libs.json.JsResult[Location]] = scala.concurrent.impl.Promise$DefaultPromise@361c5860 scala> futureResult.onComplete(println) scala> Success(JsSuccess(Location(37.4192,-122.0574,California,United States),))
Now, let's create a sample controller that invokes a web service to retrieve some data from the App Store:
package controllers
import play.api._
import play.api.mvc._
import play.api.libs.ws.WS
import scala.concurrent.ExecutionContext.Implicits.global
import play.api.libs.json._
import play.api.libs.functional.syntax._
import scala.concurrent.Future
import views._
import models._
object MarketplaceController extends Controller {
val pageSize = 10
val appStoreUrl = "https://itunes.apple.com/search"
def list(page: Int, orderBy: Int, filter: String = "*") = Action.async { implicit request =>
val futureWSResponse =
WS.url(appStoreUrl)
.withQueryString("term" -> filter, "country" -> "se", "entity" -> "software")
.get()
futureWSResponse map { resp =>
val json = resp.json
val jsResult = json.validate[AppResult]
jsResult.map {
case AppResult(count, res) =>
Ok(html.marketplace.list(
Page(res,
page,
offset = pageSize * page,
count),
orderBy,
filter))
}.recoverTotal {
e => BadRequest("Detected error:" + JsError.toFlatJson(e))
}
}
}
}Here, the call to the web service is illustrated by invoking methods on the WS class, first the url method giving the URL, then the withQueryString method with input parameters given as a sequence of key->value pairs. Notice that the returned type is a Future, meaning our web service is asynchronous. recoverTotal takes a function that will return a default value after managing the error. The line json.validate[AppResult] makes the JSON response validated against an AppResult object that is specified here (as part of a Marketplace.scala file in app/models/ folder):
package models
import play.api.libs.json._
import play.api.libs.functional.syntax._
case class AppInfo(id: Long, name: String, author: String, authorUrl:String,
category: String, picture: String, formattedPrice: String, price: Double)
object AppInfo {
implicit val appInfoFormat = (
(__ "trackId").format[Long] and
(__ "trackName").format[String] and
(__ "artistName").format[String] and
(__ "artistViewUrl").format[String] and
(__ "primaryGenreName").format[String] and
(__ "artworkUrl60").format[String] and
(__ "formattedPrice").format[String] and
(__ "price").format[Double])(AppInfo.apply, unlift(AppInfo.unapply))
}
case class AppResult(resultCount: Int, results: Array[AppInfo])
object AppResult {
implicit val appResultFormat = (
(__ "resultCount").format[Int] and
(__ \ "results").format[Array[AppInfo]])(AppResult.apply, unlift(AppResult.unapply))
}The AppResult and AppInfo case classes are created to encapsulate the elements that we care about for our service. As you may have seen when first experimenting with the API, most of the search queries to the App Store return a large amount of elements, most of which we may not need. This is why, using some Scala syntactic sugar with combinators, we can validate the JSON response on the fly and directly extract the elements of interest. Before trying out this web service call, we just need to add the needed route to the routes file under conf/, as shown in the following code:
GET /marketplace controllers.MarketplaceController.list(p:Int ?= 0, s:Int ?= 2, f ?= "*")
Finally, before launching the application in a web browser, we also need the sample view that is referred to in the MarketplaceController.scala file by html.marketplace.list and created in a list.scala.html file under views/marketplace/ in several parts as shown in the following code:
@(currentPage: Page[AppInfo], currentOrderBy: Int, currentFilter:
String)(implicit flash: play.api.mvc.Flash)
...
@main("Welcome to Play 2.0") {
<h1>@Messages("marketplace.list.title", currentPage.total)</h1>
@flash.get("success").map { message =>
<div class="alert-message warning">
<strong>Done!</strong> @message
</div>
}
<div id="actions">
@helper.form(action=routes.MarketplaceController.list()) { <input
type="search" id="searchbox" name="f" value="@currentFilter"
placeholder="Filter by name..."> <input type="submit"
id="searchsubmit" value="Filter by name" class="btn primary">
}
</div>
...The first part of the view only consists of helper methods to navigate and is generated the same way as we did for the CRUD sample generation in Chapter 6, Database Access and the Future of ORM. The second part of the view includes the JSON elements we have retrieved from the web service:
...
@Option(currentPage.items).filterNot(_.isEmpty).map { entities =>
<table class="computers zebra-striped">
<thead>
<tr>
@header(2, "Picture")
@header(4, "Name")
@header(5, "Author")
@header(6, "IPO")
@header(7, "Category")
@header(8, "Price")
</tr>
</thead>
<tbody>
@entities.map{ entity =>
<tr>
<td>
<img
src="@entity.picture"
width="60" height="60" alt="image description" />
</td>
<td>@entity.name</td>
<td><a href="@entity.authorUrl" class="new-btn btn-back">@entity.author</a></td>
<td>@entity.category</td>
<td>@entity.formattedPrice</td>
</tr>
}
</tbody>
</table>
...The third and final part of the view is handling pagination:
...
<div id="pagination" class="pagination">
<ul>
@currentPage.prev.map { page =>
<li class="prev"><a href="@link(page)">← Previous</a></li>
}.getOrElse {
<li class="prev disabled"><a>← Previous</a></li> }
<li class="current"><a>Displaying @(currentPage.offset + 1)
to @(currentPage.offset + entities.size) of @currentPage.total</a></li>
@currentPage.next.map { page =>
<li class="next"><a href="@link(page)">Next →</a></li>
}.getOrElse {
<li class="next disabled"><a>Next →</a></li> }
</ul>
</div>
}.getOrElse {
<div class="well">
<em>Nothing to display</em>
</div>
} }Once we re-launch the Play app with > play run and access (through a web browser) our local http://localhost:9000/marketplace?f=candy+crush URL that includes a default search from the App Store (the f parameter stands for filter), we will obtain a page similar to the following screenshot:
