Chapter 5. Handling JSON

This chapter covers

• Consuming and producing JSON using the Scalatra JSON module
• Handling heterogeneity when working with JSON
• Using JSONP

JSON is a common data interchange format for semistructured data. It’s human-readable and supports primitive, array, and object values. Additionally, for web services it’s the natural data format of the browser, due to its close relationship with JavaScript.

Scalatra offers a module that supports an application working with JSON. This chapter covers its use with practical examples taken from the food domain. Who knows, maybe you’ll also learn a useful recipe!

5.1. Introducing JsonSupport

The Scalatra JSON module extends an application’s request handling with two facets:

• An incoming JSON request is parsed to a JSON value.
• A JSON value is written to the response as JSON text, as a result of a route action.

Let’s see how you can add JSON support to an application.

5.1.1. Adding JSON support to an application

Because Scalatra’s JSON support is an optional module, it needs to be added as a dependency to the sbt build definition:

libraryDependencies ++= Seq(
  "org.scalatra" %% "scalatra-json" % ScalatraVersion,
  "org.json4s"   %% "json4s-jackson" % "3.3.0")

Those dependencies pull in scalatra-json and Json4s. Json4s is a Scala JSON library; scalatra-json is the actual JSON module that builds on top of Json4s, reusing its JSON-handling methods and JValue data type. This chapter covers a fair amount of Json4s, but for full documentation refer to the official website at http://json4s.org.

Let’s look at what an application needs to do in order to handle JSON. Listing 5.1 shows a minimal example application defining two routes in a trait. You’ll need to mix these into a class in order to run them.

The GET route builds and returns a JSON document representing a delicious snack. The POST route extracts a tuple out of a JSON request and prints it to the console.

Listing 5.1. A basic JSON example application

Let’s take a closer look at what happens in this listing. The JacksonJsonSupport trait is responsible for parsing an incoming JSON request to a JValue as well as for writing a result of type JValue as JSON text to a response. This is accomplished by hooking into Scalatra’s request-response cycle.

The GET route returns a JValue as a result, which is written as JSON text to the response. Generally, if you intend to output JSON using Scalatra’s JSON support, your JSON routes should always return a value of type JValue. When the route result is of type JValue, it’s set implicitly to application/json by JacksonJsonSupport.

Because an HTTP request contains the JSON in textual form, JacksonJsonSupport provides the function parsedBody, which parses the JSON text and returns a value of type JValue. parsedBody does several things:

• Parses the JSON text from the HTTP request body and always returns a JValue.
• Returns JNothing if the JSON isn’t well-formed.
• Returns JNothing if the HTTP request doesn’t have the Content-Type header set to application/json.
• Parses once for each request, and caches the result. Subsequent calls return the cached result.

The request will be parsed eagerly before the route is invoked. With the JSON parsed from the request, the next logical step is usually to extract useful information from the JSON data. This can include the following:

• Selecting specific parts of a JSON document
• Extracting values of other types from a JSON primitive, array, or object
• Handling optional, missing, and null values

You may have noticed the implicit jsonFormats value. It holds serialization configuration telling Json4s how to handle specific cases when parsing and writing JSON. For instance, a JSON number can either be handled as a Double or a BigDecimal. The JSON support requires you to define this value.

The Formats type is explained in detail in section 5.3.1, and we’ll look at constructing and working with JSON in more detail in section 5.2. For now we’ll stay with the DefaultFormats settings. As a rule, you should always return a JValue from an action and use the parsedBody method to access the request’s JSON.

Let’s now take a closer look at the JValue type.

5.1.2. Introducing the JValue type

The JValue type represents a JSON intermediate model. You can think of a value of type JValue as the abstract representation of a JSON document, often called its abstract syntax tree. This simplifies further read and modify operations on that JSON data.

JSON is a typed language, so a JValue needs to support all types that may appear in a JSON document. Those are objects, arrays, and primitive values (such as string, number, Boolean, and null). For each JSON type, there’s a counterpart in the intermediate model. Listing 5.2 shows the various types. Note that a JValue boxes a native Scala value—it acts as a container for a value of that type.

Listing 5.2. JValue type

sealed trait JValue
case class JString(s: String) extends JValue
case class JBool(value: Boolean) extends JValue

trait JNumber
case class JInt(num: BigInt) extends JValue with JNumber
case class JDouble(num: Double) extends JValue with JNumber
case class JDecimal(num: BigDecimal) extends JValue with JNumber

type JField = (String, JValue)
case class JObject(obj: List[JField]) extends JValue
case class JArray(arr: List[JValue]) extends JValue

case object JNothing extends JValue
case object JNull extends JValue

Let’s look at how a JSON document is represented as a JValue. The following listing shows a sample JSON document.

Listing 5.3. A sample JSON document

{
  "label" : "Foo bar",
  "fairTrade" : true,
  "tags" : [ "bio", "chocolate" ]
}

Figure 5.1 shows how the document looks as a JValue. A JSON document forms a tree. The top-level object is a JObject containing a set of key-value pairs representing the JSON object. The keys (label, tags, and fairTrade) are of type String. The values are of type JValue.

Figure 5.1. A JSON document as a JValue

A document tree can be arbitrarily deep. The array at the key tags is represented as a value of type JArray.

Constructing this document from scratch using the JValue types is straightforward. Let’s construct some objects and see what the resulting Scala types look like. You can play around with all of the JSON code in a Scala console. Result types are shown in comments below the assignment code.

val fooBar = JObject(
  "label" -> JString("Foo bar"),
  "fairTrade" -> JBool(true),
  "tags" -> JArray(List(JString("bio"), JString("chocolate"))))

// fooBar: org.json4s.JValue =
//   JObject(List((label,JString(Foo bar)), ...

The JValue can also be parsed from JSON text using the parse method, which is defined on the JsonMethods object:

import org.json4s.jackson.JsonMethods.parse

val txt =
  """{
     | "tags": ["bio","chocolate"],
     | "label": "Foo bar",
     | "fairTrade": true
     |}""".stripMargin

val parsed = parse(txt)
// parsed: org.json4s.JValue =
//   JObject(List((label,JString(Foo bar)), ...

When comparing the previously constructed value against the parsed value, they are equal. Note that the order of the fields in a JObject doesn’t matter for equality:

fooBar == parsed
// res: Boolean = true

You should now have a basic understanding of the scope of Scalatra’s JSON support. Remember that parsing and serialization is done by the JSON support transparently in a request-response cycle. Also note that it is easy to integrate another JSON library on your own, such as argonaut or play-json.

Now let’s look at how you can work with JSON data.

5.2. Producing and consuming JSON

When receiving JSON, you’ll probably want to be able to extract useful information. When sending JSON, you’ll need to be able to construct it. Let’s start with getting to know the JSON intermediate data model. We’ll serialize a Recipe object instance as an example.

Applications use JSON to encode data that’s delivered to the client, so the application needs to be able to create a JSON message. In this section, you’ll learn a simple but effective DSL for constructing a JSON value that’s then delivered to a client as the body of an HTTP response.

There are two ways to create JSON, which can also be combined:

• Construct it from scratch using the JValue types and the DSL.
• Decompose existing values to a JValue.

When your application receives a JSON request, it needs to be able to interpret the JSON contained in that request. This section will introduce you to the following:

• Parsing a JSON value from an HTTP request
• Navigating a JSON value to find the required information
• Extracting a value of a type from a JSON value
• Handling of extraction failures

The following listing shows a recipe for an Italian pasta dish as a JSON document. We’ll use this JSON document to explain the concepts in this section.

Listing 5.4. Example recipe as JSON text

{
  "title": "Penne with cocktail tomatoes, Rucola and Goat cheese",
  "details": {
    "cuisine": "italian",
    "vegetarian": true
  },
  "ingredients": [{
      "label": "Penne",
      "quantity": "250g"
    }, {
      "label": "Cocktail tomatoes",
      "quantity": "300g"
    }, {
      "label": "Rucola",
      "quantity": "2 handful"
    }, {
      "label": "Goat cheese",
      "quantity": "200g"
    }, {
      "label": "Garlic cloves",
      "quantity": "2 tsps"
    }],
  "steps": [
    "Cook noodles until aldente.",
    "Quarter the tomatoes, wash the rucola, dice
       the goat's cheese and cut the garlic.",
    "Heat olive oil in a pan, add the garlic and the tomatoes and
       steam short (approx. for 5 minutes).",
    "Shortly before the noodles are ready add the rucola
       to the tomatoes.",
    "Drain the noodles and mix with the tomatoes,
      finally add the goat's cheese and serve."
  ]
}

Although it’s possible to stick with the JValue types when processing JSON, it’s often helpful or necessary to also employ classes. The JSON language features basic types, but classes give you additional expressivity and type safety. In addition, other third-party libraries, such as database-mapping layers, may require the use of classes.

The domain model consists of the types Recipe, RecipeDetails, and IngredientLine. The code is shown in the following listing.

Listing 5.5. Example recipe domain model

case class Recipe(title: String,
                  details: RecipeDetails,
                  ingredients: List[IngredientLine],
                  steps: List[String])

case class RecipeDetails(cuisine: String, vegetarian: Boolean,
                         diet: Option[String])

case class IngredientLine(label: String, quantity: String)

5.2.1. Producing JSON

There are three methods of creating a JValue in the Json4s library:

• Using the JValue types
• Using the JValue DSL
• Decomposing values to a JValue using a generic reflection-based method

Creating a JValue from scratch using the JValue types has the benefit that you can be explicit about the resulting JSON. But it’s also the most verbose approach and can lead to cumbersome code. You saw this approach in section 5.1.2, so we’ll employ the other two approaches here.

The DSL is based on a few simple operators and implicit conversion. In order to use the JSON DSL, it needs to be made available in the current scope by importing org.json4s.JsonDSL._. This enables conversion from primitive and collection types to a JValue by specifying the expected type for an expression:

val jsString: JValue = "italian"
// jsString: org.json4s.JValue = JString(italian)

val jsBool: JValue = true
// jsBool: org.json4s.JValue = JBool(true)

A JSON object is constructed from Tuple2[String, A], where A has an implicit view to JValue. The ~ operator combines multiple fields into a single JSON object. JSON arrays are created from Scala collections:

val detailsJson =
  ("cuisine" ->  "italian") ~ ("vegetarian" ->  true)
// detailsJson: org.json4s.JsonAST.JObject =
//   JObject(List((cuisine,JString(italian)),
//     (vegetarian,JBool(true))))

val tags: JValue = List("higher", "cuisine")

The next listing shows an example using nested objects, a list of primitives, and a list of nested objects, and constructs the JSON document from listing 5.4.

Listing 5.6. Producing a JValue with the DSL

val recipeJson =
  ("title"   ->
    "Penne with cocktail tomatoes, Rucola and Goat cheese") ~
  ("details" -> detailsJson) ~
  ("ingredients" -> List(
    ("label" ->  "Penne")             ~ ( "quantity" ->  "250g"),
    ("label" ->  "Cocktail tomatoes") ~ ( "quantity" ->  "300g"),
    ("label" ->  "Rucola")            ~ ( "quantity" ->  "2 handful"),
    ("label" ->  "Goat cheese")       ~ ( "quantity" ->  "250g"),
    ("label" ->  "Garlic cloves")     ~ ( "quantity" ->  "250g"))) ~
  ("steps" -> List(
     "Cook noodles until aldente.",
     "Quarter the tomatoes, wash the rucola,
        dice the goat's cheese ...",
     "Heat olive oil in a pan, add the garlic and the tomatoes and ...",
     "Shortly before the noodles are ready add the rucola to the ...",
     "Drain the noodles and mix with the tomatoes,
       finally add the ..."))

One goal of the DSL is to construct a valid JValue with very little boilerplate code. Because the DSL is based on implicit conversion, this means a conversion function needs to be available for all values.

Say you want to support a class from the recipe domain model shown earlier, in listing 5.5. Assume that you want to convert a RecipeDetails value to a JValue. When trying to use a value of type RecipeDetails directly in the DSL, this won’t work:

val jsObject: JValue =
  ("details" -> RecipeDetails("italian", true, None))
// <console>:23: error: No implicit view available
// from RecipeDetails => org.json4s.JsonAST.JValue.

The error tells you that no conversion function is available in the current scope. Now let’s see how you can extend the DSL for your own types by providing one. The implicit function details2jvalue in the following listing takes a RecipeDetails and returns a JValue. This allows you to use a RecipeDetails.

Listing 5.7. Extending the DSL

implicit val formats = DefaultFormats

implicit def details2jvalue(rd: RecipeDetails): JValue =
  Extraction.decompose(rd)

val jsObject: JValue =
  ("details" -> RecipeDetails("italian", true, None))
// jsObject: org.json4s.JValue =
//  JObject(List(
//    (details,JObject(List((cuisine,JString(italian)),
//    (vegetarian,JBool(true)))))))

The function relies on decomposition to convert a RecipeDetails to a JValue. Decomposing is a generic reflection-based method. Depending on the concrete type of a value, either a JObject, a JArray, or a primitive value is constructed. It’s exposed through the function Extraction.decompose(a: Any): JValue.

Generally, the following rules are used:

• A primitive value is converted to its matching JSON primitive (for example, a String will result in a JString).
• A collection value is converted to a JSON array, with the elements of the collection being converted recursively.
• An object is converted to a JSON object, with all constructor fields being converted recursively.

It’s possible to override the default conversion behavior for a type by registering a custom serializer for that type. This is discussed in section 5.3.

The JValue DSL leads to concise code and allows you to produce JSON with little typing overhead. Decomposition doesn’t involve writing any manual conversion code, and it’s good for the average conversion case. It relies on general conventions that may not be suitable in all cases, so you may need to write a custom serializer.

Both approaches can be combined: decomposition can be used in the DSL, and the DSL can be used to write a custom serializer. Now let’s move on and look at how to process JSON.

5.2.2. Consuming JSON

In this section, you’ll learn how to traverse a JSON document and extract values from it. Let’s say you’re interested in a specific part of the JSON document, such as the title of the recipe in listing 5.4. In order to read this information, you need to be able to address that part of the document. Because a JSON object consists of nested key-value pairs, the keys can be used to perform a traversal trough the JSON document.

For example, selecting the value of the field with the name "title" works like this:

recipeJson  "title"
// res: JString(Penne with cocktail tomatoes, Rucola and Goat cheese)

The function (nameToFind: String): JValue represents a one-step select that tries to find a field with the given key in the current value. Multiple operations can be chained consecutively:

recipeJson  "details"  "cuisine"
// res: JString(italian)

This one-step selection process allows you to select values deep in nested objects. The syntax resembles XPath in the XML world, which is used to address parts of an XML document.

In an array, the select operation is applied to each contained value, and the results of these invocations are merged in a single JArray:

recipeJson  "ingredients"  "label"
// res: JArray(List(JString(Penne), JString(Cocktail tomatoes), ...))

In addition to simple selects, it’s possible to do a recursive selection that searches all nested values. This is what the function \(nameToFind: String): JValue does:

scala> recipeJson \ "cuisine"
// res: org.json4s.JValue = JString(italian)

There are two special cases that we’ll take a short look at now: one regarding missing values and the other regarding null values.

Missing values happen when a traversal can’t find a suitable value in the JSON document. A missing value is represented by the JValue zero value, JNothing:

recipeJson  "details"  "prerequisites"
// res: JNothing
// there are no prerequisites, returns JNothing

Applying a select on a JNothing yields JNothing again.

A null value is often used to imply a missing value. It’s represented as the unique value JNull.

Now that you know how to navigate through a JSON document, let’s look at how you can extract a value of some arbitrary type, A, from a JValue. This is called extraction, and it’s handled by the function extract[A], which is defined on a JValue.

The following types are valid targets to extract to:

• Case classes and classes
• Value types (Boolean, Int, Float, Double) or primitive reference types (String, Date)
• Standard collection types (List[T], Seq[T], Map[String, _], Set[T])
• Any type that has a configured custom deserializer (this is shown in section 5.3.3)

When extracting a value from a JValue, that JValue is checked for compatibility with the target type. When the two are compatible, an instance of the target type is constructed. Primitives and arrays are compatible with Scala’s primitives and collections. For example, the title of the recipe can be extracted as a String by calling extract with String as the target type:

(recipeJson  "title").extract[String]
// res: String = Penne with cocktail tomatoes, Rucola and Goat cheese

Collections are read from a JSON array. A list of preparation steps from the recipe can be extracted to a List[String]:

(recipeJson  "steps").extract[List[String]]
// res: List[String] = List(Cook noodles until aldente., ...)

Class instances are extracted from JSON objects using a 1:1 mapping approach between a JSON object and a class. For each constructor parameter of the class type, a matching field in the object is required. This is similar to decomposition, described in section 5.2.2.

For example, the following code extracts a RecipeDetails value:

(recipeJson  "details").extract[RecipeDetails]
// res: RecipeDetails = RecipeDetails(italian,true,None)

This is how extraction works. Note that because there is no diet in the JSON, a None is extracted.

The extraction fails at runtime when the JSON is incompatible with the type. For example, a JBool can’t be extracted as a String, and a missing value is incompatible with every type. In this case an exception is raised:

JNothing.extract[String]
// MappingException:
// Did not find value which can be converted into java.lang.String

In the case where an extraction may fail, the value can optionally be extracted. This means the value is read into an instance of Option[A]. This is what extractOpt[A] (json: JValue): Option[A] does, with the following possible results:

• Some(v)—If the value can be extracted
• None—If the extraction fails
• None—If extractOpt is called on JNull or JNothing

The following code shows various examples of optional extraction. Note that JNull and JNothing are both handled as missing values:

JString("foo").extractOpt[String]
// res: Option[String] = Some(foo)

JString("foo").extractOpt[Boolean]
// res: Option[Boolean] = None

JNull.extractOpt[String]
// res: Option[String] = None

JNothing.extractOpt[String]
// res: Option[String] = None

The optional extraction can be combined with the / type from Scalaz (see https://github.com/scalaz/scalaz), resulting in a simple validation. Listing 5.8 shows how this can be done. The computation succeeds with an Ok(..) when all values are present. Otherwise, a BadRequest() is returned.

Listing 5.8. JSON validation using optional extraction and Scalaz’s / type

This should give you some methods for extracting basic information from JSON. Let’s now look at how to handle more-specific cases.

5.3. Customizing JSON support and handling mismatches

The conversion presented in the previous section is sufficient in cases where the structures of a class and a JSON object are very similar. In reality, there’s often some form of discrepancy. This can be a simple variation in the naming of fields or a difference in the structures.

Sometimes you can adjust one side so that both match, but in some cases this may not be feasible. For example, when you’re working with an existing data model, a code refactoring would be impossible due to dependencies. Similarly, when you’re supporting a standardized JSON format, designing Scala classes to exactly match the JSON format can be impractical.

In these cases, a custom conversion function is a cheaper and more sensible alternative. This section covers common approaches for handling such cases.

5.3.1. Customizing JSON support and using field serializers

In Json4s, an implicit value of type Formats determines the concrete JSON handling. When mixing in the Scalatra JSON support, the implicit value needs to be defined, as shown earlier in listing 5.1.

There’s an implementation with common default values in the DefaultFormats trait. The trait defines properties that can be overridden when the default behavior isn’t sufficient. These are a few use cases where a custom Formats saves the day:

• When a date value should be read or written in a non-default date format
• When a JSON number should be read as a BigDecimal instead as a Double
• When there’s a difference between a JSON object and some type, A, that requires specific conversion using a custom serializer or field serializer

Table 5.1 lists properties that can be overridden. For a full reference, see the official Json4s documentation at http://json4s.org.

Table 5.1. Properties of Formats

Formats property	Explanation	Default
wantsBigDecimal: Boolean	If true, a decimal value is read to JDecimal when decomposing. Otherwise, it’s read to a JDouble.	false
strictOptionParsing: Boolean	If true, parsing to an Option[T] is strict. Incompatible values are then not mapped to None but throw an error.	false
emptyValueStrategy: EmptyValueStrategy	When set to skip, a None is skipped. preserve writes None as null.	skip
dateFormatter: SimpleDateFormat	Specifies the format used to parse and write dates.	yyyy-MM-dd'T' HH:mm:ss'Z'
typeHints: TypeHints	Specifies what type hints variant should be used.	NoTypeHints
typeHintsFieldName: String	Sets the name used as the key of the type hint field.	jsonClass
customSerializers: List[Serializer[_]]	Specifies a list of custom serializers.	Empty list
customKeySerializers: List[KeySerializer[_]]	Specifies a list of custom key serializers.	Empty list
fieldSerializers: List[(Class[_], FieldSerializer[_])]	Specifies a list of field serializers.	Empty list

Let’s see how the default formats handle dates. You’ll define an implicit Default-Formats value and decompose a map with a java.util.Date. After that, you’ll parse it again and extract the date. The date is written following the ISO 8601 standard.

Listing 5.9. Using the default Formats

import org.json4s._
import org.json4s.Extraction.decompose
import org.json4s.jackson.JsonMethods.{parse, compact}
import java.util.Date

implicit val formats = DefaultFormats

val txt = compact(decompose(Map("date" -> new Date())))
// txt: String = {"date":"2014-11-19T19:34:34Z"}

val date = (parse(txt)  "date").extractOpt[Date]
// date: Option[java.util.Date] = Some(Wed Nov 19 20:34:34 CET 2014)

A custom Formats is most easily created by extending the DefaultFormats trait. As an exercise, you can create one that employs a custom date format, MM/dd/YYYY.

Listing 5.10. Creating a custom Formats

import java.text.SimpleDateFormat

implicit val formats = new DefaultFormats {
  override protected def dateFormatter: SimpleDateFormat = {
    new SimpleDateFormat("MM/dd/yyyy")
  }
}
val txt = compact(decompose(Map("date" -> new Date())))
// txt: String = {"date":"11/19/2014"}

val date = (parse(txt)  "date").extractOpt[Date]
// date: Option[java.util.Date] = Some(Wed Nov 19 00:00:00 CET 2014)

A field serializer is useful when those fields that don’t appear in a class constructor should be serialized. Assume you have a class, Foo, that you want to serialize with all its fields. The constructor has two parameters: x and y. Because y is prefixed by a val, there’s also a y field. Additionally, there are a and b fields:

class Foo(x: String, val y: String) {
  private val a: String = "a"
  var b: String = "b"
}

With the default serialization, only the single field, y, that appears in the constructor is written to the JSON during serialization:

import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.jackson.Serialization.write

implicit val formats = DefaultFormats

val foo = new Foo("x", "y")

val txt1 = write(foo)
// txt1: String = {"y":"y"}

In this example, you want all fields, y, a, and b, to appear. A field serializer does exactly that.

You can add a FieldSerializer[Food] using the + operator, resulting in a new Formats. Now all fields are taken into account:

import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.jackson.Serialization

implicit val formats = DefaultFormats + new FieldSerializer[Foo]()

val foo = new Foo("x", "y")

val txt1 = Serialization.write(foo)
// txt1: String = {"y":"y","a":"a","b":"b"}

Section 5.3.3 shows how to further specialize the JSON handling for a type featuring custom serializers. Let’s next look at how to employ type hints.

5.3.2. Handling polymorphism with type hints

In this section, we’ll discuss how to use type hints, which are a way to work with polymorphic values. Let’s assume that you need to work with the class hierarchy shown next. The Measure data type describes common measures that may appear in a recipe, and it improves the expressivity of the recipe model:

sealed trait Measure
case class Gram(value: Double) extends Measure
case class Teaspoon(value: Double) extends Measure
case class Tablespoon(value: Double) extends Measure
case class Handful(value: Double) extends Measure
case class Pieces(value: Double) extends Measure
case class Milliliter(value: Double) extends Measure

Let’s further assume that you want to create a JSON array from a list of such values and read it back. This is the JSON that’s generated by default:

val amounts = List(Handful(2), Gram(300), Teaspoon(2))
val amountsJson = Extraction.decompose(amounts)

[ {
  "value" : 2
}, {
  "value" : 300
}, {
  "value" : 2
} ]

Each element consists of a single field, v.

Note that by looking at the JSON representation, there’s no clear way to determine what subtype of Amount a single element of this array represents. Consequently, it’s not possible to read this array back to a List[Amount]. Instead, this results in the following exception:

amountsJson.extract[List[Measure]]
// MappingException:
// No constructor for type Measure, JObject(List((v,JInt(2))))

One way out of this dilemma is to use a synthetic field that holds type information. In Json4s, such a field is called a type hint. By default, the key is jsonClass and the value is equal to the name of the respective type. When a value is decomposed to a JSON object, a type hint field is added to it automatically, and when a value is extracted from a JSON object, the type hint is used to infer the actual type. In order to enable type hints, you can use the withHints method and provide a type hint configuration, as follows.

Listing 5.11. Using type hints

import org.json4s.Extraction.decompose
import org.json4s.jackson.JsonMethods.pretty

val hints = ShortTypeHints(List(
    classOf[Gram],
    classOf[Tablespoon],
    classOf[Teaspoon],
    classOf[Handful]))

implicit val formats = DefaultFormats.withHints(hints)

val amountsJson = decompose(amounts)

pretty(amountsJson)
// [ {
//  "jsonClass" : "Handful",
//  "v" : 2
// }, ...
// ]

amountsJson.extract[List[Measure]]
// res: List[Amount] = List(Handful(2), Gram(300), Teaspoon(2))

Here you use ShortTypeHints with a list of all classes where type hints should be enabled. The resulting JSON correctly interprets the objects, because there are no long ambiguities.

The format of the type hint field can also be customized. For example, there could already be a different naming convention for a type hint field other than jsonClass, or the class name could follow a special format. You can customize these two things.

Let’s say you need to use the key _type. In order to define a non-default field name, the value typeHintFieldName can be overridden:

implicit val jsonFormats = new DefaultFormats {
  override val typeHintFieldName: String = "_type"
}

For the type hint value, there’s an alternative implementation in the form of FullTypeHints that uses the full class name.

The type hints can be overridden in the Formats as well:

implicit val jsonFormats = DefaultFormats.withHints(FullTypeHints(List(
      classOf[Gram],
      classOf[Tablespoon],
      classOf[Teaspoon],
      classOf[Handful])))

Type hints are a simple but effective approach to working with class hierarchies in polymorphic collections and fields. Let’s now move on and look at how to work with custom serializers.

5.3.3. Handling heterogeneity with custom serializers

A custom serializer defines the JSON conversion for a specific type. In the case of a syntactical mismatch between JSON and a Scala type, a custom serializer can help you align the two formats through syntactical and structural transformations. In this section, we’ll look at how you can make use of custom serializers.

Let’s say your application wants to retrieve and send nutritional facts about food products, and the facts are represented as values of type NutritionFacts. Instead of typing the facts as primitives, they’re represented as subtypes of Fact. This improves the expressivity of the model and prevents errors by early and strongly typing. The model is shown in the following listing.

Listing 5.12. Nutritional facts domain

sealed trait Fact
case class Energy(value: Int) extends Fact
case class Carbohydrate(value: Double) extends Fact
case class Fat(value: Double) extends Fact
case class Protein(value: Double) extends Fact

case class NutritionFacts(
   energy: Energy,
   carbohydrate: Carbohydrate,
   fat: Fat,
   protein: Protein)

The nutritional facts should be sent and received as JSON. The expected JSON format for the public API is defined as follows:

{
  "energy": 2050
  "fat": 33.9,
  "carbohydrate": 36.2,
  "protein": 7.9
}

Your goal is to be able to read and write a NutritionFacts value in exactly that format. Serializing a value to JSON using the default conversion yields the following:

val facts = NutritionFacts(
  Energy(2050),
  Carbohydrate(36.2),
  Fat(33.9),
  Protein(7.9))

pretty(decompose(facts))
// {
//   "energy": {
//     "value": 2050
//   },
//   ...
// }

This isn’t exactly what you want. Each fact should be written as a JSON number and not as an object. Trying to read your expected JSON document also results in an extraction error:

val jsObj = parse("""
  { "energy": 2050,
    "carbohydrate": 36.2,
    "fat": 33.9,
    "protein": 7.9 }
""")

jsObj.extractOpt[NutritionFacts]
// None

In order to fix the mismatch between the two formats, you could adjust either your domain model or your public API. For example, you could primitively type the fact’s values as Double instead.

But obviously this isn’t always possible or preferable. Take as an example a big project employing a legacy data model or legacy API. Furthermore, adjusting a model just to overcome a technical problem is rarely an ideal solution.

What can you do instead? You can write a custom serializer. A custom serializer basically represents two partial functions: one function accepts a JValue and returns a NutritionFacts; the other function knows how to create a NutritionFacts from a JSON document.

The following listing shows how to define a custom serializer for the example. Note that you can use the full power of operations discussed in section 5.2.

Listing 5.13. Custom serializer for NutritionFacts

class NutritionFactsSerializer
  extends CustomSerializer[NutritionFacts](implicit formats => ({
    case jv: JValue =>
      val e = (jv  "energy").extract[Int]
      val c = (jv  "carbohydrate").extract[Double]
      val f = (jv  "fat").extract[Double]
      val p = (jv  "protein").extract[Double]

      NutritionFacts(
        Energy(e), Carbohydrate(c), Fat(f), Protein(p)) },
  {
    case facts: NutritionFacts =>
        ("energy" -> facts.energy.value) ~
        ("carbohydrate" -> facts.carbohydrate.value) ~
        ("fat" -> facts.fat.value) ~
        ("protein" -> facts.protein.value)
  }))

The custom serializer needs to be registered in the implicit Formats value. This can be achieved by using the + operator, which takes a CustomSerializer as an argument and returns a new Formats. The following listing shows an example with two routes using the custom serializer.

Listing 5.14. Using a custom serializer

trait FoodRoutes extends ScalatraBase with JacksonJsonSupport {

  implicit val jsonFormats = DefaultFormats +
    new NutritionFactsSerializer

  get("/foods/foo_bar/facts") {
    val facts = NutritionFacts(
      Energy(2050), Carbohydrate(36.2), Fat(33.9), Protein(7.9))

    val factsJson = Extraction.decompose(facts)

    factsJson
  }

  post("/foods/:name/facts") {
    val facts = parsedBody.extractOpt[NutritionFacts]
    println(f"updated facts: $facts")
  }

}

This should give you enough power to handle even major format mismatches.

5.4. JSONP

If you’ve already developed a browser-based web 2.0 application, you’ve probably heard of the same origin policy (SOP). This is a security measure that should prevent a resource from accessing resources located on a different host than the resource itself. It therefore prevents cross-site requests.

But sometimes you’ll want your website to do exactly that, such as querying a web service located at another host using an Ajax request. With the SOP, this isn’t possible without further ado. JSONP (JSON with Padding) allows you to work around SOP. It issues an HTTP request by inserting a <script /> tag into the DOM instead of using Ajax. This works because the src attribute of a script isn’t subject to SOP.

A JSONP request can be detected by the JSON module, and in the case of a JSONP request, the web service returns JavaScript source code instead of JSON. That JavaScript represents a function call passing the JSON data as argument. The name of the function is given as a query parameter.

A JSONP request is detected by comparing the request parameters against a predefined set of callback parameter names. Those parameters can be set by overriding the jsonpCallbackParameterNames: Iterable[String] method, as follows.

Listing 5.15. Supporting JSONP in an application

trait MyJsonpRoutes extends ScalatraBase with JacksonJsonSupport {

  override def jsonpCallbackParameterNames = Seq("jsonp")

  implicit val jsonFormats = DefaultFormats

  get("/foods/foo_bar") {
    val productJson =
      ("label" -> "Foo bar") ~
      ("fairTrade" -> true) ~
      ("tags" -> List("bio", "chocolate"))

    productJson
  }

}

The route can now be called with a query parameter, jsonp, that specifies the name of the local JavaScript method that should handle the response. The resulting JSON is returned as a JSONP response. Note that the content type of the response is text/javascript:

curl -v http://localhost:8080/foods/foo_bar?jsonp=handleResponse
> GET /foods/foo_bar?jsonp=handleResponse HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Wed, 01 Jul 2015 12:05:00 GMT
< Content-Type: text/javascript; charset=UTF-8
< Content-Length: 84
<
/**/handleResponse({"label":"Foo bar","fairTrade":true,"tags":["bio","chocolate"]});

5.5. Summary

• The Scalatra JSON module integrates Json4s in Scalatra’s request-response cycle and enables an application to handle JSON requests and answer with JSON responses.
• A JValue represents a JSON value. There is a DSL to create JSON as well as to navigate and extract information from it.
• A value can be converted to and from a JValue using automatic extraction and decomposition and by writing a custom serializer.
• JSONP enables a website to query JSON data from different sites in a browser.