Chapter 6. Handling files

This chapter covers

• Serving files to a client via HTTP
• Receiving files from a client as an HTTP file upload

This chapter discusses how to implement file exchange over HTTP with Scalatra. As an example, you’ll build a basic document store application that acts as an HTTP-based file server. It will serve documents from the filesystem, and new documents can be uploaded by a client. The user interface is depicted in figure 6.1.

Figure 6.1. User interface for the document store example

6.1. Serving files

First we’ll discuss how to serve non-HTML files, such as text documents, web assets, and media files from a route. You’ll also learn how to serve static resources and how to apply gzip compression to HTTP responses.

6.1.1. Serving files through a route

A route can serve a file by returning a file as a result. There’s built-in support for the types java.io.File, java.io.InputStream, and scala.Array[Byte].

When the type of a returned value is supported, that value is written to the HTTP response body. The file itself can be read by the route’s action from various places, such as local or remote filesystems or databases:

get("/sample") {
  new File("images/cats.jpg")
}

When the file is written to the response, it ensures that a content type header is set. This header indicates the type of the file contained in the response body, and this information enables a client to appropriately interpret the response. Valid content types include text/plain for text, image/jpeg for JPEG images, and application/octet-stream for arbitrary binary data. When no content type is explicitly set by the route, the content type of the file is inferred. This is accomplished by a partial analysis of the file’s data.

Let’s build a file-server application as a more advanced example. The application should have a web API offering read and write access to documents in a document store. A Document represents the file’s meta-information, including the file’s ID, name, an optional content type, and a description. The DocumentStore will have methods to create, find, and list documents. A new document is stored as a map entry in an internal map, and the file content is written to the filesystem with the Document ID as the filename. The following listing shows the code.

Listing 6.1. Document and DocumentStore classes

Two optional response header fields can be useful when serving files: Content-Disposition and Content-Description. The Content-Disposition field contains information about the processing of the file contained in the response. If the disposition type is set to inline, then the document in the response should be displayed directly to the user. The disposition type defaults to attachment, which usually requires further action from the user to display a result (the browser usually presents the user with a Save As dialog box or just downloads the file in the background). Additionally, a filename parameter can be set in the Content-Disposition field. This provides the client with a default filename that can be used when storing the file in the filesystem. The Content-Description response header field can contain a short description about the request payload.

Listing 6.2 shows how to serve documents from the document store and how to include meta-information by setting the HTTP headers just discussed. A document can be queried by its ID. If it can be found, it’s returned with the headers set; otherwise, a 404 error is returned.

Listing 6.2. Serving a file

class DocumentsApp(store: DocumentStore)
  extends ScalatraServlet with FileUploadSupport with ScalateSupport {

  get("/documents/:documentId") {
    val id = params.as[Long]("documentId")
    val doc = store.getDocument(id) getOrElse
      halt(404, reason = "could not find document")
    doc.contentType foreach { ct => contentType = ct }
    response.setHeader("Content-Disposition",
      f"""attachment; filename="${doc.name}"""")
    response.setHeader("Content-Description", doc.description)
    store.getFile(id)
  }
}

Let’s turn to the document store now. The DocumentStore is created in the ScalatraBootstrap class when the application starts. You provide the DocumentStore to the application through constructor injection. Because the document store is initially empty, the following code adds some sample documents. (Adding documents to the store via an HTTP file upload is shown in section 6.2.)

import org.scalatra._
import javax.servlet.ServletContext
import org.scalatra.book.chapter05.{DocumentStore, DocumentsApp}

class ScalatraBootstrap extends LifeCycle {
  override def init(context: ServletContext) {
    val store = DocumentStore("data")
    store.add("strategy.jpg",
      new FileInputStream(new File("data/strategy.jpg")),
      Some("image/jpeg"),
      "bulletproof business strategy")
    store.add("manual.pdf",
      new FileInputStream(new File("data/manual.pdf")),
      Some("application/pdf"),
      "the manual about foos")

    val app = new DocumentsApp(store)
    context.mount(app, "/*")
  }
}

A request querying the sample document would query the URL http://localhost:8080/documents/0:

HTTP/1.1 200 OK
Content-Type: application/jpeg;charset=UTF-8
Content-Disposition: attachment; filename="strategy.jpg"
Content-Description: bulletproof business strategy

<<binary data>>

The response contains the stored document.

6.1.2. Serving static resources

A web application often consists of static web assets, like images, CSS, and HTML. They’re usually located as static resources in the src/main/webapp source directory. Those resources can be served to a client in a generic way with the serveStatic-Resource method.

The serveStaticResource method resolves a resource from the request URL, and if one can be found, it’s written to the response. Internally, the resource is resolved by using the ServletContext.getResource method.

If no action can be found for a URL, Scalatra tries to serve a static resource by invoking serveStaticResource. If no resource can be found, a 404 response is returned with the previously set content type removed. This default behavior is implemented in the notFound handler:

notFound {
  contentType = null
  serveStaticResource() getOrElse halt(404, <h1>Not found.</h1>)
}

If an application requires nonstandard handling of static resources, you can overwrite the notFound handler.

6.1.3. Applying gzip compression to responses

HTTP allows you to apply content encoding to the body of a response. In practice, this is often a compression algorithm that reduces the bandwidth used by a website.

Scalatra offers the option to encode outgoing responses with the gzip algorithm. In order to make use of it, the ContentEncodingSupport trait needs to be mixed into the application:

class DocumentsApp extends ScalatraServlet with ContentEncodingSupport {

  get("/sample") {
    new File("data/strategy.jpg")
  }

}

Now, when a client indicates that it’s able to receive a response compressed with gzip (by sending an appropriate Accept-Encoding header), the response body is encoded and a Content-Encoding header is added to the response:

curl -H "Accept-Encoding: gzip" http://localhost:8080

6.2. Receiving files

A multipart request combines one or more different sets of data in a single message body. When your application receives a multipart request, Scalatra handles it as a multipart/form-data request, where each body part represents a form field consisting of the field’s name and value. The value can be a simple string or binary data representing a document. Often a body part also contains a suggested filename and a description.

We’ll start this section with a basic introduction to receiving files in an action. Then we’ll discuss the possible configuration settings and how to handle errors during the upload. Along the way, we’ll extend the document store code with a file upload.

6.2.1. Supporting file uploads

Scalatra’s support for file uploads needs to be explicitly mixed into an application through the FileUploadSupport trait. When that trait is mixed in, and if a multipart request is detected, the body parts are extracted and made available to the application:

import org.scalatra.ScalatraServlet
import org.scalatra.servlet.FileUploadSupport
import org.scalatra.scalate.ScalateSupport

class DocumentsApp extends ScalatraServlet
  with FileUploadSupport with ScalateSupport {

}

Each form field with a specified filename is handled as a file; all other form fields are handled as standard parameters.

Each file is represented as an instance of FileItem. A FileItem describes the file’s name, size, and original field name in the multipart request. The content type as well as the charset are available if they have been specified in the request; otherwise they are None. The FileItem fields and methods are listed in table 6.1.

Table 6.1. FileItem fields and methods

Name	Description
size: Long	Size of the file
name: String	Name of the file
fieldName: String	Name of the form field
contentType: Option[String]	Content type of the file
charset: Option[String]	Charset of the file
write(file: java.io.File)	Writes the data to the filesystem via a java.io.File
write(fileName: String)	Writes the data to the filesystem
get: Array[Byte]	Returns the data as a byte array
getInputStream: java.io.InputStream	Returns an InputStream to the data

A FileItem can be written to the filesystem, or the content can be retrieved as an Array[Byte] or java.io.InputStream for further processing. Parameters are merged with GET and POST parameters and can be accessed with either the params or multiParams method, as discussed in chapter 4.

A FileItem can be retrieved with the method fileParams(key: String). The key parameter is the name of the form field, which can contain multiple files. In that case, the fileMultiParams(key: String) method can be used, and it returns a Seq[FileItem].

The following code performs a simple file-upload action:

post("/sample") {
  val file = fileParams("sample")
  val desc = params("description")
  <div>
    <h1>Received {file.getSize} bytes</h1>
    <p>Description: {desc}</p>
  </div>
}

This POST action expects a file with the name sample and a parameter with the name description. A sample multipart/form-data message conforming to those requirements is shown next.

Listing 6.3. Sample multipart/form-data message

--a93f5485f279c0
content-disposition: form-data; name="file"; filename="foobar.txt"

FOOBAZ
--a93f5485f279c0
content-disposition: form-data; name="description"

A document about foos.
--a93f5485f279c0--

You can send this message as an HTTP request via curl:

curl http://localhost:8080/sample 
  --data-binary @data/multipart-message.data 
  -X POST 
  -i -H "Content-Type: multipart/form-data; boundary=a93f5485f279c0"

You’ve now seen how to upload a document manually.

Next, let’s extend the document store application with a file-upload form and functionality to receive uploads. The upload form is shown in the following listing. It’s integrated in WEB-INF/templates/views/index.scaml and rendered with the main page.

Listing 6.4. A basic file-upload form

<div class="col-lg-3">
  <h4>Create a new document</h4>
  <form enctype="multipart/form-data" method="post" action="/documents">
    <div class="form-group">
      <label>File:</label>
      <input type="file" name="file">
    </div>
    <div class="form-group">
      <label>Description:</label>
      <input class="form-control"
        type="text" name="description" value="">
    </div>
    <input class="btn btn-default" type="submit">
  </form>
</div>

The next listing shows the upload handler.

Listing 6.5. Upload handler in the DocumentStorage example

post("/documents") {
  val file = fileParams.get("file") getOrElse
    halt(400, reason = "no file in request")
  val desc = params.get("description") getOrElse
    halt(400, reason = "no description given")
  val name = file.getName
  val in = file.getInputStream
  store.add(name, in, file.getContentType, desc)
  redirect("/")
}

First the input is validated, halting with a status code of 400 if a parameter is missing. The file is then added (with the given description) to the document store, and the client is redirected to the main page again.

6.2.2. Configuring the upload support

Having file-upload support in your application allows users to upload files to the server. But this means users can upload a lot of big files and consume your memory and disk space. It can therefore be useful to set certain limits on multipart request handling.

In listing 6.6, limits are applied to an application: a 30-MB maximum for a single file, and a 100-MB maximum for the entire request. When a request exceeds one of these limits, an exception is thrown, which you can handle as shown in section 6.2.3. The configuration is set by invoking the configureMultipartHandling method, providing a value of type MultipartConfig.

Listing 6.6. Configuring upload support in an application

import org.scalatra.ScalatraServlet
import org.scalatra.servlet.FileUploadSupport
import org.scalatra.servlet.MultipartConfig
import org.scalatra.scalate.ScalateSupport

class DocumentsApp extends ScalatraServlet
  with FileUploadSupport with ScalateSupport {

  configureMultipartHandling(MultipartConfig(
    maxFileSize = Some(30 * 1024 * 1024),
    maxRequestSize = Some(100 * 1024 * 1024),
    ))

  // ...

}

Table 6.2 lists the available fields of MultipartConfig, all of which are optional. If no field value is provided, the default value is used. Here a value of -1 means unlimited, and in theory it could use all available memory. In reality, the concrete behavior also depends on the servlet container that’s used. For example, some servlet containers ignore the fileSizeThreshold setting for body parts representing files, and write those to the disk by default.

Table 6.2. The fields of the MultipartConfig type

Name	Type	Default	Description
maxRequestSize	Option[Long]	Some(-1)	The maximum size allowed for a full multipart/form-data request
maxFileSize	Option[Long]	Some(-1)	The maximum size allowed for a single uploaded body part
location	Option[String]	"" (empty string)	The directory location where body parts that are cached to the file-system will be stored
fileSizeThreshold	Option[Int]	0	The size threshold after which a body part will be written to disk

Listing 6.6 showed the preferred approach to upload a configuration, but the web.xml deployment descriptor can also be used to configure upload support, as shown in listing 6.7. This can be useful when an application already makes substantial use of web.xml. It’s possible because Scalatra’s upload support builds on the Servlet 3.0 multipart API.

Listing 6.7. Configuration of upload support in web.xml

<servlet>
  <servlet-name>documents</servlet-name>
  <servlet-class>org.scalatra.book.chapter06.Documents</servlet-class>
  <multipart-config>
    <max-file-size>31457280</max-file-size>
    <max-request-size>104857600</max-request-size>
    <location>/tmp/uploads</location>
  </multipart-config>
</servlet>

This XML is functionally equivalent to the Scala code in listing 6.6.

6.2.3. Handling upload errors

When handling file uploads, errors can occur. For example, the file being uploaded may exceed the configured size limit, or there may not be enough space left on the filesystem. In such cases, an exception is thrown. By default, that exception is then shown to the user.

Listing 6.8 shows an error handler that handles two exceptions related to file uploads. A SizeConstraintExceededException is thrown when the uploaded file exceeds a file limit. An IOException implies that an I/O operation failed, such as when the permissions aren’t sufficient to execute the operation.

Listing 6.8. Error handling for uploads

error {
  case e: SizeConstraintExceededException =>     halt(500, "Too much!")
  case e: IOException =>     halt(500, "Server denied me my meal, thanks anyway.")
}

6.3. Summary

• Serving files through a route isn’t as fast as serving files statically, but it can be useful if you want to apply some processing to files, serve a file from an arbitrary location, or construct an entirely new file in your response.
• Static file serving is useful when you want to quickly serve a file directly from the filesystem.
• Gzipping lets you compress your responses.