Build Web Applications with the Kemal and Amber Frameworks

Ruby’s most explosive growth came from Rails, the outstanding web framework. As Crystal is a direct descendant of Ruby, it’s not surprising that developers have poured energy into Crystal web frameworks. If you read through the company testimonies, you’ll see that many of them use Kemal.

The Built-in Web Server

Crystal’s standard library has a module HTTP for basic web techniques such as static file handling, logging, error-handling, form data, websockets, and so on. It also contains a built-in web server, the class HTTP::Server. You can set that up in just a few lines of code, and see the foundations of a running web server:

To start using it, we need to require the HTTP module’s code. Then, create an instance of the Server class in line ​①​. A fiber spawns to handle each incoming request, so the Crystal web server is fully concurrent. The fiber will be given the code in the do end block to execute. It has access to a ctx parameter (an object of class Context), which contains info from the request, like its path: the address the client requested. Together with the block, a list of HTTP handlers can be processed (see the docs^[65]).

In reply to the request, the processing fiber will set up the response of the server by specifying its content-type in line ​②​. Because this is a minimalist reply, it’s just text/plain. At this stage you could also set the status and header properties. Afterwards it will write (with print) to the response in line ​③​. The server will send that to the web client (probably a browser), which will process the content and show it.

The server itself is bound to an IP address and port using the bind_tcp methods, and then started with the listen method in line ​④​. If you add true to the end of the +argument list for bind_tcp, the web server will enable port reuse. This means that multiple sockets are allowed to bind to the same port on the same host, simulating multi-threaded behavior.

To parameterize the server’s address as well as the port, you can bind the server like this:

Compile this script with $ crystal web_server.cr. You’ll see the message

appearing on the server console. Opening up a browser with this address will show you the server’s response:

Stop the server with CTRL+C or use close in the code.

This default web server has very good performance, but because you have to provide the basic building blocks yourself, you’ll have to write a lot more code. An example of this is routing specific requests to their appropriate response. You may want to add tools to deal with this. router_cr^[66] is a good minimal middleware for a Crystal web server.

The next sections visit other frameworks that are built on top of the basic http-server. Comparison benchmarks show that Crystal web frameworks are among the fastest, if not number one (see ​Speeding Up the Web​).

Your Turn 1

➤ a. An XML Time Server: Create a web server on port 5000 that shows the time and a greeting according to the time of day. Use XML as the output format. (Hint: Use “text/xml” as the content-type.)

➤ b. Serving an Index Page: Create a web server that does error-handling and logging, and serves an index.html page when the browser requests the URL http://127.0.0.1:3000/public. (Hints: Use the docs for the Handlers, and copy the page to output with IO.copy.)

Kemal: A Lightweight Web Framework

Kemal^[67] is the lightning-fast defacto standard web framework for Crystal, developed by Serdar Dogruyol. It’s the successor to Frank, a proof-of-concept framework written by the Crystal team. Kemal has a modular architecture. Most of its idioms come from Sinatra (a lightweight Ruby web framework), which is why simplicity is one of its hallmarks. Its CPU and memory requirements (+- 1MB) are very low: one server can handle tens of thousands of clients.

A Basic Kemal App

To start, let’s build the example from Chapter 1, ​Speeding Up the Web​ from scratch.

You know the drill by now:

• Create a new app with crystal init (call it simple_kemal_app).

• Add the following section to the shard.yml file:

• Issue the shards command.

• The output shows that in addition to kemal, two other dependent shards (radix, used for routing, and kilt, used for view templating) are also installed:

  ​	Installing kemal
  ​	Installing radix (0.3.8)
  ​	Installing kilt (0.4.0)

  Remember: This step installs the source code of Kemal in the /lib folder, and it is pure Crystal, so you can see for yourself how it all works!

Now add the following code into src/simple_kemal_app.cr:

Then run the app with crystal src/simple_kemal_app.cr (or use crystal build and run the executable). The following message will appear:

Open up a browser with the address http://localhost:3000/ (the default port is 3000) and you’ll see the message: “My first Kemal app is alive!”

Meanwhile, on the Kemal console you see something like this:

This default output logging can be disabled by setting logging false. You can also define your own custom logging if wanted. Or, to change the default port, for example to 8080, use the code Kemal.config.port = 8080.

The code for responding to requests is elegant and simple:

​①​

require the Kemal shard.

​②​

Match a GET "http://localhost:3000/" request by returning an output string to the client.

​③​

Start the web server by calling the run method.

If you would prefer to tell the browser to visit an index page to answer this request, you can do it like this:

env is the environment variable and must be given as a parameter to the handler block, so get "/" do must change to get "/" do |env|. env is useful because it can access all info about the request and parameters (received through the URL, query, or as posted form parameter), and it can configure and fill in the response. It has also set and get methods to store a value during the request/response cycle.

Kemal can handle all HTTP methods such as GET, POST, PUT, PATCH, and DELETE: the routing handler is always a do end code block, which can have a context variable. This provides your app with a RESTful web service architecture. Routes are handled in the order they are defined: the first route that matches the request URL will be activated.

Using Views with ECR

Let’s start building a simple website, exploring more of Kemal’s features along the way. To show dynamic pages constructed partly through code, Kemal can use the ECR (Embedded Crystal)^[68] module.

This is a template language from the standard library used for embedding Crystal code into text like HTML, much as ERB does in Ruby. Unlike ERB, however, all templates are compiled into just one string, which is pretty efficient: you suffer no performance penalty when doing heavy templating. ECR templates are also type-safe.

But enough theory—let’s get started. First create a project called appviews. To ease development you can use the sentry^[69] command-line interface tool, written for Crystal by Samuel Eaton. This watches your source files, and it will build and run the app automatically when it detects a change in them. That way, you don’t have to stop and restart an app on every change! Install it within the app folder with:

$ curl -fsSLo- https://raw.githubusercontent.com/samueleaton/sentry/master/install.cr | crystal eval

(Notice how this uses crystal eval, evaluating the script install.cr.)

To execute this tool, do $ ./sentry. It starts watching:

Add the following to .gitignore to remove sentry and the executables from source control:

Then follow the steps from the previous section to install Kemal. Also write its code inside src/appviews.cr: because sentry is at work, it will see this, compile your app, and start running Kemal!

Now we probably want to add some static files, like JavaScripts and stylesheets. We’ll use Bootstrap here (use the minified version for a production app) and our app’s stylesheet in appviews.css. All these assets are placed in a public folder like this:

Kemal will find assets in the public folder automatically, pointing to them with paths like /js/appviews.js. If you have no static files, you can disable this by setting serve_static false in your code. Or you can change the location as well, like this: public_folder "src/public". We won’t use an index.html in this example.

Instead of a simple greeting, it’s time to show a home page using ECR. We can do this by changing the get handler in src/appviews.cr to:

This introduces two variables, title and page_message. For the first time, we see how ECR is used: you render an .ecr template, which is placed in src/views, the location for these template files or views. Before trying that out, you need an ECR file. Edit a new file, home.ecr, in that folder, adding this content:

Now sentry compiles and your browser shows this at http://localhost:3000:

What has happened here? In <%= page_message %> the value of page_message was pasted in the home view, which is what <%= %> does. You can also put Crystal code inside <% %> to control execution.

For common navigation, header, footer code, and so on, you can use a layout or several. These are also .ecr pages that you put in a subfolder of views called layouts. Create a main_layout.cr in that folder. It’s mainly a lot of HTML, which we aren’t going to show here (but it’s in the sample code file).

This layout will be used if you replace the previous render line with the following:

How does it work? The layout page sets the web page title via <%= title %>. It also contains this section:

home.ecr is shown within the layout page because, in Kemal, the variable content captures the content of the first argument to render. Depending on the view, you might need different JavaScript or other chunks of HTML or stylesheets. In that case, content_for and yield_content come in handy; see the Kemal guide^[70] for more detail.

We also see a code section, which sits in the <div> navbar:

This code, <% if [condition] %>[effect]<% end %>, tests the requested path through the env variable, setting a CSS class when it matches. At this stage, compilation will again be successful, and our website starts to have some design:

Add the about.ecr and contact.ecr pages to the views folder, and add two get routing handlers to appviews.cr. Our two new pages will inherit the same look:

Click on Register or Login, and Kemal will point out to you that these views don’t yet exist. Play around to start building your own Kemal website!

Streaming Database Data with JSON

Sometimes you just want to send data, not HTML pages. Let’s stream database data from the server to the client in JSON format, an example inspired by a blog article^[71] from Brian Cardiff. We’ll combine what we have learned in ​Accessing Databases​ about accessing databases and work with the same sqlite3 database, chinook.db. You can do the preliminary work by yourself: create an app called db_json and add the shards kemal and sqlite3.

The app’s logic structure looks like this:

• Open a database connection.

• Define the GET route handlers:

  1) A root request / will return a list of the names of all tables.

  2) A request like /:table_name, for example /artists, will return all records in JSON format.

• Start the Kemal web server.

• Close the db connection.

The first handler is extremely concise. First, set the type of the server’s response with env.response.content_type. Then, get the data. Execute the SQL query with query_all. Its result is an array, so you can use the to_json method on it. This creates a string with the JSON representation and sends it to the client. That’s it!

And here is the resulting output:

If you would rather see these table names in an HTML-based view, change the handler to:

and add a view, tables.ecr, which loops over the tables array like this:

As shown in the figure variables from the script (such as tables) are automatically known in the view!

The second handler is slightly more complicated. The table name is given as a parameter in the URL. This can be read through env.params.url as in line ​①​ in the following code. To avoid SQL injection attacks, we check in line ​②​ that the parameter is an existing table. We execute the query in line ​③​ and loop over each record in line ​④​, sending a JSON string to the client in the method write_json.

The mechanism used here is to stream one row at a time, minimizing memory usage on the server. The client will be able to process the data as it comes through. This is done by applying the flush method to the output buffer in line ​⑥​, sending the JSON string immediately to the client, sometimes called a chunked response. The JSON string is constructed in the write_json method, which is called for each record:

The transformation of a ResultSet with column names and field values to JSON is a little more involved, so we’ll need to use the JSON^[72] module here.

Crystal-db is designed for speed and provides direct access to the database values in memory without the need to create temporary arrays or hashes. (If you create a lot of your own temporary arrays or hashes, of course, that may not help.) We create a JSON::Builder object in line ​①​ in the preceding code that writes directly to io, which is the web server’s output passed in line ​⑤​.

To make it concrete, here is the JSON output of the first record of table artists: {"ArtistId":1,"Name":"AC/DC"}. json.object starts and ends the JSON string for one record and invokes the do code block. For each column, the name col is written, together with the value rs.read. This is done in json_encode_field in line ​②​ below:

This invokes the method to_json on all simple database values in line ​②​, but for the Bytes type in line ​①​ we need to be more careful, treating it as an array and looping over each value. This way does not create a temporary array, so it is the most efficient way to process objects that could be big blobs.

You can find the source code as a whole in the download code db_json/src/db_json.cr. Here you see the output of the first ten rows from the table artists:

Other Features and Example Web Apps

Based on the example code we’ve discussed so far, you should be able to combine ECR views with database data to build more real-world database web apps. But there’s a lot more you can do with Crystal and Kemal. Here are some other Kemal examples that might inspire you:

• A todo app^[73] demo, developed with Kemal, React, and PostgreSQL using websockets.

• A chat app^[74] demo here.^[75]

• An app^[76] using JWT authentication with auth0 described in detail here.^[77]

• A blog app.^[78]

• A blog server called kamber.^[79]

• Kemal-watcher^[80] is a plugin-like sentry plugin that watches client files.

• See how you can easily send SMS messages^[81] or make telephone calls^[82] by using the Twilio API.

These few examples give just a taste of Kemal’s strengths. For inspiration, here are more possibilities you should explore:

• Request parameters are easy to work with—for example, if the request is /users/108, then the handler get "/users/:id" matches this, and the value 108 is retrieved through env.params.url["id"].

• You can insert filters in the response cycle that let you create and encapsulate your own logic, manipulating the request or response. They come as before_x or after_x, where x can be: get, post, put, patch, delete or all.

• Kemal’s modularity is achieved through handlers (or middlewares) similar to Rack middleware: these are classes that inherit from Kemal::Handler, and can be called conditionally based on the request method. Built-in are logging, exception, static-file, and route handling. Other middlewares for basic authentication and csrf protection are available.

• Kemal can store info in sessions by including the kemal-session^[83] shard. The data can be stored in memory, file, Redis, and even MySQL.

• Kemal has easy websocket support (see the chat app, for example), which works much faster than Node.js. (Though really, this power comes from the built-in web server.)

• You can test your web app by using the spec-kemal shard.

• Kemal gives you access to cookies, file uploads, built-in SSL support, ability to differentiate between test and production environments, custom error handling, email-sending via smtp.cr,^[84] and multi-DB support.

• You can deploy your app on Heroku or through a Capistrano script.

Kemal’s^[85] website does an excellent job showing how to use the many different features of this framework. Raze^[86] is a younger framework that aims to compete with Kemal.

Amber: A Rails-Like Web Framework

Kemal is a fast and fully featured web server, but it’s deliberately minimal. If you’d like to have a framework do more for you, offering services like Ruby on Rails, Crystal offers a few Rails-like frameworks. Unsurprisingly, they rely heavily on macros internally.

Amber^[87] is inspired by Kemal, Rails, Phoenix, and other popular application frameworks, and it follows the MVC (Model-View-Controller) pattern. You can find its source code here,^[88] where you can see that it’s growing at a fast pace. They also have joined forces with Kemalyst,^[89] which is also based on Kemal.

The Amber developers aim to build a Rails for Crystal. Amber uses the conventions of a typical MVC framework, providing code generation and scaffolding for quick prototype construction. By default, it uses the lightweight granite^[90] ORM (Object Relational Mapper) for working with databases using minimal configuration, providing you with easy queries such as find_by and all, and one-to-many relations belong_to and has_many. But you can also opt for the Crecto (like Phoenix ecto) or (more full featured) Jennifer ORMs.

Amber uses a built-in command-line tool called amber. It can generate a project skeleton for you, complete with models, controllers, and views. Route handling is done with pipes, as in Phoenix.

By default, applications are configured to use a PostgreSQL database adapter, but this can be switched to MySQL, SQLite, or any other datastore or microservice you need to use. The default rendering engine is Slang, but ECR can be used as well. Amber builds in Sentry for smooth development. In addition to handling HTTP requests, Amber apps can work with websockets and can do mailing out-of-the-box.

If all this makes you want to get started right away with Amber, start here.^[91]

There are some other frameworks in this area worth mentioning, such as Amethyst,^[92] carbon-crystal,^[93] and Lucky^[94] by the Thoughtbot company. For a websocket-first framework, look at lattice-core.^[95]