At this point, it might seem like you have all you need to create process-oriented projects with Elixir. You know how to create useful functions, can work with recursion, know the data structures Elixir offers, and probably most important, know how to create and manage processes. What more could you need?
Process-oriented programming is great, but the details matter. The basic Elixir tools are powerful but can also bring you to frustrating mazes debugging race conditions that happen only once in a while. Mixing different programming styles can lead to incompatible expectations, and code that worked well in one environment may prove harder to integrate in another.
Ericsson encountered these problems early when developing Erlang (remember, Elixir runs on Erlang’s virtual machine), and created a set of libraries that eases them. OTP, the Open Telecom Platform, is useful for pretty much any large-scale project you want to do with Elixir and Erlang, not just telecom work. It’s included with Erlang, and though it isn’t precisely part of the language, it is definitely part of Erlang culture. The boundaries of where Elixir and Erlang end and OTP begins aren’t always clear, but the entrypoint is definitely behaviors. You’ll combine processes built with behaviors and managed by supervisors into an OTP application.
So far, the lifecycle of the processes shown in the previous chapters has been pretty simple. If needed, they set up other resources or processes to get started. Once running, they listen for messages and process them, collapsing if they fail. Some of them might restart a failed process if needed.
OTP formalizes those activities, and a few more, into a set of behaviors (or behaviours—this was originally created with British spelling). The most common behaviors are GenServer
(generic server) and Supervisor
. Through Erlang, you can use the gen_fsm
(finite state machine) and gen_event
behaviors. Elixir provides the Mix build tool for creating applications so that you can package your OTP code into a single runnable (and updatable) system.
The behaviors pre-define the mechanisms you’ll use to create and interact with processes, and the compiler will warn you if you’re missing some of them. Your code will handle the callbacks, specifying how to respond to particular kinds of events, and you will need to decide upon a structure for your application.
If you’d like a free one-hour video introduction to OTP, though it is Erlang-centric, see Steve Vinoski’s “Erlang’s Open Telecom Platform (OTP) Framework” at http://www.infoq.com/presentations/Erlang-OTP-Behaviors. You probably already know the first half hour or so of it, but the review is excellent. In a very different style, if you’d like an explanation of why it’s worth learning OTP and process-oriented development in general, Francesco Cesarini’s slides at https://www.erlang-factory.com/upload/presentations/719/francesco-otp.pdf work even without narration, especially the second half.
Much of the work you think of as the core of a program—calculating results, storing information, and preparing replies—will fit neatly into the GenServer
behavior. It provides a core set of methods that let you set up a process, respond to requests, end the process gracefully, and even pass state to a new process if this one needs to be upgraded in place.
Table 13-1 shows the methods you need to implement in a service that uses GenServer
. For a simple service, the first two or three are the most important, and you may just use placeholder code for the rest.
Method | Triggered by | Does |
---|---|---|
|
|
Sets up the process |
|
|
Handles synchronous calls |
|
|
Handles asynchronous calls |
|
random messages |
Deals with non-OTP messages |
|
failure or shutdown signal from supervisor |
Cleans up the process |
|
system libraries for code upgrades |
Lets you switch out code without losing state |
Example 13-1, which you can find in ch12/ex1-drop, shows an example that you can use to get started. It mixes a simple calculation from way back in Example 2-1 with a counter like that in Example 9-4.
defmodule
DropServer
do
use
GenServer
defModule
State
do
defstruct
count
:
0
end
# This is a convenience method for startup
def
start_link
do
GenServer
.
start_link
(
__MODULE__
,
[],
[{
:name
,
__MODULE__
}])
end
# These are the callbacks that GenServer.Behaviour will use
def
init
([])
do
{
:ok
,
%
State
{}}
end
def
handle_call
(
request
,
_from
,
state
)
do
distance
=
request
reply
=
{
:ok
,
fall_velocity
(
distance
)}
new_state
=
%
State
{
count
:
state
.
count
+
1
}
{
:reply
,
reply
,
new_state
}
end
def
handle_cast
(
_msg
,
state
)
do
IO
.
puts
(
"So far, calculated
#{
state
.
count
}
velocities."
)
{
:noreply
,
state
}
end
def
handle_info
(
_info
,
state
)
do
{
:noreply
,
state
}
end
def
terminate
(
_reason
,
_state
)
do
{
:ok
}
end
def
code_change
(
_old_version
,
state
,
_extra
)
do
{
:ok
,
state
}
end
# internal function
def
fall_velocity
(
distance
)
do
:math
.
sqrt
(
2
*
9.8
*
distance
)
end
end
The module name (DropServer
) should be familiar from past examples. The second line specifies that the module is going to be using the GenServer
module.
The nested defModule
declaration should be familiar; it creates a structure that contains only one field, to keep a count of the number of calls made. Many services will have more fields, including things like database connections, references to other processes, perhaps network information, and metadata specific to this particular service. It is also possible to have services with no state, which would be represented by an empty tuple here. As you’ll see further down, every single GenServer
function will reference the state.
The State
structure declaration is a good example of a declaration you should make inside of a module and not declare in a separate file. It is possible that you’ll want to share state models across different processes that use GenServer
, but it’s easier to see what State
should contain if the information is right there.
The first function in the sample, start_link/0
, is not one of the required GenServer
functions. Instead, it calls Elixir’s GenServer.start_link
function to start up the process. When you’re just getting started, this is useful for testing. As you move toward production code, you may find it easier to leave these out and use other mechanisms.
The start_link/0
function uses the built-in __MODULE__
declaration, which
returns the name of the current module..
# This is a convenience method for startup def start_link do GenServer.start_link(__MODULE__, [], [{:name, __MODULE__}]) end
The first argument is an atom (__MODULE__
) that will be expanded to the name of the current module, and that will be used as the name for this process. This is followed by a list of arguments to be passed to the module’s initialization procedure and a list of options. Options can specify things like debugging, timeouts, and options for spawning the process. By default, the name of the process is registered with just the local Elixir instance. Because we want it registered with all associated nodes, we have put the tuple {:name, __MODULE__}
in the options list.
You may also see a form of GenServer.start_link
with :via
as an atom in an option tuple. This lets you set up custom process registries, of which gproc
is the best known. For more on that, see https://github.com/uwiger/gproc.
All of the remaining functions are part of GenServer
’s behavior. init/1
creates a new state structure instance whose count
field is zero—no velocities have yet been calculated. The two functions that do most of the work here are handle_call/3
and handle_cast/2
. For this demonstration, handle_call/3
expects to receive a distance in meters and returns a velocity for a fall from that height on earth, while handle_cast/2
is a trigger to report the number of velocities calculated.
handle_call/3
makes synchronous communications between Erlang processes simple.
def
handle_call
(
request
,
_from
,
state
)
do
distance
=
request
reply
=
{
:ok
,
fall_velocity
(
distance
)}
new_state
=
%
State
{
count
:
state
.
count
+
1
}
{
:reply
,
reply
,
new_state
}
end
This extracts the distance
from the request
, which isn’t necessary except that I wanted to leave the variable names for the function almost the same as they were in the template. (handle_call(distance, _from, state)
would have been fine.) Your request
is more likely to be a tuple or a list rather than a bare value, but this works for simple calls.
The function then creates a reply based on sending that distance
to the simple fall_velocity/1
function at the end of the module. It then creates a new_state
containing an incremented count. Then the atom :reply
, the reply
tuple containing the velocity, and the new_state
containing the updated count get passed back.
Because the calculation is really simple, treating the drop as a simple synchronous call is perfectly acceptable. For more complex situations where you can’t predict how long a response might take, you may want to considering responding with a :noreply
response and using the _from
argument to send a response later. (There is also a :stop
response available that will trigger the :terminate/2
method and halt the process.)
By default, OTP will time out any synchronous calls that take longer than five seconds to calculate. You can override this by making your call using GenServer.call/3
to specify a timeout (in milliseconds) explicitly, or by using the atom :infinity
.
The handle_cast/2
function supports asynchronous communications. It isn’t supposed to return anything directly, though it does report :noreply
(or :stop
) and updated state. In this case, it takes a very weak approach, but one that does well for a demonstration, calling IO:puts/1
to report on the number of calls:
def
handle_cast
(
_msg
,
state
)
do
IO
.
puts
(
"So far, calculated
#{
state
.
count
}
velocities."
)
{
:noreply
,
state
}
end
The state doesn’t change, because asking for the number of times the process has calculated a fall velocity is not the same thing as actually calculating a fall velocity.
Until you have good reason to change them, you can leave handle_info/2
, terminate/2
, and code_change/3
alone.
Making a GenServer
process run and calling it looks a little different than starting the processes you saw in Chapter 9.
iex(1)>
DropServer
.
start_link
()
{:ok,#PID<0.46.0>}
iex(2)>
GenServer
.
call
(
DropServer
,
20
)
{:ok,19.79898987322333}
iex(3)>
GenServer
.
call
(
DropServer
,
40
)
{:ok,28.0}
iex(4)>
GenServer
.
call
(
DropServer
,
60
)
{:ok,34.292856398964496}
iex(5)>
GenServer
.
cast
(
DropServer
,
{})
So far, calculated 3 velocities.
:ok
The call to DropServer.start_link()
sets up the process and makes it available. Then, you’re free to use GenServer.call
or GenServer.cast
to send it messages and get responses.
While you can capture the pid, you don’t have to keep it around to use the process. Because start_link
returns a tuple, if you want to capture the pid you can do something like {:ok, pid} = Drop.start_link()
.
Because of the way OTP calls GenServer
functions, there’s an additional bonus—or perhaps a hazard—in that you can update code on the fly. For example, I tweaked the fall_velocity/1
function to lighten Earth’s gravity a little, using 9.1 as a constant instead of 9.8. Recompiling the code and asking for a velocity returns a different answer:
iex(6)>r(DropServer)
warning: redefining module DropServer (current version loaded from
_build/dev/lib/drop_server/ebin/Elixir.DropServer.beam)
lib/drop_server.ex:1
warning: redefining module DropServer.State (current version loaded from
_build/dev/lib/drop_server/ebin/Elixir.DropServer.State.beam)
lib/drop_server.ex:4
{:reloaded, DropServer, [DropServer.State, DropServer]}
iex(7)>
GenServer
.
call
(
DropServer
,
60
)
{:ok,33.04542328371661}
This can be very convenient during the development phase, but be careful doing anything like this on a production machine. OTP has other mechanisms for updating code on the fly. There is also a built-in limitation to this approach: init
gets called only when start_link
sets up the service. It does not get called if you recompiled the code. If your new code requires any changes to the structure of its state, your code will break the next time it’s called.
When you started the DropServer
module from the shell, you effectively made the shell the supervisor for the module—though the shell doesn’t really do any supervision. You can break the module easily:
iex(8)>
GenServer
.
call
(
DropServer
,
-
60
)
** (EXIT from #PID<0.141.0>) an exception was raised:
** (ArithmeticError) bad argument in arithmetic expression
(stdlib) :math.sqrt(-1176.0)
(drop_server) lib/drop_server.ex:44: DropServer.fall_velocity/1
(drop_server) lib/drop_server.ex:20: DropServer.handle_call/3
(stdlib) gen_server.erl:615: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:647: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Interactive Elixir (1.3.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
10:50:58.899 [error] GenServer DropServer terminating
** (ArithmeticError) bad argument in arithmetic expression
(stdlib) :math.sqrt(-1176.0)
(drop_server) lib/drop_server.ex:44: DropServer.fall_velocity/1
(drop_server) lib/drop_server.ex:20: DropServer.handle_call/3
(stdlib) gen_server.erl:615: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:647: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: -60
State: %DropServer.State{count: 5}
The error message is nicely complete, even telling you the last message and the state, but when you go to call the service again, you can’t, because the IEx
shell has restarted. You can restart it with DropServer.start_link/0
again, but you’re not always going to be watching your processes personally.
Instead, you want something that can watch over your processes and make sure they restart (or not) as appropriate. OTP formalizes the process management you saw in Example 9-10 with its supervisor behavior.
A basic supervisor needs to support only one callback function, init/1
, and can also have a start_link
function to fire it up. The return value of that init/1
function tells OTP which child processes your supervisor manages and how how you want to handle their failures. A supervisor for the drop
module might look like Example 13-2, which is in ch12/ex2-drop-sup.
defmodule
DropSup
do
use
Supervisor
# convenience method for startup
def
start_link
do
Supervisor
.
start_link
(
__MODULE__
,
[],
[{
:name
,
__MODULE__
}])
end
# supervisor callback
def
init
([])
do
child
=
[
worker
(
DropServer
,
[],
[])]
supervise
(
child
,
[{
:strategy
,
:one_for_one
},
{
:max_restarts
,
1
},
{
:max_seconds
,
5
}])
end
# Internal functions (none here)
end
The init/1
function’s job is to specify the process or processes that the supervisor is to keep track of, and specify how it should handle failure.
The worker/3
function specifies a module that the supervisor should start, its argument list, and any options to be given to the worker’s start_link
function. In this example, there is only one child process to supervise, and the options are given as list of key/value tuples.
You can also specify the options as a keyword list, which you would write this way:
supervise(child, strategy: :one_for_one, max_restarts: 1, max_seconds: 5)
The supervise/2
function takes the list of child processes as its first argument and a list of options as its second argument.
The :strategy
of :one_for_one
tells OTP that it should create a new child process every time a process that is supposed to be :permanent
(the default) fails. You can also go with :one_for_all
, which terminates and restarts all of the processes the supervisor oversees when one fails, or :rest_for_one
, which restarts the process and any processes that began after the failed process had started.
When you’re ready to take more direct control of how your processes respond to their environment, you might explore working with the dynamic functions Supervisor.start/2
, Supervisor.terminate_child/2
, Supervisor.restart_child/2
, and Supervisor.delete_child/2
, as well as the restart strategy :simple_one_for_one
.
The next two values define how often the worker processes can crash before terminating the supervisor itself. In this case, it’s one restart every five seconds. Customizing these values lets you handle a variety of conditions but probably won’t affect you much initially. (Setting :max_restarts
to zero means that the supervisor will just terminate if a worker has an error.)
The supervise
function takes those arguments and creates a data structure that OTP will use. By default, this is a :permanent
service, so the supervisor should always restart a child when it fails. You can specify a :restart
option when defining the worker if you want to change this to a different value. The supervisor can wait five seconds before shutting down the worker completely; you can change this with the :shutdown
option when defining the worker. More complex OTP applications can contain trees of supervisors managing other supervisors, which themselves manage other supervisors or workers. To create a child process that is a supervisor, you use the supervisor/3
function, whose arguments are the same as those of worker/3
.
OTP wants to know the dependencies so that it can help you upgrade software in place. It’s all part of the magic of keeping systems running without ever bringing them to a full stop.
Now that you have a supervisor process, you can set up the drop function by just calling the supervisor. However, running a supervisor from the shell using the start_link/0
function call creates its own set of problems; the shell is itself a supervisor, and will terminate processes that report errors. After a long error report, you’ll find that both your worker and the supervisor have vanished.
In practice this means that you need a way to test supervised OTP processes (that aren’t yet part of an application) directly from the shell. This method explicitly breaks the bond between the shell and the supervisor process by catching the pid of the supervisor (line 2) and then using the Process.unlink/1
function to remove the link (line 3). Then you can call the process as usual with GenServer.call/2
and get answers. If you get an error (line 6), it’ll be okay. The supervisor will restart the worker, and you can make new calls successfully. The calls to Process.whereis(DropServer)
on lines 4 and 7 demonstrate that the supervisor has restarted DropServer
with a new pid.
iex(1)>
{
:ok
,
pid
}
=
DropSup
.
start_link
()
{:ok,#PID<0.44.0>}
iex(2)>
Process
.
unlink
(
pid
)
true
iex(3)>
Process
.
whereis
(
DropServer
)
#PID<0.45.0>
iex(4)>
GenServer
.
call
(
DropServer
,
60
)
{:ok,34.292856398964496}
iex(5)>
GenServer
.
call
(
DropServer
,
-
60
)
** (exit) exited in: GenServer.call(DropServer, -60, 5000)
** (EXIT) an exception was raised:
** (ArithmeticError) bad argument in arithmetic expression
(stdlib) :math.sqrt(-1176.0)
(drop_sup) lib/drop_server.ex:44: DropServer.fall_velocity/1
(drop_sup) lib/drop_server.ex:20: DropServer.handle_call/3
(stdlib) gen_server.erl:615: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:647: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
11:05:00.438 [error] GenServer DropServer terminating
** (ArithmeticError) bad argument in arithmetic expression
(stdlib) :math.sqrt(-1176.0)
(drop_sup) lib/drop_server.ex:44: DropServer.fall_velocity/1
(drop_sup) lib/drop_server.ex:20: DropServer.handle_call/3
(stdlib) gen_server.erl:615: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:647: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: -60
State: %DropServer.State{count: 1}
(elixir) lib/gen_server.ex:604: GenServer.call/3
iex(5)>
GenServer
.
call
(
DropServer
,
60
)
{:ok,34.292856398964496}
iex(6)>
Process
.
whereis
(
DropServer
)
#PID<0.46.0>
You can also open the Process Manager in Observer and whack away at worker processes through the Kill option on the Trace menu and watch them reappear.
This works, but it is only the tiniest taste of what supervisors can do. They can create child processes dynamically and manage their lifecycle in greater detail.
In this section, you will use Mix to create an application for the drop supervisor and server that you have written.
Reiterating what we did in “Firing It Up”, create a directory to hold your application by typing mix new name
, as in the following example:
$ mix new drop_app * creating README.md * creating .gitignore * creating mix.exs * creating config * creating config/config.exs * creating lib * creating lib/drop_app.ex * creating test * creating test/test_helper.exs * creating test/drop_app_test.exs Your Mix project was created successfully. You can use "mix" to compile it, test it, and more: cd drop_app mix test Run "mix help" for more commands.
Mix creates a set of files and directories for you. Change directory to the drop_app directory that Mix created. Then open up the mix.exs
file in your favorite text editor. We haven’t talked about this file before because we never
need to make changes to it, but now we will need to
modify it. So take a look:
defmodule
DropApp.Mixfile
do
use
Mix.Project
def
project
do
[
app
:
:drop_app
,
version
:
"0.0.1"
,
elixir
:
"~> 1.3"
,
build_embedded
:
Mix
.
env
==
:prod
,
start_permanent
:
Mix
.
env
==
:prod
,
deps
:
deps
]
end
# Configuration for the OTP application
#
# Type "mix help compile.app" for more information
def
application
do
[
applications
:
[
:logger
]]
end
# Dependencies can be Hex packages:
#
# {:mydep, "~> 0.3.0"}
#
# Or git/path repositories:
#
# {:mydep, git: "https://github.com/elixir-lang/mydep.git", tag: "0.1.0"}
#
# Type "mix help deps" for more examples and options
defp
deps
do
[]
end
end
The project/0
function lets you name your application, give it a version
number, and specify the dependencies for building the project.
The dependencies are returned by the deps/0
function. The commented example says that you need to have the mydep
project version 0.3.0 or higher, and it is available via git
at the specified URL. In addition to git:
, you may specify the location of a dependency as a local file (path:
)
In this example, the application doesn’t have any dependencies, so you may leave everything exactly as it is.
Throughout the book, we have been going directly into IEx, but, again, it’s time for a change. If you type the command mix compile
, Mix will compile your empty project. If you look in your directory, you will see that Mix has created an _build
directory for the compiled code.
$ mix compile Compiling 1 file (.ex) Generated drop_app app $ ls _build config lib mix.exs README.md test
An empty application isn’t very exciting, so copy the drop_server.ex
and drop_sup.ex
files that you wrote into the lib
folder. Then start
iex -S mix
. Mix will compile the new files, and you can start using the server straightaway.
$ iex -S mix Erlang/OTP 19 [erts-8.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Compiling 2 files (.ex) Generated drop_app app Interactive Elixir (1.3.1) - press Ctrl+C to exit (type h() ENTER for help) iex(1)> {:ok, pid} = DropServer.start_link() {:ok,#PID<0.60.0>}
The last steps you need to do are to write the application code itself and then tell Mix where everything is.
Inside mix.exs
, change the application/0
function to look like this:
def application do [ applications: [:logger], registered: [:drop_app], mod: {DropApp, []} ] end
The :registered
key is a list of all the names that your application registers (in this case, just :drop_app
), and :mod
is a tuple that gives the name of the module to be run when the application starts up and a list of any arguments to be passed to that module. :applications
lists any applications that your application depends on at runtime.
Here is the code that we have added to the DropApp
Module, which is in a file named drop_app.ex
in the ch12/ex3-drop-app/drop_app/lib directory.
defmodule
DropApp
do
use
Application
def
start
(
_type
,
_args
)
do
IO
.
puts
(
"Starting the app..."
)
# show that app is really starting.
DropSup
.
start_link
()
end
end
The start/2
function is required. The first argument tells how you want
the virtual machine that Elixir runs on to handle application crashes. The second argument gives the arguments that you defined in the :mod
key. The start/2
function should return a tuple of the form {:ok, pid}
, which is exactly what DropSup.start_link/0
does.
If you type mix compile
at the command prompt, Mix will generate a file _build/dev/lib/drop_app/ebin/drop_app.app. (If you look at that file, you will see an Erlang tuple that contains much of the information gleaned from the files you have already created.) You may then run the application from the command line.
$ elixir -pa _build/dev/lib/drop_app/ebin --app drop_app Starting the app...
There is much, much more to learn. OTP deserves a book or several all on its own. Hopefully this chapter provides you with enough information to try some things out and understand those books. However, the gap between what this chapter can reasonably present and what you need to know to write solid OTP-based programs is, to say the least, vast.
3.145.103.154