Drawing the Curtain Back on the Magic Elixir - Part 1

Introduction
Last June I took over an Elixir based project that involves a central server communicating with various devices over a TCP/IP network.  This was my first real exposure to Elixir, so I quickly got a copy of Dave Thomas’ Programming Elixir and went through Code School’s Elixir tutorials so that I could at least start understanding the code I was inheriting.  Since then, I’ve spent a fair amount of time changing and adding to this project and feel that I’m starting to come to terms with Elixir and have something to say about it.

My purpose in these posts is not to offer another introductory course in the language but rather to look at some of the features that are said to distinguish it from other languages and think about how different these features really are and how they affect the readability and expressivity of the code that one writes.  In this first entry, we’ll look at pattern matching, which constitute the biggest difference between Elixir’s biggest differentiator, and the data structures available in Elixir.  A forthcoming post will look at supervision trees, error handling and the handling of state in Elixir.

Pattern Matching
Pattern matching is perhaps the feature that most distinguishes Elixir from traditional imperative languages and, for me, is the most interesting part of the language.  Take a statement like:

x = 1

In an imperative language, this assigns the value of 1 to the variable x.  In Elixir, it effectively does the same thing in this case.  But in Elixir, the equals symbol is called a match operator and actually performs a match between the left-hand side and the right-hand side of the operator.  So, we can do things like

{:ok, x} = Enum.fetch([2, 4, 6, 8], 2)

do_something(x)
report_success()

In this case, the fetch function of the Enum module would return {:ok, 6}, which would match the left-hand side of the expression and the variable x would take on the value of 6.  We then call the do_something method, passing it the value of x.  What would happen, though, were we to call fetch with a position that didn’t exist in the collection we passed it?  In this case, fetch returns the atom :error, which would not match the left-hand side of the expression.  If we don’t do anything to handle that non-matching expression, the VM will throw an error and stop the process that tried to execute the failed match.  (This is not as drastic as it sounds, see the section on Supervisors, below.)  All is not lost, though.  There are several ways we could avoid terminating the process when the match fails, but I’ll just describe two: the with statement and pattern matching as part of the formal parameters to a function declaration.  Strictly speaking, the former is a macro, which I’m not going to cover, but it acts like a flow control statement and quacks like a flow control statement, so I’m not going to worry about the distinctions.  Rewriting the above to avoid terminating the running process using a with statement would look like

with {:ok, x} <- Enum.fetch([2, 4, 6, 8], 2) do

do_something(x)

report_success()

else

:error -> report_fetch_error()

end

In this case, we’re saying that we’ll call do_something if and only if the result of calling fetch matches {:ok, x}; if the result of calling fetch is :error, we’re going to call report_error, and if the result is something else, which isn’t actually possible with this function, we’d still throw an error.  We can actually have multiple matches in both the with and the else clauses, so we could also do something like:

with {:ok, x} <- Enum.fetch([2, 4, 6, 8], 2),
:ok <- do_something(x) do
report_success()
else
:do_something_error -> report_do_something_error()
:error -> report_fetch_error()
end

Here, do_something will still not be executed unless the result of fetch matches the first clause, but the else clauses are executed with the first non-matching value in the with clauses.  If do_something also just returned :error, though, we’d have to separate this out into two with statements.

The other way we could handle the original problem of the non-matching expression would be to use pattern matching in the formal parameters of a set of functions we define.  For example, we could express the code above as:

do_something(Enum.fetch([2, 4, 6, 8], 2))

def do_something({:ok, x}) do
result = …  # :ok or :error
report_do_something_result(result)
end

def do_something(:error) do
report_fetch_error()
end

def report_do_something_result(:ok) do
report_success()
end

def report_do_something_result(:error) do
# whatever it takes to report the error
end

In this case, we’re using pattern matching to overload a function definition and do different things based on what parameters are passed instead of using flow control statements.  The Elixir literature tends to say that this is actually preferable to using flow control statements as it makes each method simpler and easier to understand, but I have my doubts about it.  I find that I tend to duplicate a lot of code when use pattern matching as a flow control mechanisms.  To be fair, I’m learning more and more techniques to avoid this, but in the end I suspect that the discouragement of flow control structures and, perhaps more importantly, the lack of subclassing makes the code harder to understand and makes it harder to avoid duplicating code than I’d like.

Sometimes, too, I feel like the cure for avoiding this kind of duplication can be worse than  the disease.  One technique I’ve seen used is essentially to use reflection, and I found that code very hard to follow and some of the code just got duplicated between modules instead of being duplicated in a single module.  There are other techniques for dealing with this kind of duplication, too, but I can’t shake the feeling that I’m being pushed into making the code less expressive when trying to remove duplication when other languages would have made removing the duplication easier and more obvious.

Another concern I have around this form of overloading methods through pattern matching is that the meaning of the patterns to match isn’t always obvious.  For example looking at two functions like:

def do_something(item, false) do
…
end

def do_something(item, true) do
…
end

How do you know what the true and false mean?  Sadly, you have to go back and find the code that’s calling these functions to see what the distinguishing parameter represents.  You can actually write the declarations to be more intention revealing:

def do_something(item, false = _active) do
…
end

def do_something(item, true = _active) do
…
end

But the majority of the code I’ve looked at is written in the former style that doesn’t reveal its intentions as well as the latter.

Pattern matching is at once both the most intriguing and a sometimes troubling element of the language.  I find it an interesting concept, but I’m still not sure whether it’s something I’m really happy about.

Data Structures
Elixir displays a curious dichotomy in regards to data structures.  On the one hand, we’re told that we shouldn’t use data structures because they represent the bad-old object oriented way of doing things.  On the other hand, Elixir does allow one to create structs, which define a set of fields, but no behavior, and set default values for them.  Structs are really just syntactic sugar on top of Elixir’s maps, though.  An Elixir map is a set of key/value mappings and a struct enforces the set of keys that one can use for that particular map.  Otherwise they’re interchangeable, so I’m going to ignore structs for the rest of this discussion.  Elixir offers two other data structure that aggregate data: lists and tuples.  Conceptually Elixir lists are pretty much like a list in other programming languages.  (Remembering that all of these data structures are immutable in Elixir.)  The documentation says that there are some low-level implementation differences that make Elixir lists more efficient to work with than in other languages, but I haven’t concerned myself terribly about this.  Finally, in terms of aggregating data, we have tuples.  One can use a tuple much like a list, except that again there are implementation details that make they more efficient to use in some situations than lists, while in other situations the lists are more efficient.  In practice, I’ve mostly seen tuples to be used when one wants to match a pattern and lists used when one is just storing the data.  One somewhat contrived example of all of this and then some discussion:

{:ok, %{grades: [g1, g2, g3], names: [n1, “student2”, n3}} = get_grades()

So, what’s going on here?  We’re again using pattern matching to expect the function get_grades to return a tuple consisting of the atom :ok and a map containing two lists of three elements each, where the name of the second student in the list must be “student2,” just to show that we can do a match to that level.

While more specific than is typical, the pattern of returning status and response is fairly common in Elixir.  This supposedly obviates a lot of exception handling, but to my way of thinking, it really just replaces it with with statements, as described previously.  I’m not convinced that it makes that much difference.

I think that the prevalent use of atoms, on the other hand, makes a large difference.  Sadly, it’s a negative difference in that atoms turn into another form of magic numbers (in the code smell sense).  Elixir has no way to define a truly global constant, so there’s actually no way around polluting one’s code with these magic numbers and there’s no compiler support for when one mistypes one of them.  For example, consider some hypothetical code dealing with a TCP socket:

def send_stuff(socket, stuff) do
with :ok <- :gen_tcp.send(socket, stuff) do
:ok
else
{:error, :econnrefused} -> …
{:error, :timeout} -> …
end
end

def open_socket(address, port, options) do
with {:ok, socket} <- :gen_tcp.connect(address, port_options) do
{:ok, socket}
else
{:error, :econnnrefused} -> …
{:error, :timeout} -> …
end
end

There’s nothing to tell me that I’ve accidentally added an extra “n” to :econnrefused in the open_socket function until it it turns into a runtime error. (This is also the kind of code that will often not get unit tested because it’s hard to deal with.  I haven’t had to write any code at this level yet, so I’m unsure if I could mock gen_tcp or not, but that’s the only way I could think of to ensure that all the error handling worked correctly.)

Conclusion
I find Elixir’s pattern matching a very interesting feature of the language.  Sometimes I think that it’s a nice step towards adding assertions as first-class elements of production code.  At other times, though, I wonder if it really adds to the readability of the code.  For example, is the send_stuff example really more readable than:

public send_stuff(socket, stuff) {
try {
socket.send(stuff)
} catch (ConnectionRefusedException cre) {
…
} catch (TimeoutException te) {
…
}
}

That being said, I often vacillate between Java’s checked exceptions and .NET’s unchecked exceptions.  Elixir has pushed me much more towards making exceptions checked by default since that makes sure that I think about them when .NET and Elixir let me forget about them, often to my detriment.

Unless and until Elixir offers truly global constants, however, I find myself distrusted the use of atoms in pattern matching to distinguish between functions.  I worry that I have too much code that’s like:

def do_something(:type1 = _device_type, other_params) …
def do_something(:type2 = _device_type, other params) …
def do_something(:type3 = _device_type, other_params) …

Granted that I could probably do more to avoid this, but I also feel like this would make the code less expressive.

Next time we’ll look at supervision trees, error handling and handling state in Elixir.

About the Author
Narti is a former CC Pace employee now working at ConnectDER.  He’s been interested in the design of programming languages since first reading The Dragon Book and the proceedings of HOPL I some thirty years ago.