Data massaging in pipes with anonymous functions

The pipe operator is great. So great, in fact, that sometimes you just want to keep that pipe juice flowing and feel bad about breaking it up just to massage data into the appropriate shape before continuing with a new pipe.

Anonymous functions can help out here. But you need to realize that each step in the pipe is a function call. Therefore, to use an anonymous function within a pipe, you have to call it with my_func.():

def to_email(full_name, domain) do
    full_name
    |> String.split()
    |> Enum.join(".")
    |> (&"#{&1}@#{domain}").()
    |> String.downcase()
 end

# iex> to_email("John Doe", "example.com")
# john.doe@example.com

On line 5, I simply wanted to create an email string without going through the fuss of writing a new function. Of course you can name the anonymous function, too:

def to_email(full_name, domain) do
    append_domain = &"#{&1}@#{domain}"

    full_name
    |> String.split()
    |> Enum.join(".")
    |> append_domain.()
    |> String.downcase()
end

As you’ve likely noticed, this technique is rife with potential for misuse and abuse: if you decide to use it, make sure your code’s readability doesn’t suffer! Otherwise, simply resort to the usual technique of grouping piped operations within larger functions (who have compatible output/input data shapes), and pipe those functions together.

Posted in Elixir, Pattern snippets | Comments Off on Data massaging in pipes with anonymous functions

Finer control in `with` failed matches

Elixir gives us the with construct to combine matching clauses, which is very handy as programs frequently to perform an operation only if/when a set of preconditions is met.

Let’s say we want to charge a product to a credit card:

with {:ok, validated_card} <- validate_cc(card),
      {:ok, amount} <- apply_sales_tax(item_price, :base),
      {:ok, amount} <- apply_sales_tax(item_price, :luxury),
      {:ok, transaction_id} <- charge_card(validated_card, amount) do
  {:ok, %{id: transaction_id, card: validated_card, amount: amount}}
else
  _ -> {:error, "unable to charge credit card"}
end

We’ve got a few steps we need to succeed, and if any one of those steps fail, we return an error. Unfortunately, as our code stands, the calling code won’t be able to differentiate what caused the error. Maybe it would ask the customer to enter credit card details again if the card validation failed, or a support ticket would be logged with the credit card processing if the card couldn’t be charged? Too bad: you can’t decide what to do without first knowing what went wrong.

But take a closer look at line 7: we’re pattern matching there just like we would in a case. So we should be able to pattern match on the returned errors, right? Unfortunately, all 3 calls on lines 1-4 just return :error if they’re not successful. Foiled again!

But maybe, just maybe, we could tag these statements? That would allow us to determine exactly what step failed even in cases where we’re calling the same function several times (like on lines 2-3). Let’s give it a go:

with {:validation, {:ok, validated_card}} <- {:validation, validate_cc(card)},
      {:base_tax, {:ok, amount}} <- {:base_tax, apply_sales_tax(item_price, :base)},
      {:luxury_tax, {:ok, amount}} <- {:luxury_tax, apply_sales_tax(item_price, :luxury)},
      {:transaction, {:ok, transaction_id}} <- {:transaction, charge_card(validated_card, amount)} do
  {:ok, %{id: transaction_id, card: validated_card, amount: amount}}
else
  {:validation, _} -> {:error, :invalid_card}
  {:base_tax, _} -> {:error, :base_tax_failed}
  {:luxury_tax, _} -> {:error, :luxury_tax_failed}
  {:transaction, _} -> {:error, :transaction_failed}
end

Now, the calling code can know exactly which step failed and respond appropriately. Or if it doesn’t case, it can simply match on {:error, _} and handle all failures the same. And of course, it could handle only a subset of the errors (e.g. invalid cards) and just use the _ match to have a catch all case for handling other errors. The point is: the power is in the caller’s hands, where it should be.

Of course, don’t forget that you can choose to tag some result values, but not others. And of course, you could tag several lines with the same atom:

with {:validation, {:ok, validated_card}} <- {:validation, validate_cc(card)},
      {:tax, {:ok, amount}} <- {:tax, apply_sales_tax(item_price, :base)},
      {:tax, {:ok, amount}} <- {:tax, apply_sales_tax(item_price, :luxury)},
      {:ok, transaction_id} <- charge_card(validated_card, amount) do
  {:ok, %{id: transaction_id, card: validated_card, amount: amount}}
else
  {:validation, _} -> {:error, :invalid_card}
  {:tax, _} -> {:error, :tax_failed}
  _ -> {:error, :unknown}
end

I hope you’ll find this concept of tagging useful. In fact, you might come across it in other places, such as in Ecto.Multi.

Posted in Elixir, Pattern snippets | Comments Off on Finer control in `with` failed matches

Pattern matching in function heads: don’t go overboard

Elixir’s pattern matching is great, but make sure it’s not reducing your code’s readability. In particular, just because you can match all the variables you’ll need from within the function’s head doesn’t mean you should.

Take this code:

  def handle_call(:checkout, %{workers: [h | t], monitors: monitors}) do
    # ...
  end

  def handle_call(:checkout, %{workers: [], idle_overflow: [h | t]}) do
    # ...
  end

  def handle_call(:checkout, %{workers: [], idle_overflow: [],
      overflow: overflow, overflow_max: max, worker_sup: sup,
      spec: spec, monitors: monitors})
      when overflow < max do
    # ...
  end

  def handle_call(:checkout, %{workers: [], idle_overflow: [],
      overflow: overflow, overflow_max: max, waiting: waiting}) do
    # ...
  end

A lot of matching is going on there, but how much of it is to differentiate the function heads, and how much is convenience (i.e. binding for later use)?

By leaving only the matches that differentiate the heads and moving other bindings to the function bodies, readability can be improved (although, sadly, this is poorly demonstrated due to the line wrapping required by this blog format):

def handle_call(:checkout, %{workers: [h | t]} = state) do
    %{monitors: monitors} = state
    # ...
  end

  def handle_call(:checkout, %{workers: [], idle_overflow: [h | t]}) do
    # ...
  end

  def handle_call(:checkout, %{workers: [], idle_overflow: [],
      overflow: overflow, overflow_max: max} = state) when overflow < max do
    %{worker_sup: sup, spec: spec, monitors: monitors} = state
    # ...
  end

  def handle_call(:checkout, state) do
    %{workers: [], idle_overflow: [], overflow: overflow,
        overflow_max: max, waiting: waiting} = state
    # ...
  end

Of course, I’m not saying that you should always split matching/binding between function head and body. But if you find yourself binding so many variables in your function heads that your code’s readability suffers, take a good look at them. The bindings that aren’t discriminating can most likely be moved to the function’s body, simplifying your code.

Update June 28th, 2018: It turns out José also uses this technique:

I tend to use this rule: if the key is necessary when matching the pattern, keep it in the pattern, otherwise move it to the body. So I end-up with code like this:

def some_fun(%{field1: :value} = struct) do
  %{field2: value2, field3: value3} = struct
  ...
end
Posted in Elixir, Pattern snippets | Comments Off on Pattern matching in function heads: don’t go overboard

Keyword list reduction

Keyword lists are often used to provide optional values, which are then processed (for example to initialize the state). One really nice way to do so, is with a reduce pattern.

I’ve never seen anyone bringing attention to it in training materials I’ve seen on Elixir (although I did see it mentioned in this thread). Given how prevalent keyword list processing is for option handling, and how readable the resulting code is, I thought I’d mention it here.

Here’s, the code:

defmodule ReducerDemo do
  @options [:name, :timeout]

  def init([type, count | opts]) when is_atom(type) and is_integer(count) and count > 0 do
    init(opts, %{type: type, count: count})
  end

  defp init([], state), do: {:ok, state}

  defp init([{:timeout, _} | _rest], %{type: :immediate}) do
    {:stop, {:conflicting_option, "cannot use `timeout` option with type :immediate"}}
  end

  defp init([{:timeout, t} | rest], state) when is_integer(t) and t > 0 do
    init(rest, Map.put(state, :timeout, t))
  end

  defp init([{:name, n} | rest], state) when is_atom(n) do
    init(rest, Map.put(state, :name, n))
  end

  defp init([{name, value} | _], _state) when name in @options do
    {:stop, {:invalid_option, "invalid value `#{value}` given to option `#{name}`"}}
  end

  defp init([{name, _value} | _], _state) do
    {:stop, {:invalid_option, "`#{name}` is not a valid option"}}
  end
end

Let’s say init/1 is a callback for a GenServer: it should return {:ok, state} or {:stop, reason} (see docs). Within the list, it expects 2 mandatory arguments, along with an optional list of options. (Note that all of these are provided within a list due to the callback’s arity.)

In the public init/1 function head, we typecheck the mandatory arguments. Then, within its body, we call a private init/2 function with the list of options and the initial state containing the mandatory values.

On line 8, we have the actual return taking place: if no options are left to process, just return the current state we’ve built up.

Let’s ignore lines 10-12 for now, and swing back to them later.

Lines 14-20 represent the happy path: match one a variable (with guards ensuring it’s the expected type), add it to the state, and recursively call init/2. Recall that a keyword list such as [a: 1, b:2] is just nicer syntax for the real representation, which is a list of tuples: [{:a, 1}, {:b, 2}] (read more about that here). In order to pattern match, we need to rely on the tuple representation.

Finally, we get to error handling. The first type of errors we handle are conflicting options on lines 10-12: if we discover an option that is incompatible with one already provided/processed, we return an error. Error functions return directly: there is no further processing of remaining options since we don’t recursively call init/2. The {:stop, reason} tuple is returned only because this example dealt with initializing state for a GenServer, and that is the expected error return value (see docs). Of course, your own code should return whatever makes sense in the current context.

We’ve got 2 more cases to handle: we were provided a valid option with an invalid value, or we were given an invalid option.

The first case is handled with the function on line 22: if the option name is in the whitelisted options (defined as a module attribute on line 2), then its value is incorrect.

If one the other hand an unrecognized option was provided, the function head on line 26 will be matched and return the appropriate message.

Let’s give it a spin:

iex(1)> c "reducer_demo.ex"
iex(2)> ReducerDemo.init([:delayed, 3, name: :foobar, timeout: 500])
{:ok, %{count: 3, name: :foobar, timeout: 500, type: :delayed}}
iex(3)> ReducerDemo.init([:delayed, 3, timeout: 500]) 
{:ok, %{count: 3, timeout: 500, type: :delayed}}
iex(4)> ReducerDemo.init([:delayed, 3]) 
{:ok, %{count: 3, type: :delayed}}
iex(5)> ReducerDemo.init([:immediate, 3, timeout: 500])
{:stop, {:conflicting_option, "cannot use `timeout` option with type :immediate"}}
iex(6)> ReducerDemo.init([:delayed, 3, nane: :foobar])
{:stop, {:invalid_option, "`nane` is not a valid option"}}
iex(7)> ReducerDemo.init([:delayed, 3, name: "foobar"])
{:stop, {:invalid_option, "invalid value `foobar` given to option `name`"}}

Finally, astute readers will have noticed that the “reason” tagged tuples on lines 23 and 27 have the same tag and just differ in their message. This was an arbitrary API choice: in this case, it would be expected that clients only match on the tag (therefore being unable to determine whether the error was caused by an invalid option name or value). In other words, the string message isn’t intended for flow control, it’s there for the developer: if an invalid option pops up, the message will tell you if you’ve mistyped an option name or if the value was invalid.

Posted in Elixir, Pattern snippets | Comments Off on Keyword list reduction

Rendering a view after multiple async functions return (using promises)

In the last post, we saw how to use jQuery’s promises to render a view once data had been returned from an asynchronous function. Let’s take the concept a little further and see how we could render the view only once we have data from multiple async sources.

Continue reading

Posted in AJAX, Backbone.js, Backbone.Marionette, jQuery | 2 Comments

Using jQuery promises to render Backbone views after fetching data

So I’ve been working through Brian Mann’s Marionette screencasts at Backbonerails.com. They’re absolutely top notch material, and I can’t recommend them enough. So if you want to learn how to go about building large Backbone applications (even if the back end won’t be rails), check them out!

In episode 6, Brian goes over several strategies for fetching remote data and updating the views in consequence. He also mentions using jQuery promises,  which isn’t shown in the screencast. So I thought I’d give it a go and attempt implementing it. Unfortunately, since this blog post depends directly on Brian’s screen cast and code, it might be harder to follow along if you don’t have them handy.

Continue reading

Posted in AJAX, Backbone.js, Backbone.Marionette, jQuery | 2 Comments

Tutorial: Nested views using Backbone Marionette’s CompositeView

A reader recently asked me how I would go about using Marionette to create an accordion like Twitter’s Bootstrap collapse component. In this post, I’ll not only give you my take on the answer, but also guide you through my reasoning. My hope is that by following my reasoning, you’ll be able to apply any time you time “how can I do this with Marionette” ? Continue reading

Posted in Backbone.js, Backbone.Marionette | 5 Comments

Tutorial: a full Backbone.Marionette application (part 2)

In the previous post, we started writing our application (see it live!) and now have a functional app, even if it still is somewhat basic.

Continue reading

Posted in Backbone.js, Backbone.Marionette | 5 Comments

Tutorial: a full Backbone.Marionette application (part 1)

In previous posts, I introduced you to Backbone.Marionette which provides many niceties to help you build complex Backbone.js apps. We’ll cover more advanced topics here, such as Backbone.History, modal windows, organizing code into sub applications, and more. The resulting app can bee seen live here. Let’s get started!

Continue reading

Posted in Backbone.js, Backbone.Marionette | 12 Comments

A simple Backbone.Marionette tutorial (part 2)

In the previous post, we started building a simple cat leader board (see it live!) and got all the way to displaying the list of cat names. Let’s finish and get to the result! Continue reading

Posted in Backbone.js, Backbone.Marionette | 6 Comments