Elixir and Livebook: concurrency, web, and AI

Mix.install(
  [
    {:kino, "~> 0.11.0"},
    {:bandit, "~> 1.0"},
    {:req, "~> 0.4"},
    {:kino_vega_lite, "~> 0.1.11"},
    {:kino_bumblebee, "~> 0.4.0"},
    {:exla, ">= 0.0.0"},
    {:kino_db, "~> 0.2.3"},
    {:exqlite, "~> 0.11.0"}
  ],
  config: [nx: [default_backend: EXLA.Backend]]
)

Welcome

Elixir is a dynamic and functional programming language that runs on the Erlang VM:

list = ["hello", 123, :banana]

Enum.fetch!(list, 0)

In Elixir it is common to write our code a series of data transformations with the |> (pipe) operator. Let’s give it a try.

Concurrency

Elixir supports pattern-matching, polymorphism via protocols, meta-programming, and more. But today, we will focus on its concurrency features. In the Erlang VM, all code runs inside lightweight threads called processes. We can literally create millions of them:

for _ <- 1..1_000_000 do
  spawn(fn -> :ok end)
end

Process communicate by sending messages between them:

parent = self()

child =
  spawn(fn ->
    receive do
      :ping -> send(parent, :pong)
    end
  end)

send(child, :ping)

receive do
  :pong -> :it_worked!
end

And Livebook can helps us see how processes communicate between them:

Kino.Process.render_seq_trace(fn ->
  parent = self()

  child =
    spawn(fn ->
      receive do
        :ping -> send(parent, :pong)
      end
    end)

  send(child, :ping)

  receive do
    :pong -> :it_worked!
  end
end)

Maybe you want to see how Elixir can perform multiple tasks at once, scaling on both CPU and IO?

Kino.Process.render_seq_trace(fn ->
  ["/foo", "/bar", "/baz", "/bat"]
  |> Task.async_stream(
    fn _ -> Process.sleep(Enum.random(100..300)) end,
    max_concurrency: 4
  )
  |> Enum.to_list()
end)

Let’s take visualizations even further!

Plotting live data

The Erlang VM provides a great set of tools for observability. Let’s gather information about all processes:

processes =
  for pid <- Process.list() do
    info = Process.info(pid, [:reductions, :memory, :status])

    %{
      pid: inspect(pid),
      reductions: info[:reductions],
      memory: info[:memory],
      status: info[:status]
    }
  end

But how to plot it?

Demo time: Web + AI

Live programming: drag and drop

Try drag-and-dropping some files!

Live programming: debugging

Use |> dbg() after a pipeline for some awesome debugging.

Live programming: doctests

Doctests sit at the intersection of documentation and testing:

defmodule HelloWorld do
  @doc """

      iex> HelloWorld.my_addition(1, 2)
      3

      iex> HelloWorld.my_addition(1, 2)
      4

      iex> HelloWorld.my_addition(1, "2")
      3

  """
  def my_addition(a, b) do
    a + b
  end
end