GenServers and Supervisors

Mix.install([
  {:kino, "~> 0.6.2"},
  {:factory, path: "./factory"}
])

Connecting Projects

• Adding remote projects by github url or version
• Adding local projects for development purposes.

GenServers

To demonstrate OTP in LiveBook, we’re going to use a Miner GenServer which digs for gold.

classDiagram
class Miner {
  gold: :integer
}

Miner --> Miner : dig

defmodule Miner do
  use GenServer

  def start_link(opts) do
    GenServer.start_link(__MODULE__, [], opts)
  end

  @impl true
  def init(_opts) do
    schedule_dig()
    {:ok, %{gold: 0}}
  end

  @impl true
  def handle_info(:dig, %{gold: amount}) do
    schedule_dig()
    {:noreply, %{gold: amount + 1}}
  end

  defp schedule_dig do
    Process.send_after(self(), :dig, 1000)
  end
end

{:ok, pid} = Miner.start_link([])

We can see this process is running and sending itself messages.

:sys.get_state(pid)

Keep in mind, every time we run the cell above, we’re starting a new process. This may produce unexpected/undesired results. (re-evaluate the Miner cell and see the number of processes increase)

Process.list() |> Enum.count()

At any time if we get into a bad state, we can press the 00 keyboard shortcut to reconnect the livebook process, thus stopping all created processes.

Named GenServers

defmodule MinerJim do
  use GenServer

  def start_link(_opts) do
    GenServer.start_link(__MODULE__, [], name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    schedule_dig()

    {:ok, %{gold: 0}}
  end

  @impl true
  def handle_info(:dig, %{gold: amount}) do
    schedule_dig()
    {:noreply, %{gold: amount + 1}}
  end

  defp schedule_dig do
    Process.send_after(self(), :dig, 5000)
  end
end

{:ok, pid} = MinerJim.start_link([])

Rerunning the cell above or attempting to start the GenServer again causes an :already_started error.

Trapping Exit

We can use Process.flag/2 to trigger the terminate/2 callback for our GenServer.

defmodule TrappedMiner do
  use GenServer

  def start_link(_opts) do
    GenServer.start_link(__MODULE__, [], name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    schedule_dig()
    # The terminate/2 callback is called for a normal exit.
    Process.flag(:trap_exit, true)

    {:ok, %{gold: 0}}
  end

  @impl true
  def handle_info(:dig, %{gold: amount}) do
    schedule_dig()
    {:noreply, %{gold: amount + 1}}
  end

  @impl true
  def terminate(_reason, _label) do
    IO.puts("Saved the Miner!")
  end

  defp schedule_dig do
    Process.send_after(self(), :dig, 1000)
  end
end

trapped_miner_pid = Process.whereis(TrappedMiner)

if trapped_miner_pid do
  Process.exit(trapped_miner_pid, :normal)
  # The process needs time to terminate.
  Process.sleep(100)
end

{:ok, pid} = TrappedMiner.start_link([])

This means we can keep killing and re-creating a named process without issue.

So far, this is the best way I’ve found but there may be better alternatives.

Notice the number of processes does not change when we re-evaluate the Elixir cell above.

Process.list() |> Enum.count()

Some additional solutions that may be preferable depending on the context.

• Write any related code in a different cell so the MatchError doesn’t cause issues.
• Use an unlinked process with start/3 instead of start_link/3 and kill the process if it is alive.
• Reconnect the livebook environment (manually or using the 00 keyboard shortcut)

unlinked_pid = Process.whereis(:bill)

if unlinked_pid do
  Process.exit(unlinked_pid, :kill)
  # Process needs time to end 
  Process.sleep(100)
end

GenServer.start(Miner, [], name: :bill)

Supervisors

We’re going to have a few named miner processes under a supervisor.

flowchart
S[Supervisor]
M1[Miner1]
M2[Miner2]
M3[Miner3]

S --> M1
S --> M2
S --> M3

defmodule Miner1 do
  use GenServer

  def start_link(_opts) do
    GenServer.start_link(__MODULE__, [], name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    schedule_dig()
    {:ok, %{gold: 0}}
  end

  @impl true
  def handle_info(:dig, %{gold: amount}) do
    schedule_dig()
    {:noreply, %{gold: amount + 1}}
  end

  defp schedule_dig do
    Process.send_after(self(), :dig, 1000)
  end
end

defmodule Miner2 do
  use GenServer

  def start_link(_opts) do
    GenServer.start_link(__MODULE__, [], name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    schedule_dig()
    {:ok, %{gold: 0}}
  end

  @impl true
  def handle_info(:dig, %{gold: amount}) do
    schedule_dig()
    {:noreply, %{gold: amount + 1}}
  end

  defp schedule_dig do
    Process.send_after(self(), :dig, 1000)
  end
end

defmodule Miner3 do
  use GenServer

  def start_link(_opts) do
    GenServer.start_link(__MODULE__, [], name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    schedule_dig()
    {:ok, %{gold: 0}}
  end

  @impl true
  def handle_info(:dig, %{gold: amount}) do
    schedule_dig()
    {:noreply, %{gold: amount + 1}}
  end

  defp schedule_dig do
    Process.send_after(self(), :dig, 1000)
  end
end

Named Supervisors suffer from the same issue as named GenServers. Arguably the issue is worse, because we cannot easily update their children without reconnecting the LiveBook instance.

children = []

# children = [
#   {Miner, []}
# ]

Supervisor.start_link(children, strategy: :one_for_one, name: :doomed_to_already_start)

Supervisor.which_children(:doomed_to_already_start)

We can avoid this issue by first stopping the supervisor if it’s already running.

sup_pid = Process.whereis(:restart_example)

if sup_pid do
  Supervisor.stop(sup_pid)
end

children = []

# children = [
#   {Miner, []}
# ]

Supervisor.start_link(children, strategy: :one_for_one, name: :restart_example)

Supervisor.which_children(:restart_example)

Visualizing Supervision Trees

Let’s use this pattern to start a :mine_supervisor that will supervise Miner1, Miner2, Miner3, and a DynamicSupervisor for more Miner processes.

sup_pid = Process.whereis(:mine_supervisor)

if sup_pid do
  Supervisor.stop(sup_pid)
end

children = [
  {DynamicSupervisor, strategy: :one_for_one, name: :miner_supervisor},
  {Miner1, []},
  {Miner2, []},
  {Miner3, []}
]

Supervisor.start_link(children, strategy: :one_for_one, name: :mine_supervisor)

Kino.Process comes with a sup_tree/2 function for visualizing supervision trees.

Kino.Process.sup_tree(:mine_supervisor)

We can add processes and see them reflected in our diagram (uncomment the code below.)

DynamicSupervisor.start_child(:miner_supervisor, {Miner, []})

Animated Supervision Trees

By leveraging Kino animations, we can even create an updating diagram.

Kino.animate(1000, 0, fn i ->
  tree = Kino.Process.sup_tree(:mine_supervisor)
  {:cont, tree, i + 1}
end)

Kino.animate(1000, 0, fn i ->
  mine_children = inspect(Supervisor.count_children(:mine_supervisor))
  miner_children = inspect(Supervisor.count_children(:miner_supervisor))

  md =
    Kino.Markdown.new("""
    **:mine_supervisor children: `#{mine_children}`**

    **:miner_supervisor children: `#{miner_children}`**
    """)

  {:cont, md, i + 1}
end)

# {:ok, pid} = DynamicSupervisor.start_child(:miner_supervisor, {Miner, [name: :name]})
{:ok, pid} = DynamicSupervisor.start_child(:miner_supervisor, {Miner, []})

We can even kill processes and see them reflected in an updated diagram above when we re-evaluate the diagram. (set pid to one of the pids under the :miner_supervisor)

pid = :c.pid(0, 330, 0)
DynamicSupervisor.terminate_child(:miner_supervisor, pid)

Or see a process restarted with a new PID under the tree.

pid = :c.pid(0, 345, 0)

Process.exit(pid, :kill)

App Supervision Trees

Kino.Process provides an app_sup/1 function we can use to explore supervision trees in our applications and in external libraries.

For sake of example we’ve included a Factory mix project you can see in the ./factory folder of this project.

Kino.Process.app_tree(:factory)

We can also use the sup_tree/2 function on the application’s supervisor. This excludes the two processes coming from livebook.

Kino.Process.sup_tree(Factory.Supervisor)

Here’s a fun example where we can monitor the state of processes in our application.

> Side Note: there’s an upcoming feature to trace processes in LiveBook by Alex Koutmos

Kino.animate(1000, 0, fn _ ->
  tree = Kino.Process.sup_tree(Factory.Supervisor)
  {:cont, tree, 0}
end)

Kino.animate(1000, 0, fn _ ->
  gold = inspect(:sys.get_state(Factory.GoldMiner))
  silver = inspect(:sys.get_state(Factory.SilverMiner))
  iron = inspect(:sys.get_state(Factory.IronMiner))

  dynamic =
    DynamicSupervisor.which_children(Factory.DynamicMinerSupervisor)
    |> Enum.map(fn {_, pid, _, _} ->
      "- #{inspect(pid)}: #{inspect(:sys.get_state(pid))}\n"
    end)

  storage = inspect(:sys.get_state(Factory.Storage))

  md =
    Kino.Markdown.new("""
    - Factory.GoldMiner: #{gold}
    - Factory.SilverMiner: #{silver}
    - Factory.IronMiner: #{iron}
    - Factory.Storage: #{storage}

    DynamicMinerSupervisor:

    #{dynamic}
    """)

  {:cont, md, 0}
end)

DynamicSupervisor.start_child(Factory.DynamicMinerSupervisor, {Factory.GoldMiner, [name: :test]})
# DynamicSupervisor.start_child(Factory.DynamicMinerSupervisor, {Factory.GoldMiner, []})
DynamicSupervisor.start_child(Factory.DynamicMinerSupervisor, {Factory.IronMiner, []})

We can use this with any application available from the livebook environment. We can use :application_controller to view available applications.

:application_controller.which_applications()

We can use this to explore other projects if they have supervisors! The :direction option is useful for viewing larger trees.

Kino.Process.app_tree(:kernel, direction: :left_right)

Observer

We can also start the observer from Livebook.

# :observer.start()

Up Next

• ETS