Powered by AppSignal & Oban Pro

CoView Diff Protocol — Interactive Tour

livebook/diff-protocol.livemd

CoView Diff Protocol — Interactive Tour

Mix.install([
  {:floki, "~> 0.36"},
  {:kino, "~> 0.13"},
  {:jason, "~> 1.4"}
])

This notebook walks through CoView’s DOM diff protocol — the same one implemented in extension/content.js and assets/js/dom_patch.js. Run the cells top-to-bottom; you’ll end up with a tiny pure-Elixir reference implementation that mirrors what the JS does in the browser.

If you’d rather read the protocol than execute it, see ARCHITECTURE.md.

Section 1 — The naive thing

The original CoView prototype sent the whole document on every mutation. The viewer ran morphdom against the new HTML. This works as a toy but is essentially a worse video stream — text is crisp, but bandwidth scales with document size, not with what actually changed.

# Same page, two states. Find what changed:
before = """

  
  • milk
  • eggs
"""
after_ = """
  • milk
  • eggs
"""
byte_size(after_)

Sending all ~110 bytes on every keystroke when only class="todo done" changed is the bandwidth bug. We want to send just the change.

Section 2 — Stable node IDs

CoView’s trick is to assign every Element a data-cv-id="N" attribute once, then refer to nodes by that ID forever.

defmodule Stamper do
  @doc """
  Walks the parsed HTML tree, assigns sequential ids on every Element,
  and returns {stamped_tree, max_id}.
  """
  def stamp(html) do
    {:ok, doc} = Floki.parse_document(html)
    {stamped, max_id} = stamp_nodes(doc, 1)
    {Floki.raw_html(stamped), max_id}
  end

  defp stamp_nodes(nodes, next_id) when is_list(nodes) do
    Enum.map_reduce(nodes, next_id, &stamp_node/2)
  end

  defp stamp_node({tag, attrs, children}, next_id) do
    attrs = [{"data-cv-id", Integer.to_string(next_id)} | attrs]
    {children, n2} = stamp_nodes(children, next_id + 1)
    {{tag, attrs, children}, n2}
  end

  defp stamp_node(other, next_id), do: {other, next_id}
end

{stamped, _} = Stamper.stamp(before)
stamped

Section 3 — Computing ops from a before/after diff

The leader’s MutationObserver naturally emits incremental ops. Outside of a browser we can simulate it by diffing two snapshots. This is a pedagogical exercise — the real leader uses MutationRecords, not diffing.

defmodule Differ do
  @doc """
  Diffs two stamped trees and emits CoView ops.
  Only attribute changes are implemented here — the real protocol covers
  insert/remove/text/value/checked/selected too.
  """
  def diff(before_tree, after_tree) do
    before_attrs = index_attrs(before_tree)
    after_attrs = index_attrs(after_tree)

    Enum.flat_map(after_attrs, fn {id, attrs} ->
      case Map.get(before_attrs, id) do
        ^attrs -> []
        nil -> []  # new node; would be an "insert" in the real protocol
        before_a ->
          Enum.flat_map(attrs, fn {k, v} ->
            if Map.get(before_a, k) != v do
              [%{op: "attr", id: id, name: k, value: v}]
            else
              []
            end
          end)
      end
    end)
  end

  defp index_attrs(html_string) do
    {:ok, doc} = Floki.parse_document(html_string)
    doc
    |> Floki.traverse_and_update(fn
      {_, attrs, _} = node ->
        case List.keyfind(attrs, "data-cv-id", 0) do
          {"data-cv-id", id} -> {:keep, id, Map.new(attrs)}
          _ -> node
        end
      other -> other
    end)
    |> Enum.flat_map(fn
      {:keep, id, attrs} -> [{id, attrs}]
      _ -> []
    end)
    |> Map.new()
  end
end

Run the diff:

{before_stamped, _} = Stamper.stamp(before)
# Mutate the "milk" li class:
{after_stamped, _} =
  Stamper.stamp(String.replace(before, ~s|class="todo">milk|, ~s|class="todo done">milk|))

ops = Differ.diff(before_stamped, after_stamped)
ops |> Jason.encode!(pretty: true) |> IO.puts()

In the real CoView, that one mutation produces a single op:

{"op":"attr", "id":"4", "name":"class", "value":"todo done"}

~50 bytes on the wire instead of re-sending the entire document.

Section 4 — Connecting to a live CoView server

The standard library can talk Phoenix Channels via Phoenix.Channels.GenSocketClient or — simpler from a notebook — Mint.WebSocket. We’ll skip the live connection here to keep this notebook offline-runnable, but the incoming wire payloads on a real connection look exactly like the ones in Section 3.

To inspect a real session, run mix phx.server in the repo, share a room, and watch the WebSocket frames in Chrome DevTools’ Network panel. Each frame is a Phoenix wire message containing exactly the op shape above.

Section 5 — Where to read next