ASmap Daily Diff

Mix.install([
  {:kino_db, "~> 0.2.8"},
  {:exqlite, "~> 0.11"},
  {:vega_lite, "~> 0.1.6"},
  {:kino_vega_lite, "~> 0.1.11"}
])

ASmap Daily Run Analysis

By diff’ing two ASmaps generated by kartograf, we get a list of which IP networks have changed assignment status. These can be moved from one AS to another, unassigned, or a new AS assignment we hadn’t seen before (i.e. it wasn’t present in the first ASmap but present in the second).

The database used here is created by running kartograf, producing an ASmap, and running the diff against the previous day’s ASmap. Thus we can see how an ASmap changes day to day. The table is called “diffs” and the columns are: previous_epoch, current_epoch, network, previous_asn, current_asn. The data covers a 6 week period, during 2024/11/22 - 2025/01/03.

Step 1: let’s focus on reassignments, i.e. trades between one AS and another.

1. daily frequency graph: per day / epoch, count of changes
2. transfers per AS: count of transfers volume.
3. top-k active trading ASes, networks traded.

Step 2: how fast does an ASmap decay? Given an initial set, what decay does an ASmap show over time? meaning, to what extent does a future ASmap have different data?

opts = [database: "../data/asmap.db"]
{:ok, conn} = Kino.start_child({Exqlite, opts})

Network Reassignments per day

How many network reassignments occur per day?

perday =
  Exqlite.query!(
    conn,
    ~S"""
    select current_epoch, count(*) from diffs 
      where previous_asn is not NULL and current_asn is not NULL
      group by current_epoch
      order by current_epoch ASC
    """,
    []
  )

perday_count = perday.rows
|> Enum.map(fn [ts, count]-> %{date: DateTime.from_unix!(ts), count: count} end)

VegaLite.new(
  width: 600,
  height: 400,
  title: "Network Reassignments per Day, 22/11/24 - 03/01/25"
)
|> VegaLite.data_from_values(perday_count, only: ["date", "count"])
|> VegaLite.mark(:bar)
|> VegaLite.encode_field(:x, "date", type: :temporal)
|> VegaLite.encode_field(:y, "count", type: :quantitative)

ASN Reassignments per Network

How much reassignment activity occurs across all AS ?

To get a sense of how often reassignments occur, let’s plot the count of reassignments by the count of networks. Again, we’ll focus on reassignments from one AS to another, excluding new assignments and unassigned networks.

result =
  Exqlite.query!(
    conn,
    ~S"select * from diffs where previous_asn is not NULL and current_asn is not NULL",
    []
  )

moves = result.rows
|> Enum.reduce(%{}, fn x, acc -> 
  network = Enum.at(x, 2)
  new_asn = Enum.at(x, 4)
  Map.update(acc, network, [new_asn], fn current -> [ new_asn | current ] end)
  end)

now count the AS reassignments, and group by the count.

transfer_freq = moves 
  |> Enum.map(fn {k, v} -> {k, Enum.count(v)} end)
  |> Enum.group_by(&(elem(&1, 1)))
  |> Enum.map(fn {k, v} -> %{reassignments_per_network: k, network_count: Enum.count(v)} end)

# a code cell so we can customize the axes scale
VegaLite.new(title: "Network Reassignment Frequency, log", width: 600, height: 400)
|> VegaLite.data_from_values(transfer_freq)
|> VegaLite.mark(:bar)
|> VegaLite.encode_field(:x, "reassignments_per_network", type: :quantitative, title: "Reassignments per Network")
|> VegaLite.encode_field(:y, "network_count", type: :quantitative, scale: [type: :log], title: "Count of Networks, log")

For networks that were reassigned, the reassignment frequency is distributed like this.

Reassignment volume per AS

Which AS see the most network reassignments? i.e. which AS are the most active traders of networks?

incoming =
  Exqlite.query!(
    conn,
    ~S"""
    select 
      current_asn,
      count(current_asn) as ct
    from diffs where previous_asn is not NULL and current_asn is not NULL
    group by current_asn
    order by ct desc
    LIMIT 50
    """,
    []
  )

outgoing =
  Exqlite.query!(
    conn,
    ~S"""
    select 
      previous_asn,
      count(previous_asn) as ct
    from diffs where previous_asn is not NULL and current_asn is not NULL
    group by previous_asn
    order by ct desc
    LIMIT 50
    """,
    []
  )

incoming_by_asn = incoming.rows |> Enum.map(fn [asn, count] -> %{asn: asn, count: count, assignment: :in} end)
outgoing_by_asn = outgoing.rows |> Enum.map(fn [asn, count] -> %{asn: asn, count: count, assignment: :out} end)
all = incoming_by_asn ++ outgoing_by_asn

VegaLite.new(width: 800, height: 400, title: "Count of Network assignments by AS")
|> VegaLite.data_from_values(all, only: ["asn", "count", "assignment"])
|> VegaLite.mark(:bar)
|> VegaLite.encode_field(:x, "asn", type: :nominal, sort: [count: :desc], title: "ASN", axis: [label_angle: -45])
|> VegaLite.encode_field(:y, "count", type: :quantitative, aggregate: :sum, title: "Count of network assignments")
|> VegaLite.encode_field(:color, "assignment", scale: [scheme: "paired"])

Reassignment Diversity

Are networks always reassigned to a new AS? Or are they transferred back and forth between a few (perhaps linked) ASNs?

a diff looks at the change from one ASmap to another, but not over time. We want to have a map of changes of a given network through time, i.e. the whole chain of transfers, which the DB does not make explicit.

result2 =
  Exqlite.query!(
    conn,
    ~S"""
    SELECT current_epoch, network, previous_asn, current_asn
    FROM diffs
    WHERE network IN (
        SELECT network
        FROM diffs
        WHERE previous_asn is not NULL and current_asn is not NULL
        GROUP BY network
        ORDER BY COUNT(*) DESC
        LIMIT 100
    )
    LIMIT 2000;
    """,
    []
  )

transfers = result2.rows 
  |> Enum.reduce(%{}, fn [epoch, network, prev_asn, current_asn], acc ->
    dt = DateTime.from_unix!(epoch)
    Map.update(acc, network, [{dt, current_asn || prev_asn}], fn existing ->
      cond do
        current_asn == nil -> existing
        # latest entry == prev
        current_asn == Enum.at(existing, -1) |> elem(1) -> existing
        # default
        true -> existing ++ [{dt, current_asn}]
      end
    end)
  end)

transfers_agg = Enum.map(transfers, fn {_k, v} ->
  total = Enum.count(v)
  uniq = Enum.map(v, fn {_, asn} -> asn end) |> Enum.uniq() |> Enum.count()
  %{total: total, uniq: uniq}
  end)
|> Enum.sort(&(&1.total >= &2.total))

VegaLite.new(width: 600, height: 400, title: "Unique ASNs by Count of Reassignments")
|> VegaLite.data_from_values(transfers_agg, only: ["total", "uniq"])
|> VegaLite.mark(:point, filled: true)
|> VegaLite.encode_field(:x, "total", type: :quantitative, title: "Total Reassignments")
|> VegaLite.encode_field(:y, "uniq", type: :quantitative, title: "Unique ASNs in Reassignments")