Powered by AppSignal & Oban Pro
Would you like to see your link here? Contact us
Notesclub
Sign in with GitHub
created by hec & contributors
terms privacy

Traffic Splitting Demo

traffic_splitting.livemd

avatar
Jason Schmidt
avatar
avatar
More notebooks
0

Traffic Splitting Demo

Scenario

This notebook will help us explore the NGINX ngx_http_split_clients_module to understand how its applications and configuration options.

Imagine a situation where we have a very old application that sits outside our main architecture which we are going to try to replace with a totally new application that has been rewritten to our current standards but has the same interface:

graph TD;
  subgraph legacy infrastructure
    FRONTEND
    backend01
    backend02
  end
  subgraph new infrastructure
    backend03-canary
    DOWNSTREAM_SERVICE-->FRONTEND
  end

  FRONTEND-- * -->backend01;
  FRONTEND-- * -->backend02;
  FRONTEND-- 20% -->backend03-canary;

We’d like to start routing a small amount of traffic to this new service as we monitor it carefully for scaling issues and bugs.

The easiest method is to leverage the existing NGINX reverse proxy (FRONTEND in the diagram above) that sits in front of the legacy service.

Next Steps

  1. Run the cells one by one in the “Setup” section. These just set up behind the scenes code for tracking request distribution
  2. Use the cells in the “Visualizing Traffic Flow” section to run requests and see how they are distributed.

Setup

The following installs libraries we’ll need to show the data. It will take a bit to run the first time, but subsequent runs should be faster.

For each cell, click “evaluate” which will appear as you hover over the cell on the upper left. Wait until the evaluation has completed before moving on to the next cell.

Mix.install([
  {:smart_cell_command, path: "/data/smart_cell_command"},
  {:jason, "~> 1.4"},
  {:vega_lite, "~> 0.1"},
  {:kino_vega_lite, "~> 0.1"},
  {:dns, "~> 2.4"},
  {:smart_cell_file_editor, path: "/data/smart_cell_file_editor"},
  {:nginx_livebook_utils, path: "/data/nginx_livebook_utils"}
])
"apt-get update"
|> String.split("\n")
|> Enum.map(fn line ->
  [cmd | args] = line |> String.split(" ")
  {result, _} = System.cmd(cmd, args)
  result |> String.trim()
end)
|> Enum.join("\n\n")
|> IO.puts()
"apt-get install -y curl"
|> String.split("\n")
|> Enum.map(fn line ->
  [cmd | args] = line |> String.split(" ")
  {result, _} = System.cmd(cmd, args)
  result |> String.trim()
end)
|> Enum.join("\n\n")
|> IO.puts()

Data Tracking Setup

Next we set up two things:

  1. An in-memory store to keep track of how many requests are routed to each backend
  2. A simple UDP server to consume output from a logspout container that will collect logs from the demo containers.

You don’t need to understand any of this code.

The following cell determines which IP addresses correspond to our named backends in order to make the diagrams easier to read.

alias NginxLivebookUtils.{TrafficCounter, UdpLogParser}

# determines which IP addresses correspond to our named
# backends in order to make the diagrams easier to read
ip_to_name_mapping =
  ["backend01", "backend02", "backend03"]
  |> Enum.reduce(%{}, fn be, acc ->
    case DNS.resolve(be) do
      {:ok, [ip]} ->
        upstream_ip =
          Tuple.to_list(ip)
          |> Enum.join(".")

        Map.put(acc, "#{upstream_ip}:80", be)

      _err ->
        acc
    end
  end)

# Give the stats counter the name to ip mappings with which to work
TrafficCounter.set_id_mappings(ip_to_name_mapping)

# Parse out log entries we care about and increment the traffic counter
UdpLogParser.set_packet_handler(fn log_message ->
  case Regex.named_captures(~r/(?\{.+\})/, log_message) do
    %{"json" => json} ->
      {:ok, %{"upstream_addr" => upstream_addr}} = Jason.decode(json)
      TrafficCounter.increment(upstream_addr)

    _ ->
      :ok
  end
end)

Visualize Traffic Split

Validating the initial split of 20%

Look at the nginx configuration below and confirm that we have a setting that looks like this:

    split_clients "${time_iso8601}" $backend_key {
        20.0%   "backend_preprod";
        *       "backend_prod";
    }

Above the server directive.

"\nuser  nginx;\nworker_processes  auto;\n\nerror_log  /var/log/nginx/error.log notice;\npid        /var/run/nginx.pid;\n\n\nevents {\n    worker_connections  1024;\n}\n\n\nhttp {\n    include       /etc/nginx/mime.types;\n    default_type  application/octet-stream;\n\n    log_format main3 escape=json '{'\n    '\"remote_addr\":\"$remote_addr\",'\n    '\"time_iso8601\":\"$time_iso8601\",'\n    '\"request_uri\":\"$request_uri\",'\n    '\"request_length\":\"$request_length\",'\n    '\"request_method\":\"$request_method\",'\n    '\"request_time\":\"$request_time\",'\n    '\"server_port\":\"$server_port\",'\n    '\"server_protocol\":\"$server_protocol\",'\n    '\"ssl_protocol\":\"$ssl_protocol\",'\n    '\"status\":\"$status\",'\n    '\"bytes_sent\":\"$bytes_sent\",'\n    '\"http_referer\":\"$http_referer\",'\n    '\"http_user_agent\":\"$http_user_agent\",'\n    '\"upstream_response_time\":\"$upstream_response_time\",'\n    '\"upstream_addr\":\"$upstream_addr\",'\n    '\"upstream_connect_time\":\"$upstream_connect_time\",'\n    '\"upstream_cache_status\":\"$upstream_cache_status\",'\n    '\"tcpinfo_rtt\":\"$tcpinfo_rtt\",'\n    '\"tcpinfo_rttvar\":\"$tcpinfo_rttvar\"'\n    '}';\n\n\n\n    access_log  /var/log/nginx/access.log  main3;\n\n    sendfile        on;\n    #tcp_nopush     on;\n\n    keepalive_timeout  65;\n\n\n    upstream backend_prod {\n        zone backend_prod 64k;\n        server backend01:80;\n        server backend02:80;\n    }\n\n    upstream backend_preprod {\n        zone backend_preprod 64k;\n        server backend03:80;\n    }\n\n    split_clients \"${time_iso8601}\" $backend_key {\n        20.0%   \"backend_preprod\";\n        *       \"backend_prod\";\n    }\n\n    server {\n        listen 80 default_server;\n        server_name $hostname;\n\n        location / {\n            proxy_pass    http://$backend_key;\n        }\n}\n\n}\n"
|> IO.puts()

The following cell will send a request to our NGINX frontend. Run this many times, then scroll down to the visualizations to see how traffic is being distributed among the backends

"curl -s -o /dev/null frontend:80"
|> String.split("\n")
|> Enum.map(fn line ->
  [cmd | args] = line |> String.split(" ")
  {result, _} = System.cmd(cmd, args)
  result |> String.trim()
end)
|> Enum.join("\n\n")
|> IO.puts()

Visualizing Traffic

# Pull the traffic stats and extract the backends
text =
  NginxLivebookUtils.TrafficCounter.raw_stats()
  # |> Map.take(["backend01", "backend02", "backend03"])
  |> Enum.reduce("", fn {backend_name, call_count}, acc ->
    acc <> "FRONTEND-- #{call_count} -->#{backend_name};\n"
  end)

Kino.Markdown.new(~s"""
```mermaid
graph TD;
  subgraph legacy infrastructure
    FRONTEND
    backend01
    backend02
  end
  subgraph new infrastructure
    backend03
    DOWNSTREAM_SERVICE-->FRONTEND
  end

  #{text}
```
""")

Next, run the following command to reload the NGINX server

"curl -X POST --unix-socket /var/run/docker.sock  http://v1.24/containers/frontend/restart"
|> String.split("\n")
|> Enum.map(fn line ->
  [cmd | args] = line |> String.split(" ")
  {result, _} = System.cmd(cmd, args)
  result |> String.trim()
end)
|> Enum.join("\n\n")
|> IO.puts()

And clear out the traffic numbers so we can see the difference

NginxLivebookUtils.TrafficCounter.clear()
Back