Powered by AppSignal & Oban Pro

Choreo Infrastructure: Cloud Network Topology Walkthrough

infrastructure_topology_walkthrough.livemd

Choreo Infrastructure: Cloud Network Topology Walkthrough

Mix.install([
  {:choreo, "~> 0.14.1"},
  # {:choreo, path: Path.expand("../..", __DIR__), force: true},
  {:kino_vizjs, "~> 0.9.0"}
])

Section

Rendering diagrams: This livebook uses Kino.VizJS to render DOT diagrams inline. Mermaid output can be rendered with Kino.Mermaid.new/1, which is supported natively in Livebook.


What is Choreo.Infrastructure?

Choreo.Infrastructure is a cloud network topology preset built on top of Choreo's existing graph and rendering stack. It provides a domain vocabulary for modelling VPCs, subnets, compute instances, databases, and load balancers — with structural validation rules to catch common networking mistakes before they reach production.

It is not a separate rendering engine: node shapes and colors are resolved through Choreo.Theme (just like Choreo), and the rendering pipeline is the same Yog-based stack. What it adds is:

  • Typed cluster boundaries — VPCs and subnets carry security semantics (public vs. private).
  • Choreo.Infrastructure.Analysis — audit rules that operate on those semantics.
  • Network vocabulary — intent-revealing builders (add_vpc, add_compute, etc.).

Node Types

Builder Node Type DOT Shape Mermaid Shape Purpose
add_internet/3 :internet ☁️ cloud Circle Public internet gateway
add_load_balancer/3 :load_balancer ▽ invhouse Hexagon ALB, NLB, Nginx, HAProxy
add_compute/3 :compute 📦 box3d Subroutine EC2, ECS task, Kubernetes pod
add_managed_db/3 :managed_db 🛢️ cylinder Cylinder RDS, Aurora, Cloud SQL
add_storage/3 :storage 📁 folder Rounded rect S3, EFS, GCS bucket

Cluster (Boundary) Types

Builder Cluster Type Style Meaning
add_vpc/3 :vpc Dashed Virtual Private Cloud — the outer envelope
add_subnet_public/3 :subnet_public Rounded Internet-facing zone (DMZ, load balancers)
add_subnet_private/3 :subnet_private Rounded Isolated zone (app servers, databases)

API Approaches: Programmatic Pipe API vs Lab DSL

Choreo provides two complementary approaches to modeling cloud infrastructure:

  1. Programmatic Pipe API (Choreo.Infrastructure): The canonical, explicit interface (Infrastructure.new() |> Infrastructure.add_vpc(...) |> Infrastructure.add_compute(...) |> Infrastructure.connect(...)). Ideal for automated infrastructure discovery, configuration ingestion, and strict programmatic pipelines.
  2. Lab DSL (Choreo.Lab.DSL.Infrastructure): An expressive sketch syntax using infrastructure do ... end, semantic node constructors (compute, service, managed_db, load_balancer), nested cluster blocks (vpc "Name" do ... end, public_subnet "Name" do ... end), direct connections (~>), and protocol modifiers (|> on("HTTPS", protocol: :https)). Ideal for Livebooks, design reviews, and rapid cloud topology sketching.

The Legend in Step 1 demonstrates the canonical programmatic pipe API, while subsequent steps showcase the concise Lab DSL.

import Choreo.Lab.DSL.Infrastructure
alias Choreo.Infrastructure
alias Choreo.Infrastructure.Analysis

Step 1: Minimal topology — a node type legend

legend =
  Infrastructure.new()
  |> Infrastructure.add_internet(:gw, label: "Internet")
  |> Infrastructure.add_load_balancer(:lb, label: "Load Balancer")
  |> Infrastructure.add_compute(:app, label: "Compute")
  |> Infrastructure.add_managed_db(:db, label: "Managed DB")
  |> Infrastructure.add_storage(:store, label: "Storage")
  |> Infrastructure.connect(:gw, :lb)
  |> Infrastructure.connect(:lb, :app)
  |> Infrastructure.connect(:app, :db)
  |> Infrastructure.connect(:app, :store)

Kino.Layout.tabs(
  Siren: Choreo.Lab.Siren.new(Infrastructure.to_mermaid(legend)),
  Graphviz: Kino.VizJS.render(Infrastructure.to_dot(legend), height: "500px"),
  Sketch: Choreo.Lab.Sketch.new(Infrastructure.to_mermaid(legend))
)

Step 2: A realistic three-tier web application

We model a production VPC with:

  • A public subnet (DMZ) holding the load balancer
  • A private subnet (App) holding compute and the database
  • A storage bucket outside the VPC (managed S3-like service)
prod =
  infrastructure do
    internet = internet("Internet")

    vpc "vpc_prod", label: "Production VPC" do
      public_subnet "subnet_dmz", label: "Public Subnet (DMZ)" do
        alb = load_balancer("Application LB")
      end

      private_subnet "subnet_app", label: "Private Subnet (App)" do
        api = service("API Service")
        worker = compute("Background Worker")
        rds = managed_db("Postgres RDS")
      end
    end

    s3 = storage("Object Store (S3)")

    internet ~> alb |> on("HTTPS", protocol: :https)
    alb ~> api |> on(protocol: :http)
    api ~> rds |> on(protocol: :tcp)
    api ~> worker |> on("Jobs", protocol: :amqp)
    api ~> s3 |> on(protocol: :https)
    worker ~> rds |> on(protocol: :tcp)
    worker ~> s3 |> on(protocol: :https)
  end

Kino.Layout.tabs(
  Siren: Choreo.Lab.Siren.new(Infrastructure.to_mermaid(prod)),
  Graphviz: Kino.VizJS.render(Infrastructure.to_dot(prod), height: "800px"),
  Sketch: Choreo.Lab.Sketch.new(Infrastructure.to_mermaid(prod))
)

Step 3: Security analysis — Choreo.Infrastructure.Analysis

The analysis module runs structural audit rules against your topology. Analysis.validate/1 returns a list of {severity, message} tuples where severity is :error or :warning.

warnings = Analysis.validate(prod)

if warnings == [] do
  IO.puts("✅ No security warnings — topology looks clean.")
else
  Enum.each(warnings, fn {severity, message} ->
    icon = if severity == :error, do: "❌", else: "⚠️"
    IO.puts("#{icon} #{message}")
  end)
end

Intentionally broken topology — triggering all audit rules

Now let's build a deliberately misconfigured topology to see the audit rules fire:

broken =
  infrastructure do
    internet = internet("Internet")

    vpc "vpc", label: "VPC" do
      public_subnet "pub", label: "Public Subnet" do
        # ❌ Database placed in the public subnet (should be private)
        db = managed_db("DB")
        # ❌ Storage placed in the public subnet
        store = storage("Store")
      end

      private_subnet "priv", label: "Private Subnet" do
        # ❌ Load balancer placed in the private subnet (should be in public/DMZ)
        lb = load_balancer("LB")
        app = compute("App")
      end
    end

    # ❌ Compute node without subnet assignment
    worker = compute("Worker")

    # ❌ Direct internet → private subnet connection (bypasses DMZ)
    internet ~> app |> on(protocol: :https)
    lb ~> app
    app ~> db
  end

Analysis.validate(broken)
|> Enum.each(fn 
  {:error, message} -> IO.puts("❌ #{message}")
  {:warning, message} -> IO.puts("⚠️ #{message}")
end)

You can also inspect specific violations programmatically:

IO.inspect(Analysis.direct_internet_violations(broken), label: "Direct internet violations")
IO.inspect(Analysis.misplaced_databases(broken), label: "Misplaced databases")
IO.inspect(Analysis.misplaced_storage(broken), label: "Misplaced storage")
IO.inspect(Analysis.misplaced_load_balancers(broken), label: "Misplaced load balancers")
IO.inspect(Analysis.unassigned_compute(broken), label: "Unassigned compute")

The five audit rules explained

Each violation message describes the problem:

Violation detected Severity Example message
Direct internet → private subnet :error "Private resource 'app' is connected directly to public internet boundary 'internet'."
Database not in private subnet :error "Managed database 'db' should be located in a private subnet, but it is in 'Public Subnet'."
Storage in public subnet :error "Storage 'store' should not be in a public subnet, but it is in 'Public Subnet'."
Load balancer not in public subnet :warning "Load balancer 'lb' should be located in a public subnet, but it is in 'Private Subnet'."
Compute node unassigned to subnet :warning "Compute node 'worker' is not assigned to any subnet. This may indicate incomplete modeling."

Step 4: Themes

All six Choreo.Theme presets work out of the box with Choreo.Infrastructure because :internet, :compute, and :managed_db are now first-class types in the theme system.

# Build a compact topology for comparing themes
demo =
  infrastructure do
    gw = internet("Internet")

    vpc "vpc", label: "VPC" do
      public_subnet "pub" do
        lb = load_balancer("ALB")
      end

      private_subnet "priv" do
        app = compute("API")
        db = managed_db("RDS")
      end
    end

    gw ~> lb |> on(protocol: :https)
    lb ~> app |> on(protocol: :http)
    app ~> db |> on(protocol: :tcp)
  end

Kino.Layout.tabs(
  Default: Kino.Mermaid.new(Infrastructure.to_mermaid(demo)),
  Architecture: Kino.Mermaid.new(Infrastructure.to_mermaid(demo, syntax: :architecture)),
  Warm: Kino.Mermaid.new(Infrastructure.to_mermaid(demo, theme: :warm)),
  Ocean: Kino.Mermaid.new(Infrastructure.to_mermaid(demo, theme: :ocean)),
  Forest: Kino.Mermaid.new(Infrastructure.to_mermaid(demo, theme: :forest))
)

Step 5: Multi-region topology

A more complex deployment: two availability zones with shared storage and a global load balancer.

multi_region =
  infrastructure do
    internet = internet("Internet")
    # Global load balancer (outside VPC)
    global_lb = load_balancer("Global LB / CDN")

    # VPC
    vpc "vpc", label: "AWS VPC (us-east-1)" do
      # AZ-1
      public_subnet "az1_pub", label: "AZ-1 Public" do
        alb1 = load_balancer("ALB (AZ-1)")
      end

      private_subnet "az1_priv", label: "AZ-1 Private" do
        api1 = compute("API (AZ-1)")
        primary_db = managed_db("Primary RDS")
      end

      # AZ-2
      public_subnet "az2_pub", label: "AZ-2 Public" do
        alb2 = load_balancer("ALB (AZ-2)")
      end

      private_subnet "az2_priv", label: "AZ-2 Private" do
        api2 = compute("API (AZ-2)")
        replica_db = managed_db("Read Replica RDS")
      end
    end

    # Shared storage (outside any subnet — managed service)
    s3 = storage("S3 (shared)")

    # Connections
    internet ~> global_lb |> on(protocol: :https)
    global_lb ~> alb1 |> on(protocol: :https)
    global_lb ~> alb2 |> on(protocol: :https)
    alb1 ~> api1 |> on(protocol: :http)
    alb2 ~> api2 |> on(protocol: :http)
    api1 ~> primary_db |> on(protocol: :tcp)
    api2 ~> replica_db |> on(protocol: :tcp)
    # Replication stream
    primary_db ~> replica_db |> on("Replication", protocol: :tcp)
    api1 ~> s3 |> on(protocol: :https)
    api2 ~> s3 |> on(protocol: :https)
  end

Kino.Layout.tabs(
  Siren: Choreo.Lab.Siren.new(Infrastructure.to_mermaid(multi_region)),
  Architecture: Choreo.Lab.Siren.new(Infrastructure.to_mermaid(multi_region, syntax: :architecture)),
  Graphviz: Kino.VizJS.render(Infrastructure.to_dot(multi_region)),
  Sketch: Choreo.Lab.Sketch.new(Infrastructure.to_mermaid(multi_region))
)
# Confirm no unexpected security errors on the multi-region design (only expected warning about global LB outside subnet)
[{:warning, message}] = Analysis.validate(multi_region)

IO.puts("⚠️  #{message}")

Step 6: Protocol-aware edge styling

connect/3 accepts a :protocol key that drives edge color in the DOT output:

  • :https / :ssl → green (#10b981) — secure traffic
  • everything else → grey (#64748b) — internal traffic
proto_demo =
  infrastructure do
    inet = internet("Internet")
    lb = load_balancer("LB")
    api = compute("API")
    db = managed_db("DB")
    store = storage("Store")

    inet ~> lb |> on("TLS", protocol: :https)
    lb ~> api |> on("Plaintext", protocol: :http)
    api ~> db |> on("TCP/5432", protocol: :tcp)
    api ~> store |> on("TLS", protocol: :https)
  end

Kino.Layout.tabs(
  Siren: Choreo.Lab.Siren.new(Infrastructure.to_mermaid(proto_demo)),
  Graphviz: Kino.VizJS.render(Infrastructure.to_dot(proto_demo)),
  Sketch: Choreo.Lab.Sketch.new(Infrastructure.to_mermaid(proto_demo))
)

Step 7: Choreo.View lens operations

Choreo.Infrastructure implements Choreo.Viewable, so the full View API works — focus, zoom, filter, and collapse.

alias Choreo.View

# Build a full topology to slice through
full =
  infrastructure do
    inet = internet("Internet")
    lb = load_balancer("Load Balancer")
    api = compute("API")
    worker = compute("Worker")
    db = managed_db("DB")
    store = storage("Store")

    inet ~> lb |> on(protocol: :https)
    lb ~> api |> on(protocol: :http)
    api ~> worker
    api ~> db |> on(protocol: :tcp)
    api ~> store |> on(protocol: :https)
    worker ~> db |> on(protocol: :tcp)
  end

Focus: show API and its immediate 1-hop neighbourhood

focused = full |> View.focus(:api, radius: 1)
focused_mermaid = focused |> Infrastructure.to_mermaid()

Kino.Layout.tabs(
  Siren: Choreo.Lab.Siren.new(focused_mermaid),
  Graphviz: Kino.VizJS.render(focused |> Choreo.to_dot()),
  Sketch: Choreo.Lab.Sketch.new(focused_mermaid)
)

Filter: show only compute nodes (and edges between them)

compute = View.filter(full, fn _, data -> data[:node_type] == :compute end)
compute_mermaid = Infrastructure.to_mermaid(compute)

Kino.Layout.tabs(
  Siren: Choreo.Lab.Siren.new(compute_mermaid),
  Graphviz: Kino.VizJS.render(compute |> Choreo.to_dot()),
  Sketch: Choreo.Lab.Sketch.new(compute_mermaid)
)

Cheat Sheet

Lab DSL Syntax (Choreo.Lab.DSL.Infrastructure)

Syntax Description
infrastructure do ... end Define an infrastructure diagram block
gw = internet("Label") Public internet entrypoint / gateway
lb = load_balancer("Label") / gateway("...") / lb("...") Load balancer proxy node
app = compute("Label") / service("...") Compute workload runner or application service
db = managed_db("Label") / database("...") / db("...") Managed database or stateful storage
store = storage("Label") / object_store("...") Object or blob storage node
q = queue("Label") / cache("Label") Message queue or cache node
vpc "Name", label: "..." do ... end VPC cluster block with automatic subnet/node nesting
public_subnet "Name" do ... end / subnet_public Public subnet (DMZ) cluster block
private_subnet "Name" do ... end / subnet_private Private subnet cluster block
cluster "Name" do ... end Generic boundary cluster block
a ~> b Unlabeled network connection
`a ~> b > on("Label")/label("Label")`
`a ~> b > on("Label", protocol: :https)`
`a ~> b > protocol(:tcp)/cost(2)`
edge a ~> b, "Label", protocol: :https Explicit edge statement with label and options
edge a ~> b, protocol: :http Explicit edge statement with options

Programmatic Pipe API & Analysis (Choreo.Infrastructure)

Task / Feature Command
Create Topology Infrastructure.new/1
Add Boundaries Infrastructure.add_vpc/3, add_subnet_public/3, add_subnet_private/3, add_cluster/3
Add Nodes Infrastructure.add_internet/3, add_load_balancer/3, add_compute/3, add_managed_db/3, add_storage/3
Connect Nodes Infrastructure.connect/4 (Opts: :protocol, :label, :cost, :type)
Query Nodes / Edges Infrastructure.nodes/1, Infrastructure.edges/1, Infrastructure.edges_with_meta/1
Query Clusters Infrastructure.clusters/1
Graph Conversion Infrastructure.to_simple_graph/2, Infrastructure.to_graph/1, Infrastructure.to_choreo/1
Security Audit Infrastructure.Analysis.validate/1 (Flags 5 rules: direct internet, DB/storage/LB placement, compute assignment)
Programmatic Queries Analysis.direct_internet_violations/1, misplaced_databases/1, misplaced_storage/1
Placement Queries Analysis.misplaced_load_balancers/1, Analysis.unassigned_compute/1, Analysis.isolated_nodes/1
Render Mermaid Infrastructure.to_mermaid/2 (Opts: :syntax, :theme, :direction, :highlighted_nodes)
Render DOT Graphviz Infrastructure.to_dot/2 (Opts: :theme, :highlighted_nodes, :highlighted_edges)
Themes Infrastructure.theme/2 (:default, :dark, :minimal, :warm, :forest, :ocean)
Lens Views Choreo.View.focus/3, Choreo.View.filter/3, Choreo.View.zoom/2, Choreo.View.collapse/4

Summary

Feature API
Create topology Infrastructure.new/0
Add boundary add_vpc/3, add_subnet_public/3, add_subnet_private/3, add_cluster/3
Add nodes add_internet/3, add_load_balancer/3, add_compute/3, add_managed_db/3, add_storage/3
Add edge connect/4 — options: :protocol, :label, :cost, :type
Query edges & meta Infrastructure.edges_with_meta/1, Infrastructure.clusters/1
Audit topology Analysis.validate/1
Audit queries Analysis.direct_internet_violations/1, misplaced_databases/1, misplaced_storage/1, etc.
Render DOT Infrastructure.to_dot/2 — option: theme:, highlighted_nodes:, highlighted_edges:
Render Mermaid Infrastructure.to_mermaid/2 — option: syntax:, theme:, direction:
Lens ops Choreo.View.focus/3, filter/3, zoom/2, collapse/4