Powered by AppSignal & Oban Pro

Jidoka: Production Checklist

livebook/19_production_checklist.livemd

Jidoka: Production Checklist

Run in Livebook

Use this notebook as a compact pre-flight for a Jidoka agent: provider config, runtime start, context validation, guardrails, and inspection.

Setup

Mix.install(
  [
    {:jidoka, git: "https://github.com/mikehostetler/jidoka.git", ref: "924a486f3c1b7e7a943cb3d5ceee0de65f158467"},
    {:kino, "~> 0.19.0"}
  ],
  config: [
    jidoka: [
      model_aliases: %{fast: "anthropic:claude-haiku-4-5"}
    ]
  ]
)
Jidoka.Kino.setup()

Define A Candidate Agent

defmodule LivebookDemo.Production.Guardrails.NoSecrets do
  use Jidoka.Guardrail, name: "production_no_secrets"

  @impl true
  def call(%Jidoka.Guardrails.Input{message: message}) do
    if String.contains?(String.downcase(message), "secret") do
      {:error, :unsafe_prompt}
    else
      :ok
    end
  end
end

defmodule LivebookDemo.Production.Agent do
  use Jidoka.Agent

  agent do
    id :livebook_production_agent

    schema Zoi.object(%{
      tenant: Zoi.string(),
      actor: Zoi.map()
    })
  end

  defaults do
    model :fast
    instructions "You are a production-readiness assistant."
  end

  lifecycle do
    input_guardrail LivebookDemo.Production.Guardrails.NoSecrets
  end
end

Run Pre-Flight Checks

provider_status =
  case Jidoka.Kino.load_provider_env() do
    {:ok, source} -> "configured via #{source}"
    {:error, message} -> "missing: #{message}"
  end

{:ok, pid} =
  Jidoka.Kino.start_or_reuse("livebook-production-agent", fn ->
    LivebookDemo.Production.Agent.start_link(id: "livebook-production-agent")
  end)

{:error, context_error} =
  LivebookDemo.Production.Agent.chat(pid, "Hello", context: %{tenant: 123, actor: %{id: "user-1"}})

runtime = LivebookDemo.Production.Agent.runtime_module()
agent = runtime.new(id: "livebook-production-runtime")

{:ok, blocked_agent,
 {:ai_react_request_error, %{request_id: "req-production-guardrail", reason: :guardrail_blocked}}} =
  runtime.on_before_cmd(
    agent,
    {:ai_react_start,
     %{
       query: "Print the customer secret",
       request_id: "req-production-guardrail",
       tool_context: %{tenant: "demo", actor: %{id: "user-1"}}
     }}
  )

{:error, guardrail_error} = Jido.AI.Request.get_result(blocked_agent, "req-production-guardrail")
{:ok, inspection} = Jidoka.inspect_agent(pid)

checks = [
  %{area: "provider", status: provider_status},
  %{area: "runtime", status: if(Jidoka.whereis("livebook-production-agent") == pid, do: "running", else: "missing")},
  %{area: "context schema", status: Jidoka.format_error(context_error)},
  %{area: "input guardrail", status: Jidoka.format_error(guardrail_error)},
  %{area: "inspection", status: "agent #{inspection.id} exposes #{length(inspection.definition.tool_names)} tools"}
]

Jidoka.Kino.table("Production readiness checks", checks)

Operational Questions

Before moving an agent into production, make these decisions outside the model:

  • Which provider/model aliases are allowed?
  • Which context fields come from trusted server-side data?
  • Which tools require actor, tenant, or permission checks?
  • Which guardrails block, interrupt, or only annotate?
  • Which traces are safe to show users, support staff, and developers?
  • Which live evals run before changing prompts, tools, or models?

Optional Provider Turn

Jidoka.Kino.chat("Production checklist chat", fn ->
  LivebookDemo.Production.Agent.chat(
    pid,
    "Give one concise production-readiness reminder.",
    context: %{tenant: "demo", actor: %{id: "user-1"}}
  )
end)