Observability in Practice

Mix.install([
  {:langchain, "~> 0.8"},
  # Optional — only needed for the OpenTelemetry integration in the later sections
  {:opentelemetry_api, "~> 1.4"},
  {:opentelemetry, "~> 1.5"},
  {:kino, "~> 0.12.0"}
])

Section

This notebook is interactively available as a Livebook named notebooks/observability_tutorial.livemd. It’s the hands-on companion to the Observability guide.

LangChain gives you two layers of observability:

1. LangChain.Telemetry — vendor-neutral :telemetry events for every LLM call, chain run, and tool call. No extra dependencies.
2. LangChain.OpenTelemetry — an optional integration that turns those events into OpenTelemetry spans and metrics following a subset of the GenAI Semantic Conventions.

We’ll start with the raw telemetry layer (which needs nothing but LangChain), then attach the OpenTelemetry integration.

For a conceptual reference, see the Observability guide.

NOTE: This assumes your OPENAI_API_KEY is available as a Livebook secret.

Application.put_env(:langchain, :openai_key, System.fetch_env!("LB_OPENAI_API_KEY"))

Part 1 — Watching the raw telemetry events

Every LLM call emits a [:langchain, :llm, :call, :stop] event when it completes. Let’s attach a handler that prints the provider, model, duration, and token usage. Notice the metadata carries a :provider and a %TokenUsage{} — no content, by design.

:telemetry.attach(
  "demo-llm-stop",
  [:langchain, :llm, :call, :stop],
  fn _event, measurements, metadata, _config ->
    ms = System.convert_time_unit(measurements.duration, :native, :millisecond)

    IO.puts("""
    ── LLM call finished ─────────────────────────────
      provider:  #{metadata.provider}
      model:     #{metadata.model}
      duration:  #{ms} ms
      tokens:    #{inspect(metadata[:token_usage])}
    ──────────────────────────────────────────────────
    """)
  end,
  nil
)

Now run a simple chain. When the call completes, the handler above fires.

alias LangChain.Chains.LLMChain
alias LangChain.ChatModels.ChatOpenAI
alias LangChain.Message
alias LangChain.Message.ContentPart

{:ok, updated_chain} =
  %{llm: ChatOpenAI.new!(%{model: "gpt-4o"})}
  |> LLMChain.new!()
  |> LLMChain.add_message(Message.new_user!("In one sentence, what is Elixir?"))
  |> LLMChain.run()

ContentPart.content_to_string(updated_chain.last_message.content)

Chain-level events aggregate token usage across all assistant messages in the run — important for multi-turn or tool-calling chains where a single “last message” would undercount. Let’s watch the [:langchain, :chain, :execute, :stop] event too.

:telemetry.attach(
  "demo-chain-stop",
  [:langchain, :chain, :execute, :stop],
  fn _event, measurements, metadata, _config ->
    ms = System.convert_time_unit(measurements.duration, :native, :millisecond)

    IO.puts(
      "Chain '#{metadata.chain_type}' finished in #{ms} ms " <>
        "(#{metadata.tools_count} tools) — total usage: #{inspect(metadata[:token_usage])}"
    )
  end,
  nil
)

Migration note: the chain metadata key is :tools_count (matching :message_count). Earlier versions used :tool_count (singular).

When you’re done experimenting, detach the demo handlers:

:telemetry.detach("demo-llm-stop")
:telemetry.detach("demo-chain-stop")

Part 2 — Turning events into OpenTelemetry spans

The OpenTelemetry integration is opt-in. Its dependencies are optional; we included :opentelemetry_api and :opentelemetry in the Mix.install/1 cell above.

For this notebook we’ll configure the SDK to log spans to the console so you can see them inline. In a real app you’d point :opentelemetry_exporter at an OTLP collector or Langfuse instead (shown in Part 3).

# Register a simple exporter that prints finished spans. In production you would
# instead configure :opentelemetry_exporter with an OTLP endpoint.
:otel_batch_processor.set_exporter(:otel_exporter_stdout, [])
:ok

Attach LangChain’s OpenTelemetry handlers. setup/1 wires the span and metrics handlers to the telemetry events. Message content capture is off by default — we leave it off here.

LangChain.OpenTelemetry.setup()

Now run a chain. Behind the scenes you’ll get a chat gpt-4o span (kind :client) nested under an invoke_agent llm_chain span (kind :internal), each carrying gen_ai.* attributes like gen_ai.request.model, gen_ai.provider.name, and token counts.

{:ok, otel_chain} =
  %{llm: ChatOpenAI.new!(%{model: "gpt-4o"})}
  |> LLMChain.new!()
  |> LLMChain.add_message(Message.new_user!("Name three functional programming languages."))
  |> LLMChain.run()

ContentPart.content_to_string(otel_chain.last_message.content)

Opting into content capture (contains PII)

If your tracing backend and data policy allow it, you can record the actual messages. These flags are off by default because they can carry sensitive data. Re-run setup/1 with them enabled:

# Detach first so we don't double-attach handlers, then re-attach with capture on.
LangChain.OpenTelemetry.teardown()

LangChain.OpenTelemetry.setup(
  capture_input_messages: true,
  capture_output_messages: true
)

Now spans will include gen_ai.input.messages / gen_ai.output.messages as JSON attributes.

Suppressing traces for utility chains

Some sub-chains — a title generator, a translation pass — are noise in your traces. Wrap them in without_tracing/1 and nothing inside is exported:

LangChain.OpenTelemetry.without_tracing(fn ->
  {:ok, quiet_chain} =
    %{llm: ChatOpenAI.new!(%{model: "gpt-4o-mini"})}
    |> LLMChain.new!()
    |> LLMChain.add_message(Message.new_user!("Say hi."))
    |> LLMChain.run()

  ContentPart.content_to_string(quiet_chain.last_message.content)
end)

Part 3 — Exporting to Langfuse (or any OTLP backend)

In a real application you don’t print spans — you export them. Langfuse ingests OpenTelemetry traces via an OTLP endpoint. In config/runtime.exs:

config :opentelemetry_exporter,
  otlp_protocol: :http_protobuf,
  otlp_endpoint: "https://your-langfuse-host/api/public/otel",
  otlp_headers: [
    {"Authorization", "Basic " <> Base.encode64("pk-lf-...:sk-lf-...")}
  ]

You can enrich the root chain span with Langfuse trace attributes (user, session, tags, metadata) via the chain’s custom_context. The langfuse_* keys are recognized by the tracing layer and mapped onto the span:

langfuse_chain =
  %{llm: ChatOpenAI.new!(%{model: "gpt-4o"})}
  |> LLMChain.new!()
  |> Map.put(:custom_context, %{
    langfuse_user_id: "user-123",
    langfuse_session_id: "session-abc",
    langfuse_tags: ["notebook", "demo"],
    langfuse_metadata: %{env: "dev", feature: "observability-demo"}
  })
  |> LLMChain.add_message(Message.new_user!("What's a trace in observability?"))

# Running this exports a trace tagged with the Langfuse attributes above.
# (Requires the exporter configured against a real Langfuse instance.)
langfuse_chain

custom_context is your own data map — it’s also passed to your tool functions and surfaced on tool/chain telemetry events. The langfuse_* names are simply reserved keys the tracing layer looks for; mix them with your application’s own context freely.

Part 4 — Metrics

setup/1 enables metrics by default, but there’s an important nuance: LangChain re-emits intermediary :telemetry events rather than recording OTel histograms directly. You attach a metrics library to turn them into real metrics.

The two events are:

• [:langchain, :otel, :operation, :duration] — %{duration_s: float()}
• [:langchain, :otel, :token, :usage] — %{tokens: integer()}, tagged with gen_ai.token.type

Here’s a quick handler that observes them directly (in production you’d wire these into Telemetry.Metrics + an OTel/Prometheus reporter):

:telemetry.attach_many(
  "demo-otel-metrics",
  [
    [:langchain, :otel, :operation, :duration],
    [:langchain, :otel, :token, :usage]
  ],
  fn event, measurements, metadata, _ ->
    IO.puts("metric #{inspect(event)} → #{inspect(measurements)} #{inspect(metadata)}")
  end,
  nil
)

{:ok, metrics_chain} =
  %{llm: ChatOpenAI.new!(%{model: "gpt-4o-mini"})}
  |> LLMChain.new!()
  |> LLMChain.add_message(Message.new_user!("Count to three."))
  |> LLMChain.run()

ContentPart.content_to_string(metrics_chain.last_message.content)

Clean up when you’re finished:

:telemetry.detach("demo-otel-metrics")
LangChain.OpenTelemetry.teardown()

Where to go next

• LangChain.Telemetry — the full event catalog and metadata shapes
• LangChain.OpenTelemetry — setup/1, teardown/0, without_tracing/1
• LangChain.OpenTelemetry.Config — every capture flag
• Observability guide — the conceptual reference behind this notebook