Imp 02: Programming, Not Prompting

imp_checkout? = fn path ->
  is_binary(path) and File.regular?(Path.join(path, "mix.exs")) and
    File.regular?(Path.join(path, "lib/imp.ex"))
end

explicit_repo = System.get_env("IMP_PATH")

if explicit_repo && not imp_checkout?.(Path.expand(explicit_repo)) do
  raise "IMP_PATH does not point to an Imp source checkout or unpacked package"
end

repo =
  [explicit_repo, Path.expand("..", __DIR__), File.cwd!()]
  |> Enum.reject(&is_nil/1)
  |> Enum.map(&Path.expand/1)
  |> Enum.find(imp_checkout?)

if repo do
  # Prefer the notebook's own Imp checkout even when Livebook was launched
  # from an unrelated Mix project. Unpacked package archives may omit
  # mix.lock, so pin it only when the source checkout actually provides it.
  install_opts =
    if File.regular?(Path.join(repo, "mix.lock")),
      do: [lockfile: Path.join(repo, "mix.lock")],
      else: []

  Mix.install([{:imp, path: repo}], install_opts)
else
  # Standalone notebook: install the tagged Git source release.
  Mix.install([{:imp, github: "deepfates/imp", tag: "v0.4.0"}])
end
live_provider_enabled? = System.get_env("LIVE_PROVIDER") == "1"

The Idea

Livebook 01 showed the real-provider shape. This chapter removes provider variance so you can inspect the same Imp ideas with a deterministic local LM.

It also includes a live-provider example near the end. The point is that Imp.LM.Static is a development tool, not a different product path.

Imp-style programming means declaring the task boundary first:

The signature is the smallest useful unit in Imp. It says what data enters the task and what data must come back. It does not say which provider to use, how many examples to show, or how the final prompt should look.

signature = Imp.signature("question -> answer")
Imp.Signature.to_spec(signature)

Then you attach an LM and call a program:

This notebook uses a deterministic local LM so the behavior is stable. The LM is deliberately boring; the point is to see the Imp program shape without network calls or provider variance.

lm =
  Imp.LM.Static.new(
    handler: fn messages, _opts ->
      prompt = Enum.map_join(messages, "\n", & &1.content)
      if prompt =~ "France", do: %{answer: "Paris"}, else: %{answer: "unknown"}
    end
  )

Imp.configure(lm: lm, adapter: Imp.Adapter.Chat)

qa_program = Imp.predict("question -> answer")
{:ok, prediction} = Imp.call(qa_program, %{question: "Capital of France?"})
Imp.to_map(prediction)

Inspect The Generated Messages

Predictions keep trace metadata:

The generated messages are still available. Imp is not asking you to trust a black box; it is giving the prompt an owner, a contract, and a place in normal Elixir data.

prediction.metadata.trace.messages

The point is not to hide prompts forever. The point is to make them a generated artifact of a typed program.

Chain Of Thought

Some tasks benefit from a separate reasoning field. In Imp that is still a structured output, not an informal convention hidden in a prompt.

cot_lm =
  Imp.LM.Static.new(
    handler: fn _messages, _opts -> %{reasoning: "2 + 2 is 4.", answer: "4"} end
  )

cot_program = Imp.chain_of_thought("question -> answer", lm: cot_lm)
{:ok, pred} = Imp.call(cot_program, %{question: "What is 2+2?"})
Imp.to_map(pred)

Schema-Constrained Output

When another part of your application consumes the result, prefer constraints over hoping the model follows prose. The JSON adapter validates the parsed fields and returns feedback when output does not fit the signature.

signature =
  Imp.signature(%{
    inputs: [:text],
    outputs: [
      %{name: :sentiment, type: :string, constraints: %{enum: ["positive", "negative"]}},
      %{name: :confidence, type: :number, constraints: %{min: 0.0, max: 1.0}}
    ]
  })

{:ok, pred} =
  Imp.Adapter.JSON.parse(
    signature,
    ~s({"sentiment":"positive","confidence":0.91}),
    []
  )

Imp.to_map(pred)

Try a bad output:

Imp.Adapter.JSON.parse(signature, ~s({"sentiment":"mixed","confidence":2.0}), [])

Examples

Examples are rows of data. Marking inputs tells Imp which fields the model can see and which fields are labels for evaluation or optimization.

example =
  Imp.example(question: "2+2?", answer: "4")
  |> Imp.with_inputs(:question)

{Imp.to_map(Imp.inputs(example)), Imp.to_map(Imp.labels(example))}

Try A Live Provider

This is the same programming model against a real LM. The cell validates the typed JSON result only when LIVE_PROVIDER=1, OPENAI_API_KEY, and OPENAI_MODEL are present.

live_lm = fn opts ->
  if live_provider_enabled? && System.get_env("OPENAI_API_KEY") && System.get_env("OPENAI_MODEL") do
    {:ok,
     Imp.req_llm("openai:#{System.fetch_env!("OPENAI_MODEL")}",
       Keyword.merge(
         [
           api_key: System.fetch_env!("OPENAI_API_KEY"),
           temperature: 0,
           max_completion_tokens: 120
         ],
         opts
       )
     )}
  else
    {:skip, "Set LIVE_PROVIDER=1, OPENAI_API_KEY, and OPENAI_MODEL to run the live provider cell."}
  end
end

case live_lm.(max_completion_tokens: 120) do
  {:ok, lm} ->
    live_program =
      Imp.predict(
        Imp.signature(
          "text -> sentiment: enum[positive,negative], confidence: number",
          "Classify the text. Return JSON only."
        ),
        lm: lm,
        adapter: Imp.Adapter.JSON,
        config: [json_retries: 1]
      )

    {:ok, live_prediction} =
      Imp.call(live_program, %{text: "Imp makes LM programs testable and useful."})

    sentiment = Imp.get(live_prediction, :sentiment)
    confidence = Imp.get(live_prediction, :confidence)

    unless sentiment == "positive" and is_number(confidence) do
      raise "live provider returned an invalid typed prediction: #{inspect(Imp.to_map(live_prediction))}"
    end

    %{
      sentiment: sentiment,
      confidence: confidence,
      trace_messages: length(live_prediction.metadata.trace.messages)
    }

  skip ->
    skip
end

Save And Load

Program state can be saved without serializing provider secrets. That makes it reasonable to treat compiled programs as artifacts while rebinding live credentials at runtime.

path = Path.join(System.tmp_dir!(), "imp-livebook-program.json")
Imp.save!(qa_program, path)
loaded = Imp.load!(path)
File.rm(path)

loaded.signature |> Imp.Signature.to_spec()

Next: open livebooks/03_evaluate_and_optimize.livemd to turn examples into a metric, evaluate the baseline, and compile a better program.