Powered by AppSignal & Oban Pro
Notesclub
Sign in with GitHub
created by hec & contributors
terms privacy

OpenaiEx User Guide

notebooks/userguide.livemd

avatar
CyberChitta
avatar
avatar
Share to X
Share to Bluesky
More notebooks

OpenaiEx User Guide

Mix.install([
  # {:openai_ex, "~> 0.9.16"},
  {:openai_ex, path: Path.join(__DIR__, "..")},
  {:kino, "~> 0.17.0"}
])

Introduction

OpenaiEx is an Elixir library that provides a community-maintained OpenAI API client.

The main user guide is a livebook, so you should be able to run everything without any setup. The user guide is also the test suite. It is run before every version release, so it is always up to date with the library.

> Portions of this project were developed with assistance from ChatGPT 3.5 and 4, as well as Claude 3 Opus and Claude Sonnets 3.5, 3.6 and 3.7. However, every line of code is human curated (by me, @restlessronin 😇). AI collaboration facilitated by llm-context.

All API endpoints and features (as of Mar 16, 2025) are supported, including the most recent Responses API (proposed replacement for Chat Completion).

Configuration of Finch pools and API base url are supported.

There are some differences compared to other elixir openai wrappers.

  • I tried to faithfully mirror the naming/structure of the official python api. For example, content that is already in memory can be uploaded as part of a request, it doesn’t have to be read from a file at a local path.
  • I was developing for a livebook use-case, so I don’t have any config, only environment variables.
  • Streaming API versions, with request cancellation, are supported.
  • The underlying transport is finch, rather than httpoison
  • 3rd Party (including local) LLMs with an OpenAI proxy, as well as the Azure OpenAI API, are considered legitimate use cases.

To learn how to use OpenaiEx, you can refer to the relevant parts of the official OpenAI API reference documentation, which we link to throughout this document.

This file is an executable Livebook, which means you can interactively run and modify the code samples provided. We encourage you to open it in Livebook and try out the code for yourself!

Installation

You can install OpenaiEx using Mix:

In Livebook

Add the following code to the first connection cell:

Mix.install(
  [
    {:openai_ex, "~> 0.9.16"}
  ]
)

In a Mix Project

Add the following to your mix.exs file:

def deps do
  [
    {:openai_ex, "~> 0.9.16"}
  ]
end

Authentication

To authenticate with the OpenAI API, you will need an API key. We recommend storing your API key in an environment variable. Since we are using Livebook, we can store this and other environment variables as Livebook Hub Secrets.

apikey = System.fetch_env!("LB_OPENAI_API_KEY")
openai = OpenaiEx.new(apikey)

You can also specify an organization if you are a member of more than one:

# organization = System.fetch_env!("LB_OPENAI_ORGANIZATION")
# openai = OpenaiEx.new(apikey, organization)

For more information on authentication, see the OpenAI API Authentication reference.

Configuration

There are a few places where configuration seemed necessary.

Receive Timeout

The default receive timeout is 15 seconds. If you are seeing longer latencies, you can override the default with

# set receive timeout to 45 seconds
openai = OpenaiEx.new(apikey) |> OpenaiEx.with_receive_timeout(45_000)

Finch Instance

In production scenarios where you want to explicitly tweak the Finch pool, you can create a new Finch instance using

Finch.start_link(
    name: MyConfiguredFinch,
    pools: ...
)

You can use this instance of Finch (instead of the default OpenaiEx.Finch) by setting the finch name

openai_with_custom_finch = openai |> with_finch_name(MyConfiguredFinch)

Base Url

There are times, such as when using a local LLM (like Ollama) with an OpenAI proxy, when you need to reset the base url of the API. This is generally only applicable for chat and chat completion endpoints and can be accomplished by

# in this example, our development livebook server is running in a docker dev container while 
# the local llm is running on the host machine
proxy_openai =
  OpenaiEx.new(apikey) |> OpenaiEx.with_base_url("http://host.docker.internal:8000/v1")

Using an LLM gateway (e.g. Portkey)

LLM gateways are used to provide a virtual interface to multiple LLM providers behind a single API endpoint.

Generally they work on the basis of additional HTTP headers being added that specify the model to use, the provider to use, and possibly other parameters.

For example, to configure your client for openai using the portkey gateway, you would do this:

openai_api_key = "an-openai-api-key"
portkey_api_key = "a-portkey-api-key"

OpenaiEx.new(openai_api_key) 
  |> OpenaiEx.with_base_url("https://api.portkey.ai/v1") 
  |> OpenaiEx.with_additional_headers(%{"x-portkey-api-key"=>portkey_api_key, "x-portkey-provider"=>"openai"})

similarly, for Anthropic, you would do this:

anthropic_api_key = "some-anthropic-api-key"

OpenaiEx.new(anthropic_api_key) 
  |> OpenaiEx.with_base_url("https://api.portkey.ai/v1") 
  |> OpenaiEx.with_additional_headers(%{"x-portkey-api-key"=>portkey_api_key, "x-portkey-provider"=>"anthropic"})

Azure OpenAI

The Azure OpenAI API replicates the Completion, Chat Completion and Embeddings endpoints from OpenAI.

However, it modifies the base URL as well as the endpoint path, and adds a parameter to the URL query. These modifications are accommodated with the following calls:

for non Entra Id

openai = OpenaiEx._for_azure(azure_api_id, resource_name, deployment_id, api_version)

and for Entra Id

openai = OpenaiEx.new(entraId) |> OpenaiEx._for_azure(resource_name, deployment_id, api_version)

These methods will be supported as long as the Azure version does not deviate too far from the base OpenAI API.

Error Handling

OpenaiEx provides robust error handling to support both interactive and non-interactive usage. There are two main ways to handle errors:

Error Tuples

Most functions in OpenaiEx return :ok and :error tuples. This allows for pattern matching and explicit error handling:

case OpenaiEx.Chat.Completions.create(openai, chat_req) do
  {:ok, response} -> # Handle successful response
  {:error, error} -> # Handle error
end

Exceptions

For scenarios where you prefer exceptions, OpenaiEx provides bang (!) versions of functions that raise exceptions on errors:

try do
  response = OpenaiEx.Chat.Completions.create!(openai, chat_req)
  # Handle successful response
rescue
  e in OpenaiEx.Error -> # Handle exception
end

Error Types

OpenaiEx closely follows the error types defined in the official OpenAI Python library. For a comprehensive list and description of these error types, please refer to the OpenAI API Error Types documentation.

In addition to these standard error types, OpenaiEx defines two specific error types for handling streaming operations:

  • SSETimeoutError: Raised when a streaming response times out
  • SSECancellationError: Raised when a user initiates a stream cancellation

For more details on specific error types and their attributes, refer to the OpenaiEx.Error module documentation.

Model

List Models

To list all available models, use the Model.list() function:

alias OpenaiEx.Models

openai |> Models.list()

Retrieve Models

To retrieve information about a specific model, use the Model.retrieve() function:

openai |> Models.retrieve("gpt-4o-mini")

For more information on using models, see the OpenAI API Models reference.

Chat Completion

To generate a chat completion, you need to define a chat completion request structure using the ChatCompletion.new() function. This function takes several parameters, such as the model ID and a list of chat messages. We have a module ChatMessage which helps create messages in the chat format.

alias OpenaiEx.Chat
alias OpenaiEx.ChatMessage
alias OpenaiEx.MsgContent

chat_req =
  Chat.Completions.new(
    model: "gpt-4o-mini",
    messages: [
      ChatMessage.user(
        "Give me some background on the elixir language. Why was it created? What is it used for? What distinguishes it from other languages? How popular is it?"
      )
    ]
  )

You are able to pass images to the API by creating a message.

ChatMessage.user(
  MsgContent.image_url(
    "https://raw.githubusercontent.com/restlessronin/openai_ex/main/assets/images/starmask.png"
  )
)

You can generate a chat completion using the ChatCompletion.create() function:

{:ok, chat_response} = openai |> Chat.Completions.create(chat_req)

For a more in-depth example of ChatCompletion, check out the Deeplearning.AI OrderBot Livebook.

You can also call the endpoint and have it stream the response. This returns the result as a series of tokens, which have to be put together in code.

To use the stream option, call the ChatCompletion.create() function with stream: true (and stream_options set to {%include_usage: true} to receive usage information)

{:ok, chat_stream} = openai |> Chat.Completions.create(chat_req |> Map.put(:stream_options, %{include_usage: true}), stream: true)
IO.puts(inspect(chat_stream))
IO.puts(inspect(chat_stream.task_pid))
chat_stream.body_stream |> Stream.flat_map(& &1) |> Enum.each(fn x -> IO.puts(inspect(x)) end)

Canceling a streaming request

The chat_stream.task_pid can be used in conjunction with OpenaiEx.HttpSse.cancel_request/1 to cancel an ongoing request.

You need to check the return chat_stream.status field. In case the status is not 2XX, the body_stream and task_pid fields are not available. Instead, an error field will be returned.

For example

bad_req = Chat.Completions.new(model: "code-llama", messages: [])
{:error, err_resp} = openai |> Chat.Completions.create(bad_req, stream: true)

For a detailed example of the use of the streaming ChatCompletion API, including how to cancel an ongoing request, check out Streaming Orderbot, the streaming equivalent of the prior example.

Stream Timeout

While OpenAI’s official API implementation typically doesn’t require explicit timeout handling for streams, some third-party implementations of the OpenAI API may benefit from custom timeout settings. OpenaiEx provides a way to set a stream-specific timeout to handle these cases.

You can set a stream-specific timeout using the with_stream_timeout function:

# Set a stream timeout of 30 seconds
openai_with_timeout = openai |> OpenaiEx.with_stream_timeout(30_000)

This is particularly useful when working with third-party OpenAI API implementations that may have different performance characteristics than the official API.

Exception Handling for Streams

When working with streams, it’s important to handle potential exceptions that may occur during stream processing. OpenaiEx uses a custom exception type for stream-related errors. Here’s how you can handle these exceptions:

alias OpenaiEx.Error

process_stream = fn openai, request ->
  response = Chat.Completions.create!(openai, request, stream: true)
  
  try do
    response.body_stream
    |> Stream.flat_map(& &1)
    |> Enum.each(fn chunk ->
      # Process each chunk here
      IO.inspect(chunk)
    end)
  rescue
    e in OpenaiEx.Error ->
      case e do
        %{kind: :sse_cancellation} -> 
          IO.puts("Stream was canceled")
          {:error, :canceled, e.message}
        %{kind: :sse_timeout_error} -> 
          IO.puts("Timeout on SSE stream")
          {:error, :timeout, e.message}
        _ -> 
          IO.puts("Unknown error occurred")
          {:error, :unknown, e.message}
      end
    e ->
      IO.puts("An unexpected error occurred")
      {:error, :unexpected, Exception.message(e)}
  end
end

# Usage
chat_req = Chat.Completions.new(
  model: "gpt-4o-mini",
  messages: [ChatMessage.user("Tell me a short story about a brave knight")],
  max_tokens: 500
)

# Use the OpenaiEx struct with custom stream timeout
result = process_stream.(openai_with_timeout, chat_req)
case result do
  {:error, type, message} ->
    IO.puts("Error type: #{type}")
    IO.puts("Error message: #{message}")
  _ ->
    IO.puts("Stream processed successfully")
end

In this example, we define a process_stream function that handles different types of stream exceptions:

  • :canceled: The stream was canceled. We return an error tuple.
  • :timeout: The stream timed out. We return an error tuple.
  • Any other OpenaiEx.Exception: We treat it as an unknown error.
  • Any other exception: We treat it as an unexpected error.

This approach allows you to gracefully handle different types of stream-related errors and take appropriate actions.

For more information on generating chat completions, see the OpenAI API Chat Completions reference.

Function(Tool) Calling

In OpenAI’s ChatCompletion endpoint, you can use the function calling feature to call a custom function and pass its result as part of the conversation. Here’s an example of how to use the function calling feature:

First, we set up the function specification and completion request. The function specification defines the name, description, and parameters of the function we want to call. In this example, we define a function called get_current_weather that takes a location parameter and an optional unit parameter. The completion request includes the function specification, the conversation history, and the model we want to use.

tool_spec =
  Jason.decode!("""
    {"type": "function",
     "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  """)

rev_msgs = [
  ChatMessage.user("What's the weather like in Boston today?")
]

fn_req =
  Chat.Completions.new(
    model: "gpt-4o-mini",
    messages: rev_msgs |> Enum.reverse(),
    tools: [tool_spec],
    tool_choice: "auto"
  )

Next, we call the OpenAI endpoint to get a response that includes the function call.

{:ok, fn_response} = openai |> Chat.Completions.create(fn_req)

We extract the function call from the response and call the appropriate function with the given parameters. In this example, we define a map of functions that maps function names to their implementations. We then use the function name and arguments from the function call to look up the appropriate function and call it with the given parameters.

fn_message = fn_response["choices"] |> Enum.at(0) |> Map.get("message")
tool_call = fn_message |> Map.get("tool_calls") |> List.first()
tool_id = tool_call |> Map.get("id")
fn_call = tool_call |> Map.get("function")

functions = %{
  "get_current_weather" => fn location, unit ->
    %{
      "location" => location,
      "temperature" => "72",
      "unit" => unit,
      "forecast" => ["sunny", "windy"]
    }
    |> Jason.encode!()
  end
}

fn_name = fn_call["name"]
fn_args = fn_call["arguments"] |> Jason.decode!()

location = fn_args["location"]
unit = unless is_nil(fn_args["unit"]), do: fn_args["unit"], else: "fahrenheit"

fn_value = functions[fn_name].(location, unit)

We then pass the returned value back to the ChatCompletion endpoint with the conversation history to that point to get the final response.

latest_msgs = [ChatMessage.tool(tool_id, fn_name, fn_value) | [fn_message | rev_msgs]]

fn_req_2 =
  Chat.Completions.new(
    model: "gpt-4o-mini",
    messages: latest_msgs |> Enum.reverse()
  )

{:ok, fn_response_2} = openai |> Chat.Completions.create(fn_req_2)

The final response includes the result of the function call integrated into the conversation.

Storing and Managing Chat Completions

OpenAI allows storing chat completions for later retrieval. This is useful for conversation history, auditing, or analytics purposes.

Store a Chat Completion

To store a chat completion, include store: true in your request:

stored_chat_req = 
  Chat.Completions.new(
    model: "gpt-4o-mini",
    messages: [
      ChatMessage.user("Explain quantum computing briefly")
    ],
    store: true
  )

{:ok, stored_completion} = openai |> Chat.Completions.create(stored_chat_req)
completion_id = stored_completion["id"]

Retrieve a Stored Chat Completion

Use Chat.Completions.retrieve() to fetch a previously stored chat completion:

{:ok, retrieved_completion} = openai |> Chat.Completions.retrieve(completion_id: completion_id)

Get Messages from a Chat Completion

To access the messages in a stored chat completion, use Chat.Completions.messages_list():

{:ok, messages} = openai |> Chat.Completions.messages_list(completion_id)

List Stored Chat Completions

To get a list of all your stored chat completions, use Chat.Completions.list():

{:ok, completions_list} = openai |> Chat.Completions.list()

You can filter by metadata, model, or use pagination parameters:

{:ok, filtered_completions} = openai |> Chat.Completions.list(
  metadata: %{project: "research"},
  model: "gpt-4o-mini",
  limit: 10,
  order: "desc"
)

Update a Chat Completion

You can update the metadata of a stored chat completion using Chat.Completions.update():

{:ok, updated_completion} = openai |> Chat.Completions.update(
  completion_id: completion_id,
  metadata: %{category: "technical", reviewed: "true"}
)

Delete a Chat Completion

To remove a stored chat completion, use Chat.Completions.delete():

{:ok, delete_result} = openai |> Chat.Completions.delete(completion_id: completion_id)

For more information on managing chat completions, see the OpenAI API Chat reference.

Responses

alias OpenaiEx.Responses

The Responses API combines features of Chat Completions and Assistants, providing a streamlined way to interact with OpenAI models. It supports single-turn and multi-turn conversations, tool use (including built-in tools), and optional server-side state management. It is the recommended approach for building applications that would previously have used the Assistants API, which will be sunset in the first half of 2026.

Create a Response

Use Responses.create() to generate a response. You can provide a simple prompt or a list of ChatMessage structs for multi-turn conversations. The store: true option enables server-side state management.

First, a simple example with a single prompt:

response_req = %{
  model: "gpt-4o-mini",
  input: "Explain the theory of relativity.",
  store: true # Optional: Enables state management
}

{:ok, response} = openai |> Responses.create(response_req)

Next, a multi-turn example using ChatMessage structs:

response_req2 = %{
  model: "gpt-4o-mini",
  input: [
    ChatMessage.user("What is the capital of France?"),
    ChatMessage.assistant("Paris."),
    ChatMessage.user("And what is the capital of Germany?")
  ],
  store: true
}

{:ok, response2} = openai |> Responses.create(response_req2)

Streaming Responses

The Responses API supports streaming, activated by passing stream: true to the create function. This returns the result as a series of semantic events, which can be processed and displayed as they arrive.

Here’s a simple example that prints each event:

streaming_req = %{
  model: "gpt-4o-mini",
  input: "Tell me a 300 word story about a cat.",
  store: true  # Streaming works with or without storing the response
}

{:ok, stream} = openai |> Responses.create(streaming_req, stream: true)

stream.body_stream
|> Stream.flat_map(& &1)
|> Enum.each(fn event ->
  IO.puts(inspect(event))
end)

Here’s a more detailed example that demonstrates how to handle different event types using pattern matching:

streaming_req = %{
  model: "gpt-4o-mini",
  input: "Tell me a 500 word story about a cat.",
  store: true  # Streaming works with or without storing the response
}

{:ok, stream} = openai |> Responses.create(streaming_req, stream: true)

stream.body_stream
|> Stream.flat_map(& &1)
|> Enum.each(fn event ->
  case event do
    %{data: data, event: "response.created"} ->
      # Initial response creation.  Contains metadata.
      IO.puts("Response Created: #{inspect(data)}")

    %{data: data, event: "response.in_progress"} ->
       # The response is still being generated.
      IO.puts("Response In Progress: #{inspect(data)}")

    %{data: data, event: "response.output_item.added"} ->
      # An output item (e.g. text, tool call) was added to the response
      IO.puts("Output Item Added: #{inspect(data)}")

    %{data: data, event: "response.content_part.added"} ->
      # A content part was added (e.g. a chunk of text for a text output item.)
      IO.puts("Content Part Added: #{inspect(data)}")

    %{data: %{"delta" => delta}, event: "response.output_text.delta"} ->
      # Text delta (a chunk of text).  Write this to the console/UI.
      IO.write(delta)

    %{data: data, event: "response.output_text.done"} ->
       # The text output is complete.
      IO.puts("\nOutput Text Done: #{inspect(data)}")

    %{data: data, event: "response.content_part.done"} ->
      # A content part is done (e.g., end of a text chunk)
      IO.puts("Content Part Done: #{inspect(data)}")

    %{data: data, event: "response.output_item.done"} ->
      # An output item is done (e.g. text generation or tool call)
      IO.puts("Output Item Done: #{inspect(data)}")

    %{data: data, event: "response.completed"} ->
      # The entire response is complete.
      IO.puts("\nResponse Completed: #{inspect(data)}")

    other ->
      # Handle other event types.  See API reference for full list.
      IO.puts("Other Event: #{inspect(other)}")
  end
end)

Streaming with Tool Calling

Streaming also works when using tools. The stream will include events related to tool usage, such as response.output_item.added when a tool call is initiated and response.output_tool_call.* events for the tool call’s progress and results.

Here’s an example using the built-in web_search_preview tool, demonstrating how to handle tool-related events in the stream:

streaming_req_with_tool = %{
  model: "gpt-4o-mini",
  input: "What's the latest news on the James Webb Telescope?",
  tools: [%{type: "web_search_preview"}],
  store: true
}

{:ok, stream} = openai |> Responses.create(streaming_req_with_tool, stream: true)

stream.body_stream
|> Stream.flat_map(& &1)
|> Enum.each(fn event ->
  case event do
    %{data: data, event: "response.created"} ->
      IO.puts("Response Created: #{inspect(data)}")

    %{data: data, event: "response.in_progress"} ->
      IO.puts("Response In Progress: #{inspect(data)}")

    %{data: data, event: "response.output_item.added"} ->
      IO.puts("Output Item Added: #{inspect(data)}")

    %{data: data, event: "response.content_part.added"} ->
      IO.puts("Content Part Added: #{inspect(data)}")

    %{data: %{"delta" => delta}, event: "response.output_text.delta"} ->
      IO.write(delta)

    %{data: data, event: "response.output_text.done"} ->
      IO.puts("\nOutput Text Done: #{inspect(data)}")

    %{data: data, event: "response.content_part.done"} ->
      IO.puts("Content Part Done: #{inspect(data)}")

    %{data: data, event: "response.output_item.done"} ->
      IO.puts("Output Item Done: #{inspect(data)}")

    %{data: data, event: "response.completed"} ->
      IO.puts("\nResponse Completed: #{inspect(data)}")

     %{data: data, event: "response.output_tool_call.started"} ->
      IO.puts("Tool Call Started: #{inspect(data)}")

    %{data: data, event: "response.output_tool_call.in_progress"} ->
      IO.puts("Tool Call In Progress: #{inspect(data)}")

    %{data: data, event: "response.output_tool_call.completed"} ->
      IO.puts("Tool Call Completed: #{inspect(data)}")

    %{data: data, event: "response.output_tool_call.failed"} ->
      IO.puts("Tool Call Failed: #{inspect(data)}")

    %{
      data: %{
        "index" => index,
        "id" => id,
        "type" => type,
        "web_search_preview" => %{"query" => query}
      },
      event: "response.output_tool_call.delta"
    } ->
      IO.puts(
        "Tool Call Delta (index=#{index}, id=#{id}, type=#{type}, query=#{query})"
      )

    other ->
      # Handle other event types.
      IO.puts("Other Event: #{inspect(other)}")
  end
end)

This example shows how to handle response.output_tool_call.* events to track the progress of tool calls within the stream, specifically for the web_search_preview tool.

Retrieve a Response

If store was set to true, you can retrieve the full response object (including conversation history) using Responses.retrieve(). Response objects are automatically deleted after 30 days.

response_id = response["id"]

{:ok, retrieved_response} = openai |> Responses.retrieve(response_id: response_id)
# With optional parameters
{:ok, retrieved_response} = openai |> Responses.retrieve([
  response_id: response_id,
  include: ["output[*].file_search_call.search_results", "output[*].web_search_call.search_results", "file_search_call.results", "web_search_call.results", "message.input_image.image_url"]
])

List Input Items from a Response

To access the input items from a stored response, use Responses.input_items_list():

{:ok, input_items} = openai |> Responses.input_items_list([
  response_id: response_id
])

You can paginate through large input lists:

{:ok, input_items} = openai |> Responses.input_items_list([
  response_id: response_id
])

# If there are more items and we want to paginate
if input_items["has_more"] do
  # Use the last_id from the first request to get the next page
  last_id = input_items["last_id"]
  
  {:ok, next_page_items} = openai |> Responses.input_items_list([
    response_id: response_id,
    limit: 10, 
    after: last_id,
    order: "desc"
  ])
  IO.puts(next_page_items)
end

Delete a Response

To remove a stored response, use Responses.delete():

{:ok, delete_result} = openai |> Responses.delete(response_id: response_id)

This allows you to clean up stored responses that are no longer needed.

For more information on the Responses API, see the OpenAI API Responses Reference.

Image

Generate Image

We define the image creation request structure using the Image.new function

alias OpenaiEx.Images

img_req = Images.Generate.new(prompt: "An adorable baby sea otter", size: "256x256", n: 1)

Then call the Image.create() function to generate the images.

{:ok, img_response} = openai |> Images.generate(img_req)

For more information on generating images, see the OpenAI API Image reference.

Fetch the generated images

With the information in the image response, we can fetch the images from their URLs

fetch_blob = fn url ->
  Finch.build(:get, url) |> Finch.request!(OpenaiEx.Finch) |> Map.get(:body)
end
fetched_images = img_response["data"] |> Enum.map(fn i -> i["url"] |> fetch_blob.() end)

View the generated images

Finally, we can render the images using Kino

fetched_images
|> Enum.map(fn r -> r |> Kino.Image.new("image/png") |> Kino.render() end)
img_to_expmt = fetched_images |> List.first()

Edit Image

We define an image edit request structure using the Images.Edit.new() function. This function requires an image and a mask. For the image, we will use the one that we received. Let’s load the mask from a URL.

# if you're having problems downloading raw github content, you may need to manually set your DNS server to "8.8.8.8" (google)
star_mask =
  fetch_blob.(
    "https://raw.githubusercontent.com/restlessronin/openai_ex/main/assets/images/starmask.png"
  )

# star_mask = OpenaiEx.new_file(path: Path.join(__DIR__, "../assets/images/starmask.png"))

Set up the image edit request with image, mask and prompt.

img_edit_req =
  Images.Edit.new(
    image: {"dummy_image.png", img_to_expmt},
    mask: {"dummy_mask.png", star_mask},
    size: "256x256",
    prompt: "Image shows a smiling Otter"
  )

We then call the Image.create_edit() function

{:ok, img_edit_response} = openai |> Images.edit(img_edit_req)

and view the result

img_edit_response["data"]
|> Enum.map(fn i -> i["url"] |> fetch_blob.() |> Kino.Image.new("image/png") |> Kino.render() end)

Image Variations

We define an image variation request structure using the Images.Variation.new() function. This function requires an image.

img_var_req = Images.Variation.new(image: img_to_expmt, size: "256x256")

Then call the Image.create_variation() function to generate the images.

###

{:ok, img_var_response} = openai |> Images.create_variation(img_var_req)
img_var_response["data"]
|> Enum.map(fn i -> i["url"] |> fetch_blob.() |> Kino.Image.new("image/png") |> Kino.render() end)

For more information on images variations, see the OpenAI API Image Variations reference.

Embedding

Define the embedding request structure using Embedding.new.

alias OpenaiEx.Embeddings

emb_req =
  Embeddings.new(
    model: "text-embedding-ada-002",
    input: "The food was delicious and the waiter..."
  )

Then call the Embedding.create() function.

{:ok, emb_response} = openai |> Embeddings.create(emb_req)

For more information on generating embeddings, see the OpenAI API Embedding reference

Audio

alias OpenaiEx.Audio

Create speech

For text to speech, we create an Audio.Speech request structure as follows

speech_req =
  Audio.Speech.new(
    model: "tts-1",
    voice: "alloy",
    input: "The quick brown fox jumped over the lazy dog",
    response_format: "mp3"
  )

We then call the Audio.Speech.create() function to create the audio response

{:ok, speech_response} = openai |> Audio.Speech.create(speech_req)

We can play the response using the Kino Audio widget.

speech_response |> Kino.Audio.new(:mp3)

Create transcription

To define an audio transcription request structure, we need to create a file parameter using Audio.File.new().

# if you're having problems downloading raw github content, you may need to manually set your DNS server to "8.8.8.8" (google)
audio_url = "https://raw.githubusercontent.com/restlessronin/openai_ex/main/assets/transcribe.mp3"
audio_file = OpenaiEx.new_file(name: audio_url, content: fetch_blob.(audio_url))

# audio_file = OpenaiEx.new_file(path: Path.join(__DIR__, "../assets/transcribe.mp3"))

The file parameter is used to create the Audio.Transcription request structure

transcription_req = Audio.Transcription.new(file: audio_file, model: "whisper-1")

We then call the Audio.Transcription.create() function to create a transcription.

{:ok, transcription_response} = openai |> Audio.Transcription.create(transcription_req)

Create translation

The translation call uses practically the same request structure, but calls the Audio.Translation.create() endpoint

For more information on the audio endpoints see the Openai API Audio Reference

File

List files

To request all files that belong to the user organization, call the File.list() function

alias OpenaiEx.Files

openai |> Files.list()

Upload files

To upload a file, we need to create a file parameter, and then the upload request

# if you're having problems downloading raw github content, you may need to manually set your DNS server to "8.8.8.8" (google)
ftf_url = "https://raw.githubusercontent.com/restlessronin/openai_ex/main/assets/fine-tune.jsonl"
fine_tune_file = OpenaiEx.new_file(name: ftf_url, content: fetch_blob.(ftf_url))

# fine_tune_file = OpenaiEx.new_file(path: Path.join(__DIR__, "../assets/fine-tune.jsonl"))

upload_req = Files.new_upload(file: fine_tune_file, purpose: "fine-tune")

Then we call the File.create() function to upload the file

{:ok, upload_res} = openai |> Files.create(upload_req)

We can verify that the file has been uploaded by calling

openai |> Files.list()

We grab the file id from the previous response value to use in the following samples

file_id = upload_res["id"]

Retrieve files

In order to retrieve meta information on a file, we simply call the File.retrieve() function with the given id

openai |> Files.retrieve(file_id)

Retrieve file content

Similarly to download the file contents, we call File.content()

openai |> Files.content(file_id)

Delete file

Finally, we can delete the file by calling File.delete()

openai |> Files.delete(file_id)

Verify that the file has been deleted by listing files again

openai |> Files.list()

Containers

Containers provide isolated environments for running code via the Code Interpreter tool. The API reference can be found at OpenAI Containers API.

List containers

To request all containers that belong to the user organization, call the Containers.list() function

alias OpenaiEx.Containers

openai |> Containers.list()

Create container

To create a new container, we need to create a container request with a name

container_req = Containers.new(name: "Example Container")

We can also specify expiration settings and attach files during creation

container_req = Containers.new(
  name: "Advanced Container",
  expires_after: %{anchor: "last_active_at", minutes: 20},
  file_ids: []
)

Then we call the Containers.create() function to create the container

{:ok, container_res} = openai |> Containers.create(container_req)

We can verify that the container has been created by listing containers again

openai |> Containers.list()

We grab the container id from the previous response value to use in the following samples

container_id = container_res["id"]

Retrieve container

In order to retrieve information on a container, we simply call the Containers.retrieve() function with the given id

openai |> Containers.retrieve(container_id)

Delete container

Finally, we can delete the container by calling Containers.delete()

openai |> Containers.delete(container_id)

Verify that the container has been deleted by listing containers again

openai |> Containers.list()

Container Files

Container Files allow you to create and manage files within OpenAI containers for use with the Code Interpreter tool. Files can be uploaded directly or referenced from existing file IDs. The API reference can be found at OpenAI Container Files API.

First, let’s create a container to work with for our file examples

alias OpenaiEx.ContainerFiles

# Create a container for our file operations
container_req = Containers.new(name: "File Demo Container")
{:ok, container_res} = openai |> Containers.create(container_req)
demo_container_id = container_res["id"]

List container files

To request all files in a container, call the ContainerFiles.list() function

openai |> ContainerFiles.list(demo_container_id)

Upload file to container

To upload a file directly to a container, we need to create a file upload request

# Create a sample file content
sample_content = """
# Sample Python Script
print("Hello from container!")

def calculate_sum(a, b):
    return a + b

result = calculate_sum(10, 20)
print(f"Sum: {result}")
"""

# Create file upload request
cf_upload_req = ContainerFiles.new_upload(file: {"hello.py", sample_content})

Then we call the ContainerFiles.create() function to upload the file

{:ok, upload_res} = openai |> ContainerFiles.create(demo_container_id, cf_upload_req)

We can verify that the file has been uploaded by listing container files

openai |> ContainerFiles.list(demo_container_id)

We grab the file id from the previous response to use in the following samples

container_file_id = upload_res["id"]

Retrieve container file

To retrieve metadata about a container file, call the ContainerFiles.retrieve() function

openai |> ContainerFiles.retrieve(demo_container_id, container_file_id)

Get container file content

To download the actual content of a container file, use ContainerFiles.content()

{:ok, file_content} = openai |> ContainerFiles.content(demo_container_id, container_file_id)
IO.puts(file_content)

Delete container file

To delete a file from a container, call ContainerFiles.delete()

openai |> ContainerFiles.delete(demo_container_id, container_file_id)

Verify that the file has been deleted by listing container files again

openai |> ContainerFiles.list(demo_container_id)

Finally, clean up by deleting the demo container

openai |> Containers.delete(demo_container_id)

FineTuning Job

To run a fine-tuning job, we minimally need a training file. We will re-run the file creation request above.

{:ok,  upload_res} = openai |> Files.create(upload_req |> Map.put(:purpose, "fine-tune"))

Next we call FineTuning.Jobs.new() to create a new request structure

alias OpenaiEx.FineTuning

ft_req = FineTuning.Jobs.new(model: "gpt-4o-mini-2024-07-18", training_file: upload_res["id"])

To begin the fine tune, we call the FineTuning.Jobs.create() function

{:ok, ft_res} = openai |> FineTuning.Jobs.create(ft_req)

We can list all fine tunes by calling FineTuning.Jobs.list()

openai |> FineTuning.Jobs.list()

The function FineTune.retrieve() gets the details of a particular fine tune.

ft_id = ft_res["id"]
openai |> FineTuning.Jobs.retrieve(fine_tuning_job_id: ft_id)

and FineTuning.Jobs.list_events() can be called to get the events

openai |> FineTuning.Jobs.list_events(fine_tuning_job_id: ft_id)

To cancel a Fine Tune job, call FineTuning.Jobs.cancel()

openai |> FineTuning.Jobs.cancel(fine_tuning_job_id: ft_id)

A fine tuned model can be deleted by calling the Model.delete()

ft_model = ft_res["fine_tuned_model"]

unless is_nil(ft_model) do
  openai |> Models.delete(ft_model)
end

For more information on the fine tune endpoints see the Openai API Moderation Reference

Batch

alias OpenaiEx.Batches

Create batch

Use the Batch.create() function to create and execute a batch from an uploaded file of requests.

First, we need to upload a file containing the batch requests using the File API.

batch_url =
  "https://raw.githubusercontent.com/restlessronin/openai_ex/main/assets/batch-requests.jsonl"

batch_file = OpenaiEx.new_file(name: batch_url, content: fetch_blob.(batch_url))

# batch_file = OpenaiEx.new_file(path: Path.join(__DIR__, "../assets/batch-requests.jsonl"))
batch_upload_req = Files.new_upload(file: batch_file, purpose: "batch")
{:ok, batch_upload_res} = openai |> Files.create(batch_upload_req)

Then, we create the batch request using Batch.new() and specify the necessary parameters.

batch_req =
  Batches.new(
    input_file_id: batch_upload_res["id"],
    endpoint: "/v1/chat/completions",
    completion_window: "24h"
  )

Finally, we call the Batch.create() function to create and execute the batch.

{:ok, batch} = openai |> Batches.create(batch_req)

Retrieve batch

Use the Batch.retrieve() function to retrieve information about a specific batch.

batch_id = batch["id"]
{:ok, batch_job} = openai |> Batches.retrieve(batch_id: batch_id)
batch_job_output_file_id = batch_job["output_file_id"]
{:ok, batch_job_output_file} = openai |> Files.retrieve(batch_job_output_file_id)
{status, batch_result} = batch_output_content = openai |> Files.content(batch_job_output_file["id"])

Note that the string is not valid json (it’s a sequence of json objects without the commas or the array ‘[‘ ‘]’ delimiters), so it cannot be parsed as such.

Cancel batch

Use the Batch.cancel() function to cancel an in-progress batch.

{status, cancel_result} = openai |> Batches.cancel(batch_id: batch_id)

List batches

Use the Batch.list() function to list your organization’s batches.

openai |> Batches.list()

For more information on the Batch API, see the OpenAI API Batch Reference.

Moderation

We use the moderation API by calling Moderation.new() to create a new request

alias OpenaiEx.Moderations

mod_req = Moderations.new(input: "I want to kill people")

The call the function Moderation.create()

mod_res = openai |> Moderations.create(mod_req)

For more information on the moderation endpoints see the Openai API Moderation Reference

Evals

The Evals API allows you to create and manage evaluation runs for your models. The API reference can be found at OpenAI Evals API.

alias OpenaiEx.Evals

Create Eval

To create an evaluation, call the Eval.create() function.

First, we setup the create request parameters:

eval_req = Evals.new(
  name: "Sentiment",
  data_source_config: %{
    type: "stored_completions",
    metadata: %{
      usecase: "chatbot"
    }
  },
  testing_criteria: [
    %{
      type: "label_model",
      model: "o3-mini",
      input: [
        %{
          role: "developer",
          content: "Classify the sentiment of the following statement as one of 'positive', 'neutral', or 'negative'"
        },
        %{
          role: "user",
          content: "Statement: {{item.input}}"
        }
      ],
      passing_labels: [
        "positive"
      ],
      labels: [
        "positive",
        "neutral",
        "negative"
      ],
      name: "Example label grader"
    }
  ]
)

Then we call the create function:

{:ok, eval} = openai |> Evals.create(eval_req)

Update Eval

Once created, only the name and metadata of an evaluation can be modified using the Eval.update() function:

update_req = Evals.new(
  name: "Updated Evaluation",
  metadata: %{
    key: "value"
  }
)
eval_id = eval["id"]
{:ok, updated_eval} = openai |> Evals.update(eval_id, update_req)

List Evals

We use Eval.list() to get a list of evaluations:

# Basic usage
{:ok, evals} = openai |> Evals.list()

# With optional parameters
{:ok, filtered_evals} = openai |> Evals.list(%{
  limit: 10,
  order: "desc"
})

Delete Eval

Finally, we can delete evaluations using the Eval.delete() function:

{:ok, delete_result} = openai |> Evals.delete(eval_id)

Working with Runs

Evals can have multiple runs. Here are the functions for managing runs:

{:ok, eval} = openai |> Evals.create(eval_req)

Note: The following examples are shown as non-executable code blocks because they contain complex data structures and variable dependencies that may not run correctly in all contexts. These examples are syntactically correct and demonstrate the proper API usage, but depend on specific evaluation and run states that may not be available during execution.

Create Run

To create a new run for an evaluation:

eval_id = eval["id"]
run_req = %{
  name: "gpt-4o-mini",
  data_source: %{
    type: "completions",
    input_messages: %{
      type: "template",
      template: [
        %{
          role: "developer",
          content: "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\"  \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\"  \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\"  \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\"  \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\"  \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n"
        },
        %{
          role: "user",
          content: "{{item.input}}"
        }
      ]
    },
    sampling_params: %{
      temperature: 1,
      max_completions_tokens: 2048,
      top_p: 1,
      seed: 42
    },
    model: "gpt-4o-mini",
    source: %{
      type: "file_content",
      content: [
        %{
          item: %{
            input: "Tech Company Launches Advanced Artificial Intelligence Platform",
            ground_truth: "Technology"
          }
        }
      ]
    }
  }
}

{:ok, run} = openai |> Evals.create_run(eval_id, run_req)

Get Run

To retrieve a specific run:

run_id = run["id"]
{:ok, run} = openai |> Evals.get_run(eval_id, run_id)

Cancel Run

To cancel an in-progress run:

{:ok, cancel_result} = openai |> Evals.cancel_run(eval_id, run_id)

List Runs

To get all runs for an evaluation:

{:ok, runs} = openai |> Evals.get_runs(eval_id)

Delete Run

To delete a run:

{:ok, delete_result} = openai |> Evals.delete_run(eval_id, run_id)

Working with Output Items

Runs produce output items that contain the evaluation results:

List Output Items

To get all output items for a run:

{:ok, output_items} = openai |> Evals.list_run_output_items(eval_id, run_id)

Get Output Item

To retrieve a specific output item:

output_item_id = "some_output_item_id"
{:ok, output_item} = openai |> Evals.get_run_output_item(eval_id, run_id, output_item_id)

Vector Stores

alias OpenaiEx.VectorStores
alias OpenaiEx.VectorStores.Files
alias OpenaiEx.VectorStores.File.Batches

Create vector store

Use VectorStores.create() to create a vector store.

vector_store_req = VectorStores.new(name: "HR Documents")
{:ok, vector_store} = openai |> VectorStores.create(vector_store_req)
vector_store_id = vector_store["id"]

Retrieve vector store

Use VectorStores.retrieve() to retrieve a vector store.

openai |> VectorStores.retrieve(vector_store_id)

Update vector store

Use VectorStores.update() to modify the vector store.

openai |> VectorStores.update(vector_store_id, %{name: "HR Documents 2"})

Delete vector store

VectorStores.delete() can be used to delete a vector store.

openai |> VectorStores.delete(vector_store_id)

List vector stores

We use VectorStores.list() to get a list of vectorstores.

openai |> VectorStores.list()

Vector Store Files

Create vector store file

We can create a vector store file by attaching a file to a vector store using VectorStores.Files.create().

First we recreate the vector store above

{:ok, vector_store} = openai |> VectorStores.create(VectorStores.new(name: "HR Documents"))
vsf_url = "https://raw.githubusercontent.com/restlessronin/openai_ex/main/assets/cyberdyne.txt"
vector_store_file = OpenaiEx.new_file(name: vsf_url, content: fetch_blob.(ftf_url))
#vector_store_file = OpenaiEx.new_file(path: Path.join(__DIR__, "../assets/cyberdyne.txt"))
upload_req = OpenaiEx.Files.new_upload(file: vector_store_file, purpose: "assistants")
{:ok, upload_res} = openai |> OpenaiEx.Files.create(upload_req)
file_id = upload_res["id"]

then attach the file id from earlier

vector_store_id = vector_store["id"]
{:ok, vs_file} = openai |> VectorStores.Files.create(vector_store_id, file_id)

Create vector store file with attributes

The new stable API supports adding attributes (metadata) to vector store files:

# Create a file with attributes for better organization
file_params = %{
  attributes: %{
    "source" => "hr_documentation",
    "category" => "policies",
    "priority" => "high",
    "last_updated" => "2024-01-15"
  }
}

{:ok, vs_file_with_attrs} = openai |> VectorStores.Files.create(vector_store_id, file_id, file_params)

Retrieve vector store file

Retrieve a vector store file using the VectorStores.Files.retrieve() function

openai |> VectorStores.Files.retrieve(vector_store_id, file_id)

Delete vector store file

Detach a file from the vector store using VectorStores.Files.delete()

openai |> VectorStores.Files.delete(vector_store_id, file_id)

List vector store files

List vector store files using VectorStores.Files.list()

openai |> VectorStores.Files.list(vector_store_id)

Vector Store File Batches

File batches allow addition of multiple files to a vector store in a single operation.

Create VS file batch

Use VectorStores.File.Batch.create() to attach a list of file ids to a vector store.

{:ok, vsf_batch} = openai |> VectorStores.File.Batches.create(vector_store_id, [file_id])

Retrieve VS file batch

Use VectorStores.File.Batch.retrieve() to retrieve the batch.

vsf_batch_id = vsf_batch["id"]
openai |> VectorStores.File.Batches.retrieve(vector_store_id, vsf_batch_id)

Cancel VS file batch

Use VectorStores.File.Batches.cancel() to cancel a batch.

openai |> VectorStores.File.Batches.cancel(vector_store_id, vsf_batch_id)

List VS file batch

Use VectorStores.File.Batches.list()

openai |> VectorStores.File.Batches.list(vector_store_id, vsf_batch_id)

Other notebooks:

Back