Elixir Image Object Detection

Mix.install([
  {:image_detection, github: "elixir-image/image_detection"},
  {:kino, "~> 0.9"},
  :req
])

Introduction

This livebook is based upon a talk on object detection by @hansihe at the Warsaw Elixir meeting in March 2023.

The demo is based upon a development version of the image_detection library that uses the techniques presented by @hansihe which is also based upon:

• image for image processing
• axon for machine learning model execution
• axon_onnx for leveraging prebuild .onnx models
• req for downloading the yolov8n.onnx model

Example

# Download the Yolo v8n model
model_path = Path.join(System.tmp_dir!(), "yolov8n.onnx")

unless File.exists?(model_path) do
  Req.get!("https://github.com/Hyuto/yolov8-onnxruntime-web/raw/master/public/model/yolov8n.onnx",
    output: model_path
  )
end

uploaded_image = Kino.Input.image("Image in which to detect objects")

# Create a `t:Vix.Vips.Image.t/0` from the input
{:ok, image} =
  uploaded_image
  |> Kino.Input.read()
  |> Image.from_kino()

# Now run image detection on the image
# This step just runs object detection and
# return a mapping from object classes
# to bounding boxes
bounding_boxes = Image.Detection.detect(image, model_path)

Rendering Bounding Boxes

Now that the objects have been detected we have a mapping from object name (standard Coco object classes) to a list of bounding boxes where those objects were detected. Using this data we can:

• Know how many objects of what type have been detected
• Render bounding boxes on the original image

# How many people were detected?
bounding_boxes
|> Map.get("person")
|> length()

# Lets draw the bounding boxes with labels on the
# original image.
Image.Detection.draw_bbox_with_labels(bounding_boxes, image)