Powered by AppSignal & Oban Pro
Notesclub
Sign in with GitHub
created by hec & contributors
terms privacy

Ultralytics Yolo11 - CPU

ultralytics_yolo.livemd

avatar
Poeticoding Ltd.
avatar
avatar
Share to X
Share to Bluesky
More notebooks

Ultralytics Yolo11 - CPU

Mix.install(
  [
    {:yolo, ">= 0.0.0"},
    {:nx, "~> 0.9"},
    {:exla, "~> 0.9"},
    
    {:image, "~> 0.54"},
    {:evision, "~> 0.2"},
    {:kino, "~> 0.16"},
    {:kino_yolo, github: "poeticoding/kino_yolo", branch: "main"},
    {:pythonx, "~> 0.4.2"},
    {:kino_pythonx, "~> 0.1.0"}
  ],
  config: [
    nx: [
      default_backend: EXLA.Backend,
      default_defn_options: [compiler: EXLA]
    ]
  ]
)

Application.put_env(:ortex, Ortex.Native, features: [:coreml])
[project]
name = "ultralytics_to_onnx"
version = "0.0.0"
requires-python = "==3.13.*"
dependencies = [
  "ultralytics",
  "onnx",
  "onnxruntime"
]

Define model and paths

Ultralytics Yolo11 to ONNX

Pythonx.eval(
  """
  from ultralytics import YOLO
  IMAGE_SIZE = 640
  model_name = model_name.decode("utf-8")
  
  model = YOLO(model_name)
  model.export(format='onnx', imgsz=IMAGE_SIZE, opset=21)
  """,
  %{"model_name" => "yolo11n.pt"}
)
:ok
Ultralytics 8.3.225 🚀 Python-3.13.5 torch-2.9.0 CPU (Apple M3)
YOLO11n summary (fused): 100 layers, 2,616,248 parameters, 0 gradients, 6.5 GFLOPs

PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB)
requirements: Ultralytics requirement ['onnxslim>=0.1.71'] not found, attempting AutoUpdate...
Requirement already satisfied: onnxslim>=0.1.71 in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (0.1.72)
Requirement already satisfied: colorama in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from onnxslim>=0.1.71) (0.4.6)
Requirement already satisfied: sympy>=1.13.3 in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from onnxslim>=0.1.71) (1.14.0)
Requirement already satisfied: onnx in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from onnxslim>=0.1.71) (1.19.1)
Requirement already satisfied: ml_dtypes in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from onnxslim>=0.1.71) (0.5.3)
Requirement already satisfied: packaging in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from onnxslim>=0.1.71) (25.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from sympy>=1.13.3->onnxslim>=0.1.71) (1.3.0)
Requirement already satisfied: numpy>=1.21 in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from ml_dtypes->onnxslim>=0.1.71) (2.2.6)
Requirement already satisfied: protobuf>=4.25.1 in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from onnx->onnxslim>=0.1.71) (6.33.0)
Requirement already satisfied: typing_extensions>=4.7.1 in /Users/alvise/code/yolo_elixir/venv/lib/python3.10/site-packages (from onnx->onnxslim>=0.1.71) (4.15.0)

requirements: AutoUpdate success ✅ 0.8s
WARNING ⚠️ requirements: Restart runtime or rerun command for updates to take effect


ONNX: starting export with onnx 1.19.1 opset 21...
WARNING ⚠️ ONNX: simplifier failure: No module named 'onnxslim'
ONNX: export success ✅ 1.0s, saved as 'yolo11n.onnx' (10.2 MB)

Export complete (1.2s)
Results saved to /Users/alvise/code/codebeam_2025
Predict:         yolo predict task=detect model=yolo11n.onnx imgsz=640  
Validate:        yolo val task=detect model=yolo11n.onnx imgsz=640 data=/usr/src/ultralytics/ultralytics/cfg/datasets/coco.yaml  
Visualize:       https://netron.app
:ok

Load and run YOLO

model = YOLO.load(
  model_path: "yolo11n.onnx", 
  classes_path: "coco_classes.json", 
  eps: [:coreml]
)

10:46:28.689 [info] Loaded model yolo11n.onnx with [:coreml] execution providers

10:46:28.693 [info] Initialized model yolo11n.onnx
%YOLO.Model{
  ref: #Ortex.Model<
    inputs: [
      {"images",
       "Tensor {\n    ty: Float32,\n    dimensions: [\n        1,\n        3,\n        640,\n        640,\n    ],\n}",
       [1, 3, 640, 640]}
    ]
    outputs: [
      {"output0",
       "Tensor {\n    ty: Float32,\n    dimensions: [\n        1,\n        84,\n        8400,\n    ],\n}",
       [1, 84, 8400]}
    ]>,
  classes: %{
    39 => "bottle",
    74 => "clock",
    59 => "bed",
    69 => "oven",
    67 => "cell phone",
    45 => "bowl",
    50 => "broccoli",
    22 => "zebra",
    51 => "carrot",
    26 => "handbag",
    63 => "laptop",
    47 => "apple",
    27 => "tie",
    77 => "teddy bear",
    0 => "person",
    5 => "bus",
    21 => "bear",
    62 => "tv",
    30 => "skis",
    16 => "dog",
    3 => "motorcycle",
    53 => "pizza",
    33 => "kite",
    14 => "bird",
    40 => "wine glass",
    37 => "surfboard",
    24 => "backpack",
    17 => "horse",
    48 => "sandwich",
    73 => "book",
    11 => "stop sign",
    57 => "couch",
    43 => "knife",
    6 => "train",
    20 => "elephant",
    60 => "dining table",
    28 => "suitcase",
    25 => "umbrella",
    1 => "bicycle",
    58 => "potted plant",
    32 => "sports ball",
    76 => "scissors",
    36 => "skateboard",
    35 => "baseball glove",
    15 => "cat",
    78 => "hair drier",
    64 => "mouse",
    75 => "vase",
    ...
  },
  model_impl: YOLO.Models.Ultralytics,
  shapes: %{input: {1, 3, 640, 640}, output: {1, 84, 8400}},
  model_data: nil
}
image_path = "images/kite.jpg"
mat = Evision.imread(image_path)
%Evision.Mat{
  channels: 3,
  dims: 2,
  type: {:u, 8},
  raw_type: 16,
  shape: {900, 1352, 3},
  ref: #Reference<0.2103773846.209059866.196730>
}
detected_objects = 
  model
  |> YOLO.detect(mat)
  |> YOLO.to_detected_objects(model.classes)
[
  %{
    class: "person",
    prob: 0.2557294964790344,
    bbox: %{h: 18, w: 13, cx: 549, cy: 526},
    class_idx: 0
  },
  %{
    class: "person",
    prob: 0.27376076579093933,
    bbox: %{h: 14, w: 10, cx: 1210, cy: 457},
    class_idx: 0
  },
  %{class: "person", prob: 0.274641752243042, bbox: %{h: 33, w: 17, cx: 527, cy: 514}, class_idx: 0},
  %{
    class: "person",
    prob: 0.3205679655075073,
    bbox: %{h: 34, w: 16, cx: 187, cy: 556},
    class_idx: 0
  },
  %{class: "person", prob: 0.6205487251281738, bbox: %{h: 43, w: 23, cx: 43, cy: 534}, class_idx: 0},
  %{class: "person", prob: 0.7554749250411987, bbox: %{h: 58, w: 24, cx: 94, cy: 537}, class_idx: 0},
  %{
    class: "person",
    prob: 0.8044017553329468,
    bbox: %{h: 149, w: 52, cx: 139, cy: 689},
    class_idx: 0
  },
  %{
    class: "person",
    prob: 0.8190996646881104,
    bbox: %{h: 159, w: 53, cx: 244, cy: 778},
    class_idx: 0
  },
  %{
    class: "kite",
    prob: 0.27058956027030945,
    bbox: %{h: 35, w: 18, cx: 315, cy: 393},
    class_idx: 33
  },
  %{
    class: "kite",
    prob: 0.44570764899253845,
    bbox: %{h: 18, w: 16, cx: 476, cy: 349},
    class_idx: 33
  },
  %{class: "kite", prob: 0.508489727973938, bbox: %{h: 26, w: 26, cx: 588, cy: 357}, class_idx: 33},
  %{
    class: "kite",
    prob: 0.6232947111129761,
    bbox: %{h: 30, w: 17, cx: 1091, cy: 409},
    class_idx: 33
  },
  %{class: "kite", prob: 0.8487096428871155, bbox: %{h: 45, w: 25, cx: 292, cy: 260}, class_idx: 33},
  %{class: "kite", prob: 0.8747634887695312, bbox: %{h: 73, w: 79, cx: 631, cy: 115}, class_idx: 33}
]
{:ok, image} = Image.from_evision(mat)
KinoYOLO.Draw.draw_detected_objects(image, detected_objects)

Real-time CAM

cap = Evision.VideoCapture.videoCapture(0)
%Evision.VideoCapture{
  fps: 24.000038,
  frame_count: 0.0,
  frame_width: 1920.0,
  frame_height: 1080.0,
  isOpened: true,
  ref: #Reference<0.2103773846.209059856.196655>
}
mat = Evision.VideoCapture.read(cap)

detected_objects = 
  model
  |> YOLO.detect(mat)
  |> YOLO.to_detected_objects(model.classes)

{:ok, image} = Image.from_evision(mat)
KinoYOLO.Draw.draw_detected_objects(image, detected_objects)

Other notebooks:

Back