TimelessMetrics Architecture

This Livebook is a current overview of the default TimelessMetrics design.

One-Sentence Summary

TimelessMetrics now uses a Rust-native engine for the hot time-series path, with Elixir providing the HTTP API, ingest workers, PromQL layer, charts, alerts, scraping, and other product features.

Default Runtime Model

Current default:

{TimelessMetrics, name: :metrics, data_dir: "/data"}

That means:

• engine: :rust by default
• raw writes and reads go through TimelessMetrics.RustEngine
• Elixir still manages the surrounding system

Supervision Shape

Typical application setup:

children = [
  {TimelessMetrics, name: :metrics, data_dir: "/data"},
  {TimelessMetrics.HTTP, store: :metrics, port: 8428}
]

The store supervisor starts a rust-default tree roughly like:

TimelessMetrics.Supervisor
├── TimelessMetrics.DB
├── TimelessMetrics.RustEngine
├── TimelessMetrics.IngestWorker x N        (non-memory mode)
├── TimelessMetrics.AlertEvaluator          (non-memory mode)
├── TimelessMetrics.SelfMonitor             (optional)
├── DynamicSupervisor                       (scraping enabled)
└── TimelessMetrics.Scraper                 (scraping enabled)

Programmatic Write Path

TimelessMetrics.write / write_batch
  -> TimelessMetrics.RustEngine
  -> Rust NIF
  -> resolve series
  -> append to partition buffers
  -> flush to chunk files

The main performance path is batch ingest.

HTTP Ingest Path

HTTP handlers do not parse and ingest the body inline.

They:

1. accept the request
2. enqueue the raw body in ETS
3. return quickly
4. let IngestWorker parse and write in the background

That queueing model is shared across the supported ingest formats:

• VictoriaMetrics JSON lines
• Prometheus text exposition
• Influx line protocol

Read Path

Native reads:

• TimelessMetrics.query/4
• TimelessMetrics.query_multi/4
• TimelessMetrics.query_aggregate/4
• TimelessMetrics.query_aggregate_multi/4

These use the Rust engine in the default configuration.

PromQL and Prometheus-compatible HTTP endpoints sit above that:

HTTP / PromQL request
  -> parse query
  -> call TimelessMetrics query functions
  -> Rust engine returns matching data
  -> Elixir formats response

Persistence Model

The Rust engine persists:

• series registry data
• chunk files
• batch chunk files
• enough metadata to rebuild the in-memory index on startup

Elixir-side SQLite data still exists for product/admin concerns such as:

• annotations
• alerts
• scrape targets
• metric metadata

So the current system is not “Elixir replaced by Rust.” It is “Rust engine inside an Elixir application.”

Memory-Only Mode

{TimelessMetrics, name: :metrics, mode: :memory}

Memory-only mode still uses the rust engine, but skips durable raw-data persistence and certain background subsystems.

Good use cases:

• tests
• ephemeral environments
• constrained devices

Legacy Caveat

Older architecture descriptions in this repository may still mention:

• ETS shard buffers as the main engine design
• SegmentBuilder as the primary write/read engine
• Gorilla or ALP as the main active compression story
• SQLite-backed raw time-series storage

Those describe the legacy engine path, not the rust-default path on main.

Benchmarks

The maintained benchmark set is documented in:

• ../bench/README.md

The main benchmark entry points are:

• bench/write_bench.exs
• bench/http_concurrency.exs
• bench/realistic_workload.exs
• bench/tsbs_bench.exs
• bench/vs_victoriametrics.exs