Model roster & benchmarks reference

Three distinct concepts share the word “model” here: the registry (what’s actually on disk), the curated catalogue (what the installer offers to download), and benchmarks (measured performance, not tied 1:1 to either).

Model registry

Each registered model is a Model record (hal0.registry.model), persisted as one atomic TOML file per model under /var/lib/hal0/registry/.

Field	Notes
id, name, path	—
size_bytes, quant	Derived from the GGUF header where possible, filename fallback.
license	—
capabilities	Freeform tags, normalized to a fixed modality set: chat, embed, rerank, vision, asr, tts.
hf_repo / hf_filename	—
backends	Which slot backends can run this model: GGUF → vulkan, rocm, cuda, cpu; moonshine → moonshine; kokoro → kokoro.
mmproj	Optional vision-projector sidecar path.
defaults	Tri-state launcher knobs the model now owns: context_size, extra_args, chat_template, profile, mtp, jinja, enable_thinking, vision.
default	Per-dispatch-type default marker.
architecture	e.g. llama, qwen2, gemma3, gpt-oss, qwen3next, mamba — drives MoE/context sizing.
capability_flags.tool_calling	Typed tri-state flag.

Registered models split into two buckets for the dashboard UI: blessed (path under /var/lib/hal0/models/<recipe>/<capability>/) vs pulled (anything downloaded or hand-registered outside that layout).

Note

As of the v1.0 slot/hardware-ownership rework, the model, not the slot, owns tuning knobs like context_size and MTP. A slot’s [model].context_size is now a ceiling only — see Config schema.

Curated catalogue

hal0.registry.curated.CuratedModel — not a file on disk, shipped in code with the release, so a fresh install can never land on a stale pick list. This is the roster the first-run wizard’s model cards render via GET /api/install/curated-models.

Field	Notes
id, display_name, description	—
family, architecture	—
size_gb, vram_gb_min	—
license, license_url	—
hf_repo, hf_file, mmproj_file (optional)	—
context_length	—
recommended_slot	chat or img
capability	chat, embed, asr, tts, image
tags, notes	—

Caution

STT/TTS curated picks are not implemented as of this writing — kokoro and vibevoice can’t be expressed in the current single-file GGUF/safetensors/checkpoint curated schema (they ship as multi-file ONNX/PyTorch bundles). This is a documented open item in source with three possible resolution paths, none landed for those two. Don’t document a curated STT/TTS pick flow as existing for kokoro or vibevoice.

Moonshine’s resolution is decided (it doesn’t wait on the curated-schema redesign): weights are operator-staged under the model store rather than registry-pulled, and both slot spawn and hal0 doctor preflight the staged path, failing loudly and by name (slot.weights_missing) instead of starting a container that 500s on first request. See ADR-0001.

Hardware-driven recommendation

hardware.recommend.recommend_primary_slot(hw) is the installer’s logic for choosing the model that seeds the primary chat slot, ordered largest-to-smallest and RAM-gated:

Curated id	Min RAM	Class	Approx throughput
Qwen3.6-35B-A3B-MTP-GGUF	≥ 48 GB	Hybrid MoE	~45 tok/s
qwen3.5-9b	≥ 16 GB	Dense	—
qwen3-4b	0 GB (always downloadable)	Dense	—
llama32-3b	CPU fallback	Dense	—
phi3-mini	CPU fallback	Dense	—

Every id in this tier list must exist in the curated catalogue — enforced by test (tests/registry/test_curation_drift.py, tests/hardware/test_recommend.py), so this table can drift only if both source and its tests change together.

Context-window resolution for the seeded primary slot: MoE/MTP models get the full architecture-max window (their KV cache is tiny relative to a hybrid model); dense models are capped at 32768 tokens; if neither applies, the fallback is 8192.

See Hardware matrix for the backend (ROCm/Vulkan/ CUDA/CPU) selection logic this recommendation composes with.

Benchmarks

The bench/ package measures real llama.cpp performance — this is a distinct concept from the roster above, not a per-curated-model score table.

Record schema (bench.schema) — stdlib dataclasses only, no Pydantic, so a record is constructible with zero dependencies on the target box. Records are appended to a JSONL file. Each record splits into:

• Identity block (feeds the dedup key cell_key()): Model (id, gguf path, sha256, quant, size_bytes, capabilities), Engine (kind: llama-bench or llama-server, image, image digest, llama.cpp build), lane (rocm, vulkan_radv, default), Config (resolved argv/env/kv, spec block, parallel, context size), Workload (kind: pp/tg/chat/batch/embed/rerank/reuse, depth, n_prompt, n_gen, sampler, concurrency).
• Environment block (host) — deliberately excluded from cell_key(), so a hal0 or kernel upgrade doesn’t fork the identity of an otherwise-identical measurement.

Outcome enum: ok, failed, skipped-contended, oom, hang — only ok counts as a measured/publishable value.

Separately, [profile] sections in config/schema.py (sourced from config/seeds.py) carry a small static per-profile tps/rtf benchmark table (PROFILE_BENCH: dict[str, dict[str, float]]), surfaced on profile cards. This is curated per-profile, not per-model, and is distinct from the JSONL pipeline above.

CLI

hal0 bench plan|run|status|worker|results|history|reindex|devices|publish|eval|bundle|upload

hal0 bench is a raw argparse passthrough (not a Typer sub-app) — run hal0 bench <verb> --help for per-verb flags. Supporting modules: bench/devices.py (device enumeration), bench/planner.py (picks what’s stale), bench/runner.py (drives llama-bench/llama-server), bench/store.py (JSONL/db layer), bench/publish.py (pushes results to the dashboard), bench/suites.py, bench/regress.py.

Sharing results

Benchmark results never leave the box on their own — nothing is uploaded in the background, and no verb here runs unattended. Sharing is always an explicit command you type. To share a run, package it as a self-contained bundle and attach it wherever you discuss it (forum thread, issue, chat):

hal0 bench bundle --list                       # see what's eligible
hal0 bench bundle --runs <run_id> --title "Strix Halo, ROCm 7" -o run.hal0bench.tar.gz

A bundle contains the schema-2 records (resolved flags, engine digests, telemetry), the profile/suite TOMLs you pass with --profile, and optional raw artifacts (--with-artifacts). host.name is redacted by default, the manifest carries a sha256 for every member, and the whole archive is content-addressed — anyone can verify and reproduce what you measured.

Records with outcome: ok but no throughput measurement at all are left out of bundles: they carry nothing to publish, and the publish API rejects a whole upload over a single one.

Publishing to the public leaderboard

Maintainers of the hal0.dev leaderboard publish a bundle to the bench API, which is what backs hal0.dev/benchmarks. This requires an admin token and is not something a normal install needs:

export HAL0_BENCH_TOKEN=…                      # admin token; never a CLI flag
hal0 bench upload run.hal0bench.tar.gz
hal0 bench bundle --runs <run_id> --upload     # or bundle + publish in one step

The token is read from the environment only — passing a credential as an argv value would expose it in ps, in shell history, and in any journald unit that logs the command line.

Uploads are idempotent. The API is content-addressed, so re-sending a bundle that is already published is a no-op that reports the existing record count, and re-sending one that was unpublished restores it. A retry after a network failure cannot duplicate records.