GPU drivers and memory

hal0’s inference containers need the host kernel to already expose the GPU (and, on Strix Halo, the NPU) as device nodes. hal0 ships the userspace runtime — ROCm, XRT, the FastFlowLM binary — inside its container images, but it cannot load a kernel driver from inside a container. This page covers what the host needs before you install, and how container/LXC device passthrough must be wired so hal0 can see it.

Kernel and firmware

The iGPU uses the in-tree amdgpu driver. On AMD Strix Halo (Ryzen AI Max+ 395/385/390), get onto a kernel ≥ 6.14 with current linux-firmware — earlier kernels are missing GTT and NPU fixes that Strix Halo depends on. If you also want the XDNA NPU, the kernel needs the amdxdna module available (modprobe amdxdna) and NPU firmware current enough to match it.

hal0 does not install or manage this layer — it is host-side operator setup done before hal0’s installer runs.

Verify amdgpu

Confirm the kernel actually bound amdgpu to the GPU before you install:

lspci -nnk | grep -A3 VGA

Look for Kernel driver in use: amdgpu in the output. If it instead shows amdgpu under “Kernel modules” but not “Kernel driver in use”, the module is available but not bound — check dmesg for why the driver probe failed (firmware mismatch is the usual cause).

Device nodes

Once amdgpu (and, for NPU hosts, amdxdna) is bound, the kernel exposes the accelerators as device nodes:

Node	Role
/dev/dri/renderD*	Render node — Vulkan / ROCm render access
/dev/kfd	ROCm compute queues
/dev/accel/accel0	XDNA NPU (FastFlowLM)

hal0’s hardware probe reads these directly; if a node is missing, the corresponding capability drops out of the probe rather than failing the install.

Host ROCm is not required

The host only needs amdgpu bound and the nodes above present — it does not need a host ROCm install. Every hal0 slot container carries its own ROCm userspace.

Render-group gids

Containers (podman slots, or an LXC passing devices through to an inner container runtime) need group access to these nodes, and the group id that matters is the device node’s owner gid, not whatever name it resolves to on the host. Check it directly:

stat -c %g /dev/dri/renderD128

Use that numeric gid with --group-add (or the container runtime’s equivalent) when granting the container access. Don’t rely on getent group render — on a given host that name can resolve to a different gid than the one that actually owns the render node, which silently leaves the container unable to open it.

Size the GTT pool

On Strix Halo the GPU’s usable memory is the amdgpu GTT pool, carved from system RAM (unified memory). A model can only allocate up to the GTT cap, not the full RAM pool.

On kernel ≥ 6.14, amdgpu grows the GTT/GART limit dynamically — hal0 reads the live pool size from mem_info_gtt_total on every request rather than trusting a boot-time snapshot, precisely because that limit can move (see src/hal0/api/routes/hardware.py). You generally don’t need to pin anything; hal0 measures whatever the kernel currently reports and sizes its model recommendations against it.

Pinning GTT/TTM parameters on the kernel command line is an operator choice, not a requirement — it reserves a large fixed pool up front instead of letting the kernel size it dynamically, which is useful if you want a guaranteed floor regardless of what else the host is doing. Here’s a worked example from a 128 GB Strix Halo host that pins one:

# /etc/default/grub — GRUB_CMDLINE_LINUX_DEFAULT on a 128 GB Strix Halo host
iommu=pt amd_iommu=on amd_pstate=active amdgpu.gttsize=118784 \
amdgpu.cwsr_enable=1 ttm.pages_limit=30408704 ttm.page_pool_size=30408704 \
amdttm.pages_limit=30408704 amdttm.page_pool_size=30408704

Parameter	Value	Meaning
amdgpu.gttsize	118784	GTT window in MiB (116 GiB)
ttm.pages_limit / ttm.page_pool_size	30408704	TTM page cap in 4 KiB pages (116 GiB — keep equal to gttsize)
amdttm.pages_limit / amdttm.page_pool_size	30408704	Same cap, duplicated for the amdttm module name used on some kernels
iommu=pt amd_iommu=on	—	IOMMU passthrough mode, left on here (this host does not disable the IOMMU)
amd_pstate=active	—	Active P-state driver for CPU frequency scaling
amdgpu.cwsr_enable=1	—	Compute wave save/restore, needed for compute-queue preemption

On this host the pinned values leave 116 GiB of a 128 GB machine as the GTT pool; the observed live pool (/sys/class/drm/renderD*/device/mem_info_gtt_total) matched the pinned gttsize/pages_limit exactly. Treat these as a starting point, not a universal constant — scale gttsize and the pages_limit/ page_pool_size pairs down together (they must stay equal) to reserve more host RAM for other tenants. After rebooting, cat /proc/cmdline should show the params.

NVIDIA

NVIDIA support is experimental. hal0 targets AMD Strix Halo first, and RTX 30/40/50-series GPUs are supported through CDI and the NVIDIA Container Toolkit rather than through the AMD device-node path above. Install the NVIDIA Container Toolkit and generate a CDI spec on the host so podman/docker can request the GPU by CDI device name; hal0’s hardware probe picks it up from there. Expect rougher edges than the AMD path — this tier gets less field validation.