Reefy GPU and NPU provider architecture

Reefy keeps large, vendor-specific GPU stacks out of its immutable base image. When an app requests an accelerator, Reefy resolves and activates a host provider built for that exact Reefy OS build. The app then receives the device through the Container Device Interface (CDI). Generic Linux DRM infrastructure and libdrm remain in the base OS, while Intel, NVIDIA, and AMD driver payloads are delivered independently.

This design reduces the base firmware size, keeps unused proprietary files off devices, and lets NVIDIA, AMD, and Intel support evolve independently.

Responsibility split

Reefy does not install a complete CUDA or ROCm development environment on the host. Those libraries belong to the app image and can be versioned with the application that uses them.

From an app request to a usable device

APP-SPEC accelerator request
  -> cloud resolves a provider for the exact Reefy build
  -> desired state contains an immutable OCI manifest digest
  -> device verifies and caches the manifest and layer digests
  -> SquashFS layers mount read-only
  -> the provider-owned activation hook runs
  -> the hook loads matching components and publishes CDI
  -> Reefy starts the app's Compose project

Explicit CDI requests select providers:

The historical APP-SPEC field gpu: true remains equivalent to requesting an NVIDIA GPU. An app may request multiple CDI resources, and Reefy prepares each provider independently.

Exact build and kernel matching

Externally built kernel modules must match more than uname -r. Reefy firmware CI assigns an immutable build ID and records a digest of the kernel ABI evidence. Provider CI builds against that exact kernel configuration and Buildroot toolchain, then records both values in the artifact metadata.

The device rejects a host provider when either value differs from the running OS. Desired state also selects providers by exact build ID, not by a mutable tag or a human-readable firmware version. This keeps A/B upgrades and rollbacks deterministic: each slot activates only its matching provider.

A provider already loaded into the running kernel is not replaced with a different payload. If desired state changes to a genuinely different provider after activation, Reefy reports that a reboot is required. Manifests that have different publication metadata but the exact same runtime config and layers are treated as the same payload.

OCI storage and activation

Host providers are OCI artifacts, but Reefy does not use Docker to activate them. A small Docker-independent client works before or alongside application reconciliation. It accepts only exact SHA-256 digest references from approved provider repositories.

Artifact blobs are stored under /mnt/reefy-data/artifacts and mounted under /run/reefy-artifacts. Provider layers are SquashFS files mounted read-only with nodev and nosuid. The only provider entry point Reefy invokes is the fixed hook at usr/lib/reefy/activate.

Transfers and read-only mounts can run concurrently, with at most two artifact admissions in progress. Activation is serialized across vendors because hooks may update the shared module dependency index, load modules, and publish CDI specifications. Per-artifact and per-provider file locks prevent duplicate downloads and activation races. Kernel file locks are released automatically if a process exits or crashes.

The cache retains the three most recently used versions of each artifact type, plus anything required by current desired state, currently active, or explicitly pinned. This bounds persistent storage while preserving current and recent rollback payloads.

Boot, offline operation, and retries

At boot, a one-shot service activates matching providers that are already in the persistent cache. It does not require network access and exits after its work is complete. Normal app reconciliation can download a missing provider later and invoke the same activation path.

A slow or unavailable provider does not block unrelated applications. Provider preparation is scoped to each app project, and different apps continue to reconcile independently.

Hardware detection and graceful fallback

Vendor knowledge lives inside each provider's activation hook. Generic Reefy code does not maintain PCI product lists or duplicate NVIDIA, AMD, or Intel probing rules. The hook decides whether compatible hardware is usable and publishes CDI resources only after successful initialization.

Reefy then checks the CDI resources that actually exist. Any unresolved CDI request is removed from that app's Compose configuration before startup. The activation output is included in Reefy's normal device logs, and failed preparation is retried with bounded backoff.

This makes accelerator access best effort. It guarantees that missing hardware does not prevent the container from starting, but it cannot guarantee that the application itself supports a non-accelerated fallback.

Current providers

Provider versions can change with a Reefy release. Check the current firmware and provider metadata before selecting hardware for a particular driver branch.

Supply-chain controls

Provider publication creates a software bill of materials (SBOM), immutable digest-addressed OCI manifests, and signed build provenance through GitHub OIDC. On the device, repository allowlists, exact manifest and blob digest verification, architecture checks, build and kernel ABI matching, read-only mounts, and the fixed activation-hook contract constrain what may run as a host extension.