Build documentation · for review by the owner

FRAME-SERV-2 — AI INFERENCE BUILD

This site records every change made to this machine (a friend's Framework Desktop) during its setup as a large-model AI inference rig. Every entry in the change log states what was done, why, and exactly how to reverse or tweak it. Nothing on the machine is undocumented.

Strix Halo · gfx1151 128G unified 2TB model store serving livebenchmarks running

The Machine

Chassis

Framework Desktop, rev A6 (SKU FRAMDACP06)

CPU

AMD Ryzen AI MAX+ 395 — 16C/32T, 64 MiB L3

GPU

Radeon 8060S integrated (Strix Halo) — kernel amdgpu, ROCm 7.1.1

NPU

XDNA2 NPU present (/dev/accel0) — optional, not required for LLM serving

Memory

128 GiB LPDDR5X unified (~256 GB/s) — GPU and CPU share one pool

Storage

1TB NVMe (OS, ext4) · 2TB NVMe → /srv/llm/models (ext4, freshly formatted for model weights)

Network

5GbE Realtek (primary) · Wi-Fi 7 (unused) · Tailscale node on the dxverm tailnet (100.126.153.18)

Fedora Linux 44, kernel 7.1.13-200, SELinux enforcing, firewalld on, BIOS 03.06

Inference

llama.cpp master (Vulkan + HIP builds) — serving live via llama-swap on :8080; rerank :18190

What this site contains

System Audit

The as-found state of the machine (2026-09-09, before any work): hardware, OS, drivers, services, security posture, storage. The baseline the change log measures against.

Read the audit →

Architecture

What was built and why: inference backend, model server topology, API endpoints, where weights live, how services start and stop.

Architecture →

Models

Which models were selected and why — quality/speed trade-offs, memory budgets on 128G unified, quant choices, tool-use / vision / embedding support.

Models →

Benchmarks

Measured numbers, not claims: tokens/s per model, prompt throughput, context behavior, and the exact commands to reproduce every number.

Benchmarks →

Runbook / How to Use

The owner's operating manual: start/stop serving, hit the API, swap models, and — equally important — how to use this documentation site itself.

Runbook →

Change Log

Every modification to the machine in chronological order, each with a rollback procedure. If anything here looks wrong, this page is the receipt.

Change log →

Ground rules of this build

Documented-or-didn't-happen. Every command that changes system state is logged here with its evidence file and a reversal path.
Reversible by default. Config files are backed up alongside their changes (dot-suffixed names ignored by their consumers); service changes are unit-file drop-ins, not edits of vendor files.
Model weights are the only large artifacts. They live on the dedicated 2TB disk (/srv/llm/models) so wiping or replacing models never touches the OS.
Benchmarks are recorded, not remembered. Every performance claim on the benchmarks page links to the raw command that produced it.

Built 2026-09-09 · documentation of record for this machine · served from Cloudflare Pages