*Editor's note: sizes, licences and download links below were checked
against each model's own Hugging Face card, and every link was verified to
resolve at time of writing. Throughput figures marked **(measured)** come
from published community benchmarks on this exact chip; those marked
`(est.)` are my own arithmetic from active parameters against this
platform's real memory bandwidth, and should be treated as a rough band, not
a promise. Where a figure is contested between sources I've given the range
rather than picking the flattering end.*

*Hardware assumed: AMD Ryzen AI Max+ 395 ("Strix Halo"), 128GB unified
LPDDR5X, Radeon 8060S integrated GPU, 50-TOPS XDNA2 NPU. No CUDA — this is
a ROCm/Vulkan machine, which rules some things out.*

## The one number that decides everything

This machine has **128GB of capacity and about 256 GB/s of bandwidth on
paper — roughly 215 GB/s in practice.** Those two numbers pull in opposite
directions, and understanding why is the difference between a great local AI
box and a frustrating one.

Capacity decides *what fits*. Bandwidth decides *how fast it talks*. For
token generation, the GPU must read every active weight from memory for
every single token it emits, so:

```
tokens/sec  ≈  memory bandwidth  ÷  bytes read per token
```

At ~215 GB/s that arithmetic is unforgiving for dense models and generous
for sparse ones:

| Model shape | Bytes read per token (Q4) | Realistic tok/s here |
|---|---|---|
| 3B active (MoE) | ~1.7 GB | ~90–100 **(measured)** |
| 5.1B active (MoE) | ~2.9 GB | ~34–55 **(measured)** |
| 12B active (MoE) | ~6.8 GB | ~20–30 `(est.)` |
| 27B dense | ~17 GB | ~10–14 `(est.)` |
| 70B dense | ~40 GB | ~5 at Q8 to ~15 at aggressive Q4 **(measured)** |
| 123B dense | ~62 GB | ~3–4 `(est.)` — unusable interactively |

**So the rule for this hardware is counterintuitive: prefer bigger-but-sparser
over smaller-but-dense.** An 80B mixture-of-experts model with 3B active
parameters occupies 43GB of your 128GB and runs at conversational speed. A
dense 27B model occupies 18GB — less than half as much — and runs six times
slower. Capacity is the thing you have in abundance; bandwidth is the thing
you're rationing. Spend the capacity.

This is also why Strix Halo is a genuinely good MoE machine and a mediocre
dense one. The 128GB lets you hold models a 24GB discrete GPU can't touch;
the bandwidth means you should only bother if those models are sparse.

## Setting the machine up first

Four things worth knowing before you download 60GB of weights:

- **You must raise the GTT limit or you can't use most of your RAM.** By
  default the GPU-addressable pool is roughly half of system memory. Getting
  to a ~96GB GPU allocation needs a small BIOS framebuffer (512MB) plus a
  kernel-level bump via `amdgpu.gttsize` or `amdttm.pages_limit`, verified
  with `mem_info_gtt_total`. Leave 4–12GB for the CPU and kernel — pushing
  GTT too high has been reported to make large loads hang rather than fail
  cleanly.
- **Vulkan or ROCm depends on your context length, and the gap is large.**
  For chat-length prompts, Vulkan (RADV) is simpler and equally fast. At 32K
  context one head-to-head measured standard ROCm at 17.82 tok/s against
  Vulkan's 31.54 — but a tuned `rocWMMA` branch hit 36.43, beating Vulkan by
  15%. At 128K depth, prompt processing was 51 tok/s on tuned ROCm versus 17
  on Vulkan, roughly 3×. **Short prompts: Vulkan. Long documents: tuned
  ROCm.** Test both; don't assume.
- **Linux, if you want the GPU properly.** ROCm has no official Windows
  support for this chip. On Windows, LM Studio's Vulkan backend works out of
  the box and is the sane choice; on Linux you get the full range including
  the tuned ROCm paths. AMD's **Lemonade** server is the most convenient
  single install — it bundles LLM serving, Stable Diffusion, Whisper and
  Kokoro TTS behind one OpenAI-compatible API.
- **Ignore the NPU for now.** The 50-TOPS XDNA2 NPU is largely unused for
  general LLM inference as of September 2026. Lemonade and FastFlowLM target
  it for specific small models, and Linux NPU support is only now arriving
  (Lemonade 10.0 via FastFlowLM; Ubuntu 26.04 adding native support). Buy
  this machine for its iGPU and its memory, not its TOPS number.

## The fifteen

Grouped by the job you're doing. **Model names link to the download.**
Download sizes are the 4-bit quantisation unless noted, since that's what
you'll actually pull.

| Model | Job | Params total / active | Download | Speed here | Licence | Why this one |
|---|---|---|---|---|---|---|
| [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b) | General reasoning, the heavy hitter | 117B / 5.1B | ~61 GB (native MXFP4) | **34–55 tok/s (measured)** | **Apache 2.0** | The most capable model that runs *comfortably* on this box, and it's measured on this exact chip |
| [Nemotron 3 Super](https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF) | Long documents, 1M context | 120B / 12B | ~62 GB Q4 | ~20–30 `(est.)` | NVIDIA Nemotron Open | A 1M-token context window on a laptop; the only entry here that can hold a whole codebase |
| [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) | Fast everyday generalist | 35B / 3B | 23 GB | ~90–100 `(est.)` | **Apache 2.0** | Multimodal (text/image/video), 262K context, and fast enough to feel instant |
| [Gemma 4 26B-A4B](https://huggingface.co/google/gemma-4-26b-a4b-it) | Multilingual work | 25.2B / 3.8B | ~16 GB | ~80–100 `(est.)` | **Apache 2.0** | 140+ languages and image input at MoE speed; the 31B dense sibling is the wrong pick here |
| [Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next) | Coding, primary | 80B / 3B | ~43 GB Q4 | ~60–90 `(est.)` | **Apache 2.0** | **The single best fit on this machine** — near-top open coding scores at 3B active cost |
| [GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash) | Coding, long context | 31.2B / 3B | ~19 GB | ~90–100 `(est.)` | **MIT** | 200K context, MIT licence, 3B active — the cheap-and-permissive coding default |
| [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) | Coding, highest quality | 27B dense | 18 GB | ~10–14 `(est.)` | **Apache 2.0** | Best single-model coding quality in this size class — but dense, so slow here. Batch work, not chat |
| [Devstral Small 2 24B](https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512) | Agentic coding / tool use | 24B dense | ~14 GB | ~12–16 `(est.)` | **Apache 2.0** | 68.0% SWE-bench Verified, purpose-built for agent loops rather than chat |
| [FLUX.2 klein 4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) | Images, commercial-safe | 4B | ~13 GB | ~30–60 s/image `(est.)` | **Apache 2.0** | The image model you can actually use in paid work — see the licence warning below |
| [Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1) | Images, best quality | 7B + 8B encoder | ~30 GB | ~1–2 min/image `(est.)` | ⚠️ **Qwen Research** | Best open image quality (RGBA, 10 reference images, editing) — **research licence, not commercial** |
| [Qwen3-VL-32B](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct) | Vision, documents, GUI | 33B dense | ~20 GB | ~10–14 `(est.)` | **Apache 2.0** | Strongest open doc/vision model; 256K→1M context, OCR in 32 languages, GUI grounding |
| [GLM-OCR](https://huggingface.co/zai-org/GLM-OCR) | OCR, invoices, scans | 0.9B | ~1 GB | near-instant | **MIT** | 94.62 on OmniDocBench V1.5, **#1 overall** — a 0.9B model beating 200B+ VLMs at OCR |
| [Parakeet TDT 0.6B v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) | Speech to text | 0.6B | ~1.5 GB | ~3,300× real time | CC-BY-4.0 | 6.34% avg WER across 25 European languages, and fast enough to run on the CPU alone |
| [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) | Text to speech | 82M | ~0.3 GB | real time | **Apache 2.0** | Tiny, good, Apache-licensed, and already bundled in Lemonade |
| [Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) | RAG retrieval | 0.6B | ~1.5 GB | instant | **Apache 2.0** | Best quality-per-GB embedder; pair with [Qwen3-Reranker-4B](https://huggingface.co/Qwen/Qwen3-Reranker-4B) for a real accuracy jump |

Quantised GGUF builds for most of the LLMs above live under
[unsloth](https://huggingface.co/unsloth) — usually the fastest route to a
file that fits, including [gpt-oss-120b](https://huggingface.co/unsloth/gpt-oss-120b-GGUF)
and [Qwen3.6-27B](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF).

## Two licence traps in that table

Both are easy to miss and both matter if money is involved:

- **Qwen-Image-2.1 is under the Qwen Research License Agreement**, not
  Apache 2.0. It is the best open image model here by output quality, and it
  is *not* licensed for commercial use. Every "best free image models 2026"
  listicle I found omitted this. **If you are generating images for paid
  work, use FLUX.2 klein 4B** (genuinely Apache 2.0) and treat Qwen-Image as
  a research toy.
- **Devstral 2 123B carries a revenue clause.** The 24B Small in the table
  is clean Apache 2.0, but its 123B sibling is "modified MIT" where the
  modification is that you may not exercise the licence if your company's
  global monthly revenue exceeds $20M. It's also a dense 123B, so on this
  hardware it would run at 3–4 tok/s anyway — avoid it on both counts.

Elsewhere: Nemotron 3 Super is under the NVIDIA Nemotron Open Model Licence
(permissive, but read it), Parakeet is CC-BY-4.0 (attribution required), and
everything marked Apache 2.0 or MIT above is genuinely unencumbered.

## What not to put on this machine

Saying this plainly because the temptation with 128GB is to reach too far:

- **Anything dense above ~30B.** It will load and it will disappoint. Dense
  70B lands at 5–15 tok/s; dense 123B at 3–4. The memory is not the
  constraint — the bandwidth is, and no quantisation fixes that.
- **DeepSeek-V4-Flash-0731.** At 166.9 GB it does not fit, and the sub-4-bit
  quants that would fit give up the quality you wanted it for.
- **GLM-5.3-Flash.** 321B total means ~180 GB at 4-bit and ~239 GB for the
  2-bit dynamic quant. Both exceed your usable pool.
- **Anything CUDA-only.** No CUDA path exists on the Radeon 8060S. Most
  mainstream inference (llama.cpp, Ollama, vLLM) is fine on ROCm/Vulkan, but
  custom-kernel projects, some training code and parts of the diffusion
  tooling ecosystem still have no ROCm story.
- **Image generation, if you need it fast.** This box does roughly 2 minutes
  per image — comparable to ComfyUI on an RTX 4060 laptop GPU. Perfectly
  usable for occasional asset work; painful as a creative loop.

## A loadout that fits in one 128GB pool

You don't need all fifteen resident. Three practical configurations:

| Loadout | Resident set | Total | Leaves free |
|---|---|---|---|
| **Everyday** | Qwen3.6-35B-A3B + GLM-4.7-Flash + GLM-OCR + Parakeet + Kokoro + embedder | ~46 GB | ~50 GB for apps and KV cache |
| **Coding day** | Qwen3-Coder-Next + Devstral Small 2 + embedder + reranker | ~62 GB | ~34 GB |
| **Heavy reasoning** | gpt-oss-120b alone, plus the small utility models | ~66 GB | ~30 GB |

The small models — OCR, ASR, TTS, embedder — total under 5GB combined and
can stay loaded permanently. They are the best value in the entire table.

## Caveats

- **Only three throughput figures here are measured on this chip**
  (gpt-oss-120b at 34–55, the ~3B-active MoE class at ~90–100, dense 70B at
  5–15). Everything else is bandwidth arithmetic and will vary with
  quantisation, backend, context depth and thermals. A thin notebook chassis
  will also throttle sustained loads in a way a mini-PC with the same chip
  won't.
- **The 34–55 tok/s spread on gpt-oss-120b is real, not sloppiness.**
  Community llama.cpp benchmarks recorded up to 55.57 tok/s on Vulkan/RADV;
  another head-to-head measured 34.13 on identical settings. Backend and
  llama.cpp build version explain the gap — which is itself the lesson about
  how much tuning matters on this platform.
- **Benchmark scores are quoted sparingly and from model cards** where
  possible. Third-party "best local model" rankings for 2026 disagree
  violently with each other and several are auto-generated.
- **Context length eats your headroom.** Every size above is weights only. A
  long context on a 1M-capable model can add tens of gigabytes of KV cache.
  Budget for it before you fill the pool.

## My take

The instinct with 128GB of unified memory is to chase the biggest model that
fits. On this hardware that instinct is wrong, and the table above is mostly
an argument against it.

**The best three picks here are all sparse.** Qwen3-Coder-Next (80B total,
3B active, Apache 2.0) is the standout — it uses a third of your memory and
runs at conversational speed while scoring near the top of open coding
benchmarks, and the only reason it's possible is that this machine has
capacity to spare and bandwidth it doesn't. gpt-oss-120b is the right choice
when you want maximum reasoning and can accept 34–55 tok/s. Qwen3.6-35B-A3B
is the one that will actually get used daily, because ~90 tok/s changes how
you interact with a model.

**The most underrated line in the table is the cheapest.** GLM-OCR at 0.9B
and MIT beats 200B-parameter vision models on document OCR, Parakeet
transcribes 25 languages at thousands of times real time, Kokoro does
passable TTS in 300MB, and the Qwen3 embedder is 1.5GB. That's a complete
document-and-voice pipeline in under 5GB, permanently resident, with room
left for a 60GB reasoning model on top. For actual work — reading invoices,
transcribing meetings, searching your own files — that stack will earn more
hours than the 120B model will.

**And the thing I'd want told to me before buying:** this is an inference
machine, not a development machine. No CUDA means quantised-inference work
is excellent and CUDA-native training or custom-kernel work is off the
table entirely. If your job is running models, 128GB at 215 GB/s is a
remarkable amount of capability in a notebook. If your job is writing code
against NVIDIA's stack, the memory size is not the spec that matters.

## Further reading

- [Strix Halo LLM benchmark results — Level1Techs forum thread](https://forum.level1techs.com/t/strix-halo-ryzen-ai-max-395-llm-benchmark-results/233796) — the community's running measurements
- [strix-halo-guide](https://github.com/hogeheer499-commits/strix-halo-guide) — evidence-backed setup notes, Vulkan vs ROCm, large GGUFs
- [strix-halo-windows-llm-bench](https://github.com/ihanesman/strix-halo-windows-llm-bench) — backend/flag/spec-decoding comparisons under Windows
- [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) — AMD's local server bundling LLM, image, ASR and TTS behind one API
- [AMD: running a trillion-parameter LLM on a Ryzen AI Max+ cluster](https://www.amd.com/en/developer/resources/technical-articles/2026/how-to-run-a-one-trillion-parameter-llm-locally-an-amd.html) — what happens when you network several of these
- [AMD ROCm — Strix Halo system optimization](https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html) — the GTT and framebuffer settings
- [Unsloth: running Qwen3.6 locally](https://unsloth.ai/docs/models/qwen3.6) — quantisation memory tables
- [Self-Hosting the Frontier on Multi-GPU Servers](/publications/local-ai-models-2026/) — the companion piece, for when one notebook isn't the target