Self-Hosting the Frontier on Multi-GPU Servers: 10 Free-to-Download Models and 5 You Pay For (September 2026)

Editor’s note: every figure below was checked against a primary source — the model’s own Hugging Face card, the vendor’s release post, or the vendor’s pricing page — because the secondary coverage of this space is now badly polluted. Searching for “best open LLM 2026” returns a large volume of auto-generated articles citing models, version numbers and benchmarks that exist on no vendor channel. Figures marked (est.) are my own arithmetic from parameter counts, not published numbers. Everything unmarked traces to a primary source.

Scope: this is written for multi-GPU server deployment — 2-GPU nodes up to NVL72-class racks. It is not a guide to running models on a desktop.

What changed in 2026

The open-weight frontier caught up. As of this month the top of the downloadable pile is a 2.8-trillion-parameter model with a 1M-token context window, and three separate labs ship models above 1.5T parameters under licences permitting commercial use. “Open weights” is no longer a synonym for “a generation behind,” which means the self-hosting decision is now a real capability-vs-cost tradeoff rather than a compromise.

Two structural shifts matter for anyone specifying hardware:

• Sparse MoE is now universal at the top. Every model in the top ten is mixture-of-experts. This decouples the memory bill from the compute bill in a way that changes procurement: a 428B/23B-active model needs the HBM of a 428B model but delivers the throughput of a 23B one. You size memory for total parameters and size your throughput expectations on active ones.
• Labs now ship natively low-precision checkpoints. Kimi K3’s released weights are MXFP4 via quantisation-aware training — not post-hoc squashed. NVIDIA ships NVFP4. DeepSeek’s V4-Flash-0731 release is 166.9 GB for a ~304B model, roughly half a byte per parameter. You are increasingly not downloading BF16 and quantising it yourself; the 4-bit artefact is the reference checkpoint, and it is the one the lab evaluated.

Reading the two size columns

The tables below separate two numbers that get conflated constantly, and that was a flaw in the first version of this article:

• Download (disk) — the size of the checkpoint you pull from Hugging Face. A one-time storage and bandwidth cost. Matters for your registry, your image builds and your cold-start time, not for your GPU count.
• GPU memory to serve — how much HBM must hold the model while it is answering requests: weights, plus KV cache, plus activation and framework overhead. This is the number that determines how many GPUs you buy.

They are not the same, and the gap between them is where capacity planning goes wrong. Two things drive the gap:

• Precision. BF16 = 2 bytes/param, FP8 = 1, MXFP4/NVFP4 ≈ 0.5. Serving the FP8 checkpoint of a 675B model needs roughly half the HBM of the BF16 one, from the same model card.
• KV cache at long context is not a rounding error. At the 1M-token context windows these models advertise, KV cache can rival or exceed the weights. Every GPU-memory figure below assumes a production context budget, not the advertised maximum — if you intend to actually serve 1M tokens to concurrent users, add substantially more headroom and measure it on your own stack.

GPU configurations quoted for reference: H100-80GB and A100-80GB = 80 GB each, H200 = 141 GB, B200 = 180 GB. So 8× H100 ≈ 640 GB, 8× H200 ≈ 1.1 TB, 8× B200 ≈ 1.4 TB.

The 10 most capable models you can download for free

Ranked by capability, which correlates loosely with size. Model names link straight to the download repository — every link was checked to resolve at time of writing.

Quantised community builds (GGUF, AWQ, MXFP4) for most of these live under unsloth — usually the fastest route to a checkpoint that fits fewer GPUs than the reference release.

Not in the ranking, but worth knowing: language-optimised models

The table above ranks on general capability, which systematically hides models built for one language. The clearest example is Rakuten AI 3.0 — a ~700B MoE optimised for Japanese, developed under Japan’s GENIAC programme with METI backing, and released under plain Apache 2.0. On parameter count and licence it sits alongside Mistral Large 3; on English general benchmarks it won’t place, because that isn’t what it was built for. If your workload is Japanese, it is likely the best thing you can download, and the licence has no rider.

Japan’s wider ecosystem is worth a look for the same reason — Preferred Networks’ PLaMo 3.0 Prime is trained from scratch but keeps its weights private, while LLM-jp publishes fully open models from the national consortium. The same logic applies to Indic, Arabic and Southeast Asian models: a global leaderboard is the wrong instrument for picking one.

Choosing by the hardware you have

The practical way to read the table above. Capability rises down the list; so does the number of GPUs you have to buy and keep busy.

Two observations that matter more than the leaderboard:

The small-node tier is the value inflection. DeepSeek-V4-Flash-0731 is 166.9 GB, MIT-licensed, 1M context, 13B active, and comes from the lab whose V4-Pro is a genuine frontier model. It fits on two H200s without you quantising anything. If I had to put one model from this list into production on hardware a mid-sized company can actually finance and keep utilised, this is it — and Nemotron 3 Super at ~124 GB on 2× H100 is the close second if you want NVIDIA’s supported stack and reproducible training.

The top tier is a sovereignty option, not a deployment plan. Kimi K3, Qwen3.8-Max and DeepSeek-V4-Pro are open in the sense that matters legally and closed in the sense that matters operationally: the weights are yours, the 1.5–5 TB of HBM is not. NVIDIA published a GB300 NVL72 serving guide for Qwen3.8-2.4T-A95B, which tells you the intended venue. Specify these when the requirement is “this model must never leave our facility” and the budget already includes racks.

Licence and capability gotchas

These cost real money if you find them after committing, and none are obvious from a spec sheet:

• Qwen3.8’s open weights are not Qwen3.8-Max. The downloadable checkpoint is text-only and thinking-mode-required. The hosted API adds vision input, non-thinking mode, 1M context by default and built-in tools. The Hugging Face discussion thread on exactly this is titled “Huge disappointment.” If you selected this model for multimodal work from Max’s spec sheet, you selected the wrong artefact.
• GLM regressed on licence between minor versions. GLM-5.2 (753B, 1M context, June 2026) is MIT. GLM-5.3 is a bespoke “GLM-5.3 Licence.” If permissiveness outweighs the last capability increment, deploy 5.2 — it is two months older. There is also GLM-5.3-Flash at 321B / 18B active (~180 GB at 4-bit), a far cheaper family member.
• Kimi K3 changed licence from K2. K2 was modified-MIT; K3 is bespoke, and secondary sources describe revenue sharing up to 30% for inference providers grossing over US$20M/year. That makes it free for essentially all self-hosters and a paid licence for commercial inference platforms. I could not confirm the exact terms from Moonshot directly — read the licence before building a product on it.
• Command A+’s 2× H100 claim implies quantisation. 218B at BF16 is ~436 GB by arithmetic; 160 GB of H100 implies roughly 4-bit. Cohere’s position is that their quantisation is lossless. Validate that on your own evals before sizing a cluster around it.
• Nemotron licences differ within the family. Ultra was reported under the Linux Foundation’s OpenMDW-1.1; Super’s model card says “NVIDIA Nemotron Open Model Licence.” Check per-model, not per-family.
• Mistral has a cheaper self-hostable option than Large 3. Medium 3.5 is a dense 128B under a modified MIT, self-hostable on as few as four GPUs. For most deployments that is the more realistic Mistral.

The 5 you pay to self-host

This list is shorter than it looks, because the commercial self-hosting market is mostly platform licensing rather than model licensing. You are rarely buying weights; you are buying a supported appliance, a licensed serving stack, or a hosted-but-isolated instance. Priced entries are from vendor pricing pages — “contact sales” means exactly that, and I have not invented numbers to fill the column.

Offering names link to the vendor’s own page — there is no download link for these, which is rather the point.

Be clear about what is being sold. Four of these five are selling support, indemnity and an appliance — not access to weights you could not otherwise obtain. The NIM container is a free pull and the Nemotron weights are openly licensed, so $4,500/GPU/year buys the licence, the optimised engines and the enterprise indemnity; vLLM plus the same weights costs nothing. Reporting also indicates Palmyra X6 is a post-training variation on Z.ai’s MIT-licensed GLM-5.2 — so there too, the base model is a free download and what Writer sells is the post-training, harness, support and indemnity. That is a legitimate product, and for a regulated buyer “someone is contractually liable for this” can be the entire value proposition. Just price it against the free path deliberately rather than by default.

The one exception is Gemini on Google Distributed Cloud — the only entry here buying you a model you genuinely cannot otherwise run on your own hardware. It is also the one with no published price.

What you still cannot self-host at any price

Anthropic’s Claude and OpenAI’s GPT models have no self-hosted product. ChatGPT Enterprise offers regional data residency, not on-premises. The one exception on record is bespoke: OpenAI deployed a reasoning model into a US national laboratory as a custom air-gapped installation, with weights physically carried into the facility. That is not a SKU you can order. Google is currently the only US frontier lab selling an on-premises product. If your requirement is a US frontier model inside your own perimeter, that is the entire market.

Caveats I’d want stated if I were reading this

• The secondary sources here are heavily contaminated. Several confidently-written articles describe releases that appear on no first-party channel — a “gpt-oss” refresh with an 8B dense model was the clearest case — and invented version numbers for major labs are common. If a spec sheet doesn’t trace to a vendor channel, don’t trust it. Meta is absent from the tables above for a real reason: its most recent open-weight generation remains Llama 4 (Scout 109B/17B active, Maverick 400B/17B active, April 2025) under the source-available Llama 4 Community Licence, and its 2026 frontier work appears under the Muse brand with no published weights.
• Benchmark numbers are deliberately sparse here. Aggregator rankings disagree violently on which model leads, and the Hugging Face Open LLM Leaderboard — the old common reference — is archived. Parameter counts, checkpoint sizes and licences are checkable facts; September 2026 leaderboard positions are not.
• (est.) figures are arithmetic, not measurements. They ignore KV cache, activation memory and framework overhead, so treat them as a lower bound — typically 20–40% optimistic against a real deployment. Benchmark on your own stack before signing a hardware PO.
• Licences drift between versions, as the gotchas section shows. Check the licence on the exact revision you deploy, not the family.
• Sub-32B models are out of scope but worth knowing for dev boxes, eval harnesses and edge tiers: Qwen3.6-27B, Gemma 4 (five variants, 2.3B–31B) and gpt-oss-120b are all Apache 2.0 and all run on one GPU.

My take

For professional deployment the interesting question isn’t “what’s the most powerful model I can download” — it’s “what’s the most capability I can keep usefully busy on hardware I can finance.” Those have different answers, and the gap between them is the whole story of self-hosting in 2026.

On free weights: the small-node tier is where the value is concentrated. DeepSeek-V4-Flash-0731 (166.9 GB, MIT, 1M context, 2× H200) and Nemotron 3 Super (~124 GB, 1M context, 2× H100) deliver most of what the frontier tier does at roughly a tenth of the memory footprint, and both fit a node you can actually saturate. Buying into the 1.5 TB+ tier only makes sense when sovereignty is a hard requirement rather than a preference — otherwise you will own an underutilised rack and a model whose marginal capability you can’t monetise.

On licences: Mistral Large 3 and Cohere Command A+ under plain Apache 2.0 are more significant than their benchmark positions suggest. The trend at the top is toward bespoke licences with revenue thresholds and use restrictions — GLM MIT → bespoke, Kimi modified-MIT → bespoke — and it runs fastest where capability is highest. A 675B Apache-2.0 model with no rider is worth real capability in trade if you are shipping a product rather than running an experiment, and DeepSeek’s MIT licensing at genuine frontier scale is the most underrated fact in this whole survey.

On paid self-hosting: price it honestly against vLLM plus openly-licensed weights, because in the Nemotron case that is literally the comparison and in the Palmyra case the base model appears to be a free download too. What you are buying is liability transfer and a supported stack. That is often worth $4,500/GPU/year — just make it a decision rather than an assumption.

Further reading

• DeepSeek V4 release notes — primary source for V4-Pro/V4-Flash specs
• deepseek-ai/DeepSeek-V4-Flash-0731 — the 166.9 GB MIT checkpoint
• moonshotai/Kimi-K3 — architecture and MXFP4 details
• Qwen/Qwen3.8-2.4T-A95B — and the discussion thread on what the open weights omit
• Serving Qwen3.8-2.4T-A95B on NVIDIA GB300 NVL72 — what the top tier actually requires
• zai-org/GLM-5.3 — and note GLM-5.2 for the MIT licence
• Introducing Mistral 3 — Apache 2.0 across the family
• NVIDIA Nemotron 3 Ultra — open data and recipes, not just weights
• nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 — the 2× H100 model card
• MiniMax M3 announcement
• Run Gemini and AI on-prem with Google Distributed Cloud
• Cohere Model Vault pricing
• NVIDIA AI Enterprise licensing and pricing
• Dell + SpaceXAI: Grok on-premises