Best Local LLM for Coding on 16GB VRAM: Qwen3.8 27B, Not This Week's Drops

Published 20 October 2026

On a 16GB graphics card, the best local model for coding is Qwen3.8 27B in a 3-bit build. It came out in August, not this week. Four big open-weight models came out from 15 to 17 October. Even their smallest builds need 22 GB to about 265 GB of memory, so none of them fits a 16GB card. Only NVIDIA’s Nemotron 3.5 Bolt comes close on a 24GB card: its file fits with under 4 GB to spare.

A model runs locally only if its file and its context fit in memory

“Runs locally” means a model can run on hardware you own. It does not mean your hardware. Here is what decides it.

A model is a large file of numbers called weights. To answer you, your graphics card has to hold the whole file in its own memory, called VRAM. A smaller “build” of the same model stores each number in fewer bits. A 4-bit build is about a quarter of the 16-bit original, and a 2-bit or 3-bit build is smaller still. Each step down loses some quality.

The card also needs room for the context: the code, files and chat you feed the model. That cache grows with every token. A coding assistant that reads a few files can use 32K tokens (32,768) or more.

llmspeed, an open-source speed calculator, uses a simple rule we follow in this post. It adds the file, the context cache and 0.6 GB for the runtime. If the total fits in 93% of the card’s memory, the model fits (llmspeed). On that rule, an 8GB card has 7.44 GB to use, a 16GB card 14.88 GB, and a 24GB card 22.32 GB. The rule is cautious. Card makers count memory in slightly bigger units than file sizes use: a card sold as 24GB holds 25.77 GB in file-size units. So a model a little over the line may still run.

Here is what this week’s four releases need:

Sources: Meta, the Muse model card, NVIDIA, the Bolt model card, the DeepSeek model card and llmspeed’s data. Build sizes are the file listings on Hugging Face. We added up the shard files where a build is split.

“Active per token” matters for speed. These are mixture-of-experts models: each token uses only some of the weights. But every weight still has to sit in memory, so the total sets what fits.

Bolt fits a 24GB card on paper, but NVIDIA’s own setups assume 32GB or more

Bolt’s file is 22.02 GB. A card sold as 24GB holds 25.77 GB in the same units, so 3.75 GB is left for the context cache, the runtime and your screen. If you also load the add-on that lets Bolt read images (1.73 GB), 2.02 GB is left (Bolt model card). By llmspeed’s cautious rule, Bolt is already over a 24GB card’s usable memory before you add any context. Counted in bytes, the file fits. How much context fits on top is the open question.

NVIDIA says it designed Bolt to stay under 24 GB with 128K tokens of context. But the two setups on its model card are for bigger machines. The smaller one, a “28GB recipe”, is for cards with 32GB or more and gives 65,536 tokens of context. Both setups add a 1.00 GB helper model that speeds up answers.

Two more limits on the card. It lists only two chip families: Ada Lovelace, which includes the RTX 4090, and Blackwell. The RTX 3090 is an older family, Ampere, so it is not on the list. The card also lists only Windows and Linux. We found no published test of Bolt on a 24GB card.

On 8GB, Qwen3.5 9B is the coding model that fits

At 4-bit, Qwen3.5 9B is a 5.68 GB file. With 32K tokens of context, it needs 7.35 GB of the 7.44 GB an 8GB card can use. That is tight, so drop to 16K tokens (6.82 GB) if other apps use the card. It scores 11 on Artificial Analysis’s Intelligence Index, a combined test of reasoning, knowledge and coding tasks (Artificial Analysis).

Two larger models do not fit:

• Gemma 4 12B (14 on the index) is a 6.98 GB file. It needs 7.81 GB even at 8K tokens. Part of it would spill into your computer’s main memory, which is much slower.
• Qwen3.8 27B at 2-bit is 7.27 GB and needs 8.40 GB at 8K tokens.

llmspeed lists no 8GB card. On its closest card, the 12GB RTX 3060, Qwen3.5 9B runs at 13 to 42 tokens per second. A token is a word or part of a word.

NVIDIA’s smaller Bolt, a 4GB model called E4B, is due on 23 October. It was not out when we checked.

For images, FLUX 3 Klein from Black Forest Labs fits an 8GB card. It has 5.6B parameters. Its weights take 11.2 GB at 16-bit and 5.6 GB at 8-bit. BFL says its INT8 build “runs on NVIDIA GeForce RTX GPUs with as little as 8GB of VRAM” (BFL). BFL gives no speed for an 8GB card. Its slowest listed card, the RTX 4070, made a 1-megapixel image in under 9 seconds.

On 16GB, Qwen3.8 27B at 3-bit is the strongest model that fits

No published coding score covers all of these small models. Artificial Analysis’s coding page shows none of them. So we rank by its Intelligence Index, which includes coding tasks, then check memory at 32K tokens of context.

“Needs” uses llmspeed’s rule, with its context-cache size for each model family. File sizes come from the Unsloth and llmspeed listings. Qwen3.8 27B scores 34 at its highest reasoning setting, 28 at medium and 20 with reasoning off. At medium, it still beats every other model in the table. On its own model card, Qwen reports 73.0 on Terminal-Bench 2.1, a test of coding in a terminal (Qwen).

One caveat could change the pick. Artificial Analysis tests the full model, not a 3-bit copy. We found no published test of how much Qwen3.8 27B loses at 3-bit. Treat the 34 as a ceiling.

The 3-bit files come from Unsloth, which publishes compressed builds of popular models. Get them from Unsloth’s Qwen3.8 27B page. “UD-IQ3_XXS” and “UD-Q3_K_XL” are Unsloth’s names for two 3-bit builds.

Qwen3.8 27B is slow on 16GB, because it reads all 27B weights for every token

Speed depends on memory bandwidth: how many gigabytes per second the card can read. To write each token, the card reads every weight the token uses. Qwen3.8 27B is dense, so it reads all of them. gpt-oss-20b uses 3.6B of its 20.91B weights per token. Qwen3.6 35B A3B uses 3B of 35.95B (llmspeed).

So on the same card:

• Qwen3.8 27B, 3-bit: about 12 to 17 tokens per second on an RTX 5060 Ti (448 GB/s), and 25 to 34 on an RTX 5070 Ti (896 GB/s). This is our projection with llmspeed’s formula, because llmspeed does not list this build.
• gpt-oss-20b: 43 to 137 tokens per second on the 5060 Ti, and 87 to 273 on the 5070 Ti, in llmspeed’s estimates from similar measured setups.

Reasoning makes the gap bigger. A model that thinks for 2,000 tokens at 15 tokens per second takes about 133 seconds before it starts its answer. So use Qwen3.8 27B when you can wait for a better answer. Use gpt-oss-20b or Qwen3.6 35B A3B for quick edits and autocomplete. Qwen3.6 35B A3B scores 18, twice gpt-oss-20b’s 9, and is also light per token.

On 24GB, run Qwen3.8 27B at 4-bit, or try Bolt on an RTX 4090

With 24GB, Qwen3.8 27B fits at 4-bit with 32K tokens and about 3 GB spare (19.21 of 22.32 GB). For its own 4-bit build, llmspeed estimates 12 to 38 tokens per second on an RTX 4090 and 11 to 35 on an RTX 3090.

Bolt scores lower (21) but uses only 9B weights per token, and it reads images and video. NVIDIA’s own table puts it ahead of Qwen3.6 35B A3B on Terminal-Bench 2.1, 50.98 to 44.38. That is NVIDIA’s test, and it ran Bolt at 2-bit against the other models at full 16-bit. Qwen’s 73.0 for its own model came from a different lab and setup, so do not compare the two numbers directly. If you have an RTX 4090 and want a fast agent that can read screenshots, Bolt is worth a try. Keep the context short at first, because nobody has published how much fits. On an RTX 3090, NVIDIA does not list your chip family.

On a Mac, the memory you bought sets the limit, and Muse Spark Open at 4-bit needs two 512GB Mac Studios

A Mac’s graphics chip uses the computer’s main memory, called unified memory. llmspeed counts about 75% of it as usable for a model. So a 24GB MacBook has about 18 GB to work with (llmspeed).

Speeds are at 32K tokens, except the Qwen 27B figure, which is at 8K.

Meta says Muse Spark Open “runs on your desktop”. It means two M5 Ultra Mac Studios, each with up to 512 GB of unified memory, together “a terabyte of unified memory” (Meta). Meta’s model card says the official 4-bit build needs at least 700 GB of GPU memory.

Smaller community builds exist. Unsloth’s 3-bit build is 340.4 GB, under the 384 GB llmspeed counts as usable on one 512GB Mac. Its 4-bit build is 505.5 GB, which needs two Macs. On paper, one Mac Studio can run Muse at 3-bit. We found no published test of that, or of how much quality 3-bit costs.

Bolt is not an option on a Mac from NVIDIA: its model card lists Windows and Linux only.

Muse and FLUX limit big companies; Qwen, Gemma and gpt-oss use Apache 2.0

“Open weights” means you can download the file. The license decides what you may do with it.

• Muse Spark Open (Muse Model License): you must “prominently display ‘Built with Muse Spark Open’” in any product that uses it. If you sell access to AI models through an API, and your company’s revenue tops $20 million in 12 months, you need a separate deal with Meta. Internal use and features inside your own product are exempt. The model is not meant for anyone under 18, and Meta approves each download request by hand (license text).
• FLUX 3 Klein (FLUX Kommunity License): free commercial use for people and companies with annual revenue under $5M. Larger companies need a paid license or BFL’s API (BFL).
• Nemotron 3.5 Bolt (OpenMDW-1.1): no revenue limit and no use limits. Keep the notices and include the license if you share the files. Your rights end if you sue claiming the model infringes your patent or copyright (OpenMDW).
• Qwen3.8 27B, Qwen3.6 35B A3B, Qwen3.5 9B, gpt-oss-20b and Gemma 4: Apache 2.0 on each model card, a standard license that allows commercial use. DeepSeek V4.1 Flash: MIT, which is similar.

This is a summary, not legal advice. Read the license file in the model’s download page before you ship a product on it.

Check speed on llmspeed.dev before you buy, but it does not list this week’s models yet

llmspeed.dev lets you pick a graphics card or Mac and a model. It then streams text at the predicted speed, including the wait before the first word. It is open source.

Each estimate carries a label. “Measured” means someone benchmarked that exact setup. “Derived” means it is interpolated from similar setups. “Projected” means it is calculated from the hardware’s specs. Ranges are wide: gpt-oss-20b on an RTX 5060 Ti spans 43 to 137 tokens per second. Use it to compare two cards, not to promise a number.

Its data, updated 17 October, covers 30 machines and 15 models. Bolt, Super VL and Muse Spark Open are not among them yet. DeepSeek V4.1 Flash is.

You can also estimate speed yourself. Divide the card’s memory bandwidth by the gigabytes the model reads per token. The RTX 5070 Ti reads 896 GB/s, twice the RTX 5060 Ti’s 448, so on the same model it runs about twice as fast. Both have 16GB, so both fit the same models.

LodeHQ’s daily AI newsletter covers new models, tools and research, with links to the original sources.

If you would rather pay for a hosted model, read our comparison of Gemini 4 Argon, Claude Opus 5.5 and GPT-6 Astra, or how ChatGPT and Claude usage limits compare.

How we checked

We read the model cards and file listings on Hugging Face, and the Meta, NVIDIA and Black Forest Labs posts, on 20 October 2026. Quality scores are Artificial Analysis’s Intelligence Index, read the same day. Speeds and memory use come from llmspeed’s open data (dated 17 October) and its published formula. Where llmspeed does not list a build, we ran its formula ourselves and say so. We did not run any of these models. File sizes are in GB of 10^9 bytes, the unit file listings use. Card sizes are as sold, which makes llmspeed’s rule cautious. Our working is in a script that reproduces every memory and speed figure here.