Every local-AI conversation starts with the processor. Which CPU, which GPU, how many TOPS. Then the model loads, the machine starts swapping to disk, and the real constraint becomes obvious: it was memory all along.
Running a model locally is a memory problem before it is a compute problem. If the weights and the context cache do not fit in RAM, the machine does not run the model — it runs a much slower approximation of it, paging to an SSD.
So here is the arithmetic, for the three model sizes people actually ask about — 7B, 14B and 32B — and an honest answer about which of them a mini PC can handle.
Key takeaways
- A 7B/8B model at Q4 needs about 10 GB in total once Windows and context are counted. 12 GB works, 16 GB is comfortable.
- A 14B model needs about 15 GB. That fits a 24 GB mini PC comfortably and a 16 GB one only just.
- A 32B model needs about 27 GB. No current BMAX mini PC can run it properly, because memory is soldered at 24 GB maximum.
- Q4_K_M is the sweet spot: roughly 40% smaller than Q8 with a small quality loss. Below Q4, the loss becomes obvious.
- Electricity is not the constraint: an always-on AI box costs about $19–26 a year at US rates.
In this guide
The memory math
Three numbers decide whether a model runs:
- Weights. The model file itself. At Q4, roughly 0.65 GB per billion parameters — about 4.8 GB for a 7B model, 9 GB for 14B, 20 GB for 32B.
- Context cache. The working memory for the conversation. It grows with context length; 1–4 GB is typical for 8k–32k tokens.
- The operating system. Windows takes about 4 GB before any AI software starts. A minimal Linux install takes 1–2 GB.
Add them up and the picture is much less generous than the model's download size suggests:
| Model | Q4 weights | + OS & context | Realistic minimum | BMAX option |
|---|---|---|---|---|
| 3B | 1.9 GB | ~7 GB | 8 GB | B3 Turbo |
| 7B / 8B | 4.8 GB | ~10 GB | 12 GB tight, 16 GB comfortable | B7A Plus / B11 Pro |
| 14B | 9.0 GB | ~15 GB | 16 GB tight, 24 GB comfortable | B12 Power |
| 32B | 19.9 GB | ~27 GB | 32 GB | None — soldered at 24 GB |
| 70B | 40 GB | ~48 GB | 64 GB | None |
Weights are typical Q4_K_M GGUF sizes. Overhead assumes Windows plus a 8k–32k context. Individual models vary with architecture, layer count and context settings.
What fits in which machine
8 GB (B3 Turbo) — 3B models. Small assistants, summarizers, translation. A 7B model will load at Q4 but leaves almost nothing for Windows, so it will swap and crawl. Use this tier for narrow tasks, not for a general chat model.
12 GB (B7A Plus) — 7B at Q4, tight. This is the entry point for a usable general model. Expect to close the browser while it runs, and keep context short.
16 GB (B11 Pro) — 7B/8B comfortable. The sensible tier for a single local model used alongside normal work. A 14B model will load at Q4, but you are within 1–2 GB of the ceiling, which is where stutter and slow first tokens come from.
24 GB (B12 Power) — 14B comfortable. This is the sweet spot for local AI in a mini PC. A 14B model at Q4 plus a 16k context fits with room to spare, and you can still keep a browser open.
The honest limitation: a 24 GB machine does not run a 32B model well. The weights alone are about 20 GB, leaving 4 GB for Windows, the context cache and everything else — not enough. It may load and produce a few tokens per second by spilling to disk, but that is a demo, not a workflow.
Quantization
Quantization is how a 14B model fits in 9 GB instead of 28 GB. It reduces the precision of the weights — from 16-bit down to roughly 4-bit — and the quality cost is smaller than the number suggests, until it is not.
- Q8_0 (15 GB) — near-lossless, but it doubles the memory requirement for a quality difference most people cannot hear in text output.
- Q6_K / Q5_K_M (10.5–11.5 GB) — a good compromise if you have the headroom and care about code or long-form writing.
- Q4_K_M (9 GB) — the default for a reason. About 40% smaller than Q8 with a small, usually acceptable loss.
- Q3 and below — noticeable degradation: repetition, broken reasoning, bad instruction-following. Worth it only to squeeze a model into a machine that otherwise cannot run it.
What about the NPU?
Every Core Ultra mini PC advertises an NPU and a TOPS number. It is worth understanding what that number does and does not do for local AI.
The NPU is a low-power accelerator designed for sustained, always-on workloads: background blur, voice isolation, some photo and video effects. It is efficient at those tasks. What it is not, in 2026, is the main path for running a language model.
- Most local LLM software does not use the NPU. Ollama, LM Studio and llama.cpp run on the CPU and, where available, the GPU. NPU backends exist but are limited and model-specific.
- TOPS does not predict tokens per second. Language model inference is limited by memory bandwidth and capacity far more than by peak accelerator throughput.
- The NPU still matters for video calls, and it is why these chips feel responsive in everyday work — just not for the thing people buy them for.
So when a spec sheet says "AI PC, 40 TOPS", the honest translation is: this machine has an accelerator for background tasks, and it can run a language model if it has enough RAM. The RAM is the part to check.
Speed expectations
We are not going to quote tokens per second, because it varies by model, quantization, context length and driver version. What is consistent:
- 7B–8B on integrated graphics feels interactive — fast enough for chat, drafting and summarisation, not instant.
- 14B is roughly half the speed of 7B on the same hardware, because there is twice as much data to move.
- Memory bandwidth matters more than TOPS. A model that fits in RAM runs at a usable speed even on a modest iGPU; a model that spills to disk runs at a fraction of that, regardless of how fast the chip is.
- Long context slows everything down, because the cache grows and attention work increases with every token.
If speed is the priority, the answer is a smaller model, not a bigger computer. A 7B model at Q4 that fits comfortably will feel better than a 14B model that is constantly at the edge of the memory ceiling.
Power and heat
An AI box is usually left on so it is ready when you need it. At idle, a mini PC draws around 12 W — about $19 a year. Two hours of inference a day averages 17 W, or $26 a year. Even running inference continuously, you are at roughly $122.
Compare that with the hardware: the difference between a 16 GB and a 24 GB model is $290. Electricity is not the reason to choose one tier over another — memory is. This is the same pattern we found across the range in our power consumption test.
Heat is the practical concern. Sustained inference is a sustained all-core load, and a compact chassis will spin its fan. If the machine sits on your desk, a 15 W chip is much easier to live with than a 45 W one — which is why the B11 Pro is a reasonable AI box despite the smaller memory.
Software setup
You do not need a framework to get started. Three tools cover most of it:
- Ollama — command-line, one command per model, handles quantization automatically. The fastest way to test what fits.
- LM Studio — a desktop app with a model browser and a chat interface. Good for Windows users who want to see memory usage live.
- llama.cpp — the engine underneath both. Worth using directly if you want to tune context size and GPU offload layers.
Two settings matter more than the tool:
- Context size. Dropping from 32k to 8k can free 2–3 GB, which is often the difference between a 14B model fitting and not.
- GPU offload layers. Moving some layers to the iGPU frees system RAM and speeds up generation. More layers is not always better — VRAM is shared with system memory on these machines.
If you are comfortable with Linux, run the model there. A minimal install leaves 2–3 GB more headroom than Windows, which is a full model tier in some cases. Our guide to how long a mini PC lasts also applies: sustained inference is the workload that ages a fan fastest.
Which BMAX fits
| What you want to run | Pick | Why |
|---|---|---|
| Small 3B assistants, summarising, translation | B3 Turbo — 8 GB, $219.99 | Enough for compact models, and the cheapest way to experiment. |
| 7B/8B chat models alongside normal work | B11 Pro — 16 GB, $459.99 | 16 GB leaves real headroom for Windows and a 8k context. |
| 14B models, longer context, multi-tasking | B12 Power — 24 GB, $749.99 | The only tier that runs 14B comfortably; 16 cores help with prompt processing. |
| 32B and above | Not a mini PC | You need 32–64 GB of upgradeable memory, which means a different form factor. |
One buying note that applies to every tier: memory is soldered on all of these models. There is no upgrade later, so the decision is permanent. If you are unsure between 16 GB and 24 GB, and local AI is part of the plan, the 24 GB machine is the one that will still be useful in three years.
FAQ
Can a mini PC run local AI?
Yes, up to about 14B parameters at Q4 quantization on a 24 GB model. 7B and 8B models run comfortably on 16 GB. Anything at 32B or above needs more memory than a compact mini PC provides.
How much RAM do I need for a 7B model?
About 10 GB in total once Windows and context are counted, so 12 GB is the minimum and 16 GB is comfortable. With a minimal Linux install, 8 GB can work for a 7B model at short context.
Can a 24 GB mini PC run a 32B model?
Not properly. A 32B model at Q4 is about 20 GB of weights, leaving too little for Windows and the context cache. It may load by paging to disk, but the speed makes it impractical for real work.
Do I need a GPU for local AI?
Not a discrete one. Integrated graphics help by offloading some layers, but memory capacity matters more than graphics power. A model that fits in RAM runs usably on an iGPU.
What speed should I expect?
Interactive for 7B–8B models, noticeably slower for 14B, and impractical for anything that spills to disk. Memory bandwidth is the limiting factor, not peak compute.
Is Q4 quantization good enough?
For chat, summarising and drafting, yes — the quality difference from Q8 is small. For code generation or precise reasoning, Q5 or Q6 is worth the extra memory if you have it.
Windows or Linux for local AI?
Linux, if you are comfortable with it. A minimal install uses 1–2 GB instead of about 4 GB, which frees enough memory to move up a model tier in some configurations.
Which BMAX mini PC is best for local AI?
The B12 Power with 24 GB for 14B models. The B11 Pro with 16 GB is the better value for 7B and 8B models used alongside everyday work. The B3 Turbo is enough for 3B models and experimentation.
The bottom line
Local AI on a mini PC is not a hardware fantasy, but it has a ceiling, and the ceiling is memory. At 24 GB you can run a 14B model well. At 16 GB you can run a 7B or 8B model comfortably. At 8 GB you are running small task-specific models, not a general assistant.
Buy the memory you need now, because it is soldered and permanent. Then pick the quietest machine at that tier — you will be sitting next to it while it thinks.
24 GB, 16 cores, 14B models. The BMAX B12 Power is the only tier in the range that runs a 14B model at Q4 with context to spare.
Sources
- Model weight sizes: typical Q4_K_M GGUF file sizes for 3B–70B parameter models (for example Qwen2.5 and Llama 3.1 families). Values are approximate and vary by model architecture.
- Overhead: 4 GB for Windows 11, 1–2 GB for a minimal Linux install, plus 1–4 GB of KV cache at 8k–32k context.
- Power and cost: 12.4 W idle from published B12 Power measurements; 17 W and 80 W modelled for mixed and continuous inference; electricity at 17.45 ¢/kWh, EIA Electric Power Monthly, January 2026.
- Memory configurations and prices: bmaxbuy.com product pages for B3 Turbo, B7A Plus, B11 Pro and B12 Power, October 2026.
Memory figures are planning estimates, not benchmarks. Actual requirements vary with model architecture, quantization, context length and the software stack. All BMAX mini PCs in this guide use soldered memory.
