LLM/fit

Method

An estimate is only worth something if you can check it. Here is every formula, every reserve, and what this site cannot do.

The formulas

Model weightsGiB = parameters × effective bits ÷ 8 ÷ 2³⁰

Effective bits are not the name of the format. A Q4_K_M file does not weigh 4 bits per parameter: llama.cpp keeps some tensors at higher precision and adds the quantization scales. The table used here is Q2_K ≈ 3.35, Q3_K_M ≈ 3.9, Q4_K_M ≈ 4.85, Q5_K_M ≈ 5.7, Q6_K ≈ 6.6, Q8_0 ≈ 8.5, FP16 ≈ 16 bits per parameter.

KV cacheGiB = 2 × layers × KV heads × head dim × context × requests × 2 ÷ 2³⁰

Two tensors (keys and values), two bytes per value because almost every engine keeps the cache in 16-bit. This is the term that explodes when you raise the context, and it is nearly always what breaks a load that "should fit".

ActivationsGiB = 0.05 + requests × min(context, 8192) × width × 2 × 3 ÷ 2³⁰

The model’s scratchpad while it writes. The context contribution is capped at 8192 tokens: during inference the engine works in blocks and does not retain the whole graph the way training does.

RuntimeGiB = max(0.75, weights × 10%)

Ollama, LM Studio or llama.cpp keep buffers of their own. This is a flat allowance, not a measurement: an engine may ask for more.

Generation speedtokens/s ≈ bandwidth ÷ (active parameters × bits ÷ 8) × efficiency

Generation is memory-bound, not compute-bound: every token reads the weights again. Real efficiency runs from 20 to 50% of theoretical bandwidth depending on the engine, hence a range rather than a figure. On a mixture-of-experts model, only the activated parameters cross the bus.

The reserves

  • Graphics card: 1 GiB taken out for display, driver and fragmentation.
  • System: the larger of 4 GB or 10% of RAM, never handed to the model.
  • Unified memory (Apple): at most 75% of physical memory, because macOS will not let the GPU take everything.
  • Past 85% of the budget the verdict becomes "it barely fits": the load often succeeds, and fails as soon as another application takes its share.
  • Capacities are taken as the manufacturer states them (24 GB, 16 GB…) and counted in gibibytes, i.e. 2³⁰ bytes — the same unit Ollama or LM Studio reports.

The sources

  • Model architecture: the config.json published by the vendor on Hugging Face — layers, KV heads, head dimension, maximum context. Nothing is typed by hand.
  • Parameter count: the repository’s safetensors index, not the name of the model.
  • Licensed repositories (Llama, Gemma): the configuration file is read from a public mirror whose content is identical. The model line says which one.
  • Graphics cards: RightNow GPU Database, derived from TechPowerUp — memory, bandwidth, release date, with a link to the spec sheet.
  • Apple Silicon: bandwidth as announced by Apple, with a link to the matching press release on every entry.

What it does not tell you

  • This is not a speed test. The tokens/s range is a physical bound, not a measurement on your machine.
  • Software compatibility is not checked: a model can fit in memory and still not be supported by your engine.
  • Latent-attention (DeepSeek V2/V3), hybrid and multimodal architectures are not in the catalog, precisely because their cache formula differs.
  • Sliding-window models (Gemma 3) are estimated over the full context: real usage can be lower.
  • Sharding across several cards is not modelled.
  • Quantization is a theoretical size: a specific file can differ by a few hundred megabytes, and nothing guarantees it exists in that format.