Will this model run on your machine?

A check before you download 40 GB for nothing. The arithmetic is written out in full, and the link keeps your settings.

Installed memory

The processor and the GPU share this memory.

03 Quantization

About 4.9 bits per parameter in a GGUF file.

04 Context

8,192tokens

Prompt + answer. The model accepts 32,768 tokens at most.

05 Parallel requests

One conversation at a time in almost every case.

What fits on this machine

The largest models in the catalog that fit at Q4_K_M with a 8,192-token context.

32layers

4,096width

32heads

8KV heads

32Kvocabulary

33KContext

2 active experts out of 8 — Mixture of experts: all the memory is loaded, but a token only crosses part of it.