Local AI
VRAM and large language models explained
How much memory a model really needs: weights, quantisation, the KV cache and overhead, with every figure computed by a script you can run.
9 min read
Topic
VRAM
Every ByteOrbital article about VRAM.
1 article