Local AI
VRAM and large language models explained
How much memory a model really needs: weights, quantisation, the KV cache and overhead, with every figure computed by a script you can run.
9 min read
Topic
Kv Cache
Every ByteOrbital article about Kv Cache.
1 article