A 27B model on one RTX 3090
· 4 min read
Most of my local inference runs on one RTX 3090 with 24 GB of VRAM. This post covers what fits on it, what doesn't, and why the limit is usually not the model weights.
One post tagged with "local-llm"
View All Tags