Running local open weight models on your own gaming GPU is a thrilling exercise. The software stack is already quite refined, and there’s a wealth of instruction available (both online and via an agent) to get you started. The largest benefit of doing so, besides the cost savings, is of course the speed.
Local inferencing on gaming GPUs is usually fantastic speed-wise. These cards boast the clocks and memory bandwidth to really get prefill and generation across most common models to scream. Tokens per second are usually always going to be higher here when compared to single AI computers like the DGX Spark or the AMD Strix Halo platform.
The only let down here will be VRAM: the 24GB of video memory on the 3090 isn’t super large. But there are some great models that will fit just fine on it.
If you’re doing anything from coding through to agentic processing, Qwen 3.8 27B is a clear winner in this area for the amount of VRAM. It’s a dense model, so it’ll perform clearly better on a GPU than it would on the compute mentioned earlier. Qwen 3.8 excels at agentic coding and would pair very well with a coding harness like OpenCode or OMP.
Failing Qwen, Gemma-4 31B would be a decent follow-up choice, though primarily for conversational purposes and intelligence.
Setting Qwen up is pretty straight forward these days – Mia, a well known AI-related tinkerer and builder on X, has got you covered with this recipe. You’ll need to be running Linux, with all your drivers installed and up to date (we’d also recommend an OS with a modern kernel, like Ubuntu or Fedora) – once you’re there clone the repository and follow the instructions to run it.