GPU for local AI

For local AI, what matters most in a graphics card is its own memory (VRAM), more than its raw power, because it decides what size of model you can load in full. With a mid-range card you can already work comfortably with small and medium models; for large ones you need a lot more graphics memory, or several cards. And if you're only going to try small models, you can even start without a dedicated card, armed with patience.

VRAM works like the size of your workbench: if the model fits entirely, the card runs fast; if it doesn't, part of it stays in the computer's regular memory and everything slows down enormously. That's why a modest card with more memory sometimes pays off more than a faster one with less. Something else that confuses people: not all brands get on equally well with AI tools. NVIDIA cards are the most compatible; others work too, but it's worth checking first that your program supports them.

Before spending money, try with what you have and see which models are too big for you; that tells you how much memory you're really missing. Which graphics cards I use, how much memory they have and how that limits the models I can run I explain in "What hardware you need to run your own artificial intelligence", along with three entry tiers to get started and the mistakes I made building the setup, which may save you an unnecessary purchase or two.