Computer for local AI

To get started with local AI, the computer you already have is enough: you can run small models on it, though not at great speed. If you want to get serious, what matters most isn't the processor but the graphics card and, above all, how much memory of its own it has (VRAM), because that determines the model size you can load comfortably.

What confuses beginners is looking at the usual specs. A powerful processor helps little if the model doesn't fit in the graphics card's memory; when it doesn't fit, the machine falls back on regular memory and everything gets much slower. That's why it's worth thinking in tiers: an ordinary laptop for tinkering with small models, a PC with a mid-range graphics card for working comfortably, and machines with more graphics memory for large models. Computers with unified memory, like Macs with Apple chips, are a separate case that also works.

Before buying anything, the prudent thing is to try with what you have and see where it falls short; that way you know what you're really missing and not what the ad says. Which machines I use, what I bought and what I already had, with three entry tiers and what you can do on each, I explain in "What hardware you need to run your own artificial intelligence". The mistakes I made setting it up are there too, and they're exactly the ones you'll want to avoid repeating.