Local AI models

Locally you can use "open-weight" models, that is, the ones their creators publish so anyone can download them and run them on their own machine. There are many families, like Llama, Qwen, DeepSeek, Gemma or Mistral, and each comes out in versions of different sizes. Closed models, like the ones behind ChatGPT or Claude, can't be downloaded: they can only be used over the internet.

The number next to the name (7B, 14B, 70B…) indicates billions of parameters, which, to put it simply, is the size of the model. Bigger usually means more capable, but also more memory. That's why almost all of them are also published in "quantized" versions, compressed so they fit on normal machines in exchange for losing some quality. What throws people at first is finding the same model in dozens of variants with almost identical names; the difference is usually in the size and the level of compression.

I've been testing more than 50 models locally: some have stayed and others I've ended up deleting. Which families I've tried, how I decide to keep one and which ones I use today I explain in "More than fifty models tested at home". If you're starting out, don't download twenty at once: pick a small one from a well-known family, try it with your usual tasks and only go up in size when it falls short.