Running AI locally

Running AI locally means running an artificial intelligence model on your own computer or server, instead of using it over the internet on the servers of a company like OpenAI or Anthropic. What changes is where everything happens: your data stays at home, you can work offline and you don't depend on someone else's service changing its terms or prices.

In exchange, you give some things up. You can't install ChatGPT or Claude on your PC, because those models can't be downloaded: you use open models, like those from DeepSeek, Qwen, Llama, Gemma or Mistral, and the ones that fit on a normal machine tend to be smaller and less clever than the big paid ones. In the cloud you pay per use or by subscription; locally, you pay in hardware, electricity and hours of wrestling with it. What confuses people is treating it as all or nothing: you don't have to pick just one, you can split tasks between the two.

In my case, local AI lives alongside the cloud, it doesn't replace it: I work mainly with Claude, and on my machines I use Ollama, LM Studio, llama.cpp and Unsloth with more than 50 models. I tell all of that in several articles: in "What hardware you need to run your own AI" I talk about the hardware, in "What surrounds the model so it's actually useful" about the utilities that surround the model and in "What good is having AI at home?" about what it's really good for, which is the question that matters.