Local AI

Local AI is an artificial intelligence model that runs on your own computer or server, without sending what you type to an outside company. It's mainly useful for three things: working with data you don't want to take out of your home or office, doing repetitive tasks without paying per use and learning how these models work inside.

What usually confuses people is thinking they're going to have "a ChatGPT at home". The models that fit on a normal machine are smaller than the big cloud ones, and it shows on difficult tasks. In exchange, they're yours: they work without internet, their terms don't change from one day to the next and there's no bill per message. You also have to accept that the model is just one piece: you need a program to run it (Ollama, LM Studio or llama.cpp, for example) and, for it to be useful, something around it that connects it to your work.

I use it alongside cloud AI, not instead of it: I have more than 50 models locally and I run them with Ollama, LM Studio, llama.cpp and Unsloth. How it all fits into the way I work I explain in "How I'm working with artificial intelligence". The hardware, the models, the utilities and the verdict on what it's good for I cover in separate articles, such as "What hardware you need to run your own AI" and "What good is having AI at home?". And where each piece runs is in "What runs on each of my machines".