What is AI at home actually good for?

Four articles about hardware, models and software, and the question that matters is still missing: what is all this actually good for? Here's the final count, with what stayed, what I threw in the bin and what it cost me compared with paying for the cloud.

The line goes around videos and forums with small variations: "install your own AI and stop paying". It hooks you for two very human reasons: nobody likes paying a subscription every month, and everybody likes the idea of owning something powerful that's all theirs.

It's a half-truth. Running AI locally means running a model on your own computer or server, instead of using it over the internet on another company's servers. But it isn't installing ChatGPT on your PC: that model can't be downloaded. What you download are open models, which their creators publish so anyone can grab them, and the ones that fit on a normal machine are smaller and less clever than the big paid ones.

I fell for it too. I started with local AI in spring 2024, convinced that within a few months I'd stop paying. That didn't happen, and this article explains why, and also why, despite that, I haven't switched the machines off.

A quick recap of what I've already covered in other articles. In What hardware you need to run your own AI I described the machines I work with and why graphics card memory trumps everything else. In More than fifty models tested at home I went over the models I've been trying and which ones survived. In Ollama, LM Studio, llama.cpp and Unsloth, which I use for what I explained the programs that run those models. And in What surrounds the model so it's actually useful I covered everything you need around a model for it to be useful and not a toy.

For newcomers, the one-sentence definition: local AI is an artificial intelligence model that runs on your own computer, without sending what you type to an outside company.

After all that testing, the uses still alive are few and very specific. I split them into what's in production, which I use for real on real work, and what's still experimental.

Client data I don't want leaving my machines (in production). This is the use that justifies everything else. Classifying and summarising a client's internal documents, such as emails and meeting minutes, which by contract must not leave their network or mine. There, the local model isn't competing with the cloud one: the cloud one simply isn't an option. I do it several times a week, a few hundred documents a month, with a mid-sized model from the Qwen family on the main server, the one with the RTX 4070.

Bulk, repetitive tasks (in production). Tagging thousands of product listings, extracting fields from invoices or cleaning up text before feeding it into another tool. Dumb but long tasks, where paying per query makes no sense because the quality I need is modest. A couple of big batches a month, several thousand items each, with a mid-sized, fast model.

Testing models before using them elsewhere (experimental, but constant). When a new open model comes out, I test it at home with my own examples before considering using it on an outside server or recommending it to anyone. One or two afternoons a month. This doesn't save me money; it saves me from putting my foot in it in front of a client.

Fine-tuning with Unsloth (experimental). Unsloth is for fine-tuning a model, that is, retraining it a little on your own examples so it does a specific task better. I've used it to fine-tune a small model that classifies support queries; it works in testing, but I haven't put it into production yet. I'm calling it what it is: a promising experiment.

In numbers, this is how each use looks:

If you look closely, none of the uses that stayed is "chatting with my homemade ChatGPT". They're all bounded tasks, each with a clear reason not to do them outside.

A word of caution: running locally doesn't guarantee privacy on its own. Some programs send usage statistics, the web interface keeps the conversation history, backups carry that data to another machine and anyone who gets into the server can read it. I turn off statistics wherever I can, delete the history of client jobs when I finish, backups are encrypted and only my user can log into that machine. Local reduces the risk; it doesn't remove it.

The list of what I dropped is longer than the list of what stayed.

The first and most painful: trying to replace the cloud assistant on hard tasks, like programming or drafting long documents. For about three weeks I tried working with local models only. The quality wasn't there, the answers were slow, and I ended up reviewing so much that it took longer than doing it myself. The day it clicked was when I asked a local model to fix a forty-line script and it handed back an eighty-line one that didn't work, with a very confident explanation of why it now did. I went back to Claude that same afternoon, tail between my legs.

The second: a home voice assistant that would understand commands and look things up in my notes. It stalled halfway because the card's memory couldn't hold the voice model and the text model at the same time, and because every update broke something. The maintenance hours ate the excitement.

The third: keeping a big model always loaded "just in case", which took up all the graphics memory and stopped me using the machine for anything else.

The lesson I take away: the model is just one piece, and the ones that fit at home don't play in the big league. It's not that they're bad; they compete in a different category.

On to the numbers, which is where you see whether this is a business or a hobby.

The hardware dedicated to local AI, meaning the complete server with the graphics card, has cost me about €2,300: the RTX 4070 I mentioned when talking about the hardware, the processor, the memory and the rest of the machine. Electricity, about €14 a month extra: the home network uses about €50 a month, measured with a plug-in meter, and I reckon a bit over a quarter of that goes on the server with the graphics card. And the hours, to be honest, more than 200 since I started, between installing, testing, breaking and fixing.

Breaking each item down per month: depreciation, spreading the €2,300 over a five-year useful life, about €38. Electricity, that €14; I later measured the server with the graphics card separately and the numbers add up, about 3 kWh a day. Cooling, nothing extra, I just moved it somewhere else. Maintenance, about two hours a month of the time I spend on the home network. Backups, almost nothing: I don't back up the models because they can be downloaded again; I only keep configurations and scripts. And a residual value: the card could be resold for part of what it cost, which I don't subtract because I don't know how much it will be.

Against that, what I would have paid to do the same in the cloud: working from my real volume of bulk tasks and documents, an estimate of about €40 to €60 a month. The assumptions: the volumes in the table above, which add up to about six million tokens a month between what the model reads and what it writes, and a mid-range model via API, with no batch discounts. I'm not quoting any provider's prices because they change often; the estimate was reviewed in September 2026. And it's theoretical: documents that can't leave the client's network have no price in the cloud.

The conclusion, even if it doesn't flatter me: if I only count money, it doesn't pay off: between depreciation and electricity it comes to about €52 a month, roughly the same as the cloud's €40 or €60, and that's without counting my hours. What tips the scales is something else. Privacy, because there's work I couldn't take on without local. Learning, because understanding how these models work inside has made me better at using the cloud ones. And independence: nobody changes the terms on me overnight.

So "free" is relative. You pay in hardware, in electricity and in hours.

The question I get asked most is what the best local AI model is. My short answer: the most capable one that fits in your memory and does your task well. There's no single best one for everybody.

The ones I still use today:

• A mid-sized Qwen on the main server, for summarising and classifying client documents, and also for the bulk tagging batches.
• Gemma 3 12B, also on the main server, for writing and reviewing texts.
• A distilled DeepSeek model, for reasoning tests and comparisons.

When a new one comes out, I choose the opposite way to how it's usually done: first the task, then the memory I have, and with that I test two or three candidates on my own real examples. Internet leaderboards help me very little. The details are in More than fifty models tested at home.

Three profiles, three answers.

Small business owner with nobody technical. Probably not, unless you handle data that genuinely can't leave your business and you have someone you trust to set it up and maintain it. The sign that it does: a specific, repeated task with sensitive data. The sign that it doesn't: "I want my own ChatGPT".

SME manager with sensitive data. Yes, if there's a specific task you can't do with the cloud today because of privacy or contracts. The sign that it does: a use case that already exists and is blocked. The sign that it doesn't: wanting to set it up before knowing what it's for.

Developer starting out. Yes, as a learning exercise, with hardware you already have. The sign that it does: curiosity about how they work inside. The sign that it doesn't: expecting it to replace the big models in your daily work.

Three decisions I'd change. I'd start with a task and not with the hardware: first the problem, then the machine. I'd test fewer models and measure more, because out of my more than fifty tests, half taught me nothing new. And I wouldn't think about another graphics card until it was clear the one I have really falls short.

What remains is a simple idea: AI at home doesn't replace AI in the cloud, it complements it where the cloud can't go. If you want to see where each piece of all this lives, it's in What runs on each of my machines.