What hardware you need to run your own AI

Having your own AI at home sounds like science fiction and, partly, it is. Here I show you the hardware I run models on at home, what really calls the shots (spoiler: the graphics card's memory) and what you can do with three levels of machine. The idea is for you to know whether your computer is good enough to start and what you shouldn't rush out and buy.

"Install your own AI for free in five minutes." The video is twelve minutes long, the presenter smiles a lot and at the end there's a chat running on their laptop that answers like an oracle.

The first time I tried it, the five minutes turned into a whole afternoon: the model download took over an hour, the program didn't recognise the graphics card and, when it finally answered, it wrote one word every two seconds, like a hungover telegraph operator. Not ChatGPT, not at home, not in five minutes. Three mistakes in one sentence, quite a record.

Here I tell you what happened next, when I stopped believing the video and started understanding what I was doing. I'm only talking iron: machines, graphics cards and memory; the models, the programs and the rest have their own articles.

I had several reasons, not all of them noble. The first, privacy: I work with client documents that I'm not at all keen on sending to other people's servers. The second, the cost of repetitive tasks, which in the cloud you pay for on every call. And the third, pure old-tinkerer curiosity: I wanted to see how these models work inside.

But first let's clarify what this is. Local AI is a model that runs on your machine without sending anything out. The model, simplifying a lot, is the "brain" you download: a big file that does the answering.

What running AI locally is not: installing ChatGPT or Claude on your PC. Those models can't be downloaded. What you use are open models, which their creators publish so anyone can download them, like those from DeepSeek, Qwen, Llama, Gemma or Mistral. They're almost always "open-weight": you can download the finished model, but not necessarily the code and data it was trained with, which is what it would take to call it truly open source. And each one comes with its own licence: some let you do almost anything and others set conditions of use, so check it before using it for work. And the ones that fit on a normal machine are smaller and less clever than the big cloud ones.

In my case, local lives alongside the cloud, it doesn't replace it. I work mainly with Claude, and the models at home are one more piece of the setup, not the centre.

I started in spring 2024, with Ollama, on my work computer at the time, running Windows: a desktop PC with a decent processor, 16 GB of RAM and a graphics card a few years old with 4 GB of memory. I had the RTX 4070 server for other things and it didn't even occur to me to use it. It seemed enough. I had a good processor, what could go wrong?

I tried a 13B model, that is, one with about thirteen billion parameters, because the forums said it was "the good one", in Q4 and with the context Ollama came with by default. And what had to happen happened: it didn't fit on the card, the program moved a good chunk into normal memory and the answer came out at a little over one token per second. On the third long message the computer froze and I had to hard-reset it.

A token is the unit a model reads and writes in: a piece of a word, sometimes a whole word or a punctuation mark. That's why speed is measured in tokens per second; in English, a token works out at roughly three quarters of a word.

I tried a smaller one, a compressed 7B Llama, and things changed: it answered at an acceptable speed, about ten tokens per second, though with noticeably weaker answers. There I was watching the processor, which was barely breaking a sweat, while the bottleneck was somewhere else.

That's where I learnt the lesson that leads into the next section: in my case it wasn't processor I was short of. It was graphics memory.

VRAM is the graphics card's own memory, and it's the part that decides what you can do. Think of it as the size of your workbench. If the whole model fits on the bench, the card works fast. If it doesn't fit, part of it goes to the computer's normal memory, which is far away, and everything crawls. Not a little: a lot.

So if you're wondering what graphics card you need for local AI, the answer isn't raw power but memory. Sometimes a modest card with more memory is a better deal than a faster one with less. And the processor, king in other tasks, matters less here than it seems as long as the model fits on the card. When it doesn't, things change: the amount of RAM and its speed come into play, as do how much of the model goes to normal memory, the context (the more text you give it, the more memory it takes), the quantisation and the program running it. That's where a better processor and memory do make a difference, though they don't work miracles.

On brands, reviewed in September 2026, as this changes fast: NVIDIA cards are still the most compatible with AI tools. AMD cards work with Ollama, LM Studio or llama.cpp, but not every card model is equally well supported, especially on Windows; check your program's compatibility list before buying. Intel cards work mainly through llama.cpp and what's built on top of it, with less of a track record. Macs with Apple chips are a case apart: they use unified memory, shared between the processor and the graphics, and they hold their own. I've only tried NVIDIA, so on the others I'm going by the documentation, not experience.

Two words you'll see a lot. Sizes, like 7B or 14B, give the billions of parameters in the model, which are, so to speak, the pieces it "thinks" with: bigger is usually smarter and heavier. And quantisation, which is compressing the model so it takes up less space at the cost of losing some quality. I go into all this in More than fifty models tested at home.

An example of mine, on the same machine, the server with the RTX 4070 with 12 GB of VRAM and 64 GB of RAM: an 8B Llama 3.1 fitted entirely and answered at about thirty-five tokens per second; a 32B Qwen 2.5 didn't fit, was split with normal memory and dropped to one or two tokens per second. Both in Q4_K_M, with Ollama, a 4,096-token context and the same question. Same machine, same program, about twenty times slower. That's the difference between fitting and not fitting.

If you're looking for which computer to buy for local AI, start here: look at the graphics memory before anything else.

This is what runs models at home: the only machine with a graphics card in my internal network of 8 servers, plus the laptop.

| Work laptop (HP Omen) | Laptop RTX 5070 | 8 GB | 32 GB | Trying models in LM Studio before moving them to the server, and away from home |

The rest of the network is mini PCs and old recycled machines doing other things: services, agents, backups. They don't run models, and they don't need to.

What I bought and what I already had: I built the main server in December 2023, with a 20-core Intel i7 processor and the RTX 4070 bought new; the complete server, card included, came to around €2,300. I bought the laptop for work, so I don't count it as a local AI expense. I do those sums in What good is having AI at home?. I'm not giving you market prices because they change every week.

An ordinary laptop. You can run small models, with patience. Summarising a long email or rewriting a paragraph with a 3B or 4B model works fine, though don't expect wonders. It's useful above all for finding out whether this interests you before spending a single euro.

A PC with a mid-range graphics card. Here you can work comfortably with small and medium models. With my 12 GB I classify documents, extract data from invoices and draft texts at a good speed. This is the level at which local AI starts to be genuinely useful.

My setup. With a machine dedicated only to this and plenty of RAM behind it I can run somewhat bigger models, if slowly, and do more serious tests, like fine-tuning a small model with my own examples. What it still doesn't let me do is get close to the big cloud models: on hard reasoning or long code tasks, the gap is still huge. Don't fool yourself: you won't have a Claude at home.

The central advice of the whole article: try with what you have before buying. That way you see which models are too big for you and know how much memory you're really short of, not what the ad says.

If you want to install an AI on your PC today, the simplest way is to download LM Studio or Ollama and, from there, pull a small model from a well-known family, such as Qwen, Llama or Gemma. No programming needed. Patience with downloads is, since they weigh several gigabytes.

And one rule that will save you grief: if everything is painfully slow, it's almost always because the model is too big for your machine. Drop to a smaller or more compressed one before blaming the program. The details of each program are in Ollama, LM Studio, llama.cpp and Unsloth, which I use for what.

I bought a power supply that was too tight and had to replace it two months later. A 4070 can draw about 200 watts on its own, with spikes above that, so the power supply needs to be sized with headroom. I underestimated the heat: in summer the server sounded like a hairdryer, the card went over 80 degrees under load and I had to move it out of the office. In the end I lowered its power limit a little: I lost some speed and gained quiet and a few degrees.

On the used card, which is what I recommend, with its risks: it may have had a hard life, for example mining cryptocurrency, with no warranty and worn fans or thermal paste. If you go that way, ask to see it run a half-hour stress test before paying, and expect to replace the thermal paste. And for weeks I left models loaded in memory without using them, with the electricity bill taking notes. Today, knowing what I know, I'd have started straight away with a used card with lots of memory, instead of paying for speed I didn't need.

Verdict on the promise? Installing a model at home can indeed be done in a short while, and for free if you already have the hardware. What's smoke is the "your own ChatGPT" bit: what runs at home is smaller, slower and more limited. It has cost me about €2,300 on the server and dozens of afternoons, and I don't regret it, but I wouldn't sell it to you as a shortcut either.

Which models I've put on these machines and which have survived, I tell you in More than fifty models tested at home.