Why people run models locally

A thread asked why people run models on their own machines. Cost is the answer most of us expect, with privacy close behind. The most upvoted comment in the thread was two words long: "It's fun." The next one down gave eight reasons, and fun appeared in three of them.

That is worth taking seriously, because the reasons people give have very little to do with a hardware budget.

The comparisons people reach for are gardening and hobbies

One commenter said running a model at home is like growing tomatoes when the supermarket is two minutes away. Another compared it to the money people spend on car parts or fishing gear. A third said renting a GPU feels like using a computer in a library, because you cannot take a machine apart that you do not own. Having your own means you can break something at 3am without asking anyone first.

Privacy runs underneath almost every answer. People write about trade secrets, private documents, and the plain preference for keeping their work off someone else's server. One commenter noted that the large providers get targeted constantly, while a home machine is a much smaller target.

The work that cloud models refuse

Reverse engineering, vulnerability research, security testing, data scraping and OSINT come up again and again. One user said his projects would be rejected by cloud models immediately, and that the value of his results had already paid for his DGX Spark several times over. Another does the same work with local models because the frontier services block it outright.

Where the money actually goes

The financial argument appears in a specific shape. It is rarely about a single API call. A coding agent resends the whole conversation at every turn, so a session of sixty thousand tokens is prefilled sixty thousand tokens at every step. Across a day of work the bill grows, while the local cost per turn is electricity. Large batches of documents behave the same way. One commenter said he had processed billions of tokens locally and had no intention of paying API prices for it.

Consistency matters to anyone who runs evaluations. A provider can move from fp8 to fp4 without announcing it, and one commenter said he suspects providers quietly serve quantized models. A file downloaded to your own disk stays the same tomorrow, so prompts and benchmarks do not drift when the model behind a name is swapped.

Independence shows up as well. Several people described the relief of having something that works with no internet connection and no company able to change the terms. One comment invoked the prepared survivors in films about the end of the world, half as a joke.

The hardware was already in the house

Most people in the thread were not buying equipment for inference. Gaming PCs, older servers and mining era GPUs were already there. A 4090 bought for 4K gaming now runs a Qwen 3.8 27B model for a friend of one commenter. A GTX 1660 Super, kept in a home server for video transcoding, generates over 40 tokens a second. A model of that size fits on a single consumer card and trades blows with what was considered top class less than a year ago.

Learning shows up too. One commenter said his knowledge of local models came up in a job interview and helped him get the role. Another builds home labs because adjusting the solution is as satisfying as having it.

What ownership changes

The question was framed around money, and the answers were about ownership. People want to keep what they write, they want a machine that behaves the same way today as it did yesterday, and they want to be able to open it up and see what happens. Access through a subscription lasts as long as the company keeps selling it.

One commenter was ready to cancel a ten dollar GitHub Copilot subscription he had barely touched in two months, after setting up llama-swap and running Qwen and Gemma models on hardware he already owned.