Dedicated AI Hardware

The Case Against Renting Inference

Article three — ownership

Renting is a good deal until it becomes a dependency. Here is where the line sits and how to find it for your own workload.

Nobody starts by building a machine. The first experiment goes through an API, because it takes five minutes and costs nothing. That is the correct decision and this page will not pretend otherwise. The question is what happens at month six, when the prototype has become something people depend on, and the meter is still running.

Three costs accumulate quietly. None of them appear on the invoice line called usage.

Cost one: the meter never stops

A rented endpoint charges per token, forever. There is no learning curve that bends the price down, no amortization, and no month where the tenth thousand request is cheaper than the first. The marginal cost is flat and permanent.

Owned hardware inverts this. Buy a machine once, and the price per token falls with every request you make, because the fixed cost is being divided by a growing number. Run it for a year at household volumes and the effective cost per token becomes comically small. Run it for five years and it approaches the cost of electricity.

The crossover depends entirely on volume. Below a few hundred requests a month, renting wins and it is not close. Above a few thousand, owning starts to win, and the gap widens every month afterward. The interesting zone is in the middle, where the answer depends on latency requirements and on whether you value the machine for anything besides inference.

Cost two: your capability is somebody else's roadmap

Rented models get retired. This is not a conspiracy, it is a business managing a portfolio, but the effect on you is the same. An endpoint you built around is deprecated. A model version is replaced by one with a different tone. A price rises or a rate limit appears at the worst possible moment. A provider changes its terms and your compliance posture changes with it.

Each of these is a small migration project with a deadline you did not choose. Over three years the pile of them becomes real work: re-testing prompts, adjusting to a new model's quirks, re-checking output against expectations, sometimes losing behaviour you had come to depend on.

An owned machine does not change unless you change it. The model you tested is the model that runs next year. For anything where consistency matters, that stability is worth more than the price difference.

Cost three: the data question

This is the one people argue about, and the argument is usually too polarized to be useful. Sending some prompts to a provider is fine. Sending others is not, and the difference is not about how much you trust the vendor. It is about what the data is.

Ask a simple question about public information and there is no meaningful exposure. Feed a machine your medical history, an unredacted contract, a set of family photographs, or a client's internal documents and you have made a decision that cannot be undone by a settings toggle. It is now somebody else's records-retention policy to manage, on your behalf, with your liability attached.

A local machine does not make those questions disappear. It changes them into backup, physical security, and disk encryption, which are problems you can solve with a shelf and a passphrase. For many people that trade is worth taking, not out of paranoia but because the amount of machine-readable personal material keeps growing and the value of keeping it private grows with it.

What owning actually requires

Be honest about the work you are taking on. Owning means you are now the operations team.

If that list reads as unacceptable, rent. That is a legitimate engineering decision and it costs you nothing but money. If it reads as ordinary, keep going.

The hybrid answer is usually right

Rent and own are not mutually exclusive, and the strongest setups use both deliberately.

  1. Route by sensitivity. Anything private goes to the local machine. Anything public goes wherever it is cheapest or fastest.
  2. Route by difficulty. Routine summarisation and classification runs locally at near-zero marginal cost. The rare hard reasoning task goes to a rented frontier model.
  3. Keep one skill portable. Write prompts and pipelines so the model is a parameter rather than an assumption. Then changing providers is a config edit instead of a rewrite.

That arrangement captures most of the cost benefit of ownership and most of the capability of renting. It also means a provider outage does not stop your household assistant from working, which is the sort of resilience you only appreciate after the first incident.

How to actually run the numbers

Collect four figures and the comparison becomes simple.

Multiply the first by the second for your monthly rental figure. Add the third and fourth for your monthly ownership figure. If you are planning to do this properly, a DIY AI build guide with real bill-of-materials numbers will serve you better than any estimate from a website, including this one.

Two things the arithmetic will not capture. First, the value of uptime when the network is down, which is real for anyone who has lost an afternoon to an outage. Second, the learning you acquire by running the thing yourself, which for anyone technical is worth more than the marginal cost difference.

The pieces you end up choosing between

If you decide to own, you have three routes and they are genuinely different.

The first is a repurposed machine you already have. Cheapest, fastest to start, worst idle behaviour, and fine for an experiment. The second is a purpose-built appliance, either assembled by you around an embedded board or bought ready to run. This is where the idle and bandwidth arguments pay off, and where a stripped-down appliance operating system built for AI workloads makes the difference between a project and a product.

The third is an off-the-shelf dedicated box that arrives configured. You trade money for your own time, and the honest question is what an hour of your attention is worth, which is usually more than people assume when they start a build on a Saturday and finish it in November.

The verdict

Rent to learn. Own to depend. That is the shortest useful summary of this entire argument.

If your use is exploratory, occasional, or needs the largest model that exists, renting is correct and will stay correct. If your use is continuous, private, latency-sensitive and modest in volume, an owned machine pays for itself and then keeps paying, in money and in a kind of quiet reliability that subscriptions do not provide.

Read the bandwidth article before you buy anything, and the idle power article before you decide where to put it. Both of those mistakes are expensive, and both are avoidable with an hour of reading.