Once a team decides to build an AI feature, the next big decision is where the model itself comes from: a hosted API like OpenAI or Anthropic, or an open-weight model you run on your own infrastructure. It’s a classic build-vs-buy question, and like most build-vs-buy questions, the loudest arguments tend to be about the wrong things.
The usual debate fixates on per-token price and “we don’t want to send our data to a third party.” Both matter, but both are frequently misjudged. Here’s a clearer way to think about it.
Start with hosted APIs—for almost everyone
For the vast majority of teams building their first AI features, a hosted API is the right starting point, and often the right permanent answer.
You get frontier capability immediately. The best hosted models are, at any given moment, more capable than anything you can practically self-host. You access that with an API key and no infrastructure.
There’s nothing to operate. No GPUs to provision, no inference servers to keep up, no model-serving stack to monitor. Someone else handles scaling, availability, and the relentless pace of model improvements.
You pay for what you use. For low and moderate volume, usage-based pricing is dramatically cheaper than standing up dedicated hardware that sits idle most of the day.
You can move fast. You can prototype, change models, and ship in days. That speed is exactly what you want while you’re still learning whether the use case works at all.
The instinct to self-host early usually comes from anticipating problems—cost, privacy, lock-in—before you’ve confirmed the feature is even valuable. That’s backwards. Prove the use case on a hosted API first. Most of the reasons to self-host only become real at scale or under specific constraints, and you can cross those bridges when you reach them.
The real cost picture
The per-token price of an API looks like the whole cost, so people compare it directly against “free” open models and conclude self-hosting is cheaper. It usually isn’t, until you’re running at serious volume.
Self-hosting a capable model means GPU infrastructure—expensive, and expensive whether it’s busy or idle. It means an inference-serving stack, autoscaling, monitoring, and on-call coverage. It means engineering time, which is the most expensive line item of all. A “free” open-weight model running on hardware you operate can easily cost more in total than an API bill, especially at low-to-moderate usage where your hardware is mostly idle.
The economics flip when volume is high and sustained. At large, steady throughput, dedicated infrastructure can be cheaper per request than API pricing—you’re keeping the GPUs busy enough to justify them. The crossover point depends on your usage pattern, but the lesson is the same: don’t self-host to save money until you’ve done the math on your actual volume, including the cost of the people who’ll operate it.
When privacy and compliance genuinely require it
“We can’t send our data to a third party” is the most common reason cited for self-hosting—and it’s sometimes right and often overstated.
It’s worth knowing what the major providers actually offer. Hosted APIs typically provide business agreements that exclude your data from being used to train their models, offer data-handling commitments, and in some cases regulated-industry compliance options. For many businesses, those terms are sufficient, and the reflexive “we can’t use the cloud for this” doesn’t survive a careful read of what’s on offer.
Self-hosting becomes genuinely necessary when you have hard constraints: regulatory requirements that data never leaves your environment, contractual obligations that rule out third-party processing, air-gapped or sovereign deployments, or data so sensitive that the residual risk of any external call is unacceptable. These are real situations—particularly in healthcare, defense, and parts of finance—and in them, running an open model in your own environment isn’t optional.
The key is to evaluate the actual requirement, not the vague discomfort. “It feels risky” and “our compliance framework prohibits it” lead to very different, and very differently priced, answers.
What self-hosting actually buys you
Beyond hard compliance needs, self-hosting open-weight models offers genuine advantages worth weighing:
Control and stability. You decide which model version you run and when it changes. Hosted models get updated and occasionally deprecated on the provider’s schedule, which can subtly shift behavior under you. Self-hosting freezes that.
Data residency and isolation. Everything stays in your environment. For some organizations that’s the whole point.
Customization depth. Full control of the weights opens up fine-tuning and modification that hosted APIs limit or don’t offer.
Predictable cost at scale. Fixed infrastructure cost instead of a variable bill that grows with success.
And the costs:
Operational burden. You now run a non-trivial production system: GPUs, serving infrastructure, scaling, monitoring, upgrades. That’s a standing commitment, not a one-time setup.
A capability gap to manage. Open models have improved enormously and are excellent, but you’re responsible for keeping current, and the absolute frontier still tends to live behind hosted APIs.
Real expertise required. Serving models efficiently and reliably is specialized work. If you don’t have that skill in-house, you’re acquiring it or hiring it.
Avoid lock-in without self-hosting
A quiet third option addresses the “lock-in” fear without taking on the operational burden: build your application so the model is swappable. Put a thin abstraction between your app and whatever model serves it, so switching providers—or moving to a self-hosted model later—is a configuration change, not a rewrite. This is good engineering regardless, and it lets you start on a hosted API today while keeping the door open.
A practical default
For most teams, the sensible path is: start on a hosted API, design for model portability, and revisit self-hosting only when a specific trigger appears—a hard compliance requirement, volume high enough to flip the cost math, or a need for control that hosted options can’t meet.
Self-hosting is a powerful option for the situations that genuinely call for it. It’s a poor default chosen out of cost assumptions that don’t hold at low volume or privacy fears that a careful read of the provider’s terms would settle. Decide based on your real constraints and your real numbers.
If you’re trying to make this call for a specific use case—and want an honest assessment that isn’t selling you either a subscription or a GPU cluster—we help teams weigh exactly this trade-off.