Build vs buy: should you train your own AI or use a vendor?

Every team building anything with AI eventually hits this question: do we train our own model, or do we just use a vendor? It generates more meetings than almost any other decision in the space.

Most of the heat in those meetings comes from treating it as one big yes/no decision. It isn’t. There are at least four positions on the spectrum, and the right one depends on specifics most arguments skip over.

The four positions

From least to most ownership:

1. Call a vendor’s API. Anthropic, OpenAI, Google. You send text, you get text. They run the model.
2. Use an open-weight model someone else hosts. Bedrock, Together, Anyscale, Replicate. You don’t manage GPUs, but you can pick from many models and pay by usage.
3. Run an open-weight model yourself. Llama, Mistral, Qwen on your own GPUs. You handle inference, scaling, and updates.
4. Train or extensively fine-tune your own model. Continued pre-training, custom architecture, or large-scale fine-tuning.

Almost nobody should be at position 4. Many teams who think they need position 4 actually need position 1.

When to lean towards “buy” (positions 1 and 2)

You’re early. Your volume is unknown. Your use case is general enough that a frontier model can handle it. You don’t have specialised infrastructure expertise on the team.

In numbers, buy is almost certainly right when:

• You’re spending under $10–20k per month on inference. Engineering time to migrate to self-hosted would outweigh savings.
• Your data isn’t subject to residency or privacy rules that block sending it to a vendor.
• The product is still finding fit. Optimising the wrong thing is the most expensive mistake at this stage.

This is also where most teams should stay. Buy doesn’t mean “lazy” — it means putting complexity budget into the parts of the system that actually differentiate you (retrieval quality, evaluation, UX) instead of into running model servers.

When to lean towards “build” (positions 3 and 4)

You have one or more of these very specific situations:

• Data residency. You can’t send the data to a third party. Healthcare, government, regulated finance.
• Real volume. You’re spending hundreds of thousands per month or more on inference, and a fine-tuned smaller model on your own infrastructure would dramatically reduce that.
• Specialised performance. You’ve measured that, on your task, a fine-tuned open model can beat the frontier model — not in theory, in your evaluation set.
• Vendor risk. Your business depends on AI in a way that single-vendor dependency is genuinely dangerous, and you have the engineering capacity to operate the alternative.

Notice that none of these are “we want more control” or “open-source is better.” Those are emotional reasons, not engineering ones.

The fine-tuning trap

A common middle ground: we’ll fine-tune our own model. This sounds clever — keeping vendor convenience while customising the model.

It can work. But it more often disappoints than people expect, because:

• Fine-tuning is most useful for changing behaviour, not adding knowledge. Adding facts to a model via fine-tuning is much worse than just retrieving them at runtime (i.e., RAG).
• A fine-tuned model is a snapshot. The base model’s next version will be better; your fine-tune doesn’t automatically benefit. You’ll re-tune.
• It requires a labelled dataset, evaluation rig, and ongoing maintenance — real engineering work.

A useful test: if you can’t articulate, in one sentence, what specific behaviour change you’re fine-tuning for, you’re not ready to fine-tune.

A cleaner framework

Instead of “build or buy,” ask three sharper questions:

1. What is the cost dominating right now?

• Engineering hours → buy. Don’t burn weeks self-hosting to save dollars.
• Inference bill → consider building, if the math is real.
• Compliance/legal → buy with strict guarantees, or build.

2. What is the work that differentiates us?

• If it’s the model itself: a rare answer. Maybe build. But really, is it?
• If it’s the data, the workflow, the UX: buy. Spend the saved time there.

3. What can we operationally support?

• A team that has shipped trained models before, with GPU infra, evaluation pipelines, and on-call: building is realistic.
• A team that’s never owned model infrastructure: don’t start with model training. Start with the next adjacent thing.

The escape hatch

A pragmatic pattern that works for most teams:

• Start at position 1 (vendor API). Get to working product.
• Identify where cost or constraints actually pinch.
• Move that specific path — not the whole system — to position 2 or 3.
• Re-evaluate quarterly.

You almost never need to commit upfront to “we are a build company” or “we are a buy company.” The right answer is usually different per code path, and usually shifts as the product matures.

The honest summary

For most teams, most of the time, “buy first, build narrowly, fine-tune rarely” is the right default. Building your own LLM is fashionable; doing the unglamorous work of evaluation, retrieval, and observability is what actually ships AI products.