How to pick the right LLM: Claude, GPT, Llama, and the rest

Pick any week and there’s a new “best” language model. Last month’s top of the leaderboard is this month’s second place. Most of these comparisons are entertaining to read and almost useless when you actually have to ship something.

Here is the framework I use when teams ask: “which model should we use?”

Start with the wrong question

The wrong question is “which model is best?” — there is no single best model, and the answer changes every six weeks anyway.

The right question is “which model is best for this use case, at this price, within this latency budget?” — those three constraints narrow the field dramatically.

The three families

Almost every model you can use in production falls into one of three families:

• Closed frontier models. Claude (Anthropic), GPT (OpenAI), Gemini (Google). These are the most capable, the easiest to start with, and the most expensive per call. You access them through an API; you don’t run them yourself.
• Open weight models. Llama (Meta), Mistral, Qwen, DeepSeek, and others. You can download the weights and run them on your own infrastructure (or on a hosting provider’s). Capability has caught up to closed models for most tasks, especially with fine-tuning.
• Specialised small models. Smaller models tuned for a specific job — embeddings, classification, code, etc. Much cheaper, much faster, but not general-purpose.

Most production systems use a combination. A frontier model for hard reasoning steps, a smaller model for routine ones, a specialised embedding model for retrieval.

When to use a closed frontier model

Default to Claude, GPT, or Gemini when:

• You’re early. You don’t yet know what your real volume or latency requirements are. Don’t optimise prematurely; ship with the easiest option.
• The task needs strong reasoning. Complex agents, multi-step planning, code generation, nuanced writing. Frontier models pull ahead here.
• You don’t want infra responsibility. No GPUs, no model serving, no quantisation decisions. The provider does that work.

The catch: cost compounds fast at scale, and you’re dependent on the vendor’s roadmap, pricing, and uptime.

When to use open weight models

Lean towards Llama, Mistral, Qwen, etc. when:

• Volume is high. At enough scale, paying per token to a closed API becomes more expensive than running your own inference.
• Data residency matters. You can’t (or shouldn’t) send the data to a third-party API — healthcare, government, finance compliance scenarios.
• You want to fine-tune. Closed providers offer fine-tuning, but you’re more limited than with open weights you control end-to-end.
• The task is well-defined. A 7B-parameter model fine-tuned on your data can match a frontier model on the specific thing you care about, while being 10× cheaper and faster.

The catch: you take on real infrastructure work — GPU procurement, inference servers, quantisation, evaluation, monitoring. Not trivial.

When to use small specialised models

Reach for small specialised models when:

• The task is narrow. Embedding text for retrieval, classifying intent, extracting structured data. A 100M-parameter model can solve these well.
• Latency is critical. Conversational UX, autocomplete, real-time anything. A large model’s latency budget alone could be your whole user experience.
• The task runs at extreme volume. Anywhere you’re calling a model millions of times per day, specialised models save real money.

A decision rule that actually works

When in doubt, in this order:

1. Prototype with a frontier closed model. Cheapest way to validate the idea actually works.
2. Measure where you spend. Once you have a working prototype, see what’s calling the model, how often, and at what latency. The 80/20 will become obvious.
3. Replace the expensive paths. Swap the 80%-volume routine paths to a smaller or open model. Keep the 20%-volume hard paths on the frontier model.
4. Fine-tune if it pays back. Only fine-tune when you have a clear, narrow task where it can outperform prompting at a meaningful cost reduction.

Skipping step 1 — “we should use open-source / fine-tune from day one” — is the most common expensive mistake. Validate the idea first, optimise later.

What to ignore

A few things to deprioritise:

• Benchmark leaderboards. They measure tasks that may have nothing to do with yours. Build your own small evaluation set; that’s the only leaderboard that matters.
• “Our model beat GPT-X on Y” press releases. These are usually carefully chosen benchmarks under carefully chosen conditions. Healthy scepticism.
• The newest model. New models are exciting; they’re also less battle-tested. For production, lagging by a release is often the right call.

The honest summary

The model is rarely the limiting factor. The retrieval, the prompt design, the evaluation rig, the guardrails, and the production monitoring almost always matter more than which of three excellent frontier models you picked.

Pick the easiest viable model. Build the system around it well. Optimise model choice once you have data telling you which path is actually expensive — not before.