The frontier still takes the headlines. In the work we actually ship, most of the value runs on models small enough to sit beside the application instead of looming over it. So we gathered the questions clients ask us most about that gap and answered them the way we would in a working session, with the numbers in front of us.

Q: Why are small models suddenly worth a serious look?

Because the price of a good answer fell off a cliff. Stanford’s AI Index put a figure on it: the cost of a query that scores like GPT-3.5 on MMLU dropped from twenty dollars per million tokens in late 2022 to seven cents by late 2024. That is a 280-fold cut in about eighteen months. Once “good enough” gets that cheap, the question stops being “which model is smartest” and becomes “what is the smallest model that clears my bar, and how do I know it clears it.”

Q: Define small. People mean different things by it.

NVIDIA’s research group drew a practical line last year: treat most models under ten billion parameters as small, with the working test being whether the thing fits on a normal consumer device and answers one user fast enough to be useful. That framing is the right one, because it ties “small” to where a model runs and what it costs you, not to a leaderboard position that will be stale by next quarter.

Q: Can a small model actually compete on quality, or is that wishful thinking?

On the right task it competes, and it can surprise you. Microsoft’s Phi-4 is a fourteen-billion-parameter model, and on its own published card it beats the much larger GPT-4o at competition math (80.4 to 74.6) and at graduate-level science questions (56.1 to 50.6). Here is the honest footnote, though: it does not win across the board. It trails GPT-4o on broad knowledge and on coding. That contrast is the whole lesson. Small models win narrow, not everywhere, so the work is knowing which of your tasks is actually narrow.

Q: Where do small models break?

Long context and long chains of reasoning. There is a clean result here. A study called Through the Valley found that models at or below three billion parameters degrade sharply once you push them into extended step-by-step reasoning; one 1B model fell to about a quarter of its baseline accuracy after only a modest dose of long-reasoning training. Larger models recover faster. So the failure is real, and it bites the smallest models hardest. The skill is spotting the task that is secretly a long-reasoning problem wearing a simple costume.

Benchmark a small model on a frontier task, decide it is bad, and you will miss that it was perfect for the task you actually have.

Q: Host a small model, or just call a cheap frontier API?

This is where teams overcomplicate it. Sometimes the win is a small open model you run yourself: NVIDIA estimates a small Llama can be ten to thirty times cheaper to serve than the 405B version, and reckons forty to seventy percent of the calls inside real agent systems could be handed to a specialized small model. But sometimes the win is just a cheap hosted model from a frontier family. a16z’s survey of a hundred enterprise buyers found teams increasingly saying the major models are all “good enough,” so they choose on price and fit, and they are doing less fine-tuning, leaning on long context instead. Both roads end in the same place: stop paying frontier prices for work that was never frontier.

Q: What does the cheap end actually look like today?

Concrete numbers help. Mistral’s Ministral models run at four cents per million tokens for the 3B and ten cents for the 8B, with a 128k context window. Google’s Gemma 3 ships from one billion up to twenty-seven billion parameters, runs on a single GPU, and the 27B holds its own against models many times its size in head-to-head preference tests. A year ago that capability needed a frontier contract. Now it fits inside a side process you barely notice on the bill.

Q: One piece of advice for a team starting now?

Build the evaluation before you pick the model. Once you can score quality on your own task, the small-versus-large argument simply ends: you try both and you read the number. Teams without an eval argue about model choice for months. Teams with one settle it in an afternoon, then spend their real effort where the return actually lives, on the workflow and the data wrapped around the model.