You Don't Need the Biggest AI Model, You Need the Right One

There's a lot of noise at the moment about smaller AI models catching up to the giant ones, at a fraction of the cost and speed. It's a genuinely useful shift, but not for the reason most people think. It's not really about small models being clever. It's a reminder that most business problems don't need the biggest tool in the shed.

I see this a lot when clients come to me wanting "an AI feature" bolted onto their software. Often what they've pictured is a chatbot powered by the most capable model on the market. What they actually need is something far more boring: a small, fast, cheap model that reads a document, pulls out three fields, and puts them in the right column. That's not a lesser solution. It's often the correct one.

Bigger isn't automatically better for your use case

The large, general-purpose models are genuinely impressive at open-ended tasks: writing, reasoning through ambiguous problems, holding a conversation. But most business automation isn't open-ended. It's narrow and repetitive. Categorise this support ticket. Extract the invoice number and total from this PDF. Flag whether this booking request mentions a cancellation. These are closed, well-defined tasks, and a smaller model tuned for that job will often do it faster, more consistently, and far more cheaply than a general-purpose model asked to "figure it out".

There's also a reliability angle. A huge, general model can be brilliant one moment and confidently wrong the next, because it's trying to be good at everything. A smaller model built or prompted for a specific, narrow job has less room to wander off. When I'm building something that feeds into an invoice, a rota, or a customer record, I want boring and predictable a lot more than I want impressive.

What this looks like in practice

A few examples from the kind of work I actually build:

• Document extraction. Pulling supplier names, dates and totals off scanned invoices doesn't need a model that can also write poetry. A lightweight model trained or prompted specifically for structured extraction is quicker, cheaper per call, and easier to keep accurate over thousands of documents a month.
• Transcription and categorisation. Turning a voicemail into text and tagging it by urgency is a job for a fast, specialised transcription model, not a full conversational assistant. You want it to run in seconds, not have a personality.
• Customer-facing chat. If someone's asking your website "what are your opening hours" or "how do I book a callback", they don't need a model capable of writing a novel. They need something quick that answers correctly and hands off to a human when it's out of its depth.

In all of these, the "smallest model that reliably does the job" is usually the right answer, not the biggest one you can afford.

Cost and speed add up faster than people expect

Every call to a large general-purpose model has a price and a delay. If your automation runs a handful of times a day, that's irrelevant. If it runs on every ticket, every booking, every incoming email, the difference between a large model and a smaller, purpose-built one can be the difference between an automation that's cheap to run indefinitely and one that quietly becomes an expensive habit. I've seen businesses build something that works fine in testing, then find the running costs climb once it's actually being used at volume. Picking the right-sized tool up front avoids that conversation later.

Speed matters too. A support agent waiting three seconds for a suggested reply is fine. A booking system waiting three seconds to confirm a card payment feels broken. Smaller models generally respond faster, which matters more than it sounds like it should when you're building something people use in the middle of their working day.

There's also the question of what leaves your building

Some smaller models can run entirely on your own infrastructure, rather than sending data off to a third party's servers. For businesses handling anything sensitive, customer records, medical information, financial details, that's not a small consideration. It's often the deciding factor between "we can automate this" and "we can't, not without a lot of extra assurance work". A smaller, self-hosted model sidesteps that entirely.

How I actually decide

When I'm scoping an automation project, I ask what the task actually is before I ask which AI to use for it. Usually the honest answer is that a chunk of it doesn't need AI at all, it needs a proper workflow and some validation. Of what's left, most of it is a narrow, repeatable job suited to a small, fast, specific tool. Only a small slice genuinely benefits from a large general-purpose model, usually where the input is unpredictable or the output needs to read naturally to a human.

Working alone means I don't have a default stack I'm trying to justify, and no incentive to reach for the most expensive option because it looks better on a proposal. I pick whatever actually solves the problem at hand, and I'm the one who has to live with the running cost and the support calls afterwards, so I've no interest in over-engineering it.

If you're looking at where AI might genuinely save you admin time rather than just add a shiny feature, that's the kind of thing worth working through properly before any code gets written. It's covered in more detail on my page about the hidden cost of doing the same admin twice, and it's usually where the real, sustainable savings are.