How We Choose Between Fine-Tuning and Better Prompts
A working note on fine tuning vs prompting — what matters, what does not, and where these projects usually go sideways.
There is a cheap version of fine tuning vs prompting and an expensive one. The difference is decided in week one. This post walks the order we actually use for fine tuning vs prompting.
The problem underneath
Two things are competing. Shipping this quarter and not regretting it next year.
Most teams pick one and pretend the other does not exist.
What this looks like in real projects
One common pattern we see: the first customer shaped the design, and the fifth one broke it. Nothing was wrong, the inputs changed.
The fix is usually smaller than the panic suggests, provided somebody maps the current state honestly.
Mistakes companies make
- Choosing tools before the workflow is written down.
- Scoping version one to cover every edge case.
- Leaving the work unowned, then blaming the tool.
- Skipping measurement, so nobody can prove it helped.
- Treating launch day as the end of the cost.
The first and the last are the expensive ones.
The engineering view on fine tuning vs prompting
From inside the codebase, fine tuning vs prompting reduces to three questions. What happens when a step fails halfway. Who finds out. How you reverse it.
Design for partial failure before you need it. Step three failing after one and two succeeded is the case people skip.
Give retries a ceiling and some jitter. A retry storm is an outage you built yourself.
How we approach it step by step
- Reproduce the pain with a real case, not a description of it.
- Write the target outcome as a single number.
- Pick the smallest change that could plausibly move that number.
- Build it with a rollback path.
- Release to one team or a slice of traffic.
- Review in two weeks, then widen, revise, or delete.
Deleting is a legitimate result. It happens less often than it should.
Practical guardrails
- Instrument before optimising.
- Cap spend and volume in code, not on the invoice.
- Write down the decision, not only the outcome.
- Keep one named owner with protected hours.
- Set a review date ninety days out and keep it.
Trade-offs worth saying out loud
Speed against flexibility. Managed service against control. Cheap now against cheap later. None of it is free.
This trade-off usually appears when the second customer wants something the first one didn't. That is the moment to revisit fine tuning vs prompting, not before.
Common misconceptions
“We need the best available option.” You need the one your team can operate at 2am. Rarely the same thing.
“We’ll do it properly later.” Sometimes true. Put a date on later or it never arrives.
“It’s a one-off.” Anything a customer touches becomes a product, support included.
Frequently asked questions
How long does fine tuning vs prompting take to get right?
A narrow first version is usually four to six weeks. Anything quoted at three months with nothing shippable in between is a risk, not a plan.
What is the most common mistake with fine tuning vs prompting?
Choosing tools before the workflow is written down. The tool then dictates the process instead of serving it.
Can we do this without touching production data?
For the first pass, yes — use a masked copy. Anything involving billing or permissions needs a rehearsal against real shapes.
How do we know whether it worked?
Choose the number before you build — hours saved, error rate, response time, or conversion — then compare a two-week window either side.
Do we need to hire someone for this?
Not at the start. One named owner with a few protected hours a week, plus a small build team, is enough to prove value.
Conclusion
The useful move on fine tuning vs prompting is almost always the smaller one. Ship a narrow slice a real user can touch this month, measure it, then decide what earns the next four weeks.
Everything gets easier once something is live.
Related reading and next steps
- AI automations — how we run this kind of work.
- growth analytics — how we run this kind of work.
- More writing from the team.
Want a second opinion on fine tuning vs prompting for your setup? Book a 30-minute call. If it is not worth building, we will say so.
FAQ
Frequently asked questions
How long does fine tuning vs prompting take to get right?+
A narrow first version is usually four to six weeks. Anything quoted at three months with nothing shippable in between is a risk, not a plan.
What is the most common mistake with fine tuning vs prompting?+
Choosing tools before the workflow is written down. The tool then dictates the process instead of serving it.
Can we do this without touching production data?+
For the first pass, yes — use a masked copy. Anything involving billing or permissions needs a rehearsal against real shapes.
How do we know whether it worked?+
Choose the number before you build — hours saved, error rate, response time, or conversion — then compare a two-week window either side.
Do we need to hire someone for this?+
Not at the start. One named owner with a few protected hours a week, plus a small build team, is enough to prove value.
Building something similar?
Let's talk in 30 minutes.