Shipping Multimodal AI Features Without Blowing Your Budget
Vision, audio, and PDF pipelines that actually work in production — and the cost traps we keep seeing in 2026.
Multimodal calls are 5–40x more expensive than text. Ship them the same way you would text and your unit economics will die within a month.
Vision
- Downscale on the client before upload — 768px is enough for 90% of tasks.
- Cache by perceptual hash, not by URL. Same photo, same answer.
- Use structured outputs so you can validate and retry cheaply.
PDFs
Do not send the whole PDF to a vision model. Extract text first, run OCR on scanned pages, then only send images of pages the LLM asks for. Costs drop 10x with no accuracy loss.
Audio
Transcribe with a cheap model, run the LLM on text, only use audio LLMs when tone or speaker separation is the product. Streaming transcription + text LLM beats direct audio LLM for most support and meeting use cases.
The unit-economics check
Before you ship, model cost per active user per month. If a heavy user costs more than 20% of their price, redesign — cache harder, downscale more, or gate features by plan.
FAQ
Frequently asked questions
Can I use one model for everything?+
Technically yes, economically no. Route by task; use vision only when text will not do.
How do I cache vision?+
Perceptual hashes plus a stable prompt hash. Store the answer for 30–90 days depending on domain.
Building something similar?
Let's talk in 30 minutes.