Where AI Features Belong on the Server vs the Client

Modern React frameworks give you a server/client boundary that's more powerful than most teams use, and easier to misuse than the docs suggest. When you add AI features on top — LLM calls, streaming, tool use, RAG — where the logic runs stops being a preference and becomes a security decision. Server components vs client components for AI features is one of those choices where getting it wrong costs you money or leaks keys.

The rule we default to

All model calls, all keys, all tool executions run on the server. The client renders. Full stop.

This isn't a stylistic preference. Any key that touches a client bundle is a key that ends up in someone's browser dev tools, then a scraper, then an abuse invoice.

What the client is actually for

• Rendering streamed tokens as they arrive.
• Local state — the input, the current conversation, UI expansion.
• Optimistic UI updates.
• Cancelling an in-flight request.

None of these need model access. All of them need a server endpoint they can call.

The pattern that works

A server function or route accepts the user's input, calls the model with the server-held key, and streams the response back over Server-Sent Events or a streaming Response. The client component reads the stream and renders it.

This is boring. It's also the pattern that survives audits, doesn't leak keys, and lets you swap providers without touching the frontend.

Where teams get this wrong

1. Calling the model directly from the client

"Just for the demo." The demo ships. The key is in the bundle. Someone finds it and racks up a $12k OpenAI bill in a weekend.

Even with a "browser-safe" SDK that hides the key, you're just relying on obscurity. Assume everything in the client is public.

2. Passing tool results through the client

The model asks for a tool call → client executes it → returns to the model. Now the client can invoke any tool the model asks for, with whatever arguments the model chose. Users can craft prompts that make the tool do things the server would have refused.

Tools execute on the server. Period.

3. Streaming through a client-side proxy

Some teams try to keep the model call server-side but let the client stream directly from the model's URL. This leaks headers, keys, and gives the client the ability to bypass your rate limits.

The server proxies the stream. Yes, that means the request path is server → model → server → client. It's the correct shape.

What server components add

Server components in modern React frameworks (Next.js App Router, TanStack Start, etc.) let you fetch and render AI-generated content without shipping the fetch logic to the client. For non-interactive AI (summaries baked into a page, precomputed suggestions), they're a clean fit.

For interactive AI (chat, streaming responses, tool use), you still need a client component to handle the stream and state. The server component renders the shell; the client component drives the interaction.

Streaming, concretely

Server function returns a ReadableStream. The response is content-type text/event-stream (SSE) or a chunked plain text stream. The client uses fetch with a reader and updates state as chunks arrive.

Two things to get right:

• Backpressure. If the client stops reading, the server should stop generating. Wire up an abort signal from the client's abort controller back to the model call.
• Error boundaries. Streams can fail mid-flight. The client needs to show a partial response with a clear "something went wrong" message and let the user retry.

Cost implications

Client-side model calls also mean you can't cache responses across users, can't dedupe identical prompts, and can't add prompt caching (Anthropic, OpenAI) which is server-only. On volume, the server-side architecture is meaningfully cheaper. Related reading: Caching LLM responses.

Trade-offs to be honest about

Server-side AI adds latency — the request has to hit your infrastructure before reaching the model. In practice, this is 50-150ms extra, invisible against a multi-second generation. The people who worry about this are usually worrying about the wrong thing.

Server-side also concentrates cost. Every request goes through your server. If you're on a per-request-priced platform, factor this in.

What we do for the boundary

• All model calls in createServerFn or route handlers.
• All keys in process.env, read inside the handler, never at module scope.
• All tools implemented as server functions with their own auth checks.
• Client components import types from server modules, not runtime.

Related reading: How we handle prompt injection in production LLM apps.

Common misconceptions

"Server components make my AI faster." They don't. They just move where the render happens. Latency comes from the model, not the framework.

"I need a separate backend for AI." Rarely. Server functions in your existing framework are enough for 99% of use cases.

"Streaming requires WebSockets." It doesn't. HTTP streaming with SSE handles the entire chat UX cleanly.

FAQ

Can I ever call the model from the client?

For genuinely public, non-authenticated features with a heavily rate-limited public key (like the OpenAI browser-only endpoint), maybe. For anything user-scoped, no.

How do I add rate limiting?

In the server function. Per-user, per-IP, and global caps. Related reading: Rate limiting AI features in SaaS.

What about doing embedding on the client?

Same rule. Even "small" embedding models cost real inference dollars at scale — and the vector then has to hit your server anyway for search.

Where to go from here

If your AI feature is calling the model from the client "just for now," it's worth an hour to move it before someone finds the key. Our AI product engineering team does this kind of tightening as part of most engagements.