Laravel + OpenAI: Streaming, Cost Control and Production Gotchas

If you are adding an OpenAI feature to a Laravel app in 2026, use the official laravel/ai SDK unless you have a specific reason not to — and treat the API key like a payment method, not a config value. I am Khaled Ahmed, a full stack developer in Cairo with 5+ years and 39+ shipped production projects across eight countries. This guide is the checklist I actually use: package choice, real streaming, queueing, rate limits, and the cost controls that stop a demo from becoming a bill.

1. The Verdict: Which Laravel OpenAI Package Should You Use in Production?

Short answer, as of August 2026: start with laravel/ai. It is the official SDK, it is maintained by the Laravel core team, and it is being pushed hard enough that the rest of the ecosystem is now orbiting it.

The honest caveat, which nobody selling you an AI feature will mention: laravel/ai is still 0.x. The version on Packagist at the time I am writing this is v0.10.3, tagged 6 August 2026, requiring PHP ^8.3 and Laravel ^12.0|^13.0. Forty-six tags have shipped since launch. That is a fast-moving package. Pin it with a tilde constraint, read the release notes before you bump, and do not put a 0.x dependency at the centre of a system you will not touch for two years.

Here is how the real options compare. I have used all four.

Why the official SDK actually changes the calculus

It is not just that it is official. It is that four things I used to hand-build are now framework-level: SSE streaming that you can return straight from a route, a queue() method with then/catch, provider failover as an array argument, and a full set of dispatched events (PromptingAgent, AgentPrompted, InvokingTool, ToolInvoked, StreamingAgent, AgentStreamed) that give you a clean hook for cost logging. That is a week of plumbing I no longer bill anyone for.

2. Installation and the Config Decisions That Matter

Installation is three commands:

composer require laravel/ai

php artisan vendor:publish --provider="Laravel\Ai\AiServiceProvider"

php artisan migrate

The migration creates agent_conversations and agent_conversation_messages. Look at those tables before you run the migration on a multi-tenant app — if you are enforcing tenancy at the database level, you need to decide where conversation rows live. I wrote about that boundary in more depth in my guide to building multi-tenant SaaS on Laravel, and the same rule applies: a conversation is tenant data, and it will contain whatever your users pasted into it.

Credentials go in config/ai.php or .env. The SDK recognises keys for OpenAI, Anthropic, Gemini, Mistral, Cohere, xAI, Groq, DeepSeek, OpenRouter, Azure OpenAI, ElevenLabs, Jina, VoyageAI and Ollama, plus a generic OpenAI-compatible driver that covers anything you self-host or run behind Bedrock. That breadth tells you how seriously the abstraction is meant.

Three config decisions people get wrong:

• Set a custom base URL if you need a control point. The url parameter on a provider config lets you route through LiteLLM, a self-hosted vLLM endpoint, or your own proxy. (Azure OpenAI does not need this workaround — it is a first-class provider in the enum.) That is how you centralise key management, enforce spend caps outside application code, and log every request even when a developer bypasses your service class. On regulated projects — banks, health, anything in Saudi Arabia touching personal data — this is usually mandatory, not optional.
• Set the timeout deliberately. The SDK's default HTTP timeout is 60 seconds. A reasoning model with tool calls can exceed that. Use the #[Timeout(120)] attribute on slow agents rather than raising the global default and hiding real hangs.
• Never put the API key in a frontend build. Obvious, and yet I have inherited two projects where VITE_OPENAI_API_KEY was sitting in a public bundle. Rotate immediately, then read my website security checklist before you ship anything else.

3. Agents: The Right Unit of Abstraction

The SDK's core idea is the agent — a PHP class that owns instructions, conversation context, tools and an output schema. Generate one with php artisan make:agent SupportTriage, or add --structured for a JSON-schema variant.

namespace App\Ai\Agents;

use Illuminate\Contracts\JsonSchema\JsonSchema;
use Laravel\Ai\Attributes\MaxTokens;
use Laravel\Ai\Attributes\Model;
use Laravel\Ai\Attributes\Provider;
use Laravel\Ai\Attributes\Temperature;
use Laravel\Ai\Contracts\Agent;
use Laravel\Ai\Contracts\HasStructuredOutput;
use Laravel\Ai\Enums\Lab;
use Laravel\Ai\Promptable;
use Stringable;

#[Provider(Lab::OpenAI)]
#[Model('gpt-5.6-luna')]
#[MaxTokens(600)]
#[Temperature(0.2)]
class SupportTriage implements Agent, HasStructuredOutput
{
    use Promptable;

    public function instructions(): Stringable|string
    {
        return 'Classify the support ticket. Never invent an order number.';
    }

    public function schema(JsonSchema $schema): array
    {
        return [
            'category' => $schema->string()->required(),
            'urgency'  => $schema->integer()->min(1)->max(5)->required(),
            'reply'    => $schema->string()->required(),
        ];
    }
}

Two things in that snippet are cost controls, not style choices. #[MaxTokens(600)] is a hard ceiling on the expensive half of the bill. #[Temperature(0.2)] on a classification task cuts the rambling that inflates output tokens. And the structured schema means you get parseable JSON instead of a paragraph you have to regex — which is exactly the same discipline I argue for in API design best practices: define the contract, then make the system honour it.

The SDK also ships #[UseCheapestModel] and #[UseSmartestModel]. Convenient, but I avoid them in production. "Cheapest" is defined by the SDK, not by you, and it will change under you on a minor version bump. Name your model explicitly and change it on purpose.

4. How Do I Stream Tokens to the Browser from Laravel?

Streaming is the single biggest perceived-quality win in an AI feature. In my own experience watching people use these features, the same total latency reads as fast when text is visibly arriving and as broken when it is a silent spinner — users will sit through a long generation they can see, and give up quickly on one they cannot.

With the official SDK the happy path is genuinely one line — the returned StreamableAgentResponse is a valid route response and emits Server-Sent Events:

use App\Ai\Agents\SupportTriage;
use Laravel\Ai\Responses\StreamedAgentResponse;

Route::post('/ai/triage', function (Request $request) {
    return SupportTriage::make()
        ->stream($request->string('ticket'))
        ->then(function (StreamedAgentResponse $response) {
            // $response->text, $response->events, $response->usage
            // Persist the message and the token usage HERE.
        });
})->middleware(['auth', 'throttle:ai']);

If your frontend is React or Next.js and you are already using the Vercel AI SDK on the client, call ->usingVercelDataProtocol() on the streamable response and the wire format will match what useChat expects. That saves you writing a custom parser, which matters if you are combining a Laravel API with a React frontend — the pattern I describe in building a SaaS MVP with Laravel and React.

The four things that break streaming in production

The code is easy. The infrastructure is where people lose two days.

1. Nginx buffering. By default Nginx buffers the upstream response and your "stream" arrives as one lump at the end. You need proxy_buffering off; and X-Accel-Buffering: no on the SSE location. This is the number one cause of "it works locally, not on the server".
2. Cloudflare and other CDNs. Proxied SSE through an aggressive edge can be buffered or cut. Exclude the streaming route from proxying, or verify it end to end before you promise it to the client. If you are still choosing infrastructure, my notes on choosing web hosting cover which setups make this painless.
3. PHP-FPM worker occupancy. Every open stream holds a PHP worker for its entire lifetime. With pm.max_children = 20, twenty concurrent chats will lock your whole site. Either move to Octane, or use the broadcasting path below, or raise worker counts with your eyes open. This is the concrete place where the Laravel vs Node.js argument stops being theoretical — Node's event loop handles many idle-ish long connections more cheaply, and PHP needs Octane or a queue to match it.
4. Load balancer idle timeouts. An ALB or a VPS reverse proxy with a 60-second idle timeout will kill a long generation mid-sentence. Raise it on the streaming route only.

The alternative: stream over WebSockets instead

If holding HTTP workers is unacceptable, push the generation to a queue and broadcast the deltas. The SDK supports this directly:

use Illuminate\Broadcasting\PrivateChannel;

SupportTriage::make()->broadcastOnQueue(
    $request->string('ticket'),
    new PrivateChannel('conversations.'.$conversation->id),
);

The HTTP request returns immediately, a queue worker does the generation, and Reverb or Pusher delivers the tokens. It costs you a WebSocket layer and more moving parts, but it is the pattern that scales past a few dozen concurrent users on ordinary PHP hosting.

5. Where Should AI Calls Live — Queue Jobs or Controllers?

The rule I apply on every project:

• Controller, streaming: the user is watching, and the output is the point. Chat, drafting, "explain this". Stream it.
• Queue job: everything else. Classification, summarisation on upload, embedding generation, enrichment, bulk translation, anything triggered by a webhook.
• Never a synchronous controller call with no streaming. That is a 3-to-40-second request that occupies a worker, dies on a 502, and cannot be retried. It is the worst of both options and it is the most common mistake I find in inherited codebases.

Queueing with the SDK is deliberately boring:

SupportTriage::make()
    ->queue($ticket->body)
    ->then(fn ($response) => $ticket->applyTriage($response))
    ->catch(fn (Throwable $e) => report($e));

Four operational rules for AI queue jobs, learned the expensive way:

1. Put AI jobs on their own queue and their own Horizon supervisor. A slow model must never block password-reset emails.
2. Set $tries low and $backoff explicit. I use public $tries = 3; and public $backoff = [10, 60, 300];. A default retry loop against a paid API is a way to pay three times for the same failure.
3. Make the job idempotent. Store a hash of the input and short-circuit if you already have a result. Queue workers get killed mid-flight during deploys; without this you regenerate and re-pay.
4. Persist usage in the same transaction as the result. If you save the answer but lose the token count, your cost dashboard is fiction.

6. How Do I Stop an AI Feature from Burning My API Budget?

This is the section clients care about after week two. Here are the current published OpenAI list prices per million tokens, checked August 2026 — verify them before quoting anyone, because this table has changed several times a year since 2023. The current lineup on the pricing page is the GPT-5.6 family: Sol as the flagship, Terra as the everyday balanced tier, and Luna as the cost-optimised tier. Terra and Luna both got cheaper on 30 July 2026 (Terra by 20%, Luna by 80%); Sol was unchanged. The GPT-5 and GPT-4.1 generations are still available and still worth knowing, because plenty of production apps are pinned to them.

Two footnotes that bite in production. On the GPT-5.6 tiers, requests above roughly 272K input tokens move to a higher long-context rate, so a RAG feature that quietly grows its context window can change price band without anyone changing a line of code. And on GPT-5.6 and later, a cache write is billed at 1.25× the uncached input rate — cheap, but not free, so caching a prefix you only ever use once is a small loss rather than a small win.

Read the shape of that table, not the numbers. Output costs six to eight times input across the whole lineup — 6× on Sol, 8× on GPT-5. Cached input costs a tenth of fresh input. The cost-optimised tier is twenty-five times cheaper than the flagship on input — Luna at $0.20 against Sol at $5.00. Every real cost control follows from those three facts.

The seven levers, in the order I apply them

1. Cap output tokens. #[MaxTokens] on every agent. No exceptions. An unbounded generation is an unbounded invoice.
2. Route by difficulty. Most production traffic is classification, extraction and short replies. Those run on gpt-5.6-luna — or on gpt-5-mini/gpt-5-nano if you are already pinned to the previous generation. Reserve Sol, or whatever the flagship is when you read this, for the 5–10% of requests that genuinely need it. In my experience this alone typically removes the majority of the bill, because teams default the whole feature to the biggest model and never revisit it.
3. Exploit prompt caching. Cached input is 90% cheaper, but read that carefully: the discount applies to the cached prefix tokens on a cache hit, not to your input bill as a whole. Two conditions decide whether you get it. First, the prompt has to be stable at the front — put the long system instructions, schema and few-shot examples first, and the volatile user content last. Second, OpenAI's automatic caching only kicks in at a prefix of about 1,024 tokens and then grows in 128-token steps, so short prompts never cache at all, and caches are dropped after a few minutes of inactivity on that prefix. Get both right and the stable part of your input drops by 90%; get the ordering wrong and you pay full price on every call. Most teams never look at it.
4. Use the Batch API for anything not user-facing. 50% off standard rates. Nightly enrichment, backfills, re-embedding a catalogue — all of it belongs in Batch.
5. Consider flex processing for async work. OpenAI's service_tier flex option is priced at Batch rates for slower, lower-priority requests. As of the docs I read in August 2026 it is still described as beta with limited model availability, and it can return 429 "resource unavailable", which you are explicitly not charged for. The official SDKs use a 10-minute request timeout on flex, and complex jobs may need more than that — raise the timeout deliberately, back off on 429, and fall back to service_tier: auto if the work has any deadline at all. Useful, but do not put a user in front of it.
6. Cache your own results. Hash the normalised input plus model plus prompt version, and store the response. In a support-reply or product-description tool a meaningful share of requests are near-duplicates. This is ordinary application caching and it is free money — see why your website loads slowly for the same principle applied to page rendering.
7. Meter per user and per tenant. Store prompt_tokens, completion_tokens, model and cost on every call, keyed to the user. Then enforce a quota. Without this, one enthusiastic user or one loop bug is an open tap. Design that table properly the first time — my notes on database design for web apps apply directly.

The SDK gives you the hook for lever seven for free: listen for AgentPrompted and AgentStreamed and write a usage row from the event listener. One listener, every call covered, no service class to remember to go through.

7. How Do I Handle Rate Limits and Retries Safely?

OpenAI returns 429 for rate limiting and includes headers you should actually read: Retry-After, x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens, and the matching reset headers. Limits are tied to your usage tier, which climbs with cumulative spend — a brand-new Tier 1 key is dramatically more constrained than a Tier 4 key, which is why a feature that worked in staging on the founder's personal key can fall over on the company account.

Four rules:

1. Honour Retry-After, then add jitter. Fixed backoff across many workers produces a synchronised retry storm that re-triggers the limit. Randomise.
2. Throttle before you send, not after you fail. Laravel's Redis::throttle() or a RateLimited job middleware keeps you under the ceiling instead of discovering it. Cheaper and calmer than reactive retries.
3. Distinguish the error classes. 429 rate limit, retry with backoff. 429 insufficient quota, do not retry — your card failed, and retrying just fills the failed-jobs table. 400 context length, do not retry, truncate. 500/503, retry a few times. Blanket retry-on-any-exception is how you turn one bad request into 300.
4. Fail over, do not fail. The SDK accepts an array of providers: provider: [Lab::OpenAI, Lab::Anthropic]. For a feature that must stay up during a provider incident, that one argument is your whole disaster-recovery plan — provided your prompts and schemas are not so OpenAI-specific that the fallback produces garbage. Test the fallback path deliberately, at least once.

8. Production Gotchas Nobody Warns You About

Streaming makes moderation harder

OpenAI's own documentation makes this point: partial output is harder to evaluate, and moderation signals arrive after generation completes. If you are streaming to end users in a consumer product, you are showing text before you have judged it. Either accept that risk explicitly, or do not stream the user-generated-content-adjacent features.

Tools are an injection surface

The moment you give an agent a tool that reads your database, a user's prompt can attempt to steer that tool. Scope every tool to the authenticated user at the query level, never trust an ID that came out of a model, and validate tool arguments with the same rigour as a form request. Treat the model as an untrusted client calling your internal API.

Logging prompts is a data-protection decision

Full prompt logging is enormously useful for debugging and a liability the day someone pastes a national ID or a medical detail. Log token counts, model, latency, a prompt hash and a truncated preview by default. Log full prompts only behind a flag, with retention, and say so in your privacy policy — especially for clients in Saudi Arabia and the UAE, where personal-data rules are tightening.

Your evals are your regression tests

You cannot unit-test "is this reply good", but you can build a fixture set of thirty real inputs with expected classifications and assert against them. The SDK makes this cheap with SupportTriage::fake(), which accepts a fixed response, an array of responses, or a closure that inspects the incoming AgentPrompt. Fake in CI so your test suite costs nothing, and run the real eval set manually before you change a model or a prompt.

Version your prompts

Store a prompt version string with every logged call. When quality drops next quarter, the first question is "what changed", and "we edited the system prompt in March" is only answerable if you wrote it down.

9. What Does This Cost to Build?

Honest ranges, not a quote. These are the bands I quote from as a senior freelance developer working remotely from Cairo; agencies in London or Dubai will typically be two to four times higher for the same scope, which I unpack in freelance developer vs agency.

Assumptions behind those numbers: an existing Laravel 12 or 13 codebase in reasonable health, you supply the API account, and the API spend column assumes sane model routing rather than running everything on the flagship. Egyptian and Gulf clients often ask for the same figures in EGP or SAR — I quote in whichever currency you prefer, and the API cost is always billed to your own OpenAI account, never marked up through me.

The variable that moves the build cost most is not the AI. It is the state of the codebase around it. Adding an agent to a clean, tested Laravel app is a week. Adding one to a five-year-old app with no queue workers, no Horizon and business logic living in controllers is a month, because you have to build the foundation first.

10. A Ship-It Checklist

• Hard billing cap set in the provider dashboard, not just an alert.
• #[MaxTokens] on every agent, and a named model — never an auto-selected one.
• Cheap model as the default; flagship only on the paths that need it.
• Stable prompt prefix, long enough to clear the caching minimum, so caching actually engages.
• Per-user token metering written from an event listener, plus an enforced quota.
• AI jobs on a dedicated queue with $tries and explicit $backoff, and idempotency by input hash.
• 429 handling that reads Retry-After and adds jitter; no blanket retries on 400 or insufficient-quota.
• Streaming route excluded from proxy buffering and given a raised idle timeout, verified on the real server.
• Tools scoped to the authenticated user at the query level.
• Prompt hashes and versions logged; full prompts behind a flag with retention.
• An eval fixture set, plus fake() in CI so tests cost nothing.
• A documented fallback: what the feature does when the provider is down.

If eleven of those twelve are done, the feature is production-ready. If fewer than six are, you have a demo — and demos are exactly what get shipped to real users and then quietly turned off six weeks later when the bill arrives.

11. Work With Me

I build these features for a living, mostly for teams who already have a working Laravel product and want the AI part done properly rather than bolted on. If you want a second opinion on an architecture, a fixed-fee build, or a rescue of something already misbehaving in production, you can see how I work on the services page or hire me as a Laravel developer directly.

Send me the feature you have in mind through the contact page. Tell me the Laravel version, your current queue setup and what the feature should do. I will reply within 24 hours with a free consultation and a fixed-fee quote — and if I think the honest answer is that you do not need AI for this, I will say that instead.