Streaming

Use streaming when you want to show model output as it is generated instead of waiting for the full response. OpenModels supports OpenAI-compatible streaming through POST /chat/completions with stream: true.

Base URL:

https://api.getopenmodels.com/v1

Endpoint:

POST /chat/completions

Request

Set stream to true in the request body:

{
  "model": "qwen3.5-flash",
  "route": { "provider": "<provider-for-this-model>" },
  "route_mode": "balanced",
  "messages": [
    {
      "role": "user",
      "content": "Write a concise launch announcement for OpenModels."
    }
  ],
  "stream": true,
  "max_tokens": 512
}

cURL

Use -N so cURL does not buffer the response:

curl -N https://api.getopenmodels.com/v1/chat/completions \
  -H "Authorization: Bearer $OM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5-flash",
    "route": { "provider": "<provider-for-this-model>" },
    "route_mode": "balanced",
    "messages": [
      {
        "role": "user",
        "content": "Write a concise launch announcement for OpenModels."
      }
    ],
    "stream": true,
    "max_tokens": 512
  }'

The response is sent as server-sent events:

data: {"choices":[{"delta":{"content":"OpenModels"}}]}
data: {"choices":[{"delta":{"content":" gives developers"}}]}
data: [DONE]

Read each data: event, append choices[0].delta.content when present, and stop when you receive [DONE].

Python

import os
from openai import OpenAI
client = OpenAI(
    base_url="https://api.getopenmodels.com/v1",
    api_key=os.environ["OM_API_KEY"],
)
stream = client.chat.completions.create(
    model="qwen3.5-flash",
    messages=[
        {
            "role": "user",
            "content": "Write a concise launch announcement for OpenModels.",
        }
    ],
    stream=True,
    max_tokens=512,
    extra_body={
        "route": {"provider": "<provider-for-this-model>"},
        "route_mode": "balanced",
    },
)
for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)

Node.js

import OpenAI from "openai";
const client = new OpenAI({
  baseURL: "https://api.getopenmodels.com/v1",
  apiKey: process.env.OM_API_KEY,
});
const stream = await client.chat.completions.create({
  model: "qwen3.5-flash",
  route: { provider: "<provider-for-this-model>" },
  route_mode: "balanced",
  messages: [
    {
      role: "user",
      content: "Write a concise launch announcement for OpenModels.",
    },
  ],
  stream: true,
  max_tokens: 512,
});
for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}

Handling streamed output

• Append text deltas in order.
• Treat missing delta.content as normal. Some chunks only carry metadata.
• Keep a local buffer if you need to save the final answer.
• If the connection drops after text has started streaming, decide whether retrying could duplicate user-visible output.
• Log your client request id, model ID, endpoint, and any error.request_id returned by the gateway.

Billing

Streaming changes how output is delivered, not how usage is billed. Credits are consumed based on actual input and output token usage.