Streaming

Every chat endpoint supports Server-Sent Events streaming. Tokens are sent as soon as the model produces them, without buffering.

OpenAI format

Send "stream": true. Add stream_options.include_usage if you want usage in the last chunk.

text/event-stream

data: {"id":"...","choices":[{"index":0,"delta":{"content":"Halo"}}]}

data: {"id":"...","choices":[{"index":0,"delta":{"content":" dunia"}}]}

data: {"id":"...","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":2,"total_tokens":14}}

data: [DONE]

The usage chunk is only sent to you when include_usage is true. MarkasAI bills from usage regardless of this setting.

Examples

curl -N https://api.markas.ai.id/v1/chat/completions \
  -H "Authorization: Bearer $MARKASAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "messages": [{"role": "user", "content": "Tell me about the Indonesian archipelago."}],
    "max_tokens": 500,
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.markas.ai.id/v1", api_key=os.environ["MARKASAI_API_KEY"])

stream = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Tell me about the Indonesian archipelago."}],
    max_tokens=500,
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.markas.ai.id/v1", apiKey: process.env.MARKASAI_API_KEY });

const stream = await client.chat.completions.create({
  model: "MODEL_ID",
  messages: [{ role: "user", content: "Tell me about the Indonesian archipelago." }],
  max_tokens: 500,
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) console.log("\n", chunk.usage);
}

Anthropic and Responses formats

/v1/messages: message_start, content_block_delta, message_delta, and message_stop events.
/v1/responses: response.created, response.output_text.delta, response.completed, and other events. Billing reads usage from response.completed.

Disconnects, cancellation, and time limits

If the provider connection drops midway, you receive an error event and then [DONE]. Tokens already sent are billed.
A single stream has a time limit, 30 minutes by default.
Close the connection to cancel. Tokens already generated are billed.
Do not put a buffering proxy in front of this path. Events must be passed on immediately.