Streaming

Semua endpoint chat mendukung streaming Server-Sent Events. Token dikirim begitu dihasilkan model, tanpa disangga.

Format OpenAI

Kirim "stream": true. Tambahkan stream_options.include_usage bila Anda ingin chunk terakhir memuat usage.

text/event-stream

data: {"id":"...","choices":[{"index":0,"delta":{"content":"Halo"}}]}

data: {"id":"...","choices":[{"index":0,"delta":{"content":" dunia"}}]}

data: {"id":"...","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":2,"total_tokens":14}}

data: [DONE]

Chunk usage hanya dikirim ke Anda bila include_usage bernilai true. Penagihan di sisi MarkasAI tetap memakai usage tanpa memandang pengaturan ini.

Contoh

curl -N https://api.markas.ai.id/v1/chat/completions \
  -H "Authorization: Bearer $MARKASAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "messages": [{"role": "user", "content": "Ceritakan tentang Nusantara."}],
    "max_tokens": 500,
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.markas.ai.id/v1", api_key=os.environ["MARKASAI_API_KEY"])

stream = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Ceritakan tentang Nusantara."}],
    max_tokens=500,
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.markas.ai.id/v1", apiKey: process.env.MARKASAI_API_KEY });

const stream = await client.chat.completions.create({
  model: "MODEL_ID",
  messages: [{ role: "user", content: "Ceritakan tentang Nusantara." }],
  max_tokens: 500,
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) console.log("\n", chunk.usage);
}

Format Anthropic dan Responses

/v1/messages: event message_start, content_block_delta, message_delta, dan message_stop.
/v1/responses: event response.created, response.output_text.delta, response.completed, dan lainnya. Penagihan membaca usage dari response.completed.

Putus, batal, dan batas waktu

Bila sambungan penyedia terputus di tengah, Anda menerima event galat lalu [DONE]. Token yang sudah terkirim tetap ditagih.
Lama satu stream dibatasi, bawaannya 30 menit.
Putuskan koneksi untuk membatalkan. Token yang sudah dihasilkan ditagih.
Jangan pasang buffering di proxy perantara untuk jalur ini. Event harus diteruskan seketika.