Inference API.

§ 05.1Base URL & authentication¶

Everything lives under https://api.gpu.ai/v1 and authenticates with Authorization: Bearer gpuai_live_.... Get a key with gpu login or mint one in the dashboard. Scopes: serverless:read lists models, serverless:write runs chat, image, and video generation, and billing:read reads usage; a full_access key covers all three.

Because the API is OpenAI-compatible, the official OpenAI SDKs work unchanged for chat — just override the base URL and API key. The video endpoints follow the same submit → poll → download layout as OpenAI's video API.

§ 05.2Models¶

The catalog is live — built from the models actually being served warm right now, refreshed roughly hourly and probe-verified. Each entry carries a modality (chat, image, or video), pricing, and the request parameters the model accepts. Filter with ?modality=video.

Most chat models offer two tiers: serverless (default — low latency, no cold start; use the bare model id) and economy (cheaper, compute-priced, may cold-start ~30–60s; opt in with the :economy suffix, e.g. gpuai/qwen2.5-7b-instruct:economy).

§ 05.3Chat completions¶

Standard OpenAI chat completions, including stream: true for server-sent events.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gpu.ai/v1",
    api_key="gpuai_live_...",
)

stream = client.chat.completions.create(
    model="gpuai/qwen2.5-7b-instruct",
    messages=[{"role": "user", "content": "Explain FRP in one sentence"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

§ 05.4Image generation¶

Image generation is synchronous and OpenAI-compatible. v1 returns images as base64 only — data[].b64_json carries the bytes, and response_format: "url" is rejected. The response adds usage.image_count, the metered unit.

curl https://api.gpu.ai/v1/images/generations \
  -H "Authorization: Bearer gpuai_live_..." \
  -H "Content-Type: application/json" \
  -d '{"model":"gpuai/flux.1-schnell","prompt":"a watercolor fox","n":1,"size":"1024x1024"}'

{ "created": 1733800000, "data": [{ "b64_json": "iVBORw0KGgo..." }], "usage": { "image_count": 1 } }

§ 05.5Video generation¶

Video generation is asynchronous: POST /v1/videos returns a job in the queued state immediately. Poll GET /v1/videos/{id} until the status reaches a terminal value, then download the mp4 from GET /v1/videos/{id}/content.

# 1. Submit — returns a queued job immediately
curl https://api.gpu.ai/v1/videos \
  -H "Authorization: Bearer gpuai_live_..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "model": "gpuai/wan-2.2-t2v",
    "prompt": "ocean waves at sunset",
    "seconds": 5
  }'

# 2. Poll until status is terminal
curl https://api.gpu.ai/v1/videos/vid_... \
  -H "Authorization: Bearer gpuai_live_..."

# 3. Download the mp4
curl https://api.gpu.ai/v1/videos/vid_.../content \
  -H "Authorization: Bearer gpuai_live_..." -o out.mp4

{
  "id":           "vid_...",
  "object":       "video",
  "model":        "gpuai/wan-2.2-t2v",
  "status":       "completed",
  "progress":     100,
  "seconds":      5,
  "cost_cents":   18,
  "created_at":   1733800000,
  "completed_at": 1733800240,
  "expires_at":   1733886640
}

§05.5.1Job lifecycle¶

§05.5.2Parameters¶

Each model declares the request parameters it accepts in its catalog entry (parameters on GET /v1/models) — typically seconds, the clip length, whose bounds and default come from the model. Parameters a model does not declare are rejected with an invalid_request_error rather than silently ignored: live video models generate at their default output resolution, so most do not accept size. Omitting seconds uses the model's default clip length.

§05.5.3Billing, idempotency & retention¶

You're charged by seconds of output video, measured when the job completes. Jobs that never complete — failed, cancelled, or expired before finishing — are never billed. The authoritative charge is the cost_cents field on the job object; it stays on the job even after the artifact expires.

Completed videos are retained for 24 hours — the job carries an expires_at (unix seconds). After that, the artifact is gone and GET /v1/videos/{id}/content returns 410 Gone with an expired error code. Download promptly and store the file on your side.

Stop an in-flight job with POST /v1/videos/{id}/cancel (a cancel verb, not DELETE). Cancellation is idempotent — a terminal job is returned unchanged, and a job whose output is already committed can no longer be cancelled.

§ 05.6Audio generation¶

§ 05.7Errors¶

Errors use the OpenAI envelope, so SDK error handling works as-is:

{ "error": { "message": "...", "type": "invalid_request_error", "code": "model_not_found" } }

• model_not_found (404) — the live catalog no longer serves that model id. Re-fetch GET /v1/models and pick another. On streaming this arrives as the final SSE error event before [DONE].
• stream_limit_exceeded (429) — too many inference requests open at once on this API key. Close streams you're no longer reading or retry shortly; this is separate from the per-second rate limit (rate_limit_exceeded).
• upstream_unavailable (503) — a model's serving capacity is temporarily degraded. Returned promptly; retry after a short interval — the platform routes around degraded capacity automatically.