OpenAI Compatibility

Arbytra exposes an OpenAI-compatible API. If you already use the OpenAI SDK, change your client initialization to point at Arbytra and every call works the same way.

Update client initialization

Change the client initialization to point at Arbytra:

Python

# With OpenAI SDK
from openai import OpenAI
client = OpenAI(api_key="sk-...")

# With Arbytra — just change the init
import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

# Everything else stays the same
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

TypeScript

// With OpenAI SDK
import OpenAI from "openai";
const client = new OpenAI({ apiKey: "sk-..." });

// With Arbytra — just change the init
import OpenAI from "openai";
const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

// Everything else stays the same
const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
});

Check compatibility

Standard OpenAI API features work through Arbytra, including chat completions, the Response API, streaming, tool calling, structured output (json_schema), error classes, async clients, and the models list endpoint. Some features have special handling:

Feature	Behavior
Response API	Preview. See Response API overview and API reference.
Legacy `functions`/`function_call`	Arbytra auto-converts to `tools`/`tool_choice`
Structured output (`json_object`)	Model-dependent. See Structured output.

Use additional features

Arbytra adds capabilities on top of the OpenAI-compatible interface:

Routing options: Optimize for cost, latency, or throughput across providers.
Cost optimization: Arbytra computes the expected cost of each request at every available provider and routes to the cheapest one.
Prompt caching: Arbytra optimizes prompt caching automatically, reducing cost and latency on repeated requests.
Budget management: Set spending limits per workspace, API key, or BYOK provider. See Error codes for the budget_exhausted code.
Response headers: Every response includes request_id, rate limit headers, and credit usage. See Python SDK or TypeScript SDK.

Use OpenAI SDK directly

You can use the OpenAI SDK with a base_url override instead of the Arbytra package:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

response = client.chat.completions.create(
    model="claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Hello from Claude via Arbytra!"}]
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "claude-sonnet-4-20250514",
    messages: [{ role: "user", content: "Hello from Claude via Arbytra!" }],
});
console.log(response.choices[0].message.content);

This gives you access to models from multiple providers (Anthropic, Google, Meta, and others) through the OpenAI client. For typed routing metadata and error mapping, use the Arbytra SDK (Python, TypeScript) instead.

Map errors

If you use the OpenAI SDK directly, Arbytra errors arrive as generic openai.APIStatusError. You can convert them to typed Arbytra errors with map_openai_error() to branch on specific error codes like budget_exhausted or rate_limit_error:

import os
import openai
from arbytra import map_openai_error, RateLimitError, PermissionDeniedError

client = openai.OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
except openai.APIStatusError as e:
    arbytra_error = map_openai_error(e)
    if isinstance(arbytra_error, RateLimitError):
        # budget_exhausted is a 429 rate_limit_error; branch on .code if needed
        if arbytra_error.code == "budget_exhausted":
            print(f"Budget exhausted: {arbytra_error.message}")
        else:
            print(f"Rate limited. Retry after: {arbytra_error.retry_after_seconds}s")
    elif isinstance(arbytra_error, PermissionDeniedError):
        print(f"Permission denied (code={arbytra_error.code}): {arbytra_error.message}")
    else:
        raise arbytra_error

map_openai_error() is Python-only. TypeScript users get typed errors automatically with the Arbytra SDK. See Error Handling for the full guide.

Access routing metadata

Every Arbytra response includes routing metadata: which provider handled the request, the cost, and latency. How you access it depends on which SDK you use.

With the Arbytra SDK

Both the Python and TypeScript SDKs expose routing_metadata as a typed property on each response:

from arbytra import Client

client = Client()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.routing_metadata.provider)

import { Client } from "@arbytra/sdk";

const client = new Client();
const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
});
console.log(response.routing_metadata?.provider);

See the Python SDK or TypeScript SDK for the full client reference.

With the OpenAI SDK

Arbytra includes routing metadata in every response. If you use the OpenAI SDK, the arbytra Python package provides two helpers to extract it with full typing: parse_routing_metadata() extracts routing metadata from an OpenAI SDK response:

from arbytra.route_types import parse_routing_metadata

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)
metadata = parse_routing_metadata(response)
if metadata:
    print(f"Provider: {metadata.provider}")
    if metadata.cost:
        print(f"Cost: ${metadata.cost.usd}")

ArbytraAsyncOpenAI (experimental) is a drop-in AsyncOpenAI subclass that captures routing metadata on every response. Use it when a framework (OpenAI Agents SDK, LangChain, LlamaIndex) requires an AsyncOpenAI instance:

import asyncio
from arbytra import ArbytraAsyncOpenAI

async def main():
    client = ArbytraAsyncOpenAI()
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(client.last_routing_metadata.provider)

asyncio.run(main())

Install with pip install "arbytra[openai-compat]". See ArbytraAsyncOpenAI for framework wiring details.

These helpers are Python-only. For typed routing metadata in TypeScript, use the Arbytra SDK or @arbytra/ai-sdk-provider with the Vercel AI SDK.

Resources

Python SDK — full native client reference
TypeScript SDK — TypeScript client reference
Error handling — complete error handling guide

Get Started

Guides

Response API

Frameworks

Integrations

Specifications

Platform

Changelog

OpenAI Compatibility

Update client initialization

Python

TypeScript

Check compatibility

Use additional features

Use OpenAI SDK directly

Map errors

Access routing metadata

With the Arbytra SDK

With the OpenAI SDK

Resources

​Update client initialization

​Python

​TypeScript

​Check compatibility

​Use additional features

​Use OpenAI SDK directly

​Map errors

​Access routing metadata

​With the Arbytra SDK

​With the OpenAI SDK

​Resources

Update client initialization

Python

TypeScript

Check compatibility

Use additional features

Use OpenAI SDK directly

Map errors

Access routing metadata

With the Arbytra SDK

With the OpenAI SDK

Resources