General principles

Platforms and frameworks ship OpenAI-compatible clients. There are only two connection points: Base URL <your-gateway>/v1 and API key (your OpenTokenRouter key).

Dify

  1. Workspace → Settings → Model providers.
  2. Add OpenAI-API-compatible: API key, API base URL <your-gateway>/v1.
  3. Add available model IDs and set a default; they become selectable in app orchestration.

n8n

  1. Add an OpenAI node to the workflow.
  2. In the credential, choose a custom Base URL: <your-gateway>/v1, API key your key.
  3. Set the model parameter to an available model ID.

Coze

Add a custom model / plugin in the workspace with an OpenAI-compatible endpoint: API base <your-gateway>/v1, API key, and an available model ID.

LangChain (Python)

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="MODEL_ID",
    base_url="<your-gateway>/v1",
    api_key="sk-YOUR_KEY",
)

LlamaIndex

from llama_index.llms.openai import OpenAI

llm = OpenAI(
    model="MODEL_ID",
    api_base="<your-gateway>/v1",
    api_key="sk-YOUR_KEY",
)

LiteLLM

config.yaml:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: "MODEL_ID"
      api_base: "<your-gateway>/v1"
      api_key: "sk-YOUR_KEY"

After the proxy starts, consumers call LiteLLM while the gateway still meters by your key and pricing.

OpenAI SDK (Python / Node.js)

from openai import OpenAI

client = OpenAI(base_url="<your-gateway>/v1", api_key="sk-YOUR_KEY")
import OpenAI from 'openai'

const client = new OpenAI({
  baseURL: "<your-gateway>/v1",
  apiKey: "sk-YOUR_KEY",
})

Notes

  • Protocol routes: default Chat Completions; Anthropic clients use the Messages route (gateway root), Codex-style clients use Responses.
  • Metering and limits: every request is billed to the key; check the usage log. Framework auto-retries create multiple records; distinguish them by request id.
  • Concurrency and rate limits: frameworks default to high concurrency; on 429 lower concurrency or back off and retry.