liter-llm-running-the-proxy · xberg-io - Agent Skill

01 前言 Front Matter

name: liter-llm-running-the-proxy
description: Use when running the `liter-llm api` OpenAI-compatible gateway — virtual keys, per-key rate limits, budgets, cost tracking, and model routing. Covers the TOML config and the 22 REST endpoints.

02 资源文件 1

参考文档index.md484 B

03 SKILL.md 正文

Running the Proxy

liter-llm api is a drop-in OpenAI-compatible gateway: 22 REST endpoints that route to 174 providers, with multi-tenant virtual keys, rate limits, budgets, and cost tracking.

Start it

liter-llm api --config liter-llm-proxy.toml

The proxy auto-discovers liter-llm-proxy.toml in the current directory.

Configuration

[server]
host = "0.0.0.0"
port = 4000

[general]
master_key = "${LITER_LLM_MASTER_KEY}"

# Each [[models]] entry maps a routable name to a provider/model and its key.
[[models]]
name = "gpt-4o"
provider_model = "openai/gpt-4o"
api_key = "${OPENAI_API_KEY}"

[[models]]
name = "claude-sonnet"
provider_model = "anthropic/claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"

# Virtual keys scope which configured model names a caller may use.
[[keys]]
key = "sk-team-frontend"
models = ["gpt-4o", "claude-sonnet"]
rpm = 60
tpm = 100000
budget_limit = 50.0

${ENV_VAR} interpolation keeps secrets out of the file.

Call it like OpenAI

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-team-frontend" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

Notes

Virtual keys scope which models a caller may use and carry their own RPM/TPM and budget limits; the master_key administers them.
The OpenAPI 3.1 spec is served at /openapi.json.
Endpoints cover chat, embeddings, images, audio, moderations, files, batches, responses, and model listing.
Docker: docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-key ghcr.io/xberg-io/liter-llm.

04 同仓库更多 Skills

xberg-io/liter-llm

liter-llm

Universal LLM API client for 174 providers with native bindings for 14 languages. Use when writing code that calls LLM APIs via liter-llm in Python, TypeScript, Rust, Go, Java, C#, Ruby, PHP, Elixir, WASM, or C, when running the OpenAI-compatible proxy, or when calling LLMs through the MCP server. Covers chat, streaming, tool calling, embeddings, image generation, speech, transcription, moderation, web search, OCR, reranking, provider routing, middleware, and configuration.

开发工程师后端与接口Skill与智能体开发

★ 266

xberg-io/liter-llm

liter-llm-using-the-mcp-server

Use when calling LLM APIs through the liter-llm MCP server's 22 tools, and to decide when MCP beats the CLI or SDK. Covers the tool surface, the auto-installing launcher, and authentication.

后端与接口Skill与智能体开发

★ 266

xberg-io/liter-llm

liter-llm-tool-calling

Use when defining functions/tools for an LLM to call through liter-llm, or requesting structured JSON outputs. Covers tool schemas, tool_calls handling, and response formats.

★ 266

xberg-io/liter-llm

liter-llm-streaming-responses

Use when streaming tokens incrementally from an LLM via liter-llm over SSE or async iterators. Covers chat_stream, delta handling, and null-content chunks.

★ 266

xberg-io/liter-llm

liter-llm-embeddings-and-search

Use when generating embeddings, calling the 12 web-search providers, or running OCR over documents with the 4 OCR providers through liter-llm. Covers embed, search, and ocr methods plus reranking.

★ 266

xberg-io/liter-llm

liter-llm-calling-llms

Use when sending chat completions through liter-llm and routing to a specific provider via the `provider/model` prefix. Covers the chat call shape, provider routing, model_hint, message roles, and error categories.

★ 266