liter-llm

Why liter-llm

Work with any LLM provider without rewrites

Reach 165 providers — OpenAI, Anthropic, Google, Bedrock, Azure, Mistral, and more — through one client. Change the model name to switch providers; your code stays the same.

Your language, identical results

The same client ships to 14 languages — Python, TypeScript, Go, Java, Ruby, PHP, and more — all built on one Rust core, so behavior matches everywhere.

Chat, embeddings, images, speech — one API

Chat and streaming, tool calling, embeddings and rerank, images, speech and transcription, moderation, search, and OCR. One consistent surface for every task.

Drop-in OpenAI-compatible proxy

Point your existing OpenAI SDK at the built-in proxy to add virtual keys, budgets, and fallbacks. The same binary also serves your AI agents.

Secure by default

API keys are wrapped, redacted from logs, and never serialized. Cloud auth for Azure, AWS, and Vertex is handled for you — tokens refresh and rotate automatically.

Stay up when a provider goes down

Route by cost, latency, or weight, and fall back to another provider on failure — so one outage doesn’t take your app down. Set it per request or once in the proxy.

Language support

Language	Package	Docs
Rust	cargo add liter-llm	API Reference
Python	pip install liter-llm	API Reference
TypeScript / Node	npm install @xberg-io/liter-llm	API Reference
WebAssembly	npm install @xberg-io/liter-llm-wasm	API Reference
Go	go get github.com/xberg-io/liter-llm/packages/go	API Reference
Java	Maven Central io.xberg.literllm:liter-llm	API Reference
Kotlin (Android)	Maven io.xberg.literllm:liter-llm-android	API Reference
C#	dotnet add package XbergIo.LiterLlm	API Reference
Ruby	gem install liter_llm	API Reference
PHP	composer require xberg-io/liter-llm	API Reference
Elixir	{:liter_llm, "~> 1.7"}	API Reference
Dart / Flutter	dart pub add liter_llm	API Reference
Swift	Swift Package Manager	API Reference
Zig	zig fetch --save from GitHub	API Reference
C (FFI)	Shared library + header	API Reference
CLI	cargo install liter-llm-cli	Proxy Server
Docker	ghcr.io/xberg-io/liter-llm	Proxy Server

Quick example

Rust

src/main.rs

use liter_llm::{ChatCompletionRequest, ClientConfigBuilder, DefaultClient, LlmClient, Message, UserContent, UserMessage};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ClientConfigBuilder::new(std::env::var("OPENAI_API_KEY")?).build();
    let client = DefaultClient::new(config, None)?;

    let request = ChatCompletionRequest {
        model: "openai/gpt-4o-mini".into(),
        messages: vec![Message::User(UserMessage {
            content: UserContent::Text("Summarize liter-llm in one sentence.".into()),
            name: None,
        })],
        ..Default::default()
    };

    let response = client.chat(request).await?;
    println!("{}", response.choices[0].message.text().unwrap_or_default());
    Ok(())
}

Python

main.py

import asyncio
import os
from liter_llm import create_client

async def main():
    client = create_client(api_key=os.environ["OPENAI_API_KEY"])
    response = await client.chat({
        "model": "openai/gpt-4o-mini",
        "messages": [{"role": "user", "content": "Summarize liter-llm in one sentence."}],
    })
    print(response["choices"][0]["message"]["content"])

asyncio.run(main())

TypeScript

index.ts

import { createClient } from "@xberg-io/liter-llm";

const client = createClient(process.env.OPENAI_API_KEY!);

const response = await client.chat({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "Summarize liter-llm in one sentence." }],
});

console.log(response.choices[0].message.content);

Part of Xberg.io

XbergDocument intelligence: text, tables, metadata from 101 formats with optional OCR.

Xberg EnterpriseManaged extraction API with SDKs, dashboards, and observability.

crawlbergWeb crawling and scraping with HTML→Markdown and headless-Chrome fallback.

html-to-markdownFast, lossless HTML→Markdown engine.

liter-llmUniversal LLM API client with native bindings for 14 languages and 165 providers.

tree-sitter-language-packTree-sitter grammars and code-intelligence primitives.

alefThe polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

Explore the docs

Get StartedInstall liter-llm for your language, set an API key, and make your first call.

GuidesChat, embeddings, media, search, fallback routing, authentication, and the proxy/MCP servers.

ConceptsArchitecture, feature flags, the tokenizer model, and the cost-estimation pipeline.

ReferencePer-language API docs, the configuration schema, type catalogue, and error matrix.

ProvidersAll 165 runtime providers, model prefixes, auth modes, and endpoint coverage.

Proxy & MCPRun the OpenAI-compatible proxy and the Model Context Protocol server from one binary.

Getting help

• Bugs & feature requests — Open an issue on GitHub
• Community chat — Join the Discord
• Contributing — Read the contributor guide