Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

[1.13.0] - 2026-08-02

Added

Anthropic-compatible providers can now target a custom base URL (#159). anthropic/-prefixed models with a base_url override route to the Anthropic provider, so the Messages-API transforms (system extraction, content blocks, tool use, thinking) work against Anthropic-compatible endpoints such as DeepSeek’s https://api.deepseek.com/anthropic.
ReasoningEffort gains Minimal and Max variants (#160), covering DeepSeek’s max thinking level and OpenAI’s minimal level. Variants are appended, so existing FFI integer discriminants are unchanged.

Changed

extra_body is now shallow-merged into the request body root for OpenAI-compatible providers (#160), matching the OpenAI SDK. Top-level fields such as DeepSeek’s thinking switch reach the wire instead of being sent as a literal extra_body object. The transport-controlled stream flag cannot be overridden via extra_body.
Synced the model catalog from models.dev.
Upgraded dependencies (cargo upgrade --incompatible).

[1.12.2] - 2026-08-01

Added

cargo binstall liter-llm-cli support — prebuilt CLI binaries can now be installed directly from GitHub Releases without compiling from source. Adds [package.metadata.binstall] to the CLI crate plus a release-time verify-binstall CI job that installs via cargo binstall, smoke-tests the binary, and gates release-finalize.

Changed

Synced the model catalog from models.dev (#158).
Updated dependencies.

[1.12.1] - 2026-08-01

Added

MiniMax now exposes region-specific endpoints (#157): the global region (https://api.minimax.io/v1 for the OpenAI protocol, https://api.minimax.io/anthropic for the Anthropic protocol) and the China region (https://api.minimaxi.com/v1, https://api.minimaxi.com/anthropic), each with its own docs root.

Changed

Regenerated the bundled model catalog, refreshing provider/model data and provenance.

Fixed

The ai-rulez-generated plugin/.mcp.json bundle is now committed instead of being swept up by the broad .mcp.json gitignore rule, so a fresh CI checkout no longer fails ai-rulez verify --plugin with a missing-file error.

[1.12.0] - 2026-07-31

Added

liter-llm-cli installs a real OTLP export pipeline under the otel feature: it builds an OTLP span exporter + SDK tracer/meter providers, registers the W3C TraceContext propagator, and bridges tracing spans (and the library’s gen_ai.* metric instruments) to OpenTelemetry via a tracing-opentelemetry layer. Export is activated at runtime by OTEL_EXPORTER_OTLP_ENDPOINT; without it the CLI falls back to the console subscriber. Both the api proxy server and the mcp server share this install path, and the server Docker image is built with --features otel. (Previously the otel feature only compiled the OTel API with no exporter, so nothing was exported standalone — the instruments only reached a collector when a host such as xberg-enterprise installed the providers.)

Changed

Raw println!/eprintln!/print!/eprint!/dbg! are denied in production code across the workspace (clippy print_stdout/print_stderr/dbg_macro); tracing is the sole diagnostic surface. Genuinely non-diagnostic output opts back in per site with #[expect(...)] (the interactive GitHub Copilot device-flow auth prompt and the CLI’s top-level error report), and the catalog-gen dev tool keeps its stdout/stderr contract. Language bindings regenerated with alef 0.48.12.
The tracing Cargo feature is removed; tracing is now an always-on dependency. Spans and events compile unconditionally (near-zero cost with no subscriber installed), aligning with the org feature-flag contract that tracing is never optional. otel remains the opt-in OTLP export layer.

Fixed

Streaming no longer aborts on a trailing metadata-only SSE event that omits id. OpenAI-compatible providers such as OpenCode Zen/Go (https://opencode.ai/zen/go/v1) emit an inference-cost event with empty choices and no id/object/created/model immediately before [DONE]; these header fields on ChatCompletionChunk are now #[serde(default)], so the event decodes to an empty chunk instead of failing with missing field 'id'. (#155)

[1.11.5] - 2026-07-30

Changed

Upgrade the MCP server to rmcp 3.0, adopting the MCP 2026-07-28 protocol revision and its sessionless streamable-HTTP transport. The existing validate_api_key middleware and per-request KeyContext auth model are retained.

[1.11.4] - 2026-07-28

Fixed

SSE streams no longer abort with “invalid UTF-8” when a provider splits a multi-byte character across HTTP chunks. The SSE parser now buffers the trailing bytes of a codepoint that lands on a chunk boundary and reassembles it with the next chunk, instead of treating the incomplete sequence as corruption and terminating the stream. Common with CJK text, accented characters, emoji, or any sufficiently long response. Genuinely malformed UTF-8 still errors. (#152)

Changed

Refresh the model catalog from models.dev (now 165 providers).
Refresh Cargo.lock to the latest compatible dependency versions.

[1.11.3] - 2026-07-27

Changed

Refresh the model catalog from models.dev.
Regenerate all language bindings on alef 0.48.4, which lowers the Maven enforcer floor (fixes Java/Maven publishing) and generates the C# runtime.json template (fixes C#/NuGet publishing).
Upgrade dependencies to their latest incompatible versions (base64 0.22 → 0.23).

[1.11.2] - 2026-07-26

Changed

Regenerate all language bindings on alef 0.48.2.
Update dependencies to their latest compatible versions.

Removed

Remove unused Java PMD ruleset and stale linter configuration.

[1.11.1] - 2026-07-26

Fixed

list_models() on providers that omit object. ModelsListResponse.object is now #[serde(default)], so listing models no longer fails with missing field 'object' against OpenRouter and other OpenAI-compatible providers that omit the top-level "object": "list" field from /v1/models. (#149)

Changed

Dependency upgrades. pnpm 11.15 → 11.17, Jackson (databind/jdk8) 2.21.2 → 2.22.1, and a sorbet-runtime bump.

[1.11.0] - 2026-07-23

Added

OpenCode Zen and OpenCode Go providers — now 165 total (up from 163). OpenCode Zen routes via the opencode/ model prefix (https://opencode.ai/zen/v1); OpenCode Go routes via the opencode-go/ model prefix (https://opencode.ai/zen/go/v1). Both are OpenAI-compatible, bearer-authenticated (OPENCODE_API_KEY), and support chat completions.
reasoning_content support — StreamDelta and AssistantMessage now carry an optional reasoning_content field, so reasoning/thinking tokens from OpenAI-compatible providers (DeepSeek R1, Qwen, etc.) are preserved in both streaming and non-streaming responses instead of being dropped. Anthropic extended-thinking blocks are mapped into the same field. AssistantMessage gains a reasoning_text() accessor.

Changed

base64 embedding responses. EmbeddingObject.embedding now accepts either a JSON float array or a base64 string of little-endian f32 bytes (the OpenAI-compatible encoding_format: "base64" response), decoding base64 vectors that previously failed to deserialize. Payloads are ~2.3× smaller on the wire and decode ~15× faster via a zero-copy Visitor. The element type narrows from Vec<f64> to Vec<f32> (float32 is what providers send on the wire); the high-level embed() API was already f32, so only code reading the raw EmbeddingObject.embedding field as f64 needs a one-line adjustment.

[1.10.1] - 2026-07-20

Fixed

Swift: tool-call responses now decode correctly — the type field on tool calls honors its wire name instead of throwing keyNotFound.
Swift: transcription segments, tool_calls, and other optional list fields on response DTOs now decode instead of throwing DecodingError.typeMismatch.
Elixir: streaming entry points now call chat_stream/2 (a non-existent chat_stream_async/2 was generated previously), fixing UndefinedFunctionError.
Regenerated all language bindings with alef 0.38.4.

[1.10.0] - 2026-07-19

Added

Public ModelInfo DTO and cost::model_info(name) accessor, exposing per-model metadata — context window, max input/output tokens, mode, and capability flags (vision, reasoning, tool calling, structured output, audio) — across all language bindings.
Opt-in runtime catalog refresh. cost::refresh_catalog(&CatalogRefreshConfig) overlays an updated catalog.json on top of the embedded catalog at runtime, without a rebuild. It is a runtime toggle, not a Cargo feature, and is disabled by default (enabled: false). A failed, unreachable, or air-gapped refresh always falls back to the embedded (or prior) catalog, so a refresh failure never degrades availability. Also exposed: RefreshOutcome, CatalogRefreshError, install_catalog_overlay_from_str, clear_catalog_overlay, and DEFAULT_CATALOG_URL.
Extended per-model pricing. The catalog now carries context-tiered pricing (e.g. distinct rates above a 200k-token context), plus audio and reasoning token costs, all consumed by completion_cost. New ModelTier type.
20 new providers — now 163 total (up from 143) — including Black Forest Labs, Reducto, Soniox, Nvidia Riva, Google Distributed Cloud (Gemini), and AWS Bedrock Mantle.
Rust catalog generator (liter-llm-catalog-gen) replacing the Python generate_pricing.py / generate_providers_doc.py scripts. It fetches and strictly validates the models.dev api.json (rejecting unknown fields so upstream drift fails loudly) and dual-writes catalog.json with a $provenance block (source SHA-256, fetch date, library version). New tasks generate:catalog and generate:catalog:check.
Catalog automation workflows. A daily sync-catalog workflow opens an auto-merging PR when models.dev drifts; publish-catalog pushes catalog snapshots to a rolling model-catalog release; catalog-verify gates PRs that touch the catalog.

Changed

Renamed the embedded pricing registry from pricing.json to catalog.json (both the canonical schemas/ copy and the crate’s crates/liter-llm/schemas/ copy), reflecting its expanded per-model metadata.
Upgrade dependencies: jsonschema 0.46 → 0.48, rmcp 2.1 → 2.2, tokio-tungstenite 0.29 → 0.30. Regenerate all bindings with alef 0.38.0.

Fixed

list_models() no longer fails for providers that omit created (e.g. DeepSeek). ModelObject.created, object, and owned_by now default when absent from a provider’s /v1/models response. Fixes #139.

[1.9.3] - 2026-07-04

Changed

Upgrade all dependencies to latest and regenerate all bindings with alef 0.31.0. Notable major bumps: rmcp 1.7 → 2.1, etcd-client 0.15 → 0.19, tokio-tungstenite 0.26 → 0.29, tower-http 0.6 → 0.7, sha2 0.10 → 0.11, cel-interpreter 0.8 → 0.10, and tikv-jemallocator 0.6 → 0.7.

Fixed

CEL guardrail: adapt to cel-interpreter 0.10. Program::compile now returns ParseErrors, and the new ANTLR-based parser can panic on some malformed input. CelGuardrail::new now catches those panics and returns a CelCompileError, so an invalid expression can never abort the caller (e.g. proxy startup).
Proxy MCP server: adapt to rmcp 2.1 model changes (RawResource → Resource, RawResourceTemplate → ResourceTemplate, Content → ContentBlock, PromptMessageRole → Role).
Proxy etcd config watcher: adapt to etcd-client 0.19, whose watch now returns a single WatchStream and moves cancel onto the stream.

[1.9.2] - 2026-07-02

Fixed

Ruby: allow Ruby 4.x. Drop the < 4.0 upper bound on the gem’s required_ruby_version (now >= 3.2.0), so installing/updating the gem no longer fails on Ruby 4.0+. Fixes #136.

[1.9.1] - 2026-07-02

Changed

Upgrade dependencies and regenerate all bindings with alef 0.30.11.
Add [workspace] extra_clippy_allows = ["single_match", "collapsible_match"] to alef.toml so the generated single-variant enum-from-int bindings pass clippy -D warnings.

Fixed

Correct the HashiCorp vendor name spelling in the proxy Vault secret backend (HashiCorpVaultProvider/Builder), its comments, docs, and this changelog.

[1.9.0] - 2026-06-27

Stable release promoting 1.9.0-rc.2 (fully published green). No code changes beyond the version bump.

[1.9.0-rc.2] - 2026-06-27

Changed

Rebrand to Xberg. All published package coordinates move off the kreuzberg brand: npm scope @kreuzberg/* → @xberg-io/* (main, the six napi platform packages, -wasm, and -cli); the Java/Maven and Kotlin Android namespace dev.kreuzberg.literllm[.android] → io.xberg.literllm[.android] (group id dev.kreuzberg → io.xberg), which relocates the generated Java/Kotlin source trees from dev/kreuzberg/… to io/xberg/…. Docs, badges, the ecosystem block (now links the Xberg product at github.com/xberg-io/xberg), the brand heading (“Part of Xberg.io”), and CI publishing references (owner: xberg-io, ghcr.io/xberg-io, packagist xberg-io, the xberg-dev-publisher app, xberg-io/homebrew-tap) are updated to match. The legal entity name “Kreuzberg, Inc.” is unchanged. (alef.toml, templates, generated bindings, docs, publish.yaml)
Regenerate with alef 0.29.3 (from 0.29.1): correct Kotlin Android Maven coordinate, drop redundant -node npm alias.

Removed

Drop the unused legacy packages/typescript wrapper package. It was a re-export barrel of the native @xberg-io/liter-llm binding and was never published. alef no longer generates it (the napi backend stopped emitting packages/typescript); the canonical TypeScript surface is the native package’s bundled index.d.ts. Removed the directory and its references in the workspace list, dependabot.yaml, the validate-versions manifest path (now crates/liter-llm-node/package.json), the node:typecheck task, and the oxlint exclude.

[1.8.2] - 2026-06-23

Fixed

Publish (zig): the published zig package compiles again. The “Stage FFI artifacts” step bundled a stale top-level crates/liter-llm-ffi/liter_llm.h (old ABI, missing literllm_create_client/*_from_json) into the ffi-* artifact, so the zig wrapper’s @cImport resolved against an out-of-date header and every test failed to compile. The step now stages the canonical include/liter_llm.h (and fails hard if absent); the orphaned top-level header is removed.
Publish (kotlin-android): the AAR no longer corrupts on download. Both ABI matrix legs emitted the same filename liter-llm-android-release.aar; the publish job’s download-artifact … merge-multiple: true collided them into one torn zip (bad zipfile offset). Each leg now uploads an ABI-suffixed liter-llm-android-<abi>.aar.
Publish (swift): a rerun can no longer desync the artifact-bundle checksum. The swift-artifactbundle job now reuses an existing release asset and its checksum instead of rebuilding a non-reproducible bundle, keeping the hosted bundle consistent with the swift-<version> tag.
Swift test-app scaffold pins the checksum-bearing ref. Regenerated from alef 0.26.7: the registry-mode Package.swift now uses .package(url:, branch: "release/swift/<version>") (which carries the substituted checksum) instead of from: "<version>" (which resolved the placeholder-bearing SemVer tag).

Changed

Bindings regenerated from alef 0.26.7 (from 0.26.5): swift e2e scaffold branch pin and kotlin-android Gradle plugin/dependency bumps. Generated node e2e/test_apps emit .npmrc disabling frozen-lockfile.

[1.8.1] - 2026-06-23

Fixed

CI E2E (kotlin_android): the host-JVM test project compiles again. The generated MockServerListener implements the JUnit Platform LauncherSessionListener SPI (referencing LauncherSession/LauncherSessionListener at compile time), but the e2e build.gradle.kts scoped junit-platform-launcher as testRuntimeOnly, so compileDebugUnitTestKotlin failed with “Unresolved reference ‘launcher’”. Regenerated from alef 0.26.5, which now scopes it testImplementation.
CI E2E (swift): the e2e link step now finds liter_llm_ffi. The swift e2e before step built liter-llm-swift and the mock server but never liter-llm-ffi, so libliter_llm_ffi.a was absent from the linker search path (ld: library 'liter_llm_ffi' not found). The [crates.test.swift].before step now also runs cargo build --release -p liter-llm-ffi.
CI E2E (node): the lockfile is back in sync. pnpm-lock.yaml was stale against crates/liter-llm-node/package.json (missing @napi-rs/cli ^3.6.2) after the 1.8.0 bump, failing pnpm install --frozen-lockfile. Regenerated.

Changed

Bindings regenerated from alef 0.26.5 (from 0.26.3): Swift bridge-glue re-materialization runs after the bridge crate is built, opaque-handle aliasing avoids capsule import collisions, and JSON-string overloads emit positional arguments for underscore-prefixed parameters.

[1.8.0] - 2026-06-22

Added

MCP server: tool annotations on every tool. Each MCP tool now advertises rmcp ToolAnnotations (a human-readable title plus readOnlyHint/destructiveHint/idempotentHint/openWorldHint) so clients can present them and decide auto-approval. Query tools are read-only; create_* mutate without being destructive; delete_*/cancel_* are destructive and idempotent; all reach external providers (openWorldHint).
MCP server: prompts, resources, and argument completion. Beyond tools, the server now exposes reusable prompt templates (summarize, translate, extract), catalog resources (liter-llm://models, liter-llm://providers, and the liter-llm://pricing/{model} / liter-llm://provider/{name} templates), and argument completion for model (from the configured models) and provider name (from the registry). get_info advertises tools, prompts, resources, and completions.

Fixed

Budget middleware: concurrent spend is no longer lost across a window rollover. On weakly-ordered architectures (arm64) the window-reset path could drop a racing fetch_add; the rollover now subtracts the snapshotted prior total instead of storing zero, preserving every concurrent charge.
CI: the Kotlin Android build provisions the wrapper-declared Gradle version (gradle-version: wrapper) in ci-mobile and ci-e2e. AGP 9.2.0 requires Gradle >= 9.4.1; the previous hardcoded 8.13 pin silently overrode the 9.6.0 wrapper and broke assembleRelease.
Swift & Dart: the untagged content-union text() accessor now references the generated payload field instead of a non-existent binding, mirroring the 1.7.6 Kotlin fix. Regenerated from alef 0.26.3.
CI/release: create the draft GitHub Release once in the prepare job so the Swift artifact-bundle upload (the only upload job without its own ensure-release step) can no longer race ahead of release creation and fail with “release not found”. Mirrors the html-to-markdown publish pipeline.

Changed

Bindings regenerated from alef 0.26.3 (from 0.25.60): Swift capsule-pointer bridging via usize plus .product dependency wiring, RustBridgeC.h preserved across alef all --clean, and the content-union accessors above.

[1.7.6] - 2026-06-22

Fixed

Kotlin Android: the untagged content-union text() accessor now references the generated value property instead of the non-existent field0, fixing a :compileReleaseKotlin failure (Unresolved reference 'field0') that broke the Android AAR build. Regenerated from alef 0.25.60; the legitimate field0 data-class properties (e.g. LiterLlmError.Serialization) are unaffected.

[1.7.5] - 2026-06-21

Changed

npm: the primary package is now the bare @kreuzberg/liter-llm (matching the crawlberg convention), renamed from @kreuzberg/liter-llm-node. The napi packageName and all per-platform sub-packages use the bare prefix (@kreuzberg/liter-llm-<rid>); the .node binaryName stays liter-llm-node. (alef.toml, generated node binding, all README badges, docs, pnpm-lock.yaml)

Fixed

Packagist: publish under the canonical xberg-io/liter-llm vendor (matching html-to-markdown, crawlberg, tree-sitter-language-pack) instead of the legacy kreuzberg/liter-llm. Fixes the registry-check coordinate and the publish-packagist step (packagist-username, package-name) in publish.yaml, plus the stale reference in llms.txt. The Maven registry-check coordinate is corrected to dev.kreuzberg.literllm:liter-llm.
CI/release: @kreuzberg/liter-llm-cli now publishes on the main release at the release version (previously decoupled on a separate cli-proxy-v* tag), gated behind the CLI-binary upload so the npm wrapper never ships ahead of the binaries it downloads.
CI/release: disable sccache for release build jobs so a transient sccache cache/DNS failure can no longer block a publish run (publish.yaml).
docs: refresh stale install snippets — Java/Kotlin/Swift/Zig version pins, the Java Maven coordinate, and the Elixir version range in docs/getting-started/installation.md, docs/index.md, and llms.txt.

[1.7.4] - 2026-06-19

Changed

chore(precommit,alef): standardize kotlin-android formatting on ktfmt –kotlinlang-style. Drop the conflicting prek ktlint hook (it ran a destructive --format that fought ktfmt and rewrote alef’s /// doc comments), scope ktfmt to packages/kotlin-android with --kotlinlang-style, switch alef.toml kotlin format/check from gradle-ktlintFormat to ktfmt so alef and prek agree, and exclude the vendored Gradle wrapper from shellcheck. detekt remains. (.pre-commit-config.yaml, alef.toml)

Fixed

Content-union e2e gate completed across the remaining bindings (regenerated with alef 0.25.49). The Dart binding now injects the AssistantContent.text() extension even when FRB emits the freezed mixin clause (sealed class … with _$… {); the PHP flat-enum conversion always emits an exhaustive wildcard for the &str tag match, so alef(skip)’d-Default enums (Message, ContentPart, AssistantPart, OcrDocument) compile; and the Node/NAPI binding keeps both From conversion directions for plain data enums used as struct fields (e.g. AuthHeaderFormat in CustomProviderConfig). Local e2e green for Python, Node, WASM, Elixir, and PHP.

Documentation

Bindings parity docs. Added a feature-by-language support matrix (chat, streaming, tool calling, embeddings, multimodal in/out, call idiom) across all 14 native bindings plus the C/FFI surface, expanded the multimodal cookbook with idiomatic examples in nine languages, and reconciled the binding count to a single “14 native bindings + C/FFI surface” across the README and docs (added Dart, Swift, Kotlin Android, and Zig to the README Language READMEs table).

[1.7.3] - 2026-06-19

Fixed

Cross-language e2e for multimodal message.content. The remaining bindings can now string-assert the assistant content union (AssistantContent), fixing the e2e suites that broke on it: Kotlin and Dart get a text() accessor on the sealed class; WASM returns the display text (String) instead of a discriminant; Swift renders property access for first-class result structs; Elixir reads the NIF struct’s .text; PHP calls the message’s text() accessor. Generated via alef 0.25.48 (untagged_union_text_types + fields_display_as_text extended to all backends).
Android AAR packaging guard — the Kotlin/Android publish now stages the cross-compiled JNI .so libs into the AAR and fails loudly if jni/ would be empty, so a jni-less AAR can never be published.

[1.7.2] - 2026-06-18

Added

Display for AssistantContent — renders the message text (Text variant verbatim; Parts variant concatenates its text segments, skipping non-text parts), enabling string assertions on message.content across the polyglot e2e suites.

Fixed

Cross-language e2e content assertions — the generated e2e suites stringified choices[0].message.content with plain-string casts that fail for the multimodal AssistantContent union (Go string(*ptr), Java Objects.toString, C# .ToString(), Rust .as_deref()). alef 0.25.45’s fields_display_as_text config now emits the per-language text accessor so the assertions compile and assert text content.

[1.7.1] - 2026-06-18

Fixed

FFI build under -D warnings — the regenerated liter-llm-ffi crate referenced #[cfg(feature = "wasm-http")] without declaring the feature, producing unexpected cfg condition value: wasm-http errors that broke CI Rust, CI E2E (Build FFI), CI Mobile, and the crates.io publish in v1.7.0. alef now declares non-default passthrough features via the configurable [crates.ffi] extra_features key, so wasm-http is declared (but not enabled) and survives regeneration.
Java PMD on generated DTOs — the new DecodedDataUrl { mime, byte[] data } value object tripped ArrayIsStoredDirectly / MethodReturnsInternalArray; alef’s PMD ruleset now excludes these for generated DTOs.
Docs strict build — fixed a broken intra-doc link in docs/usage/multimodal.md (../concepts/providers.md → ../providers.md).

[1.7.0] - 2026-06-18

Added

Typed multimodal builders — liter_llm::image::{encode_data_url, decode_data_url, DecodedDataUrl} with IMAGE_PNG/IMAGE_JPEG/IMAGE_WEBP/IMAGE_TIFF MIME constants. decode_data_url returns a named DecodedDataUrl { mime, bytes } struct rather than a tuple so polyglot bindings extract it as a typed object.
Message::user_with_parts(parts) — ergonomic constructor for multimodal user messages.
ContentPart::{text, image_data_url, image_url, image_with_detail, image_png, image_jpeg, image_webp, image_tiff} — typed constructors replacing hand-rolled struct construction.
ResponseFormat::{json_schema, json_object, text} + JsonSchemaFormat::new(name, schema).strict(bool).description(d) fluent builder. new defaults to strict = Some(true). Provider-mapping rustdoc on ResponseFormat (OpenAI passthrough, Gemini/Vertex responseMimeType+responseSchema, Anthropic system-instruction injection).
Multimodal output: AssistantContent enum (Text / Parts) with #[serde(untagged)] back-compat; AssistantPart (Text / Refusal / OutputImage / OutputAudio) with #[serde(tag = "type", rename_all = "snake_case")]; AssistantMessage::{text, refusal_text, output_images, output_audio} accessors; Message::{assistant_with_parts, system_with_parts} constructors; ChatCompletionRequest.modalities: Option<Vec<Modality>> with Modality::{Text, Audio, Image}. Vertex transform_response preserves inline_data as OutputImage/OutputAudio (no base64 re-encode); OpenAI transform_response hoists message.audio into Parts([Text(transcript), OutputAudio]).

Changed

BREAKING: AssistantMessage.content: Option<String> → Option<AssistantContent>. Back-compat via From<String>/From<&str> for AssistantContent and the untagged serde variant — providers returning scalar content strings still deserialize as Text(_).
BREAKING: SystemMessage.content: String → UserContent. Back-compat via From<String>/From<&str> for UserContent.
liter-llm makes base64 an unconditional dependency (previously gated behind native-http/wasm-http) — liter_llm::image::* helpers are transport-agnostic.
All polyglot bindings regenerated for the multimodal surface (alef 0.25.40 pin). PHP/WASM extraction of #[serde(untagged)] enums (AssistantContent) and nested complex enum-variant types (OutputImage { image_url: ImageUrl }, OutputAudio { audio: AudioContent }) is resolved upstream; bindings construct/access multimodal types via each language’s native idiom.

[1.6.4] - 2026-06-17

Changed

Gate etcd-client behind optional etcd-watch feature; default CLI builds no longer require protoc. Resolves the Homebrew bottle source-build failure on v1.6.3.
Bump alef codegen pin to 0.25.24 (swift opaque-handle class triples, dart () -> () cleanup, kotlin per-file ktfmt invocation, java PMD/palantir-java-format alignment, c e2e FFI tarball-name alignment, php exclude_functions filtering through to user-facing wrapper).
liter_llm::tower::metrics::global_meter() is now pub so downstream crates (liter-llm-proxy) can read the shared OTel meter without re-initialising.

Fixed

ContentPart crate-root shadow — src/lib.rs did pub use types::* (which brings types::ContentPart to the crate root) and then explicitly pub use realtime::ContentPart, causing the realtime variant to shadow the types one. Downstream consumers writing use liter_llm::ContentPart; received the realtime variant, which has no ImageUrl variant, producing E0599: no variant named 'ImageUrl' found for enum 'liter_llm::ContentPart' in any VLM-OCR call site. The realtime ContentPart re-export is removed from lib.rs; callers that need it must import it explicitly as liter_llm::realtime::ContentPart. A compile-time regression test in src/tests.rs asserts that crate::types::ContentPart::ImageUrl is constructible through the crate root.
rustdoc ICE in liter-llm — bracketed intra-doc links to private items (MAX_POOL_BUFFER_CAPACITY, post_stream_with_cancel, post_json, post_json_raw, retry::should_retry, LiterLlmError, ConfigDrivenProvider::transform_request) in http//provider/ triggered an internal compiler error on rustc 1.95.0 during doc_link_resolutions. Stripped the link brackets so the doc strings reference the names as plain code identifiers. Also fixed bare URL in CustomProviderConfig::base_url.
Missing-docs cleanup — added documentation for UsageSinkErased trait and method, the four tenant modules (context, etcd, in_memory, resolver), the http::transport module, and the RouterError::Discover { source } struct field.

[1.6.3] - 2026-06-16

Fixed

WASM build: tokio::time inside wait_for_batch_impl — crates/liter-llm/src/client/mod.rs polled with tokio::time::Instant and tokio::time::sleep unconditionally, but tokio is gated behind the native-http feature. Under wasm-http-only builds (the WASM crate) compilation failed with E0433: cannot find module or crate 'tokio'. The function now switches to web_time::Instant + gloo_timers::future::sleep on target_arch = "wasm32", with web-time added as an optional dep on the wasm-http feature.
WASM build: leaked tower DTOs — alef regen at v1.6.0 started emitting From<liter_llm::tower::{CircuitState, HealthStatus, IntentPrototype, SingleflightResult}> impls in crates/liter-llm-wasm/src/lib.rs, but tower is not enabled under wasm-http. Added the four types to alef.toml [crates.wasm].exclude_types alongside the existing config-DTO exclusions.
Ruby gem build: missing feature declarations — packages/ruby/ext/liter_llm_rb/native/Cargo.toml did not declare native-http/opendal-cache/wasm-http, so alef-emitted #[cfg(feature = "native-http")] gates in src/lib.rs resolved as false and the ensure_crypto_provider call site failed with E0425: cannot find value in this scope. Regen now writes the features section. Affected every Ruby gem variant (linux-x86_64, linux-aarch64, macos-arm64).
PHP windows builds: ext-php-rs macro lookup — LiterLlmApi::ensure_crypto_provider resolution failed inside #[php_impl]-generated code on target_os = "windows" (E0599: no associated function or constant). Other bindings (PyO3, NAPI, Rustler) handle the same pattern correctly, so this looks like an ext-php-rs upstream gap. Excluded ensure_crypto_provider from the PHP binding entirely via alef.toml — the function is a no-op on Windows anyway, and downstream PHP users rely on transitive invocation from internal reqwest::Client constructors.
PHP PIE matrix: php8.5 macos-arm64 — shivammathur/setup-php@v2 does not yet ship a PHP 8.5 image for macOS arm64 runners (PHP 8.5 was released November 2025). Excluded that matrix cell from publish.yaml until the upstream action catches up. Linux + Windows PHP 8.5 builds remain in the matrix.

[1.6.2] - 2026-06-16

Fixed

to_singleflight_error dead-code lint — gated LiterLlmError::to_singleflight_error with #[cfg(feature = "tower")] so the method is not emitted when cargo publish --verify builds with default features (default = ["native-http"]). The method’s only call sites live in src/tower/cache_singleflight.rs, which is itself feature-gated. Under -D warnings the dead-code lint rejected the build, blocking Publish crates.io and every per-platform Build CLI binary, Build WASM, and Build Kotlin Android natives job in the v1.6.0 and v1.6.1 release workflows. No artifacts reached PyPI, crates.io, npm, RubyGems, Maven Central, Packagist, Hex, NuGet, pub.dev, or the Homebrew tap from those tags.
Release-runner protoc toolchain dep — xberg-io/actions/setup-rust@v1.8.70 now installs protobuf-compiler on every Linux/macOS/Windows runner. etcd-client v0.15’s build.rs shells out to protoc; liter-llm-cli pulls etcd-client transitively through liter-llm-proxy, so the v1.6.1 CLI binary builds panicked with Failed to compile proto files: Could not find protoc. v1.6.2’s publish workflow consumes the floating v1 tag which now carries the fix.

[1.6.1] - 2026-06-16

Fixed

FFI crate wasm-http feature — declared wasm-http as a no-op feature on liter-llm-ffi/Cargo.toml so the alef-emitted #[cfg(any(feature = "native-http", feature = "wasm-http"))] gates in crates/liter-llm-ffi/src/lib.rs resolve under cargo build -D warnings. v1.6.0’s publish workflow failed on every Rust FFI, CLI, WASM, and Kotlin Android native build with unexpected_cfgs errors because the gate was emitted without the corresponding feature declaration.

[1.6.0] - 2026-06-16

Bindings

All 14 language bindings regenerated against alef 0.25.18. Covers Python (PyO3), TypeScript (NAPI-RS), Ruby (Magnus), PHP (ext-php-rs), Elixir (Rustler), Go (cgo), Java (Panama FFM), C# (P/Invoke), Swift (swift-bridge), Dart (flutter_rust_bridge), Kotlin Android (JNI), Zig, C, and WASM.
Tier A/B/C API triage applied to the v1.6.0 surface: internal types stay Rust-only (Tier A), trait-generic helpers like wait_for_batch_impl and the BatchRetriever trait stay Rust-only (Tier B), and binding-exposed concrete methods like DefaultClient::wait_for_batch cross the FFI boundary (Tier C). Tier B trait methods (BatchRetriever::fetch_batch_for_polling) now emit correctly in JNI without a workaround.
Swift chat_stream streaming adapter restored across all bindings. The swift-bridge extern block emitted by alef now declares type DefaultClient; inside the streaming extern block so the owner reference resolves.

Tooling

alef bumped from 0.23.7 → 0.25.18. New upstream fixes consumed:
- JNI backend emits per-trait use core_crate::{path}; clauses for non-root trait methods.
- swift-bridge streaming-adapter extern block declares owner types alongside handle types.
- cleanup_orphaned_files recognises hash-less stale files (self-referential // auto-generated by alef header).
- Dart and PHP cfg-feature forwarding fixes for the wasm-http feature gate.

Added

KeyContext::tenant_id: TenantId — tenant identifier carried in every resolved auth context. Master-key auth always sets TenantId("master") (see MASTER_TENANT_ID constant); virtual-key auth propagates the tenant_id returned by the configured KeyResolver.
KeyContext::from_resolved(key_id, &ResolvedKey) — constructor that builds a KeyContext from a KeyResolver-resolved record, preserving the canonical tenant_id from the resolver rather than falling back to the raw key token.
auth::MASTER_TENANT_ID: &str — well-known constant ("master") used by KeyContext::master() so downstream budget and usage layers can identify master-key traffic without a special-case enum.

Changed

KeyContext now carries tenant_id: TenantId resolved by the configured KeyResolver. Master-key auth resolves to TenantId("master").
Every proxy HTTP handler now propagates tenant_id into LlmRequest::with_tenant_id, activating BudgetLedger::Tenant, TenantScopedStrategy, and UsageEvent.tenant_id for all in-proxy traffic. The dispatch helper in routes/mod.rs applies with_tenant_id centrally; chat.rs (which bypasses dispatch for stream/non-stream branching) sets it inline.
validate_api_key middleware now resolves virtual keys via AppState.key_resolver.resolve() (previously used KeyStore.get() directly), so custom KeyResolver backends injected via ProxyServer::with_key_resolver receive calls for all virtual-key traffic.
ProxyServer::with_key_resolver and ProxyServer::with_usage_sink builder methods for embedder-supplied dependencies. Default behaviour unchanged.
AppState.usage_sink: Option<Arc<dyn UsageSinkErased>>; HooksLayer is now wired outermost in the Tower stack when a sink is configured.
UsageEvent.effective_model: Option<String> — provider-echoed model name from the response, distinct from model which is the requested name. Populated for Chat, Embed, Moderate, Ocr, and Search response variants; None for streaming, speech, transcription, rerank, image generation, and list-models variants, and on error/timeout paths.
UsageEvent.cache_state is now accurate. Previously hardcoded Bypass; now reflects Miss, ExactHit, SemanticHit, or StaleHit set by CacheLayer and SingleflightService (followers) via a tokio::task_local! cell read by HooksLayer after the inner service resolves. Requests with no CacheLayer in the stack continue to report Bypass.
DefaultClient::wait_for_batch(batch_id, WaitForBatchConfig) — poll a batch until terminal status (Completed, Failed, Expired, Cancelled) with exponential backoff, configurable intervals, and optional timeout. Returns Ok(BatchObject) on completion, Err(BatchWaitError::{Failed, Timeout, Client}) on failure.
WaitForBatchConfig — configuration struct with initial_interval_secs: f64, max_interval_secs: f64, backoff_multiplier: f32, and optional timeout_secs: Option<f64> fields; implements Default with 5.0s initial, 60.0s max, 1.5x multiplier, no timeout. All fields use f64 seconds (not Duration) for FFI bridgeability.
BatchWaitError — error enum with struct-style variants Failed { status: BatchStatus }, Timeout { timeout_secs: f64 }, and Client { message: String, code: u32 } for batch polling failures. All fields are FFI-friendly primitives.
liter_llm::observability::{UsageEvent, UsageSink, CacheState, UsageEventOutcome, UsageSinkError, LoggingUsageSink, MultiUsageSink} — canonical per-request usage events with pluggable sinks. UsageEvent is billing/observability-vendor-agnostic; downstream consumers translate it into metrics, ledgers, or OTel events.
HooksLayer::with_usage_sink — wires a UsageSink into the Tower stack; emits one event per request completion (success or error). Sink errors are best-effort: logged and not propagated to callers.
liter_llm::tower::IdempotencyLayer — Idempotency-Key dedup with pluggable IdempotencyStore (default InMemoryIdempotencyStore via DashMap, 24h TTL). Mismatched body for same key returns LiterLlmError::IdempotencyConflict; in-flight key returns LiterLlmError::IdempotencyInFlight (error-out, not sleep-poll).
LlmRequest::idempotency_key: Option<String> field and LlmRequest::with_idempotency_key builder for opt-in idempotency.
LiterLlmError::IdempotencyConflict { key: String } and LiterLlmError::IdempotencyInFlight { key: String } error variants (HTTP 409 equivalent).
liter_llm::tower::FallbackChainLayer — walk an ordered Vec<S> of services, advancing on transient errors via pluggable RetryPolicy. DefaultRetryPolicy treats 5xx/timeouts/429 as transient; auth and validation errors as terminal. Exports RetryClass, RetryPolicy, DefaultRetryPolicy, FallbackChainLayer, and FallbackChainService.
liter_llm::tenant::{TenantId, TenantContext, KeyResolver, ResolvedKey, KeyResolverError, InMemoryKeyResolver} — generic multi-tenant primitives.
liter_llm::tenant::EtcdKeyResolver — distributed KeyResolver backed by an etcd cluster. Behind feature etcd-key-resolver. Reads JSON-serialised ResolvedKey at {prefix}/{sha256(api_key)}. Configure via EtcdKeyResolverConfig (endpoints, prefix, timeouts, optional auth).
LlmRequest::with_tenant_id / LlmRequest::tenant_id for tenant propagation through the Tower stack.
LlmRequestKind — the discriminant enum extracted from LlmRequest to carry the variant payload; re-exported from liter_llm::tower.

Changed

Migration — RealtimeEvent::RateLimitsUpdated: field reset_at: SystemTime renamed to reset_at_unix_ms: i64 (Unix milliseconds). Pattern-match sites must update the field name; the value can be reconstructed with SystemTime::UNIX_EPOCH + Duration::from_millis(reset_at_unix_ms as u64) if needed.
Migration — WaitForBatchConfig: fields renamed for FFI bridgeability — initial_interval: Duration → initial_interval_secs: f64, max_interval: Duration → max_interval_secs: f64, timeout: Option<Duration> → timeout_secs: Option<f64>. Update all construction sites to use f64 seconds.
Migration — BatchWaitError: variants changed to struct-style for FFI bridgeability — Failed(BatchStatus) → Failed { status: BatchStatus }, Timeout(Duration) → Timeout { timeout_secs: f64 }, Client(LiterLlmError) → Client { message: String, code: u32 }. Update all pattern-match and construction sites.
LlmRequest is now a struct (kind: LlmRequestKind, tenant_id: Option<TenantId>) rather than a plain enum. All existing constructor call sites (LlmRequest::Chat(r), LlmRequest::Embed(r), etc.) continue to compile unchanged via #[allow(non_snake_case)] associated functions. Pattern-match sites that directly match on LlmRequest::Variant must be updated to match on req.kind.
liter-llm-proxy AppState gains key_resolver: Arc<dyn KeyResolver> alongside the existing key_store: Arc<KeyStore>. KeyStore implements KeyResolver; behaviour is unchanged.
Cache (CacheService, SingleflightService) reads tenant_id from the request via LlmRequest::tenant_id() so cached responses are scoped to the correct tenant automatically.
KeyResolver::resolve now takes an owned String and returns a 'static future so the future can be spawned across tower::Service::call boundaries.
ProviderCredential.api_key is now secrecy::SecretString (was String); zeroed on drop, Debug impl redacts to [REDACTED].
liter-llm-proxy activates liter_llm::provider::set_outbound_policy at startup based on SecurityConfig.outbound_policy. Defaults to DenyPrivate (blocks RFC1918, loopback, link-local, multicast, unspecified); Allowlist parse errors are logged and skipped with a startup summary line.
liter-llm-proxy virtual-key lookup uses subtle::ConstantTimeEq over a full iteration of the key store, preventing timing-side-channel inference of key existence.
tower::HooksLayer::with_usage_sink now dispatches sink emit via tokio::spawn; slow sinks no longer add latency to caller-observed responses.

Fixed

tower::circuit — maybe_half_open is wired into the request path; circuits no longer get permanently stuck Open after tripping. probe_in_flight is now a ProbeGuard<P> RAII enum so the probe slot is reclaimed even if the future panics or is cancelled. should_allow gates HalfOpen to exactly one probe via atomic CAS.
tower::hedge — HedgeService::call uses mem::replace correctly so the polled-ready inner becomes the primary attempt and hedge attempts clone from a fresh standby; eliminates Tower-readiness contract violation and ConcurrencyLimit permit double-consumption.
tower::cache::CacheService::call — applies the mem::replace(&mut self.inner, self.inner.clone()) Tower swap so permit-bearing inner services (e.g. ConcurrencyLimit, Buffer) are not double-consumed.
tower::cache_singleflight — tx.send(result) now precedes map.remove(&key) so late-arriving followers don’t become duplicate leaders. Added LeaderDropGuard to clean up the in-flight map when the leader is cancelled; leader errors propagate to followers with variant preserved (no longer mapped to InternalError).
tower::cache_key::ExactHashStrategy — uses ahash::RandomState::generate_with(fixed_seeds) via OnceLock so cache keys are deterministic across process restarts and distributed nodes (was std::hash::DefaultHasher, which is randomized per process).
tower::idempotency::compute_body_hash — same deterministic-ahash fix as the cache key strategy; required for distributed IdempotencyStore backends to function.
tower::idempotency::IdempotencyService::call — store key is now format!("{tenant_id}:{idempotency_key}") so two tenants with the same idempotency key value do not cross-pollinate responses.
tower::fallback_chain::FallbackChainService::call — drives svc.ready().await? before each svc.call(...), restoring the Tower readiness contract for chain elements (previously silently bypassed ConcurrencyLimit permits). FallbackChainLayer implements Layer<()> with a prepend(head) helper for ServiceBuilder composition.
tower::budget — window rollover is atomic via CAS; concurrent writers across the rollover boundary cannot torn-read a $0 window mid-zero-then-add.
liter-llm-proxy::routes::realtime — the WebSocket upgrade now resolves the upstream credential via the per-model ProviderCredential registry instead of general.master_key; restricted virtual keys can no longer trigger Realtime sessions that bill the master key. The handler also calls key_ctx.can_access_model(&model) before ws.on_upgrade, so the model allowlist is enforced before any upstream connection is opened.
liter-llm-proxy::secrets (aws.rs, vault.rs) — cache values are stored as secrecy::SecretString so secrets are zeroed on TTL eviction; previously plain String left key material on the heap.
liter-llm-proxy::secrets::vault — HashiCorpVaultProviderBuilder::build validates the address against the outbound policy (validate_outbound_url_sync) before constructing the VaultClient; misconfigured addresses pointing at internal endpoints (link-local, loopback, RFC1918) are rejected with SecretError::Forbidden.
guardrail::cel — CEL evaluation errors are no longer returned to the caller verbatim (could leak expression internals); callers see a fixed "policy evaluation error" reason while the full error is logged server-side via tracing::error!.
http::transport::TransportConfig — re-exported at the crate root so the rustdoc example compiles; previously the type was reachable only via the pub(crate) http module.
Trait re-exports — CacheKeyStrategy, Guardrail, GuardrailContext, GuardrailDecision, GuardrailStage, VectorStore, VectorMatch, EmbeddingProvider, NoOpEmbeddingProvider, TenantId, TenantContext, KeyResolver, ResolvedKey, KeyResolverError, InMemoryKeyResolver, IdempotencyStoreError are now reachable from liter_llm::* or liter_llm::tower::* without spelling out the full module path.

Security

All findings from three rounds of critical security audit are resolved: per-model credential isolation on the Realtime route, model-allowlist enforcement before WebSocket upgrade, constant-time virtual-key lookup, SecretString for cached secrets and provider credentials (zeroed on drop), SSRF outbound-policy guard activated at proxy startup, Vault address validation, CEL error redaction, fail-closed CEL guardrails, and JSON-tree redaction for regex guardrails.

[1.6.0-rc.0] - 2026-06-15

Added

tower::circuit module — CircuitPolicy trait with ExponentialBackoffCircuit default impl, CircuitState enum (Closed→Open→HalfOpen), CircuitLayer and CircuitService for fault isolation. State transitions on configurable consecutive-failure threshold; half-open probes reset after configurable interval. (crates/liter-llm/src/tower/circuit.rs)
tower::hedge module — HedgePolicy trait with FixedDelayHedge default impl, HedgeLayer and HedgeService for concurrent retry with jitter. Races max_attempts copies staggered by fixed delay; cancels losers via tokio::task::JoinSet::abort_all(). Fast path when max_attempts == 1 skips JoinSet entirely. (crates/liter-llm/src/tower/hedge.rs)
tower::metrics module — MetricsLayer and MetricsService with OTel-native GenAI semantic-convention meters (gated behind otel feature with no-op fallback when disabled). Emits: gen_ai.client.operation.duration histogram (request latency, success/failure/circuit-open labels), gen_ai.cache.{hit,miss,stale} counters, gen_ai.circuit.trip counter, gen_ai.retry.attempt counter, plus gen_ai.client.token.usage histogram and gen_ai.request.cost_usd histogram. Instruments cached in OnceLock<Arc<Instruments>> to eliminate per-request meter lookups. (crates/liter-llm/src/tower/metrics.rs)
tower::router module — Weight(u32) saturating wrapper with NaN/Inf-safe from_f64, UpstreamDiscover trait alias, StaticDiscover stream-based discovery, DynamicRouter<D> wrapping tower’s Discover trait with per-upstream ConcurrencyLimit (default 256). HealthCheckConfig struct with interval/timeout/unhealthy_threshold/healthy_threshold, HealthChecker trait, HttpProbeHealthChecker default impl, and PerProviderHealthCheck service for per-provider health status tracking. (crates/liter-llm/src/tower/health.rs, crates/liter-llm/src/tower/router.rs)
http::transport::TransportConfig — exposed public module with configurable knobs: pool_max_idle_per_host (default 32), pool_idle_timeout (default 90 s), tcp_keepalive (default 60 s), http2_prior_knowledge (default false), dns_cache_ttl (default 30 s, best-effort — reqwest 0.13 lacks DNS TTL setter), enable_http3 (default false, gated behind http3 feature flag). Builder pattern with sensible defaults. Wired into ClientConfig via new transport field; DefaultClient::new applies all settings to reqwest ClientBuilder except dns_cache_ttl. (crates/liter-llm/src/http/transport.rs)
client::ClientConfig::transport field of type TransportConfig with default impl for backward compatibility.
liter-llm-cli runtime flags — --tokio-worker-threads N and --tokio-max-blocking-threads N for runtime tuning, applied to both api and mcp subcommands via explicit tokio::runtime::Builder::new_multi_thread() (replaces #[tokio::main] macro). Defaults: physical CPU count (workers), 512 (blocking threads).
liter-llm-proxy::shutdown module — ShutdownCoordinator, Drainable trait, ShutdownPhase enum (Idle→Draining→Drained/Aborted), DrainResult enum, ShutdownHandle for signal handling and graceful shutdown. Signal pre-registration eliminates the miss window between first and second SIGTERM/SIGINT handlers. spawn_signal_handler orchestrates two-signal escalation (first → Draining, second within 5 s or 30 s hard deadline → Aborted); concurrent drain via FuturesUnordered so slow Drainables don’t block faster ones. (crates/liter-llm-proxy/src/shutdown.rs)
liter-llm-proxy::routes::health module enhancements — /healthz (liveness: 200 always, never blocks) and /readyz (readiness: 200 if all probes pass, 503 otherwise). ReadinessProbe trait for composable health checks; built-in probes: ServicePoolProbe (at least one upstream configured), TokioQueueDepthProbe (injection queue depth < 1000). Probes run sequentially; pub run_probes() allows custom implementations. (crates/liter-llm-proxy/src/routes/health.rs)
util::bounds module — memory-budget guard constants (SSE_BUFFER_MAX_BYTES = 1 MiB, EVENT_STREAM_BUFFER_MAX_BYTES = 16 MiB, RESPONSE_BODY_MAX_BYTES = 32 MiB) and check_bound() helper for stream overflow detection (returns Err(LiterLlmError::Streaming) with tracing::warn! if exceeded). (crates/liter-llm/src/util/bounds.rs)
Workspace [lints.clippy] policy — deny correctness/suspicious/perf; warn style. Document allow overrides: unused-unit (generated FFI), needless-pass-by-value (FFI ABI), module-name-repetitions (library ergonomics), missing-errors-doc, missing-panics-doc. (Cargo.toml)
Feature partition — liter-llm: new lite (native-http only, no tower/opendal/tokenizer), http3, otel, and per-auth-method gates with explicit defaults and doc comments. liter-llm-proxy: otel, opendal-cache, proxy (named surface); default = proxy. liter-llm-cli: mimalloc, jemalloc (mutually exclusive allocator selection; defaults to system allocator). Workspace dependency pinning added for allocators. (Cargo.toml, crates/*/Cargo.toml)
Global allocator selection — crates/liter-llm-cli/src/allocator.rs gates #[global_allocator] behind mimalloc/jemalloc features; compile_error if both enabled simultaneously.
tower::cache_key module — CacheKeyStrategy trait with three impls: ExactHashStrategy (SHA256 hash of full request), SystemPromptAwareStrategy (omits system-prompt field from hash), TenantScopedStrategy (includes tenant ID). All three hash deterministically via serde_json to stable JSON. (crates/liter-llm/src/tower/cache_key.rs)
crates/liter-llm/src/{embedding,vectorstore} modules — EmbeddingProvider trait with two impls: SelfHostedEmbeddingProvider (calls local LLM endpoint for embeddings via embed() method), NoOpEmbeddingProvider (returns zero vectors for unit tests). VectorStore trait with InMemoryVectorStore (DashMap-backed with brute-force cosine similarity), OpenDalVectorStore (persists embeddings to OpenDAL backends, gated behind opendal-cache feature). Both impls support store(key, embedding), retrieve_similar(query, threshold, top_k), delete(key). (crates/liter-llm/src/tower/vectorstore/{mod,memory,opendal}.rs, crates/liter-llm/src/tower/embedding.rs)
tower::cache.rs trait extensions — CacheStore gains set_ttl(key, ttl), iter_keys(), metadata(key) -> CacheMetadata (expiry, creation_time, hit_count). Default no-op bodies preserve backward compat for existing impls. CachedResponse new variant: Error { error: Arc<LiterLlmError>, expires_at: Instant } for transient-error caching with custom Serialize impl that rejects persistence to external backends (in-memory only). (crates/liter-llm/src/tower/cache.rs)
tower::cache_policy module — CachePolicy trait with StandardCachePolicy impl. Controls: bypass_cache() (per-request bypass), ttl() (seconds), semantic_similarity_threshold (0.85), stale_while_revalidate (5 minutes). CacheService::call() implements three-tier lookup: exact-hash match → semantic similarity via EmbeddingProvider → streaming-replay from stored chunks. warm(requests) async warming hook for batch pre-population. (crates/liter-llm/src/tower/cache_policy.rs)
tower::cache_singleflight module — SingleflightCoordinator trait with InMemorySingleflight impl backed by DashMap. Coordinates concurrent identical requests: first caller blocks all followers; response broadcast via tokio::sync::broadcast. Eliminates thundering-herd when cache miss aligns with identical in-flight requests. (crates/liter-llm/src/tower/cache_singleflight.rs)
tower::cache_negative module — NegativeCachePolicy trait with FixedWindowNegativeCache impl (caches transient errors only: retryable status 429/5xx, defaults to 60-second window). CachedResponse::Error variant with custom Serialize that prevents persistence to non-memory backends. (crates/liter-llm/src/tower/cache_negative.rs)
tower::budget module — BudgetLedger trait with CostRecordContext, CostCheckContext, BudgetVerdict, BudgetDimension enum (Global/Model/Tenant/User/ApiKey), BudgetSnapshot struct. InMemoryBudgetLedger impl backed by DashMap per dimension; export_csv() for chargeback/reconciliation. Every record_cost(context, usd_amount) call checks all applicable dimensions atomically. (crates/liter-llm/src/tower/budget.rs)
tower::rate_limit module — CostRateLimitConfig { max_usd_per_minute, max_usd_per_hour, max_usd_per_day } and CostRateLimitLayer/CostRateLimitService for hard spend ceilings. Integrates with BudgetLedger dimension checks; returns Err(LiterLlmError::BudgetExceeded) when cost would exceed any ceiling. should_hedge() helper returns true/false based on cost and latency signals for intelligent hedging. (crates/liter-llm/src/tower/rate_limit.rs)
tower::metrics OTel additions — new meters: gen_ai.budget.spend_usd (histogram, labeled by dimension), gen_ai.budget.rejection (counter, labeled by dimension + reason). Emitted by BudgetLedger impls and CostRateLimitService. (crates/liter-llm/src/tower/metrics.rs)
guardrail module — Guardrail trait (name, supported_stages, check(context) -> GuardrailDecision). GuardrailStage enum (Input/Output/OutputChunk). GuardrailDecision enum (Allow/Block/Mutate). GuardrailContext struct (request, response, reason). Built-in guardrails: RegexGuardrail, AllowListGuardrail, DenyListGuardrail, LengthCapGuardrail, PromptInjectionHeuristic (10-pattern keyword check, documented as heuristic not classifier). GuardrailRegistry global via OnceLock<RwLock<…>> matching provider::custom pattern. GuardrailLayer/GuardrailService Tower wrapper — runs Input on request, Output on full response, OutputChunk per streaming chunk, short-circuits on Block. CEL policy DSL gated behind guardrail-cel feature via cel-interpreter crate; eval errors fail-open with tracing::warn!. (crates/liter-llm/src/guardrail/{mod,builtin,registry,cel,tests}.rs, crates/liter-llm/src/tower/guardrail.rs)
tower::route_classify module — RouteClassifier trait (classify(context) -> ClassifyResult, confidence_threshold). Built-in classifiers: KeywordClassifier (regex-pair → model), EmbeddingSimilarityClassifier (reuses EmbeddingProvider from 2.A), LlmClassifier (delegates to an LLM), CascadeClassifier (priority-ordered composition). ClassifierVerdictCache caches verdicts via CacheStore. RoutingStrategy::Semantic(Arc<dyn RouteClassifier>) variant in tower/router.rs; falls back to round-robin when classifier defers. OTel meters: gen_ai.route.classify.duration histogram + gen_ai.route.classify.tier{keyword,embedding,llm}.hit counters. (crates/liter-llm/src/tower/route_classify.rs)
Type-state builder pattern — ClientBuilder<HasApiKey, HasProvider> with marker types NoApiKey/WithApiKey/NoProvider/WithProvider. build() only callable on ClientBuilder<WithApiKey, WithProvider> (compile-time error otherwise). Enforces API key and provider selection before use. (crates/liter-llm/src/client/builder.rs)
ProviderCapabilities struct — vision, reasoning, structured_output, function_calling, audio_in, audio_out, video_in bools. Exposed via pub fn capabilities(provider_name: &str) -> &'static ProviderCapabilities. (crates/liter-llm/src/provider/mod.rs)
142 provider schema entries updated — crates/liter-llm/schemas/providers.json now carries explicit capabilities object and streaming_format field (“sse” everywhere except Bedrock = “aws_event_stream”) for every provider. Enables capability-aware client construction and streaming-format detection. (crates/liter-llm/schemas/providers.json)

Phase 3 — Realtime streaming, secret backends, credential rotation, and config hot-reload

streaming module — unified ingress/egress streaming with three composable layers: IngressStream<S, P> (typed SSE decoder), StreamPipeline<S> (ordered per-chunk middleware via ChunkMiddleware trait), EgressStream<S> (typed OpenAI SSE encoder). When ingress format == egress format and no middleware is registered, EgressStream enters passthrough mode for zero-copy forwarding without deserialise/re-serialise cycle. StreamFormat (SSE vs. AWS EventStream) promoted to pub for explicit wire-format selection. Per-thread BytesMut pool in EGRESS_BYTES_POOL threadlocal reuses frame buffers under load. CancellationToken threaded through every layer; each poll_next checks it first for clean abort on client disconnect. (crates/liter-llm/src/streaming.rs)
liter-llm-proxy::secrets module — SecretManager trait (object-safe via Pin<Box<dyn Future>>) with get(name) -> SecretValue (field: zeroed SecretString + SecretMetadata), set(name, value, tags), delete(name). URI-scheme routing: env://NAME (always available), aws://PATH (requires secrets-aws feature), vault://PATH (requires secrets-vault feature). Built-in impls: EnvVarSecretManager (environment variables), AwsSecretsManagerProvider (AWS Secrets Manager with key rotation warnings), HashiCorpVaultProvider (Vault KV-v2 with expiry tracking). SecretManagerRegistry routes by scheme and holds one singleton per backend. OTel gauge gen_ai.secret.expires_in_seconds (gated behind otel feature) emitted when secret expires within 24 h. (crates/liter-llm-proxy/src/secrets/{mod,env,aws,vault}.rs, crates/liter-llm-proxy/src/secrets/registry.rs)
liter-llm-proxy::config::ConfigProvider trait — load() -> ProxyConfig (single snapshot) and watch() -> mpsc::Receiver<ConfigEvent> (live updates). Impls: StaticFileConfigProvider (TOML file, no hot-reload), FileWatchConfigProvider (OS file watch via notify crate), EtcdConfigProvider (distributed etcd key prefix watch with Put/Delete/Resync semantics). ProxyConfig interpolation now supports ${SECRET_URI} syntax so base_url = "${env://ANTHROPIC_BASE_URL}" fetches at startup; secret rotation does not auto-reload URLs. (crates/liter-llm-proxy/src/config/{provider,watcher}.rs)
liter-llm-proxy::provider::CredentialPool trait — rotates per-provider API keys on 429/5xx rate-limit signals. Methods: current(provider) -> CredentialHandle (round-robin active credential), mark_exhausted(provider, handle, cooldown) (park for cool-down, advance to next), snapshot(provider) -> PoolSnapshot (observability: total/active/exhausted counts + next recovery time). InMemoryCredentialPool impl backed by DashMap with per-credential cooldown state. ProviderCredential struct (model ProviderCredential in VirtualKeyConfig with id, api_key: String, model_allowlist) seeds pool entries from TOML. Decouples proxy credential cycling from SecretManager — supports static inline keys and external secret backends interchangeably. (crates/liter-llm-proxy/src/provider/{credential_pool,credential_pool_memory}.rs)
liter-llm::realtime module — unified envelope + event types for vendor-neutral realtime streaming. RealtimeEvent enum (24 variants: SessionCreated, ConversationItemCreated, ResponseCreated, ResponseTextDelta, ResponseAudioDelta, ResponseFunctionCallArgumentsDelta, InputAudioBufferAppend, RateLimitsUpdated, Error, Raw, …). ContentPart enum (Text, Audio, ImageRef) used in conversation items. ResponseStatus enum (Completed, Cancelled, Failed, Incomplete). RealtimeEnvelope wraps event + optional event_id. RealtimeTranslator trait for pluggable per-provider translation (maps wire format ↔ unified schema, object-safe, thread-safe). Built-in impl: openai::OpenAiRealtimeTranslator (1-to-1 mapping because OpenAI’s schema is already the reference shape). crates/liter-llm/src/realtime/{mod,openai}.rs)
AppState refactor — config field changed from Arc<ProxyConfig> to Arc<ArcSwap<ProxyConfig>> for atomic hot-reload without blocking in-flight requests. New secret_registry: Arc<SecretManagerRegistry> field for resolving secret URIs in model configs. Callers must call state.config.load() to obtain a consistent snapshot per request. (crates/liter-llm-proxy/src/state.rs)

Migration notes

AppState now requires secret_registry: Arc<SecretManagerRegistry> and config: Arc<ArcSwap<ProxyConfig>> fields. Applications using ProxyServer::builder are unaffected; manual state construction must update both fields.
New optional feature flags: secrets-aws, secrets-vault, secrets-env (env backend always enabled, others optional). mimalloc, jemalloc for allocator selection. http3 for HTTP/3 support. tokenizer for count_tokens availability.
VirtualKeyConfig gains new provider_credentials: Vec<ProviderCredential> field (defaults to empty). Inline credentials in TOML via repeated [[keys.provider_credentials]] blocks; proxy auto-rotates among them on 429/5xx.
Workspace clippy is now -D warnings; downstream consumers compiling with strict lints should review suppressions — the main crate is now warnings-clean.
LlmRequest pattern-matching change — LlmRequest was previously an enum; it is now a struct with a kind: LlmRequestKind field. match req { LlmRequest::Chat(r) => ... } no longer compiles. Migrate to match req.kind() { LlmRequestKind::Chat(r) => ... } (preferred, using the new kind() accessor) or match req.kind { LlmRequestKind::Chat(r) => ... } (direct field access). The PascalCase constructor aliases (LlmRequest::Chat(r) etc.) remain callable for constructing requests and continue to compile unchanged; they are marked #[doc(hidden)] and will be removed in a future minor release.
KeyResolver::resolve signature change — the method now takes api_key: String (owned) instead of api_key: &str, and returns Pin<Box<dyn Future<Output = ...> + Send + 'static>> instead of ... + 'a. Custom KeyResolver implementations must update their signature. Call sites that previously passed a &str literal must add .to_owned(): resolver.resolve("sk-...".to_owned()).

Changed

Bindings regenerated against alef v0.25.9; refreshes all 16 language surfaces, e2e suites, and README templates. New [crates.e2e.fields_c_types] entry chat_completion_response.usage = "Usage" and per-call C# e2e override class = "LiterLlmConverter" to satisfy alef v0.25.9’s stricter intermediate-accessor checks.
tower/router.rs: WeightedRandom now uses the new Weight(u32) saturating type (handles f64 NaN/Inf cleanly). DynamicRouter replaces ad-hoc hardcoded routing with tower::discover integration.
tower/health.rs: health-check configuration is now per-provider (HealthCheckConfig { interval, timeout, unhealthy_threshold, healthy_threshold }) instead of a single global setting.
http/streaming.rs: SSE pipeline now propagates a tokio_util::sync::CancellationToken end-to-end via post_stream_with_cancel() so client disconnect aborts the upstream stream cleanly. Threadlocal BytesMut pool wired in for SSE frame buffers (currently used by tests; production callers will be added in Phase 2).
cli/main.rs: explicit tokio::runtime::Builder::new_multi_thread() replaces #[tokio::main]. Worker/blocking-thread counts now configurable.
Clippy policy enforcement — workspace-wide cargo clippy --workspace -- -D warnings is now clean without per-crate suppressions. Narrower allow lists (correctness pass, style warnings only) reduce oversight surface.
tower/cache.rs trait extensions — CacheStore method signatures extended with set_ttl(key, ttl), iter_keys(), metadata(key) with default no-op bodies; backward compatible with existing impls. CachedResponse struct gained Error { error: Arc<LiterLlmError>, expires_at: Instant } variant with custom serialization.
tower/router.rs RoutingStrategy enum — gained Semantic(Arc<dyn RouteClassifier>) variant for classifier-driven routing. Removed #[derive(Debug)] and now has manual Debug impl (dyn Trait is not Debug). Round-robin fallback when classifier defers.

Fixed

Test diagnostic clarity — all 417 test .unwrap() calls replaced with .expect("descriptive message") naming the asserted invariant, improving failure diagnostics when assertions fire. Production code paths remain unwrap-clean. (crates/liter-llm-*/src/**)
tower/circuit.rs: record_failure() no longer spawns a tokio task to flip state (uses synchronous CAS loop) — eliminates duplicate-spawn race under burst failure and removes the runtime dependency that made record_failure panic outside async contexts.
tower/hedge.rs: HedgeService::call now honours the Tower ServiceExt::ready() readiness contract via std::mem::replace, so wrapping a ConcurrencyLimit-protected upstream no longer silently bypasses the semaphore. Hedge fast-path (max_attempts == 1) skips JoinSet entirely.
tower/metrics.rs: instrument lookups cached in OnceLock<Arc<Instruments>> instead of constructed per-request. Removes ~8k redundant meter lookups/sec at 1k req/s production load.
http/streaming.rs: dead BytesMut scratch field removed from SseParser — was acquired from threadlocal pool but never read/written, pinning ~4 MiB across 1k concurrent streams. Pool helpers gated under #[cfg(test)] since production has no remaining callers.
liter-llm-proxy/shutdown.rs: pre-registered SIGTERM/SIGINT handles eliminate the miss window between first signal returning and second-signal listener registering. Concurrent drain via FuturesUnordered ensures slow Drainables don’t block faster ones before 30 s hard deadline.
liter-llm-proxy/routes/health.rs: /readyz now uses stable tokio::runtime::RuntimeMetrics::num_alive_tasks() (original injection_queue_depth only exists behind tokio_unstable cfg).
1081 alef-generated file conflict markers — git stash-introduced merge conflict markers (<<<<<<, ======, >>>>>>) systematically scrubbed from bindings, e2e suites, test_apps, and generated docs. The C# LiterLlmConverter.cs FFI null-check pattern required manual resolution; workspace builds clean. (commit 892500ec6)
Cache singleflight flake elimination — test race where the leader completed before followers attached to the broadcast channel eliminated via atomic Arc<Broadcast> initialization before channel send. Fast mock services under parallel load now stable. (commit 4e3a3e51e)

Tooling

Workspace clippy lint policy enforcement via [workspace.lints.clippy] blocks; per-crate suppressions consolidated at source.
Feature flag audit — split composite features (e.g., native-http still depends on http2, now gated on both); avoid silent breakage from feature interaction.
Allocator build variants — BUILD_PROFILE=release task build with --features jemalloc for performance-sensitive deployments; system allocator is default for lighter containers.
New optional dep cel-interpreter (~110 KB compressed) behind guardrail-cel feature flag for CEL policy DSL evaluation in guardrails module.
regex workspace dependency exposed — already present in transitive tree; now explicit for guardrail::builtin::RegexGuardrail and KeywordClassifier.

[1.5.1] - 2026-06-13

Changed

publish workflow: migrate every push, release-asset upload, and homebrew-tap commit to the kreuzberg-dev-publisher[bot] GitHub App via actions/create-github-app-token@v2, replacing secrets.GITHUB_TOKEN and secrets.HOMEBREW_TOKEN with scoped app installation tokens.
Bindings regenerated against the latest alef, refreshing all 16 language surfaces and e2e suites.

Fixed

Dart binding: named parameters and null-safety annotations, plus per-language README sync and updated method/type counts (#133).
PyO3 0.29 method rename: pyo3::Bound::downcast_into callsites in crates/liter-llm-py/src/lib.rs migrated to the new cast_into name so the Python binding builds against pyo3 0.29.
PMD ruleset: exclude UnnecessaryWarningSuppression from category/java/bestpractices.xml. Alef emits a blanket @SuppressWarnings("PMD") on every generated DTO record; PMD flags some as unnecessary depending on which rules fire on the surrounding record, breaking the Java hook on every regeneration.

[1.5.0] - 2026-06-07

Security

External security audit identified six exploitable gaps in the v1.4.1 codebase. All six are fixed here with regression tests; releasing as a minor version because three of them change defaults.

(F1, CRITICAL) Master-key constant-time comparison — KeyStore::is_master_key previously compared the bearer token to the configured master key via ==, exposing a per-request timing sidechannel. Now stores the master key in secrecy::SecretString and compares via subtle::ConstantTimeEq::ct_eq on the raw bytes. (crates/liter-llm-proxy/src/auth/key_store.rs, new subtle = "2.6" dep in crates/liter-llm-proxy/Cargo.toml.)
(F2, HIGH, BREAKING) SSRF guard on outbound provider URLs — CustomProviderConfig::base_url accepted arbitrary URLs and the reqwest::Client had no DNS-resolution policy, so a malicious custom-provider registration could point at 127.0.0.1 / 169.254.169.254 / RFC1918 networks. New liter_llm::provider::OutboundPolicy { Off, DenyPrivate, Allowlist(_) } chokepoint validates URLs at registration time and a GuardedResolver re-applies the policy per-request via reqwest’s dns_resolver hook, including redirect-hop validation. Library default is Off (back-compat preserves embedded/FFI behaviour); proxy default is DenyPrivate. New LiterLlmError::OutboundForbidden variant maps to HTTP 502. New TOML key [security] outbound_policy = "deny_private" | "off" | { allowlist = ["…"] }. (crates/liter-llm/src/provider/outbound_policy.rs, crates/liter-llm/src/provider/custom.rs, crates/liter-llm/src/client/mod.rs, crates/liter-llm-proxy/src/config/server.rs, crates/liter-llm-cli/src/commands/serve.rs.)
(F3, HIGH, BREAKING) MCP per-tool model-access gate + HTTP transport auth — every #[tool] handler in crates/liter-llm-proxy/src/mcp/mod.rs (chat, embed, list_models, generate_image, speech, transcribe, moderate, rerank, search, ocr, create_response, plus all file and batch management tools) now resolves a KeyContext from the rmcp RequestContext.extensions and pre-flight-checks can_access_model(&params.model) or is_master before routing through ServicePool / FileStore. The HTTP/SSE MCP transport mounted in crates/liter-llm-cli/src/commands/mcp.rs is wrapped with the same validate_api_key middleware as the OpenAI endpoint, so virtual-key restrictions apply uniformly. Stdio transport requires an explicit mcp.stdio_key_id / mcp.stdio_trust_local = true opt-in or refuses to start.
(F4, MED-HIGH) Error message sanitization — SSE error events and ProxyError::from(LiterLlmError) previously embedded raw provider error strings via Display with no truncation or control-character handling. New crates/liter-llm-proxy/src/error.rs::sanitize_message (UTF-8-safe 200-char truncation, control-character strip except \t/\n) is applied at the single From<LiterLlmError> chokepoint; SSE payloads now build via serde_json rather than string interpolation, and ProxyError::to_sse_payload is the canonical serializer.
(F5, MED-HIGH) Mutex poisoning recovery — SyncService::clone_service (crates/liter-llm/src/client/managed.rs) previously panicked when the inner std::sync::Mutex was poisoned. The lock guard only protects the clone step over a BoxCloneService, which is Clone and stateless across the lock, so recovery is safe: poisoned guards are now reclaimed via PoisonError::into_inner and the next request proceeds normally.
(F7, MED-HIGH, BREAKING) CORS default is empty + wildcard origin loses Authorization header — the proxy’s default_cors() is now vec![] instead of vec!["*"]; with no cors_origins, the router skips the CorsLayer entirely. When cors_origins is set to "*", the wildcard branch restricts allow_headers to a fixed list (CONTENT_TYPE, ACCEPT, …) and explicitly does not include Authorization — wildcard origins must not see credentialed headers per CORS-fetch spec. liter-llm-cli serve also logs a tracing::warn! when cors_origins.contains("*") && host == "0.0.0.0".

Changed

Bindings regenerated against alef v0.23.28 (was v0.23.16). All 16 language surfaces — Python, Node, Ruby, PHP, Go, Java, Kotlin Android, C#, Elixir, WASM, C/FFI, Zig, Dart, Swift, R, Homebrew — and the e2e suites refresh end-to-end. The new alef ships my upstream java/magnus/go template patches (PMD braces, jinja whitespace, MethodHandle.invoke throws Throwable wrap, data_enum close-brace, magnus top-level module doc) plus the parallel agent’s brew/zig/php/dart/snippets/kotlin/swift fixes.
Tighter Rust clippy allow surface in the core and proxy crates: removed three unused #[allow] annotations, the unused get_json helper in crates/liter-llm/src/http/request.rs, and a now-dead serde::de::DeserializeOwned import. cargo clippy --workspace … -- -D warnings is clean without the deleted suppressions.

Tooling

xberg-io/pre-commit-hooks bumped to v2.1.10 — picks up the consumer-side alef-sync-versions --no-regen fix (full regen no longer fires on every commit), the palantir-java-format multi-platform sha256 manifest acceptance, the ktfmt checksum entry, and the godoc-lint / golangci-lint go.work-aware module discovery (no longer scans stale test_apps/swift_e2e/.build/checkouts/.../e2e/go/).
Project-local PMD ruleset at packages/java/pmd-ruleset.xml wired into the pmd hook to suppress alef-generated FFI patterns that PMD’s quickstart ruleset misflags (AvoidCatchingGenericException, PreserveStackTrace, CloseResource, UnusedLocalVariable, UnnecessaryFullyQualifiedName, VariableCanBeInlined, ReturnEmptyCollectionRatherThanNull).
deny.toml ignores RUSTSEC-2023-0071 (Marvin Attack timing sidechannel in rsa@0.9.x, transitive via opendal -> reqsign-core). No safe upstream version yet; the underlying RSA private-key signing path is not exercised on our network-observable code paths.
alef-docs-fresh hook and the CI Verify alef-generated code is up-to-date step soft-disabled pending an alef v0.23.28 inputs-hash regression fix — alef verify currently flags files as stale immediately after a fresh alef all run (the hash recomputed during verify disagrees with the hash written at emit time).
markdownlint-rumdl-strict exclude expanded to cover the root README.md (alef-generated badge row uses inline HTML), CONTRIBUTING.md, templates/readme/, and .github/PULL_REQUEST_TEMPLATE.md.

Migration notes

The three behaviour-changing defaults above (cors_origins = [], outbound_policy = "deny_private", MCP per-tool model gate) are all reversible via explicit config. Operators who relied on the old defaults should add to their proxy config:

cors_origins = ["*"]                # opt back into the v1.4.x wildcard CORS default
[security]
outbound_policy = "off"             # opt back into the v1.4.x unguarded outbound HTTP

Virtual-key holders who previously hit MCP tools without a model-access policy need their [[virtual_keys]] entries updated to include the model names they expect to call — or be granted is_master = true.

[1.4.1] - 2026-06-05

Fixed

Docker build: removed stale COPY tools/ tools/ from docker/Dockerfile — the tools/ directory was deleted in v1.3.0 and the unfixed copy was failing every Docker image build since.
publish-crates job timeout: bumped .github/workflows/publish.yaml publish-crates timeout-minutes from 30 to 60. The 30-minute ceiling was cancelling mid-publish on busy crates.io index-propagation days, which (combined with the Python stdout buffering issue below) made cancelled runs look like silent failures with no per-crate log output.
Upstream xberg-io/actions to v1.8.29: publish-crates/scripts/publish.py now line-buffers stdout/stderr (sys.stdout.reconfigure(line_buffering=True)), so per-crate “Publishing X (n/total)…” progress survives job cancellation. Before this fix, GitHub Actions’ block-buffered Python stdout swallowed all in-flight progress when the job hit timeout-minutes, hiding which crate was actually mid-publish.

Notes

v1.4.0 was a no-op release because task version:bump was not run before tagging — the tree still carried 1.4.0-rc.61 in Cargo.toml, so every publish job either re-shipped rc.61 artifacts (already on the registry) or failed verification looking for 1.4.0. v1.4.1 is the first real 1.4.x release.
alef pin advanced to 0.23.16 (was 0.23.12) — no functional codegen changes vs. 0.23.12; bump tracks the latest released 0.23.x.

1.4.0 - 2026-06-05

Added

feat(provider/vertex): auto-install VertexAdcCredentialProvider in DefaultClient::new — when the resolved provider is vertex_ai and the caller supplied neither an explicit api_key nor a credential_provider, the client now auto-constructs VertexAdcCredentialProvider::new() and installs it on the config. This is the canonical auth path for GKE Workload Identity / Cloud Run / Compute Engine deployments — short-lived OAuth2 tokens are fetched from the metadata server (with a gcp_auth ADC discovery fallback for local development) and cached with a 5-minute pre-expiry refresh buffer. Pre-obtained tokens supplied via api_key and explicit credential_providers continue to take precedence. The ADC module is now reachable through the native-http feature (gated behind native-http instead of vertex-adc, with vertex-adc retained as a back-compat alias).
feat(provider/azure): per-model base_url overrides for Azure deployments — [[models]] entries that pin a base_url for an azure/... provider_model now route through AzureProvider::with_base_url(...), producing the required {base_url}/openai/deployments/{model}{path}?api-version=… shape instead of the generic OpenAI-compatible URL. Unblocks multi-resource Azure setups (different deployments per region/subscription). Closes #83.
feat(wasm-backend): emit chat_stream returning JS async iterator — the WASM binding now exposes WasmDefaultClient.chat_stream(req) alongside the existing chat, embed, etc. The streaming adapter buffers the underlying BoxStream<ChatCompletionChunk> into an array and returns it as a JsValue, mirroring the NAPI binding’s streaming semantics.
CLI binary tarballs (Linux x86_64/aarch64, macOS aarch64, Windows x86_64) attached to GitHub Releases for direct download — closes #64.
schemas/pricing.json regenerated from models.dev and now covers 4,219 models (up from 35); scripts/generate_pricing.py wired into task generate:pricing, task update, and task upgrade. Closes #48.
Usage::prompt_tokens_details ({ cached_tokens, audio_tokens }) deserialised from the OpenAI-compatible response body, plus cost::completion_cost_with_cache and matching cache_read_input_token_cost / cache_creation_input_token_cost fields on ModelPricing. ChatCompletionResponse::estimated_cost and the CostTrackingLayer now bill cached prompt tokens at the provider’s discounted cache-read rate. schemas/pricing.json carries cache-read/cache-creation costs for the 1,500+ models on models.dev that publish them. Closes #65.
ci-mobile: new .github/workflows/ci-mobile.yaml running android-check (ubuntu, arm64-v8a + x86_64 via cargo ndk), ios-check (macos, aarch64-apple-ios + aarch64-apple-ios-sim), and xcframework-build (macos, SPM-ready XCFramework + SHA256 checksum). Uses shared composite actions from xberg-io/actions@v1.
Alef migration to v0.23.11: the entire polyglot surface (16 language bindings — Python, Node, Ruby, PHP, Go, Java, C#, Kotlin Android, Elixir, WASM, C/FFI, Zig, Dart, Swift, Homebrew + Rust core) is regenerated end-to-end via alef. Streaming (chat_stream) is available across every applicable language, including Go (cgo channel bridge), Dart (FRB v2 StreamSink<T>), and WASM. Skipped-assertion total across e2e suites: 354 → 0.

Changed

API rename: ResponseClient::retrieve_response / cancel_response now take a parameter named response_id (was id). Positional callers are unaffected; named-arg callers must update. Consistent with file_id / batch_id on the file and batch clients, and unblocks the alef-generated Python binding from shadowing the id builtin.
GitHub Release CLI assets ship a single sorted SHA256SUMS-<version>.txt instead of one .sha256 per archive — closes #67.
WebAssembly build verified mio-free. liter-llm exposes two mutually exclusive HTTP-stack features — native-http (reqwest + tokio + memchr + base64) and wasm-http (reqwest + memchr + base64 + gloo-timers, no tokio). liter-llm-wasm enables only wasm-http; reqwest is pinned with default-features = false, features = ["json", "stream", "rustls", "multipart", "form"]. cargo build --target wasm32-unknown-unknown -p liter-llm-wasm pulls neither mio nor tokio — reqwest auto-routes to the browser/Node fetch API on wasm32 targets.
Ruby publish vendors core crates exclusively via the shared xberg-io/actions/rewrite-native-deps@v1 action (alef publish prepare, vendor_mode = "core-only"). The bespoke scripts/ci/ruby/vendor-liter-llm-core.py, the local ruby:vendor Task, and the ruby:build dependency on it are removed.
Repo hygiene: .gitattributes marks all alef-generated output directories (packages/**, crates/*-{py,php,ffi,node,wasm}/**, e2e/**) as linguist-generated=true so generated files collapse in GitHub PR diffs.

Fixed

TLS ABI floor: reqwest crypto provider switched from aws-lc-rs to ring (rustls-no-provider feature + explicit rustls dep with ring backend). Eliminates __isoc23_strtol and related glibc 2.38+ symbols emitted by aws-lc-sys 0.40.0, restoring the GLIBC_2.28 ABI floor required by downstream users (e.g. Node.js aarch64 bindings).
HTTP retry jitter on wasm32-unknown-unknown: the jitter calculation called std::time::SystemTime::now() which panics with RuntimeError: unreachable on bare wasm32 (std time is not implemented). On wasm32 the jitter step is skipped; native targets keep the existing [0.5x, 1.0x] jitter. Unblocks WASM e2e tests that exercise 429/5xx retry paths.
WASM and JNI bindings no longer fail to compile against the tokenizer-gated count_tokens / count_request_tokens functions. Both now exclude_functions in alef.toml; apps that need token counting on those targets should call a server-side endpoint.
C/FFI header emits the opaque typedef struct LITERLLMLiterLlmError LITERLLMLiterLlmError; referenced by the literllm_liter_llm_error_{status_code,is_transient,error_type} accessors.
Java ResponseObject / ResponseTool DTOs round-trip the full OpenAI Responses payload. ResponseOutputItem.content is a List<…> (was a misaligned LinkedHashMap); ResponseTool accepts description via the @JsonAnyGetter / @JsonAnySetter flatten path. Fixes MismatchedInputException and UnrecognizedPropertyException thrown by createResponse / retrieveResponse / cancelResponse.
Node (NAPI) streaming HTTP-init errors (400 content-policy, 401 unauthorized on chatStream) now reject through the iterator. Binding remains lazy (parity with Python’s async for _ in stream: pass).
Python api.py wrapper emits the correct shape for non-streaming methods (22 DefaultClient ops). Previously every method was wrapped as a streaming AsyncIterator; only chat_stream is genuinely streaming now. Also fixes String → str and bytes::Bytes → bytes mappings.

1.3.0 - 2026-04-23

Changed

Alef migration: All language bindings are now auto-generated by alef instead of hand-written
BoxFuture/BoxStream type aliases no longer wrap Result<T> — all method signatures now explicitly return Result<T>
provider module is now public (was pub(crate))
ChatCompletionRequest.stream field is now public (was pub(crate))
Switched spell checker from codespell to typos
CI no longer runs code generation — only alef verify --exit-code for freshness checks
Updated alef to v0.5.9

Added

alef.toml configuration for 10 language targets, 23 API method call configs, mock server support
bindings.rs adapter module with create_client and create_client_from_json binding-friendly constructors
Default derives on all public types for binding compatibility
Clone derive on DefaultClient
E2E test fixtures converted to alef format (167+ fixtures across 23 categories)
E2E tests regenerated for 13 languages with mock HTTP server support
Test apps generated with alef e2e generate --registry
API reference documentation auto-generated with alef docs for all 10 languages
Package READMEs generated with alef readme using restored Jinja templates
alef-verify and alef-sync-versions pre-commit hooks
alef verify --exit-code step in CI validation workflow
.lychee.toml link checker configuration
_typos.toml spell checker configuration
Auto-load API keys from environment variables
FFI callback streaming support
chat_stream method across all bindings

Removed

liter-llm-bindings-core crate — replaced by alef codegen
tools/e2e-generator crate — replaced by alef e2e generate
scripts/sync_versions.py — replaced by alef sync-versions
scripts/generate_readme.py — replaced by alef readme
scripts/readme_config.yaml and scripts/readme_templates/ — replaced by templates/readme/
tests/test_apps/ — replaced by test_apps/ (alef registry mode)
Hand-written binding source in crates/liter-llm-{py,node,ffi,wasm,php}/src/
Hand-written package source in packages/{go,java,csharp,ruby,elixir}/

1.2.2 - 2026-04-18

Added

GitHub Copilot OAuth Device Flow credential provider (copilot-auth feature) — use your Copilot subscription as an LLM backend via github_copilot/ model prefix (#12)
GitHub Copilot provider with OpenAI-compatible routing, required Copilot headers, per-request UUID, and X-Initiator header
E2E test fixtures for GitHub Copilot provider (chat + auth error)

Fixed

Provider registry audit: corrected base URLs for 20 providers (aiml, assemblyai, clarifai, dashscope, deepseek, elevenlabs, firecrawl, friendliai, gradient_ai, gmi, helicone, lambda_ai, minimax, moonshot, morph, nlp_cloud, ollama, poe, stability, wandb)
Provider registry audit: corrected env var names for 5 providers (cometapi, fal_ai, gradient_ai, jina_ai, venice)
Provider registry audit: corrected endpoint lists for 6 providers (cometapi, deepinfra, elevenlabs, jina_ai, mistral, nvidia_nim)
Added missing base_url and auth config for 11 previously non-functional providers (amazon_nova, baseten, compactifai, datarobot, docker_model_runner, duckduckgo, langgraph, lemonade, v0, vercel_ai_gateway, zai)
Added 18 stub/infrastructure providers to complex_providers list to prevent incorrect config-driven routing
Added nanogpt param mapping (max_completion_tokens → max_tokens)

1.2.1 - 2026-04-17

Added

LlmClientRaw trait with _raw variants of all LlmClient methods, returning RawExchange<T> that exposes the final request body and raw provider response before normalization (#13)
RawExchange<T> and RawStreamExchange<S> types for wire-level debugging and custom parsing
MCP & IDE integration documentation with setup guides for VS Code, GitHub Copilot, Claude Desktop, Cursor (#12)

Fixed

Docker image now published to ghcr.io/xberg-io/liter-llm (#11)
Docker publish workflow timeout increased from 60 to 360 minutes (multi-arch Rust builds via QEMU were timing out)
Bedrock build_url tests no longer flake due to BEDROCK_CROSS_REGION env var race condition

1.2.0 - 2026-04-07

Added

Local LLM provider support: Ollama, LM Studio, vLLM, llama.cpp, LocalAI, llamafile – use any local inference engine via OpenAI-compatible API
Docker Compose setup for local LLM integration testing with Ollama
Integration test suite for local LLM providers

Fixed

PHP onError hook now passes a proper \Exception object instead of a plain string (PHP strict types requires \Throwable)
README templates fixed for rumdl compliance (MD040 code fence language, MD031 blank lines, MD032 list spacing, MD020 closed headings)
Added 404 to all POST endpoint OpenAPI specs (model not found on default model names)
Homebrew badge added to all READMEs

1.1.1 - 2026-03-29

Fixed

Java Maven plugins downgraded to 3.x stable (was 4.0.0-beta, incompatible with Maven 3.9.x CI)
PHP hook isolation (per-client instead of global), budget per-model enforcement, onError hook invocation, shutdown segfault
PHP e2e tests set max_retries=0 to prevent retry delays on mock 500s
OpenAPI spec: added 400/415/422/503 status codes to all endpoints for schemathesis compliance
first_client() returns 503 Service Unavailable instead of 500 for “no models configured”
Schemathesis CI checks aligned (removed content_type_conformance, not_a_server_error)
Docker cache: per-platform TARGETARCH cache IDs prevent multi-arch build races

Added

Homebrew formula: brew tap xberg-io/tap && brew install liter-llm
Homebrew bottle builds (arm64_sequoia) in publish workflow
liter-llm-proxy and liter-llm-cli added to crates.io publish pipeline
Installation docs: CLI/Docker/Homebrew tabs
scripts/publish/upload-homebrew-bottles.sh and ensure-github-release-exists.sh

1.1.0 - 2026-03-29

OpenAI-compatible LLM proxy server with CLI, MCP tool server, and Docker support.

Proxy Server (`liter-llm-proxy`)

22 REST endpoints — full OpenAI-compatible API surface: chat completions (streaming + non-streaming), embeddings, models, images, audio (speech + transcription), moderations, rerank, search, OCR, files CRUD, batches CRUD, responses CRUD, health
Tower middleware stack — reuses core middleware: cache, rate limit, budget, cost tracking, cooldown, health check, tracing
Virtual API keys — in-memory key store with per-key model restrictions, RPM/TPM limits, budget limits
Model routing — name-based routing to provider deployments, wildcard aliases, deterministic default client
OpenDAL file storage — configurable backend (memory, S3, GCS, filesystem) for file operations
SSE streaming — chat completion chunks proxied as Server-Sent Events with [DONE] sentinel
OpenAPI 3.1 — utoipa-generated spec served at /openapi.json with bearer auth security scheme
TOML configuration — liter-llm-proxy.toml with env var interpolation (${VAR}), auto-discovery, deny_unknown_fields
CORS — configurable origins from config (default: allow all)
Graceful shutdown — SIGINT/SIGTERM handling via tokio::signal

MCP Server (`rmcp`)

22 tools — full parity with REST API: chat, embed, list_models, generate_image, speech, transcribe, moderate, rerank, search, ocr, file CRUD (5), batch CRUD (4), response CRUD (3)
Transports — stdio (default) and HTTP/SSE via StreamableHttpService
Parameter schemas — schemars::JsonSchema derives for MCP tool discovery

CLI (`liter-llm`)

liter-llm api — start proxy server with config, host/port overrides, debug logging
liter-llm mcp — start MCP server with stdio or HTTP transport
3-tier config precedence: CLI flags > env vars > config file > defaults

Docker

Multi-stage build: rust:1.91-bookworm builder, cgr.dev/chainguard/glibc-dynamic runtime (35MB)
Non-root execution, OCI labels, port 4000 exposed
ENTRYPOINT ["liter-llm"], CMD ["api", "--host", "0.0.0.0", "--port", "4000"]

Testing

74 unit tests — config parsing, error mapping, auth key store, service pool, file store, streaming
32 integration tests — auth middleware, chat/embedding/models routes, error propagation, CORS, health, OpenAPI
12 proxy e2e fixtures — chat (basic + streaming), embeddings, models, auth errors, upstream errors, health, images, moderation, reranking
Schemathesis — contract testing against OpenAPI spec via Docker (task proxy:schemathesis)

CI/CD

.github/workflows/ci-docker.yaml — build + health test + schemathesis contract tests
.github/workflows/publish-docker.yaml — multi-arch (amd64/arm64) publish to ghcr.io/xberg-io/liter-llm
Taskfile: proxy:test, proxy:schemathesis

1.0.0 - 2026-03-28

Initial stable release. Universal LLM API client with native bindings for 11 languages and 142+ providers.

Core

LlmClient trait with chat, chat_stream, embed, list_models, image_generate, speech, transcribe, moderate, rerank, search, ocr
FileClient, BatchClient, ResponseClient traits for file/batch/response operations
DefaultClient with reqwest + tokio, SSE streaming, retry with exponential backoff
ManagedClient with composable Tower middleware stack
142 LLM providers embedded at compile time from schemas/providers.json
Per-request provider routing from model name prefix (e.g. anthropic/claude-sonnet-4-20250514)
secrecy::SecretString for API keys (zeroized on drop, never logged)
TOML configuration file loading with auto-discovery (liter-llm.toml)
Custom provider registration at runtime

Middleware (Tower)

CacheLayer — in-memory LRU + pluggable backends via CacheStore trait
OpenDAL cache — 40+ storage backends (Redis, S3, GCS, filesystem, etc.) via Apache OpenDAL
BudgetLayer — global + per-model spending limits with hard/soft enforcement
HooksLayer — request/response/error lifecycle callbacks with guardrail pattern
CooldownLayer — circuit breaker after transient errors
ModelRateLimitLayer — per-model RPM/TPM rate limiting
HealthCheckLayer — background health probing
CostTrackingLayer — per-request cost calculation from embedded pricing registry
TracingLayer — OpenTelemetry GenAI semantic convention spans
FallbackLayer — automatic failover to backup provider
RouterLayer — multi-deployment load balancing (round-robin, latency, cost, weighted)

Language Bindings

All bindings expose the full API surface with language-idiomatic conventions:

Python (PyO3) — async/await, typed kwargs, full .pyi stubs
TypeScript / Node.js (NAPI-RS) — camelCase, .d.ts types, Promise-based
Rust — native, zero-cost
Go (cgo) — FFI wrapper with build tags, context.Context support
Java (Panama FFM) — JDK 25+, AutoCloseable, builder pattern
C# / .NET (P/Invoke) — async/await, IAsyncEnumerable streaming, IDisposable
Ruby (Magnus) — RBS type signatures, Enumerator streaming
Elixir (Rustler NIF) — {:ok, result} tuples, OTP-compatible
PHP (ext-php-rs) — PHP 8.2+, JSON in/out, PIE packages
WebAssembly (wasm-bindgen) — browser + Node.js, Fetch API
C / FFI (cbindgen) — extern "C" with opaque handles

Authentication

Static API keys (Bearer, x-api-key)
Azure AD OAuth2 client credentials
Vertex AI service account JWT
AWS STS Web Identity (EKS/IRSA)
AWS SigV4 signing for Bedrock

Provider Transforms

Anthropic: message format, tool use v1, thinking blocks, max_tokens default
AWS Bedrock: Converse API, EventStream binary framing, cross-region routing
Vertex AI: Gemini format, embedding :predict endpoint
Google AI: embedding/list_models response transforms
Cohere: citation handling
Mistral: API compatibility
param_mappings for config-driven field renaming (8 providers)

Documentation

MkDocs Material site at docs.liter-llm.xberg.io
170+ code snippets across 10 languages
11 API reference docs with full method coverage
Usage pages: Chat & Streaming, Embeddings & Rerank, Media, Search & OCR, Files & Batches, Configuration
TOML configuration reference
llms.txt (218 lines) with capabilities, examples, provider list
Skills directory (4,072 lines) for Claude Code integration
README generation from Jinja templates via scripts/generate_readme.py

Testing

500+ unit and integration tests
Middleware stack composition tests (cache + budget + hooks + rate limit + cooldown)
Per-request provider routing tests
File/batch/response CRUD operation tests
Concurrency tests (budget atomicity, cache contention, rate limit fairness)
Redis cache backend integration tests (Docker Compose)
Live provider tests for 7 providers (OpenAI, Anthropic, Google AI, Vertex AI, Mistral, Azure, Bedrock)
Smoke test apps for all 10 languages against real APIs
E2E test generation from JSON fixtures across all languages
Contract test fixtures for binding API parity

CI/CD

Multi-platform publish pipeline: crates.io, PyPI, npm, RubyGems, Hex.pm, Maven Central, NuGet, Packagist, Go FFI, PHP PIE
Pre-commit hooks: 43 linters across all languages
Post-generation formatting in e2e-generator
Version sync script across 27+ manifests with README regeneration

Previous RC Releases

Release candidate history (rc.1 through rc.9)

rc.1 (2026-03-27): Initial release — core crate, 11 bindings, e2e generator
rc.2 (2026-03-27): Packaging fixes for crates.io, RubyGems, Elixir NIF, Node NAPI, publish workflow
rc.3 (2026-03-27): Cache, budget, hooks middleware; custom providers; TDD e2e fixtures
rc.4 (2026-03-28): Shared bindings-core crate; camelCase conversion; real streaming across all bindings
rc.5 (2026-03-28): OpenDAL cache; search/OCR endpoints; full middleware wiring; Go/Java/C# FFI rewrites; serde deny_unknown_fields; documentation overhaul
rc.6 (2026-03-28): Full API documentation coverage; Rust crate README; version sync improvements
rc.7 (2026-03-28): Binding parity (5 middleware params + search/ocr in all 10); contract test fixtures; skills directory; PHP PIE packages
rc.8 (2026-03-28): CI fixes (PHP publish, crate order, Maven GPG, Ruby deps, Bedrock test)
rc.9 (2026-03-28): Live provider tests; Anthropic/Bedrock/Google streaming fixes; TOML config loading; per-request provider routing; integration test suite