Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[Unreleased]
Section titled “[Unreleased]”[2.7.0] - 2026-11-06
Section titled “[2.7.0] - 2026-11-06”- Anthropic:
reasoning_efforton Claude Opus 4.6+, Sonnet 4.6+, Haiku 5.5, Fable and Mythos now sends adaptive thinking withoutput_config.effort(minimalbecomeslow, andxhighbecomesmaxon Opus and Sonnet 4.6), instead of athinking.budget_tokensthat Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 reject. When the request sets nomax_tokens, it is raised to at least 16,000 (64,000 atxhighandmax), since adaptive thinking spends it on thinking and answer together. Older models keep the token budget, withxhighbetweenhighandmax, and only they still send thethinking-2025-04-14beta header.noneon Claude Sonnet 5.5 sendsthinking: {type: "between_tools"}, its documented way to turn thinking off (#473). - Anthropic: thinking blocks survive a tool loop. A reply keeps each
thinkingblock, with itssignature, and eachredacted_thinkingblock in the newAssistantMessage.thinking_blocks(ThinkingBlock, with the block’s position among the turn’s text and tool calls), and sending the message back returns them to Anthropic unchanged at that position, as Anthropic requires alongside tool results. A block without itssignature(ordata) is rejected withBadRequestnaming the field. Streams carry them asStreamDelta.thinking_blocksfragments keyed by block index, including the signature that was dropped before; in Rust,ThinkingBlockAccumulatormerges a stream’s fragments into the blocks a reply without streaming returns. Other providers never receive the field. Rust code that buildsAssistantMessage,StreamDeltaorThinkingBlockwith a struct literal and without..Default::default()must add the new fields (#474).
[2.6.1] - 2026-11-06
Section titled “[2.6.1] - 2026-11-06”Changed
Section titled “Changed”- macOS 15 is the minimum for every macOS artifact (2.6.0 required macOS 26): the FFI libraries, language extensions, CLI, Swift bundle, Python wheels (now tagged
macosx_15_0) and the Swift package (.macOS(.v15)). The floor is the oldest macOS release Apple still supports (#470).
[2.6.0] - 2026-11-05
Section titled “[2.6.0] - 2026-11-05”Changed
Section titled “Changed”- Breaking for macOS 25 and earlier: every macOS binary the release ships (FFI libraries, language extensions, CLI, Swift bundle and Python wheels) and the Swift package now require macOS 26, instead of macOS 11 or 13 for the binaries and macOS 14 for the 2.5.0
Package.swift. The macOS Python wheels are taggedmacosx_26_0(#470). - Regenerate the bindings with Alef 0.124.2. The published Python package now declares
requires-python = ">=3.11"and buildsabi3-py311wheels, matching the Python 3.11 minimum. Python option enums subclassStrEnum.
- Swift: the core
vX.Y.Ztag’sPackage.swiftnow pins the artifact bundle checksum, sofrom:andexact:consumers resolve a working manifest. A release is now two steps: dispatch Prepare Swift release on main, which calls the shared xberg-io/actions workflow to build the bundle once, push the checksum pin with the tag and attach the exact ZIP and receipt to a draft release; then dispatch Publish Release on the tag, which verifies that state, never rebuilds the bundle and createsrelease/swift/<version>at the tagged commit (procedure). Publish Release verifies the tag and the attached Swift bundle before any registry publishes, rechecks them before it publishes the draft, and rejectsrepublishandforce_republishoutside dry runs. Theswift-<version>tags are no longer created (#460).
[2.5.0] - 2026-11-03
Section titled “[2.5.0] - 2026-11-03”blocked_networks: a per-client list of CIDR networks the client never connects to. A host name is refused when any address it resolves to is in a listed network, and an IP-literal URL is refused before the request is sent. Set it onClientConfigBuilder,ClientBuilder,ClientOptions(every binding) or the config file. A refused request fails withOutboundForbidden.wasm32refuses a list that is not empty with aBadRequesterror.
Changed
Section titled “Changed”- Minimum supported versions raised: Node.js 22, PHP 8.3, Elixir 1.17, Go 1.27, Android API 24, Dart 3.13, Zig 0.17, macOS 14.
- Python 3.11 is now the minimum supported version; Python 3.10 reached end of life.
- Refresh served skill lock digests after regenerating coding-agent plugin artifacts.
- Keep macOS extension linker flags out of iOS checks and XCFramework builds.
- Keep CI Python imports sorted and use the builtin exception group in the Hermes bridge.
- Regenerate Rust E2E suites with compatible Alef 0.115.15 and per-call client trait imports, preserving stream checks without unused bindings under strict warnings.
- Refresh generated error-family interfaces and required collection JSON serializers while preserving the existing Dart bindings.
- Align the generated Python package README with the Python 3.11 minimum requirement.
- Mistral: preserve parallel tool-call control and presence/frequency penalties, including false and zero values (#458).
- Outbound policy: refuse IPv6 forms that carry a blocked IPv4 address, such as IPv4-mapped, IPv4-compatible, SIIT, NAT64, 6to4 and Teredo addresses, and every ISATAP address (#462).
[2.4.4] - 2026-10-31
Section titled “[2.4.4] - 2026-10-31”- Refresh stale coding-agent plugin provenance that blocked OpenCode and Pi package publication.
Changed
Section titled “Changed”- Regenerate bindings with Alef 0.115.10, including private Swift
CodingKeysand Java Maven Enforcer 3.7.0.
[2.4.3] - 2026-10-30
Section titled “[2.4.3] - 2026-10-30”- Cohere: preserve streamed thinking content separately from visible answer text (#453).
Changed
Section titled “Changed”- Upgrade Rust dependencies, including PyO3 0.30, RMCP 3.8, JSON Schema 0.60 and OpenDAL 0.59.
- Regenerate bindings and release scaffolding with Alef 0.115.7.
[2.4.2] - 2026-10-29
Section titled “[2.4.2] - 2026-10-29”- WebAssembly: an assistant message has a
contentValuegetter. It returns the content as sent, a string or the list of parts. Thecontentgetter still returns the display text only (#412).
Changed
Section titled “Changed”- Build: mise replaces go-task.
mise.tomlpins the toolchain at the xberg-io/toolchains v1.7.1 baseline, every task keeps its name undermise run, and CI sets up tools withsetup-mise.Taskfile.ymland.task/are gone. - Ruby: a message read back from the native extension is an instance of its variant class, such as
LiterLlm::MessageUser, not a plain Hash. Code that expects a Hash can callto_hon it (#412). - Regenerate bindings and end-to-end suites with alef 0.115.5 (#412).
- Sync the model catalog from models.dev (#446, #451).
- Security: update pulldown-cmark to 0.13.5 for RUSTSEC-2026-0414. The crate is a build dependency through prost-build (#448).
- Build:
mise run version:syncrefreshes the ai-rulez lock, so a version bump no longer fails the lock check (#444). - CI:
alef verifyand the poly format check pass on main again (#448). - CI: pin poly to 0.31.2, so a new poly release cannot turn main red without a commit (#449).
- Tests: the bad-purpose file fixture is skipped in Elixir, PHP and Swift, which reject the value before any HTTP request (#416). Every shell script has a Bats suite, and the generated Homebrew consumer checks run through Bats (#443).
[2.4.1] - 2026-10-27
Section titled “[2.4.1] - 2026-10-27”Changed
Section titled “Changed”- Regenerate bindings with alef 0.114.0. Five end-to-end fixtures that skipped bindings now run (#391).
- Anthropic:
json_schemastructured output closes every object schema withadditionalProperties: false(#403). - Anthropic: a list-form system message skips blank text parts (#399).
- Anthropic and Bedrock: streamed tool calls are numbered from 0 across the whole stream, in the order they start. Argument deltas keep the number of their own call (#435).
- Anthropic: a streamed refusal reports the finish reason
content_filter, as the reply that is not streamed does, notother(#436). - Gemini: a tool result carries the name of the function that was called, not
tool(#417). - Gemini: a reply that calls a tool finishes with
tool_calls(#420). - Gemini: presence penalty and frequency penalty are sent (#421). The seed is sent (#422).
- Gemini and Vertex: streamed tool calls are numbered across the whole stream. The first call is 0, the second is 1, also when the calls arrive in different chunks (#439).
- Ollama: a stream that stops at the token limit after a tool call finishes with
tool_calls(#440). - An API key that Google reports as not valid is an authentication error, for Google AI and Vertex AI only, on streams and on calls that are not streams (#418, #426).
- llama.cpp and LM Studio: a request that does not fit the context window raises
ContextWindowExceeded, notBadRequest(#437). list_catalog_models(provider)accepts the routing namesgemini,google_aiandvertex_ai. They list the models of the catalog idsgoogleandgoogle-vertex, as the cost lookup already resolves them (#405).- Build: the stale pnpm and uv lock files are refreshed, and the PHP end-to-end config no longer holds a macOS path (#413).
- CI: the release notes start from the CHANGELOG entry of the version (#392). The hook check matches the ai-rulez
version exactly (#401). The CHANGELOG entry test passes
checktosubprocess.run(#428).
Not in this release
Section titled “Not in this release”The next alef bump waits for the alef fix for the dropped end-to-end assertions (xberg-io/alef#412).
[2.4.0] - 2026-10-24
Section titled “[2.4.0] - 2026-10-24”Changed
Section titled “Changed”ollama/(and its aliasollama_chat/) now uses Ollama’s own API,/api/chatand/api/embed, instead of the OpenAI-compatible path. Settings inextra_bodysuch asnum_ctx,top_kandrepeat_penaltyreach Ollama asoptions, where Ollama applies them. Chat streams read newline-delimited JSON. The defaultbase_urlis nowhttp://localhost:11434. Abase_urlthat ends in/v1still works: native calls strip one trailing/v1before they append/api/.... An Ollama hint always means the native API: a client built with the hintollamaorollama_chat, or with a hint that starts withollama/orollama_chat/, sends every bare model to/api/chatand/api/embedwhateverbase_urlis, and itslist_modelsreads/api/tags, which Ollama serves, instead of/models, and setscreatedfrommodified_at. On such a client a model with another provider’s prefix, such asopenai/m,groq/mor the name of a registered custom provider, still goes to the OpenAI-compatible endpoint at thebase_urlas given, with the model name as given; a model whose name holds a slash but no provider prefix, such ashf.co/user/model, is bare. To reach Ollama’s OpenAI-compatible endpoint, use theopenai/prefix with the Ollamabase_url.reasoning_effortbecomesthink, withnonesent asthink: false, andlogprobsreach Ollama. Anextra_bodykey set tonullis not sent. Fields Ollama has no equivalent for are dropped, as the Anthropic provider does; the Ollama section of the local LLMs page lists them. A reply that only calls tools keeps an emptycontent. Error text of other providers is unchanged (#308).ollama/requests that Ollama’s API cannot take now fail withBadRequestbefore they are sent (#308):- An
image_urlpart with anhttporhttpsURL, or a data URL without;base64: send the image as a base64 data URL. - A message content part other than
textandimage_url: send text and images only. - Tool-call
argumentsin an assistant message that are not valid JSON: send them as a JSON object. - Embedding
inputthat is not a string or a list of strings: send the text as strings.
- An
- Regenerate bindings with alef 0.112.0. The Rust, Go, Java, PHP and C streaming tests read the stream before they assert a mid-stream error.
- Swift: the enum case
noneis nownone_(the serialized value is unchanged). - Zig: wrappers report native conversion failures through error unions.
- Python: getters of untagged enums return the payload of the matching variant. The stub declares them.
- Python: error classes carry attributes, such as
messageandstatus. - PHP: errors are typed exception classes (29 classes under
LiterLlmErrorandCatalogRefreshError). - Dart: errors are typed values, and
retryAfterandmodelare nullable. - Java: builder setters of optional primitive fields take
Longand have alongoverload, so the signature changes. - Elixir: specs name struct types instead of
map(). - Ruby: an optional string argument of the wrong type raises instead of becoming an empty string.
- The client option
connect_timeout_secs(Rustconnect_timeout) sets how long the client waits for a connection to open. The default is 10 seconds; before, the connect step had no limit of its own, so this changes a public default. It is set in the client options, the TOML and JSON config files, andLlmConfigof every binding. A connection that does not open in time fails withTimeoutand is retried up tomax_retriestimes, because the server never saw the request. A request that times out after the connection is open is still sent once. The requesttimeout_secsends the call first when it is shorter than the connect timeout. A value of zero is aBadRequesterror when the client is created, on every target.wasm32ignores any other value (#341).
StreamFormathas a newNdjsonvariant for providers that stream newline-delimited JSON (#308).list_catalog_models(provider)lists the models of the built-in catalog without a network call or an API key, in every binding. Each entry has the full catalog name, the provider, the model, the mode, the token limits and the ability flags. The list is sorted by name in byte order and covers an installed catalog refresh. The optionalproviderargument keeps the models of one catalog provider id, such asopenai; a routing name fromall_providersthat is not a catalog id gives an empty list, and an emptyproviderkeeps every provider (#310).- The embedding, rerank, image, speech, transcription, moderation, search and OCR requests take an
extra_bodyfield in every binding. Its keys are merged into the provider request body and override fields with the same name, as the chatextra_bodydoes. A Bedrock Cohere embed request takesinput_typefromextra_body, such assearch_query, and still sendssearch_documentwhenextra_bodyhas none. Bedrock Cohere and Titan embed requests send otherextra_bodykeys, such astruncateornormalize, in the request body. The embedding cache key includesextra_body(#334). - Gemini API (
gemini/andgoogle_ai/) embed requests merge theextra_bodykeys, such astaskTypeandtitle, into each embed request (#334).
Changed
Section titled “Changed”- Swift:
CreateImageRequest,CreateSpeechRequest,CreateTranscriptionRequest,EmbeddingRequest,ModerationRequest,OcrRequest,RerankRequestandSearchRequestare nowtypealiases over the Rust bridge types, likeChatCompletionRequest, rather than nativestructs. They gain theextra_bodyfield, which holds JSON. Code that built them with the memberwise initialiser must use the bridge initialiser, and they are no longerCodableorHashablevalue types. The 16 public*Refand*RefMuttypealiases of these eight types, such asEmbeddingRequestRefandEmbeddingRequestRefMut, are removed (#334).
- One rule now decides the provider route from
model_hint, andbase_urlchanges only the host. Nowcohere,cohere/command-r-plusandcommand-r-plusselect the same provider with or without abase_url, and the request keeps that provider’s body, path, query and reply handling. A path the provider serves outside its own base URL keeps its whole path at thebase_urlhost: the Cohere model list goes to/v1/models?page_size=1000there. A request model without aprovider/prefix stays on the provider that the hint selected, and this holds for a model-name hint too: under agpt-4ohint, every bare model is sent as an OpenAI model. Behaviour change: these hints now reach a different route. With abase_url, a bare or model-name hint for Anthropic, Gemini, Vertex AI, Voyage, Groq, Cohere or Mistral sends that provider’s native request to thebase_urlhost, where it sent the generic OpenAI-shaped request before, so a gateway that expects an OpenAI-shaped request now receives the native one. Drop the hint to keep the OpenAI-shaped request. With abase_url, a model-name hint for OpenAI (such asgpt-4o) now strips theopenai/prefix fromopenai/gpt-4o. Without abase_url, a barevoyage,groq,mistral,cohere,geminiorvertex_aihint now reaches that provider’s own host instead ofapi.openai.com, and avertex_aihint no longer reaches Google AI Studio. With abase_urland an emptyapi_key, a bare or model-name hint (anthropic,claude-sonnet-4,azure,gemini,google_ai,vertex_ai,bedrock,cohere) reads no key from the environment, as before. Only aprovider/hint reads one there, as before:anthropic/,azure/,gemini/andgoogle_ai/readANTHROPIC_API_KEY,AZURE_OPENAI_API_KEYandGEMINI_API_KEY,vertex_ai/gets a Google access token, andbedrock/usesAWS_BEARER_TOKEN_BEDROCKor signs with your AWS keys. A barebedrockhint at abase_urlsends the request unsigned unless the client config carries AWS keys. A provider that needs no key, such aslmstudio, sends noAuthorizationheader at its own host and sends theapi_keyas a bearer token at abase_url(#330). - Send Vertex AI and Bedrock image, rerank and list-models calls to the paths those services serve. Vertex image generation goes to the Imagen
:predictmethod with aninstancesandparametersbody, andnandsizebecomesampleCountandaspectRatio. Vertex list models readsv1beta1/publishers/google/modelsand followsnextPageTokento the last page. Bedrock rerank goes to/rerankon thebedrock-agent-runtimehost with aqueries,sourcesandrerankingConfigurationbody, and itsrelevanceScorereplies parse as results. Bedrock list models reads/foundation-modelson thebedrockhost. Abase_urlon a Bedrock client moves every call, rerank and list models included. Before, each call went to a chat-shaped path and failed with a serialization error (#332). ollama/embed requests now sendextra_bodyto/api/embed. The keystruncate,keep_aliveanddimensionsstay at the top level. Every other key, such asnum_ctx, goes intooptions, and the keys of an explicitoptionsobject win on a conflict. Before, the embed request droppedextra_body(#329).- An error reply with a plain-text
errorfield, as Ollama sends, now shows that text as the error message instead of the raw JSON body (#308). chat_stream_rawnow ends inLiterLlmError::Streamingwhen a stream is cut before the provider’s final event, aschat_streamdoes. This covers Ollama, Anthropic, Gemini, Vertex AI, Cohere and Bedrock (#308).- Parse the rerank replies of
fireworks_ai/(hits underdata),nvidia_nim/(hits underrankings, with alogitfor the score) andhuggingface/Text Embeddings Inference (a bare top-level array, with ascoreand an optionaltext), so these calls return the sameresultsas other providers instead of raisingSerializationError. The reply shape follows the model prefix or the client’smodel_hint, as for Voyage. A reply that hasrelevance_scorekeeps it. The TEItextbecomes the resultdocument. The NIMlogitis passed through unchanged and is not limited to[0, 1](#327). - Send rerank requests to
nvidia_nim/andhuggingface/in the shape those providers document. NVIDIA NIM getsPOST /rankingwithqueryas{"text": ...}and the documents aspassages. TEI getstextsfor the documents andreturn_textforreturn_documents. Neither documents a field that limits the number of results, sotop_nis not sent to them (#327). - A rerank request to a
voyage/model, or from a client whosemodel_hintisvoyage, now sendstop_nastop_k, the field Voyage documents. Other providers keeptop_n(#319). - Vertex AI embed requests send
dimensionsasparameters.outputDimensionality. Before, the value was dropped without an error. Anextra_body.parametersobject adds its keys tooutputDimensionalityand does not replace it (#334). - Refuse
elevenlabs/,deepgram/,stability/,tavily/,fal_ai/,runwayml/,assemblyai/andserper/models when nobase_urlis set, with the same error as other providers that have no built-in route. These services have native APIs, and the client sent them OpenAI-shaped requests that they reject. The provider list no longer declares audio, image, search or chat endpoints for them, so the providers page shows none. A call with abase_urlis unchanged, and an emptybase_urlcounts as unset. (#333) - Refuse calls to an endpoint that a provider has no route for, before any request is sent, with the same error as other providers that have no built-in route. The error names the endpoint. This covers search for
brave/,perplexity/,exa_ai/,firecrawl/,linkup/,parallel_ai/,google_pse/andduckduckgo/, and audio and image foredenai/. These services have native APIs for those endpoints, and the client sent them OpenAI-shaped requests that fail. Their other endpoints, such as Brave, Perplexity and Eden AI chat, keep working. A call with abase_urlis unchanged, and so is a custom provider registered under one of these names. (#350) - Cohere embed and list-models calls now use Cohere’s own endpoints. Embed sends
POST /v2/embedwithtexts(orinputsfor multimodal input),input_type(search_document) andembedding_types: ["float"], mapsdimensionstooutput_dimension, and readsembeddings.float. List models sendsGET /v1/modelsand readsmodels. The bare model nameembed-english-v3.0takes the same route ascohere/embed-english-v3.0. List models asks forpage_size=1000, the largest page Cohere allows, and returns an error if Cohere still reports a next page. A Cohere embed reply takes the model of the request, because Cohere sends none (other providers keep the replymodelas sent), and billed image tokens count as input tokens. An empty input list is rejected before any request. Anextra_bodyon a Cohere embed request is merged into the request body, so a caller can setinput_type(#331). - The Cohere reply step rebuilds only chat replies. Before, it replaced every other reply it received with an empty chat completion and dropped all of its fields. Other replies now pass through unchanged (#318).
- Parse Voyage rerank replies, which list the scored documents under
datainstead ofresults, sovoyage/rerank calls return the sameresultsas other providers. The reply shape follows the model in the request, and a client that names Voyage as itsmodel_hintreads Voyage replies for a bare model such asrerank-2. A custom provider that claims thevoyage/prefix still decides where the request goes, but the reply is read as a Voyage reply. Aresults: nullbeside a validdataarray parses fromdata. Thevoyage/prefix is case-sensitive, like every other provider prefix. (#306) - Rerank replies no longer pass through the chat transform of the provider that sends the request. This fixes
cohere/rerank calls, whose replies the Cohere chat transform rebuilt without theresultsarray, and any rerank call from a client pinned to a provider with a chat transform. (#306) reasoning_effort: "none"is now accepted by the typedReasoningEffortfield in every binding and reaches the server as"none"on OpenAI-compatible, Azure and Ollama endpoints. Anthropic and Bedrock sendthinking: {type: "disabled"}, and Vertex AI and Gemini send a thinking budget of 0 (#301).- A model for a registry provider that has no route of its own, such as
snowflake/arcticorwatsonx/ibm-slate, now fails with an error that names the provider and asks for abase_url. Before, a call withoutbase_urlfell back to the OpenAI endpoint. The gateways that need abase_url(custom,custom_openai,hosted_vllm,infinity,litellm_proxy,openai_like) fail the same way when it is missing. A client hint that names such a provider without amodelprefix, such assnowflake, fails the same way when nobase_urlis set. The proxy refuses to start when a[[models]]entry names such a provider and has nobase_url, andliter-llm apinow exits with that error instead of staying up with no listener. Plain OpenAI models and providers with a route behave as before. - Publish an
abi3Python wheel for Intel macOS (macosx_11_0_x86_64), cross-built on the arm64 runner, and fail the build when any wheel’s platform tag or machine code is not the one its build leg promises. (#297) - An error event inside an OpenAI-compatible chat stream that already returned 200 now raises the same error kind as the same body on an HTTP error, for example
RateLimitedErrorwith status 429. An event with nostatus_codetakes its status from the errorcodeortype(#304). - An error event inside a chat stream now carries the event’s
retry_after(seconds, as a number or a numeric string, in the error object or at the top level; an unusable value in the error object does not hide the top level), soretry_after()on theRateLimitederror returns it as it does for an HTTP reply (#339). - An error event inside a Gemini or Vertex AI chat stream, such as
{"error": {"code": 429, ...}}, now raises the error kind that the same body gives as an HTTP reply. Before, the stream ended with no error. An error event inside an Anthropic or Cohere chat stream, and an exception frame inside a Bedrock chat stream, raise that kind too, for exampleRateLimitedErrorfor an Anthropicrate_limit_erroror a BedrockthrottlingException. Anthropicbilling_error,request_too_largeandtimeout_errorkeep the status 402, 413 and 504. A BedrockmodelStreamErrorExceptionandmodelErrorExceptionkeep the status 424. The Bedrock exception type is read from the:exception-typeheader of the frame. A Bedrock frame with the message typeerrorraises a streaming error with its error code and message. A Gemini or Vertex AI error without a numericcodetakes its status from thestatusname, such asRESOURCE_EXHAUSTEDfor 429, and acodethat is a string reads as a number. An error event readsstatus_codeas a number or a string. Astatus_codeat the top level of the event counts only when it is an error status (400 to 599), and the one in the error object wins. A Coheremessage-endevent with an error text or the finish reasonERRORraises a streaming error. Before, Anthropic and Bedrock raised a generic streaming error and Cohere dropped the event (#340, #343). - A chat stream that the server closes cleanly with no
data: [DONE]and no chunk that carries afinish_reasonnow ends with aStreamingerror. Before, a cut that fell between two events ended the stream with no error, so a short reply looked complete. This reverses the 2.3.0 line that OpenAI-compatible streams keep ending cleanly without[DONE](#295): a close with neither[DONE]nor afinish_reasonis now an error. A stream that ends with[DONE], or with a chunk that has afinish_reasonand no[DONE], ends as before. The raw chat stream (chat_stream_raw) of Anthropic, Gemini, Vertex AI and Cohere now ends with aStreamingerror on a cut too. Before, it ended with no error. The parsed Anthropic, Gemini, Vertex AI and Cohere streams keep their own error, which names the provider. The Responses API stream is checked too (#367). - A Responses API stream (
create_response_stream) that the server closes cleanly beforeresponse.completed,response.incompleteorresponse.failednow ends with aStreamingerror after the last event. Before, it ended with no error, so a short reply looked complete. Anerrorevent followed by a clean close now ends with theStreamingerror after the error item. A stream that ends with one of these events, or withdata: [DONE], ends as before. The error text no longer names[DONE]or a finish reason, which do not apply to every stream (#367). - A request that times out is no longer sent again. Before, one slow reply cost about
timeout_secs * (max_retries + 1)plus back-off, and the server ran (and billed) the request each time. The caller now getsLiterLlmError::Timeoutafter abouttimeout_secs. A timeout after the connection is open is sent once. A connection that fails or closes without a reply,429and5xxreplies keep their retries (#316). - Apply the client timeout on WASM (wasm32). Before, the WASM client set no timeout, so a call waited for the server for as long as it took.
timeout_secsnow limits each attempt of a call that returns one reply, from the request to the end of the reply body, and the caller getsLiterLlmError::Timeout. It limits a stream by the silence between chunks, as on native targets. A request that times out is not sent again. This covers chat, embeddings and every other JSON call, binary calls, file uploads, and server-sent event and Bedrock event streams. A client on wasm32 with no timeout set uses the 60 s default timeout, as on native targets. Theliter-llmcrate alone now builds for wasm32 with thewasm-httpfeature (#365, #366).
Security
Section titled “Security”- Reject distrusted emSign certificate chains by updating TLS validation and bundled trust stores (RUSTSEC-2026-0381, RUSTSEC-2026-0382, RUSTSEC-2026-0383).
Not in this release
Section titled “Not in this release”These tests stay skipped until alef, the binding generator, fixes the generated test code:
- The parts reply test in Go, C, Java and Kotlin reads the parts through an accessor the binding does not have (xberg-io/alef#688).
- The Ruby empty response body test names an error class the extension does not define (xberg-io/alef#687).
- The streamed usage test, in 11 bindings, does not read usage from the last chunk (xberg-io/alef#689).
- Two wasm file tests: the create-file skip is stale, and the file-content test needs a test-config override.
Two Swift binding defects are open in alef and ship unchanged in this release:
- The Swift binding has no public member that returns the parts of a reply, so its parts reply test is also skipped (xberg-io/alef#691).
- In Swift,
result.choices()[0].message()in one expression reads freed memory. Keep the list in a local variable first (xberg-io/alef#693).
[2.3.0] - 2026-10-19
Section titled “[2.3.0] - 2026-10-19”Changed
Section titled “Changed”- Anthropic, Gemini, Vertex AI, Cohere and Bedrock chat streams that end before the provider’s final event now yield
LiterLlmError::Streaminginstead of ending like a complete response. OpenAI-compatible streams keep ending cleanly without[DONE](#295). - Regenerate bindings with alef 0.111.0: the Python binding moves to pyo3 0.30, and the Dart
LiterLlmError.retryAfter()accessor is nowgetRetryAfter().
liter-llm config-check --config <path>validates a proxy configuration without starting the server, printing one JSON line and exiting non-zero when the config is invalid or unreadable (#294).TransportConfig::disable_proxies(with_proxies_disabled) makes a client ignore ambientHTTP(S)_PROXYand system proxy settings even when the outbound policy isOff(#293).
Security
Section titled “Security”- Remove vulnerable ZIP 8.6 dependencies from PHP binding builds.
- Limit a stream by the silence between chunks instead of its total time.
timeout_secs(60 s by default) now ends a stream only when no data arrives for that long, so a long answer that keeps sending chunks completes. A call that returns one reply keeps the whole-request limit. A stalled stream still fails with the timeout error. (#314) - Reject a Bedrock
messageStopevent withoutstopReasoninstead of reporting it asend_turn, and stop reporting intermediate Gemini and Vertex AI stream chunks as finished (#295). - Omit an absent assistant
refusalfrom request message history, which strict OpenAI-compatible providers such as Groq reject, while completion responses keep the requiredrefusal: nullkey (#292). - Replace the
php:testreference to a nonexistent example with a real local smoke check that loads the built extension in isolation, verifies its version and core classes, and propagates failures (#290). - Accept the optional upstream models.dev model
typefield in the catalog generator while still rejecting unknown fields and model types, so scheduled catalog refreshes succeed again (#289). - Stream chat completions through
LlmService, and therefore the proxy, as the provider sends them instead of buffering the whole completion before the first chunk. Abandoned streams still settle spend: a background task drains them to the usage-bearing final chunk, andDropno longer polls the live provider stream. - Hold the in-flight request permit for the lifetime of a streaming response, and settle spend with the usage seen so far when the task draining an abandoned stream is cancelled.
- Validate canonical package versions during release dry runs while preserving genuine prerelease checks.
- Keep public coding-agent skills synchronized with native plugin resources during version updates.
[2.2.5] - 2026-10-16
Section titled “[2.2.5] - 2026-10-16”- Send each deployment’s configured provider model upstream when proxy clients use aliases, including streaming, deployment pools, and default image/moderation models. (#286)
- Keep release dry runs from moving existing tags when republishing is requested.
- Reject whitespace-only resolved Bedrock credentials before sending requests while preserving nonblank signing bytes (#285).
- Send Gemini and Vertex structured-output schemas through
responseJsonSchemato preserve nullable fields and nested JSON Schema (#284). - Preserve Gemini model metadata and request-model fallbacks across standard and streaming responses, and resolve Gemini routing aliases for cost attribution (#283).
- Sign Bedrock requests with per-request resolved AWS credentials across standard and streaming clients, including refreshed session tokens (#285).
[2.2.4] - 2026-10-15
Section titled “[2.2.4] - 2026-10-15”Dependency refresh and regenerated polyglot bindings with Alef 0.110.13.
Changed
Section titled “Changed”- Upgrade Rust and language-package dependencies and refresh their lockfiles.
- Regenerate bindings, package scaffolding, documentation, and test suites with the latest Alef.
- Remove unused-argument lint warnings from the Hermes plugin test fixture.
- Install protobuf development headers in the shared lint runner so etcd-client builds succeed.
- Mark the native Python client as safe to share across async worker threads.
- Keep iOS builds compatible with the Flutter Rust bridge by holding libc below its incompatible Apple symbol change.
- Run mobile CI when Cargo manifests, the workspace lockfile, or Alef configuration changes.
[2.2.3] - 2026-10-10
Section titled “[2.2.3] - 2026-10-10”Reliable Go publication, smaller bundled native libraries, and static C distribution artifacts. Generated with Alef 0.107.12.
- Release catalog uploads retain explicit GitHub repository context, so the prepare gate no longer blocks every downstream package and Go module-tag job after staging files outside the checkout (#281).
- Go FFI shared libraries strip release debug information and enforce a 40 MiB size budget before packaging, preventing oversized native binaries from entering the module bundle (#282).
Changed
Section titled “Changed”- C FFI release archives include the static library and its native linker metadata through the
shared
xberg-io/actions/build-rust-ffiaction. - Removed repository-wide JavaScript lint exemptions for Unicode regexes, array indexing, and string replacement, plus the obsolete async-plugin exemption.
[2.2.2] - 2026-10-09
Section titled “[2.2.2] - 2026-10-09”Security, provider correctness, bounded Go clients, truthful catalog pricing, and repaired release assets. Generated with Alef 0.107.11.
- Go and the other generated bindings expose
ClientOptions.max_response_bytes; provider-quota exhaustion has its own stable cross-language error variant (#275, #278). - Model metadata exposes
price_known, allowing callers to distinguish unsupported or missing per-token prices from genuinely free models (#276).
- Authenticated native clients never follow redirects, preventing prompt bodies and credentials from being replayed to another origin even when outbound policy enforcement is disabled (#277).
- Empty API keys omit authentication headers, and provider error bodies redact both full bearer values and bare echoed tokens (#272, #279).
- Anthropic’s final streaming usage is cumulative and retains prompt-cache details; 529 overloads preserve their status and malformed successful responses are serialization errors (#274, #275).
- Tagged releases publish catalog checksums, macOS static archives are no longer corrupted by
strip -S, and the Go module-tag/native-asset publish path uses the corrected Alef packager (#270, #271, #273).
Changed
Section titled “Changed”- Updated Rust, Node/WASM, and generated language dependency locks.
[2.2.1] - 2026-10-09
Section titled “[2.2.1] - 2026-10-09”Provider correctness, safer transport errors, refreshed catalog metadata, and portable Go release assets. Generated with Alef 0.107.7.
- Go setup supports explicit static linking, offline native-library directories, and Linux musl targets; release builds now publish stripped musl archives (#267).
- Catalog responses expose provenance and whether model lookup was exact or used a fallback (#260, #262).
- Network failures have stable cross-language error code 109, including
ErrNetworkin Go (#258).
- Gemini and Vertex embedding requests preserve every input, requested dimensions, and response order (#268).
- OpenAI-compatible streams request usage by default, while preserving explicit caller overrides (#266).
- Anthropic uses native JSON-schema output configuration, preserves caller system-message order, reports ignored strict/seed options, and derives the default output limit from catalog metadata (#264, #265).
- OpenAI reasoning requests use
max_completion_tokens; custom endpoints retain the native Google, Vertex, Bedrock, Azure, and Anthropic transforms (#256, #264). - Retry classification handles quota exhaustion, 529 responses, 503 retry hints,
retry-after-ms, HTTP dates, and context-window error variants correctly (#257). - Provider error bodies redact configured secrets, network errors omit request URLs, and custom base URLs append paths before query strings (#259).
- Gemini and Anthropic usage maps cached, cache-creation, tool-use, and thinking tokens into the public usage fields used for pricing (#254, #255).
- Catalog refresh uses isolated atomic downloads with integrity checks, layers overlays, refreshes the embedded snapshot, and corrects model modes and embedding dimensions (#261, #262, #263).
Changed
Section titled “Changed”- Paid down the remaining tracked provider, fallback, and generated-documentation lint complexity debt (#201).
- Completed the ai-rulez integration tracker for usage, pricing, errors, routing, and distribution (#269).
[2.2.0] - 2026-10-08
Section titled “[2.2.0] - 2026-10-08”Gemini thinking-token accounting and the corresponding cross-language usage API. Generated with Alef 0.106.1.
- Usage responses expose
completion_tokens_details.reasoning_tokensin Rust and every generated language binding.
- Gemini usage includes thinking tokens. Google AI and Vertex AI responses now add
thoughtsTokenCountto completion tokens, preservetotalTokenCount, and report usage on streaming chunks (#253).
Changed
Section titled “Changed”- Lint-debt pay-down (#201): split the provider documentation generator into focused helpers and
removed its
function-too-longbaseline exemption.
[2.1.4] - 2026-10-06
Section titled “[2.1.4] - 2026-10-06”Publishes the 2.1.3 changes. The 2.1.3 publish run stopped at version validation, so 2.1.3 reached only Packagist; every other registry goes from 2.1.2 to 2.1.4. Generated with Alef 0.106.1.
- Kotlin Android: the library compiles again.
LiterLlmError.RateLimitedredeclaredretryAfter, which the sealed base now exposes as anopen valaccessor; the field is now declaredoverride. - Plugin bundles and test-app coordinates are pinned to the release version.
[2.1.3] - 2026-10-06
Section titled “[2.1.3] - 2026-10-06”Go/FFI fixes for the issues filed against 2.1.2, client-config fixes, and nine new providers. Generated with Alef 0.105.0.
- Providers: cognition, cortecs, prism, reka, sail, nadir, scx-ai, edenai and
qwen_ai_platform (174 providers total). Registry entries can now declare
aliases, solm_studio/,ollama_chat/andqwencloud/route and strip like the provider name. DashScope and Qwen AI Platform support rerank via the compatible-api route. - Cache: an
opendal:<scheme>backend value selects any OpenDAL service, includingopendal:memory. - Errors: every
LiterLlmErrorvariant has a stable error code, and reqwest timeouts map toTimeout.
- Go: typed errors. Native errors surface as
*literllm.Errorwith code, status, transient flag, error type andRetryAfter;errors.Ismatches the sentinels (#244). - Go: per-call cancellation.
ChatWithContext/EmbedWithContextcancel the in-flight native request when the context is done (#245). - Go: no more
(nil, nil). Serialisation and decode failures return an error (#246). - FFI static library links on its own. The cgo preamble declares the system libraries, and
the Go release asset ships
native-static-libs.txt(#247). macOS FFI builds pinMACOSX_DEPLOYMENT_TARGET=11.0so C objects match the Rust objects (#248). base_urlprefix stripping. A knownprovider/prefix is stripped from the model name whenbase_urlis set; unknown org prefixes such asmeta-llama/are kept (#249).- JSON client config is applied.
create_client_from_jsonnow builds the middleware stack (budget, cache, cost tracking, rate limits, hooks) and registers custom providers, which it previously parsed and ignored (#251). The budget and cache snippets configure a client in every language. - Go README documents the real release asset names, static vs dynamic linking, the current API, typed errors and the slim FFI build (#250, #252).
- Swift basic chat snippet compiles.
Changed
Section titled “Changed”- Lint-debt pay-down (#201): table-driven outbound IPv4 policy, smaller tower hedge, hooks and rate-limit functions.
[2.1.1] - 2026-09-25
Section titled “[2.1.1] - 2026-09-25”Two binding fixes from Alef 0.96.4. No Rust, Kotlin or wire-format change — 0.96.4’s third fix widens the Kotlin container-payload serializer, which this tree’s bindings already satisfied.
-
Ruby:
tool_choiceaccepts a bare string or hash again. 2.1.0 emittedToolChoiceas a native wrapped class and made that class the only accepted input, sotool_choice: 'auto'andtool_choice: { 'type' => 'function', ... }raisedTypeError: no implicit conversion of String into LiterLlm::ToolChoice. The Rust type is#[serde(untagged)], so both of those are valid representations and the binding was wrong. The wrapped instance is now tried first and anything else falls through to serde’s own reader, so the explicitLiterLlm::ToolChoice.from_mode/.from_specificconstructors introduced in 2.1.0 keep working and the pre-2.1.0 forms work again. This closes the known issue recorded against 2.1.0. -
Java: multimodal content keeps its
typediscriminator.UserContent.ofObject,AssistantContent.ofObjectandEmbeddingInput.ofObjectwrapped aListwithMAPPER.valueToTree. Java erases a collection’s element type, so Jackson resolved each element with its untyped lookup and never wrote the@JsonTypeInfodiscriminator — a list ofContentPartserialized as[{"text":...},{"image_url":...}]with no"type"key, and the request was rejected before reaching a provider. Because the tag-less tree is frozen into the wrapper, nothing downstream could recover it. Lists are now written through a writer with the element type pinned. This is not a 2.1.0 regression — it has been present since at least 1.19.0, in every release carrying the declarative@JsonTypeInforepresentation. Single values andList<String>payloads were never affected and are unchanged.
[2.1.0] - 2026-09-25
Section titled “[2.1.0] - 2026-09-25”AWS_BEARER_TOKEN_BEDROCKauthenticates the Bedrock provider. A Bedrock API key (ABSK...) is sent asAuthorization: Bearer <token>and needs no signing, so it works with thebedrockfeature off. Precedence is any explicitly configured credential field, then the token, then environment SigV4 credentials; with the feature off, the token alone (#227).
Breaking changes
Section titled “Breaking changes”All wire formats are unchanged. Every break below is source compatibility only, so an upgrade surfaces at compile or type-check time rather than at runtime.
- Node, WASM and TypeScript:
Messageis now a flat discriminated union. A message was typed{ role: "user", user: UserMessage }, nesting the payload under a key named after the role. The RustMessageis#[serde(tag = "role")], so that shape never matched what the core actually serializes. It is now{ role: "user" } & UserMessage, i.e.{ role: "user", content: "hi" }, matching both the Rust representation and the OpenAI wire format. WASM consumers additionally move from constructing a#[wasm_bindgen]class to passing a plain object with camelCase payload fields. - Java: generated enum constants are now
SCREAMING_SNAKE_CASE—AuthType.BearerbecomesAuthType.BEARER,ApiKeybecomesAPI_KEY, across 53 constants. The wire strings the constants carry are untouched. - Swift:
BudgetConfig,LlmBudgetConfig,LlmCacheConfig,LlmConfigandProviderConfigare now nativestructs with a memberwise initialiser, rather thantypealiases over the Rust bridge types. This turns a reference type into a value type. - PHP:
AuthHeaderFormatconstants become static constructors —AuthHeaderFormat::BEARERbecomesAuthHeaderFormat::bearer()— andgetAuthHeader()now returnsAuthHeaderFormatinstead ofstring. - Python: 13
Enum | strparameter unions in_internal_bindings.pyiare now the enum alone. Passing a bare string still works at runtime but is a type error. - Ruby:
AuthTypeinsig/types.rbsis now atypealias rather than a class, with ~13 accompanying retypes. An RBS/Steep break only. - Ruby: untagged enums must now be constructed explicitly.
ToolChoiceis emitted as a wrapped class, sotool_choice: 'auto'andtool_choice: { 'type' => 'function', ... }raiseTypeError: no implicit conversion of String into LiterLlm::ToolChoice. Build the value withLiterLlm::ToolChoice.from_modeor.from_specificinstead. This is a generator defect, not an intended API change — the RustToolChoiceis#[serde(untagged)], so the bare string is a valid representation, and the coercion is being restored in 2.1.1. Tracked as a known issue. - Bedrock credentials that cannot be signed are now rejected instead of sent unsigned. In a
build without the
bedrockfeature — WASM, the proxy and the CLI — explicitbedrock_credentials(...)with noAWS_BEARER_TOKEN_BEDROCKfailsvalidate()with a 401 rather than emitting a request with noAuthorizationheader and an opaque upstream 403. Note that this also rejects a configuration that previously worked: a default-feature build pointingBEDROCK_BASE_URLat a signing sidecar or mock, with credentials still in config, now fails at client construction.
Changed
Section titled “Changed”- Pin Alef 0.96.3 (from 0.85.19) and regenerate every binding. This is where the
Messagereshape comes from; it also moves the Ruby native extension to magnus 0.9, narrows the gem’s file glob so sibling packages stay out of the archive, and makes the generated Python package type-check clean under Pyrefly’sstrictpreset without adding a runtime dependency. - Upgrade every dependency to its latest release:
opentelemetry,opentelemetry-otlpandopentelemetry_sdk0.32 to 0.33 withtracing-opentelemetry0.33 to 0.34 (the four move as one set —tracing-opentelemetry0.34 requiresopentelemetry0.33 — and theotelfeature needed no source changes), utoipa 5.5 to 6.0, rmcp 3.3 to 3.4, opendal 0.59.1 to 0.59.3, jsonschema 0.56 to 0.57 (dev only), and the Java, Node, Ruby, Elixir, Python, PHP and Dart manifests — including Dart’sfreezedcode generator from 3.2 to 4.0, verified against the committed generated bindings withdart analyze. Thegetrandom0.2 and 0.3 aliases andflutter_rust_bridge2.13.0 stay pinned deliberately: the first keeps a wasm-js backend available to transitive consumers at each major, the second must matchpackages/dart. - Drop
RUSTSEC-2023-0071,RUSTSEC-2026-0194andRUSTSEC-2026-0195fromdeny.toml. None matched a crate in the dependency graph, so they disarmed the advisory gate for those IDs while appearing to be active exceptions. The two remaining ignores areunmaintainednotices for transitivenumber_prefixandpaste, not vulnerabilities. - Pay down five single-entry quality-debt baselines —
azure.rs(thanks @peteraisher, #228),tokenizer.rs,commands/mcp.rs,routes/mod.rsandclient/config_file.rs— taking the baseline from 76 findings across 50 files to 71 across 45. Behaviour is unchanged (#201). - Refresh the model catalog from models.dev: 430 model ids added, 94 removed. Removals are
user-visible, and catalog pricing feeds budget enforcement while
contextfeeds request validation.
- Ruby: the native extension compiles again. The regenerated kwargs constructors handed
Default::default()toEmbeddingRequest.input,ModerationRequest.input,OcrRequest.documentandcontenton the system, tool and user messages — all required fields whose types are mirrored into the generated crate without theDefaultimpl thatcoremarksalef(skip)— producing sixerror[E0277]s. A missing required keyword now raises instead. - Kotlin/Android: array-valued content keeps its
typediscriminator.UserContent.Parts,AssistantContent.PartsandEmbeddingInput.Multimodalserialized every element as{}, so the Rust side rejected the request withdata did not match any variant of untagged enumbefore it was sent. Only the first had test coverage; the other two were silently broken.
[2.0.3] - 2026-09-18
Section titled “[2.0.3] - 2026-09-18”OutboundPolicy::DenyPrivateExceptHosts: keepDenyPrivateDNS filtering and redirect restrictions for every public provider while letting explicitly trusted exact hosts or.suffixdomains resolve to private addresses. The same matcher applies during URL validation and during fresh or cached connection-time DNS resolution (#217).
- Redact resolved addresses from
OutboundForbiddenerrors and align the forbidden address ranges between URL validation and connection-time resolution (#217).
Security
Section titled “Security”- Bump rustls to 0.23.45 for RUSTSEC-2026-0285 (TLS 1.3 handshake messages accepted across encryption level boundaries).
[2.0.2] - 2026-09-14
Section titled “[2.0.2] - 2026-09-14”- Add optional native HTTP response limits to client builders. Enforce the limit before retaining or parsing successful and final error bodies, including decoded gzip output, while preserving unconfigured defaults and successful streaming behavior.
[2.0.1] - 2026-09-12
Section titled “[2.0.1] - 2026-09-12”Changed
Section titled “Changed”- Refresh the model catalog from models.dev (#211).
- Upgrade jsonschema to 0.56 and rmcp to 3.3, and regenerate bindings with Alef 0.85.19.
-
Allow explicitly allowlisted hostnames to resolve to internal addresses while preserving
DenyPrivateDNS filtering, exact-origin checks, and redirect restrictions. Allowlisting a hostname now trusts its DNS results, including private, loopback, and link-local addresses; literal private IP URLs remain blocked. -
Deserialize generated Python DTOs through their Rust types so serde annotations remain effective (Alef #375).
[2.0.0] - 2026-09-09
Section titled “[2.0.0] - 2026-09-09”Breaking changes
Section titled “Breaking changes”- Rust callers of
OpenDalCacheStore::newandOpenDalVectorStore::newmust pass anopendal0.59 operator instead of 0.58. Upgrade the application dependency alongside Liter. - Go consumers must use
github.com/xberg-io/liter-llm/packages/go/v2in imports and module requirements.
Changed
Section titled “Changed”- Upgrade Rust dependency requirements, including jsonschema 0.55, rmcp 3.2, and OpenDAL 0.59.1, and regenerate bindings with Alef 0.85.11.
-
Serve actually compressed gzip bodies for mock HTTP fixtures that declare gzip encoding.
-
Propagate C# native failures from fallible void calls and preserve numeric error codes in typed exceptions.
-
Narrow Java boolean native return values before comparison while retaining the compatible wide call layout.
-
Resolve the current Zig release archive and its actual content hash before running published consumer tests.
-
Render Cargo installation commands in the Rust package README.
-
Keep task lint coverage compatible with current Poly reports while rejecting formatter errors.
-
Update Ruby E2E Faraday and JSON dependencies to resolve three security advisories.
-
Synchronize Java and Kotlin test harnesses with the patched Jackson dependencies used by generated packages. Verify both Kotlin registry pins against the release version before native tests.
-
Update vulnerable JavaScript tooling and documentation dependencies, preserving scoped security overrides across regeneration.
-
Include every generated Zig streaming test in the native test build and check source-to-target coverage before running it.
-
Align PHP enum conversions with enabled core features and require the expected native exception in Ruby streaming error tests.
-
Build Swift native artifacts for the advertised macOS 13 and iOS 16 minimum versions.
-
Remove five Rust advisories by selecting the modern AWS Secrets Manager transport with Tokio runtime support and updating rust_decimal.
-
Preserve complete Python streaming requests and Ruby renamed and internally tagged fields.
-
Accept both native Elixir client references and typed wrappers in top-level client methods.
-
Make generated PHP error tests assert actual exceptions and Kotlin content tests check nonempty text.
-
Correct Swift stream and content assertions, and make generated streaming examples inspect individual chunks and release native resources.
-
Preserve Alef-owned Node declarations and loaders during native builds.
-
Remove obsolete standalone Swift bridge headers so the generated umbrella module passes strict compiler checks.
-
Resolve Java example dependencies from the Maven manifest and distinguish mocked fixture URLs from hosted samples.
-
Generate numeric C handles for file and batch list calls.
-
Correct the C# client factory and Java exception handling in documentation examples, and type-check them.
-
Synchronize the C# consumer using its actual
XbergIo.LiterLlmNuGet package reference. -
Restore Kotlin Android publishing after verifying the regenerated AAR and JNI bridge.
-
Refresh the root workspace lockfile from a current WASM build, including all six native Node packages, so frozen installation matches the release manifests.
-
Install prose tools from the standalone documentation workspace and check MDX as well as Markdown with real failure controls.
-
Refresh the Python and Rust consumer locks to published liter-llm 2.0.0.
-
Remove endpoint URLs from transport-retry warnings so URL path and query credentials do not reach application logs; retain retry counts and the transport error category.
[1.19.2] - 2026-09-03
Section titled “[1.19.2] - 2026-09-03”[1.19.1] - 2026-09-01
Section titled “[1.19.1] - 2026-09-01”- The plugin publish jobs no longer run ahead of version validation.
publish-plugin-opencodeandpublish-plugin-hermesdepended only onprepare, so they published whilevalidate-versionswas failing. In v1.19.0 that shipped an npm bundle whose manifests still declared 1.18.4, and rebuilt the Hermes wheel at 1.18.4 where the skip-existing guard reported success having published nothing for the tag. Both jobs now requirevalidate-versionsto succeed. plugin/.ai-rulez/config.tomlis back in sync with the release version. The 1.19.0 release commit bumpedCargo.tomland the alef-managed manifests but not the plugin’s own version source, so every ai-rulez-generated plugin artifact stayed at 1.18.4 andvalidate-versionsblocked the entire publish.
- v1.19.0 reached no registry except the npm plugin bundle described above; 1.19.1 is the first complete 1.19 release.
[1.19.0] - 2026-08-31
Section titled “[1.19.0] - 2026-08-31”- Scoop is now a live release channel alongside Homebrew: a release publishes a Scoop manifest for
the CLI, and the install lists and badges cover it. The channel was previously inert because
scoopwas held out of the workflow’savailable-targets– alef only gained the target after v0.79.2, and naming a target it does not recognise hard-errors the whole publish pipeline. The pinned alef is now 0.79.5, which carries it, so the gate is enabled.
- The
uv-bumpdevelopment dependency pointed at a GitHub fork that no longer exists, so resolving the dev dependency group failed outright and took the lint pipeline down with it. It now tracks the publisheduv-bumppackage on PyPI — the same tool, at 0.6.0 instead of 0.3.0. - Dependency scanning now actually runs. The
cargo denystep used a Docker container action that failed for two independent reasons: the runner pre-builds such an image at job start, but theFree disk spacestep prunes Docker images and deleted it before the step ran, and the action’s Dockerfile downloads anx86_64cargo-deny binary that cannot execute on theubuntu-24.04-armrunner anyway. No advisory, license, ban, or source check had ever been evaluated in CI since the scan was added. cargo-deny is now installed as a native binary and invoked directly. - The release announcement no longer fires for a release that shipped nothing.
announce-discordgated on!contains(needs.*.result, 'failure'), which cannot see askippedjob — andskippedis exactly what a publish job becomes when an upstream gate fails. This is the same defectrelease-finalizealready documents and fixed for itself; the announcement now depends onrelease-finalizeand gates on its verdict. CI Rust’s path filters now name the files its own gates read. Editingdeny.toml,alef.toml,rust-toolchain.toml, or.cargo/config.tomlpreviously started no run at all, so the dependency scan and the binding-freshness check could not disagree with the very configuration that drives them.
[1.18.4] - 2026-08-28
Section titled “[1.18.4] - 2026-08-28”Changed
Section titled “Changed”- Upgraded
etcd-client0.19 -> 0.20 andjsonschema0.51 -> 0.52.
[1.18.3] - 2026-08-27
Section titled “[1.18.3] - 2026-08-27”Note on 1.18.2
Section titled “Note on 1.18.2”1.18.2 was tagged and left as an unpublished Draft release; it never reached crates.io or PyPI
(both remained at 1.18.1). Its publish run aborted on the sync_plugin_version.py --expect gate,
because at that tag Cargo.toml was 1.18.2 while plugin/.ai-rulez/config.toml was still 1.18.1.
The plugin pin was corrected on main after the tag was cut, so the tag could never satisfy the
gate on a retry. 1.18.3 supersedes it and ships the corrected pin.
[1.18.2] - 2026-08-25
Section titled “[1.18.2] - 2026-08-25”Changed
Section titled “Changed”- Regenerated all language bindings on alef 0.68.0.
-
Outbound policies now cover every provider request URL, including streaming, multipart uploads, health probes, catalog refreshes, Vault, credential acquisition, and native DNS connection attempts and redirect targets. Authenticated clients reject cross-origin redirects so provider credentials cannot leak to a redirected host, while credential-free catalog downloads may follow a cross-origin redirect only after validating its target and never across an HTTPS-to-HTTP downgrade. Policy failures retain their non-retryable
OutboundForbiddenclassification, and active policies disable HTTP proxies so connection-time DNS checks cannot be delegated around the guarded resolver. Vertex ADC limits its private-network exception to the fixed, redirect-free metadata endpoint and disables the opaquegcp_authfallback under active policies; the embedded-library default remainsOff, preserving local mock endpoints. -
A skipped
publish-cratesno longer reads as a passing gate, and a release that published nothing can no longer report success. Five places in.github/workflows/publish.yamlgated downstream build, publish and release-promotion jobs onneeds.publish-crates.result != 'failure'. That expression is TRUE when the dependency was skipped, andpublish-cratesskips for two opposite reasons: the version is already on crates.io (a re-run or a resumed release, where downstream must proceed) or an upstream gate such as version validation or crate packaging failed (where downstream must not).resultalone cannot separate them, so every one of those conditions was gating on nothing – tree-sitter-language-pack v1.15.5 promoted a GitHub release toLatestwith 40+ failed jobs and every registry publish skipped, and still reported success.A new always-running
crates-gatejob resolves the ambiguity once, into an explicitoutcome(published/already-present/not-required/dry-run/blocked) and anokflag that every consumer now tests instead ofresult. Because the job always runs, its outputs always exist; because it never fails, depending on it cannot skip a consumer. The legitimate already-published path stays exactly as permissive as before, and only the gate-failed path is newly blocked.release-finalizegated finalization on!contains(needs.*.result, 'failure'), which cannot see a skipped job – so a release whose targets all silently shipped nothing was finalized out of draft and reported green. It now verifies every enabled target individually before flipping the draft, and holds the release as a draft and fails when an enabled, not-yet-published target did not succeed.One of the five sites,
publish-homebrew-formula, had the mirror-image defect: the job carries noalways()or!cancelled(), so GitHub skipped it outright wheneverpublish-crateswas skipped and itsif:never ran at all. The formula therefore went unupdated on exactly the re-runs it was meant to cover. Depending oncrates-gate– which always succeeds – lets the condition be evaluated, so the formula now updates on a resumed release and is still held back when the crates.io prerequisite was never met. -
A drifted coding-agent plugin now fails the release instead of shipping.
CI Pluginalready re-ran onCargo.toml, so--checkcaught drift onmain— but nothing asserted the plugin version against the tag actually being published.scripts/sync_plugin_version.pygains--expect <version>, which requires core andplugin/.ai-rulez/config.toml[plugin].versionto both equal the version being released, andpublish.yaml’svalidate-versionsjob runs it againstneeds.prepare.outputs.version..task/tools/version-sync.ymljoins theci-plugin.yamlpath filter so edits to the sync task re-run the gate as well.
Changed
Section titled “Changed”-
Repinned
alef.tomlalef_versionfrom0.67.2to0.67.5and regenerated. Behaviour picked up from the three upstream releases:- Swift e2e assertions now decide a leaf’s shape from the bridged getter rather than the accessor
text, so
tool_callsassertions that swift-bridge JSON-bridges toRustStringare emitted as an explicit skip instead of an assertion that could not compile against the real binding. - Zig e2e assertions no longer wrap expected strings in
std.mem.trim, which silently masked leading and trailing whitespace differences. - Java FFI wrappers rethrow
LiterLlmRsExceptioninstead of re-wrapping it in a secondLiterLlmRsException("FFI call failed"), so the original typed error survives the call boundary. poly.tomladdsMD025to the markdown formatter’s disable list.
- Swift e2e assertions now decide a leaf’s shape from the bridged getter rather than the accessor
text, so
-
alef.toml[crates.exclude].functionsnow excludesconfigure_outbound_client_builder. It takes and returns areqwest::ClientBuilder, which has no binding representation, so alef sanitized both toStringand failed extraction. It is a Rust-only escape hatch and is excluded the same way its outbound-policy siblings (set_outbound_policy,current_policy,validate_outbound_url) already were. -
Repinned
alef.tomlalef_versionfrom0.66.0to0.67.2and regenerated. Behaviour picked up from the two upstream releases:crates/liter-llm-ffi/build.rs: C header generation is now opt-in behindALEF_EXPORT_GENERATED_HEADERS=1and writes bothcrates/liter-llm-ffi/include/liter_llm.handpackages/go/include/liter_llm.hatomically with rollback. A plaincargo build -p liter-llm-ffino longer rewrites either header, so ordinary builds stop mutating committed source; regenerating with the flag set reproduces the committed headers byte for byte.alef verifyis the freshness gate now.packages/dart: flutter_rust_bridge moved from 2.12.0 to 2.13.0 (pubspec.yaml,packages/dart/rust/Cargo.toml,Cargo.lock, regenerated bridge). The generated bridge now allowsmismatched_lifetime_syntaxesand fully qualifiesstd::result::Result::Ok.packages/dart/rust/build.rspasses--no-deps-checktoflutter_rust_bridge_codegenand only re-applies the FRB cfg gates after a regeneration — the committed bridge already carries them, so the previous unconditional repair on every build was redundant.docs-site/src/snippets: 1,691 snippets regenerated. The generator now prints a field of the result instead of the whole value (System.out.println(result.data())rather thanSystem.out.println(result)). Where the fixture’s assertion path is not a field of the returned type —is_trueoncost_tracked,equalsonerror.status_code, oraudioon a response the binding flattens to a byte array — the emitted accessor does not exist and the snippet no longer typechecks. Verified against built packages: 17 C# and 78 Java snippets regress this way. Tracked upstream against alef; these snippets are generated and cannot be fixed in this repo.
[1.18.1] - 2026-08-22
Section titled “[1.18.1] - 2026-08-22”-
packages/ruby/sig/types.rbs: thefine-tuneenum value was emitted as a bare symbol,:fine-tune, which is not valid RBS — the hyphen terminates an unquoted symbol literal, so the declaration failed to parse and steep could not type-check the Ruby package. It is now emitted quoted,:"fine-tune", andpoly lint’s steep check passes again. Fixed upstream in alef 0.64.0 and picked up here by regenerating the bindings. -
.task/tools/docs.yml:docs:snippets:validateanddocs:snippets:validate:langinvokedalef snippets validate, a subcommand that does not exist (alef snippetshas onlylist,check,parse,audit,gaps), so both targets failed immediately witherror: unrecognized subcommand 'validate'. Both now callalef snippets check --strict, with the per-language target using--lang. The canonicaldocs:snippets:allgate was never affected — it already calledcheckdirectly. The removedLEVELvariable has no replacement: validation level is config-owned (validation_level = "typecheck"), so per-language runs now typecheck instead of defaulting to syntax. -
.github/workflows/publish.yaml: thedry_runworkflow_dispatch input declareddefault: "false"(a quoted string) againsttype: boolean, which made the wholeon:block fail GitHub’s workflow schema (gau --checkreportedschema validation failed at /on: ... is not valid under any of the schemas listed in the 'oneOf' keyword). Its sibling inputsforce_republishandrepublishalready used an unquotedfalse. No job behaviour changes: everyif:in the workflow gates on the normalized string outputssteps.meta.outputs.dry_run/needs.prepare.outputs.dry_run, never oninputs.dry_run.
Removed
Section titled “Removed”-
Retired 18 of 20
legacy_python_*hand-maintained doc snippets underdocs-site/src/snippets/pythonwhose API surface (basic/streaming/tool-calling chat, multi-turn, configuration, embeddings, batches, files, responses, image generation, local providers, moderation, OCR, rerank, search, speech, transcription) is already covered by alef-generated fixture snippets. Doc pages underdocs-site/src/content/docs/usage/*.mdxnow import the equivalent generated fixtures instead.legacy_python_guides_error_handlingandlegacy_python_multimodalare kept: they demonstrate per-exception-type branching and multimodal/audio/document/structured-output patterns that the generated fixtures do not currently express. -
Retired 18 of 19
legacy_csharp_*hand-maintained doc snippets underdocs-site/src/snippets/csharpcovering the same topic set as the Python legacy snippets (the C# tree has nomultimodallegacy file).legacy_csharp_guides_error_handlingis kept for the same reason as its Python counterpart. -
Retired 18 of 19
legacy_dart_*hand-maintained doc snippets underdocs-site/src/snippets/dart, same topic set and same triage as csharp/python.legacy_dart_guides_error_handlingis kept. These snippets were only referenced fromreference/legacy-snippet-catalog.mdx; that page now points at the equivalent generated fixtures instead. -
Retired 18 of 19
legacy_elixir_*hand-maintained doc snippets underdocs-site/src/snippets/elixir, same triage as csharp.legacy_elixir_guides_error_handlingis kept. -
Retired 18 of 20
legacy_go_*hand-maintained doc snippets underdocs-site/src/snippets/go, same triage as python.legacy_go_guides_error_handlingandlegacy_go_usage_multimodalare kept. -
Retired 18 of 19
legacy_java_*hand-maintained doc snippets underdocs-site/src/snippets/java, same triage as csharp.legacy_java_guides_error_handlingis kept. -
Retired 18 of 19
legacy_kotlin_*hand-maintained doc snippets underdocs-site/src/snippets/kotlin. Alef’s generated fixtures for this target live underdocs-site/src/snippets/kotlin-android(the crate key iskotlin_android, notkotlin); the legacy-snippet-catalog page now points there.legacy_kotlin_guides_error_handlingis kept. -
Retired 18 of 20
legacy_php_*hand-maintained doc snippets underdocs-site/src/snippets/php, same triage as python/go.legacy_php_guides_error_handlingandlegacy_php_usage_multimodalare kept. -
Retired 18 of 19
legacy_ruby_*hand-maintained doc snippets underdocs-site/src/snippets/ruby, same triage as csharp.legacy_ruby_guides_error_handlingis kept. -
Retired 18 of 19
legacy_rust_*hand-maintained doc snippets underdocs-site/src/snippets/rust, same triage as csharp.legacy_rust_guides_error_handlingis kept. -
Retired 18 of 19
legacy_swift_*hand-maintained doc snippets underdocs-site/src/snippets/swift, same topic set and same triage as dart (catalog-only references).legacy_swift_guides_error_handlingis kept. -
Retired 18 of 19
legacy_typescript_*hand-maintained doc snippets underdocs-site/src/snippets/typescript, same triage as python.legacy_typescript_guides_error_handlingis kept. -
Retired 18 of 19
legacy_wasm_*hand-maintained doc snippets underdocs-site/src/snippets/wasm, same triage as csharp.legacy_wasm_guides_error_handlingis kept. -
Retired 18 of 19
legacy_zig_*hand-maintained doc snippets underdocs-site/src/snippets/zig, same topic set and same triage as dart/swift (catalog-only references).legacy_zig_guides_error_handlingis kept.This completes the doc-snippet triage across all 14 languages that carried
legacy_*snippets (269 files total): 252 retired as superseded by alef-generated fixtures, 17 kept (guides_error_handlingin every language, plususage_multimodalin go/php andpython_multimodalin python) because they demonstrate patterns — per-exception-type branching, and multimodal/audio/document/structured-output/typed-constructor usage — that the generated fixtures do not currently express.
[1.18.0] - 2026-08-22
Section titled “[1.18.0] - 2026-08-22”- Multimodal embedding inputs can combine tagged text, image URL, and base64 image parts. Self-hosted
embedding providers forward the complete input, and vector metadata can retain an image payload
and convert it directly to
ContentPart::ImageUrlfor multimodal RAG. - A queueing
InFlightLimitLayerand managed-client configuration now bound simultaneous provider requests globally per client. Cache hits bypass the limiter,Noneremains unlimited, and zero is rejected as invalid configuration.
Changed
Section titled “Changed”- Breaking:
EmbeddingProvider::embednow accepts&EmbeddingInputinstead of&str, andVectorMetadataadds animage_urlfield. Custom provider implementations and metadata struct literals must be updated. Built-in Bedrock, Google AI, and Vertex AI embedding adapters reject multimodal inputs explicitly; use a compatible custom or self-hosted endpoint for image embeddings.
[1.17.3] - 2026-08-21
Section titled “[1.17.3] - 2026-08-21”- The release actually publishes. v1.17.2 was tagged and released but published nothing to any
registry: the
Validate versionsgate failed on staleCargo.lockfiles undere2e/rustandpackages/ruby/ext/liter_llm_rb/native, which skipped the crates.io publish job. Every language-package build behind it then failed withfailed to select a version for the requirement ^1.17.2, becausealef publish preparewas retrying against a registry version that had never been pushed. The lockfiles are refreshed and the gate passes. Note that Packagist did ingest the 1.17.2 tag through its own webhook, independently of CI, so PHP is the one ecosystem where 1.17.2 resolves; every other ecosystem is still on 1.17.1. Use this version instead.
[1.17.2] - 2026-08-21
Section titled “[1.17.2] - 2026-08-21”-
Generated bindings regenerated on alef 0.62.8. This carries alef’s fixes for JNI and FFI casts to a value’s own type (which tripped
clippy::unnecessary_castin the JNI crate, linted with-D warnings), a deadjava.util.Listimport, and akotlin.test.assertNotNullimport emitted where nothing referenced it. -
packages/ruby/ext/liter_llm_rb/native/Cargo.tomlande2e/rust/Cargo.tomlnow follow the project version. Both are alef-owned but were never reached by the version sync, so each release left them pinned to the previous version. -
The publish workflow pins
xberg-io/actionsat@v1rather than@main, so it no longer tracks untagged changes to that repository.
Security
Section titled “Security”h2advanced to 0.4.18, resolving RUSTSEC-2026-0258 (unbounded empty DATA frames: a peer could queue empty frames without limit, risking unbounded memory use or a panic on length overflow). Low severity.
Changed
Section titled “Changed”-
All Rust dependencies taken to their latest versions (
cargo upgrade --incompatiblefollowed bycargo update):jsonschema0.49 to 0.50 andrand0.9 to 0.10 as direct dependencies, plus 105 transitive bumps and two new packages. Nothing was downgraded.rand0.10 required source changes: theos_rngfeature is nowsys_rng,small_rngis gone (SmallRngcompiles unconditionally),SmallRng::from_os_rng()becomesrand::make_rng::<SmallRng>(), andRng::random_rangemoved to a newRngExttrait. -
alef pinned to 0.62.8.
Known issues
Section titled “Known issues”- The Dart package does not analyze cleanly.
packages/dart/lib/src/liter_llm.dartcallscountTokens,countRequestTokensandrecordCostUsd, which the committed flutter_rust_bridge bridge does not declare, sodart analyzereports threeundefined_functionerrors. The three functions are declared in the bridge facade; the bridge generator emits them only when invoked as acargobuild-script subprocess and not when invoked as a bare CLI, which is how alef calls it. Tracked upstream; other language bindings are unaffected.
[1.17.1] - 2026-08-15
Section titled “[1.17.1] - 2026-08-15”Changed
Section titled “Changed”-
wasm/typescript:
#[serde(untagged)]data enums now generate real structural TypeScript types instead ofany. Six types change shape in the generated.d.ts:ModerationInputandEmbeddingInput(string | string[]),StopSequence(string | string[]),RerankDocument(string | { text: string; }),UserContent(string | any[]), andToolChoice("auto" | "required" | "none" | WasmSpecificToolChoice). TypeScript consumers may see new compile errors at call sites that previously typechecked only becauseanyhid a genuine mismatch between the value passed and the shape the binding actually accepts. These are pre-existing bugs the old typing concealed, not new runtime restrictions — the accepted runtime values are unchanged.AssistantContentis deliberately exempt viauntagged_union_text_typesand continues to bridge as a plainstringon its field, getter, and setter. -
catalog: refreshed the bundled model catalog (
schemas/catalog.json) from models.dev. The catalog is embedded viainclude_str!and backs pricing and capability lookups incost.rs. Providers went from 184 to 185 and models from 6293 to 6321: 70 models added, 42 removed, and no existing model repriced — input and output costs are byte-identical for every model present both before and after. The single new provider iscrusoe.The removals are the upgrade risk, since a removed identifier no longer resolves for cost lookup. 36 of the 42 are under
cloudflare-ai-gateway, and they are genuine deletions rather than renames — nothing was added back under those names. Notably every Anthropic model behind that gateway is gone (claude-3-haiku,claude-3-sonnet,claude-3-opus,claude-3.5-haiku,claude-3-5-haiku,claude-3.5-sonnet,claude-sonnet-4,claude-opus-4,claude-opus-4-1), along with 24workers-aimodels,openai/gpt-5.1-codex,openai/gpt-5.2-codex, andmoonshotai/kimi-k3. The remaining six removals areinclusionai/ling-3.0-tinyvariants acrosskilo,nano-gpt,openrouter, andvercel, plusnano-gpt/TEE/minimax-m2.5. If you route Anthropic models throughcloudflare-ai-gateway, cost lookup for those ids will no longer resolve after this upgrade.
- client: stop re-inserting
streaminto request bodies that providers rebuilt without it.prepare_requestset the transport-controlledstreamflag beforetransform_request, then restored it after theextra_bodymerge soextra_bodycould not desync the wire flag from the chosen transport. That restore ran for every provider, including those whose native wire format has nostreamfield at all. Vertex/Gemini rebuilds the body intogenerateContentshape and deliberately drops the key, so a plain non-streaming call shipped"stream": falseand Vertex rejected the entire request with HTTP 400Invalid JSON payload received. Unknown name "stream": Cannot find field.— breaking every non-streaming chat completion against Vertex AI, including vision and structured-extraction calls, which surfaced only as a silently absent result. The restore is now scoped to providers whose body still carriedstreamafter their own transform, so OpenAI-compatible, Anthropic, and Cohere bodies keep the flag while Vertex and Bedrock — which select streaming by endpoint — no longer receive a field their native request formats do not define. - version: stop the per-language binding crates from drifting behind the core workspace
version, so a binding crate published from this release carries the same version as the
liter-llmcore it was built against. - docs: label the MCP HTTP transport snippet as
bashinstead oftomlso it renders with the correct syntax highlighting.
[1.17.0] - 2026-08-13
Section titled “[1.17.0] - 2026-08-13”Upgrade notes
Section titled “Upgrade notes”-
tenant_idno longer defaults to the virtual key token — this resets budget spend and cache scope for every unconfigured key. The key token was silently doubling as the tenant identifier, which meant a live credential flowed into the OTLPgen_ai.budget.tenant_idattribute, the budget-ledger CSV chargeback export, tenant-scoped cache keys at rest, andUsageEventlogs. That could not be preserved as the default — it was the vulnerability. Virtual keys now take an optionaltenant_idfield; if left unset, a new tenant id is derived instead of reusing the key. On deploy this means: month-to-date budget spend resets to zero for every key that does not settenant_idexplicitly, and that key’s existing tenant-scoped cache entries become orphaned (unreachable, not deleted).If you need billing continuity across this upgrade, set
tenant_idto the old key value for every affected virtual key in the same deploy, before traffic hits the new build.Understand what that costs before you do it. Setting
tenant_idto the key token puts the live credential back into every sink listed above — the OTLP attribute, the chargeback export, cache keys at rest andUsageEventlogs. None of those sinks were hardened by this release; only the default changed. The code deliberately printstenant_idin plain text (inKeyContext’sDebug, for one) precisely because it is no longer expected to be a secret, so this remedy reinstates the exposure the release exists to remove. Treat it as a temporary migration step: carry spend across the deploy, then rotate the affected keys and drop the explicittenant_id. If you can absorb a spend reset instead, that is the safer choice.Cache continuity is not available by any route — see the next note. Every entry is unreachable after this deploy regardless of what
tenant_idis set to, so do not accept the credential-reexposure cost expecting to keep a warm cache. -
Deploying this release invalidates every existing cache entry. The cache key now folds in
tenant_idandsystem_prompt(both previously hard-coded toNone) and additionally hashestools,response_format,seed,presence_penalty,frequency_penalty,logit_bias,tool_choice,parallel_tool_calls,reasoning_effort,modalitiesandextra_body. Old keys become unreachable the moment this deploys — there is no error and no warning, just a cold cache and a burst of upstream traffic and spend until it refills. This is intended: the old keys were the defect. Expect the cold-start cost and do not read the traffic spike as a regression. -
Wire format: chat responses now always include
role,logprobsandrefusal. OpenAI’s schema requires all three onchoices[]/choices[].message; a strict client deserializing a response from this proxy previously failed on the missing keys.refusalis now serialized even whennull, which as a side effect also makes it appear on request messages you send — that is valid per OpenAI’s schema, but if you diff request payloads byte-for-byte, expect the new key. -
API: exhaustive struct literals against
ChoiceorCreateResponseRequestwill not compile.Choicegainedlogprobs;CreateResponseRequestgainedextra_body. -
API:
CacheDecisionlosesstale_while_revalidateanduse_streaming_replay. Both were public, documented, and read nowhere — an operator could set either and believe it took effect.stale_while_revalidatenever served a stale entry, and the “opt-in via policy” streaming replay did nothing.semantic_ttlstays, but its doc no longer implies a separate duration: a semantic hit resolves to the same physical entry, which has exactly one TTL. -
API:
HttpProbeHealthChecker::newno longer takes a timeout parameter. See the health-gate entry under Fixed. -
API:
GuardrailServiceandHooksServicegained anS: Clonebound. The bound is on theServiceimpl, not the struct, so it breaks composition rather than construction: a stack wrapping an inner service that is notCloneno longer compiles. Both needed it to defer the inner call until after their own pre-flight decision.
Security
Section titled “Security”- Singleflight deduplication ignored tenant identity, so one tenant could be served another’s
response.
singleflight_keyhashed only the inner chat/embedding request;tenant_idlives on theLlmRequestenvelope, whoseSerializeforwards to that inner value alone, so tenant could not reach the hasher even indirectly — while the exact and semantic cache tiers, already fixed for this, read it explicitly. Two virtual keys posting byte-identical bodies inside one in-flight window collapsed to a single upstream call, and the follower skipped the budget, cost and guardrail layers entirely: its spend went unrecorded and its cap unenforced, while it was reported as a cache hit on a call made for someone else. Only reachable if you composeSingleflightLayeryourself; no in-tree stack did. - The proxy ran no guardrails at all. The trait, the stage enum and the whole apply-path shipped, but nothing populated a guardrail set — see “Added” for the config surface that fixes it.
- The virtual key token was echoed into client-visible 403 error bodies from six call sites,
across
require_master, guardrail rejections and other proxy error paths — the live credential landed in the caller’s own logs plus every access log, reverse proxy and error tracker on the response path.Debugoutput onKeyContexthad the same problem: bothkey_idandtenant_idprinted verbatim into any captured span or panic message. Both are now redacted; error bodies andDebugreport a stable, non-reversible correlation id instead. Two tests that had pinned the leak by asserting the raw token was present now assert the opposite. tenant_idwas the virtual key token itself, and it flowed into weakly-protected sinks with no rotation-linked purge: the OTLPgen_ai.budget.tenant_idmetric attribute, the budget-ledger CSV chargeback export, tenant-scoped cache entries at rest, andUsageEventusage logs. See “Upgrade notes” above — this is now fixed, and the fix changes tenant identity for unconfigured keys.- An empty master key was a full authentication bypass, not a weak credential.
Authorization: Bearer(empty, trailing space) resolved toKeyContext::master()— full read/delete access to every tenant’s files, batches and responses — because unset env-var interpolation yields"", and the constant-time comparison treated two empty byte slices as equal. Config load now refuses a master or virtual key that interpolates to empty, and the comparison itself rejects empty operands. Virtual keys had the same hazard and are fixed the same way. - The realtime proxy documented guardrail enforcement it has never performed. The module doc
stated as fact that client → upstream messages are checked at
GuardrailStage::Inputand upstream → client atGuardrailStage::OutputChunk. The machinery to do so exists and is correct, but the only production call site passes an empty guardrail set, and there is no proxy config field, noAppStatefield and no other caller that populates one — so both checks iterate nothing and always allow. No behaviour changed here; what changed is that the docs no longer assert a control that is not running. If you relied on realtime sessions being moderated, they were not. Wiring a real config surface for this is tracked separately. - MCP tool calls and realtime sessions bypassed per-key rate limits and budgets entirely.
Model-routed MCP calls never attached
tenant_idto the Tower stack, and a missingtenant_idis treated as intentionally unlimited; realtime never entered the Tower stack at all. Both are now routed through the same rpm/tpm/budget enforcement a unary HTTP call gets — realtime is necessarily pre-flight-only, since a live session has no discrete request to meter. server.request_timeout_secswas parsed from config and had no effect. It is now applied to unary routes, bounding an upstream that never responds. (Realtime and already-flowing stream bodies are not covered — see the commit notes for why.)- CEL guardrail expressions could exhaust the native stack through an unparenthesized operator chain
(
"!!!!...true", a long&&chain) that the existing bracket-depth cap could not see, since it counts only( [ {. Operator count is now bounded too. Not a confirmed exploit — defence in depth alongside the existing depth cap. - Proxy: any valid virtual key could read, download and delete another tenant’s files, batches and
responses. The nine REST handlers in
routes/files.rsandroutes/batches.rsboundKeyContextas_key_ctxand discarded it. These IDs are opaque identifiers on a single shared upstream provider account with no per-tenant ownership record in the proxy, so they are now scoped to master keys — matching what the MCP layer already enforced for the same twelve operations. - Proxy: revoked virtual keys and rotated master keys kept working until restart. The config
watcher swapped only the config
Arc;KeyStoreis now built once, shared with the watcher, and reloaded on everyPut/Resyncevent. - Cache: enabling the semantic tier silently defeated tenant isolation. The exact tier folded
tenant_idinto its key, but the semantic tier wrote every entry withtenant_id: NoneandVectorStore::searchtook no tenant argument, so it could not have filtered even if the metadata were populated — tenant B asking a semantically similar question was served tenant A’s cached response.searchnow takes a tenant and both backends filter on it.Nonematches onlyNone; treating it as a wildcard would reopen the hole it closes. - Bedrock: a SigV4 signing failure sent the request to AWS unauthenticated.
signing_headerscalledsigv4_sign(...).unwrap_or_default(), so failure yielded empty headers. Credentials are now validated on every request and fail hard before any network I/O. - Credentials appeared in
{:?}output.LlmConfigandBedrockConfigderivedDebugoverapi_key,access_key_id,secret_access_keyandsession_token, so any debug formatting — a tracing event, a panic message — printed live credentials. Both now redact, matching the existingClientConfigconvention. Proxy config structs holding virtual keys and provider credentials are redacted likewise. - Output guardrails did nothing for streamed responses.
GuardrailStage::OutputChunkwas defined and documented but never invoked, so a guardrail that blocks a phrase had no effect once the caller streamed. Chunks now pass through it, and a blocked chunk terminates the stream: bytes already sent cannot be recalled, but nothing further reaches the caller. - CEL guardrail expressions reached the parser with no bound on length or nesting depth. A stack
overflow aborts the process and
catch_unwindcannot catch it, so one malicious rule string could take the process down. Expressions are now validated before compilation. The abort was never reproduced — this is defence in depth, not a confirmed exploit. - A panicking guardrail poisoned the global registry lock and permanently disabled guardrail enforcement for the rest of the process; the guard is now recovered with a warning.
- The CLI now warns when a master key is passed as a command-line argument, where it is visible in the process table.
Changed
Section titled “Changed”- Chat responses now always serialize
role(onchoices[].message),logprobs(onchoices[]) andrefusal(present even whennull) to satisfy OpenAI’s response schema. See “Upgrade notes.” Choicegained alogprobsfield;CreateResponseRequestgained anextra_bodyfield;GuardrailServiceandHooksServiceboth gained anS: Clonebound. See “Upgrade notes.”DenyListGuardrailnever blocked anything, andAllowListGuardrailwould have blocked everything. Both decide onGuardrailContext::metadata, and nothing populated it per call —GuardrailServicepassed only the static per-layer map, andLlmRequest::tenant_idwas never copied in. A deny-list ontenant_idtherefore readNone, which that guardrail treats as “nothing to deny”, so a configured and documented access control silently allowed every request; an allow-list readNoneas “required field absent” and would have blocked all traffic. CEL expressions referencingmetadata.tenant_idwere broken the same way.tenant_idis now merged into the per-call context. Static per-layer metadata still wins on a key collision, with a warning, so an operator’s explicit value is never silently overwritten.- A hook-rejected request no longer consumes rate-limit and budget accounting.
HooksServicecalled the inner service before running itson_requesthooks, so a request the hooks rejected had already executed every inner layer’s synchronous body — andModelRateLimitServiceincrements its RPM counter there. A burst of hook-rejected traffic therefore exhausted the RPM window and started rejecting legitimate requests, none of which had reached a provider. No upstream call was ever made, so this cost bookkeeping rather than money, but it is a self-DoS any client could trigger by repeatedly sending content a guardrail hook blocks. The inner call now happens after the hooks pass. The existing short-circuit test could not catch this: its mock counted inside the async body, which is never polled either way. - An Azure-routed Responses call sent a malformed URL instead of failing. All four
ResponseClientmethods pass an empty model tobuild_url— none of create/retrieve/cancel has one to give — and Azure embeds the model in the path unconditionally, so the request went out to.../openai/deployments//responsesand relied on Azure’s 404. It now returnsEndpointNotSupportedbefore any network call. The check tests the URL shape rather than the provider name, so it cannot reject OpenAI-compatible gateways running under a different provider name, and will catch any future provider with the same URL-embedding pattern. Bedrock, Vertex and Google AI were never affected — contrary to what theResponseClientdoc claimed, they fall through to the plain{base}{path}branch; that doc is corrected too. top_pnever reached Cohere at any value. Cohere’s v2/chatendpoint has notop_pfield — its nucleus-sampling parameter is namedp— so the value was forwarded under a name Cohere does not recognise and silently dropped, leaving nucleus sampling entirely unset on every Cohere request that asked for it. It is now renamed topon the way out and range-checked against Cohere’s documented[0.01, 0.99]. Note the floor is0.01, not0.0:top_p: 0.0is legal for OpenAI and was legal per this crate’s own docs, and is now rejected for Cohere rather than quietly coerced. The error namestop_p, the field you set, not Cohere’s internalp.- A single failed health probe took the whole service offline. The global gate stored
result.is_ok()from oneListModelsprobe, so any failure rejected all traffic with 503 until the next tick — and a provider that does not implementListModelsreturnsEndpointNotSupportedand was taken down permanently for a capability gap, while ordinary chat through it would have worked. The gate now uses the same consecutive-failure thresholds the per-provider checker already had (3 to open, 2 to close), andEndpointNotSupportedis not counted at all. Set both thresholds to1for the old behaviour.HealthCheckConfig::timeoutwas also applied to nothing, so a stalled checker froze the probe loop and left a dead provider marked healthy forever; it now bounds every probe. Breaking:HttpProbeHealthChecker::newno longer takes a timeout — two independent deadlines for one probe is worse than one. - Every cached error replayed as a 500. The negative cache rewrote each error to
InternalError, so a cached 429 came back without itsRetry-Afterand withis_transient()false — clients stopped retrying. The variant is now preserved on both the write and read paths. Fixing only that would have been worse than the bug: replays re-entered the write path and pushed the expiry out every time, which was masked solely because the replayed error was non-transient. The window is now set once, by the call that saw the real failure, so steady polling can no longer keep an entry alive forever. - The OpenDAL cache backend silently discarded every configured TTL — it never overrode
set_ttl, so the trait’s no-op ran and entries kept the store’s construction-time TTL regardless of config or per-model policy. Both existing tests named for this defect used the in-memory store, which does override it. Error entries are also no longer logged at WARN on this backend, which previously produced one warning per failed request during exactly the outage the negative cache exists for. - Metrics mislabelled everything after the first request for a model. The attribute cache was
keyed on
(system, model)but also cachedgen_ai.response.modelandgen_ai.operation.name, so one early failure fixed both for good — and requests with no model (ListModels, and image or moderation calls without one) all collapsed onto a single key and inherited whichever operation arrived first. Token usage also dropped every streamed request while cost did not, so a streaming deployment reported spend against zero tokens.gen_ai.systemwas empty on all cache-tier metrics, making hit rate impossible to break down by provider. temperatureabove a provider’s documented cap is now rejected locally instead of forwarded.ChatCompletionRequestdocumentedtemperatureas[0.0, 2.0]— OpenAI’s range — while Anthropic and Amazon Bedrock both cap it at1.0, so a value legal per our own API docs failed at the provider after a network round trip. Those two now return aBadRequestnaming the field, the value and the provider’s range. The same outcome as before, just earlier and legible; nothing that previously succeeded now fails. Deliberately not clamped — the parameter is supported and only the scale differs, so clamping would silently rewrite the caller’s request. Enforcement applies only where the provider’s own schema states a bound: Cohere documentstemperatureas a non-negative float with no maximum, Google’s docs contradict each other on the ceiling, and Mistral publishes a recommendation rather than a bound, so those are left forwarded unchecked.- The Responses API is documented as OpenAI-only. No behaviour changed, but the docs previously
implied otherwise —
parse_response_stream_eventdescribed the event shape as “uniform across providers that support it”, a set that does not exist: the provider registry models noresponsesendpoint for any of its 165 providers, no provider overridesresponses_path(), and neithertransform_requestnortransform_responseruns on this path. Aprovider/modelprefix onCreateResponseRequest.modelis not stripped and does not re-route. If you point this path at a non-OpenAI provider today, it does not work and is not expected to.
-
Cohere streaming was entirely non-functional. The parser matched Cohere’s legacy v1 NDJSON event names and field paths against the v2 endpoint the provider actually targets, so a stream opened cleanly and then yielded empty text, empty tool calls, and no
finish_reasonor usage — ever. All twelve existing streaming tests encoded the v1 shapes and asserted the broken behaviour; rewritten against the real v2 wire format. -
Every non-streaming Cohere completion failed deserialization. Cohere’s v2 chat response has no
choiceswrapper —id,finish_reason,messageandusageare top level — sotransform_responsewas a no-op on real payloads. The old tests passed only because they fed the wrong shape as input. -
Gemini streaming hit the non-streaming endpoint and failed with a misleading error.
build_stream_urlreused the non-streaming:generateContentURL with?alt=sseappended; Gemini ignoresalt=ssethere and returns one ordinary JSON body, which surfaced as “SSE stream truncated” with no content. Both Google AI Studio and Vertex AI now call:streamGenerateContent. -
Bedrock embedding requests were silently discarded. Bedrock has no unified embeddings API, so embedding calls fell through to the Converse (chat) transform, which rebuilds the body and dropped the input entirely. Titan and Cohere embedding shapes are now dispatched on model prefix and their responses normalised to OpenAI’s
data[]list form; a batched Titan input errors rather than silently truncating to one vector, and an unrecognised embedding model errors instead of guessing. -
Vertex, Google AI and Bedrock silently dropped documented OpenAI request fields (
logprobs/top_logprobs,service_tier,metadata, and others) because those providers rebuild the request body wholesale. Fields with a real equivalent are now mapped (e.g.logprobs/top_logprobs→ Gemini’sresponseLogprobs/logprobs); anything without one now warns instead of vanishing. Anthropic had the opposite defect — it mutates the body in place, so seven unmapped fields reached its wire verbatim, and a caller’smetadatacollided with Anthropic’s ownmetadata.user_id. Anthropic now strips what it doesn’t support. -
Setting
modalitiesorseedon an Anthropic request failed the entire call with a 400. Two fields were still missing from Anthropic’s strip list, and the Messages API rejects any unrecognised top-level key with"Extra inputs are not permitted"rather than ignoring it — so these did not degrade the request, they broke it, includingmodalities: ["text"], which is a no-op for a provider that only emits text. Both are now stripped;seedwarns, since a caller pinning it for reproducibility gets none. -
The
cohereandcohere_chatregistry entries pointed atapi.cohere.aiwhile the provider itself usedapi.cohere.com. Both hosts are live aliases so nothing failed, but anything reading the registry to locate Cohere disagreed with the client that sends the request. Both now sayapi.cohere.com, and a test pins them together. -
The proxy rejected documented OpenAI chat request fields with a hard 400 instead of ignoring them:
logprobs,top_logprobs,max_completion_tokens,service_tier,store,metadata,prediction,audio,web_search_options. All are now accepted and forwarded to the provider. -
The Responses API had no escape hatch for provider extensions.
CreateResponseRequesthad noextra_body, and because the type denies unknown fields, a caller-suppliedextra_bodywas rejected outright — for example, there was no way to reach OpenAI’sreasoning.efforton this path.extra_bodyis now merged on both the streaming and non-streaming Responses paths, reusing the same merge the chat path already used (GH #174, reported by @huangmiuXyz). -
Guardrail
Mutatedecisions were applied at theOutputChunkstreaming stage but silently ignored at theInputandOutputstages — a redaction guardrail (including the built-inRegexGuardrailin Redact mode) sent the provider the unredacted content and returned the unredacted response to the caller. Both stages now apply the mutated payload, and fail closed: if a mutated payload cannot be applied, the call aborts rather than silently forwarding the original. -
The circuit breaker could be force-closed by a stale in-flight request. A request that started while the circuit was
Closedand completed after other requests had since tripped it toOpenunconditionally reset the breaker toClosedand zeroed the failure count on success, sending traffic back at a backend that was still down. Closing now requires aHalfOpen -> Closedtransition; a success recorded against an already-Opencircuit is ignored. -
A realtime session hanging up could shut down the entire proxy. The handler passed the process-wide shutdown token straight into the relay loop, so a normal session close cancelled the same token graceful shutdown awaits — any client could trigger it deliberately, and a legitimate user closing a call could trigger it by accident. Sessions now cancel a child token instead.
-
Streamed requests that a caller abandoned early recorded no spend, even though the full completion is generated and buffered server-side before the first byte reaches the caller — opening and dropping streams in a loop was unmetered usage against a budget-limited endpoint. Spend now also settles on stream
Drop. -
Several in-memory maps had no bound and were reachable with attacker-controlled keys, each a memory-exhaustion vector in a long-running proxy: the per-key idempotency store (also fixed to hash bodies deterministically across restarts and replicas, rather than per-process-random), the budget ledger’s per-user/per-key spend maps (reclaims only zero-spend entries — an already-tracked principal’s recorded spend is never discarded), and the semantic-routing classifier verdict cache (capped at 4096 entries by default).
-
Cache TTL configuration was ignored on two of three write paths.
CacheLayer::newhardcoded a 300s policy TTL regardless ofCacheConfig.ttl;with_store(used by every custom store and every OpenDAL backend viaManagedClient) took no config at all and had the same default.CacheLayer::newnow honours it, and a newCacheLayer::with_store_and_configconstructor carries the config through —ManagedClientuses it.with_storeitself is unchanged and still cannot honour a TTL, because it takes noCacheConfig; if you build a custom store through it, switch towith_store_and_configor you will keep getting the 300s default. -
Weighted-random routing seeded its RNG from
SystemTimesubsecond nanos, so requests arriving in the same timer tick drew near-identical values and a burst of traffic collapsed onto one deployment instead of spreading per the configured weights. Now seeded once from OS entropy. -
A request body containing a non-ASCII character within the first 64 bytes panicked while computing the idempotency body hash, which sliced a fixed-length prefix on a byte boundary that could fall mid-character. Now cuts on the nearest character boundary; previously-working hashes are unchanged.
-
LiterLlmErrorgained aretry_after()accessor alongsidestatus_code()anderror_type(), exposing the delay already parsed from aRetry-Afterheader onRateLimitederrors. Consumers previously had to invent their own backoff from the status code alone; bindings pick this up on their next regen. -
The npm package’s
bin/liter-llm.jswas committed non-executable, so a checkout — or any install path that preserves archive file modes — could not run it directly. -
The C# README badge linked to
nuget.org/packages/LiterLlm, a package this project does not own. -
Budget: month-to-date spend reset on every config reload, not on restart.
InMemoryBudgetLedgerhad no way to update its limits, so the proxy rebuilt and swapped the whole ledger to apply a reloaded config, discarding every sliding window. With hot-reload enabled and frequent config pushes a tenant’s budget could effectively never be enforced, and it failed open. Limits now sit behind anArcSwapand only that pointer is swapped, so a lowered limit applies to spend already recorded rather than forgiving it. -
Budget enforcement, cost tracking and rate limiting skipped every streamed request.
LlmResponse::usage()is alwaysNoneforChatStream; streams are now wrapped to fire accounting on completion. -
Requests differing only in
tools,response_format,seed,logit_bias,tool_choiceorextra_bodycollided in the cache and were served each other’s responses. See the upgrade note above. -
The negative cache was dead code: it wrote keys with a std
DefaultHasherover raw JSON while the read path used a seededahashover a curated field set, so an entry could never be read back. Both paths now share oneCacheKeyStrategy. -
SSE stream truncation was reported as a clean EOF — silent data loss. Truncated lines and truncated UTF-8 codepoints now surface as errors, and transport errors go through the retry budget instead of propagating immediately. Anthropic 529 is treated as retryable and its
Retry-Afteris honored. -
Gemini reasoning parts carry a
thoughtflag but were concatenated into visible content; they now route toreasoning_content, andreasoning_effortmaps tothinkingConfiginstead of being dropped. -
ImageUrlandAudioContentare shared between request and response types but carrieddeny_unknown_fields, so a provider adding a field to its image or audio output hard-failed the whole response. -
The proxy routed to only one deployment and silently dropped the rest; per-key rpm, tpm and
budget_limitwere parsed from config and never enforced by anything. -
Idempotency keys are released by an RAII guard when a request is cancelled, instead of being stranded for the full 24h TTL.
-
Metrics attribute caches were unbounded, so label cardinality grew without limit in a long-running proxy. The cost histogram was declared and never recorded to, which made any dashboard built on it read zero forever.
-
Documentation examples are generated from every E2E fixture across the configured binding-language matrix, keeping examples correlated with executable coverage and making missing target renderers visible during generation.
-
Dart: the native loader downloads and caches the library again on a cold cache. It only read the versioned cache and then threw a
StateError, even thoughnativeDownloadAndCacheLibrary()was defined and exported for exactly that case. The loader also now searches for the_dart-suffixed cdylib that is actually built, opens every candidate by absolute path (a hardened runtime rejects a relativedlopen), and names the real environment variable in its error message instead of printing the identifier$nativeLibDirEnvliterally. Fixed upstream in alef 0.55.6.Behavior change: an unresolvable native now throws a descriptive
StateErrornaming the asset URL and the download command, where it previously returnednulland let flutter_rust_bridge attempt its own relative-pathdlopen.
[1.16.0] - 2026-08-05
Section titled “[1.16.0] - 2026-08-05”-
[[guardrails]]— the proxy can now actually enforce guardrails. One registry is built at startup, layered into every model’s Tower stack, and handed to the realtime WebSocket relay as the sameArc, so a rule cannot be live on one path and absent on the other. Supportsregex(block or redact),length_cap,prompt_injection, andcelbehind theguardrail-celfeature, each scoped to explicitstages. Nothing degrades: an invalid pattern or expression, an unknown or misspelled key, an emptystageslist, a duplicate name, or acelentry in a build without the feature all abort startup before the listener binds — a partially-applied set is indistinguishable from an unguarded proxy.allow_list/deny_listare deliberately not exposed yet; they key on a named metadata field and onlytenant_idis populated, so any other field would fail silently. Guardrails are not hot-reloadable — the watcher warns rather than let a reloaded config describe a set the proxy is not running. -
Tool results can carry multimodal content.
ToolMessage.contentwasString, so a tool returning an image had to describe it in prose or smuggle it through a separate user turn. It is nowUserContent— the same untagged text-or-parts type user messages already use — withMessage::tool_result()for the common text case, so existingToolMessage { content: "…".into() }call sites keep working and a plain JSON string still deserializes (#165).
- Bedrock no longer silently drops non-text content parts. Converting a multi-part message kept only the text and discarded images without a trace; conversion is now shared with the other providers and warns on any part it cannot represent. This bug predates the change above.
- Go and Ruby bindings compile again.
LlmConfig.providerswas generated as both a struct field and a method —field and method with the same name Providersin Go, a duplicate method definition plus an RBSDuplicatedMethodDefinitionin Ruby. Fixed upstream in alef 0.55.0, which also regenerates Swift and C#. - The nightly model-catalog sync no longer reports failure after doing its job.
It opened and merged the PR correctly, then died on
gh pr merge --auto, which GitHub rejects once the PR is already mergeable (“clean status”) — so the job went red on roughly every other run. - CI runs poly’s whole-project lint phase. It was skipped entirely because the
shared validate job could install only Rust, Python and Java, so golangci-lint,
rubocop, steep, dart-analyze and credo ran in the git hooks and never in CI —
which is why the Go and Ruby breakage above reached
mainunnoticed.
Changed
Section titled “Changed”- Python:
config.providersis an attribute again rather than a bound method. The generated getter was shadowed by a same-named method wrapper, so the attribute the type stub and the constructor keyword both promised did not exist at runtime. Any caller written against the accidentalconfig.providers()spelling must drop the parentheses.
[1.15.0] - 2026-08-04
Section titled “[1.15.0] - 2026-08-04”LlmConfig, a canonical, binding-friendly client configuration type (client::LlmConfig, re-exported at the crate root). UnlikeClientConfigandFileConfig, it is plain data (String/Option<primitive>/HashMap/Vec/ plain sub-structs — no secrets, no trait objects, noDuration), derivesSerialize/Deserialize, and is not excluded from generated language bindings. Field-name parity with the fieldsmodel,api_key,base_url,timeout_secs,max_retries,temperature,max_tokens,load_env, andheaders, plus additive support for custom providers, cache, budget, rate-limit, cost tracking, tracing, cooldown, health-check, and AWS Bedrock configuration. Convert to a runtime client viaLlmConfig::into_client_builder.schemacargo feature: derivesutoipa::ToSchemaonLlmConfigand its sub-structs for OpenAPI schema generation.- AWS Bedrock is now fully configurable instead of environment-only:
ClientConfigBuilder::bedrock_region,bedrock_cross_region_prefix, andbedrock_credentialsset the region, cross-region inference profile prefix, and explicit AWS credentials onClientConfig.BedrockProvidergainedfrom_configand builder methods (with_cross_region_prefix,with_credentials) that fall back to the existing environment variables (AWS_DEFAULT_REGION/AWS_REGION,BEDROCK_CROSS_REGION,AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/AWS_SESSION_TOKEN) when left unset, so existing env-driven setups are unaffected.
[1.14.0] - 2026-08-04
Section titled “[1.14.0] - 2026-08-04”- Streaming support for the Responses API:
create_response_streamon theResponseClienttrait plus aResponseStreamEventtype and astreamfield onCreateResponseRequest, reusing the SSE machinery so Responses tool-calling apps get progressive output and streaming tool-call arguments (#162). Available in the Rust core crate; the language bindings are unchanged this release (the streaming event types are core-crate-only).
Changed
Section titled “Changed”- Regenerated all language bindings on alef 0.51.2.
- Ruby binding no longer publishes generated types into the global
Objectnamespace (theParsercollision with theparsergem); types stay namespaced under the binding module (tree-sitter-language-pack #173, via alef 0.51.1).
[1.13.0] - 2026-08-02
Section titled “[1.13.0] - 2026-08-02”- Anthropic-compatible providers can now target a custom base URL (#159).
anthropic/-prefixed models with abase_urloverride route to the Anthropic provider, so the Messages-API transforms (system extraction, content blocks, tool use, thinking) work against Anthropic-compatible endpoints such as DeepSeek’shttps://api.deepseek.com/anthropic. ReasoningEffortgainsMinimalandMaxvariants (#160), covering DeepSeek’smaxthinking level and OpenAI’sminimallevel. Variants are appended, so existing FFI integer discriminants are unchanged.
Changed
Section titled “Changed”extra_bodyis now shallow-merged into the request body root for OpenAI-compatible providers (#160), matching the OpenAI SDK. Top-level fields such as DeepSeek’sthinkingswitch reach the wire instead of being sent as a literalextra_bodyobject. The transport-controlledstreamflag cannot be overridden viaextra_body.- Synced the model catalog from models.dev.
- Upgraded dependencies (
cargo upgrade --incompatible).
[1.12.2] - 2026-08-01
Section titled “[1.12.2] - 2026-08-01”cargo binstall liter-llm-clisupport — prebuilt CLI binaries can now be installed directly from GitHub Releases without compiling from source. Adds[package.metadata.binstall]to the CLI crate plus a release-timeverify-binstallCI job that installs viacargo binstall, smoke-tests the binary, and gates release-finalize.
Changed
Section titled “Changed”- Synced the model catalog from models.dev (#158).
- Updated dependencies.
[1.12.1] - 2026-08-01
Section titled “[1.12.1] - 2026-08-01”- MiniMax now exposes region-specific endpoints (#157): the global region
(
https://api.minimax.io/v1for the OpenAI protocol,https://api.minimax.io/anthropicfor the Anthropic protocol) and the China region (https://api.minimaxi.com/v1,https://api.minimaxi.com/anthropic), each with its own docs root.
Changed
Section titled “Changed”- Regenerated the bundled model catalog, refreshing provider/model data and provenance.
- The ai-rulez-generated
plugin/.mcp.jsonbundle is now committed instead of being swept up by the broad.mcp.jsongitignore rule, so a fresh CI checkout no longer failsai-rulez verify --pluginwith a missing-file error.
[1.12.0] - 2026-07-31
Section titled “[1.12.0] - 2026-07-31”liter-llm-cliinstalls a real OTLP export pipeline under theotelfeature: it builds an OTLP span exporter + SDK tracer/meter providers, registers the W3C TraceContext propagator, and bridgestracingspans (and the library’sgen_ai.*metric instruments) to OpenTelemetry via atracing-opentelemetrylayer. Export is activated at runtime byOTEL_EXPORTER_OTLP_ENDPOINT; without it the CLI falls back to the console subscriber. Both theapiproxy server and themcpserver share this install path, and the server Docker image is built with--features otel. (Previously theotelfeature only compiled the OTel API with no exporter, so nothing was exported standalone — the instruments only reached a collector when a host such as xberg-enterprise installed the providers.)
Changed
Section titled “Changed”- Raw
println!/eprintln!/print!/eprint!/dbg!are denied in production code across the workspace (clippyprint_stdout/print_stderr/dbg_macro);tracingis the sole diagnostic surface. Genuinely non-diagnostic output opts back in per site with#[expect(...)](the interactive GitHub Copilot device-flow auth prompt and the CLI’s top-level error report), and thecatalog-gendev tool keeps its stdout/stderr contract. Language bindings regenerated with alef 0.48.12. - The
tracingCargo feature is removed;tracingis now an always-on dependency. Spans and events compile unconditionally (near-zero cost with no subscriber installed), aligning with the org feature-flag contract that tracing is never optional.otelremains the opt-in OTLP export layer.
- Streaming no longer aborts on a trailing metadata-only SSE event that omits
id. OpenAI-compatible providers such as OpenCode Zen/Go (https://opencode.ai/zen/go/v1) emit aninference-costevent with emptychoicesand noid/object/created/modelimmediately before[DONE]; these header fields onChatCompletionChunkare now#[serde(default)], so the event decodes to an empty chunk instead of failing withmissing field 'id'. (#155)
[1.11.5] - 2026-07-30
Section titled “[1.11.5] - 2026-07-30”Changed
Section titled “Changed”- Upgrade the MCP server to
rmcp3.0, adopting the MCP 2026-07-28 protocol revision and its sessionless streamable-HTTP transport. The existingvalidate_api_keymiddleware and per-requestKeyContextauth model are retained.
[1.11.4] - 2026-07-28
Section titled “[1.11.4] - 2026-07-28”- SSE streams no longer abort with “invalid UTF-8” when a provider splits a multi-byte character across HTTP chunks. The SSE parser now buffers the trailing bytes of a codepoint that lands on a chunk boundary and reassembles it with the next chunk, instead of treating the incomplete sequence as corruption and terminating the stream. Common with CJK text, accented characters, emoji, or any sufficiently long response. Genuinely malformed UTF-8 still errors. (#152)
Changed
Section titled “Changed”- Refresh the model catalog from models.dev (now 165 providers).
- Refresh
Cargo.lockto the latest compatible dependency versions.
[1.11.3] - 2026-07-27
Section titled “[1.11.3] - 2026-07-27”Changed
Section titled “Changed”- Refresh the model catalog from models.dev.
- Regenerate all language bindings on alef 0.48.4, which lowers the Maven
enforcer floor (fixes Java/Maven publishing) and generates the C#
runtime.jsontemplate (fixes C#/NuGet publishing). - Upgrade dependencies to their latest incompatible versions (base64 0.22 → 0.23).
[1.11.2] - 2026-07-26
Section titled “[1.11.2] - 2026-07-26”Changed
Section titled “Changed”- Regenerate all language bindings on alef 0.48.2.
- Update dependencies to their latest compatible versions.
Removed
Section titled “Removed”- Remove unused Java PMD ruleset and stale linter configuration.
[1.11.1] - 2026-07-26
Section titled “[1.11.1] - 2026-07-26”list_models()on providers that omitobject.ModelsListResponse.objectis now#[serde(default)], so listing models no longer fails withmissing field 'object'against OpenRouter and other OpenAI-compatible providers that omit the top-level"object": "list"field from/v1/models. (#149)
Changed
Section titled “Changed”- Dependency upgrades. pnpm 11.15 → 11.17, Jackson (databind/jdk8)
2.21.2 → 2.22.1, and a
sorbet-runtimebump.
[1.11.0] - 2026-07-23
Section titled “[1.11.0] - 2026-07-23”- OpenCode Zen and OpenCode Go providers — now 165 total (up from 163).
OpenCode Zen routes via the
opencode/model prefix (https://opencode.ai/zen/v1); OpenCode Go routes via theopencode-go/model prefix (https://opencode.ai/zen/go/v1). Both are OpenAI-compatible, bearer-authenticated (OPENCODE_API_KEY), and support chat completions. reasoning_contentsupport —StreamDeltaandAssistantMessagenow carry an optionalreasoning_contentfield, so reasoning/thinking tokens from OpenAI-compatible providers (DeepSeek R1, Qwen, etc.) are preserved in both streaming and non-streaming responses instead of being dropped. Anthropic extended-thinking blocks are mapped into the same field.AssistantMessagegains areasoning_text()accessor.
Changed
Section titled “Changed”- base64 embedding responses.
EmbeddingObject.embeddingnow accepts either a JSON float array or a base64 string of little-endianf32bytes (the OpenAI-compatibleencoding_format: "base64"response), decoding base64 vectors that previously failed to deserialize. Payloads are ~2.3× smaller on the wire and decode ~15× faster via a zero-copyVisitor. The element type narrows fromVec<f64>toVec<f32>(float32 is what providers send on the wire); the high-levelembed()API was alreadyf32, so only code reading the rawEmbeddingObject.embeddingfield asf64needs a one-line adjustment.
[1.10.1] - 2026-07-20
Section titled “[1.10.1] - 2026-07-20”- Swift: tool-call responses now decode correctly — the
typefield on tool calls honors its wire name instead of throwingkeyNotFound. - Swift: transcription
segments,tool_calls, and other optional list fields on response DTOs now decode instead of throwingDecodingError.typeMismatch. - Elixir: streaming entry points now call
chat_stream/2(a non-existentchat_stream_async/2was generated previously), fixingUndefinedFunctionError. - Regenerated all language bindings with alef 0.38.4.
[1.10.0] - 2026-07-19
Section titled “[1.10.0] - 2026-07-19”- Public
ModelInfoDTO andcost::model_info(name)accessor, exposing per-model metadata — context window, max input/output tokens, mode, and capability flags (vision, reasoning, tool calling, structured output, audio) — across all language bindings. - Opt-in runtime catalog refresh.
cost::refresh_catalog(&CatalogRefreshConfig)overlays an updatedcatalog.jsonon top of the embedded catalog at runtime, without a rebuild. It is a runtime toggle, not a Cargo feature, and is disabled by default (enabled: false). A failed, unreachable, or air-gapped refresh always falls back to the embedded (or prior) catalog, so a refresh failure never degrades availability. Also exposed:RefreshOutcome,CatalogRefreshError,install_catalog_overlay_from_str,clear_catalog_overlay, andDEFAULT_CATALOG_URL. - Extended per-model pricing. The catalog now carries context-tiered pricing
(e.g. distinct rates above a 200k-token context), plus audio and reasoning
token costs, all consumed by
completion_cost. NewModelTiertype. - 20 new providers — now 163 total (up from 143) — including Black Forest Labs, Reducto, Soniox, Nvidia Riva, Google Distributed Cloud (Gemini), and AWS Bedrock Mantle.
- Rust catalog generator (
liter-llm-catalog-gen) replacing the Pythongenerate_pricing.py/generate_providers_doc.pyscripts. It fetches and strictly validates the models.devapi.json(rejecting unknown fields so upstream drift fails loudly) and dual-writescatalog.jsonwith a$provenanceblock (source SHA-256, fetch date, library version). New tasksgenerate:catalogandgenerate:catalog:check. - Catalog automation workflows. A daily
sync-catalogworkflow opens an auto-merging PR when models.dev drifts;publish-catalogpushes catalog snapshots to a rollingmodel-catalogrelease;catalog-verifygates PRs that touch the catalog.
Changed
Section titled “Changed”- Renamed the embedded pricing registry from
pricing.jsontocatalog.json(both the canonicalschemas/copy and the crate’scrates/liter-llm/schemas/copy), reflecting its expanded per-model metadata. - Upgrade dependencies:
jsonschema0.46 → 0.48,rmcp2.1 → 2.2,tokio-tungstenite0.29 → 0.30. Regenerate all bindings with alef 0.38.0.
list_models()no longer fails for providers that omitcreated(e.g. DeepSeek).ModelObject.created,object, andowned_bynow default when absent from a provider’s/v1/modelsresponse. Fixes #139.
[1.9.3] - 2026-07-04
Section titled “[1.9.3] - 2026-07-04”Changed
Section titled “Changed”- Upgrade all dependencies to latest and regenerate all bindings with alef
0.31.0. Notable major bumps:
rmcp1.7 → 2.1,etcd-client0.15 → 0.19,tokio-tungstenite0.26 → 0.29,tower-http0.6 → 0.7,sha20.10 → 0.11,cel-interpreter0.8 → 0.10, andtikv-jemallocator0.6 → 0.7.
- CEL guardrail: adapt to
cel-interpreter0.10.Program::compilenow returnsParseErrors, and the new ANTLR-based parser can panic on some malformed input.CelGuardrail::newnow catches those panics and returns aCelCompileError, so an invalid expression can never abort the caller (e.g. proxy startup). - Proxy MCP server: adapt to
rmcp2.1 model changes (RawResource→Resource,RawResourceTemplate→ResourceTemplate,Content→ContentBlock,PromptMessageRole→Role). - Proxy etcd config watcher: adapt to
etcd-client0.19, whosewatchnow returns a singleWatchStreamand movescancelonto the stream.
[1.9.2] - 2026-07-02
Section titled “[1.9.2] - 2026-07-02”- Ruby: allow Ruby 4.x. Drop the
< 4.0upper bound on the gem’srequired_ruby_version(now>= 3.2.0), so installing/updating the gem no longer fails on Ruby 4.0+. Fixes #136.
[1.9.1] - 2026-07-02
Section titled “[1.9.1] - 2026-07-02”Changed
Section titled “Changed”- Upgrade dependencies and regenerate all bindings with alef 0.30.11.
- Add
[workspace] extra_clippy_allows = ["single_match", "collapsible_match"]toalef.tomlso the generated single-variant enum-from-int bindings passclippy -D warnings.
- Correct the HashiCorp vendor name spelling in the proxy Vault secret backend
(
HashiCorpVaultProvider/Builder), its comments, docs, and this changelog.
[1.9.0] - 2026-06-27
Section titled “[1.9.0] - 2026-06-27”Stable release promoting 1.9.0-rc.2 (fully published green). No code changes beyond the version bump.
[1.9.0-rc.2] - 2026-06-27
Section titled “[1.9.0-rc.2] - 2026-06-27”Changed
Section titled “Changed”- Rebrand to Xberg. All published package coordinates move off the
kreuzbergbrand: npm scope@kreuzberg/*→@xberg-io/*(main, the six napi platform packages,-wasm, and-cli); the Java/Maven and Kotlin Android namespacedev.kreuzberg.literllm[.android]→io.xberg.literllm[.android](group iddev.kreuzberg→io.xberg), which relocates the generated Java/Kotlin source trees fromdev/kreuzberg/…toio/xberg/…. Docs, badges, the ecosystem block (now links theXbergproduct atgithub.com/xberg-io/xberg), the brand heading (“Part of Xberg.io”), and CI publishing references (owner: xberg-io,ghcr.io/xberg-io, packagistxberg-io, thexberg-dev-publisherapp,xberg-io/homebrew-tap) are updated to match. The legal entity name “Kreuzberg, Inc.” is unchanged. (alef.toml, templates, generated bindings, docs,publish.yaml) - Regenerate with alef 0.29.3 (from 0.29.1): correct Kotlin Android Maven coordinate, drop redundant
-nodenpm alias.
Removed
Section titled “Removed”- Drop the unused legacy
packages/typescriptwrapper package. It was a re-export barrel of the native@xberg-io/liter-llmbinding and was never published. alef no longer generates it (the napi backend stopped emittingpackages/typescript); the canonical TypeScript surface is the native package’s bundledindex.d.ts. Removed the directory and its references in the workspace list,dependabot.yaml, thevalidate-versionsmanifest path (nowcrates/liter-llm-node/package.json), thenode:typechecktask, and the oxlint exclude.
[1.8.2] - 2026-06-23
Section titled “[1.8.2] - 2026-06-23”- Publish (zig): the published zig package compiles again. The “Stage FFI artifacts” step bundled a stale top-level
crates/liter-llm-ffi/liter_llm.h(old ABI, missingliterllm_create_client/*_from_json) into theffi-*artifact, so the zig wrapper’s@cImportresolved against an out-of-date header and every test failed to compile. The step now stages the canonicalinclude/liter_llm.h(and fails hard if absent); the orphaned top-level header is removed. - Publish (kotlin-android): the AAR no longer corrupts on download. Both ABI matrix legs emitted the same filename
liter-llm-android-release.aar; the publish job’sdownload-artifact … merge-multiple: truecollided them into one torn zip (bad zipfile offset). Each leg now uploads an ABI-suffixedliter-llm-android-<abi>.aar. - Publish (swift): a rerun can no longer desync the artifact-bundle checksum. The swift-artifactbundle job now reuses an existing release asset and its checksum instead of rebuilding a non-reproducible bundle, keeping the hosted bundle consistent with the
swift-<version>tag. - Swift test-app scaffold pins the checksum-bearing ref. Regenerated from alef 0.26.7: the registry-mode
Package.swiftnow uses.package(url:, branch: "release/swift/<version>")(which carries the substituted checksum) instead offrom: "<version>"(which resolved the placeholder-bearing SemVer tag).
Changed
Section titled “Changed”- Bindings regenerated from alef 0.26.7 (from 0.26.5): swift e2e scaffold branch pin and kotlin-android Gradle plugin/dependency bumps. Generated node e2e/test_apps emit
.npmrcdisabling frozen-lockfile.
[1.8.1] - 2026-06-23
Section titled “[1.8.1] - 2026-06-23”- CI E2E (kotlin_android): the host-JVM test project compiles again. The generated
MockServerListenerimplements the JUnit PlatformLauncherSessionListenerSPI (referencingLauncherSession/LauncherSessionListenerat compile time), but the e2ebuild.gradle.ktsscopedjunit-platform-launcherastestRuntimeOnly, socompileDebugUnitTestKotlinfailed with “Unresolved reference ‘launcher’”. Regenerated from alef 0.26.5, which now scopes ittestImplementation. - CI E2E (swift): the e2e link step now finds
liter_llm_ffi. The swift e2ebeforestep builtliter-llm-swiftand the mock server but neverliter-llm-ffi, solibliter_llm_ffi.awas absent from the linker search path (ld: library 'liter_llm_ffi' not found). The[crates.test.swift].beforestep now also runscargo build --release -p liter-llm-ffi. - CI E2E (node): the lockfile is back in sync.
pnpm-lock.yamlwas stale againstcrates/liter-llm-node/package.json(missing@napi-rs/cli ^3.6.2) after the 1.8.0 bump, failingpnpm install --frozen-lockfile. Regenerated.
Changed
Section titled “Changed”- Bindings regenerated from alef 0.26.5 (from 0.26.3): Swift bridge-glue re-materialization runs after the bridge crate is built, opaque-handle aliasing avoids capsule import collisions, and JSON-string overloads emit positional arguments for underscore-prefixed parameters.
[1.8.0] - 2026-06-22
Section titled “[1.8.0] - 2026-06-22”- MCP server: tool annotations on every tool. Each MCP tool now advertises rmcp
ToolAnnotations(a human-readable title plusreadOnlyHint/destructiveHint/idempotentHint/openWorldHint) so clients can present them and decide auto-approval. Query tools are read-only;create_*mutate without being destructive;delete_*/cancel_*are destructive and idempotent; all reach external providers (openWorldHint). - MCP server: prompts, resources, and argument completion. Beyond tools, the server now exposes reusable prompt templates (
summarize,translate,extract), catalog resources (liter-llm://models,liter-llm://providers, and theliter-llm://pricing/{model}/liter-llm://provider/{name}templates), and argument completion formodel(from the configured models) and providername(from the registry).get_infoadvertises tools, prompts, resources, and completions.
- Budget middleware: concurrent spend is no longer lost across a window rollover. On weakly-ordered architectures (arm64) the window-reset path could drop a racing
fetch_add; the rollover now subtracts the snapshotted prior total instead of storing zero, preserving every concurrent charge. - CI: the Kotlin Android build provisions the wrapper-declared Gradle version (
gradle-version: wrapper) inci-mobileandci-e2e. AGP 9.2.0 requires Gradle >= 9.4.1; the previous hardcoded8.13pin silently overrode the 9.6.0 wrapper and brokeassembleRelease. - Swift & Dart: the untagged content-union
text()accessor now references the generated payload field instead of a non-existent binding, mirroring the 1.7.6 Kotlin fix. Regenerated from alef 0.26.3. - CI/release: create the draft GitHub Release once in the
preparejob so the Swift artifact-bundle upload (the only upload job without its own ensure-release step) can no longer race ahead of release creation and fail with “release not found”. Mirrors the html-to-markdown publish pipeline.
Changed
Section titled “Changed”- Bindings regenerated from alef 0.26.3 (from 0.25.60): Swift capsule-pointer bridging via
usizeplus.productdependency wiring,RustBridgeC.hpreserved acrossalef all --clean, and the content-union accessors above.
[1.7.6] - 2026-06-22
Section titled “[1.7.6] - 2026-06-22”- Kotlin Android: the untagged content-union
text()accessor now references the generatedvalueproperty instead of the non-existentfield0, fixing a:compileReleaseKotlinfailure (Unresolved reference 'field0') that broke the Android AAR build. Regenerated from alef 0.25.60; the legitimatefield0data-class properties (e.g.LiterLlmError.Serialization) are unaffected.
[1.7.5] - 2026-06-21
Section titled “[1.7.5] - 2026-06-21”Changed
Section titled “Changed”- npm: the primary package is now the bare
@kreuzberg/liter-llm(matching the crawlberg convention), renamed from@kreuzberg/liter-llm-node. The napipackageNameand all per-platform sub-packages use the bare prefix (@kreuzberg/liter-llm-<rid>); the.nodebinaryNamestaysliter-llm-node. (alef.toml, generated node binding, all README badges, docs,pnpm-lock.yaml)
- Packagist: publish under the canonical
xberg-io/liter-llmvendor (matching html-to-markdown, crawlberg, tree-sitter-language-pack) instead of the legacykreuzberg/liter-llm. Fixes the registry-check coordinate and thepublish-packagiststep (packagist-username,package-name) inpublish.yaml, plus the stale reference inllms.txt. The Maven registry-check coordinate is corrected todev.kreuzberg.literllm:liter-llm. - CI/release:
@kreuzberg/liter-llm-clinow publishes on the main release at the release version (previously decoupled on a separatecli-proxy-v*tag), gated behind the CLI-binary upload so the npm wrapper never ships ahead of the binaries it downloads. - CI/release: disable sccache for release build jobs so a transient sccache cache/DNS failure can no longer block a publish run (
publish.yaml). - docs: refresh stale install snippets — Java/Kotlin/Swift/Zig version pins, the Java Maven coordinate, and the Elixir version range in
docs/getting-started/installation.md,docs/index.md, andllms.txt.
[1.7.4] - 2026-06-19
Section titled “[1.7.4] - 2026-06-19”Changed
Section titled “Changed”- chore(precommit,alef): standardize kotlin-android formatting on ktfmt –kotlinlang-style. Drop the conflicting prek ktlint hook (it ran a destructive
--formatthat fought ktfmt and rewrote alef’s///doc comments), scope ktfmt topackages/kotlin-androidwith--kotlinlang-style, switchalef.tomlkotlin format/check from gradle-ktlintFormat to ktfmt so alef and prek agree, and exclude the vendored Gradle wrapper from shellcheck. detekt remains. (.pre-commit-config.yaml,alef.toml)
- Content-union e2e gate completed across the remaining bindings (regenerated with alef 0.25.49). The Dart binding now injects the
AssistantContent.text()extension even when FRB emits the freezed mixin clause (sealed class … with _$… {); the PHP flat-enum conversion always emits an exhaustive wildcard for the&strtag match, soalef(skip)’d-Defaultenums (Message,ContentPart,AssistantPart,OcrDocument) compile; and the Node/NAPI binding keeps bothFromconversion directions for plain data enums used as struct fields (e.g.AuthHeaderFormatinCustomProviderConfig). Local e2e green for Python, Node, WASM, Elixir, and PHP.
Documentation
Section titled “Documentation”- Bindings parity docs. Added a feature-by-language support matrix (chat, streaming, tool calling, embeddings, multimodal in/out, call idiom) across all 14 native bindings plus the C/FFI surface, expanded the multimodal cookbook with idiomatic examples in nine languages, and reconciled the binding count to a single “14 native bindings + C/FFI surface” across the README and docs (added Dart, Swift, Kotlin Android, and Zig to the README Language READMEs table).
[1.7.3] - 2026-06-19
Section titled “[1.7.3] - 2026-06-19”- Cross-language e2e for multimodal
message.content. The remaining bindings can now string-assert the assistant content union (AssistantContent), fixing the e2e suites that broke on it: Kotlin and Dart get atext()accessor on the sealed class; WASM returns the display text (String) instead of a discriminant; Swift renders property access for first-class result structs; Elixir reads the NIF struct’s.text; PHP calls the message’stext()accessor. Generated via alef 0.25.48 (untagged_union_text_types+fields_display_as_textextended to all backends). - Android AAR packaging guard — the Kotlin/Android publish now stages the cross-compiled
JNI
.solibs into the AAR and fails loudly ifjni/would be empty, so a jni-less AAR can never be published.
[1.7.2] - 2026-06-18
Section titled “[1.7.2] - 2026-06-18”DisplayforAssistantContent— renders the message text (Textvariant verbatim;Partsvariant concatenates its text segments, skipping non-text parts), enabling string assertions onmessage.contentacross the polyglot e2e suites.
- Cross-language e2e content assertions — the generated e2e suites stringified
choices[0].message.contentwith plain-string casts that fail for the multimodalAssistantContentunion (Gostring(*ptr), JavaObjects.toString, C#.ToString(), Rust.as_deref()). alef 0.25.45’sfields_display_as_textconfig now emits the per-language text accessor so the assertions compile and assert text content.
[1.7.1] - 2026-06-18
Section titled “[1.7.1] - 2026-06-18”- FFI build under
-D warnings— the regeneratedliter-llm-fficrate referenced#[cfg(feature = "wasm-http")]without declaring the feature, producingunexpected cfg condition value: wasm-httperrors that broke CI Rust, CI E2E (Build FFI), CI Mobile, and the crates.io publish in v1.7.0. alef now declares non-default passthrough features via the configurable[crates.ffi] extra_featureskey, sowasm-httpis declared (but not enabled) and survives regeneration. - Java PMD on generated DTOs — the new
DecodedDataUrl { mime, byte[] data }value object trippedArrayIsStoredDirectly/MethodReturnsInternalArray; alef’s PMD ruleset now excludes these for generated DTOs. - Docs strict build — fixed a broken intra-doc link in
docs/usage/multimodal.md(../concepts/providers.md→../providers.md).
[1.7.0] - 2026-06-18
Section titled “[1.7.0] - 2026-06-18”- Typed multimodal builders —
liter_llm::image::{encode_data_url, decode_data_url, DecodedDataUrl}withIMAGE_PNG/IMAGE_JPEG/IMAGE_WEBP/IMAGE_TIFFMIME constants.decode_data_urlreturns a namedDecodedDataUrl { mime, bytes }struct rather than a tuple so polyglot bindings extract it as a typed object. Message::user_with_parts(parts)— ergonomic constructor for multimodal user messages.ContentPart::{text, image_data_url, image_url, image_with_detail, image_png, image_jpeg, image_webp, image_tiff}— typed constructors replacing hand-rolled struct construction.ResponseFormat::{json_schema, json_object, text}+JsonSchemaFormat::new(name, schema).strict(bool).description(d)fluent builder.newdefaults tostrict = Some(true). Provider-mapping rustdoc onResponseFormat(OpenAI passthrough, Gemini/VertexresponseMimeType+responseSchema, Anthropic system-instruction injection).- Multimodal output:
AssistantContentenum (Text/Parts) with#[serde(untagged)]back-compat;AssistantPart(Text/Refusal/OutputImage/OutputAudio) with#[serde(tag = "type", rename_all = "snake_case")];AssistantMessage::{text, refusal_text, output_images, output_audio}accessors;Message::{assistant_with_parts, system_with_parts}constructors;ChatCompletionRequest.modalities: Option<Vec<Modality>>withModality::{Text, Audio, Image}. Vertextransform_responsepreservesinline_dataasOutputImage/OutputAudio(no base64 re-encode); OpenAItransform_responsehoistsmessage.audiointoParts([Text(transcript), OutputAudio]).
Changed
Section titled “Changed”- BREAKING:
AssistantMessage.content: Option<String>→Option<AssistantContent>. Back-compat viaFrom<String>/From<&str>forAssistantContentand the untagged serde variant — providers returning scalarcontentstrings still deserialize asText(_). - BREAKING:
SystemMessage.content: String→UserContent. Back-compat viaFrom<String>/From<&str>forUserContent. liter-llmmakesbase64an unconditional dependency (previously gated behindnative-http/wasm-http) —liter_llm::image::*helpers are transport-agnostic.- All polyglot bindings regenerated for the multimodal surface (alef 0.25.40 pin). PHP/WASM extraction of
#[serde(untagged)]enums (AssistantContent) and nested complex enum-variant types (OutputImage { image_url: ImageUrl },OutputAudio { audio: AudioContent }) is resolved upstream; bindings construct/access multimodal types via each language’s native idiom.
[1.6.4] - 2026-06-17
Section titled “[1.6.4] - 2026-06-17”Changed
Section titled “Changed”- Gate
etcd-clientbehind optionaletcd-watchfeature; default CLI builds no longer requireprotoc. Resolves the Homebrew bottle source-build failure on v1.6.3. - Bump
alefcodegen pin to 0.25.24 (swift opaque-handle class triples, dart() -> ()cleanup, kotlin per-file ktfmt invocation, java PMD/palantir-java-format alignment, c e2e FFI tarball-name alignment, phpexclude_functionsfiltering through to user-facing wrapper). liter_llm::tower::metrics::global_meter()is nowpubso downstream crates (liter-llm-proxy) can read the shared OTel meter without re-initialising.
ContentPartcrate-root shadow —src/lib.rsdidpub use types::*(which bringstypes::ContentPartto the crate root) and then explicitlypub use realtime::ContentPart, causing the realtime variant to shadow the types one. Downstream consumers writinguse liter_llm::ContentPart;received the realtime variant, which has noImageUrlvariant, producingE0599: no variant named 'ImageUrl' found for enum 'liter_llm::ContentPart'in any VLM-OCR call site. The realtimeContentPartre-export is removed fromlib.rs; callers that need it must import it explicitly asliter_llm::realtime::ContentPart. A compile-time regression test insrc/tests.rsasserts thatcrate::types::ContentPart::ImageUrlis constructible through the crate root.- rustdoc ICE in
liter-llm— bracketed intra-doc links to private items (MAX_POOL_BUFFER_CAPACITY,post_stream_with_cancel,post_json,post_json_raw,retry::should_retry,LiterLlmError,ConfigDrivenProvider::transform_request) inhttp//provider/triggered an internal compiler error on rustc 1.95.0 duringdoc_link_resolutions. Stripped the link brackets so the doc strings reference the names as plain code identifiers. Also fixed bare URL inCustomProviderConfig::base_url. - Missing-docs cleanup — added documentation for
UsageSinkErasedtrait and method, the fourtenantmodules (context,etcd,in_memory,resolver), thehttp::transportmodule, and theRouterError::Discover { source }struct field.
[1.6.3] - 2026-06-16
Section titled “[1.6.3] - 2026-06-16”- WASM build:
tokio::timeinsidewait_for_batch_impl—crates/liter-llm/src/client/mod.rspolled withtokio::time::Instantandtokio::time::sleepunconditionally, buttokiois gated behind thenative-httpfeature. Underwasm-http-only builds (the WASM crate) compilation failed withE0433: cannot find module or crate 'tokio'. The function now switches toweb_time::Instant+gloo_timers::future::sleepontarget_arch = "wasm32", withweb-timeadded as an optional dep on thewasm-httpfeature. - WASM build: leaked tower DTOs — alef regen at v1.6.0 started emitting
From<liter_llm::tower::{CircuitState, HealthStatus, IntentPrototype, SingleflightResult}>impls incrates/liter-llm-wasm/src/lib.rs, buttoweris not enabled underwasm-http. Added the four types toalef.toml[crates.wasm].exclude_typesalongside the existing config-DTO exclusions. - Ruby gem build: missing feature declarations —
packages/ruby/ext/liter_llm_rb/native/Cargo.tomldid not declarenative-http/opendal-cache/wasm-http, so alef-emitted#[cfg(feature = "native-http")]gates insrc/lib.rsresolved as false and theensure_crypto_providercall site failed withE0425: cannot find value in this scope. Regen now writes the features section. Affected every Ruby gem variant (linux-x86_64, linux-aarch64, macos-arm64). - PHP windows builds:
ext-php-rsmacro lookup —LiterLlmApi::ensure_crypto_providerresolution failed inside#[php_impl]-generated code ontarget_os = "windows"(E0599: no associated function or constant). Other bindings (PyO3, NAPI, Rustler) handle the same pattern correctly, so this looks like an ext-php-rs upstream gap. Excludedensure_crypto_providerfrom the PHP binding entirely viaalef.toml— the function is a no-op on Windows anyway, and downstream PHP users rely on transitive invocation from internalreqwest::Clientconstructors. - PHP PIE matrix:
php8.5macos-arm64 —shivammathur/setup-php@v2does not yet ship a PHP 8.5 image for macOS arm64 runners (PHP 8.5 was released November 2025). Excluded that matrix cell frompublish.yamluntil the upstream action catches up. Linux + Windows PHP 8.5 builds remain in the matrix.
[1.6.2] - 2026-06-16
Section titled “[1.6.2] - 2026-06-16”to_singleflight_errordead-code lint — gatedLiterLlmError::to_singleflight_errorwith#[cfg(feature = "tower")]so the method is not emitted whencargo publish --verifybuilds with default features (default = ["native-http"]). The method’s only call sites live insrc/tower/cache_singleflight.rs, which is itself feature-gated. Under-D warningsthe dead-code lint rejected the build, blockingPublish crates.ioand every per-platformBuild CLI binary,Build WASM, andBuild Kotlin Android nativesjob in the v1.6.0 and v1.6.1 release workflows. No artifacts reached PyPI, crates.io, npm, RubyGems, Maven Central, Packagist, Hex, NuGet, pub.dev, or the Homebrew tap from those tags.- Release-runner
protoctoolchain dep —xberg-io/actions/setup-rust@v1.8.70now installsprotobuf-compileron every Linux/macOS/Windows runner.etcd-client v0.15’sbuild.rsshells out toprotoc;liter-llm-clipullsetcd-clienttransitively throughliter-llm-proxy, so the v1.6.1 CLI binary builds panicked withFailed to compile proto files: Could not find protoc. v1.6.2’s publish workflow consumes the floatingv1tag which now carries the fix.
[1.6.1] - 2026-06-16
Section titled “[1.6.1] - 2026-06-16”- FFI crate
wasm-httpfeature — declaredwasm-httpas a no-op feature onliter-llm-ffi/Cargo.tomlso the alef-emitted#[cfg(any(feature = "native-http", feature = "wasm-http"))]gates incrates/liter-llm-ffi/src/lib.rsresolve undercargo build -D warnings. v1.6.0’s publish workflow failed on every Rust FFI, CLI, WASM, and Kotlin Android native build withunexpected_cfgserrors because the gate was emitted without the corresponding feature declaration.
[1.6.0] - 2026-06-16
Section titled “[1.6.0] - 2026-06-16”Bindings
Section titled “Bindings”- All 14 language bindings regenerated against alef
0.25.18. Covers Python (PyO3), TypeScript (NAPI-RS), Ruby (Magnus), PHP (ext-php-rs), Elixir (Rustler), Go (cgo), Java (Panama FFM), C# (P/Invoke), Swift (swift-bridge), Dart (flutter_rust_bridge), Kotlin Android (JNI), Zig, C, and WASM. - Tier A/B/C API triage applied to the v1.6.0 surface: internal types stay Rust-only (Tier A), trait-generic helpers like
wait_for_batch_impland theBatchRetrievertrait stay Rust-only (Tier B), and binding-exposed concrete methods likeDefaultClient::wait_for_batchcross the FFI boundary (Tier C). Tier B trait methods (BatchRetriever::fetch_batch_for_polling) now emit correctly in JNI without a workaround. - Swift
chat_streamstreaming adapter restored across all bindings. The swift-bridge extern block emitted by alef now declarestype DefaultClient;inside the streaming extern block so the owner reference resolves.
Tooling
Section titled “Tooling”- alef bumped from
0.23.7→0.25.18. New upstream fixes consumed:- JNI backend emits per-trait
use core_crate::{path};clauses for non-root trait methods. - swift-bridge streaming-adapter extern block declares owner types alongside handle types.
cleanup_orphaned_filesrecognises hash-less stale files (self-referential// auto-generated by alefheader).- Dart and PHP
cfg-feature forwarding fixes for thewasm-httpfeature gate.
- JNI backend emits per-trait
KeyContext::tenant_id: TenantId— tenant identifier carried in every resolved auth context. Master-key auth always setsTenantId("master")(seeMASTER_TENANT_IDconstant); virtual-key auth propagates thetenant_idreturned by the configuredKeyResolver.KeyContext::from_resolved(key_id, &ResolvedKey)— constructor that builds aKeyContextfrom aKeyResolver-resolved record, preserving the canonicaltenant_idfrom the resolver rather than falling back to the raw key token.auth::MASTER_TENANT_ID: &str— well-known constant ("master") used byKeyContext::master()so downstream budget and usage layers can identify master-key traffic without a special-case enum.
Changed
Section titled “Changed”-
KeyContextnow carriestenant_id: TenantIdresolved by the configuredKeyResolver. Master-key auth resolves toTenantId("master"). -
Every proxy HTTP handler now propagates
tenant_idintoLlmRequest::with_tenant_id, activatingBudgetLedger::Tenant,TenantScopedStrategy, andUsageEvent.tenant_idfor all in-proxy traffic. Thedispatchhelper inroutes/mod.rsapplieswith_tenant_idcentrally;chat.rs(which bypassesdispatchfor stream/non-stream branching) sets it inline. -
validate_api_keymiddleware now resolves virtual keys viaAppState.key_resolver.resolve()(previously usedKeyStore.get()directly), so customKeyResolverbackends injected viaProxyServer::with_key_resolverreceive calls for all virtual-key traffic. -
ProxyServer::with_key_resolverandProxyServer::with_usage_sinkbuilder methods for embedder-supplied dependencies. Default behaviour unchanged. -
AppState.usage_sink: Option<Arc<dyn UsageSinkErased>>;HooksLayeris now wired outermost in the Tower stack when a sink is configured. -
UsageEvent.effective_model: Option<String>— provider-echoed model name from the response, distinct frommodelwhich is the requested name. Populated forChat,Embed,Moderate,Ocr, andSearchresponse variants;Nonefor streaming, speech, transcription, rerank, image generation, and list-models variants, and on error/timeout paths. -
UsageEvent.cache_stateis now accurate. Previously hardcodedBypass; now reflectsMiss,ExactHit,SemanticHit, orStaleHitset byCacheLayerandSingleflightService(followers) via atokio::task_local!cell read byHooksLayerafter the inner service resolves. Requests with noCacheLayerin the stack continue to reportBypass. -
DefaultClient::wait_for_batch(batch_id, WaitForBatchConfig)— poll a batch until terminal status (Completed, Failed, Expired, Cancelled) with exponential backoff, configurable intervals, and optional timeout. ReturnsOk(BatchObject)on completion,Err(BatchWaitError::{Failed, Timeout, Client})on failure. -
WaitForBatchConfig— configuration struct withinitial_interval_secs: f64,max_interval_secs: f64,backoff_multiplier: f32, and optionaltimeout_secs: Option<f64>fields; implementsDefaultwith 5.0s initial, 60.0s max, 1.5x multiplier, no timeout. All fields usef64seconds (notDuration) for FFI bridgeability. -
BatchWaitError— error enum with struct-style variantsFailed { status: BatchStatus },Timeout { timeout_secs: f64 }, andClient { message: String, code: u32 }for batch polling failures. All fields are FFI-friendly primitives. -
liter_llm::observability::{UsageEvent, UsageSink, CacheState, UsageEventOutcome, UsageSinkError, LoggingUsageSink, MultiUsageSink}— canonical per-request usage events with pluggable sinks.UsageEventis billing/observability-vendor-agnostic; downstream consumers translate it into metrics, ledgers, or OTel events. -
HooksLayer::with_usage_sink— wires aUsageSinkinto the Tower stack; emits one event per request completion (success or error). Sink errors are best-effort: logged and not propagated to callers. -
liter_llm::tower::IdempotencyLayer—Idempotency-Keydedup with pluggableIdempotencyStore(defaultInMemoryIdempotencyStoreviaDashMap, 24h TTL). Mismatched body for same key returnsLiterLlmError::IdempotencyConflict; in-flight key returnsLiterLlmError::IdempotencyInFlight(error-out, not sleep-poll). -
LlmRequest::idempotency_key: Option<String>field andLlmRequest::with_idempotency_keybuilder for opt-in idempotency. -
LiterLlmError::IdempotencyConflict { key: String }andLiterLlmError::IdempotencyInFlight { key: String }error variants (HTTP 409 equivalent). -
liter_llm::tower::FallbackChainLayer— walk an orderedVec<S>of services, advancing on transient errors via pluggableRetryPolicy.DefaultRetryPolicytreats 5xx/timeouts/429 as transient; auth and validation errors as terminal. ExportsRetryClass,RetryPolicy,DefaultRetryPolicy,FallbackChainLayer, andFallbackChainService. -
liter_llm::tenant::{TenantId, TenantContext, KeyResolver, ResolvedKey, KeyResolverError, InMemoryKeyResolver}— generic multi-tenant primitives. -
liter_llm::tenant::EtcdKeyResolver— distributedKeyResolverbacked by an etcd cluster. Behind featureetcd-key-resolver. Reads JSON-serialisedResolvedKeyat{prefix}/{sha256(api_key)}. Configure viaEtcdKeyResolverConfig(endpoints, prefix, timeouts, optional auth). -
LlmRequest::with_tenant_id/LlmRequest::tenant_idfor tenant propagation through the Tower stack. -
LlmRequestKind— the discriminant enum extracted fromLlmRequestto carry the variant payload; re-exported fromliter_llm::tower.
Changed
Section titled “Changed”- Migration —
RealtimeEvent::RateLimitsUpdated: fieldreset_at: SystemTimerenamed toreset_at_unix_ms: i64(Unix milliseconds). Pattern-match sites must update the field name; the value can be reconstructed withSystemTime::UNIX_EPOCH + Duration::from_millis(reset_at_unix_ms as u64)if needed. - Migration —
WaitForBatchConfig: fields renamed for FFI bridgeability —initial_interval: Duration→initial_interval_secs: f64,max_interval: Duration→max_interval_secs: f64,timeout: Option<Duration>→timeout_secs: Option<f64>. Update all construction sites to usef64seconds. - Migration —
BatchWaitError: variants changed to struct-style for FFI bridgeability —Failed(BatchStatus)→Failed { status: BatchStatus },Timeout(Duration)→Timeout { timeout_secs: f64 },Client(LiterLlmError)→Client { message: String, code: u32 }. Update all pattern-match and construction sites. LlmRequestis now a struct (kind: LlmRequestKind,tenant_id: Option<TenantId>) rather than a plain enum. All existing constructor call sites (LlmRequest::Chat(r),LlmRequest::Embed(r), etc.) continue to compile unchanged via#[allow(non_snake_case)]associated functions. Pattern-match sites that directly match onLlmRequest::Variantmust be updated to match onreq.kind.liter-llm-proxyAppStategainskey_resolver: Arc<dyn KeyResolver>alongside the existingkey_store: Arc<KeyStore>.KeyStoreimplementsKeyResolver; behaviour is unchanged.- Cache (
CacheService,SingleflightService) readstenant_idfrom the request viaLlmRequest::tenant_id()so cached responses are scoped to the correct tenant automatically. KeyResolver::resolvenow takes an ownedStringand returns a'staticfuture so the future can be spawned acrosstower::Service::callboundaries.ProviderCredential.api_keyis nowsecrecy::SecretString(wasString); zeroed on drop,Debugimpl redacts to[REDACTED].liter-llm-proxyactivatesliter_llm::provider::set_outbound_policyat startup based onSecurityConfig.outbound_policy. Defaults toDenyPrivate(blocks RFC1918, loopback, link-local, multicast, unspecified);Allowlistparse errors are logged and skipped with a startup summary line.liter-llm-proxyvirtual-key lookup usessubtle::ConstantTimeEqover a full iteration of the key store, preventing timing-side-channel inference of key existence.tower::HooksLayer::with_usage_sinknow dispatches sink emit viatokio::spawn; slow sinks no longer add latency to caller-observed responses.
tower::circuit—maybe_half_openis wired into the request path; circuits no longer get permanently stuckOpenafter tripping.probe_in_flightis now aProbeGuard<P>RAII enum so the probe slot is reclaimed even if the future panics or is cancelled.should_allowgates HalfOpen to exactly one probe via atomic CAS.tower::hedge—HedgeService::callusesmem::replacecorrectly so the polled-ready inner becomes the primary attempt and hedge attempts clone from a fresh standby; eliminates Tower-readiness contract violation andConcurrencyLimitpermit double-consumption.tower::cache::CacheService::call— applies themem::replace(&mut self.inner, self.inner.clone())Tower swap so permit-bearing inner services (e.g.ConcurrencyLimit,Buffer) are not double-consumed.tower::cache_singleflight—tx.send(result)now precedesmap.remove(&key)so late-arriving followers don’t become duplicate leaders. AddedLeaderDropGuardto clean up the in-flight map when the leader is cancelled; leader errors propagate to followers with variant preserved (no longer mapped toInternalError).tower::cache_key::ExactHashStrategy— usesahash::RandomState::generate_with(fixed_seeds)viaOnceLockso cache keys are deterministic across process restarts and distributed nodes (wasstd::hash::DefaultHasher, which is randomized per process).tower::idempotency::compute_body_hash— same deterministic-ahash fix as the cache key strategy; required for distributedIdempotencyStorebackends to function.tower::idempotency::IdempotencyService::call— store key is nowformat!("{tenant_id}:{idempotency_key}")so two tenants with the same idempotency key value do not cross-pollinate responses.tower::fallback_chain::FallbackChainService::call— drivessvc.ready().await?before eachsvc.call(...), restoring the Tower readiness contract for chain elements (previously silently bypassedConcurrencyLimitpermits).FallbackChainLayerimplementsLayer<()>with aprepend(head)helper forServiceBuildercomposition.tower::budget— window rollover is atomic via CAS; concurrent writers across the rollover boundary cannot torn-read a $0 window mid-zero-then-add.liter-llm-proxy::routes::realtime— the WebSocket upgrade now resolves the upstream credential via the per-modelProviderCredentialregistry instead ofgeneral.master_key; restricted virtual keys can no longer trigger Realtime sessions that bill the master key. The handler also callskey_ctx.can_access_model(&model)beforews.on_upgrade, so the model allowlist is enforced before any upstream connection is opened.liter-llm-proxy::secrets(aws.rs,vault.rs) — cache values are stored assecrecy::SecretStringso secrets are zeroed on TTL eviction; previously plainStringleft key material on the heap.liter-llm-proxy::secrets::vault—HashiCorpVaultProviderBuilder::buildvalidates the address against the outbound policy (validate_outbound_url_sync) before constructing theVaultClient; misconfigured addresses pointing at internal endpoints (link-local, loopback, RFC1918) are rejected withSecretError::Forbidden.guardrail::cel— CEL evaluation errors are no longer returned to the caller verbatim (could leak expression internals); callers see a fixed"policy evaluation error"reason while the full error is logged server-side viatracing::error!.http::transport::TransportConfig— re-exported at the crate root so the rustdoc example compiles; previously the type was reachable only via thepub(crate)httpmodule.- Trait re-exports —
CacheKeyStrategy,Guardrail,GuardrailContext,GuardrailDecision,GuardrailStage,VectorStore,VectorMatch,EmbeddingProvider,NoOpEmbeddingProvider,TenantId,TenantContext,KeyResolver,ResolvedKey,KeyResolverError,InMemoryKeyResolver,IdempotencyStoreErrorare now reachable fromliter_llm::*orliter_llm::tower::*without spelling out the full module path.
Security
Section titled “Security”- All findings from three rounds of critical security audit are resolved: per-model credential isolation on the Realtime route, model-allowlist enforcement before WebSocket upgrade, constant-time virtual-key lookup,
SecretStringfor cached secrets and provider credentials (zeroed on drop), SSRF outbound-policy guard activated at proxy startup, Vault address validation, CEL error redaction, fail-closed CEL guardrails, and JSON-tree redaction for regex guardrails.
[1.6.0-rc.0] - 2026-06-15
Section titled “[1.6.0-rc.0] - 2026-06-15”tower::circuitmodule —CircuitPolicytrait withExponentialBackoffCircuitdefault impl,CircuitStateenum (Closed→Open→HalfOpen),CircuitLayerandCircuitServicefor fault isolation. State transitions on configurable consecutive-failure threshold; half-open probes reset after configurable interval. (crates/liter-llm/src/tower/circuit.rs)tower::hedgemodule —HedgePolicytrait withFixedDelayHedgedefault impl,HedgeLayerandHedgeServicefor concurrent retry with jitter. Racesmax_attemptscopies staggered by fixed delay; cancels losers viatokio::task::JoinSet::abort_all(). Fast path whenmax_attempts == 1skipsJoinSetentirely. (crates/liter-llm/src/tower/hedge.rs)tower::metricsmodule —MetricsLayerandMetricsServicewith OTel-native GenAI semantic-convention meters (gated behindotelfeature with no-op fallback when disabled). Emits:gen_ai.client.operation.durationhistogram (request latency, success/failure/circuit-open labels),gen_ai.cache.{hit,miss,stale}counters,gen_ai.circuit.tripcounter,gen_ai.retry.attemptcounter, plusgen_ai.client.token.usagehistogram andgen_ai.request.cost_usdhistogram. Instruments cached inOnceLock<Arc<Instruments>>to eliminate per-request meter lookups. (crates/liter-llm/src/tower/metrics.rs)tower::routermodule —Weight(u32)saturating wrapper with NaN/Inf-safefrom_f64,UpstreamDiscovertrait alias,StaticDiscoverstream-based discovery,DynamicRouter<D>wrapping tower’sDiscovertrait with per-upstreamConcurrencyLimit(default 256).HealthCheckConfigstruct with interval/timeout/unhealthy_threshold/healthy_threshold,HealthCheckertrait,HttpProbeHealthCheckerdefault impl, andPerProviderHealthCheckservice for per-provider health status tracking. (crates/liter-llm/src/tower/health.rs,crates/liter-llm/src/tower/router.rs)http::transport::TransportConfig— exposed public module with configurable knobs:pool_max_idle_per_host(default 32),pool_idle_timeout(default 90 s),tcp_keepalive(default 60 s),http2_prior_knowledge(default false),dns_cache_ttl(default 30 s, best-effort — reqwest 0.13 lacks DNS TTL setter),enable_http3(default false, gated behindhttp3feature flag). Builder pattern with sensible defaults. Wired intoClientConfigvia newtransportfield;DefaultClient::newapplies all settings to reqwestClientBuilderexcept dns_cache_ttl. (crates/liter-llm/src/http/transport.rs)client::ClientConfig::transportfield of typeTransportConfigwith default impl for backward compatibility.liter-llm-cliruntime flags —--tokio-worker-threads Nand--tokio-max-blocking-threads Nfor runtime tuning, applied to bothapiandmcpsubcommands via explicittokio::runtime::Builder::new_multi_thread()(replaces#[tokio::main]macro). Defaults: physical CPU count (workers), 512 (blocking threads).liter-llm-proxy::shutdownmodule —ShutdownCoordinator,Drainabletrait,ShutdownPhaseenum (Idle→Draining→Drained/Aborted),DrainResultenum,ShutdownHandlefor signal handling and graceful shutdown. Signal pre-registration eliminates the miss window between first and second SIGTERM/SIGINT handlers.spawn_signal_handlerorchestrates two-signal escalation (first → Draining, second within 5 s or 30 s hard deadline → Aborted); concurrent drain viaFuturesUnorderedso slowDrainables don’t block faster ones. (crates/liter-llm-proxy/src/shutdown.rs)liter-llm-proxy::routes::healthmodule enhancements —/healthz(liveness: 200 always, never blocks) and/readyz(readiness: 200 if all probes pass, 503 otherwise).ReadinessProbetrait for composable health checks; built-in probes:ServicePoolProbe(at least one upstream configured),TokioQueueDepthProbe(injection queue depth < 1000). Probes run sequentially; pubrun_probes()allows custom implementations. (crates/liter-llm-proxy/src/routes/health.rs)util::boundsmodule — memory-budget guard constants (SSE_BUFFER_MAX_BYTES = 1 MiB,EVENT_STREAM_BUFFER_MAX_BYTES = 16 MiB,RESPONSE_BODY_MAX_BYTES = 32 MiB) andcheck_bound()helper for stream overflow detection (returnsErr(LiterLlmError::Streaming)withtracing::warn!if exceeded). (crates/liter-llm/src/util/bounds.rs)- Workspace
[lints.clippy]policy — denycorrectness/suspicious/perf; warnstyle. Document allow overrides:unused-unit(generated FFI),needless-pass-by-value(FFI ABI),module-name-repetitions(library ergonomics),missing-errors-doc,missing-panics-doc. (Cargo.toml) - Feature partition —
liter-llm: newlite(native-http only, no tower/opendal/tokenizer),http3,otel, and per-auth-method gates with explicit defaults and doc comments.liter-llm-proxy:otel,opendal-cache,proxy(named surface); default =proxy.liter-llm-cli:mimalloc,jemalloc(mutually exclusive allocator selection; defaults to system allocator). Workspace dependency pinning added for allocators. (Cargo.toml,crates/*/Cargo.toml) - Global allocator selection —
crates/liter-llm-cli/src/allocator.rsgates#[global_allocator]behindmimalloc/jemallocfeatures;compile_errorif both enabled simultaneously. tower::cache_keymodule —CacheKeyStrategytrait with three impls:ExactHashStrategy(SHA256 hash of full request),SystemPromptAwareStrategy(omits system-prompt field from hash),TenantScopedStrategy(includes tenant ID). All three hash deterministically viaserde_jsonto stable JSON. (crates/liter-llm/src/tower/cache_key.rs)crates/liter-llm/src/{embedding,vectorstore}modules —EmbeddingProvidertrait with two impls:SelfHostedEmbeddingProvider(calls local LLM endpoint for embeddings viaembed()method),NoOpEmbeddingProvider(returns zero vectors for unit tests).VectorStoretrait withInMemoryVectorStore(DashMap-backed with brute-force cosine similarity),OpenDalVectorStore(persists embeddings to OpenDAL backends, gated behindopendal-cachefeature). Both impls supportstore(key, embedding),retrieve_similar(query, threshold, top_k),delete(key). (crates/liter-llm/src/tower/vectorstore/{mod,memory,opendal}.rs,crates/liter-llm/src/tower/embedding.rs)tower::cache.rstrait extensions —CacheStoregainsset_ttl(key, ttl),iter_keys(),metadata(key) -> CacheMetadata(expiry, creation_time, hit_count). Default no-op bodies preserve backward compat for existing impls.CachedResponsenew variant:Error { error: Arc<LiterLlmError>, expires_at: Instant }for transient-error caching with customSerializeimpl that rejects persistence to external backends (in-memory only). (crates/liter-llm/src/tower/cache.rs)tower::cache_policymodule —CachePolicytrait withStandardCachePolicyimpl. Controls:bypass_cache()(per-request bypass),ttl()(seconds),semantic_similarity_threshold(0.85),stale_while_revalidate(5 minutes).CacheService::call()implements three-tier lookup: exact-hash match → semantic similarity viaEmbeddingProvider→ streaming-replay from stored chunks.warm(requests)async warming hook for batch pre-population. (crates/liter-llm/src/tower/cache_policy.rs)tower::cache_singleflightmodule —SingleflightCoordinatortrait withInMemorySingleflightimpl backed by DashMap. Coordinates concurrent identical requests: first caller blocks all followers; response broadcast viatokio::sync::broadcast. Eliminates thundering-herd when cache miss aligns with identical in-flight requests. (crates/liter-llm/src/tower/cache_singleflight.rs)tower::cache_negativemodule —NegativeCachePolicytrait withFixedWindowNegativeCacheimpl (caches transient errors only: retryable status 429/5xx, defaults to 60-second window).CachedResponse::Errorvariant with custom Serialize that prevents persistence to non-memory backends. (crates/liter-llm/src/tower/cache_negative.rs)tower::budgetmodule —BudgetLedgertrait withCostRecordContext,CostCheckContext,BudgetVerdict,BudgetDimensionenum (Global/Model/Tenant/User/ApiKey),BudgetSnapshotstruct.InMemoryBudgetLedgerimpl backed by DashMap per dimension;export_csv()for chargeback/reconciliation. Everyrecord_cost(context, usd_amount)call checks all applicable dimensions atomically. (crates/liter-llm/src/tower/budget.rs)tower::rate_limitmodule —CostRateLimitConfig { max_usd_per_minute, max_usd_per_hour, max_usd_per_day }andCostRateLimitLayer/CostRateLimitServicefor hard spend ceilings. Integrates withBudgetLedgerdimension checks; returnsErr(LiterLlmError::BudgetExceeded)when cost would exceed any ceiling.should_hedge()helper returns true/false based on cost and latency signals for intelligent hedging. (crates/liter-llm/src/tower/rate_limit.rs)tower::metricsOTel additions — new meters:gen_ai.budget.spend_usd(histogram, labeled by dimension),gen_ai.budget.rejection(counter, labeled by dimension + reason). Emitted byBudgetLedgerimpls andCostRateLimitService. (crates/liter-llm/src/tower/metrics.rs)guardrailmodule —Guardrailtrait (name,supported_stages,check(context) -> GuardrailDecision).GuardrailStageenum (Input/Output/OutputChunk).GuardrailDecisionenum (Allow/Block/Mutate).GuardrailContextstruct (request, response, reason). Built-in guardrails:RegexGuardrail,AllowListGuardrail,DenyListGuardrail,LengthCapGuardrail,PromptInjectionHeuristic(10-pattern keyword check, documented as heuristic not classifier).GuardrailRegistryglobal viaOnceLock<RwLock<…>>matchingprovider::custompattern.GuardrailLayer/GuardrailServiceTower wrapper — runs Input on request, Output on full response, OutputChunk per streaming chunk, short-circuits on Block. CEL policy DSL gated behindguardrail-celfeature viacel-interpretercrate; eval errors fail-open withtracing::warn!. (crates/liter-llm/src/guardrail/{mod,builtin,registry,cel,tests}.rs,crates/liter-llm/src/tower/guardrail.rs)tower::route_classifymodule —RouteClassifiertrait (classify(context) -> ClassifyResult,confidence_threshold). Built-in classifiers:KeywordClassifier(regex-pair → model),EmbeddingSimilarityClassifier(reusesEmbeddingProviderfrom 2.A),LlmClassifier(delegates to an LLM),CascadeClassifier(priority-ordered composition).ClassifierVerdictCachecaches verdicts viaCacheStore.RoutingStrategy::Semantic(Arc<dyn RouteClassifier>)variant intower/router.rs; falls back to round-robin when classifier defers. OTel meters:gen_ai.route.classify.durationhistogram +gen_ai.route.classify.tier{keyword,embedding,llm}.hitcounters. (crates/liter-llm/src/tower/route_classify.rs)- Type-state builder pattern —
ClientBuilder<HasApiKey, HasProvider>with marker typesNoApiKey/WithApiKey/NoProvider/WithProvider.build()only callable onClientBuilder<WithApiKey, WithProvider>(compile-time error otherwise). Enforces API key and provider selection before use. (crates/liter-llm/src/client/builder.rs) ProviderCapabilitiesstruct —vision,reasoning,structured_output,function_calling,audio_in,audio_out,video_inbools. Exposed viapub fn capabilities(provider_name: &str) -> &'static ProviderCapabilities. (crates/liter-llm/src/provider/mod.rs)- 142 provider schema entries updated —
crates/liter-llm/schemas/providers.jsonnow carries explicitcapabilitiesobject andstreaming_formatfield (“sse” everywhere except Bedrock = “aws_event_stream”) for every provider. Enables capability-aware client construction and streaming-format detection. (crates/liter-llm/schemas/providers.json)
Phase 3 — Realtime streaming, secret backends, credential rotation, and config hot-reload
Section titled “Phase 3 — Realtime streaming, secret backends, credential rotation, and config hot-reload”-
streamingmodule — unified ingress/egress streaming with three composable layers:IngressStream<S, P>(typed SSE decoder),StreamPipeline<S>(ordered per-chunk middleware viaChunkMiddlewaretrait),EgressStream<S>(typed OpenAI SSE encoder). When ingress format == egress format and no middleware is registered,EgressStreamenters passthrough mode for zero-copy forwarding without deserialise/re-serialise cycle.StreamFormat(SSE vs. AWS EventStream) promoted topubfor explicit wire-format selection. Per-threadBytesMutpool inEGRESS_BYTES_POOLthreadlocal reuses frame buffers under load.CancellationTokenthreaded through every layer; eachpoll_nextchecks it first for clean abort on client disconnect. (crates/liter-llm/src/streaming.rs) -
liter-llm-proxy::secretsmodule —SecretManagertrait (object-safe viaPin<Box<dyn Future>>) withget(name) -> SecretValue(field: zeroedSecretString+SecretMetadata),set(name, value, tags),delete(name). URI-scheme routing:env://NAME(always available),aws://PATH(requiressecrets-awsfeature),vault://PATH(requiressecrets-vaultfeature). Built-in impls:EnvVarSecretManager(environment variables),AwsSecretsManagerProvider(AWS Secrets Manager with key rotation warnings),HashiCorpVaultProvider(Vault KV-v2 with expiry tracking).SecretManagerRegistryroutes by scheme and holds one singleton per backend. OTel gaugegen_ai.secret.expires_in_seconds(gated behindotelfeature) emitted when secret expires within 24 h. (crates/liter-llm-proxy/src/secrets/{mod,env,aws,vault}.rs,crates/liter-llm-proxy/src/secrets/registry.rs) -
liter-llm-proxy::config::ConfigProvidertrait —load() -> ProxyConfig(single snapshot) andwatch() -> mpsc::Receiver<ConfigEvent>(live updates). Impls:StaticFileConfigProvider(TOML file, no hot-reload),FileWatchConfigProvider(OS file watch vianotifycrate),EtcdConfigProvider(distributed etcd key prefix watch withPut/Delete/Resyncsemantics).ProxyConfiginterpolation now supports${SECRET_URI}syntax sobase_url = "${env://ANTHROPIC_BASE_URL}"fetches at startup; secret rotation does not auto-reload URLs. (crates/liter-llm-proxy/src/config/{provider,watcher}.rs) -
liter-llm-proxy::provider::CredentialPooltrait — rotates per-provider API keys on 429/5xx rate-limit signals. Methods:current(provider) -> CredentialHandle(round-robin active credential),mark_exhausted(provider, handle, cooldown)(park for cool-down, advance to next),snapshot(provider) -> PoolSnapshot(observability: total/active/exhausted counts + next recovery time).InMemoryCredentialPoolimpl backed byDashMapwith per-credential cooldown state.ProviderCredentialstruct (modelProviderCredentialinVirtualKeyConfigwithid,api_key: String,model_allowlist) seeds pool entries from TOML. Decouples proxy credential cycling fromSecretManager— supports static inline keys and external secret backends interchangeably. (crates/liter-llm-proxy/src/provider/{credential_pool,credential_pool_memory}.rs) -
liter-llm::realtimemodule — unified envelope + event types for vendor-neutral realtime streaming.RealtimeEventenum (24 variants: SessionCreated, ConversationItemCreated, ResponseCreated, ResponseTextDelta, ResponseAudioDelta, ResponseFunctionCallArgumentsDelta, InputAudioBufferAppend, RateLimitsUpdated, Error, Raw, …).ContentPartenum (Text, Audio, ImageRef) used in conversation items.ResponseStatusenum (Completed, Cancelled, Failed, Incomplete).RealtimeEnvelopewraps event + optionalevent_id.RealtimeTranslatortrait for pluggable per-provider translation (maps wire format ↔ unified schema, object-safe, thread-safe). Built-in impl:openai::OpenAiRealtimeTranslator(1-to-1 mapping because OpenAI’s schema is already the reference shape).crates/liter-llm/src/realtime/{mod,openai}.rs) -
AppStaterefactor —configfield changed fromArc<ProxyConfig>toArc<ArcSwap<ProxyConfig>>for atomic hot-reload without blocking in-flight requests. Newsecret_registry: Arc<SecretManagerRegistry>field for resolving secret URIs in model configs. Callers must callstate.config.load()to obtain a consistent snapshot per request. (crates/liter-llm-proxy/src/state.rs)
Migration notes
Section titled “Migration notes”AppStatenow requiressecret_registry: Arc<SecretManagerRegistry>andconfig: Arc<ArcSwap<ProxyConfig>>fields. Applications usingProxyServer::builderare unaffected; manual state construction must update both fields.- New optional feature flags:
secrets-aws,secrets-vault,secrets-env(env backend always enabled, others optional).mimalloc,jemallocfor allocator selection.http3for HTTP/3 support.tokenizerforcount_tokensavailability. VirtualKeyConfiggains newprovider_credentials: Vec<ProviderCredential>field (defaults to empty). Inline credentials in TOML via repeated[[keys.provider_credentials]]blocks; proxy auto-rotates among them on 429/5xx.- Workspace clippy is now
-D warnings; downstream consumers compiling with strict lints should review suppressions — the main crate is now warnings-clean. LlmRequestpattern-matching change —LlmRequestwas previously an enum; it is now a struct with akind: LlmRequestKindfield.match req { LlmRequest::Chat(r) => ... }no longer compiles. Migrate tomatch req.kind() { LlmRequestKind::Chat(r) => ... }(preferred, using the newkind()accessor) ormatch req.kind { LlmRequestKind::Chat(r) => ... }(direct field access). The PascalCase constructor aliases (LlmRequest::Chat(r)etc.) remain callable for constructing requests and continue to compile unchanged; they are marked#[doc(hidden)]and will be removed in a future minor release.KeyResolver::resolvesignature change — the method now takesapi_key: String(owned) instead ofapi_key: &str, and returnsPin<Box<dyn Future<Output = ...> + Send + 'static>>instead of... + 'a. CustomKeyResolverimplementations must update their signature. Call sites that previously passed a&strliteral must add.to_owned():resolver.resolve("sk-...".to_owned()).
Changed
Section titled “Changed”- Bindings regenerated against alef v0.25.9; refreshes all 16 language surfaces, e2e suites, and README templates. New
[crates.e2e.fields_c_types]entrychat_completion_response.usage = "Usage"and per-call C# e2e overrideclass = "LiterLlmConverter"to satisfy alef v0.25.9’s stricter intermediate-accessor checks. tower/router.rs:WeightedRandomnow uses the newWeight(u32)saturating type (handles f64 NaN/Inf cleanly).DynamicRouterreplaces ad-hoc hardcoded routing with tower::discover integration.tower/health.rs: health-check configuration is now per-provider (HealthCheckConfig { interval, timeout, unhealthy_threshold, healthy_threshold }) instead of a single global setting.http/streaming.rs: SSE pipeline now propagates atokio_util::sync::CancellationTokenend-to-end viapost_stream_with_cancel()so client disconnect aborts the upstream stream cleanly. ThreadlocalBytesMutpool wired in for SSE frame buffers (currently used by tests; production callers will be added in Phase 2).cli/main.rs: explicittokio::runtime::Builder::new_multi_thread()replaces#[tokio::main]. Worker/blocking-thread counts now configurable.- Clippy policy enforcement — workspace-wide
cargo clippy --workspace -- -D warningsis now clean without per-crate suppressions. Narrower allow lists (correctness pass, style warnings only) reduce oversight surface. tower/cache.rstrait extensions —CacheStoremethod signatures extended withset_ttl(key, ttl),iter_keys(),metadata(key)with default no-op bodies; backward compatible with existing impls.CachedResponsestruct gainedError { error: Arc<LiterLlmError>, expires_at: Instant }variant with custom serialization.tower/router.rsRoutingStrategyenum — gainedSemantic(Arc<dyn RouteClassifier>)variant for classifier-driven routing. Removed#[derive(Debug)]and now has manualDebugimpl (dyn Trait is not Debug). Round-robin fallback when classifier defers.
- Test diagnostic clarity — all 417 test
.unwrap()calls replaced with.expect("descriptive message")naming the asserted invariant, improving failure diagnostics when assertions fire. Production code paths remain unwrap-clean. (crates/liter-llm-*/src/**) tower/circuit.rs:record_failure()no longer spawns a tokio task to flip state (uses synchronous CAS loop) — eliminates duplicate-spawn race under burst failure and removes the runtime dependency that maderecord_failurepanic outside async contexts.tower/hedge.rs:HedgeService::callnow honours the TowerServiceExt::ready()readiness contract viastd::mem::replace, so wrapping aConcurrencyLimit-protected upstream no longer silently bypasses the semaphore. Hedge fast-path (max_attempts == 1) skipsJoinSetentirely.tower/metrics.rs: instrument lookups cached inOnceLock<Arc<Instruments>>instead of constructed per-request. Removes ~8k redundant meter lookups/sec at 1k req/s production load.http/streaming.rs: deadBytesMutscratch field removed fromSseParser— was acquired from threadlocal pool but never read/written, pinning ~4 MiB across 1k concurrent streams. Pool helpers gated under#[cfg(test)]since production has no remaining callers.liter-llm-proxy/shutdown.rs: pre-registered SIGTERM/SIGINT handles eliminate the miss window between first signal returning and second-signal listener registering. Concurrent drain viaFuturesUnorderedensures slowDrainables don’t block faster ones before 30 s hard deadline.liter-llm-proxy/routes/health.rs:/readyznow uses stabletokio::runtime::RuntimeMetrics::num_alive_tasks()(originalinjection_queue_depthonly exists behindtokio_unstablecfg).- 1081 alef-generated file conflict markers —
git stash-introduced merge conflict markers (<<<<<<, ======, >>>>>>) systematically scrubbed from bindings, e2e suites, test_apps, and generated docs. The C#LiterLlmConverter.csFFI null-check pattern required manual resolution; workspace builds clean. (commit892500ec6) - Cache singleflight flake elimination — test race where the leader completed before followers attached to the broadcast channel eliminated via atomic
Arc<Broadcast>initialization before channel send. Fast mock services under parallel load now stable. (commit4e3a3e51e)
Tooling
Section titled “Tooling”- Workspace clippy lint policy enforcement via
[workspace.lints.clippy]blocks; per-crate suppressions consolidated at source. - Feature flag audit — split composite features (e.g.,
native-httpstill depends onhttp2, now gated on both); avoid silent breakage from feature interaction. - Allocator build variants —
BUILD_PROFILE=release task buildwith--features jemallocfor performance-sensitive deployments; system allocator is default for lighter containers. - New optional dep
cel-interpreter(~110 KB compressed) behindguardrail-celfeature flag for CEL policy DSL evaluation in guardrails module. regexworkspace dependency exposed — already present in transitive tree; now explicit forguardrail::builtin::RegexGuardrailandKeywordClassifier.
[1.5.1] - 2026-06-13
Section titled “[1.5.1] - 2026-06-13”Changed
Section titled “Changed”- publish workflow: migrate every push, release-asset upload, and homebrew-tap commit to the
kreuzberg-dev-publisher[bot]GitHub App viaactions/create-github-app-token@v2, replacingsecrets.GITHUB_TOKENandsecrets.HOMEBREW_TOKENwith scoped app installation tokens. - Bindings regenerated against the latest alef, refreshing all 16 language surfaces and e2e suites.
- Dart binding: named parameters and null-safety annotations, plus per-language README sync and updated method/type counts (#133).
- PyO3 0.29 method rename:
pyo3::Bound::downcast_intocallsites incrates/liter-llm-py/src/lib.rsmigrated to the newcast_intoname so the Python binding builds against pyo3 0.29. - PMD ruleset: exclude
UnnecessaryWarningSuppressionfromcategory/java/bestpractices.xml. Alef emits a blanket@SuppressWarnings("PMD")on every generated DTO record; PMD flags some as unnecessary depending on which rules fire on the surrounding record, breaking the Java hook on every regeneration.
[1.5.0] - 2026-06-07
Section titled “[1.5.0] - 2026-06-07”Security
Section titled “Security”External security audit identified six exploitable gaps in the v1.4.1 codebase. All six are fixed here with regression tests; releasing as a minor version because three of them change defaults.
- (F1, CRITICAL) Master-key constant-time comparison —
KeyStore::is_master_keypreviously compared the bearer token to the configured master key via==, exposing a per-request timing sidechannel. Now stores the master key insecrecy::SecretStringand compares viasubtle::ConstantTimeEq::ct_eqon the raw bytes. (crates/liter-llm-proxy/src/auth/key_store.rs, newsubtle = "2.6"dep incrates/liter-llm-proxy/Cargo.toml.) - (F2, HIGH, BREAKING) SSRF guard on outbound provider URLs —
CustomProviderConfig::base_urlaccepted arbitrary URLs and thereqwest::Clienthad no DNS-resolution policy, so a malicious custom-provider registration could point at127.0.0.1/169.254.169.254/ RFC1918 networks. Newliter_llm::provider::OutboundPolicy { Off, DenyPrivate, Allowlist(_) }chokepoint validates URLs at registration time and aGuardedResolverre-applies the policy per-request viareqwest’sdns_resolverhook, including redirect-hop validation. Library default isOff(back-compat preserves embedded/FFI behaviour); proxy default isDenyPrivate. NewLiterLlmError::OutboundForbiddenvariant maps to HTTP 502. New TOML key[security] outbound_policy = "deny_private" | "off" | { allowlist = ["…"] }. (crates/liter-llm/src/provider/outbound_policy.rs,crates/liter-llm/src/provider/custom.rs,crates/liter-llm/src/client/mod.rs,crates/liter-llm-proxy/src/config/server.rs,crates/liter-llm-cli/src/commands/serve.rs.) - (F3, HIGH, BREAKING) MCP per-tool model-access gate + HTTP transport auth — every
#[tool]handler incrates/liter-llm-proxy/src/mcp/mod.rs(chat, embed, list_models, generate_image, speech, transcribe, moderate, rerank, search, ocr, create_response, plus all file and batch management tools) now resolves aKeyContextfrom the rmcpRequestContext.extensionsand pre-flight-checkscan_access_model(¶ms.model)oris_masterbefore routing throughServicePool/FileStore. The HTTP/SSE MCP transport mounted incrates/liter-llm-cli/src/commands/mcp.rsis wrapped with the samevalidate_api_keymiddleware as the OpenAI endpoint, so virtual-key restrictions apply uniformly. Stdio transport requires an explicitmcp.stdio_key_id/mcp.stdio_trust_local = trueopt-in or refuses to start. - (F4, MED-HIGH) Error message sanitization — SSE error events and
ProxyError::from(LiterLlmError)previously embedded raw provider error strings viaDisplaywith no truncation or control-character handling. Newcrates/liter-llm-proxy/src/error.rs::sanitize_message(UTF-8-safe 200-char truncation, control-character strip except\t/\n) is applied at the singleFrom<LiterLlmError>chokepoint; SSE payloads now build viaserde_jsonrather than string interpolation, andProxyError::to_sse_payloadis the canonical serializer. - (F5, MED-HIGH) Mutex poisoning recovery —
SyncService::clone_service(crates/liter-llm/src/client/managed.rs) previously panicked when the innerstd::sync::Mutexwas poisoned. The lock guard only protects the clone step over aBoxCloneService, which isCloneand stateless across the lock, so recovery is safe: poisoned guards are now reclaimed viaPoisonError::into_innerand the next request proceeds normally. - (F7, MED-HIGH, BREAKING) CORS default is empty + wildcard origin loses Authorization header — the proxy’s
default_cors()is nowvec![]instead ofvec!["*"]; with nocors_origins, the router skips theCorsLayerentirely. Whencors_originsis set to"*", the wildcard branch restrictsallow_headersto a fixed list (CONTENT_TYPE,ACCEPT, …) and explicitly does not includeAuthorization— wildcard origins must not see credentialed headers per CORS-fetch spec.liter-llm-cli servealso logs atracing::warn!whencors_origins.contains("*") && host == "0.0.0.0".
Changed
Section titled “Changed”- Bindings regenerated against alef v0.23.28 (was v0.23.16). All 16 language surfaces — Python, Node, Ruby, PHP, Go, Java, Kotlin Android, C#, Elixir, WASM, C/FFI, Zig, Dart, Swift, R, Homebrew — and the e2e suites refresh end-to-end. The new alef ships my upstream java/magnus/go template patches (PMD braces, jinja whitespace,
MethodHandle.invokethrows Throwablewrap,data_enumclose-brace, magnus top-level module doc) plus the parallel agent’s brew/zig/php/dart/snippets/kotlin/swift fixes. - Tighter Rust clippy allow surface in the core and proxy crates: removed three unused
#[allow]annotations, the unusedget_jsonhelper incrates/liter-llm/src/http/request.rs, and a now-deadserde::de::DeserializeOwnedimport.cargo clippy --workspace … -- -D warningsis clean without the deleted suppressions.
Tooling
Section titled “Tooling”xberg-io/pre-commit-hooksbumped to v2.1.10 — picks up the consumer-sidealef-sync-versions --no-regenfix (full regen no longer fires on every commit), the palantir-java-format multi-platform sha256 manifest acceptance, the ktfmt checksum entry, and thegodoc-lint/golangci-lintgo.work-aware module discovery (no longer scans staletest_apps/swift_e2e/.build/checkouts/.../e2e/go/).- Project-local PMD ruleset at
packages/java/pmd-ruleset.xmlwired into thepmdhook to suppress alef-generated FFI patterns that PMD’s quickstart ruleset misflags (AvoidCatchingGenericException,PreserveStackTrace,CloseResource,UnusedLocalVariable,UnnecessaryFullyQualifiedName,VariableCanBeInlined,ReturnEmptyCollectionRatherThanNull). deny.tomlignoresRUSTSEC-2023-0071(Marvin Attack timing sidechannel inrsa@0.9.x, transitive viaopendal -> reqsign-core). No safe upstream version yet; the underlying RSA private-key signing path is not exercised on our network-observable code paths.alef-docs-freshhook and the CIVerify alef-generated code is up-to-datestep soft-disabled pending an alef v0.23.28inputs-hashregression fix —alef verifycurrently flags files as stale immediately after a freshalef allrun (the hash recomputed during verify disagrees with the hash written at emit time).markdownlint-rumdl-strictexclude expanded to cover the rootREADME.md(alef-generated badge row uses inline HTML),CONTRIBUTING.md,templates/readme/, and.github/PULL_REQUEST_TEMPLATE.md.
Migration notes
Section titled “Migration notes”The three behaviour-changing defaults above (cors_origins = [], outbound_policy = "deny_private", MCP per-tool model gate) are all reversible via explicit config. Operators who relied on the old defaults should add to their proxy config:
cors_origins = ["*"] # opt back into the v1.4.x wildcard CORS default[security]outbound_policy = "off" # opt back into the v1.4.x unguarded outbound HTTPVirtual-key holders who previously hit MCP tools without a model-access policy need their [[virtual_keys]] entries updated to include the model names they expect to call — or be granted is_master = true.
[1.4.1] - 2026-06-05
Section titled “[1.4.1] - 2026-06-05”- Docker build: removed stale
COPY tools/ tools/fromdocker/Dockerfile— thetools/directory was deleted in v1.3.0 and the unfixed copy was failing every Docker image build since. publish-cratesjob timeout: bumped.github/workflows/publish.yamlpublish-cratestimeout-minutesfrom 30 to 60. The 30-minute ceiling was cancelling mid-publish on busycrates.ioindex-propagation days, which (combined with the Python stdout buffering issue below) made cancelled runs look like silent failures with no per-crate log output.- Upstream
xberg-io/actionsto v1.8.29:publish-crates/scripts/publish.pynow line-buffers stdout/stderr (sys.stdout.reconfigure(line_buffering=True)), so per-crate “Publishing X (n/total)…” progress survives job cancellation. Before this fix, GitHub Actions’ block-buffered Python stdout swallowed all in-flight progress when the job hittimeout-minutes, hiding which crate was actually mid-publish.
- v1.4.0 was a no-op release because
task version:bumpwas not run before tagging — the tree still carried1.4.0-rc.61inCargo.toml, so every publish job either re-shippedrc.61artifacts (already on the registry) or failed verification looking for1.4.0. v1.4.1 is the first real1.4.xrelease. - alef pin advanced to
0.23.16(was0.23.12) — no functional codegen changes vs.0.23.12; bump tracks the latest released0.23.x.
1.4.0 - 2026-06-05
Section titled “1.4.0 - 2026-06-05”feat(provider/vertex): auto-install VertexAdcCredentialProvider in DefaultClient::new— when the resolved provider isvertex_aiand the caller supplied neither an explicitapi_keynor acredential_provider, the client now auto-constructsVertexAdcCredentialProvider::new()and installs it on the config. This is the canonical auth path for GKE Workload Identity / Cloud Run / Compute Engine deployments — short-lived OAuth2 tokens are fetched from the metadata server (with agcp_authADC discovery fallback for local development) and cached with a 5-minute pre-expiry refresh buffer. Pre-obtained tokens supplied viaapi_keyand explicitcredential_providers continue to take precedence. The ADC module is now reachable through thenative-httpfeature (gated behindnative-httpinstead ofvertex-adc, withvertex-adcretained as a back-compat alias).feat(provider/azure): per-modelbase_urloverrides for Azure deployments—[[models]]entries that pin abase_urlfor anazure/...provider_modelnow route throughAzureProvider::with_base_url(...), producing the required{base_url}/openai/deployments/{model}{path}?api-version=…shape instead of the generic OpenAI-compatible URL. Unblocks multi-resource Azure setups (different deployments per region/subscription). Closes #83.feat(wasm-backend): emit chat_stream returning JS async iterator— the WASM binding now exposesWasmDefaultClient.chat_stream(req)alongside the existingchat,embed, etc. The streaming adapter buffers the underlyingBoxStream<ChatCompletionChunk>into an array and returns it as aJsValue, mirroring the NAPI binding’s streaming semantics.- CLI binary tarballs (Linux x86_64/aarch64, macOS aarch64, Windows x86_64) attached to GitHub Releases for direct download — closes #64.
schemas/pricing.jsonregenerated from models.dev and now covers 4,219 models (up from 35);scripts/generate_pricing.pywired intotask generate:pricing,task update, andtask upgrade. Closes #48.Usage::prompt_tokens_details({ cached_tokens, audio_tokens }) deserialised from the OpenAI-compatible response body, pluscost::completion_cost_with_cacheand matchingcache_read_input_token_cost/cache_creation_input_token_costfields onModelPricing.ChatCompletionResponse::estimated_costand theCostTrackingLayernow bill cached prompt tokens at the provider’s discounted cache-read rate.schemas/pricing.jsoncarries cache-read/cache-creation costs for the 1,500+ models on models.dev that publish them. Closes #65.ci-mobile: new.github/workflows/ci-mobile.yamlrunningandroid-check(ubuntu,arm64-v8a+x86_64viacargo ndk),ios-check(macos,aarch64-apple-ios+aarch64-apple-ios-sim), andxcframework-build(macos, SPM-ready XCFramework + SHA256 checksum). Uses shared composite actions fromxberg-io/actions@v1.- Alef migration to v0.23.11: the entire polyglot surface (16 language bindings — Python, Node, Ruby, PHP, Go, Java, C#, Kotlin Android, Elixir, WASM, C/FFI, Zig, Dart, Swift, Homebrew + Rust core) is regenerated end-to-end via alef. Streaming (
chat_stream) is available across every applicable language, including Go (cgo channel bridge), Dart (FRB v2StreamSink<T>), and WASM. Skipped-assertion total across e2e suites: 354 → 0.
Changed
Section titled “Changed”- API rename:
ResponseClient::retrieve_response/cancel_responsenow take a parameter namedresponse_id(wasid). Positional callers are unaffected; named-arg callers must update. Consistent withfile_id/batch_idon the file and batch clients, and unblocks the alef-generated Python binding from shadowing theidbuiltin. - GitHub Release CLI assets ship a single sorted
SHA256SUMS-<version>.txtinstead of one.sha256per archive — closes #67. - WebAssembly build verified
mio-free.liter-llmexposes two mutually exclusive HTTP-stack features —native-http(reqwest + tokio + memchr + base64) andwasm-http(reqwest + memchr + base64 + gloo-timers, no tokio).liter-llm-wasmenables onlywasm-http; reqwest is pinned withdefault-features = false, features = ["json", "stream", "rustls", "multipart", "form"].cargo build --target wasm32-unknown-unknown -p liter-llm-wasmpulls neithermionortokio— reqwest auto-routes to the browser/NodefetchAPI onwasm32targets. - Ruby publish vendors core crates exclusively via the shared
xberg-io/actions/rewrite-native-deps@v1action (alefpublish prepare,vendor_mode = "core-only"). The bespokescripts/ci/ruby/vendor-liter-llm-core.py, the localruby:vendorTask, and theruby:builddependency on it are removed. - Repo hygiene:
.gitattributesmarks all alef-generated output directories (packages/**,crates/*-{py,php,ffi,node,wasm}/**,e2e/**) aslinguist-generated=trueso generated files collapse in GitHub PR diffs.
- TLS ABI floor: reqwest crypto provider switched from
aws-lc-rstoring(rustls-no-providerfeature + explicitrustlsdep withringbackend). Eliminates__isoc23_strtoland related glibc 2.38+ symbols emitted byaws-lc-sys0.40.0, restoring the GLIBC_2.28 ABI floor required by downstream users (e.g. Node.js aarch64 bindings). - HTTP retry jitter on
wasm32-unknown-unknown: the jitter calculation calledstd::time::SystemTime::now()which panics withRuntimeError: unreachableon bare wasm32 (std time is not implemented). Onwasm32the jitter step is skipped; native targets keep the existing[0.5x, 1.0x]jitter. Unblocks WASM e2e tests that exercise 429/5xx retry paths. - WASM and JNI bindings no longer fail to compile against the
tokenizer-gatedcount_tokens/count_request_tokensfunctions. Both nowexclude_functionsinalef.toml; apps that need token counting on those targets should call a server-side endpoint. - C/FFI header emits the opaque
typedef struct LITERLLMLiterLlmError LITERLLMLiterLlmError;referenced by theliterllm_liter_llm_error_{status_code,is_transient,error_type}accessors. - Java
ResponseObject/ResponseToolDTOs round-trip the full OpenAI Responses payload.ResponseOutputItem.contentis aList<…>(was a misalignedLinkedHashMap);ResponseToolacceptsdescriptionvia the@JsonAnyGetter/@JsonAnySetterflatten path. FixesMismatchedInputExceptionandUnrecognizedPropertyExceptionthrown bycreateResponse/retrieveResponse/cancelResponse. - Node (NAPI) streaming HTTP-init errors (400 content-policy, 401 unauthorized on
chatStream) now reject through the iterator. Binding remains lazy (parity with Python’sasync for _ in stream: pass). - Python
api.pywrapper emits the correct shape for non-streaming methods (22DefaultClientops). Previously every method was wrapped as a streamingAsyncIterator; onlychat_streamis genuinely streaming now. Also fixesString→strandbytes::Bytes→bytesmappings.
1.3.0 - 2026-04-23
Section titled “1.3.0 - 2026-04-23”Changed
Section titled “Changed”- Alef migration: All language bindings are now auto-generated by alef instead of hand-written
BoxFuture/BoxStreamtype aliases no longer wrapResult<T>— all method signatures now explicitly returnResult<T>providermodule is now public (waspub(crate))ChatCompletionRequest.streamfield is now public (waspub(crate))- Switched spell checker from codespell to typos
- CI no longer runs code generation — only
alef verify --exit-codefor freshness checks - Updated alef to v0.5.9
alef.tomlconfiguration for 10 language targets, 23 API method call configs, mock server supportbindings.rsadapter module withcreate_clientandcreate_client_from_jsonbinding-friendly constructorsDefaultderives on all public types for binding compatibilityClonederive onDefaultClient- E2E test fixtures converted to alef format (167+ fixtures across 23 categories)
- E2E tests regenerated for 13 languages with mock HTTP server support
- Test apps generated with
alef e2e generate --registry - API reference documentation auto-generated with
alef docsfor all 10 languages - Package READMEs generated with
alef readmeusing restored Jinja templates alef-verifyandalef-sync-versionspre-commit hooksalef verify --exit-codestep in CI validation workflow.lychee.tomllink checker configuration_typos.tomlspell checker configuration- Auto-load API keys from environment variables
- FFI callback streaming support
chat_streammethod across all bindings
Removed
Section titled “Removed”liter-llm-bindings-corecrate — replaced by alef codegentools/e2e-generatorcrate — replaced byalef e2e generatescripts/sync_versions.py— replaced byalef sync-versionsscripts/generate_readme.py— replaced byalef readmescripts/readme_config.yamlandscripts/readme_templates/— replaced bytemplates/readme/tests/test_apps/— replaced bytest_apps/(alef registry mode)- Hand-written binding source in
crates/liter-llm-{py,node,ffi,wasm,php}/src/ - Hand-written package source in
packages/{go,java,csharp,ruby,elixir}/
1.2.2 - 2026-04-18
Section titled “1.2.2 - 2026-04-18”- GitHub Copilot OAuth Device Flow credential provider (
copilot-authfeature) — use your Copilot subscription as an LLM backend viagithub_copilot/model prefix (#12) - GitHub Copilot provider with OpenAI-compatible routing, required Copilot headers, per-request UUID, and
X-Initiatorheader - E2E test fixtures for GitHub Copilot provider (chat + auth error)
- Provider registry audit: corrected base URLs for 20 providers (aiml, assemblyai, clarifai, dashscope, deepseek, elevenlabs, firecrawl, friendliai, gradient_ai, gmi, helicone, lambda_ai, minimax, moonshot, morph, nlp_cloud, ollama, poe, stability, wandb)
- Provider registry audit: corrected env var names for 5 providers (cometapi, fal_ai, gradient_ai, jina_ai, venice)
- Provider registry audit: corrected endpoint lists for 6 providers (cometapi, deepinfra, elevenlabs, jina_ai, mistral, nvidia_nim)
- Added missing
base_urlandauthconfig for 11 previously non-functional providers (amazon_nova, baseten, compactifai, datarobot, docker_model_runner, duckduckgo, langgraph, lemonade, v0, vercel_ai_gateway, zai) - Added 18 stub/infrastructure providers to
complex_providerslist to prevent incorrect config-driven routing - Added
nanogptparam mapping (max_completion_tokens→max_tokens)
1.2.1 - 2026-04-17
Section titled “1.2.1 - 2026-04-17”LlmClientRawtrait with_rawvariants of allLlmClientmethods, returningRawExchange<T>that exposes the final request body and raw provider response before normalization (#13)RawExchange<T>andRawStreamExchange<S>types for wire-level debugging and custom parsing- MCP & IDE integration documentation with setup guides for VS Code, GitHub Copilot, Claude Desktop, Cursor (#12)
- Docker image now published to
ghcr.io/xberg-io/liter-llm(#11) - Docker publish workflow timeout increased from 60 to 360 minutes (multi-arch Rust builds via QEMU were timing out)
- Bedrock
build_urltests no longer flake due toBEDROCK_CROSS_REGIONenv var race condition
1.2.0 - 2026-04-07
Section titled “1.2.0 - 2026-04-07”- Local LLM provider support: Ollama, LM Studio, vLLM, llama.cpp, LocalAI, llamafile – use any local inference engine via OpenAI-compatible API
- Docker Compose setup for local LLM integration testing with Ollama
- Integration test suite for local LLM providers
- PHP
onErrorhook now passes a proper\Exceptionobject instead of a plain string (PHP strict types requires\Throwable) - README templates fixed for rumdl compliance (MD040 code fence language, MD031 blank lines, MD032 list spacing, MD020 closed headings)
- Added 404 to all POST endpoint OpenAPI specs (model not found on default model names)
- Homebrew badge added to all READMEs
1.1.1 - 2026-03-29
Section titled “1.1.1 - 2026-03-29”- Java Maven plugins downgraded to 3.x stable (was 4.0.0-beta, incompatible with Maven 3.9.x CI)
- PHP hook isolation (per-client instead of global), budget per-model enforcement, onError hook invocation, shutdown segfault
- PHP e2e tests set
max_retries=0to prevent retry delays on mock 500s - OpenAPI spec: added 400/415/422/503 status codes to all endpoints for schemathesis compliance
first_client()returns 503 Service Unavailable instead of 500 for “no models configured”- Schemathesis CI checks aligned (removed
content_type_conformance,not_a_server_error) - Docker cache: per-platform
TARGETARCHcache IDs prevent multi-arch build races
- Homebrew formula:
brew tap xberg-io/tap && brew install liter-llm - Homebrew bottle builds (arm64_sequoia) in publish workflow
liter-llm-proxyandliter-llm-cliadded to crates.io publish pipeline- Installation docs: CLI/Docker/Homebrew tabs
scripts/publish/upload-homebrew-bottles.shandensure-github-release-exists.sh
1.1.0 - 2026-03-29
Section titled “1.1.0 - 2026-03-29”OpenAI-compatible LLM proxy server with CLI, MCP tool server, and Docker support.
Proxy Server (liter-llm-proxy)
Section titled “Proxy Server (liter-llm-proxy)”- 22 REST endpoints — full OpenAI-compatible API surface: chat completions (streaming + non-streaming), embeddings, models, images, audio (speech + transcription), moderations, rerank, search, OCR, files CRUD, batches CRUD, responses CRUD, health
- Tower middleware stack — reuses core middleware: cache, rate limit, budget, cost tracking, cooldown, health check, tracing
- Virtual API keys — in-memory key store with per-key model restrictions, RPM/TPM limits, budget limits
- Model routing — name-based routing to provider deployments, wildcard aliases, deterministic default client
- OpenDAL file storage — configurable backend (memory, S3, GCS, filesystem) for file operations
- SSE streaming — chat completion chunks proxied as Server-Sent Events with
[DONE]sentinel - OpenAPI 3.1 — utoipa-generated spec served at
/openapi.jsonwith bearer auth security scheme - TOML configuration —
liter-llm-proxy.tomlwith env var interpolation (${VAR}), auto-discovery,deny_unknown_fields - CORS — configurable origins from config (default: allow all)
- Graceful shutdown — SIGINT/SIGTERM handling via
tokio::signal
MCP Server (rmcp)
Section titled “MCP Server (rmcp)”- 22 tools — full parity with REST API: chat, embed, list_models, generate_image, speech, transcribe, moderate, rerank, search, ocr, file CRUD (5), batch CRUD (4), response CRUD (3)
- Transports — stdio (default) and HTTP/SSE via
StreamableHttpService - Parameter schemas —
schemars::JsonSchemaderives for MCP tool discovery
CLI (liter-llm)
Section titled “CLI (liter-llm)”liter-llm api— start proxy server with config, host/port overrides, debug loggingliter-llm mcp— start MCP server with stdio or HTTP transport- 3-tier config precedence: CLI flags > env vars > config file > defaults
Docker
Section titled “Docker”- Multi-stage build:
rust:1.91-bookwormbuilder,cgr.dev/chainguard/glibc-dynamicruntime (35MB) - Non-root execution, OCI labels, port 4000 exposed
ENTRYPOINT ["liter-llm"],CMD ["api", "--host", "0.0.0.0", "--port", "4000"]
Testing
Section titled “Testing”- 74 unit tests — config parsing, error mapping, auth key store, service pool, file store, streaming
- 32 integration tests — auth middleware, chat/embedding/models routes, error propagation, CORS, health, OpenAPI
- 12 proxy e2e fixtures — chat (basic + streaming), embeddings, models, auth errors, upstream errors, health, images, moderation, reranking
- Schemathesis — contract testing against OpenAPI spec via Docker (
task proxy:schemathesis)
.github/workflows/ci-docker.yaml— build + health test + schemathesis contract tests.github/workflows/publish-docker.yaml— multi-arch (amd64/arm64) publish toghcr.io/xberg-io/liter-llm- Taskfile:
proxy:test,proxy:schemathesis
1.0.0 - 2026-03-28
Section titled “1.0.0 - 2026-03-28”Initial stable release. Universal LLM API client with native bindings for 11 languages and 142+ providers.
LlmClienttrait with chat, chat_stream, embed, list_models, image_generate, speech, transcribe, moderate, rerank, search, ocrFileClient,BatchClient,ResponseClienttraits for file/batch/response operationsDefaultClientwith reqwest + tokio, SSE streaming, retry with exponential backoffManagedClientwith composable Tower middleware stack- 142 LLM providers embedded at compile time from
schemas/providers.json - Per-request provider routing from model name prefix (e.g.
anthropic/claude-sonnet-4-20250514) secrecy::SecretStringfor API keys (zeroized on drop, never logged)- TOML configuration file loading with auto-discovery (
liter-llm.toml) - Custom provider registration at runtime
Middleware (Tower)
Section titled “Middleware (Tower)”- CacheLayer — in-memory LRU + pluggable backends via
CacheStoretrait - OpenDAL cache — 40+ storage backends (Redis, S3, GCS, filesystem, etc.) via Apache OpenDAL
- BudgetLayer — global + per-model spending limits with hard/soft enforcement
- HooksLayer — request/response/error lifecycle callbacks with guardrail pattern
- CooldownLayer — circuit breaker after transient errors
- ModelRateLimitLayer — per-model RPM/TPM rate limiting
- HealthCheckLayer — background health probing
- CostTrackingLayer — per-request cost calculation from embedded pricing registry
- TracingLayer — OpenTelemetry GenAI semantic convention spans
- FallbackLayer — automatic failover to backup provider
- RouterLayer — multi-deployment load balancing (round-robin, latency, cost, weighted)
Language Bindings
Section titled “Language Bindings”All bindings expose the full API surface with language-idiomatic conventions:
- Python (PyO3) — async/await, typed kwargs, full .pyi stubs
- TypeScript / Node.js (NAPI-RS) — camelCase, .d.ts types, Promise-based
- Rust — native, zero-cost
- Go (cgo) — FFI wrapper with build tags,
context.Contextsupport - Java (Panama FFM) — JDK 25+,
AutoCloseable, builder pattern - C# / .NET (P/Invoke) — async/await,
IAsyncEnumerablestreaming,IDisposable - Ruby (Magnus) — RBS type signatures, Enumerator streaming
- Elixir (Rustler NIF) —
{:ok, result}tuples, OTP-compatible - PHP (ext-php-rs) — PHP 8.2+, JSON in/out, PIE packages
- WebAssembly (wasm-bindgen) — browser + Node.js, Fetch API
- C / FFI (cbindgen) —
extern "C"with opaque handles
Authentication
Section titled “Authentication”- Static API keys (Bearer, x-api-key)
- Azure AD OAuth2 client credentials
- Vertex AI service account JWT
- AWS STS Web Identity (EKS/IRSA)
- AWS SigV4 signing for Bedrock
Provider Transforms
Section titled “Provider Transforms”- Anthropic: message format, tool use v1, thinking blocks, max_tokens default
- AWS Bedrock: Converse API, EventStream binary framing, cross-region routing
- Vertex AI: Gemini format, embedding
:predictendpoint - Google AI: embedding/list_models response transforms
- Cohere: citation handling
- Mistral: API compatibility
param_mappingsfor config-driven field renaming (8 providers)
Documentation
Section titled “Documentation”- MkDocs Material site at docs.liter-llm.xberg.io
- 170+ code snippets across 10 languages
- 11 API reference docs with full method coverage
- Usage pages: Chat & Streaming, Embeddings & Rerank, Media, Search & OCR, Files & Batches, Configuration
- TOML configuration reference
- llms.txt (218 lines) with capabilities, examples, provider list
- Skills directory (4,072 lines) for Claude Code integration
- README generation from Jinja templates via
scripts/generate_readme.py
Testing
Section titled “Testing”- 500+ unit and integration tests
- Middleware stack composition tests (cache + budget + hooks + rate limit + cooldown)
- Per-request provider routing tests
- File/batch/response CRUD operation tests
- Concurrency tests (budget atomicity, cache contention, rate limit fairness)
- Redis cache backend integration tests (Docker Compose)
- Live provider tests for 7 providers (OpenAI, Anthropic, Google AI, Vertex AI, Mistral, Azure, Bedrock)
- Smoke test apps for all 10 languages against real APIs
- E2E test generation from JSON fixtures across all languages
- Contract test fixtures for binding API parity
- Multi-platform publish pipeline: crates.io, PyPI, npm, RubyGems, Hex.pm, Maven Central, NuGet, Packagist, Go FFI, PHP PIE
- Pre-commit hooks: 43 linters across all languages
- Post-generation formatting in e2e-generator
- Version sync script across 27+ manifests with README regeneration
Previous RC Releases
Section titled “Previous RC Releases”Release candidate history (rc.1 through rc.9)
- rc.1 (2026-03-27): Initial release — core crate, 11 bindings, e2e generator
- rc.2 (2026-03-27): Packaging fixes for crates.io, RubyGems, Elixir NIF, Node NAPI, publish workflow
- rc.3 (2026-03-27): Cache, budget, hooks middleware; custom providers; TDD e2e fixtures
- rc.4 (2026-03-28): Shared bindings-core crate; camelCase conversion; real streaming across all bindings
- rc.5 (2026-03-28): OpenDAL cache; search/OCR endpoints; full middleware wiring; Go/Java/C# FFI rewrites; serde deny_unknown_fields; documentation overhaul
- rc.6 (2026-03-28): Full API documentation coverage; Rust crate README; version sync improvements
- rc.7 (2026-03-28): Binding parity (5 middleware params + search/ocr in all 10); contract test fixtures; skills directory; PHP PIE packages
- rc.8 (2026-03-28): CI fixes (PHP publish, crate order, Maven GPG, Ruby deps, Bedrock test)
- rc.9 (2026-03-28): Live provider tests; Anthropic/Bedrock/Google streaming fixes; TOML config loading; per-request provider routing; integration test suite