llama-swap added support for establishing live voice connections before choosing a model and for exposing project documents as AI tools. This can reduce connection errors caused by missing model names. It also lets help-agent developers reuse document integration features.
1. Select and route to a model after establishing a bidirectional WebSocket connection
This change targets clients using OpenAI's documented real-time API connection method without specifying a model name upfront. Such requests previously hit a 404 error before the WebSocket upgrade. In v271, llama-swap completes the connection first and forwards requests once a model is selected.
When a client connects for transcription, the server buffers incoming events until a model is specified in session.update. It checks session.model first, then the transcription model key if absent. It forwards buffered events through the existing routing path.
The approach accepts the connection first, then passes queued requests to the model once it is selected. Processing failures are reported as OpenAI-format error events.
2. Expose documents and optional source code as MCP tools
v271 also adds docbot, a package exposing llama-swap documents as MCP tools. MCP lets AI applications request external capabilities as tools. Help-agent developers can import the document-serving package, attach it to an HTTP route, and optionally expose source code.
The package includes a stateless MCP handler. Its SystemPrompt() and Tools() helpers provide a system prompt and tool list. Developers can use it by attaching document tools to their application's HTTP routes.
The release notes describe the package as groundwork for the llama-swap.ai help agent and include that site's help MCP endpoint. It bundles connectivity features for reuse in document-based agents.
3. Released training materials and evaluation results
The release also documents fine-tuning a small help-agent model, refining its behavior through additional training. Training scripts, conversations, and run notes were published as references for future help-model training workflows.
According to the release notes, LFM3-1.7B-based lfm3-v2 Q4_K_M scored 95.5% on training cases and 96.1% on held-out cases under a 60-second limit per problem. The results cover both training questions and questions set aside from training.
A separate document-agent evaluation added 66 training cases and 29 held-out cases and corrected four faulty checks. The release notes say document and evaluation updates raised gemma-4-12B-r9700's training score from 96.0% to 97.3%.
The default output limit for the get_doc document tool also rose from 250 to 400 lines. The release notes cite truncation of a kubeswap GPU budget configuration example. Document links were also cleaned up to point to guides containing answers.
The released training materials do not include weights, the numerical information learned by a model. The two models cannot be ranked directly because their scores were not presented as having identical evaluation conditions.
4. What changes for developers and users
Document tools bring both benefits and costs. A larger default output can preserve previously truncated examples, but callers must process more text at once. Help systems therefore need to handle results according to their required scope.
Teams building their own help systems can use both document-tool integration and model-training references. They can import the document-serving package and consult published conversations and run notes when tuning their own help models.
v271 centers on two extensions: routing real-time requests to the model selected by the session and exposing documents as reusable AI tools.