janas-server

janas-server model.jns [options]

janas-server puts a Janas-LLM model behind an HTTP API that follows OpenAI's, on `http://127.0.0.1:8080/v1` by default, so SDKs, chat front ends, and editors written for that API can use a model running on this machine.model is a `.jns` file (or a catalog name resolved the same way as janas-chat). It takes the model options the chat takes (--ctx, --cache, --reserve, --mtp, --draft, --mode, --no-gpu) plus listen, auth, FIM, and store flags.Routes that answer:- `GET /v1/models` and `GET /v1/models/{model}`- `POST /v1/chat/completions` (tools, JSON output, log-probabilities, streaming)- stored completions under `/v1/chat/completions/{id}`- `POST /v1/completions` (raw continuation, including a `suffix` for fill-in-the-middle)- `POST /infill` (llama.cpp's route, which llama.vscode asks)- `POST /v1/responses` and `/v1/conversations`- `POST /v1/embeddings` when --embedding-model is set- `POST /v1/moderations` (asked of the chat model)- `GET /health` (`{"status":"ok"}`)Every other OpenAI operation answers 501. Default bind is loopback. --api-key (or JANAS_API_KEY) requires `Authorization: Bearer KEY`. --test serves *Janas-Chat Web* at `/test`, a page inside the program that talks only to this server.A client that resends the same messages plus a new one pays for the new tokens alone; `usage.prompttokensdetails.cached_tokens` says how much was already computed. Conversations kept in memory (--keep) and on disk (--keep-disk, --store) outlive a single request. Tools of type `mcp` in `/v1/responses` make the server reach the MCP server the request names; --no-mcp turns that off.

model.jns

Chat model to load.

Listen address (default 127.0.0.1; 0.0.0.0 opens it to the network).

Port (default 8080).

Require this bearer key (JANAS_API_KEY sets it too).

Model id for clients (default: the file's name without `.jns`).

Context in tokens (default 16384).

Expert-cache size, and memory left to other programs when the cache is automatic.

Prediction file, none, or a small draft model of the same family.

Power mode (default auto).

Never give the GPU work.

Second model for `/v1/embeddings` (for example Qwen3-Embedding-0.6B).

Reasoning when a request does not say (default: the model's own).

Extra conversations kept computed besides the one in use (default 8; 0: none).

Memory and disk for those conversations (disk default 8 GiB under `~/.cache/janas`).

Where stored completions, responses, and conversations live (default `~/.local/share/janas/server`).

Keep them in memory only until the process stops.

Requests allowed to wait (default 16).

Largest request body (default 32).

Open connections at most (default 64).

Longest input in tokens; longer requests get HTTP 400. Nothing is cut to fit.

Longer inputs are taken and logged on stderr.

Fill-in-the-middle window for editor completions (defaults 2048 tokens before the cursor, 8 lines after, 256 tokens written).

JSON lines of what every reply costs (never the text).

A line on stderr every S seconds while a long prompt is read.

Refuse tools of type `mcp` in `/v1/responses`.

Serve the built-in chat page at `/test`.

One line per request on stderr.

Default listen is loopback; do not bind 0.0.0.0 without --api-key. --no-mcp is the switch that stops a reachable server from opening outbound MCP connections a request names. OpenAI's `maxcompletiontokens` counts reasoning; the older `max_tokens` of a chat is taken for the answer alone. Models with a recurrent state (Qwen3-Next, Qwen3.5, Qwen3.6) re-read a conversation from the start when earlier turns differ. The process is experimental, x86-64 Linux or Windows, and hard on RAM and disk like the rest of Janas.

janas-server ships with Janas, written by Maurizio "camauri" Cammalleri (first public release September 2026, GNU GPL 3 or later). It is the only Janas program that compiles in GNU libmicrohttpd and yyjson.

janas-chat(1), janas-mcp(1), janas-get(1), janus(1), ollama(1), vllm(1), curl(1)

Source code · Documentation