deltafin

deltafin [run] [options]deltafin serve [options]deltafin setup [--full | --stream] [options]deltafin doctor [--runtime-only | --model-root path]deltafin upgradedeltafin benchmark [options]deltafin setup-dspark | setup-k3 | setup-qwen [options]deltafin fetch-weights | warm-expert-cache | pack-spine | convert-spine-int8 | convert-experts-scale4 [options]

deltafin is a single native Rust binary that runs Moonshot AI's Kimi K3, a 2.8-trillion-parameter mixture-of-experts model, on one consumer machine. It keeps all 16 routed experts and uses K3 itself as the sole authority for every token. Small draft models (DSpark for chat, optional Qwen for raw completion) may guess ahead; K3 verifies every guess before text is emitted.The runtime is in-process: Rust core, C-ABI providers, native tokenization, and expert streaming from disk. It does not prune, requantize the expert bank to a few bits, or skip experts. Setup downloads pinned K3 weights (full local corpus or on-demand streaming), the DSpark checkpoint (~6.6 GiB on disk), and optionally Qwen (~4.3 GiB). Converted resident storage defaults to a row-int8 spine. The original BF16 checkpoint stays on disk as the conversion source and verification authority, and is used for inference only with --spine bf16.serve exposes a deliberately small, strictly checked subset of the OpenAI API: text-only, one generation at a time, always greedy and reproducible. Fields it cannot honor exactly are refused with an OpenAI-shaped error rather than silently ignored. Point a client at http://127.0.0.1:8000/v1 with any non-empty local API key the client requires. The advertised model name is deltafin-kimi-k3.Deltafin ships as a Git source tree, not crates.io packages. Build with cargo build --locked --release; the binary lands at target/release/deltafin. The default model root is the current working directory.

run (default)

Generate text. Omitting a subcommand is the same as run. Default prompt is The capital of France is.

Text to continue (raw completion) or the user message (with --chat).

Apply K3's audited chat template and stop at the model's end marker. Raw completion has no chat end boundary and should normally use --max-new.

Stop after at most n generated tokens. Does not alter the prompt or context.

Chat thinking depth. Falls back to K3_REASONING_EFFORT. Template default is max.

Print a live cumulative throughput line per token instead of streaming generated text.

Print a per-layer, per-chunk phase breakdown (~93 lines per token). Independent of --stats.

auto, cpu, mps, cuda, or cuda:N. Falls back to K3_DEV.

Routed-expert backend: auto, cpu, metal, or cuda. Falls back to K3_MOE.

Resident spine format. auto and int8 use the quantized row-int8 default; bf16 keeps the original weights. Falls back to K3_SPINE.

Install and model root (default: current directory, or DELTAFIN_ROOT for setup/doctor).

Append native expert-route JSONL (relative to the model root).

OpenAI-compatible HTTP server implementing /v1/chat/completions, /v1/completions, and /v1/models, including SSE streaming.

Listen address (default 127.0.0.1).

Listen port (default 8000).

Per-response ceiling (default 1000000).

Concurrent requests allowed to wait for the single generation slot (default 0 = immediate 429; max 64). Falls back to K3_SERVER_QUEUE.

Bounded JSON request size (default 134217728, max 1073741824).

One-shot install of K3 weights and the DSpark draft checkpoint. With neither --full nor --stream, chooses full when disk allows, else stream.

Require the complete local expert pool (fastest inference; about 1.7 TB).

Install the resident spine; fetch exact experts on demand (about 215 GB to start). Mutually exclusive with --full.

Also install the optional Qwen 0.6B/1.7B proposal-only models used for raw completions.

Plan exact downloads and peak disk use without network or writes. Mutually exclusive with --check.

Audit that the capacity-selected installation is complete, without network access.

Parallel HTTPS transfers (default 8, maximum 16).

Install (or --check) the optional Qwen draft models used only for raw /v1/completions and run --prompt.

Install or verify Inferact's Kimi-K3-DSpark checkpoint. --check is network-free; --audit-only inspects pinned remote metadata without the tensor payload.

Requires exactly one of --meta-only (pinned inert K3 metadata) or --check.

Read-only installation audit. --runtime-only validates the host, compiled executable, provider ABI, and native canaries without opening model data.

Fetch runtime updates and rebuild the binary. Leaves models, converted weights, and caches alone. Accepts no arguments. Reuses the recorded build profile (CPU vs CUDA); it ignores build environment variables.

Interleaved A/B runs with optional token-id or text oracles. Default --max-new is 4, --reps is 3, timeout is 3600 seconds.

Download selected tensors. --spine-only and --experts-only are mutually exclusive; --layers 1-40,45,92 selects routed expert layers.

Rank and optionally fetch missing experts from router traces. --convert-npz losslessly migrates legacy six-member NPZ experts to raw .bin.

Pack the resident spine into contiguous DFSP files, convert the spine to row-int8 (quantized, not weight-exact), or build lossless scale4 expert sidecars.

Print the full help document.

Print the binary version.

Normal operation needs no environment overrides. Flags win over environment values.--device / K3_DEV

auto, mps, cuda, cuda:N, or cpu. Capability-gated auto selection.

auto, metal, cuda, or cpu.

auto and int8 select the quantized row-int8 default (not weight-exact). bf16 keeps the original weights. If the int8 spine is missing, auto fails rather than silently substituting BF16.

auto, off, or force-qualified on. K3 verification is never bypassed.

Bounded auxiliary draft-state context (default 8192); full K3 continues above it.

Optional Qwen raw-completion policy: auto, off, or on.

Chat thinking depth: low, high, or max. On the server, the per-request reasoning_effort field overrides it.

Native router trace mode and path; CLI --router-trace flags are preferred.

auto, off, or require for complete lossless scale4 sidecars.

Adaptive admission for speculative expert reads: on, measure, or off (default).

Decimal GB budget for a permanent learned-expert RAM tier on CPU/Metal (0 keeps it off).

Fallback for serve --queue when the flag is omitted.

Default install/model root for setup and doctor when --model-root is omitted.

Absolute path to a CUDA-enabled LibTorch tree. Used only when building NVIDIA/CUDA (or ROCm) binaries; upgrade ignores these and rebuilds with the recorded profile.

Full K3 is extreme on home hardware. A complete expert corpus is about 1.7 TB; streaming still needs about 215 GB plus growing on-disk expert cache. Decode on an M1 Max laptop is on the order of 0.3 token/s; even a four-SSD Apple Silicon setup is around 1 token/s. Prefill (time to first token) can be minutes. Long chats are much slower than short completions.Supported hosts are macOS 14+ on Apple Silicon and Linux with glibc 2.28+ on x86-64 or aarch64. Windows is not supported. macOS needs full Xcode plus the Metal toolchain (xcodebuild -downloadComponent MetalToolchain); Command Line Tools are not enough. CUDA builds require a user-supplied CUDA LibTorch tree and NVCC; the pinned CPU runtime is not CUDA-capable. ROCm/HIP expert kernels exist but have no hardware evidence; default AMD expert path is CPU MXFP4.serve has no authentication. Keep it on loopback unless you add your own network boundary. It is greedy-only and single-generation. upgrade refuses unexpected git state and will not switch CPU/CUDA/HIP configuration by itself. convert-spine-int8 is quantized and not weight-exact. Model weights, DSpark, and Qwen retain their upstream licenses; only Deltafin's tracked project code is MIT.

deltafin is an independent MIT-licensed project under the gavamedia GitHub organization, first published in July 2026. It began as a Python prototype and was rewritten as a Python-free native Rust runtime so full Kimi K3 could run on a single device without pruning the expert bank. Kimi K3 itself is released by Moonshot AI; Deltafin is not affiliated with Moonshot. Community Linux/CUDA work landed via pull request, and later forks (including ARGODRIVE streaming layouts) reuse the same CLI.

ollama(1), llama.cpp(1), llama-cli(1), llamafile(1), vllm(1), kimi(1)

Source code · Documentation