SparkDuet
Run DeepSeek, Qwen, and your own fine-tunes on two NVIDIA DGX Sparks: one endpoint, four lanes, honest numbers. The operating layer the Lab runs on its own pair daily, released MIT so your pair can run it tomorrow.
ONE ENDPOINT · FOUR LANESWhy This Exists
The problem, then the ideaThe problem
Two DGX Sparks give you 256 GB of unified GPU memory, a 200 Gb/s private interconnect, and a problem: every published recipe treats the pair as one big GPU for one model, all day, whether or not that is what your work needs.
The idea: lanes
SparkDuet treats the pair as a small cluster with lanes. Pick the topology per model and per workload, switch between them in one command, revert in one command, and measure everything in a way you can publish without embarrassment.
What that means in a day
On the same two boxes, in the same day: serve the 284B DeepSeek-V4-Flash across both nodes, keep a library of smaller models loading on demand without touching the flagship, fine-tune up to 70B with QLoRA on the idle node, and put it all back exactly the way it was.
Who benefits
Anyone pointing coding agents, team serving, or a private assistant at local hardware: OpenCode, Cursor, the DeepSeek CLI, and Hermes configs ship in the runbook. Everything speaks the OpenAI API on one port, so your tools point at it without client-side code.
The Architecture
One endpoint · four lanes
Lane D · Depth (TP=2)
One model too big for one box, sharded across both nodes over the 200G RoCE link. This is how the pair serves the 284B-parameter DeepSeek-V4-Flash with DSpark speculative decoding at 128K context. Worker-first launch, NCCL preflight gate, warm-up before first traffic.
Lane F · Fleet (DP=2)
Two independent replicas of a model that fits one node, one per Spark, load-balanced by the router. No cross-node collective on the serving path: a node failure degrades capacity by half instead of taking the service down. The right lane for agent fleets and team serving.
Lane P · Split (experimental)
Prefill/decode disaggregation: one node ingests long prompts, the other holds the KV pool and decodes. The 200G link moves a 128K-token KV in under a tenth of a second. Ships behind an explicit enable flag with its failure semantics documented.
The fine-tune lane
The node that is not serving is a 128 GB training box: QLoRA to about 70B, LoRA to 27B, full fine-tune to 7B, with a three-minute smoke gate before you commit a weekend. Both nodes together run distributed fine-tunes over the same fabric NCCL uses for serving.
The Fit Rule
~121 GiB GPU-usable per nodeEverything in the repository follows from one number: a DGX Spark exposes about 121 GiB of GPU-usable unified memory. An earlier ecosystem claim that the FP8 flagship loads in 79 GiB on one node misread a per-rank number; the checkpoint does not fit one Spark. The lane configs enforce the rule instead of letting you discover it at OOM time, twenty minutes into a model load.
Measured, Projected, or Hearsay
Every number carries a labelEvery number in the repository carries a label: [M-here] was measured on this pair with the JSON artifact committed, [M-else] is third-party and cited, [P] is a projection with its derivation shown. The benchmark protocol is binding: token counts come from usage frames, never from counting SSE chunks; TTFT is reported p50/p95; short runs are refused.
Read the spread as the headline: the same deployment is 72 tok/s on math and 34 tok/s on prose, because speculative decoding lives and dies by draft acceptance. Any two-Spark tok/s claim that omits the workload class and speculation status is not comparable to anything.
How You Run It
Gates before start · revert built inInstall and start
Clone, run install.sh (it detects fabric interfaces, writes the env, syncs the worker, runs the gates), then sparkduetctl start. Start refuses to run until doctor passes: SSH, fabric, RDMA, disk, and images are proven before a model loads.
Revert is a first-class verb
Every start captures the state needed to revert, and stop returns the boxes to exactly what ran before. If you already serve something you care about, capture-incumbent snapshots every running container's full spec to a dated file first.
Point your tools at it
Everything speaks the OpenAI API on one port. Working configs for OpenCode, Cursor, the DeepSeek CLI, and Hermes ship in the runbook, and the router exposes lanes as model suffixes or a header, so a harness pins a lane per request without client-side code.
Running it cool
A pair at full serve is warm and audible; there is no reason to keep it there around the clock. Stop drops idle draw to single-digit watts per GPU, the on-demand library loads models only while they work, and off is a supported state with fast cold-boot from persisted caches.
git clone https://github.com/zorost/sparkduet && cd sparkduet ./install.sh # env, worker sync, gates ./scripts/sparkduetctl.sh doctor # SSH, fabric, RDMA, disk, images ./scripts/nccl-check.sh --full # proves a real 2-node all-reduce ./scripts/sparkduetctl.sh start depth # worker first, head, health gate ./scripts/warmup.sh # do not skip python3 scripts/bench.py --suite standard --lane depth
When the Next Model Drops
Branches, not rewritesModel identity lives in env
DeepSeek-V4-Flash is the flagship today, not forever. Model identity lives in environment configuration, not in code, so swapping the flagship never means editing the operating layer.
One branch per recipe
Each complete recipe lives on a model/* branch: model/deepseek-v4-flash-0731, model/qwen3.8-27b, and yours next. A branch carries the lane configs, the retuning table, and the measured artifacts for that model, so recipes never overwrite each other.
The 30-minute swap
docs/MODEL-SWAP.md is the checklist: download against the fit rule, pick the lane, retune the handful of knobs the table names, run the bench suite, commit the artifact. A new flagship is a branch and an afternoon, not a migration project.
What lands back here
The Lab runs new recipes on its own pair first; measured branches graduate into the register and this page's numbers update with dated artifacts behind them. Contributions follow the same law: a recipe without measurements is a draft.
The Engineering Discipline
Honest about what it is notError analysis over vibes
The project's habit is the AI-engineering habit: look at the real counters, find the dominant failure, fix that. The acceptance-counter scraper bug the Lab shipped was caught and corrected the same night, and the artifact documents the exact correction.
Evals as the gate
The bench harness refuses short runs and missing counters, and the fine-tune lane's smoke test asserts loss actually falls before a weekend is committed. Numbers that cannot defend themselves do not get published.
What it is not
Not an inference engine: vLLM does the serving. Not a managed product: no auth, no TLS, no multi-tenancy; put your own gateway in front. Not a benchmark-winning machine: where another recipe is better for your case, the comparison doc says so and links it.
Standing on shoulders
The two-node DeepSeek-on-Spark lineage is credited in CREDITS.md and sourced in the research doc. SparkDuet's contribution is the lane model, the fit-rule honesty, the fine-tuning integration, and tooling that refuses to publish a number it cannot defend.
Questions
Answer-firstWhat is SparkDuet?
An open source operating layer for two NVIDIA DGX Sparks: it serves DeepSeek, Qwen, and your own fine-tunes behind one OpenAI-compatible endpoint with four lanes, keeps an on-demand model library, runs QLoRA fine-tuning on the idle node, and measures everything with a benchmark protocol strict enough to publish. MIT licensed, at github.com/zorost/sparkduet.
Why does a two-Spark pair need an operating layer at all?
Because every published recipe treats the pair as one big GPU for one model, all day, whether or not that is what your work needs. SparkDuet treats the pair as a small cluster with lanes: pick the topology per model and per workload, switch in one command, revert in one command.
How fast is DeepSeek on two DGX Sparks?
Measured on this pair with artifacts committed: DeepSeek-V4-Flash FP8 across both nodes runs 72.2 tok/s on math, 68.1 on code, 51.2 on tool calls, and 33.6 on prose at single concurrency, because speculative decoding lives and dies by draft acceptance. Any tok/s claim that omits workload class and speculation status is not comparable to anything.
What is the fit rule?
A DGX Spark exposes about 121 GiB of GPU-usable unified memory. The 156 GiB FP8 flagship therefore does not fit one node and must run tensor-parallel across both; a 29 GiB model fits easily and can run as a replica per node. SparkDuet's lane configs enforce the rule instead of letting you discover it at out-of-memory time, twenty minutes into a model load.
What happens when the next model drops?
Model identity lives in environment configuration, not in code, and each complete recipe lives on a model/* branch (model/deepseek-v4-flash-0731, model/qwen3.8-27b, yours next). The repo ships a 30-minute swap checklist and a retuning table in docs/MODEL-SWAP.md, so a new flagship is a branch and an afternoon, not a rewrite.
Can I fine-tune on the same pair that serves?
Yes; that is half the point. The node that is not serving is a 128 GB training box: QLoRA up to about 70B, LoRA to 27B, full fine-tune to 7B, with a three-minute smoke test that asserts loss actually falls before you commit a weekend. The verified smoke run took loss from 2.12 to 0.21 in 81 seconds.
What is SparkDuet not?
Not an inference engine (vLLM does the serving), not a managed product (no auth, no TLS, no multi-tenancy; put your own gateway in front), and not a benchmark-winning machine. Where another recipe or engine is better for your case, docs/COMPARISON.md says so and links it.
Serving those who
need to stay ahead.
We don't pitch slide decks. We show you what we've already built in your domain, then engineer what your mission requires.