SparkDuet
Four frontier residents on two NVIDIA DGX Sparks. One OpenAI port. One command to swap. DeepSeek, Qwen 27B, Flash-Next, and GLM-5.3 stay on disk. RAM holds one. The Lab runs this pair daily. The recipe is MIT so your pair can run it tomorrow.
FOUR MODELS · TWO SPARKSThe Noise, Then the Product
Uninstall, buy a second pair, or switchTwo DGX Sparks give you 256 GB of unified memory and a 200 Gb/s private link. Most published recipes still treat that as one GPU for one model, all day. When the next checkpoint lands, the owner uninstalls a 150 GiB resident to make room, or buys a second pair so DeepSeek and GLM never share a wall. That is the noise. SparkDuet is the operating layer that ends it.
You stage every resident on NVMe. RAM holds one. switch drains the incumbent, boots the next lane, and warms it. revert puts the last incumbent back. The OpenAI port never moves. OpenCode, Cursor, Chat, and Hermes keep the same URL. The Lab runs this pair daily. The recipe is MIT so a second pair can copy the Lab instead of copying a screenshot.
Uninstall and reinstall
Weights leave disk. The next try costs another download, another first-boot hour, and a night when nothing serves. Fine for a hobby box. Wrong for a desk that has to answer in the morning.
Buy another pair
Two more Sparks keep a second MoE warm. Capital doubles. Power doubles. You still have no verb that moves a resident between boxes. Isolation is what you bought.
Switch on the same two
Four residents stay on disk. One lives in RAM. A swap is minutes you can publish, not a rebuild. The on-demand library still loads smaller GGUF and merged fine-tunes. The idle node still trains.
Who this is for
Teams pointing coding agents or a private assistant at local metal. One port. Four named lanes. Honest boot times. A public recipe the Lab already runs.
Four Residents, Two Sparks
One endpoint · one live laneClients hit one OpenAI-compatible port. Behind it, one resident occupies the GPUs. DeepSeek-V4-Flash, Qwen 27B, Qwen3.8-Flash-Next, and GLM-5.3-Flash take turns. Smaller models sit in the library and load when called. That is a lab cluster, not a single-model appliance.
Lane D · Depth (TP=2)
DeepSeek-V4-Flash, 284B MoE, 156 GiB FP8, sharded across both nodes over the 200G RoCE link. DSpark speculative decoding. Flagship until a measured A/B says otherwise.
Lane F · Fleet (DP=2)
Two independent Qwen 27B replicas, one per Spark. No cross-node collective on the serving path. A node failure halves capacity instead of taking the service down. Fastest swap.
Lane N · Next (TP=2)
Qwen3.8-Flash-Next NVFP4 on the same fabric as Depth. Dedicated image plus the house PLE patch so the ModelOpt hybrid checkpoint loads. switch next drains the incumbent first.
Lane G · GLM (TP=2)
GLM-5.3-Flash NVFP4, 181 GiB, vision, MIT. Dedicated image, FlashInfer 0.6.18, SM90 NoPE path. First honest boot is the long one. switch glm / switch depth.
On-demand library
Smaller GGUF and merged fine-tunes load when called and unload after a TTL. The resident lane stays untouched. The pair only heats up while those models work.
The fine-tune lane
The node that is not serving is a 128 GB training box: QLoRA to about 70B, LoRA to 27B, full fine-tune to 7B, with a three-minute smoke gate before you commit a weekend.
One Command to Swap
Drain, boot, confirm, then a new chatA swap is a state change, not a reinstall. Off, booting, then one of Depth, Fleet, Next, or GLM. Confirm is a short completion on a new chat after warmup, not the container Up line. One switch at a time. A second click during boot fights the first.
Wall Clock to First Token
Lab pair · 27 August 2026You wait on weight load, then kernels. Confirm is a short completion after warmup, not the Up line. First-boot is a cold or first-of-session load. Later is caches warm. GLM first honest boot is the long one.
The Fit Rule
~121 GiB GPU-usable per nodeEverything in the repository follows from one number: a DGX Spark exposes about 121 GiB of GPU-usable unified memory. Two 150 GiB MoEs do not share that pool. The lane configs enforce the rule instead of letting you discover it at OOM time, twenty minutes into a model load.
Measured, Projected, or Hearsay
Every number carries a labelEvery number in the repository carries a label: [M-here] was measured on this pair with the JSON artifact committed, [M-else] is third-party and cited, [P] is a projection with its derivation shown. The benchmark protocol is binding: token counts come from usage frames, never from counting SSE chunks; TTFT is reported p50/p95; short runs are refused.
Read the spread as the headline: the same deployment is 72 tok/s on math and 34 tok/s on prose, because speculative decoding lives and dies by draft acceptance. Any two-Spark tok/s claim that omits the workload class and speculation status is not comparable to anything.
The Full Recipe
Stage once · swap many timesInstall and gate
Clone, run install.sh, then doctor and the NCCL check. Start refuses until SSH, fabric, RDMA, disk, and images are proven. Stage every resident you plan to swap. Weights stay. RAM holds one.
Revert is a first-class verb
Every start captures the incumbent. stop and revert put the boxes back. If something you care about is already up, capture-incumbent first. A second switch during boot fights the first.
Point your tools at it
Everything speaks the OpenAI API on one port. Working configs for OpenCode, Cursor, Chat, and Hermes ship in the runbook. After a healthy swap, open a new chat. Old threads that saw the previous engine die stay poisoned.
Running it cool
A pair at full serve is warm and audible. Stop drops idle draw to single-digit watts per GPU. The on-demand library loads models only while they work. Off is a supported state.
git clone https://github.com/zorost/sparkduet && cd sparkduet ./install.sh ./scripts/sparkduetctl.sh doctor ./scripts/nccl-check.sh --full ./scripts/prepare-models.sh --model deepseek ./scripts/prepare-models.sh --model qwen ./scripts/prepare-models.sh --model flash-next ./scripts/prepare-models.sh --model glm-flash ./scripts/sparkduetctl.sh start depth ./scripts/warmup.sh ./scripts/sparkduetctl.sh switch fleet ./scripts/sparkduetctl.sh switch next ./scripts/sparkduetctl.sh switch glm ./scripts/sparkduetctl.sh switch depth
When the Next Model Drops
Branches, not rewritesDeepSeek-V4-Flash is the flagship today, not forever. Model identity lives in environment configuration. Each complete recipe lives on a branch. The Lab measures first. A recipe without artifacts stays a draft.
Model identity lives in env
DeepSeek-V4-Flash is the flagship today, not forever. Model identity lives in environment configuration, not in code, so swapping the flagship never means editing the operating layer.
One branch per recipe
lane-n-flash-next and lane-g-glm-flash carry the compose files, entry patches, and retuning table for those checkpoints. A branch never overwrites another recipe.
The afternoon swap
docs/MODEL-SWAP.md is the checklist: fit rule, stage weights, pick the lane, retune the handful of knobs the table names, wait the wall-clock table, bench, commit the artifact.
What lands back here
The Lab runs new recipes on its own pair first. Measured branches graduate into the register and this page updates with dated artifacts behind them. A recipe without measurements is a draft.
The Engineering Discipline
Honest about what it is notError analysis over vibes
The project's habit is the AI-engineering habit: look at the real counters, find the dominant failure, fix that. Flash-Next needed a PLE scale buffer. GLM needed FlashInfer 0.6.18. Both patches ship in the repo.
Evals as the gate
The bench harness refuses short runs and missing counters, and the fine-tune lane's smoke test asserts loss actually falls before a weekend is committed. Numbers that cannot defend themselves do not get published.
What it is not
Not an inference engine: vLLM does the serving. Not a managed product: no auth, no TLS, no multi-tenancy; put your own gateway in front. Not a single-model appliance: the comparison doc names the jobs that want a dedicated one-model pair, and leaves SparkDuet as the complete operating layer.
A complete pair
SparkDuet is a Zorost AI Lab operating layer. The contribution is the lane model, the four-resident swap, the fit-rule honesty, and tooling that refuses to publish a number it cannot defend. Updates come from lessons learned on this pair.
Questions
Answer-firstWhat is SparkDuet?
An open source operating layer for two NVIDIA DGX Sparks. It keeps four resident models on disk (DeepSeek-V4-Flash, Qwen 27B, Qwen3.8-Flash-Next, GLM-5.3-Flash), holds one in RAM, serves them behind one OpenAI-compatible endpoint, keeps a smaller on-demand library, and fine-tunes on the idle node. MIT licensed, at github.com/zorost/sparkduet.
Why does a two-Spark pair need an operating layer at all?
Because the field still treats the pair as one GPU for one model. Owners uninstall a 150 GiB checkpoint to try the next one, or they buy a second pair so two MoEs never share a wall. SparkDuet stages every checkpoint, switches in one command, and reverts in one command.
How many models run on the Lab pair?
Four residents swap on the same two boxes: DeepSeek-V4-Flash (Lane D, TP=2), Qwen 27B (Lane F, one replica per box), Qwen3.8-Flash-Next (Lane N, TP=2), and GLM-5.3-Flash (Lane G, TP=2). RAM holds one. The on-demand library still loads smaller GGUF and merged fine-tunes without touching the resident.
How long does a swap take?
Confirm is a short completion on a new chat, not the container Up line. On the Lab pair on 27 August 2026: Qwen fleet 3 to 6 minutes first boot and 2 to 5 later; DeepSeek 5 to 8 then 4 to 7, about 6 minutes to API tonight; Flash-Next 10 to 15 then 8 to 12, 12 minutes tonight; GLM-5.3 20 to 60 on the first honest boot and 12 to 20 later. Run warmup.sh after READY.
How fast is DeepSeek on two DGX Sparks?
Measured on this pair with artifacts committed: DeepSeek-V4-Flash FP8 across both nodes runs 72.2 tok/s on math, 68.1 on code, 51.2 on tool calls, and 33.6 on prose at single concurrency, because speculative decoding lives and dies by draft acceptance. Any tok/s claim that omits workload class and speculation status is not comparable to anything.
What is the fit rule?
A DGX Spark exposes about 121 GiB of GPU-usable unified memory. The 156 GiB DeepSeek FP8, 135 GiB Flash-Next, and 181 GiB GLM NVFP4 do not fit one node and must run tensor-parallel across both. A 29 GiB Qwen 27B fits easily and runs as a replica per node. Lane configs enforce the rule instead of letting you discover it at out-of-memory time.
What happens when the next model drops?
Model identity lives in environment configuration. Each complete recipe lives on a branch (lane-n-flash-next, lane-g-glm-flash, model/deepseek-v4-flash-0731). The repo ships a swap checklist and a wall-clock table in docs/MODEL-SWAP.md. A new flagship is a branch and an afternoon, not a rewrite.
Can I fine-tune on the same pair that serves?
Yes. The node that is not serving is a 128 GB training box: QLoRA up to about 70B, LoRA to 27B, full fine-tune to 7B, with a three-minute smoke test that asserts loss actually falls before you commit a weekend. The verified smoke run took loss from 2.12 to 0.21 in 81 seconds.
What is SparkDuet not?
Not an inference engine (vLLM does the serving), not a managed product (no auth, no TLS, no multi-tenancy; put your own gateway in front), and not a single-model appliance. SparkDuet is the complete operating layer for the pair. docs/COMPARISON.md says where a one-model or one-node job is a different tool.
Serving those who
need to stay ahead.
We don't pitch slide decks. We show you what we've already built in your domain, then engineer what your mission requires.