Sharded download

Run models bigger than any one machine. For a model like Llama 3.3 70B (~42 GB) split across three nodes, each node downloads only the layers it will serve — dppan pull fetches just those byte ranges and writes a small, valid GGUF shard; dppan join --gguf serves it. No node ever needs the full file.

How it works

A GGUF file starts with a tensor directory that maps every tensor to its exact byte range. dppan pull reads that index (a few MB), selects the tensors for your layer range, range-fetches only those bytes, and reconstructs a smaller GGUF that the engine loads like any other model. A shard also records its own provenance — source digest and layer range — so join knows what's inside.

Three-node example (Llama 3.3 70B)

# ── Node 0: orchestrator + worker, layers 0–26 (~16 GB instead of 42) ──
dppan pull llama3.3:70b --layers 0-26 --orch
DPPAN_GGUF_LLAMA3_3_70B=~/.dppan/models/llama3.3-70b-layers0-26-orch.gguf \
  PORT=8001 dppan-orchestrator &
dppan join --orchestrator http://localhost:8001 --model llama3.3:70b \
           --gguf ~/.dppan/models/llama3.3-70b-layers0-26-orch.gguf

# ── Node 1: worker, layers 27–53 (~13 GB) ──
dppan pull llama3.3:70b --layers 27-53
dppan join --orchestrator http://node0:8001 --model llama3.3:70b \
           --gguf ~/.dppan/models/llama3.3-70b-layers27-53.gguf

# ── Node 2: worker, layers 54–79 (~13 GB) ──
dppan pull llama3.3:70b --layers 54-79
dppan join --orchestrator http://node0:8001 --model llama3.3:70b \
           --gguf ~/.dppan/models/llama3.3-70b-layers54-79.gguf

Then chat as usual — the OpenAI-compatible endpoint on node 0 routes tokens through the layer chain automatically.

Good to know

• --layers is inclusive on both ends (0-26 = 27 layers) — the same convention everywhere it appears (dppan pull, dppan join, and layer ranges shown in the dashboard and dppan status).
• A shard remembers its own range, so join --gguf needs no --layers — the orchestrator assigns the node exactly the slice it has on disk. If a node is ever assigned layers its file doesn't contain, it refuses loudly with the exact dppan pull command to fix it.
• --orch additionally includes the shared tensors (embeddings, output norm, LM head) needed by the machine running dppan-orchestrator. Exactly one shard needs it. Orch-only shards (no --layers) work too, for a coordinator-only machine. If the source publishes a projector sidecar, this pull also downloads and validates it automatically.
• Projectors belong only to the orchestrator host. They are stored under ~/.dppan/models/projectors/<canonical-model-id>/ and are never included in worker shards or sent to worker nodes. Each projector has an adjacent .dppan.json record containing its exact source identity, digest, size, and inspected modalities; the orchestrator uses that record rather than guessing compatibility from its filename.
• The orchestrator finds its GGUF via the DPPAN_GGUF_<MODEL> env var — the model id uppercased with non-alphanumerics as _ (llama3.3:70b → DPPAN_GGUF_LLAMA3_3_70B) — or via gguf_path in config/models.toml. Without either, it falls back to the --gguf path of the first node that joins, which only works when that node runs on the same machine and joins with the --orch shard (the embed engine needs token_embd.weight; a layers-only shard fails with "token_embd.weight not found").
• If the full model is already in ~/.ollama, pull slices it locally with zero network traffic.
• Shards default to ~/.dppan/models/ and show up in dppan models with their layer range.

dppan pull reference

dppan pull <MODEL> [OPTIONS]

Flag	Default	Description
<MODEL>	required	llama3.3:70b (Ollama) · hf:owner/repo[:QUANT] · inclavate:slug
--layers	all	Layer range to download, inclusive (e.g. 0-26)
--orch	off	Include orchestrator tensors and auto-download a published projector
--projector-only	off	Download only the orchestrator projector; write no decoder shard
--projector <FILE>	auto	Select an exact sidecar; automatic preference is F16, then BF16
--projector-out <PATH>	model store	Override the sidecar output path for an --orch pull
--component <ROLE[=SELECTOR]>	off	Download one component of a model published as a set of files — see Models
--out	~/.dppan/models/…	Output shard path, or the bundle directory with --component
--verify	off	Check every tensor's sha256 against the integrity manifest
--manifest	auto	Manifest file or URL for --verify
--remote	off	Range-fetch even if the full blob exists locally

Sources and quant selection (including split -0000N-of-… HF quants, which are merged automatically) are covered on the Models page.

For a manual sidecar-only pull:

dppan pull hf:owner/repo:Q4_K_M --projector-only
dppan pull hf:owner/repo:Q4_K_M --projector-only --projector mmproj-BF16.gguf

The first command uses the automatic projector preference. The second selects a published variant explicitly. Both verify the source digest and inspect the completed GGUF with libmtmd before publishing it into the model store. If --projector-out places the sidecar outside the default store, set projector_path on that model's config/models.toml entry so the orchestrator can associate it explicitly.

Picking a split

Divide the model's layer count by your node count, weighted by VRAM. For 80 layers on three similar machines: 0-26, 27-53, 54-79. The node running the orchestrator does extra work (embeddings + LM head), so give it the smaller slice if the machines differ. If you omit --layers hints entirely and let the orchestrator assign, size the pulls after joining once — or just start with the automatic flow on a model that fits locally.

One family does not divide evenly: Gemma 4 E-series. E2B and E4B share a single KV cache across their last layers, and those layers have to stay on the same machine as the layer they read from. That caps them at two nodes, with the boundary at or below layer 13 for E2B (35 layers) and layer 22 for E4B (42 layers). 0-13 / 14-34 works for E2B; an even three-way split does not, whatever the VRAM says. Every other supported model splits freely.

The orchestrator refuses an illegal layout before running anything and names the boundary to use, so you cannot get a wrong answer this way — you would just rather know before pulling the shards.