Download backends

Overview

modde resolves and fetches mod archives through a set of pluggable download backends. Every backend implements one trait (DownloadSource) with two phases:

1. resolve — turn a download directive into a concrete DownloadHandle (a final URL, optional mirror candidates, headers, an expected hash, and a size hint).
2. download_with_progress — stream the bytes to disk, report progress, and verify the hash.

At runtime the backends are wrapped in an AnySource enum so a heterogeneous list can dispatch to the right one by directive type. Every backend verifies the downloaded file against a Wabbajack-compatible hash: it computes both xxhash64 (xxh64, seed 0) and xxh3-64 (xxh3) while streaming, and accepts the file if either digest matches the expected value. On a mismatch the partial file is deleted and the download fails with a typed HashMismatch error.

First-class vs. directive-driven. In normal end-to-end user flows, Nexus is the first-class download surface (the Browse/Install UI and modde install mod). The other backends exist for completeness and are primarily exercised by Wabbajack and collection directive flows, where a manifest names exactly which transport and hash to use. That broader backend surface is Partial: solid transport code, but not all of it is wired into a polished standalone UX. See the parity audit.

Backend reference

Nexus

The Nexus backend asks the v1 API for a time-limited CDN URL (…/files/{file_id}/download_link.json) and streams it to disk. Generating that link is Premium-gated: modde validates the account first and refuses non-Premium keys. Authentication, the six-source key chain, browse/search, collections, and update checking are all documented in the Nexus Mods guide. This page covers Nexus only as a transport.

GitHub Releases

GitHub { user, repo, tag, asset, hash }

Resolves to a release asset by exact name within a tag (GET /repos/{user}/{repo}/releases/tags/{tag}), then downloads browser_download_url. If a GITHUB_TOKEN environment variable is set, modde sends it as a Bearer token — this lifts the unauthenticated rate limit and allows private-repo assets. All requests send a User-Agent: modde header (GitHub rejects requests without one). Transfers are wrapped in the shared retry/backoff helper (up to 3 attempts with exponential backoff and jitter). The backend can also list releases and fetch a release summary by tag for tooling.

Direct HTTPS

DirectURL { url, headers, mirror_resolver, hash }

The most capable transport:

• Range resume. If a partial file already exists at the destination, modde sends a Range: bytes=N- header. On a 206 Partial Content it appends to the existing file and seeds the running hash with the bytes already on disk, so resumed downloads still verify end to end. If the server ignores the range (returns 200), modde restarts cleanly from scratch.
• Mirror fallback. When the directive carries a mirror_resolver, modde scrapes the resolver page for candidate URLs and tries each in order, deleting the partial file between failed candidates. A single-candidate download surfaces its own error; a multi-candidate download reports an aggregated list of every URL that failed. A resolver-supplied User-Agent is injected unless the directive already set one.
• Custom headers. Arbitrary per-directive request headers are forwarded verbatim.

Direct is also the engine behind MediaFire (below).

Google Drive

GoogleDrive { id, hash }

Google interrupts large-file downloads with an HTML “can’t scan this file for viruses” interstitial instead of serving the bytes. modde handles both paths:

• It first hits the modern drive.usercontent.google.com/download?id=…&confirm=t host, which usually skips the interstitial entirely.
• If Google still returns text/html, modde parses the page for the confirm token (it tries three patterns: an &confirm=TOKEN query parameter, a name="confirm" value="TOKEN" hidden input, and the id="uc-download-link" anchor) and re-requests with &confirm=<token> to fetch the real bytes.

MEGA

Mega { url, hash }

MEGA encrypts files client-side, so modde decrypts them locally:

• Both URL formats are parsed: the modern https://mega.nz/file/HANDLE#KEY and the legacy https://mega.nz/#!HANDLE!KEY.
• It calls the MEGA API (https://g.api.mega.co.nz/cs) with the file handle to obtain the encrypted download URL and the file size.
• The 32-byte base64url key is decoded; the AES-128 key is the XOR of its two 16-byte halves, and the IV is bytes 16–24 zero-padded (the low 8 bytes are the CTR counter, starting at 0).
• As the encrypted stream arrives, modde applies an AES-128-CTR keystream chunk by chunk and writes the plaintext to disk, then verifies the hash.

MediaFire

MediaFire { url, hash }

MediaFire serves a file page, not a direct link. modde fetches the page with a browser-like User-Agent, scrapes the aria-label="Download file" anchor to extract its href (the real download…mediafire.com/… URL), and then delegates the transfer to the Direct backend — so MediaFire downloads inherit Direct’s range-resume and retry behaviour for free.

Manual

Manual { url, prompt, expected_name, … }

Some Wabbajack archives are gated behind interactive pages that cannot be fetched automatically. modde mirrors Wabbajack’s behaviour: it fails fast at resolve time with a clear, actionable message naming the expected file, the upstream URL to visit, and any prompt the list author left:

manual download required for <expected_name>: visit <url> and place the downloaded
file in the modde downloads directory. Prompt from list author: <prompt>

This is intentional — a Manual directive is a signal that you must download the file by hand and drop it into the downloads directory, after which the install can continue.

Resumable .meta sidecars

modde models each download as a JSON .meta sidecar written next to the destination file (mod_file.zip → mod_file.zip.meta). The sidecar records:

• url, expected_hash, bytes_downloaded, total_bytes, and a status string (queued / downloading / paused / complete / failed: …)
• Nexus provenance: nexus_mod_id, nexus_file_id, game_domain, mod_name, version

Sidecars are the persistence layer that lets a queue be rebuilt across process restarts: on load, complete entries are restored only if the file still exists, and any downloading/paused entry is restored as paused (because no transport is running after startup). This pairs with the Direct backend’s range resume to continue a partial transfer rather than restart it.

Status: Partial. The sidecar data model and round-trip are implemented and tested, but writing and reloading sidecars is not yet a shipped end-to-end workflow in the GUI. Treat durable cross-restart resume as in-progress; see the parity audit.

The download queue

DownloadQueue is a pure, synchronous state machine that tracks tasks and enforces a concurrency limit — it performs no I/O itself; the caller spawns the actual async transfers. Each task moves through Queued → Active → {Complete | Paused | Failed}:

• enqueue(url, dest, expected_hash, meta) adds a task and returns its ID.
• take_next() promotes the next Queued task to Active, but only while the active count is below the limit — this is how concurrency is bounded.
• pause(id) / resume(id) move a task to Paused and back to Queued; cancel(id) removes it.
• save_sidecars() / load_from_sidecars(dir, limit) persist and rebuild the queue from .meta files (see above).

The GUI currently constructs the queue with a concurrency of 2 and renders real queue state in its Downloads view.

Status: Partial. The queue’s UI state (queued/active/paused/failed/complete) is shipped, but transport-level pause/resume is not yet wired to the network layer, and there is no speed/ETA calculation yet (progress callbacks report bytes, not rate). The Wabbajack engine has its own concurrent download path (default parallelism 4) used during modlist installs.

Tuning environment variables

These environment variables tune the download and staging machinery. The byte cache, zstd staging compression, and archive-retention knobs are read by the Wabbajack install engine and are mainly relevant to large modlist installs (Partial); they are not part of the everyday single-mod download path.

Related authentication / transport variables documented elsewhere:

MODDE_ARCHIVE_RETENTION also has a CLI equivalent on the Wabbajack installer (--archive-retention keep|prune-applied|auto); the flag wins when both are set.

See also

• Nexus Mods — the first-class download surface: auth, browse, install, updates
• Wabbajack modlists — the directive-driven install flow that exercises every backend
• Troubleshooting — download failures, hash mismatches, rate limits
• Parity audit — what is Done vs Partial