Building qroissant wheels

This project ships as a maturin / PyO3 mixed Rust+Python package. The native extension uses #![feature(portable_simd)] so nightly Rust is required, pinned via rust-toolchain.toml.

All builds run inside a Docker image (scripts/Dockerfile.build) so the host needs only Docker and Python 3 — no Rust, no MSVC SDK, no mingw to install.

Prerequisites

Docker (tested with 29.x)
Python 3.11+ on the host, only if you want to install/test the produced Linux wheel locally via scripts/build.sh check.

One-time setup

scripts/build.sh image

Builds qroissant-build:latest. Pulls ghcr.io/rust-cross/cargo-xwin, installs the nightly toolchain + x86_64-pc-windows-msvc target + maturin. ~5 min on a cold cache, then it's a no-op.

Build commands

scripts/build.sh linux      # -> dist-linux/qroissant-*-manylinux_*_x86_64.whl
scripts/build.sh windows    # -> dist-windows/qroissant-*-win_amd64.whl
scripts/build.sh all        # image + linux + windows
scripts/build.sh check      # install latest linux wheel into .venv, import-smoke,
                            # then `zipfile -l` the windows wheel
scripts/build.sh clean      # rm dist-linux dist-windows
scripts/build.sh clean-cache  # also drop the cargo/target/xwin Docker volumes

The wheels target abi3-py311, so a single artifact per platform covers CPython 3.11, 3.12, 3.13, and onward.

What lives where

Artifact	Path
Build image definition	`scripts/Dockerfile.build`
Wrapper script	`scripts/build.sh`
Toolchain pin	`rust-toolchain.toml`
Linux wheels	`dist-linux/` (gitignored)
Windows wheels	`dist-windows/` (gitignored)
Cargo registry cache	Docker volume `qroissant-cargo-registry`
Cargo target dir	Docker volume `qroissant-target`
cargo-xwin MSVC cache	Docker volume `qroissant-xwin`

The three Docker volumes persist between runs so reruns only rebuild changed crates; nuke them with scripts/build.sh clean-cache if anything looks stale.

Debug symbols on Windows

The release profile sets debug = "line-tables-only", so a _native.pdb is produced alongside the wheel. scripts/build.sh windows copies it into dist-windows/. To get resolved frames in a Rust panic backtrace on a Windows machine:

pip install qroissant-*-win_amd64.whl
Locate the installed extension, e.g. Lib\site-packages\qroissant\_native.pyd
Drop _native.pdb next to it (same directory, same basename).

Symbol info is line-tables-only, so frames resolve to file:line but local variables and types aren't included. That's enough for backtraces and is ~30 MB; full debug info would be much larger.

How Windows cross-compile works

The Windows path uses cargo-xwin, which downloads the Microsoft CRT + Windows SDK headers (cached in the xwin volume; license auto-accepted via XWIN_ACCEPT_LICENSE=1). PyO3's generate-import-lib feature — enabled in crates/qroissant-python/Cargo.toml — lets us produce the python3.dll import library at build time, so no Windows Python install is required on the host.

Quick wheel sanity checks

# Listing
python3 -m zipfile -l dist-windows/qroissant-*-win_amd64.whl

# Validate the dist-info
pipx run twine check dist-linux/*.whl dist-windows/*.whl

Iterating on Python code

For Python-side changes (no Rust touched), the fastest loop is still maturin develop against a local rustup install. The Docker flow above is optimized for producing wheels, not for inner-loop iteration. If you have a nightly rustup toolchain on the host, you can do:

python3 -m venv .venv && .venv/bin/pip install 'maturin>=1.8,<2.0'
.venv/bin/maturin develop --release
.venv/bin/python -c 'import qroissant'

Troubleshooting

"manylinux_2_34" wheel won't install on old distros — the image base is Debian 13 (glibc 2.38). For broader compatibility, build inside a manylinux2014 image or run auditwheel repair against the produced wheel.
"failed to generate python3.dll import library" — means llvm-dlltool is missing inside the image. Rebuild it with scripts/build.sh image.
xwin download is slow / fails — Microsoft's CDN occasionally rate-limits. Drop the cache (docker volume rm qroissant-xwin) and retry.

4.3 KiB Raw Permalink Blame History