Installation

UniRL requires Python >=3.12,<3.14. Both supported rollout-engine stacks use CUDA 13 / PyTorch +cu130, but vllm and sglang pin different PyTorch versions and are declared as conflicting extras. Install them in separate virtual environments; never combine both engine extras in one environment.

Python Package

pyproject.toml is the dependency source of truth. The root setup.py and requirements.txt are legacy compatibility snapshots and do not encode the current engine/CUDA stacks.

# vLLM / vLLM-Omni
uv venv --python 3.12 --seed .venv
source .venv/bin/activate
uv pip install -e ".[vllm,train,infer]" --prerelease=allow

# SGLang, step 1: create and activate a different environment
deactivate
uv venv --python 3.12 --seed .venv-sglang
source .venv-sglang/bin/activate

The SGLang stack reaches causal-conv1d, which compiles a CUDA extension. A CUDA 12 nvcc on PATH cannot compile against the CUDA 13 PyTorch runtime. After activating .venv-sglang, install the CUDA 13 compiler wheels, configure the build environment, and only then install UniRL with the SGLang extras:

# SGLang, step 2: install the CUDA 13 compiler toolchain
uv pip install "nvidia-cuda-nvcc==13.0.*" "nvidia-cuda-crt==13.0.*" \
  "nvidia-nvvm==13.0.*" "nvidia-cuda-cccl==13.0.*" "nvidia-cuda-runtime==13.0.*"

# SGLang, step 3: select that toolchain and bound parallel compilation
export CUDA_HOME="$VIRTUAL_ENV/lib/python3.12/site-packages/nvidia/cu13"
export PATH="$CUDA_HOME/bin:$PATH"
export MAX_JOBS=4

# SGLang, step 4: install UniRL and the engine stack
uv pip install -e ".[sglang,train,infer]" --prerelease=allow

On driver-535 hosts, make NVIDIA's cuda-compat-13-0 forward-compat libraries available through LD_LIBRARY_PATH. SGLang wheels also require glibc >=2.34; the vLLM stack requires glibc >=2.28.

Optional Extras

Install only the capabilities required by the selected recipe:

# OCR reward (PaddleOCR is included; no separate OCR package is needed)
uv pip install -e ".[vllm,train,infer,eval]" --prerelease=allow

# Dataset converters; droid100 additionally needs torch from an engine env
uv pip install -e ".[dataset-prep]"

# Restricted model/backend examples
uv pip install -e ".[vllm,train,cosmos3]" --prerelease=allow
uv pip install -e ".[sglang,train,veomni,infer]" --prerelease=allow
uv pip install -e ".[vllm,train,fastvideo]" --prerelease=allow

Optional Model and Reward Dependencies

The legacy GenEval/OpenMMLab scorer must run in a separate Python 3.10 sidecar; its MMCV stack is incompatible with the core Python 3.12 / CUDA 13 environment. See Geneval MMCV Setup.

To run heavy reward models on their own GPU node instead of in-process, use the standalone remote reward service in unirl-reward-service/ (it ships its own dependencies and README). See Rewards for wiring the remote backend.

The optional rollout→trainer data-plane bus (TransferQueue / Mooncake) is also installed separately. See TransferQueue Installation.

Documentation Site

The Fumadocs site is isolated in docs/ so Node dependencies do not affect the Python package:

cd docs
npm install
npm run dev

Build the static site:

npm run build

The exported static files are emitted by Next.js into docs/out/.