TransferQueue Installation

TransferQueue (TQ) is the optional rollout→trainer data-plane bus for UniRL: bulky rollout outputs (conditions, latents, rewards) flow through it instead of the driver in separate / colocate sampling modes. It is not part of UniRL's declared dependencies — it is imported lazily and must be installed into the same environment separately. See unirl/distributed/weight_sync/README.md ("Transfer Queue — Separate Concern") for how it differs from weight sync, and unirl/distributed/tensor/backend/transfer_queue/ for the integration code (runtime.py, simple.py, mooncake.py, transport.py).

UniRL supports two inline transfer_queue: backend blocks:

TQ is off unless a recipe sets transport_kind: transfer_queue and provides an inline backend block. There is no Hydra +transfer_queue=... config group. Use examples/diffusion/sd3/sd3_trainside_tq_mooncake.yaml as the canonical working recipe.

1. Prerequisites

• UniRL already installed in one target engine venv, Python >=3.12,<3.14, with PyTorch present. See Installation.
• Install TQ into the same environment.
• Mooncake only: an RDMA-capable NIC (InfiniBand / RoCE) on every node, and — on TaiJi, for the from-source build — root.

2. Install the base TransferQueue package

Both backends need the transfer_queue Python package.

Option A — PyPI (Simple backend only)

pip install TransferQueue

Option B — From source (required for Mooncake)

The zero-copy Mooncake client lives on the v0.1.5_mooncake branch (matched to Mooncake v0.3.10.post1); the PyPI release does not carry it. Install editable, --no-deps so it does not perturb UniRL's pinned dependencies:

git clone -b v0.1.5_mooncake git@git.woa.com:MMRL_Infra/TransferQueue.git
cd TransferQueue
pip install -e . --no-deps

TQ's runtime deps are mostly already in UniRL (ray[default], hydra-core, numpy<2.0.0, torch). Install the few it doesn't already provide — mind the numpy<2.0.0 ceiling:

pip install "numpy<2.0.0" "tensordict>=0.10.0" pyzmq msgspec psutil

Verify

python -c "import transfer_queue; print(transfer_queue.__version__)"

The source v0.1.5_mooncake branch (Option B) reports 0.1.5; the PyPI release (Option A) reports the latest published version (e.g. 0.1.7).

3. Simple backend (in-memory)

No native dependencies — it uses in-Ray storage units. Select it with an inline recipe block:

transport_kind: transfer_queue
transfer_queue:
  _target_: unirl.distributed.tensor.backend.transfer_queue.simple.SimpleBackend
  num_units: 64
  unit_size: 16384

Best for single-node runs and functional testing. For production sizing, use Mooncake.

4. Mooncake backend (zero-copy RDMA)

UniRL's MooncakeBackend is a pure client — the storage segments live on an external Mooncake service that UniRL does not start for you. Four steps: install the engine, satisfy RDMA prerequisites, run the services, wire the config.

4.1 Install the Mooncake engine

This provides the mooncake.store Python module and the mooncake_master binary.

Generic Linux (prebuilt wheel — works where the wheel's glibc/ABI matches your host):

pip install mooncake-transfer-engine   # use the release matching Mooncake v0.3.10.post1

TaiJi / from source (needed for RDMA against the pod's drivers, or on glibc mismatch). From the TransferQueue checkout (§2 Option B):

cd TransferQueue/scripts/install_mooncake
sudo ./install_mooncake.sh

What that script does — read before running: requires root; installs system packages via yum; clones and builds Mooncake v0.3.10.post1 plus Go 1.23.8, boost 1.90, gflags 2.3, yaml-cpp 0.9, gtest 1.17, yalantinglibs 0.5.7; appends /usr/local/lib64:/usr/local/lib to LD_LIBRARY_PATH in ~/.bashrc. It yum removes the distro gtest/yaml-cpp/boost dev packages before rebuilding them from source, so run it on a disposable pod. Tunables: MOONCAKE_WORKDIR (default /dockerdata/data/Mooncake), GITHUB_PROXY, http_proxy / https_proxy. See scripts/install_mooncake/README.md in the TransferQueue repo.

Verify:

python -c "from mooncake.store import MooncakeDistributedStore; print('mooncake ok')"
mooncake_master --help            # binary on PATH
source ~/.bashrc                  # if the source build just appended LD_LIBRARY_PATH

4.2 RDMA prerequisites

ibv_devices ; ibstat               # list RDMA NICs (needs libibverbs + drivers)
ls /sys/class/infiniband           # UniRL auto-discovers device_name from here

UniRL auto-discovers device_name (a comma-list of RDMA bonds from /sys/class/infiniband) and sets MC_ENABLE_DEST_DEVICE_AFFINITY=1 so each process binds the PIX-distance HCA for its GPU — you normally do not set device_name. Override only for ops debugging: transfer_queue.device_name=mlx5_0.

protocol: tcp is accepted by MooncakeBackendConfig, but it is not a no-RDMA-host fallback in the current UniRL runtime. Mooncake client creation unconditionally probes /sys/class/infiniband and fills device_name before initializing the storage manager, regardless of protocol. Therefore, setting only transfer_queue.protocol=tcp on a host without an HCA still raises "no InfiniBand device found". Use SimpleBackend for functional testing on such hosts; validate Mooncake only on an RDMA-capable host until the runtime gates HCA discovery by protocol.

4.3 Run the external Mooncake services (head node)

Use the repository launcher on the head node. Its defaults are metadata port 50041 and RPC port 50051; if either is occupied, override it consistently for both the service and recipe:

export HEAD_IP=10.0.0.1
export TQ_MC_METADATA_SERVER_PORT=50041
export TQ_MC_RPC_PORT=50051
bash examples/mooncake_master.sh start
bash examples/mooncake_master.sh status

This yields the two endpoints the client config needs:

• master_server_address → <head_ip>:50051
• metadata_server → http://<head_ip>:50041/metadata

Keep it running for the duration of training, then run bash examples/mooncake_master.sh stop.

4.4 Wire the UniRL config

HEAD_IP=10.0.0.1 \
bash examples/run_experiment_single_node.sh \
  diffusion/sd3/sd3_trainside_tq_mooncake

That recipe contains the canonical inline block:

transport_kind: transfer_queue
transfer_queue:
  _target_: unirl.distributed.tensor.backend.transfer_queue.mooncake.MooncakeBackend
  metadata_server: http://${oc.env:HEAD_IP}:${oc.env:TQ_MC_METADATA_SERVER_PORT,50041}/metadata
  master_server_address: ${oc.env:HEAD_IP}:${oc.env:TQ_MC_RPC_PORT,50051}
  protocol: rdma
  global_segment_size_gb: 64
  local_buffer_size_gb: 10

Fields are defined in unirl/distributed/tensor/backend/transfer_queue/mooncake.py:

5. Environment variables

6. Verify end-to-end

# Imports
python -c "import transfer_queue; print(transfer_queue.__version__)"
python -c "from mooncake.store import MooncakeDistributedStore; print('mooncake ok')"   # Mooncake only

# Canonical recipe compose check
HEAD_IP=127.0.0.1 python -m unirl.train_diffusion \
  --config-name=diffusion/sd3/sd3_trainside_tq_mooncake \
  --cfg job --resolve

For Mooncake, the full RDMA path must be validated on a TaiJi GPU pod: start examples/mooncake_master.sh, launch the canonical recipe (§4.4), and confirm there is no ImportError/-800 and that rollout→train data flows.

7. Troubleshooting

8. References

• UniRL integration: unirl/distributed/tensor/backend/transfer_queue/{runtime,simple,mooncake,transport}.py
• Backend separation vs weight sync: unirl/distributed/weight_sync/README.md
• TransferQueue upstream (canonical): https://github.com/Ascend/TransferQueue (developed by the Ascend team; the older https://github.com/TransferQueue/TransferQueue is archived). UniRL pins the internal Mooncake fork git@git.woa.com:MMRL_Infra/TransferQueue.git (v0.1.5_mooncake); upstream Mooncake install notes: scripts/install_mooncake/README.md
• Mooncake: https://github.com/kvcache-ai/Mooncake (v0.3.10.post1) — deployment guide, error codes