Running Isaac Lab Teleoperation on Meta Quest 3

NVIDIA CloudXR enables seamless, high-fidelity immersive streaming to extended reality (XR) devices over the network. Developers can use CloudXR with NVIDIA Isaac Lab to build teleoperation workflows that require immersive XR rendering for increased spatial acuity and hand tracking for teleoperation of dexterous robots.

In these workflows, a compatible XR system captures operator head and hand motion. CloudXR sends this data to Isaac Lab, which renders and submits stereo views of the robot simulation to CloudXR. CloudXR then encodes and streams the rendered views back to the XR system for display in realtime using a low-latency, GPU-accelerated pipeline.

This guide provides the Isaac Lab-specific details referenced in the First Run Guide, including installation, CloudXR Runtime configuration, and teleoperation settings.

System Requirements

Prior to using CloudXR with Isaac Lab, please review the following system requirements:

🖥️ Isaac Lab Workstation

• Ubuntu 22.04 or Ubuntu 24.04
  • Docker 26.0.0+, Docker Compose 2.25.0+, and the NVIDIA Container Toolkit. Refer to the Isaac Lab Docker Guide for how to install.
  • For details on driver requirements, please see the Technical Requirements guide
  • Follow the Isaac Lab installation guide to install Isaac Sim and Isaac Lab.
  • Required for best performance: 16 cores Intel Core i9, X-series or higher AMD Ryzen 9, Threadripper or higher
  • Required for best performance: 64GB RAM
  • Required for best performance: 2x RTX PRO 6000 GPUs (or equivalent e.g. 2x RTX 5090)

🥽 Meta Quest 3

Meta Quest 3 (OS version 79 or later).

🌐 WiFi 6 Capable Router

• A strong wireless connection is essential for a high-quality streaming experience. Refer to the requirements of Networking Setup for more details.
• We recommend using a dedicated router, as concurrent usage will degrade quality
• The Meta Quest 3 and Isaac Lab workstation must be IP-reachable from one another
  • Many institutional wireless networks will prevent devices from reaching each other, resulting in the Meta Quest 3 being unable to find the Isaac Lab workstation on the network
  • See Networking Setup guide for more details

Next

1. Running Isaac Lab with the CloudXR Runtime
2. Running Teleop Client on Meta Quest 3

If you encounter issues, please see Troubleshooting.