hangX/mindbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
510e610a27a97a8fc4cf2e7a86c0cc3ea2c76ab5
mindbot/deps/cloudxr/docs/documents/Getting_Started.html
yt lee 623e05f250 Add CloudXR VR streaming support for PICO 4 Ultra (Early Access) 
Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.

Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
  disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
  teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
  HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
  EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide

Usage:
  # Start CloudXR Runtime + Isaac Lab
  cd ~/IsaacLab && ./docker/container.py start \
      --files docker-compose.cloudxr-runtime.patch.yaml \
      --env-file .env.cloudxr-runtime

  # Run teleop with CloudXR
  ~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
      --task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr

  # Serve web client
  cd deps/cloudxr/isaac && npm run dev-server:https
2026-03-26 14:29:03 +08:00

104 lines
13 KiB
HTML
Raw Blame History

<!DOCTYPE html><html class="default" lang="en" data-base=".."><head><meta charset="utf-8"/><meta http-equiv="x-ua-compatible" content="IE=edge"/><title>Getting Started | CloudXR.js SDK Documentation - v6.0.0-beta</title><meta name="description" content="Documentation for CloudXR.js SDK Documentation"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="stylesheet" href="../assets/style.css"/><link rel="stylesheet" href="../assets/highlight.css"/><script defer src="../assets/main.js"></script><script async src="../assets/icons.js" id="tsd-icons-script"></script><script async src="../assets/search.js" id="tsd-search-script"></script><script async src="../assets/navigation.js" id="tsd-nav-script"></script></head><body><script>document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os";document.body.style.display="none";setTimeout(() => app?app.showPage():document.body.style.removeProperty("display"),500)</script><header class="tsd-page-toolbar"><div class="tsd-toolbar-contents container"><div class="table-cell" id="tsd-search"><div class="field"><label for="tsd-search-field" class="tsd-widget tsd-toolbar-icon search no-caption"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-search"></use></svg></label><input type="text" id="tsd-search-field" aria-label="Search"/></div><div class="field"><div id="tsd-toolbar-links"></div></div><ul class="results"><li class="state loading">Preparing search index...</li><li class="state failure">The search index is not available</li></ul><a href="../index.html" class="title">CloudXR.js SDK Documentation - v6.0.0-beta</a></div><div class="table-cell" id="tsd-widgets"><a href="#" class="tsd-widget tsd-toolbar-icon menu no-caption" data-toggle="menu" aria-label="Menu"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-menu"></use></svg></a></div></div></header><div class="container container-main"><div class="col-content"><div class="tsd-page-title"><ul class="tsd-breadcrumb"><li><a href="../modules.html">CloudXR.js SDK Documentation</a></li><li><a href="Getting_Started.html">Getting Started</a></li></ul></div><div class="tsd-panel tsd-typography"><a id="getting-started" class="tsd-anchor"></a><h1 class="tsd-anchor-link">Getting Started<a href="#getting-started" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h1><!-- Below we include the sample README.md for the overview sections. -->
<p>It is <em>strongly recommended</em> that you work through this guide if you have never run CloudXR.js before.</p>
<a id="the-pieces-of-a-cloudxrjs-deployment" class="tsd-anchor"></a><h2 class="tsd-anchor-link">The Pieces of a CloudXR.js Deployment<a href="#the-pieces-of-a-cloudxrjs-deployment" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>Even for development, you'll need all the pieces of a CloudXR.js deployment in order to build and test a client. These are:</p>
<ul>
<li>a CloudXR Server
<ul>
<li>with a <em>top-end</em> NVIDIA GPU or 2 (e.g. dual RTX 6000 Ada)</li>
<li>which will run
<ul>
<li>the CloudXR Runtime</li>
<li>an OpenXR application (the thing you want to render on the server but see on the client)</li>
</ul>
</li>
</ul>
</li>
<li>a CloudXR.js development workstation
<ul>
<li>with Node.js and <code>npm</code></li>
<li>which will run
<ul>
<li>the CloudXR.js sample client build</li>
<li>a Node-based development web server</li>
</ul>
</li>
</ul>
</li>
<li>a CloudXR.js client
<ul>
<li>which is one of:
<ul>
<li>a Meta Quest 3 with its built-in Browser app</li>
<li>a desktop browser: <a href="https://www.google.com/chrome">Google Chrome</a> or Edge (IWER automatically loads for WebXR emulation)</li>
</ul>
</li>
<li>which will run...
<ul>
<li>the CloudXR.js sample client <em>served from the development web server</em>.</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>We <em>recommend</em> that for your first experience, all above run on <em>the same computer</em> to eliminate networking related issues.</p>
<a id="high-level-workflow" class="tsd-anchor"></a><h2 class="tsd-anchor-link">High Level Workflow<a href="#high-level-workflow" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>You need both a working client and a working server in order to test. Typically we follow a startup flow where server starts before the client:</p>
<ol>
<li>CloudXR Runtime</li>
<li>Server XR application</li>
<li>Sample client build + web server</li>
<li>Test from the same computer</li>
<li>Test from Quest 3 or a different computer</li>
</ol>
<a id="detailed-requirements" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Detailed Requirements<a href="#detailed-requirements" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><a id="cloudxr-server" class="tsd-anchor"></a><h3 class="tsd-anchor-link">CloudXR Server<a href="#cloudxr-server" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p><strong>Minimum:</strong> Single NVIDIA RTX 6000 Ada / L40 GPU or equivalent</p>
<p><strong>Recommended:</strong> Two (2x) NVIDIA RTX 6000-class Ada (or later) GPUs:</p>
<ul>
<li>RTX 6000 Ada</li>
<li>L40 / L40S</li>
<li>RTX 6000 PRO Blackwell (Workstation, Max-Q Workstation, or Server)</li>
</ul>
<p><strong>Additional requirements:</strong></p>
<ul>
<li>Linux OS (required for Isaac Lab Teleoperation)</li>
<li>Docker and NVIDIA Container Toolkit (Linux only)</li>
<li>Network: Wired Ethernet connection recommended</li>
</ul>
<a id="client-device" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Client Device<a href="#client-device" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p><strong>Supported headset:</strong></p>
<ul>
<li>Meta Quest 3 (OS version 79 or later) - only headset validated for this release</li>
<li>Uses built-in Browser app (no special software installation needed)</li>
<li>Some manual browser configuration required (see <a href="Client_Setup.html">Client Setup Guide</a>)</li>
</ul>
<p><strong>For development/testing:</strong></p>
<ul>
<li>Desktop <a href="https://www.google.com/chrome">Google Chrome</a> (IWER automatically loads for WebXR emulation)</li>
</ul>
<a id="network-requirements" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Network Requirements<a href="#network-requirements" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p><strong>Critical requirements:</strong></p>
<ul>
<li>Latency: ≤ 40ms required, ≤ 20ms recommended</li>
<li>Bandwidth: 200+ Mbps per streaming connection</li>
<li>WiFi: WiFi 6 (WiFi 6e recommended) at 5GHz or 6GHz</li>
<li><strong>At most one WiFi hop:</strong> Either server or client must use wired Ethernet</li>
</ul>
<p>See <a href="Networking_Setup.html">Networking Setup Guide</a> for detailed configuration examples.</p>
<a id="step-by-step-guides" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Step-by-Step Guides<a href="#step-by-step-guides" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>Following the high-level workflow above, use these guides to set up each component:</p>
<a id="1-set-up-cloudxr-runtime-and-server-application" class="tsd-anchor"></a><h3 class="tsd-anchor-link">1. Set Up CloudXR Runtime and Server Application<a href="#1-set-up-cloudxr-runtime-and-server-application" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>Choose an OpenXR server application to stream:</p>
<ul>
<li><a href="Show_Cases.Isaac_Lab_Teleop.html">Isaac Lab Teleoperation</a> - Robot learning and teleoperation workflows</li>
</ul>
<p>The guide includes instructions for setting up the CloudXR Runtime alongside the server application.</p>
<a id="2-build-and-serve-a-sample-client" class="tsd-anchor"></a><h3 class="tsd-anchor-link">2. Build and Serve a Sample Client<a href="#2-build-and-serve-a-sample-client" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>Build one of our example web clients:</p>
<ul>
<li><a href="Getting_Started.Sample_Client_-_WebGL.html">WebGL Sample</a> - Simple example using vanilla WebGL</li>
<li><a href="Getting_Started.Sample_Client_-_React.html">React Sample</a> - Example using React and Three.js</li>
</ul>
<a id="3-test-and-configure" class="tsd-anchor"></a><h3 class="tsd-anchor-link">3. Test and Configure<a href="#3-test-and-configure" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><ul>
<li><strong>Test from the same computer first</strong> - Follow the desktop testing sections in the sample guides</li>
<li><strong>Test from Meta Quest 3</strong> - See <a href="Client_Setup.html">Client Setup Guide</a> for browser configuration</li>
<li><strong>Configure networking</strong> (if needed) - See <a href="Networking_Setup.html">Networking Setup Guide</a> for firewall and proxy examples</li>
</ul>
<a id="advanced-topics" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Advanced Topics<a href="#advanced-topics" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><ul>
<li><a href="Session_API.html">Session APIs</a> - Programmatic control of CloudXR sessions</li>
<li><a href="Telemetry.html">Telemetry</a> - Performance monitoring and debugging</li>
</ul>
</div></div><div class="col-sidebar"><div class="page-menu"><div class="tsd-navigation settings"><details class="tsd-accordion"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>Settings</h3></summary><div class="tsd-accordion-details"><div class="tsd-filter-visibility"><span class="settings-label">Member Visibility</span><ul id="tsd-filter-options"><li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-inherited" name="inherited" checked/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Inherited</span></label></li></ul></div><div class="tsd-theme-toggle"><label class="settings-label" for="tsd-theme">Theme</label><select id="tsd-theme"><option value="os">OS</option><option value="light">Light</option><option value="dark">Dark</option></select></div></div></details></div><details open class="tsd-accordion tsd-page-navigation"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>On This Page</h3></summary><div class="tsd-accordion-details"><a href="#getting-started"><span>Getting <wbr/>Started</span></a><ul><li><a href="#the-pieces-of-a-cloudxrjs-deployment"><span>The <wbr/>Pieces of a <wbr/>CloudXR.js <wbr/>Deployment</span></a></li><li><a href="#high-level-workflow"><span>High <wbr/>Level <wbr/>Workflow</span></a></li><li><a href="#detailed-requirements"><span>Detailed <wbr/>Requirements</span></a></li><li><ul><li><a href="#cloudxr-server"><span>CloudXR <wbr/>Server</span></a></li><li><a href="#client-device"><span>Client <wbr/>Device</span></a></li><li><a href="#network-requirements"><span>Network <wbr/>Requirements</span></a></li></ul></li><li><a href="#step-by-step-guides"><span>Step-<wbr/>by-<wbr/>Step <wbr/>Guides</span></a></li><li><ul><li><a href="#1-set-up-cloudxr-runtime-and-server-application"><span>1. <wbr/>Set <wbr/>Up <wbr/>CloudXR <wbr/>Runtime and <wbr/>Server <wbr/>Application</span></a></li><li><a href="#2-build-and-serve-a-sample-client"><span>2. <wbr/>Build and <wbr/>Serve a <wbr/>Sample <wbr/>Client</span></a></li><li><a href="#3-test-and-configure"><span>3. <wbr/>Test and <wbr/>Configure</span></a></li></ul></li><li><a href="#advanced-topics"><span>Advanced <wbr/>Topics</span></a></li></ul></div></details></div><div class="site-menu"><nav class="tsd-navigation"><a href="../modules.html">CloudXR.js SDK Documentation - v6.0.0-beta</a><ul class="tsd-small-nested-navigation" id="tsd-nav-container"><li>Loading...</li></ul></nav></div></div></div><footer><p class="tsd-generator">Generated using <a href="https://typedoc.org/" target="_blank">TypeDoc</a></p></footer><div class="overlay"></div></body></html>
Reference in New Issue View Git Blame Copy Permalink