hangX/mindbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
510e610a27a97a8fc4cf2e7a86c0cc3ea2c76ab5
mindbot/deps/cloudxr/docs/documents/Getting_Started.Sample_Client_-_WebGL.html
yt lee 623e05f250 Add CloudXR VR streaming support for PICO 4 Ultra (Early Access) 
Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.

Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
  disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
  teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
  HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
  EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide

Usage:
  # Start CloudXR Runtime + Isaac Lab
  cd ~/IsaacLab && ./docker/container.py start \
      --files docker-compose.cloudxr-runtime.patch.yaml \
      --env-file .env.cloudxr-runtime

  # Run teleop with CloudXR
  ~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
      --task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr

  # Serve web client
  cd deps/cloudxr/isaac && npm run dev-server:https
2026-03-26 14:29:03 +08:00

119 lines
16 KiB
HTML
Raw Blame History

<!DOCTYPE html><html class="default" lang="en" data-base=".."><head><meta charset="utf-8"/><meta http-equiv="x-ua-compatible" content="IE=edge"/><title>Sample Client - WebGL | CloudXR.js SDK Documentation - v6.0.0-beta</title><meta name="description" content="Documentation for CloudXR.js SDK Documentation"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="stylesheet" href="../assets/style.css"/><link rel="stylesheet" href="../assets/highlight.css"/><script defer src="../assets/main.js"></script><script async src="../assets/icons.js" id="tsd-icons-script"></script><script async src="../assets/search.js" id="tsd-search-script"></script><script async src="../assets/navigation.js" id="tsd-nav-script"></script></head><body><script>document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os";document.body.style.display="none";setTimeout(() => app?app.showPage():document.body.style.removeProperty("display"),500)</script><header class="tsd-page-toolbar"><div class="tsd-toolbar-contents container"><div class="table-cell" id="tsd-search"><div class="field"><label for="tsd-search-field" class="tsd-widget tsd-toolbar-icon search no-caption"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-search"></use></svg></label><input type="text" id="tsd-search-field" aria-label="Search"/></div><div class="field"><div id="tsd-toolbar-links"></div></div><ul class="results"><li class="state loading">Preparing search index...</li><li class="state failure">The search index is not available</li></ul><a href="../index.html" class="title">CloudXR.js SDK Documentation - v6.0.0-beta</a></div><div class="table-cell" id="tsd-widgets"><a href="#" class="tsd-widget tsd-toolbar-icon menu no-caption" data-toggle="menu" aria-label="Menu"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-menu"></use></svg></a></div></div></header><div class="container container-main"><div class="col-content"><div class="tsd-page-title"><ul class="tsd-breadcrumb"><li><a href="../modules.html">CloudXR.js SDK Documentation</a></li><li><a href="Getting_Started.html">Getting Started</a></li><li><a href="Getting_Started.Sample_Client_-_WebGL.html">Sample Client - WebGL</a></li></ul></div><div class="tsd-panel tsd-typography"><a id="cloudxrjs-simple-example" class="tsd-anchor"></a><h1 class="tsd-anchor-link">CloudXR.js Simple Example<a href="#cloudxrjs-simple-example" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h1><p>A minimal WebGL example demonstrating WebXR streaming from a CloudXR server to a web browser. This example shows how to integrate WebXR with CloudXR to stream immersive VR/AR content.</p>
<blockquote>
<p><strong>Note:</strong> This example is for learning purposes, not production use.</p>
</blockquote>
<a id="quick-start" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Quick Start<a href="#quick-start" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><a id="prerequisites" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Prerequisites<a href="#prerequisites" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><ul>
<li>Node.js (v20 or higher)</li>
</ul>
<a id="installation" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Installation<a href="#installation" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><ol>
<li>
<p><strong>Navigate to the example folder</strong></p>
<pre><code class="bash"><span class="hl-0">cd</span><span class="hl-1"> </span><span class="hl-2">simple</span>
</code><button type="button">Copy</button></pre>
</li>
<li>
<p><strong>Install Dependencies</strong></p>
<pre><code class="bash"><span class="hl-3"># For this early access release, please run the following to install SDK from the given tarball. This step will not be needed when SDK is publicly accessible.</span><br/><span class="hl-0">npm</span><span class="hl-1"> </span><span class="hl-2">install</span><span class="hl-1"> </span><span class="hl-2">../nvidia-cloudxr-6.0.0-beta.tgz</span><br/><br/><span class="hl-0">npm</span><span class="hl-1"> </span><span class="hl-2">install</span>
</code><button type="button">Copy</button></pre>
</li>
<li>
<p><strong>Build the Application</strong></p>
<pre><code class="bash"><span class="hl-0">npm</span><span class="hl-1"> </span><span class="hl-2">run</span><span class="hl-1"> </span><span class="hl-2">build</span>
</code><button type="button">Copy</button></pre>
</li>
<li>
<p><strong>Start Development Server</strong></p>
<pre><code class="bash"><span class="hl-0">npm</span><span class="hl-1"> </span><span class="hl-2">run</span><span class="hl-1"> </span><span class="hl-2">dev-server</span>
</code><button type="button">Copy</button></pre>
</li>
<li>
<p><strong>Open in Browser</strong></p>
<ul>
<li>Navigate to <code>http://localhost:8080</code> (or the port shown in terminal)</li>
<li>For desktop browsers, IWER (Immersive Web Emulator Runtime) will automatically load to emulate a Meta Quest 3 headset</li>
</ul>
</li>
</ol>
<a id="basic-usage" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Basic Usage<a href="#basic-usage" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><ol>
<li>
<p><strong>Configure Connection</strong></p>
<ul>
<li>Enter CloudXR server IP address (default: localhost)</li>
<li>Set port (default: 49100)</li>
<li>Select AR or VR mode</li>
</ul>
</li>
<li>
<p><strong>Adjust Settings (Optional)</strong></p>
<ul>
<li>Per-eye resolution (must be multiples of 16)</li>
<li>Target frame rate (72, 90, or 120 FPS)</li>
<li>Streaming bitrate</li>
<li>XR reference space and camera offsets</li>
</ul>
</li>
<li>
<p><strong>Start Streaming</strong></p>
<ul>
<li>Click &quot;CONNECT&quot;</li>
<li>Grant XR permissions when prompted</li>
</ul>
</li>
</ol>
<p><strong>Requirements:</strong></p>
<ul>
<li>CloudXR server running and accessible</li>
<li>WebXR-compatible device (VR/AR headset) or desktop browser (IWER loads automatically for emulation)</li>
</ul>
<a id="architecture" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Architecture<a href="#architecture" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><a id="cloudxrclient-class" class="tsd-anchor"></a><h3 class="tsd-anchor-link">CloudXRClient Class<a href="#cloudxrclient-class" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>The main application class (<code>CloudXRClient</code> in <code>main.ts</code>) handles:</p>
<p><strong>Initialization:</strong></p>
<ul>
<li>UI element management and localStorage persistence</li>
<li>Browser capability checks (WebXR, WebGL2, WebRTC)</li>
<li>Event listener setup</li>
</ul>
<p><strong>Connection Flow:</strong></p>
<ol>
<li><strong>WebGL Setup</strong> - Creates high-performance WebGL2 context</li>
<li><strong>WebXR Session</strong> - Enters immersive VR/AR mode</li>
<li><strong>Reference Space</strong> - Configures coordinate system for tracking</li>
<li><strong>CloudXR Session</strong> - Establishes streaming connection to server</li>
<li><strong>Render Loop</strong> - Sends tracking data, receives video, renders frames</li>
</ol>
<p><strong>Key Components:</strong></p>
<ul>
<li><strong>WebXR Session</strong> - Hardware access (headset, controllers)</li>
<li><strong>WebGL Context</strong> - Video rendering</li>
<li><strong>CloudXR Session</strong> - Streaming management (WebRTC-based)</li>
<li><strong>XRWebGLLayer</strong> - Bridge between WebXR and WebGL</li>
</ul>
<a id="project-structure" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Project Structure<a href="#project-structure" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><pre><code><span class="hl-4">simple</span><span class="hl-1">/</span><br/><span class="hl-1">├── </span><span class="hl-4">src</span><span class="hl-1">/</span><br/><span class="hl-1">│ └── </span><span class="hl-4">main</span><span class="hl-1">.</span><span class="hl-4">ts</span><span class="hl-1"> # </span><span class="hl-4">Main</span><span class="hl-1"> </span><span class="hl-4">application</span><span class="hl-1"> </span><span class="hl-4">sample</span><br/><span class="hl-1">├── </span><span class="hl-4">index</span><span class="hl-1">.</span><span class="hl-4">html</span><span class="hl-1"> # </span><span class="hl-5">UI</span><span class="hl-1"> </span><span class="hl-4">and</span><span class="hl-1"> </span><span class="hl-4">form</span><span class="hl-1"> </span><span class="hl-4">elements</span><span class="hl-1"> </span><span class="hl-4">sample</span><br/><span class="hl-1">├── </span><span class="hl-4">package</span><span class="hl-1">.</span><span class="hl-4">json</span><span class="hl-1"> # </span><span class="hl-4">Dependencies</span><span class="hl-1"> </span><span class="hl-4">and</span><span class="hl-1"> </span><span class="hl-4">scripts</span><br/><span class="hl-1">├── </span><span class="hl-4">webpack</span><span class="hl-1">.</span><span class="hl-4">common</span><span class="hl-1">.</span><span class="hl-4">js</span><span class="hl-1"> # </span><span class="hl-4">Webpack</span><span class="hl-1"> </span><span class="hl-4">base</span><span class="hl-1"> </span><span class="hl-4">configuration</span><span class="hl-1"> </span><span class="hl-4">sample</span><br/><span class="hl-1">├── </span><span class="hl-4">webpack</span><span class="hl-1">.</span><span class="hl-4">dev</span><span class="hl-1">.</span><span class="hl-4">js</span><span class="hl-1"> # </span><span class="hl-4">Development</span><span class="hl-1"> </span><span class="hl-4">configuration</span><span class="hl-1"> </span><span class="hl-4">sample</span><br/><span class="hl-1">├── </span><span class="hl-4">webpack</span><span class="hl-1">.</span><span class="hl-4">prod</span><span class="hl-1">.</span><span class="hl-4">js</span><span class="hl-1"> # </span><span class="hl-4">Production</span><span class="hl-1"> </span><span class="hl-4">configuration</span><span class="hl-1"> </span><span class="hl-4">sample</span><br/><span class="hl-1">└── </span><span class="hl-4">tsconfig</span><span class="hl-1">.</span><span class="hl-4">json</span><span class="hl-1"> # </span><span class="hl-4">TypeScript</span><span class="hl-1"> </span><span class="hl-4">configuration</span><span class="hl-1"> </span><span class="hl-4">sample</span>
</code><button>Copy</button></pre>
<a id="code-overview" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Code Overview<a href="#code-overview" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>The <code>main.ts</code> file contains well-commented code explaining each step:</p>
<ol>
<li><strong>Browser Checks</strong> - Validates WebXR, WebGL2, and WebRTC support</li>
<li><strong>Connection Setup</strong> - Reads form inputs and validates configuration</li>
<li><strong>WebGL Initialization</strong> - Creates optimized rendering context</li>
<li><strong>WebXR Session</strong> - Enters immersive mode with requested features</li>
<li><strong>CloudXR Setup</strong> - Configures streaming session with event handlers</li>
<li><strong>Render Loop</strong> - Runs 72-120 times per second:
<ul>
<li>Sends tracking data to server</li>
<li>Receives video frame</li>
<li>Renders to display</li>
</ul>
</li>
</ol>
<p>Each method includes inline comments explaining the purpose and key concepts.</p>
<a id="license" class="tsd-anchor"></a><h2 class="tsd-anchor-link">License<a href="#license" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>See the <a href="../media/LICENSE-1">LICENSE</a> file for details.</p>
<a id="next-steps" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Next Steps<a href="#next-steps" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>Experience one of the compatible OpenXR applications with CloudXR Runtime</p>
<ul>
<li><a href="Show_Cases.Isaac_Lab_Teleop.html">Isaac Lab Teleoperation</a></li>
</ul>
</div></div><div class="col-sidebar"><div class="page-menu"><div class="tsd-navigation settings"><details class="tsd-accordion"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>Settings</h3></summary><div class="tsd-accordion-details"><div class="tsd-filter-visibility"><span class="settings-label">Member Visibility</span><ul id="tsd-filter-options"><li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-inherited" name="inherited" checked/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Inherited</span></label></li></ul></div><div class="tsd-theme-toggle"><label class="settings-label" for="tsd-theme">Theme</label><select id="tsd-theme"><option value="os">OS</option><option value="light">Light</option><option value="dark">Dark</option></select></div></div></details></div><details open class="tsd-accordion tsd-page-navigation"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>On This Page</h3></summary><div class="tsd-accordion-details"><a href="#cloudxrjs-simple-example"><span>CloudXR.js <wbr/>Simple <wbr/>Example</span></a><ul><li><a href="#quick-start"><span>Quick <wbr/>Start</span></a></li><li><ul><li><a href="#prerequisites"><span>Prerequisites</span></a></li><li><a href="#installation"><span>Installation</span></a></li><li><a href="#basic-usage"><span>Basic <wbr/>Usage</span></a></li></ul></li><li><a href="#architecture"><span>Architecture</span></a></li><li><ul><li><a href="#cloudxrclient-class"><span>CloudXRClient <wbr/>Class</span></a></li></ul></li><li><a href="#project-structure"><span>Project <wbr/>Structure</span></a></li><li><a href="#code-overview"><span>Code <wbr/>Overview</span></a></li><li><a href="#license"><span>License</span></a></li><li><a href="#next-steps"><span>Next <wbr/>Steps</span></a></li></ul></div></details></div><div class="site-menu"><nav class="tsd-navigation"><a href="../modules.html">CloudXR.js SDK Documentation - v6.0.0-beta</a><ul class="tsd-small-nested-navigation" id="tsd-nav-container"><li>Loading...</li></ul></nav></div></div></div><footer><p class="tsd-generator">Generated using <a href="https://typedoc.org/" target="_blank">TypeDoc</a></p></footer><div class="overlay"></div></body></html>
Reference in New Issue View Git Blame Copy Permalink