yt lee
623e05f250
Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.
Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide
Usage:
# Start CloudXR Runtime + Isaac Lab
cd ~/IsaacLab && ./docker/container.py start \
--files docker-compose.cloudxr-runtime.patch.yaml \
--env-file .env.cloudxr-runtime
# Run teleop with CloudXR
~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
--task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr
# Serve web client
cd deps/cloudxr/isaac && npm run dev-server:https
22 lines
18 KiB
HTML
22 lines
18 KiB
HTML
<!DOCTYPE html><html class="default" lang="en" data-base=".."><head><meta charset="utf-8"/><meta http-equiv="x-ua-compatible" content="IE=edge"/><title>createSession | CloudXR.js SDK Documentation - v6.0.0-beta</title><meta name="description" content="Documentation for CloudXR.js SDK Documentation"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="stylesheet" href="../assets/style.css"/><link rel="stylesheet" href="../assets/highlight.css"/><script defer src="../assets/main.js"></script><script async src="../assets/icons.js" id="tsd-icons-script"></script><script async src="../assets/search.js" id="tsd-search-script"></script><script async src="../assets/navigation.js" id="tsd-nav-script"></script></head><body><script>document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os";document.body.style.display="none";setTimeout(() => app?app.showPage():document.body.style.removeProperty("display"),500)</script><header class="tsd-page-toolbar"><div class="tsd-toolbar-contents container"><div class="table-cell" id="tsd-search"><div class="field"><label for="tsd-search-field" class="tsd-widget tsd-toolbar-icon search no-caption"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-search"></use></svg></label><input type="text" id="tsd-search-field" aria-label="Search"/></div><div class="field"><div id="tsd-toolbar-links"></div></div><ul class="results"><li class="state loading">Preparing search index...</li><li class="state failure">The search index is not available</li></ul><a href="../index.html" class="title">CloudXR.js SDK Documentation - v6.0.0-beta</a></div><div class="table-cell" id="tsd-widgets"><a href="#" class="tsd-widget tsd-toolbar-icon menu no-caption" data-toggle="menu" aria-label="Menu"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-menu"></use></svg></a></div></div></header><div class="container container-main"><div class="col-content"><div class="tsd-page-title"><ul class="tsd-breadcrumb"><li><a href="../modules.html">CloudXR.js SDK Documentation</a></li><li><a href="../modules/CloudXR.html">CloudXR</a></li><li><a href="CloudXR.createSession.html">createSession</a></li></ul><h1>Function createSession</h1></div><section class="tsd-panel"><ul class="tsd-signatures"><li class=""><div class="tsd-signature tsd-anchor-link"><a id="createsession" class="tsd-anchor"></a><span class="tsd-kind-call-signature">createSession</span><span class="tsd-signature-symbol">(</span><span class="tsd-kind-parameter">options</span><span class="tsd-signature-symbol">:</span> <a href="../interfaces/CloudXR.SessionOptions.html" class="tsd-signature-type tsd-kind-interface">SessionOptions</a><span class="tsd-signature-symbol">,</span> <span class="tsd-kind-parameter">delegates</span><span class="tsd-signature-symbol">?:</span> <a href="../interfaces/CloudXR.SessionDelegates.html" class="tsd-signature-type tsd-kind-interface">SessionDelegates</a><span class="tsd-signature-symbol">)</span><span class="tsd-signature-symbol">:</span> <a href="../interfaces/CloudXR.Session.html" class="tsd-signature-type tsd-kind-interface">Session</a><a href="#createsession" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></div><div class="tsd-description"><div class="tsd-comment tsd-typography"><p>Creates a new CloudXR streaming session with the provided configuration.</p>
|
|
<p>This is the primary entry point for creating CloudXR streaming sessions.
|
|
The returned session is ready to be connected to the CloudXR Runtime using
|
|
the connect() method.</p>
|
|
</div><div class="tsd-parameters"><h4 class="tsd-parameters-title">Parameters</h4><ul class="tsd-parameter-list"><li><span><span class="tsd-kind-parameter">options</span>: <a href="../interfaces/CloudXR.SessionOptions.html" class="tsd-signature-type tsd-kind-interface">SessionOptions</a></span><div class="tsd-comment tsd-typography"><p>Configuration options for the session. Must include all
|
|
required parameters such as serverAddress, serverPort,
|
|
gl context, and per-eye dimensions.</p>
|
|
</div><div class="tsd-comment tsd-typography"></div></li><li><span><code class="tsd-tag">Optional</code><span class="tsd-kind-parameter">delegates</span>: <a href="../interfaces/CloudXR.SessionDelegates.html" class="tsd-signature-type tsd-kind-interface">SessionDelegates</a></span><div class="tsd-comment tsd-typography"><p>Optional delegate object to receive essential session
|
|
events such as onStreamStarted, onStreamStopped, and
|
|
WebGL state change notifications.</p>
|
|
</div><div class="tsd-comment tsd-typography"></div></li></ul></div><h4 class="tsd-returns-title">Returns <a href="../interfaces/CloudXR.Session.html" class="tsd-signature-type tsd-kind-interface">Session</a></h4><p>A Session object ready to connect to the CloudXR Runtime</p>
|
|
<div class="tsd-comment tsd-typography"><div class="tsd-tag-throws"><h4 class="tsd-anchor-link"><a id="throws" class="tsd-anchor"></a>Throws<a href="#throws" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><p>When perEyeWidth is not a positive number or not a multiple of 16</p>
|
|
</div><div class="tsd-tag-throws"><h4 class="tsd-anchor-link"><a id="throws-1" class="tsd-anchor"></a>Throws<a href="#throws-1" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><p>When perEyeHeight is not a positive number or not a multiple of 16</p>
|
|
</div><div class="tsd-tag-throws"><h4 class="tsd-anchor-link"><a id="throws-2" class="tsd-anchor"></a>Throws<a href="#throws-2" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><p>When derived stream dimensions are not multiples of 16</p>
|
|
</div><div class="tsd-tag-example"><h4 class="tsd-anchor-link"><a id="example" class="tsd-anchor"></a>Example<a href="#example" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="typescript"><span class="hl-3">// Basic session creation</span><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">session</span><span class="hl-1"> = </span><span class="hl-0">createSession</span><span class="hl-1">({</span><br/><span class="hl-1"> </span><span class="hl-4">serverAddress:</span><span class="hl-1"> </span><span class="hl-2">'192.168.1.100'</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">serverPort:</span><span class="hl-1"> </span><span class="hl-7">49100</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">useSecureConnection:</span><span class="hl-1"> </span><span class="hl-6">false</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">gl:</span><span class="hl-1"> </span><span class="hl-4">webglContext</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeWidth:</span><span class="hl-1"> </span><span class="hl-7">2048</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeHeight:</span><span class="hl-1"> </span><span class="hl-7">1792</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">referenceSpace:</span><span class="hl-1"> </span><span class="hl-4">xrReferenceSpace</span><br/><span class="hl-1">});</span><br/><br/><span class="hl-3">// With event delegates</span><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">session</span><span class="hl-1"> = </span><span class="hl-0">createSession</span><span class="hl-1">(</span><span class="hl-4">sessionOptions</span><span class="hl-1">, {</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStarted</span><span class="hl-4">:</span><span class="hl-1"> () </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">info</span><span class="hl-1">(</span><span class="hl-2">'CloudXR streaming started'</span><span class="hl-1">);</span><br/><span class="hl-1"> },</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStopped</span><span class="hl-4">:</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-9">if</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Streaming error:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">);</span><br/><span class="hl-1"> } </span><span class="hl-9">else</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">info</span><span class="hl-1">(</span><span class="hl-2">'Streaming stopped normally'</span><span class="hl-1">);</span><br/><span class="hl-1"> }</span><br/><span class="hl-1"> }</span><br/><span class="hl-1">});</span><br/><br/><span class="hl-3">// Connect to CloudXR Runtime</span><br/><span class="hl-9">if</span><span class="hl-1"> (</span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">connect</span><span class="hl-1">()) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">info</span><span class="hl-1">(</span><span class="hl-2">'Connection initiated'</span><span class="hl-1">);</span><br/><span class="hl-1">}</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
</div><div class="tsd-tag-example"><h4 class="tsd-anchor-link"><a id="example-1" class="tsd-anchor"></a>Example<a href="#example-1" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="typescript"><span class="hl-3">// Complete WebXR integration example</span><br/><span class="hl-6">async</span><span class="hl-1"> </span><span class="hl-6">function</span><span class="hl-1"> </span><span class="hl-0">setupCloudXR</span><span class="hl-1">() {</span><br/><span class="hl-1"> </span><span class="hl-3">// Request WebXR session</span><br/><span class="hl-1"> </span><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">xrSession</span><span class="hl-1"> = </span><span class="hl-9">await</span><span class="hl-1"> </span><span class="hl-4">navigator</span><span class="hl-1">.</span><span class="hl-4">xr</span><span class="hl-1">.</span><span class="hl-0">requestSession</span><span class="hl-1">(</span><span class="hl-2">'immersive-vr'</span><span class="hl-1">, {</span><br/><span class="hl-1"> </span><span class="hl-4">requiredFeatures:</span><span class="hl-1"> [</span><span class="hl-2">'local-floor'</span><span class="hl-1">]</span><br/><span class="hl-1"> });</span><br/><br/><span class="hl-1"> </span><span class="hl-3">// Get WebGL context and reference space</span><br/><span class="hl-1"> </span><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">gl</span><span class="hl-1"> = </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-4">renderState</span><span class="hl-1">.</span><span class="hl-4">baseLayer</span><span class="hl-1">.</span><span class="hl-4">context</span><span class="hl-1">;</span><br/><span class="hl-1"> </span><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">referenceSpace</span><span class="hl-1"> = </span><span class="hl-9">await</span><span class="hl-1"> </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-0">requestReferenceSpace</span><span class="hl-1">(</span><span class="hl-2">'local-floor'</span><span class="hl-1">);</span><br/><br/><span class="hl-1"> </span><span class="hl-3">// Create CloudXR streaming session</span><br/><span class="hl-1"> </span><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">session</span><span class="hl-1"> = </span><span class="hl-0">createSession</span><span class="hl-1">({</span><br/><span class="hl-1"> </span><span class="hl-4">serverAddress:</span><span class="hl-1"> </span><span class="hl-2">'your-server-ip'</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">serverPort:</span><span class="hl-1"> </span><span class="hl-7">49100</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">useSecureConnection:</span><span class="hl-1"> </span><span class="hl-6">false</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">gl:</span><span class="hl-1"> </span><span class="hl-4">gl</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeWidth:</span><span class="hl-1"> </span><span class="hl-7">2048</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeHeight:</span><span class="hl-1"> </span><span class="hl-7">1792</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">referenceSpace:</span><span class="hl-1"> </span><span class="hl-4">referenceSpace</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">deviceFrameRate:</span><span class="hl-1"> </span><span class="hl-7">90</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">maxStreamingBitrateKbps:</span><span class="hl-1"> </span><span class="hl-7">150000</span><br/><span class="hl-1"> }, {</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStarted</span><span class="hl-4">:</span><span class="hl-1"> () </span><span class="hl-6">=></span><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">info</span><span class="hl-1">(</span><span class="hl-2">'Ready to render'</span><span class="hl-1">),</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStopped</span><span class="hl-4">:</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) </span><span class="hl-6">=></span><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">info</span><span class="hl-1">(</span><span class="hl-2">'Streaming stopped'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">)</span><br/><span class="hl-1"> });</span><br/><br/><span class="hl-1"> </span><span class="hl-3">// Connect and start rendering</span><br/><span class="hl-1"> </span><span class="hl-9">if</span><span class="hl-1"> (</span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">connect</span><span class="hl-1">()) {</span><br/><span class="hl-1"> </span><span class="hl-6">function</span><span class="hl-1"> </span><span class="hl-0">renderFrame</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">sendTrackingStateToServer</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">render</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">, </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-4">renderState</span><span class="hl-1">.</span><span class="hl-4">baseLayer</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-0">requestAnimationFrame</span><span class="hl-1">(</span><span class="hl-4">renderFrame</span><span class="hl-1">);</span><br/><span class="hl-1"> }</span><br/><span class="hl-1"> </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-0">requestAnimationFrame</span><span class="hl-1">(</span><span class="hl-4">renderFrame</span><span class="hl-1">);</span><br/><span class="hl-1"> }</span><br/><span class="hl-1">}</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
</div></div></div></li></ul></section></div><div class="col-sidebar"><div class="page-menu"><div class="tsd-navigation settings"><details class="tsd-accordion"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>Settings</h3></summary><div class="tsd-accordion-details"><div class="tsd-filter-visibility"><span class="settings-label">Member Visibility</span><ul id="tsd-filter-options"><li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-inherited" name="inherited" checked/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Inherited</span></label></li></ul></div><div class="tsd-theme-toggle"><label class="settings-label" for="tsd-theme">Theme</label><select id="tsd-theme"><option value="os">OS</option><option value="light">Light</option><option value="dark">Dark</option></select></div></div></details></div></div><div class="site-menu"><nav class="tsd-navigation"><a href="../modules.html">CloudXR.js SDK Documentation - v6.0.0-beta</a><ul class="tsd-small-nested-navigation" id="tsd-nav-container"><li>Loading...</li></ul></nav></div></div></div><footer><p class="tsd-generator">Generated using <a href="https://typedoc.org/" target="_blank">TypeDoc</a></p></footer><div class="overlay"></div></body></html>
|