yt lee
623e05f250
Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.
Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide
Usage:
# Start CloudXR Runtime + Isaac Lab
cd ~/IsaacLab && ./docker/container.py start \
--files docker-compose.cloudxr-runtime.patch.yaml \
--env-file .env.cloudxr-runtime
# Run teleop with CloudXR
~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
--task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr
# Serve web client
cd deps/cloudxr/isaac && npm run dev-server:https
93 lines
41 KiB
HTML
93 lines
41 KiB
HTML
<!DOCTYPE html><html class="default" lang="en" data-base=".."><head><meta charset="utf-8"/><meta http-equiv="x-ua-compatible" content="IE=edge"/><title>Session API | CloudXR.js SDK Documentation - v6.0.0-beta</title><meta name="description" content="Documentation for CloudXR.js SDK Documentation"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="stylesheet" href="../assets/style.css"/><link rel="stylesheet" href="../assets/highlight.css"/><script defer src="../assets/main.js"></script><script async src="../assets/icons.js" id="tsd-icons-script"></script><script async src="../assets/search.js" id="tsd-search-script"></script><script async src="../assets/navigation.js" id="tsd-nav-script"></script></head><body><script>document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os";document.body.style.display="none";setTimeout(() => app?app.showPage():document.body.style.removeProperty("display"),500)</script><header class="tsd-page-toolbar"><div class="tsd-toolbar-contents container"><div class="table-cell" id="tsd-search"><div class="field"><label for="tsd-search-field" class="tsd-widget tsd-toolbar-icon search no-caption"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-search"></use></svg></label><input type="text" id="tsd-search-field" aria-label="Search"/></div><div class="field"><div id="tsd-toolbar-links"></div></div><ul class="results"><li class="state loading">Preparing search index...</li><li class="state failure">The search index is not available</li></ul><a href="../index.html" class="title">CloudXR.js SDK Documentation - v6.0.0-beta</a></div><div class="table-cell" id="tsd-widgets"><a href="#" class="tsd-widget tsd-toolbar-icon menu no-caption" data-toggle="menu" aria-label="Menu"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-menu"></use></svg></a></div></div></header><div class="container container-main"><div class="col-content"><div class="tsd-page-title"><ul class="tsd-breadcrumb"><li><a href="../modules.html">CloudXR.js SDK Documentation</a></li><li><a href="Session_API.html">Session API</a></li></ul></div><div class="tsd-panel tsd-typography"><a id="session-api" class="tsd-anchor"></a><h1 class="tsd-anchor-link">Session API<a href="#session-api" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h1><p>The CloudXR.js Session API provides comprehensive management of streaming sessions with the CloudXR Runtime. This guide covers the core session lifecycle, configuration, and key operations.</p>
|
|
<a id="overview" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Overview<a href="#overview" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>CloudXR.js is a JavaScript Client SDK that enables WebXR streaming from the CloudXR Runtime. It requires compatible server applications such as Isaac Lab, or any OpenXR-compatible application running with the CloudXR Runtime.</p>
|
|
<a id="session-lifecycle" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Session Lifecycle<a href="#session-lifecycle" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>The CloudXR session follows a well-defined lifecycle with distinct states and transitions:</p>
|
|
<pre><code><span class="hl-4">Initialized</span><span class="hl-1"> → </span><span class="hl-4">Connecting</span><span class="hl-1"> → </span><span class="hl-4">Connected</span><span class="hl-1"> → </span><span class="hl-4">Disconnecting</span><span class="hl-1"> → </span><span class="hl-4">Disconnected</span>
|
|
</code><button>Copy</button></pre>
|
|
|
|
<a id="session-states" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Session States<a href="#session-states" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><ul>
|
|
<li><strong><a href="../enums/CloudXR.SessionState.html#initialized" class="tsd-kind-enum-member"> SessionState.Initialized</a></strong>: Session created but not yet connected</li>
|
|
<li><strong><a href="../enums/CloudXR.SessionState.html#connecting" class="tsd-kind-enum-member"> SessionState.Connecting</a></strong>: Attempting to establish connection with CloudXR Runtime</li>
|
|
<li><strong><a href="../enums/CloudXR.SessionState.html#connected" class="tsd-kind-enum-member"> SessionState.Connected</a></strong>: Active streaming session with video streaming active</li>
|
|
<li><strong><a href="../enums/CloudXR.SessionState.html#disconnecting" class="tsd-kind-enum-member"> SessionState.Disconnecting</a></strong>: Session is in the process of disconnecting from the CloudXR Runtime</li>
|
|
<li><strong><a href="../enums/CloudXR.SessionState.html#disconnected" class="tsd-kind-enum-member"> SessionState.Disconnected</a></strong>: Session has been disconnected and is ready to be reconnected</li>
|
|
<li><strong><a href="../enums/CloudXR.SessionState.html#error" class="tsd-kind-enum-member"> SessionState.Error</a></strong>: Session encountered a critical error and cannot continue</li>
|
|
</ul>
|
|
<a id="key-operations" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Key Operations<a href="#key-operations" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><a id="creating-a-session" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Creating a Session<a href="#creating-a-session" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>Full API reference: <a href="../functions/CloudXR.createSession.html" class="tsd-kind-function"> createSession</a></p>
|
|
<pre><code class="javascript"><span class="hl-9">import</span><span class="hl-1"> { </span><span class="hl-4">createSession</span><span class="hl-1"> } </span><span class="hl-9">from</span><span class="hl-1"> </span><span class="hl-2">'@nvidia/cloudxr'</span><span class="hl-1">;</span><br/><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">session</span><span class="hl-1"> = </span><span class="hl-0">createSession</span><span class="hl-1">({</span><br/><span class="hl-1"> </span><span class="hl-4">serverAddress:</span><span class="hl-1"> </span><span class="hl-2">'your-server'</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">serverPort:</span><span class="hl-1"> </span><span class="hl-7">49100</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">useSecureConnection:</span><span class="hl-1"> </span><span class="hl-6">false</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">gl:</span><span class="hl-1"> </span><span class="hl-4">webglContext</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeWidth:</span><span class="hl-1"> </span><span class="hl-7">2048</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeHeight:</span><span class="hl-1"> </span><span class="hl-7">1792</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">referenceSpace:</span><span class="hl-1"> </span><span class="hl-4">xrReferenceSpace</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-3">// Additional configuration options</span><br/><span class="hl-1">});</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<p><strong>Note</strong>: The <code>perEyeWidth</code> and <code>perEyeHeight</code> parameters specify the resolution for each eye. The actual stream resolution will be calculated automatically:</p>
|
|
<ul>
|
|
<li>Stream width = <code>perEyeWidth * 2</code></li>
|
|
<li>Stream height = <code>perEyeHeight * 9 / 4</code></li>
|
|
</ul>
|
|
<p>See <a href="../interfaces/CloudXR.SessionOptions.html" class="tsd-kind-interface"> SessionOptions</a> for all available configuration options.</p>
|
|
<a id="session-management" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Session Management<a href="#session-management" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>Full API reference: <a href="../interfaces/CloudXR.Session.html" class="tsd-kind-interface"> Session</a></p>
|
|
<a id="connect" class="tsd-anchor"></a><h4 class="tsd-anchor-link">Connect<a href="#connect" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="javascript"><span class="hl-3">// Connect throws an error if connection cannot be initiated</span><br/><span class="hl-9">try</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">connect</span><span class="hl-1">();</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Session connection initiated'</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><span class="hl-3">// Connection state changes will be reported via delegates</span><br/><span class="hl-1">} </span><span class="hl-9">catch</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Failed to initiate connection:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">.</span><span class="hl-4">message</span><span class="hl-1">);</span><br/><span class="hl-1">}</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="disconnect" class="tsd-anchor"></a><h4 class="tsd-anchor-link">Disconnect<a href="#disconnect" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="javascript"><span class="hl-3">// Disconnect from the server</span><br/><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">disconnect</span><span class="hl-1">();</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<p>The session will transition to <a href="../enums/CloudXR.SessionState.html#disconnecting" class="tsd-kind-enum-member"> SessionState.Disconnecting</a> state immediately.
|
|
Then to <a href="../enums/CloudXR.SessionState.html#disconnected" class="tsd-kind-enum-member"> SessionState.Disconnected</a> state when the disconnection is complete.</p>
|
|
<blockquote>
|
|
<p><strong>Note</strong>: CloudXR.js does not currently support pause/resume functionality. To temporarily stop streaming, disconnect the session and reconnect when needed.</p>
|
|
</blockquote>
|
|
<a id="send-tracking-state" class="tsd-anchor"></a><h4 class="tsd-anchor-link">Send Tracking State<a href="#send-tracking-state" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="javascript"><span class="hl-3">// Send viewer pose and input tracking to server</span><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">success</span><span class="hl-1"> = </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">sendTrackingStateToServer</span><span class="hl-1">(</span><span class="hl-4">timestamp</span><span class="hl-1">, </span><span class="hl-4">xrFrame</span><span class="hl-1">);</span><br/><span class="hl-9">if</span><span class="hl-1"> (!</span><span class="hl-4">success</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">warn</span><span class="hl-1">(</span><span class="hl-2">'Failed to send tracking state'</span><span class="hl-1">);</span><br/><span class="hl-1">}</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="render-frame" class="tsd-anchor"></a><h4 class="tsd-anchor-link">Render Frame<a href="#render-frame" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="javascript"><span class="hl-3">// Render a frame from CloudXR Runtime</span><br/><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">render</span><span class="hl-1">(</span><span class="hl-4">timestamp</span><span class="hl-1">, </span><span class="hl-4">xrFrame</span><span class="hl-1">, </span><span class="hl-4">xrWebGLLayer</span><span class="hl-1">);</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="send-server-message" class="tsd-anchor"></a><h4 class="tsd-anchor-link">Send Server Message<a href="#send-server-message" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="javascript"><span class="hl-3">// Send custom message to server</span><br/><span class="hl-9">try</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">sendServerMessage</span><span class="hl-1">(</span><span class="hl-4">messageObject</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Message sent successfully'</span><span class="hl-1">);</span><br/><span class="hl-1">} </span><span class="hl-9">catch</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Failed to send server message:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">.</span><span class="hl-4">message</span><span class="hl-1">);</span><br/><span class="hl-1">}</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="event-handling" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Event Handling<a href="#event-handling" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>CloudXR uses a delegate-based event system. Pass delegates when creating the session:</p>
|
|
<pre><code class="javascript"><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">sessionDelegates</span><span class="hl-1"> = {</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStarted</span><span class="hl-4">:</span><span class="hl-1"> () </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Streaming started'</span><span class="hl-1">);</span><br/><span class="hl-1"> },</span><br/><span class="hl-1"> </span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStopped</span><span class="hl-4">:</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-9">if</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Streaming stopped due to error:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">);</span><br/><span class="hl-1"> } </span><span class="hl-9">else</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Streaming stopped normally'</span><span class="hl-1">);</span><br/><span class="hl-1"> }</span><br/><span class="hl-1"> },</span><br/><span class="hl-1"> </span><br/><span class="hl-1"> </span><span class="hl-0">onWebGLStateChangeBegin</span><span class="hl-4">:</span><span class="hl-1"> () </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-3">// Called before session changes WebGL state</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'WebGL state change beginning'</span><span class="hl-1">);</span><br/><span class="hl-1"> },</span><br/><span class="hl-1"> </span><br/><span class="hl-1"> </span><span class="hl-0">onWebGLStateChangeEnd</span><span class="hl-4">:</span><span class="hl-1"> () </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-3">// Called after session changes WebGL state</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'WebGL state change complete'</span><span class="hl-1">);</span><br/><span class="hl-1"> }</span><br/><span class="hl-1">};</span><br/><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">session</span><span class="hl-1"> = </span><span class="hl-0">createSession</span><span class="hl-1">(</span><span class="hl-4">sessionOptions</span><span class="hl-1">, </span><span class="hl-4">sessionDelegates</span><span class="hl-1">);</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<p>See <a href="../interfaces/CloudXR.SessionDelegates.html" class="tsd-kind-interface"> SessionDelegates</a> for more details.</p>
|
|
<a id="monitoring-session-state" class="tsd-anchor"></a><h4 class="tsd-anchor-link">Monitoring Session State<a href="#monitoring-session-state" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="javascript"><span class="hl-3">// Check current session state</span><br/><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Current state:'</span><span class="hl-1">, </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-4">state</span><span class="hl-1">);</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="webxr-integration" class="tsd-anchor"></a><h2 class="tsd-anchor-link">WebXR Integration<a href="#webxr-integration" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><a id="basic-webxr-setup" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Basic WebXR Setup<a href="#basic-webxr-setup" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><pre><code class="javascript"><span class="hl-3">// Request WebXR session</span><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">xrSession</span><span class="hl-1"> = </span><span class="hl-9">await</span><span class="hl-1"> </span><span class="hl-4">navigator</span><span class="hl-1">.</span><span class="hl-4">xr</span><span class="hl-1">.</span><span class="hl-0">requestSession</span><span class="hl-1">(</span><span class="hl-2">'immersive-vr'</span><span class="hl-1">, {</span><br/><span class="hl-1"> </span><span class="hl-4">requiredFeatures:</span><span class="hl-1"> [</span><span class="hl-2">'local-floor'</span><span class="hl-1">]</span><br/><span class="hl-1">});</span><br/><br/><span class="hl-3">// Get WebGL context and reference space</span><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">gl</span><span class="hl-1"> = </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-4">renderState</span><span class="hl-1">.</span><span class="hl-4">baseLayer</span><span class="hl-1">.</span><span class="hl-4">context</span><span class="hl-1">;</span><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">referenceSpace</span><span class="hl-1"> = </span><span class="hl-9">await</span><span class="hl-1"> </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-0">requestReferenceSpace</span><span class="hl-1">(</span><span class="hl-2">'local-floor'</span><span class="hl-1">);</span><br/><br/><span class="hl-3">// Create CloudXR session</span><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">sessionOptions</span><span class="hl-1"> = {</span><br/><span class="hl-1"> </span><span class="hl-4">serverAddress:</span><span class="hl-1"> </span><span class="hl-2">'your-server'</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">serverPort:</span><span class="hl-1"> </span><span class="hl-7">49100</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">useSecureConnection:</span><span class="hl-1"> </span><span class="hl-6">false</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">gl:</span><span class="hl-1"> </span><span class="hl-4">gl</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeWidth:</span><span class="hl-1"> </span><span class="hl-7">2048</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">perEyeHeight:</span><span class="hl-1"> </span><span class="hl-7">1792</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">referenceSpace:</span><span class="hl-1"> </span><span class="hl-4">referenceSpace</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">deviceFrameRate:</span><span class="hl-1"> </span><span class="hl-7">90</span><span class="hl-1">,</span><br/><span class="hl-1"> </span><span class="hl-4">maxStreamingBitrateKbps:</span><span class="hl-1"> </span><span class="hl-7">150000</span><br/><span class="hl-1">};</span><br/><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">sessionDelegates</span><span class="hl-1"> = {</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStarted</span><span class="hl-4">:</span><span class="hl-1"> () </span><span class="hl-6">=></span><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'CloudXR streaming started'</span><span class="hl-1">),</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStopped</span><span class="hl-4">:</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) </span><span class="hl-6">=></span><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'CloudXR streaming stopped'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">)</span><br/><span class="hl-1">};</span><br/><br/><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">cloudxrSession</span><span class="hl-1"> = </span><span class="hl-0">createSession</span><span class="hl-1">(</span><span class="hl-4">sessionOptions</span><span class="hl-1">, </span><span class="hl-4">sessionDelegates</span><span class="hl-1">);</span><br/><br/><span class="hl-3">// Connect to CloudXR Runtime</span><br/><span class="hl-9">try</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">cloudxrSession</span><span class="hl-1">.</span><span class="hl-0">connect</span><span class="hl-1">();</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'CloudXR connection initiated'</span><span class="hl-1">);</span><br/><span class="hl-1">} </span><span class="hl-9">catch</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Failed to initiate CloudXR connection:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">.</span><span class="hl-4">message</span><span class="hl-1">);</span><br/><span class="hl-1">}</span><br/><br/><span class="hl-3">// Begin rendering loop</span><br/><span class="hl-6">function</span><span class="hl-1"> </span><span class="hl-0">renderFrame</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-3">// Send tracking state to server</span><br/><span class="hl-1"> </span><span class="hl-4">cloudxrSession</span><span class="hl-1">.</span><span class="hl-0">sendTrackingStateToServer</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><br/><span class="hl-1"> </span><span class="hl-3">// Render CloudXR frame</span><br/><span class="hl-1"> </span><span class="hl-4">cloudxrSession</span><span class="hl-1">.</span><span class="hl-0">render</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">, </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-4">renderState</span><span class="hl-1">.</span><span class="hl-4">baseLayer</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><br/><span class="hl-1"> </span><span class="hl-3">// Continue rendering</span><br/><span class="hl-1"> </span><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-0">requestAnimationFrame</span><span class="hl-1">(</span><span class="hl-4">renderFrame</span><span class="hl-1">);</span><br/><span class="hl-1">}</span><br/><br/><span class="hl-4">xrSession</span><span class="hl-1">.</span><span class="hl-0">requestAnimationFrame</span><span class="hl-1">(</span><span class="hl-4">renderFrame</span><span class="hl-1">);</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="controller-and-hand-tracking" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Controller and Hand Tracking<a href="#controller-and-hand-tracking" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>CloudXR automatically handles controller input and hand tracking through the WebXR session. The tracking data is sent to the server via <code>sendTrackingStateToServer()</code>:</p>
|
|
<pre><code class="javascript"><span class="hl-3">// Controllers and hand tracking are automatically handled</span><br/><span class="hl-3">// when you call sendTrackingStateToServer with the XR frame</span><br/><span class="hl-6">function</span><span class="hl-1"> </span><span class="hl-0">renderFrame</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-3">// This automatically includes controller and hand tracking data</span><br/><span class="hl-1"> </span><span class="hl-4">cloudxrSession</span><span class="hl-1">.</span><span class="hl-0">sendTrackingStateToServer</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><br/><span class="hl-1"> </span><span class="hl-3">// Render the frame</span><br/><span class="hl-1"> </span><span class="hl-4">cloudxrSession</span><span class="hl-1">.</span><span class="hl-0">render</span><span class="hl-1">(</span><span class="hl-4">time</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">, </span><span class="hl-4">xrWebGLLayer</span><span class="hl-1">);</span><br/><span class="hl-1">}</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<p>The server will receive all input data including:</p>
|
|
<ul>
|
|
<li>Controller button presses and triggers</li>
|
|
<li>Controller pose and orientation</li>
|
|
<li>Hand tracking data (if supported by the device)</li>
|
|
<li>Head tracking and viewer pose</li>
|
|
</ul>
|
|
<a id="error-handling" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Error Handling<a href="#error-handling" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><a id="error-handling-in-delegates" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Error Handling in Delegates<a href="#error-handling-in-delegates" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><pre><code class="javascript"><span class="hl-6">const</span><span class="hl-1"> </span><span class="hl-5">sessionDelegates</span><span class="hl-1"> = {</span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStarted</span><span class="hl-4">:</span><span class="hl-1"> () </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Streaming started successfully'</span><span class="hl-1">);</span><br/><span class="hl-1"> },</span><br/><span class="hl-1"> </span><br/><span class="hl-1"> </span><span class="hl-0">onStreamStopped</span><span class="hl-4">:</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) </span><span class="hl-6">=></span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-9">if</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Streaming stopped due to error:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">);</span><br/><span class="hl-1"> </span><span class="hl-3">// Handle specific error types</span><br/><span class="hl-1"> </span><span class="hl-9">if</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">.</span><span class="hl-4">message</span><span class="hl-1">.</span><span class="hl-0">includes</span><span class="hl-1">(</span><span class="hl-2">'connection'</span><span class="hl-1">)) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Connection error - check network and server'</span><span class="hl-1">);</span><br/><span class="hl-1"> }</span><br/><span class="hl-1"> } </span><span class="hl-9">else</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">log</span><span class="hl-1">(</span><span class="hl-2">'Streaming stopped normally'</span><span class="hl-1">);</span><br/><span class="hl-1"> }</span><br/><span class="hl-1"> }</span><br/><span class="hl-1">};</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="error-handling-1" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Error Handling<a href="#error-handling-1" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><pre><code class="javascript"><span class="hl-3">// Connect throws an error if connection cannot be initiated</span><br/><span class="hl-9">try</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">connect</span><span class="hl-1">();</span><br/><span class="hl-1">} </span><span class="hl-9">catch</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Failed to initiate connection:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">.</span><span class="hl-4">message</span><span class="hl-1">);</span><br/><span class="hl-1">}</span><br/><br/><span class="hl-3">// Send tracking state returns false if not connected</span><br/><span class="hl-9">if</span><span class="hl-1"> (!</span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">sendTrackingStateToServer</span><span class="hl-1">(</span><span class="hl-4">timestamp</span><span class="hl-1">, </span><span class="hl-4">frame</span><span class="hl-1">)) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">warn</span><span class="hl-1">(</span><span class="hl-2">'Cannot send tracking state - session not connected'</span><span class="hl-1">);</span><br/><span class="hl-1">}</span><br/><br/><span class="hl-3">// Send server message throws an error if not connected or invalid message</span><br/><span class="hl-9">try</span><span class="hl-1"> {</span><br/><span class="hl-1"> </span><span class="hl-4">session</span><span class="hl-1">.</span><span class="hl-0">sendServerMessage</span><span class="hl-1">(</span><span class="hl-4">message</span><span class="hl-1">);</span><br/><span class="hl-1">} </span><span class="hl-9">catch</span><span class="hl-1"> (</span><span class="hl-4">error</span><span class="hl-1">) {</span><br/><span class="hl-1"> </span><span class="hl-4">console</span><span class="hl-1">.</span><span class="hl-0">error</span><span class="hl-1">(</span><span class="hl-2">'Failed to send server message:'</span><span class="hl-1">, </span><span class="hl-4">error</span><span class="hl-1">.</span><span class="hl-4">message</span><span class="hl-1">);</span><br/><span class="hl-1">}</span>
|
|
</code><button type="button">Copy</button></pre>
|
|
|
|
<a id="performance-monitoring" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Performance Monitoring<a href="#performance-monitoring" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><a id="enable-webxr-native-performance-monitoring-in-meta-quest" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Enable WebXR Native Performance Monitoring in Meta Quest<a href="#enable-webxr-native-performance-monitoring-in-meta-quest" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><p>For comprehensive WebXR performance monitoring and debugging tools, refer to the <a href="https://developers.meta.com/horizon/documentation/web/webxr-perf-tools">Meta Horizon WebXR Performance Tools</a> documentation.</p>
|
|
<a id="other-tools" class="tsd-anchor"></a><h3 class="tsd-anchor-link">Other Tools<a href="#other-tools" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h3><ul>
|
|
<li><strong>WebXR Performance Guide</strong>: <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API/Performance">MDN WebXR Performance Guide</a> - Comprehensive performance optimization strategies</li>
|
|
<li><strong>Immersive Web Emulator</strong>: <a href="https://immersiveweb.dev/">immersiveweb.dev</a> - Browser extension for testing WebXR performance on various devices</li>
|
|
</ul>
|
|
<a id="server-requirements" class="tsd-anchor"></a><h2 class="tsd-anchor-link">Server Requirements<a href="#server-requirements" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h2><p>CloudXR.js requires a compatible server running:</p>
|
|
<ul>
|
|
<li><strong>CloudXR Runtime</strong> - The streaming server component</li>
|
|
<li><strong>OpenXR-compatible application</strong> - Such as:
|
|
<ul>
|
|
<li>Isaac Lab simulations</li>
|
|
<li>Custom OpenXR applications</li>
|
|
<li>Unity/Unreal Engine applications with OpenXR support</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
<p>The server must be accessible via WebSocket connection and properly configured for CloudXR streaming.</p>
|
|
<p>See our <a href="Getting_Started.html">Getting Started Guide</a> for more details.</p>
|
|
</div></div><div class="col-sidebar"><div class="page-menu"><div class="tsd-navigation settings"><details class="tsd-accordion"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>Settings</h3></summary><div class="tsd-accordion-details"><div class="tsd-filter-visibility"><span class="settings-label">Member Visibility</span><ul id="tsd-filter-options"><li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-inherited" name="inherited" checked/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Inherited</span></label></li></ul></div><div class="tsd-theme-toggle"><label class="settings-label" for="tsd-theme">Theme</label><select id="tsd-theme"><option value="os">OS</option><option value="light">Light</option><option value="dark">Dark</option></select></div></div></details></div><details open class="tsd-accordion tsd-page-navigation"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>On This Page</h3></summary><div class="tsd-accordion-details"><a href="#session-api"><span>Session API</span></a><ul><li><a href="#overview"><span>Overview</span></a></li><li><a href="#session-lifecycle"><span>Session <wbr/>Lifecycle</span></a></li><li><ul><li><a href="#session-states"><span>Session <wbr/>States</span></a></li></ul></li><li><a href="#key-operations"><span>Key <wbr/>Operations</span></a></li><li><ul><li><a href="#creating-a-session"><span>Creating a <wbr/>Session</span></a></li><li><a href="#session-management"><span>Session <wbr/>Management</span></a></li><li><ul><li><a href="#connect"><span>Connect</span></a></li><li><a href="#disconnect"><span>Disconnect</span></a></li><li><a href="#send-tracking-state"><span>Send <wbr/>Tracking <wbr/>State</span></a></li><li><a href="#render-frame"><span>Render <wbr/>Frame</span></a></li><li><a href="#send-server-message"><span>Send <wbr/>Server <wbr/>Message</span></a></li></ul></li><li><a href="#event-handling"><span>Event <wbr/>Handling</span></a></li><li><ul><li><a href="#monitoring-session-state"><span>Monitoring <wbr/>Session <wbr/>State</span></a></li></ul></li></ul></li><li><a href="#webxr-integration"><span>WebXR <wbr/>Integration</span></a></li><li><ul><li><a href="#basic-webxr-setup"><span>Basic <wbr/>WebXR <wbr/>Setup</span></a></li><li><a href="#controller-and-hand-tracking"><span>Controller and <wbr/>Hand <wbr/>Tracking</span></a></li></ul></li><li><a href="#error-handling"><span>Error <wbr/>Handling</span></a></li><li><ul><li><a href="#error-handling-in-delegates"><span>Error <wbr/>Handling in <wbr/>Delegates</span></a></li><li><a href="#error-handling-1"><span>Error <wbr/>Handling</span></a></li></ul></li><li><a href="#performance-monitoring"><span>Performance <wbr/>Monitoring</span></a></li><li><ul><li><a href="#enable-webxr-native-performance-monitoring-in-meta-quest"><span>Enable <wbr/>WebXR <wbr/>Native <wbr/>Performance <wbr/>Monitoring in <wbr/>Meta <wbr/>Quest</span></a></li><li><a href="#other-tools"><span>Other <wbr/>Tools</span></a></li></ul></li><li><a href="#server-requirements"><span>Server <wbr/>Requirements</span></a></li></ul></div></details></div><div class="site-menu"><nav class="tsd-navigation"><a href="../modules.html">CloudXR.js SDK Documentation - v6.0.0-beta</a><ul class="tsd-small-nested-navigation" id="tsd-nav-container"><li>Loading...</li></ul></nav></div></div></div><footer><p class="tsd-generator">Generated using <a href="https://typedoc.org/" target="_blank">TypeDoc</a></p></footer><div class="overlay"></div></body></html>
|