Add CloudXR VR streaming support for PICO 4 Ultra (Early Access)

Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.

Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
  disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
  teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
  HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
  EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide

Usage:
  # Start CloudXR Runtime + Isaac Lab
  cd ~/IsaacLab && ./docker/container.py start \
      --files docker-compose.cloudxr-runtime.patch.yaml \
      --env-file .env.cloudxr-runtime

  # Run teleop with CloudXR
  ~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
      --task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr

  # Serve web client
  cd deps/cloudxr/isaac && npm run dev-server:https
This commit is contained in:
2026-03-26 14:29:03 +08:00
parent eef7ff838d
commit 623e05f250
133 changed files with 24869 additions and 2 deletions

307
deps/cloudxr/react/LICENSE vendored Normal file
View File

@@ -0,0 +1,307 @@
NVIDIA SOFTWARE EVALUATION LICENSE AGREEMENT
IMPORTANT NOTICE PLEASE READ AND AGREE BEFORE USING THE SOFTWARE
This software evaluation license agreement (“Agreement”) is a legal agreement between you, whether an
individual or entity, (“you”) and NVIDIA Corporation and its affiliates (“NVIDIA”) and governs the use of certain
NVIDIA CloudXR software and documentation that NVIDIA delivers to you under this Agreement (“Software”).
NVIDIA and you are each a “party” and collectively the “parties.”
This Agreement can be accepted only by an adult of legal age of majority in the country in which the Software is
used. If you dont have the required age or authority to accept this Agreement, or if you dont accept all the
terms and conditions of this Agreement, do not use the Software.
1. License Grants.
1.1 License Grant to You. The Software is licensed, not sold. Subject to the terms of this Agreement,
NVIDIA grants you a limited, non-exclusive, revocable, non-transferable, non-sublicensable (except
as expressly granted in this Agreement), license to:
(a) access, install and use copies of the Software,
(b) configure the Software using configuration files provided (if applicable),
(c) modify and create derivative works of any source code NVIDIA delivers to you as part of the
Software (“Derivatives”) (if applicable).
All the foregoing grants are only for internal test and evaluation purposes and, as applicable, for use (a) in
client systems, or (b) in server systems with NVIDIA GPUs (“Purpose”).
1.2 License Grant to NVIDIA. Subject to the terms of this Agreement, you grant NVIDIA a non-exclusive,
perpetual, irrevocable, sublicensable, worldwide, royalty-free, fully paid-up and transferable license,
under your intellectual property rights, to publicly perform, publicly display, reproduce, use, make,
have made, sell, offer for sale, distribute (through multiple tiers of distribution), import, create
derivative works of and otherwise commercialize and exploit at NVIDIAs discretion any Derivatives
created by or for you. You may, but are not required to, deliver any Derivatives to NVIDIA.
2. License Restrictions. Your license to use the Software and Derivatives is restricted as stated in this “License
Restrictions” Section. You will cooperate with NVIDIA and, upon NVIDIAs written request, you will confirm
in writing and provide reasonably requested information to verify your compliance with the terms of this
Agreement. You may not:
2.1 Use the Software or Derivatives for any purpose other than the Purpose, including but not limited to
in production;
2.2 Sell, rent, sublicense, transfer, distribute or otherwise make available to others (except Authorized
Users as stated in the “Authorized Users” Section) any portion of the Software or Derivatives, except
as expressly granted in Section 1.1 (“License Grant to You”);
2.3 Reverse engineer, decompile, or disassemble the Software components provided in binary form, nor
attempt in any other manner to obtain source code of such Software;
2.4 Modify or create derivative works of the Software, except as expressly granted in Section 1.1
(“License Grant to You”);
2.5 Change or remove copyright or other proprietary notices in the Software;
2.6 Bypass, disable, or circumvent any technical limitation, encryption, security, digital rights
management or authentication mechanism in the Software;
2.7 Use the Software or Derivatives in any manner that would cause them to become subject to an open
source software license; subject to the terms in Section 7 (“Components Under Other Licenses”);
2.8 Use the Software or Derivatives for the purpose of developing competing products or technologies
or assist a third party in such activities;
2.9 Replace any Software components governed by this Agreement with other software that
implements NVIDIA APIs;
2.10 Use the Software or Derivatives in violation of any applicable law or regulation in the relevant
jurisdictions; or
2.11 Use the Software in or with any system or application where the use or failure of such system or
application developed or deployed with Software could result in injury, death or catastrophic
damage (“Mission Critical Applications”). NVIDIA will not be liable to you or any third party, in whole
or in part, for any claims or damages arising from uses in Mission Critical Applications.
2.12 Disclose any evaluation or test results regarding the Software or Derivatives without NVIDIAs prior
written consent.
3. Authorized Users. You may allow employees and contractors of your entity or of your subsidiary(ies), and
for educational institutions also enrolled students, to internally access and use the Software as authorized
by this Agreement from your secure network to perform the work authorized by this Agreement on your
behalf. You are responsible for the compliance with the terms of this Agreement by your authorized users.
Any act or omission that if committed by you would constitute a breach of this Agreement will be deemed to
constitute a breach of this Agreement if committed by your authorized users.
4. Pre-Release. Software versions identified as alpha, beta, preview, early access or otherwise as pre-release
may not be fully functional, may contain errors or design flaws, and may have reduced or different security,
privacy, availability and reliability standards relative to NVIDIA commercial offerings. You use pre-release
Software at your own risk. NVIDIA did not design or test the Software for use in production or business
critical systems. NVIDIA may choose not to make available a commercial version of pre-release Software.
NVIDIA may also choose to abandon development and terminate the availability of pre-release Software at
any time without liability.
5. Your Privacy: Collection and Use of Information.
5.1 Privacy Policy. Please review the NVIDIA Privacy Policy, located at https://www.nvidia.com/enus/about-nvidia/privacy-policy, which explains NVIDIAs policy for collecting and using data, as well
as visit the NVIDIA Privacy Center, located at https://www.nvidia.com/en-us/privacy-center, to
manage your consent and privacy preferences.
5.2 Collection Purposes. You also acknowledge that the Software collects data for the following
purposes: (a) properly configure and optimize products for use with Software; and (b) improve
NVIDIA products and services. Information collected by the Software includes: (i) application
configuration; (ii) browser version; (iii) and session metadata (i.e. performance and usage
statistics). Additionally, NVIDIA may collect certain personal information, such as your name
and email address or those of your authorized users, and other information necessary to
authenticate and enable you or your authorized users access to the Software. Where appropriate
you will disclose to, and obtain any necessary consent from, your authorized users to allow NVIDIA
to collect such information.
5.3 Third Party Privacy Practices. The Software may contain links to third party websites and services.
NVIDIA encourages you to review the privacy statements on those sites and services that you choose
to visit to understand how they may collect, use and share your data. NVIDIA is not responsible for
the privacy statements or practices of third-party sites or services.
6. Updates. NVIDIA may at any time and at its option, change, discontinue, or deprecate any part, or all, of the
Software, or change or remove features or functionality, or make available patches, workarounds or other
updates to the Software. Unless the updates are provided with their separate governing terms, they are
deemed part of the Software licensed to you under this Agreement, and your continued use of the Software
is deemed acceptance of such changes.
7. Components Under Other Licenses. The Software may include or be distributed with components provided
with separate legal notices or terms that accompany the components, such as open source software licenses
and other license terms (“Other Licenses”). The components are subject to the applicable Other Licenses,
including any proprietary notices, disclaimers, requirements and extended use rights; except that this
Agreement will prevail regarding the use of third-party open source software, unless a third-party open
source software license requires its license terms to prevail. Open source software license means any
software, data or documentation subject to any license identified as an open source license by the Open
Source Initiative (http://opensource.org), Free Software Foundation (http://www.fsf.org) or other similar
open source organization or listed by the Software Package Data Exchange (SPDX) Workgroup under the
Linux Foundation (http://www.spdx.org).
8. Ownership.
8.1 NVIDIA Ownership. The Software, including all intellectual property rights, is and will remain the sole
and exclusive property of NVIDIA or its licensors. Except as expressly granted in this Agreement,
(a) NVIDIA reserves all rights, interests and remedies in connection with the Software, and (b) no
other license or right is granted to you by implication, estoppel or otherwise.
8.2 Your Ownership. Subject to the rights of NVIDIA and its suppliers in the Software, which continue to
be licensed as stated in this Agreement, even when incorporated in your products, and the extent
permitted by applicable law, as between you and NVIDIA, you hold all rights, title and interest in and
to your services, applications and Derivatives you develop as permitted in this Agreement including
their respective intellectual property rights.
9. Feedback. You may, but are not obligated to, provide suggestions, requests, fixes, modifications,
enhancements or other feedback regarding your use of the Software (“Feedback”). Feedback, even if
designated as confidential by you, will not create any confidentiality obligation for NVIDIA or its affiliates. If
you provide Feedback, you hereby grant NVIDIA, its affiliates and its designees a nonexclusive, perpetual,
irrevocable, sublicensable, worldwide, royalty-free, fully paid-up and transferable license, under your
intellectual property rights, to publicly perform, publicly display, reproduce, use, make, have made, sell,
offer for sale, distribute (through multiple tiers of distribution), import, create derivative works of and
otherwise commercialize and exploit the Feedback at NVIDIAs discretion.
10. Confidentiality. You may use confidential information only to exercise your rights and perform your
obligations under this Agreement. You will not disclose, nor authorize others to disclose NVIDIA Confidential
Information to any third party, except as expressly authorized in this Agreement and as necessary for the
Purpose, without obtaining NVIDIAs prior written approval. Each recipient of confidential information must
be subject to a written agreement that includes confidentiality obligations consistent with these terms and
must have a need to know for the Purpose. You will protect the NVIDIA Confidential Information with at
least the same degree of care that you use to protect your own similar confidential and proprietary
information, but no less than a reasonable degree of care. Confidential information includes, but is not
limited to, the Software, including its features and functionality, Derivatives, and any results of
benchmarking or other competitive analysis or regression or performance data relating to the Software.
No Publicity. You may not issue any public statements about this Agreement, disclose the Software or
Derivatives, or any information or results related to your use of the Software, without prior written approval
of NVIDIA.
11. Term and Termination.
11.1 Term. This Agreement has a duration of twelve (12) months starting from the date of initial
download (even if you download the same version or updates of the Software later and it is
accompanied by this Agreement or another Agreement), unless terminated earlier in accordance
with this Agreement.
11.2 Termination for Convenience. Either party may terminate this Agreement at any time with thirty (30)
days advance written notice to the other party.
11.3 Termination for Cause. If you commence or participate in any legal proceeding against NVIDIA with
respect to the Software, this Agreement will terminate immediately without notice. Either party may
terminate this Agreement upon notice for cause if:
(a) the other party fails to cure a material breach of this Agreement within ten (10) days of the
non-breaching partys notice of the breach; or
(b) the other party breaches its confidentiality obligations or license rights under this
Agreement, which termination will be effective immediately upon written notice.
11.4 Effect of Termination. Upon any expiration or termination of this Agreement, you will promptly
(a) stop using and return, delete or destroy NVIDIA confidential information and all Software
received under this Agreement, and (b) delete or destroy Derivatives created under this Agreement,
unless an authorized NVIDIA representative provides prior written approval that you may keep a
copy of the Derivatives solely for archival purposes. Upon written request, you will certify in writing
that you have complied with your obligations under this “Effect of Termination” Section.
11.5 Survival. The “License Grant to NVIDIA”, “Updates”, “Components Under Other Licenses”,
“Ownership”, “Feedback”, “Confidentiality”, “No Publicity”, “Effect of Termination”, “Survival”,
“Disclaimer of Warranties”, “Limitation of Liability”, “Indemnity” and all “General” Sections of this
Agreement will survive any expiration or termination of this Agreement.
12. Disclaimer of Warranties. THE SOFTWARE IS PROVIDED BY NVIDIA AS-IS AND WITH ALL FAULTS. TO THE
MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, NVIDIA DISCLAIMS ALL WARRANTIES AND
REPRESENTATIONS OF ANY KIND, WHETHER EXPRESS, IMPLIED OR STATUTORY, RELATING TO OR ARISING
UNDER THIS AGREEMENT, INCLUDING, WITHOUT LIMITATION, THE WARRANTIES OF TITLE,
NONINFRINGEMENT, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, USAGE OF TRADE AND
COURSE OF DEALING. NVIDIA DOES NOT WARRANT OR ASSUME RESPONSIBILITY FOR THE ACCURACY OR
COMPLETENESS OF ANY THIRD-PARTY INFORMATION, TEXT, GRAPHICS, LINKS CONTAINED IN THE
SOFTWARE. WITHOUT LIMITING THE FOREGOING, NVIDIA DOES NOT WARRANT THAT THE SOFTWARE WILL
MEET YOUR REQUIREMENTS, ANY DEFECTS OR ERRORS WILL BE CORRECTED, ANY CERTAIN CONTENT WILL
BE AVAILABLE; OR THAT THE SOFTWARE IS FREE OF VIRUSES OR OTHER HARMFUL COMPONENTS. NO
INFORMATION OR ADVICE GIVEN BY NVIDIA WILL IN ANY WAY INCREASE THE SCOPE OF ANY WARRANTY
EXPRESSLY PROVIDED IN THIS AGREEMENT. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE
APPROPRIATENESS OF USING THE SOFTWARE OR DERIVATIVES AND ASSUME ANY RISKS ASSOCIATED WITH
YOUR USE OF THE SOFTWARE OR DERIVATIVES.
13. Limitations of Liability.
13.1 EXCLUSIONS. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT WILL
NVIDIA BE LIABLE FOR ANY (A) INDIRECT, PUNITIVE, SPECIAL, INCIDENTAL OR CONSEQUENTIAL
DAMAGES, OR (B) DAMAGES FOR THE (I) COST OF PROCURING SUBSTITUTE GOODS OR (II) LOSS OF
PROFITS, REVENUES, USE, DATA OR GOODWILL ARISING OUT OF OR RELATED TO THIS AGREEMENT,
WHETHER BASED ON BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), STRICT LIABILITY, OR
OTHERWISE, AND EVEN IF NVIDIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES AND
EVEN IF A PARTYS REMEDIES FAIL THEIR ESSENTIAL PURPOSE.
13.2 DAMAGES CAP. ADDITIONALLY, TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW,
NVIDIAS TOTAL CUMULATIVE AGGREGATE LIABILITY FOR ANY AND ALL LIABILITIES, OBLIGATIONS OR
CLAIMS ARISING OUT OF OR RELATED TO THIS AGREEMENT WILL NOT EXCEED ONE HUNDRED U.S.
DOLLARS (US$100).
14. Indemnity. You will defend, indemnify and hold harmless NVIDIA and its affiliates, and their respective
employees, contractors, agents, officers and directors, from and against any and all third party claims,
damages, obligations, losses, liabilities, costs or debt, fines, restitutions and expenses (including but not
limited to attorneys fees and costs incident to establishing the right of indemnification) arising out of use of
the Software and Derivatives outside of the scope of this Agreement or in breach of the terms of this
Agreement.
15. General.
15.1 Governing Law and Jurisdiction. This Agreement will be governed in all respects by the laws of the
United States and the laws of the State of Delaware, without regard to conflict of laws principles or
the United Nations Convention on Contracts for the International Sale of Goods. The state and
federal courts residing in Santa Clara County, California will have exclusive jurisdiction over any
dispute or claim arising out of or related to this Agreement, and the parties irrevocably consent to
personal jurisdiction and venue in those courts; except that either party may apply for injunctive
remedies or an equivalent type of urgent legal relief in any jurisdiction.
15.2 Independent Contractors. The parties are independent contractors, and this Agreement does not
create a joint venture, partnership, agency or other form of business association between the
parties. Neither party will have the power to bind the other party or incur any obligation on its
behalf without the other partys prior written consent. Nothing in this Agreement prevents either
party from participating in similar arrangements with third parties.
15.3 No Assignment. NVIDIA may assign, delegate or transfer its rights or obligations under this
Agreement by any means or operation of law. You may not, without NVIDIAs prior written consent,
assign, delegate or transfer any of your rights or obligations under this Agreement by any means or
operation of law, and any attempt to do so is null and void.
15.4 No Waiver. No failure or delay by a party to enforce any term or obligation of this Agreement will
operate as a waiver by that party, or prevent the enforcement of such term or obligation later.
15.5 Trade Compliance. You agree to comply with all applicable export, import, trade and economic
sanctions laws and regulations, as amended, including without limitation U.S. Export Administration
Regulations and Office of Foreign Assets Control regulations. You confirm (a) your understanding
that export or reexport of certain NVIDIA products or technologies may require a license or other
approval from appropriate authorities and (b) that you will not export or reexport any products or
technology, directly or indirectly, without first obtaining any required license or other approval from
appropriate authorities, (i) to any countries that are subject to any U.S. or local export restrictions
(currently including, but not necessarily limited to, Belarus, Cuba, Iran, North Korea, Russia, Syria,
the Region of Crimea, Donetsk Peoples Republic Region and Luhansk Peoples Republic Region); (ii)
to any end-user who you know or have reason to know will utilize them in the design, development
or production of nuclear, chemical or biological weapons, missiles, rocket systems, unmanned air
vehicles capable of a maximum range of at least 300 kilometers, regardless of payload, or intended
for military end-use, or any weapons of mass destruction; (iii) to any end-user who has been
prohibited from participating in the U.S. or local export transactions by any governing authority; or
(iv) to any known military or military-intelligence end-user or for any known military or military-
intelligence end-use in accordance with U.S. trade compliance laws and regulations.
15.6 Government Rights. The Software, documentation and technology (“Protected Items”) are
“Commercial products” as this term is defined at 48 C.F.R. 2.101, consisting of “commercial
computer software” and “commercial computer software documentation” as such terms are used
in, respectively, 48 C.F.R. 12.212 and 48 C.F.R. 227.7202 & 252.227-7014(a)(1). Before any Protected
Items are supplied to the U.S. Government, you will (i) inform the U.S. Government in writing that
the Protected Items are and must be treated as commercial computer software and commercial
computer software documentation developed at private expense; (ii) inform the U.S. Government
that the Protected Items are provided subject to the terms of the Agreement; and (iii) mark the
Protected Items as commercial computer software and commercial computer software
documentation developed at private expense. In no event will you permit the U.S. Government to
acquire rights in Protected Items beyond those specified in 48 C.F.R. 52.227-19(b)(1)-(2) or 252.227-
7013(c) except as expressly approved by NVIDIA in writing.
15.7 Notices. Please direct your legal notices or other correspondence to legalnotices@nvidia.com with a
copy mailed to NVIDIA Corporation, 2788 San Tomas Expressway, Santa Clara, California 95051,
United States of America, Attention: Legal Department. If NVIDIA needs to contact you about the
Software, you consent to receive the notices by email and agree that such notices will satisfy any
legal communication requirements.
15.8 Severability. If a court of competent jurisdiction rules that a provision of this Agreement is
unenforceable, that provision will be deemed modified to the extent necessary to make it
enforceable and the remainder of this Agreement will continue in full force and effect.
15.9 Amendment. Any amendment to this Agreement must be in writing and signed by authorized
representatives of both parties.
15.10 Entire Agreement. Regarding the subject matter of this Agreement, the parties agree that (a) this
Agreement constitutes the entire and exclusive agreement between the parties and supersedes all
prior and contemporaneous communications and (b) any additional or different terms or conditions,
whether contained in purchase orders, order acknowledgments, invoices or otherwise, will not be
binding and are null and void.
(v. February 25, 2025)
NVIDIA Confidential

207
deps/cloudxr/react/README.md vendored Normal file
View File

@@ -0,0 +1,207 @@
# React Three Fiber Example
This is a comprehensive CloudXR.js React Three Fiber example application that demonstrates how to integrate CloudXR streaming with modern React development patterns. This example showcases the power of combining CloudXR.js with React Three Fiber, React Three XR, and React Three UIKit to create immersive XR experiences with rich 3D user interfaces.
> NOTE: This example is not meant to be used for production.
## Overview
This example showcases the integration of CloudXR.js with the React Three Fiber ecosystem, providing:
- **React Three Fiber Integration**: Seamless integration with Three.js through React components
- **React Three XR**: WebXR session management with React hooks and state management
- **React Three UIKit**: Rich 3D user interface components for VR/AR experiences
- **CloudXR Streaming**: Real-time streaming of XR content from a CloudXR server
- **Modern React Patterns**: Hooks, context, and component-based architecture
- **Dual UI System**: 2D HTML interface for configuration and 3D VR interface for interaction
## Quick Start
### Prerequisites
- Node.js (v20 or higher)
- A CloudXR server running and accessible
- A WebXR-compatible device (VR headset, AR device)
### Installation
1. **Navigate to the example folder**
```bash
cd react
```
2. **Install Dependencies**
```bash
# For this early access release, please run the following to install SDK from the given tarball. This step will not be needed when SDK is publicly accessible.
npm install ../nvidia-cloudxr-6.0.0-beta.tgz
npm install
```
3. **Build the Application**
```bash
npm run build
```
4. **Start Development Server**
```bash
npm run dev-server
```
5. **Open in Browser**
- Navigate to `http://localhost:8080` (or the port shown in terminal)
- For desktop browsers, IWER (Immersive Web Emulator Runtime) will automatically load to emulate a Meta Quest 3 headset
### Basic Usage
1. **Configure Connection**
- Enter your CloudXR server IP address
- Set the port (default: 49100)
- Select AR or VR immersive mode
2. **Adjust Settings** (Optional)
- Configure per-eye resolution (perEyeWidth and perEyeHeight, must be multiples of 16)
- Set target frame rate and bitrate
- Adjust XR reference space
3. **Start Streaming**
- Click "CONNECT" to initiate the XR session
- Grant XR permissions when prompted
> NOTE: In order to connect to an actual server and start streaming, you need:
>
> - A CloudXR server running and accessible
> - A WebXR-compatible device (VR/AR headset) or desktop browser (IWER loads automatically for emulation)
## Technical Architecture
### Core Components
#### `App.tsx`
Main React application component managing:
- XR store configuration and session state
- CloudXR component integration
- 2D UI management and event handling
- Error handling and capability checking
- React Three Fiber Canvas setup
#### `CloudXRComponent.tsx`
Handles the core CloudXR streaming functionality:
- CloudXR session lifecycle management
- WebXR session event handling
- WebGL state management and render target preservation
- Frame-by-frame rendering loop with pose tracking
- Integration with Three.js WebXRManager
#### `CloudXR2DUI.tsx`
Manages the 2D HTML interface:
- Form field management and localStorage persistence
- Proxy configuration based on protocol
- Event listener management and cleanup
- Error handling and user feedback
- Configuration validation and updates
#### `CloudXRUI.tsx` (3D UI)
Renders the in-VR user interface:
- React Three UIKit components for 3D UI
- Interactive control buttons with hover effects
- Server information and status display
- Event handler integration
## Development
### Project Structure
```bash
react/
├── src/
│ ├── App.tsx # Main React application
│ ├── CloudXRComponent.tsx # CloudXR streaming component
│ ├── CloudXR2DUI.tsx # 2D UI management class
│ ├── CloudXRUI.tsx # 3D VR UI component
│ ├── index.tsx # React app entry point
│ └── index.html # HTML template
├── public/
│ ├── play-circle.svg # Play button icon (Heroicons)
│ ├── stop-circle.svg # Stop button icon (Heroicons)
│ ├── arrow-uturn-left.svg # Reset button icon (Heroicons)
│ └── arrow-left-start-on-rectangle.svg # Disconnect button icon (Heroicons)
├── package.json # Dependencies and scripts
├── webpack.common.js # Webpack configuration
├── webpack.dev.js # Development webpack config
├── webpack.prod.js # Production webpack config
└── tsconfig.json # TypeScript configuration
```
## React Three Fiber Integration
### XR Store Configuration
The application uses React Three XR's store for XR session management:
```javascript
const store = createXRStore({
foveation: 0,
emulate: { syntheticEnvironment: false },
});
```
### Canvas Setup
React Three Fiber Canvas with WebXR integration:
```typescript
<Canvas events={noEvents} gl={{ preserveDrawingBuffer: true }}>
<XR store={store}>
<CloudXRComponent config={config} />
<CloudXR3DUI onAction1={handleAction1} />
</XR>
</Canvas>
```
### Custom Render Loop
The CloudXR component uses `useFrame` for custom rendering:
```typescript
useFrame((state, delta) => {
if (webXRManager.isPresenting && session) {
// CloudXR rendering logic
cxrSession.sendTrackingStateToServer(timestamp, xrFrame);
cxrSession.render(timestamp, xrFrame, layer);
}
}, -1000);
```
## 3D User Interface
### React Three UIKit Components
The 3D UI uses React Three UIKit for modern VR/AR interfaces:
- **Container**: Layout and positioning components
- **Text**: 3D text rendering with custom fonts
- **Button**: Interactive buttons with hover effects
- **Image**: Texture-based image display
- **Root**: Main UI container with pixel-perfect rendering
### UI Positioning
3D UI elements are positioned in world space:
```typescript
<group position={[1.8, -0.5, -1.3]} rotation={[0, -0.3, 0]}>
<Root pixelSize={0.001} width={1920} height={1440}>
{/* UI components */}
</Root>
</group>
```
### WebGL State Tracking
This example uses WebGL state tracking to prevent rendering conflicts between React Three Fiber and CloudXR. Both libraries render to the same WebGL context, but CloudXR's rendering operations modify WebGL state (framebuffers, textures, buffers, VAOs, shaders, blend modes, etc.) which can interfere with React Three Fiber's expectations. The example wraps the WebGL context with `bindGL()` from `@helpers/WebGLStateBinding`, then uses CloudXR's `onWebGLStateChangeBegin` and `onWebGLStateChangeEnd` callbacks to automatically save and restore state around CloudXR's rendering. This ensures React Three Fiber always finds the WebGL context in the expected state after each CloudXR render operation.
See `examples/helpers/WebGLStateBinding.ts`, `WebGLState.ts`, and `WebGLStateApply.ts` for implementation details. Comprehensive tests are available in `tests/unit/WebGLState.test.ts` and `tests/playwright/WebGLTests/src/WebGLStateBindingTests.ts`.
## License
See the [LICENSE](LICENSE) file for details.
### Third-Party Assets
Icons used in the immersive UI are from [Heroicons](https://heroicons.com/) by Tailwind Labs, licensed under the MIT License. See [HEROICONS_LICENSE](public/HEROICONS_LICENSE) for details.

BIN
deps/cloudxr/react/favicon.ico vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

47
deps/cloudxr/react/package.json vendored Normal file
View File

@@ -0,0 +1,47 @@
{
"name": "cloudxr-react-example",
"version": "6.0.0-beta",
"private": true,
"description": "React Three Fiber WebXR example for CloudXR",
"author": "NVIDIA Corporation",
"license": "SEE LICENSE IN LICENSE",
"keywords": [
"react",
"three.js",
"webxr",
"cloudxr",
"vr"
],
"scripts": {
"build": "webpack --config ./webpack.prod.js",
"dev": "webpack --config ./webpack.dev.js",
"dev-server": "webpack serve --config ./webpack.dev.js --no-open",
"dev-server:https": "HTTPS=true webpack serve --config ./webpack.dev.js --no-open",
"clean": "rimraf dist"
},
"dependencies": {
"@nvidia/cloudxr": "dev",
"@react-three/drei": "^10.6.1",
"@react-three/fiber": "^9.3.0",
"@react-three/uikit": "^1.0.0",
"@react-three/uikit-default": "^1.0.0",
"@react-three/xr": "^6.6.22",
"react": "^19.2.0",
"react-dom": "^19.2.0",
"three": "^0.172.0"
},
"devDependencies": {
"@types/react": "^19.2.0",
"@types/react-dom": "^19.2.0",
"@types/three": "^0.172.0",
"css-loader": "^6.8.1",
"rimraf": "^5.0.5",
"ts-loader": "^9.5.1",
"typescript": "^5.8.2",
"copy-webpack-plugin": "^13.0.0",
"html-webpack-plugin": "^5.6.3",
"style-loader": "^3.3.3",
"webpack-dev-server": "^5.2.1",
"webpack-cli": "^6.0.1"
}
}

View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) Tailwind Labs, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" data-slot="icon">
<path fill-rule="evenodd" d="M16.5 3.75a1.5 1.5 0 0 1 1.5 1.5v13.5a1.5 1.5 0 0 1-1.5 1.5h-6a1.5 1.5 0 0 1-1.5-1.5V15a.75.75 0 0 0-1.5 0v3.75a3 3 0 0 0 3 3h6a3 3 0 0 0 3-3V5.25a3 3 0 0 0-3-3h-6a3 3 0 0 0-3 3V9A.75.75 0 1 0 9 9V5.25a1.5 1.5 0 0 1 1.5-1.5h6ZM5.78 8.47a.75.75 0 0 0-1.06 0l-3 3a.75.75 0 0 0 0 1.06l3 3a.75.75 0 0 0 1.06-1.06l-1.72-1.72H15a.75.75 0 0 0 0-1.5H4.06l1.72-1.72a.75.75 0 0 0 0-1.06Z" clip-rule="evenodd"/>
</svg>

After

Width:  |  Height:  |  Size: 556 B

View File

@@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" data-slot="icon">
<path fill-rule="evenodd" d="M9.53 2.47a.75.75 0 0 1 0 1.06L4.81 8.25H15a6.75 6.75 0 0 1 0 13.5h-3a.75.75 0 0 1 0-1.5h3a5.25 5.25 0 1 0 0-10.5H4.81l4.72 4.72a.75.75 0 1 1-1.06 1.06l-6-6a.75.75 0 0 1 0-1.06l6-6a.75.75 0 0 1 1.06 0Z" clip-rule="evenodd"/>
</svg>

After

Width:  |  Height:  |  Size: 380 B

View File

@@ -0,0 +1,13 @@
<svg width="24" height="24" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<!-- Diamond (top-left) -->
<polygon points="6,2 10,6 6,10 2,6" fill="black"/>
<!-- Triangle (top-right) -->
<polygon points="18,2 22,10 14,10" fill="black"/>
<!-- Circle (bottom-left) -->
<circle cx="6" cy="18" r="4" fill="black"/>
<!-- Square (bottom-right) -->
<rect x="14" y="14" width="8" height="8" fill="black"/>
</svg>

After

Width:  |  Height:  |  Size: 435 B

View File

@@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" data-slot="icon">
<path fill-rule="evenodd" d="M2.25 12c0-5.385 4.365-9.75 9.75-9.75s9.75 4.365 9.75 9.75-4.365 9.75-9.75 9.75S2.25 17.385 2.25 12Zm14.024-.983a1.125 1.125 0 0 1 0 1.966l-5.603 3.113A1.125 1.125 0 0 1 9 15.113V8.887c0-.857.921-1.4 1.671-.983l5.603 3.113Z" clip-rule="evenodd"/>
</svg>

After

Width:  |  Height:  |  Size: 402 B

View File

@@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" data-slot="icon">
<path fill-rule="evenodd" d="M2.25 12c0-5.385 4.365-9.75 9.75-9.75s9.75 4.365 9.75 9.75-4.365 9.75-9.75 9.75S2.25 17.385 2.25 12Zm6-2.438c0-.724.588-1.312 1.313-1.312h4.874c.725 0 1.313.588 1.313 1.313v4.874c0 .725-.588 1.313-1.313 1.313H9.564a1.312 1.312 0 0 1-1.313-1.313V9.564Z" clip-rule="evenodd"/>
</svg>

After

Width:  |  Height:  |  Size: 430 B

555
deps/cloudxr/react/src/App.tsx vendored Normal file
View File

@@ -0,0 +1,555 @@
/**
* App.tsx - Main CloudXR React Application
*
* This is the root component of the CloudXR React example application. It sets up:
* - WebXR session management and XR store configuration
* - CloudXR server configuration (IP, port, stream settings)
* - UI state management (connection status, session state)
* - Integration between CloudXR rendering component and UI components
* - Entry point for AR/VR experiences with CloudXR streaming
*
* The app integrates with the HTML interface which provides a "CONNECT" button
* to enter AR mode and displays the CloudXR UI with controls for teleop actions
* and disconnect when in XR mode.
*/
import { checkCapabilities } from '@helpers/BrowserCapabilities';
import { loadIWERIfNeeded } from '@helpers/LoadIWER';
import { overridePressureObserver } from '@helpers/overridePressureObserver';
import * as CloudXR from '@nvidia/cloudxr';
import { Environment } from '@react-three/drei';
import { Canvas } from '@react-three/fiber';
import { setPreferredColorScheme } from '@react-three/uikit';
import { XR, createXRStore, noEvents, PointerEvents, XROrigin, useXR } from '@react-three/xr';
import { useState, useMemo, useEffect, useRef } from 'react';
import { CloudXR2DUI } from './CloudXR2DUI';
import CloudXRComponent from './CloudXRComponent';
import CloudXR3DUI from './CloudXRUI';
// Override PressureObserver early to catch errors from buggy browser implementations
overridePressureObserver();
const store = createXRStore({
foveation: 0,
emulate: false, // Disable IWER emulation from react in favor of custom iwer loading function
// Configure WebXR input profiles to use local assets
// Use relative path from current page location
baseAssetPath: `${new URL('.', window.location).href}npm/@webxr-input-profiles/assets@${process.env.WEBXR_ASSETS_VERSION}/dist/profiles/`,
hand: {
model: false, // Disable hand models but keep pointer functionality
},
});
setPreferredColorScheme('dark');
const START_TELEOP_COMMAND = {
type: 'teleop_command',
message: {
command: 'start teleop',
},
} as const;
// Environment component like controller-test
function NonAREnvironment() {
// Use local HDR file instead of preset so client doesn't need to download it from CDN
return (
<Environment
blur={0.2}
background={false}
environmentIntensity={2}
files="assets/hdri/potsdamer_platz_1k.hdr"
/>
);
}
function App() {
const COUNTDOWN_MAX_SECONDS = 9;
const COUNTDOWN_STORAGE_KEY = 'cxr.react.countdownSeconds';
// 2D UI management
const [cloudXR2DUI, setCloudXR2DUI] = useState<CloudXR2DUI | null>(null);
// IWER loading state
const [iwerLoaded, setIwerLoaded] = useState(false);
// Capability state management
const [capabilitiesValid, setCapabilitiesValid] = useState(false);
const capabilitiesCheckedRef = useRef(false);
// Connection state management
const [isConnected, setIsConnected] = useState(false);
// Session status management
const [sessionStatus, setSessionStatus] = useState('Disconnected');
// Error message management
const [errorMessage, setErrorMessage] = useState('');
// CloudXR session reference
const [cloudXRSession, setCloudXRSession] = useState<CloudXR.Session | null>(null);
// XR mode state for UI visibility
const [isXRMode, setIsXRMode] = useState(false);
// Server address being used for connection
const [serverAddress, setServerAddress] = useState<string>('');
// Teleop countdown and state
const [isCountingDown, setIsCountingDown] = useState(false);
const [countdownRemaining, setCountdownRemaining] = useState(0);
const [isTeleopRunning, setIsTeleopRunning] = useState(false);
const countdownTimerRef = useRef<number | null>(null);
const [countdownDuration, setCountdownDuration] = useState<number>(() => {
try {
const saved = localStorage.getItem(COUNTDOWN_STORAGE_KEY);
if (saved != null) {
const value = parseInt(saved, 10);
if (!isNaN(value)) {
return Math.min(COUNTDOWN_MAX_SECONDS, Math.max(0, value));
}
}
} catch (_) {}
return 3;
});
// Persist countdown duration on change
useEffect(() => {
try {
localStorage.setItem(COUNTDOWN_STORAGE_KEY, String(countdownDuration));
} catch (_) {}
}, [countdownDuration]);
// Load IWER first (must happen before anything else)
// Note: React Three Fiber's emulation is disabled (emulate: false) to avoid conflicts
useEffect(() => {
const loadIWER = async () => {
const { supportsImmersive, iwerLoaded: wasIwerLoaded } = await loadIWERIfNeeded();
if (!supportsImmersive) {
setErrorMessage('Immersive mode not supported');
setIwerLoaded(false);
setCapabilitiesValid(false);
capabilitiesCheckedRef.current = false; // Reset check flag on failure
return;
}
// IWER loaded successfully, now we can proceed with capability checks
setIwerLoaded(true);
// Store whether IWER was loaded for status message display later
if (wasIwerLoaded) {
sessionStorage.setItem('iwerWasLoaded', 'true');
}
};
loadIWER();
}, []);
// Update button state when IWER fails and UI becomes ready
useEffect(() => {
if (cloudXR2DUI && !iwerLoaded && !capabilitiesValid) {
cloudXR2DUI.setStartButtonState(true, 'CONNECT (immersive mode not supported)');
}
}, [cloudXR2DUI, iwerLoaded, capabilitiesValid]);
// Check capabilities once CloudXR2DUI is ready and IWER is loaded
useEffect(() => {
const checkCapabilitiesOnce = async () => {
if (!cloudXR2DUI || !iwerLoaded) {
return;
}
// Guard: only check capabilities once
if (capabilitiesCheckedRef.current) {
return;
}
capabilitiesCheckedRef.current = true;
// Disable button and show checking status
cloudXR2DUI.setStartButtonState(true, 'CONNECT (checking capabilities)');
let result: { success: boolean; failures: string[]; warnings: string[] } = {
success: false,
failures: [],
warnings: [],
};
try {
result = await checkCapabilities();
} catch (error) {
cloudXR2DUI.showStatus(`Capability check error: ${error}`, 'error');
setCapabilitiesValid(false);
cloudXR2DUI.setStartButtonState(true, 'CONNECT (capability check failed)');
capabilitiesCheckedRef.current = false; // Reset on error for potential retry
return;
}
if (!result.success) {
cloudXR2DUI.showStatus(
'Browser does not meet required capabilities:\n' + result.failures.join('\n'),
'error'
);
setCapabilitiesValid(false);
cloudXR2DUI.setStartButtonState(true, 'CONNECT (capability check failed)');
capabilitiesCheckedRef.current = false; // Reset on failure for potential retry
return;
}
// Show final status message with IWER info if applicable
const iwerWasLoaded = sessionStorage.getItem('iwerWasLoaded') === 'true';
if (result.warnings.length > 0) {
cloudXR2DUI.showStatus('Performance notice:\n' + result.warnings.join('\n'), 'info');
} else if (iwerWasLoaded) {
// Include IWER status in the final success message
cloudXR2DUI.showStatus(
'CloudXR.js SDK is supported. Ready to connect!\nUsing IWER (Immersive Web Emulator Runtime) - Emulating Meta Quest 3.',
'info'
);
} else {
cloudXR2DUI.showStatus('CloudXR.js SDK is supported. Ready to connect!', 'success');
}
setCapabilitiesValid(true);
cloudXR2DUI.setStartButtonState(false, 'CONNECT');
};
checkCapabilitiesOnce();
}, [cloudXR2DUI, iwerLoaded]);
// Track config changes to trigger re-renders when form values change
const [configVersion, setConfigVersion] = useState(0);
// Initialize CloudXR2DUI
useEffect(() => {
// Create and initialize the 2D UI manager
const ui = new CloudXR2DUI(() => {
// Callback when configuration changes
setConfigVersion(v => v + 1);
});
ui.initialize();
ui.setupConnectButtonHandler(
async () => {
// Start XR session
if (ui.getConfiguration().immersiveMode === 'ar') {
await store.enterAR();
} else if (ui.getConfiguration().immersiveMode === 'vr') {
await store.enterVR();
} else {
setErrorMessage('Unrecognized immersive mode');
}
},
(error: Error) => {
setErrorMessage(`Failed to start XR session: ${error}`);
}
);
setCloudXR2DUI(ui);
// Cleanup function
return () => {
if (ui) {
ui.cleanup();
}
};
}, []);
// Update HTML error message display when error state changes
useEffect(() => {
if (cloudXR2DUI) {
if (errorMessage) {
cloudXR2DUI.showError(errorMessage);
} else {
cloudXR2DUI.hideError();
}
}
}, [errorMessage, cloudXR2DUI]);
// Listen for XR session state changes to update button and UI visibility
useEffect(() => {
const handleXRStateChange = () => {
const xrState = store.getState();
if (xrState.mode === 'immersive-ar' || xrState.mode === 'immersive-vr') {
// XR session is active
setIsXRMode(true);
if (cloudXR2DUI) {
cloudXR2DUI.setStartButtonState(true, 'CONNECT (XR session active)');
}
} else {
// XR session ended
setIsXRMode(false);
if (cloudXR2DUI) {
cloudXR2DUI.setStartButtonState(false, 'CONNECT');
}
if (xrState.error) {
setErrorMessage(`XR session error: ${xrState.error}`);
}
}
};
// Subscribe to XR state changes
const unsubscribe = store.subscribe(handleXRStateChange);
// Cleanup
return () => {
unsubscribe();
setIsXRMode(false);
};
}, [cloudXR2DUI]);
// CloudXR status change handler
const handleStatusChange = (connected: boolean, status: string) => {
setIsConnected(connected);
setSessionStatus(status);
};
// UI Event Handlers
const handleStartTeleop = () => {
console.log('Start Teleop pressed');
if (!cloudXRSession) {
console.error('CloudXR session not available');
return;
}
if (isCountingDown || isTeleopRunning) {
return;
}
// Begin countdown before starting teleop (immediately if 0)
if (countdownDuration <= 0) {
setIsCountingDown(false);
setCountdownRemaining(0);
try {
cloudXRSession.sendServerMessage(START_TELEOP_COMMAND);
console.log('Start teleop command sent');
setIsTeleopRunning(true);
} catch (error) {
console.error('Failed to send teleop command:', error);
setIsTeleopRunning(false);
}
return;
}
setIsCountingDown(true);
setCountdownRemaining(countdownDuration);
countdownTimerRef.current = window.setInterval(() => {
setCountdownRemaining(prev => {
if (prev <= 1) {
// Countdown finished
if (countdownTimerRef.current !== null) {
clearInterval(countdownTimerRef.current);
countdownTimerRef.current = null;
}
setIsCountingDown(false);
// Send start teleop command
try {
cloudXRSession.sendServerMessage(START_TELEOP_COMMAND);
console.log('Start teleop command sent');
setIsTeleopRunning(true);
} catch (error) {
console.error('Failed to send teleop command:', error);
setIsTeleopRunning(false);
}
return 0;
}
return prev - 1;
});
}, 1000);
};
const handleStopTeleop = () => {
console.log('Stop Teleop pressed');
// If countdown is active, cancel it and reset state
if (isCountingDown) {
if (countdownTimerRef.current !== null) {
clearInterval(countdownTimerRef.current);
countdownTimerRef.current = null;
}
setIsCountingDown(false);
setCountdownRemaining(0);
return;
}
if (!cloudXRSession) {
console.error('CloudXR session not available');
return;
}
// Send stop teleop command
const teleopCommand = {
type: 'teleop_command',
message: {
command: 'stop teleop',
},
};
try {
cloudXRSession.sendServerMessage(teleopCommand);
console.log('Stop teleop command sent');
setIsTeleopRunning(false);
} catch (error) {
console.error('Failed to send teleop command:', error);
}
};
const handleResetTeleop = () => {
console.log('Reset Teleop pressed');
// Cancel any active countdown
if (countdownTimerRef.current !== null) {
clearInterval(countdownTimerRef.current);
countdownTimerRef.current = null;
}
setIsCountingDown(false);
setCountdownRemaining(0);
if (!cloudXRSession) {
console.error('CloudXR session not available');
return;
}
// Send stop teleop command first
const stopCommand = {
type: 'teleop_command',
message: {
command: 'stop teleop',
},
};
// Send reset teleop command
const resetCommand = {
type: 'teleop_command',
message: {
command: 'reset teleop',
},
};
try {
cloudXRSession.sendServerMessage(stopCommand);
console.log('Stop teleop command sent');
cloudXRSession.sendServerMessage(resetCommand);
console.log('Reset teleop command sent');
setIsTeleopRunning(false);
} catch (error) {
console.error('Failed to send teleop commands:', error);
}
};
const handleDisconnect = () => {
console.log('Disconnect pressed');
// Cleanup countdown state on disconnect
if (countdownTimerRef.current !== null) {
clearInterval(countdownTimerRef.current);
countdownTimerRef.current = null;
}
setIsCountingDown(false);
setCountdownRemaining(0);
setIsTeleopRunning(false);
const xrState = store.getState();
const session = xrState.session;
if (session) {
session.end().catch((err: unknown) => {
setErrorMessage(
`Failed to end XR session: ${err instanceof Error ? err.message : String(err)}`
);
});
}
};
// Countdown configuration handlers (0-5 seconds)
const handleIncreaseCountdown = () => {
if (isCountingDown) return;
setCountdownDuration(prev => Math.min(COUNTDOWN_MAX_SECONDS, prev + 1));
};
const handleDecreaseCountdown = () => {
if (isCountingDown) return;
setCountdownDuration(prev => Math.max(0, prev - 1));
};
// Memo config based on configVersion (manual dependency tracker incremented on config changes)
// eslint-disable-next-line react-hooks/exhaustive-deps
const config = useMemo(
() => (cloudXR2DUI ? cloudXR2DUI.getConfiguration() : null),
[cloudXR2DUI, configVersion]
);
// Sync XR mode state to body class for CSS styling
useEffect(() => {
if (isXRMode) {
document.body.classList.add('xr-mode');
} else {
document.body.classList.remove('xr-mode');
}
return () => {
document.body.classList.remove('xr-mode');
};
}, [isXRMode]);
return (
<>
<Canvas
events={noEvents}
style={{
background: '#000',
width: '100vw',
height: '100vh',
position: 'fixed',
top: 0,
left: 0,
zIndex: -1,
}}
gl={{
preserveDrawingBuffer: true, // Keep buffer for custom rendering
antialias: true,
}}
camera={{ position: [0, 0, 0.65] }}
onWheel={e => {
e.preventDefault();
}}
>
<PointerEvents batchEvents={false} />
<XR store={store}>
<NonAREnvironment />
<XROrigin />
{cloudXR2DUI && config && (
<>
<CloudXRComponent
config={config}
onStatusChange={handleStatusChange}
onError={error => {
if (cloudXR2DUI) {
cloudXR2DUI.showError(error);
}
}}
onSessionReady={setCloudXRSession}
onServerAddress={setServerAddress}
/>
<CloudXR3DUI
onStartTeleop={handleStartTeleop}
onStopTeleop={handleStopTeleop}
onDisconnect={handleDisconnect}
onResetTeleop={handleResetTeleop}
serverAddress={serverAddress || config.serverIP}
sessionStatus={sessionStatus}
playLabel={
isTeleopRunning
? 'Running'
: isCountingDown
? `Starting in ${countdownRemaining} sec...`
: 'Play'
}
playDisabled={isCountingDown || isTeleopRunning}
countdownSeconds={countdownDuration}
onCountdownIncrease={handleIncreaseCountdown}
onCountdownDecrease={handleDecreaseCountdown}
countdownDisabled={isCountingDown}
position={[0, 1.6, -1.8]}
rotation={[0, 0, 0]}
/>
</>
)}
</XR>
</Canvas>
</>
);
}
export default App;

438
deps/cloudxr/react/src/CloudXR2DUI.tsx vendored Normal file
View File

@@ -0,0 +1,438 @@
/**
* CloudXR2DUI.tsx - CloudXR 2D User Interface Management
*
* This class handles all the HTML form interactions, localStorage persistence,
* and form validation for the CloudXR React example. It follows the same pattern
* as the simple example's CloudXRWebUI class, providing a clean separation
* between UI management and React component logic.
*
* Features:
* - Form field management and localStorage persistence
* - Proxy configuration based on protocol
* - Form validation and default value handling
* - Event listener management
* - Error handling and logging
*/
import { CloudXRConfig, enableLocalStorage, setupCertificateAcceptanceLink } from '@helpers/utils';
/**
* 2D UI Management for CloudXR React Example
* Handles the main user interface for CloudXR streaming, including form management,
* localStorage persistence, and user interaction controls.
*/
export class CloudXR2DUI {
/** Button to initiate XR streaming session */
private startButton!: HTMLButtonElement;
/** Input field for the CloudXR server IP address */
private serverIpInput!: HTMLInputElement;
/** Input field for the CloudXR server port number */
private portInput!: HTMLInputElement;
/** Input field for proxy URL configuration */
private proxyUrlInput!: HTMLInputElement;
/** Dropdown to select between AR and VR immersive modes */
private immersiveSelect!: HTMLSelectElement;
/** Dropdown to select device frame rate (FPS) */
private deviceFrameRateSelect!: HTMLSelectElement;
/** Dropdown to select max streaming bitrate (Mbps) */
private maxStreamingBitrateMbpsSelect!: HTMLSelectElement;
/** Input field for per-eye width configuration */
private perEyeWidthInput!: HTMLInputElement;
/** Input field for per-eye height configuration */
private perEyeHeightInput!: HTMLInputElement;
/** Dropdown to select server backend type */
private serverTypeSelect!: HTMLSelectElement;
/** Dropdown to select application type */
private appSelect!: HTMLSelectElement;
/** Dropdown to select reference space for XR tracking */
private referenceSpaceSelect!: HTMLSelectElement;
/** Input for XR reference space X offset (cm) */
private xrOffsetXInput!: HTMLInputElement;
/** Input for XR reference space Y offset (cm) */
private xrOffsetYInput!: HTMLInputElement;
/** Input for XR reference space Z offset (cm) */
private xrOffsetZInput!: HTMLInputElement;
/** Text element displaying proxy configuration help */
private proxyDefaultText!: HTMLElement;
/** Error message box element */
private errorMessageBox!: HTMLElement;
/** Error message text element */
private errorMessageText!: HTMLElement;
/** Certificate acceptance link container */
private certAcceptanceLink!: HTMLElement;
/** Certificate acceptance link anchor */
private certLink!: HTMLAnchorElement;
/** Flag to track if the 2D UI has been initialized */
private initialized: boolean = false;
/** Current form configuration state */
private currentConfiguration: CloudXRConfig;
/** Callback function for configuration changes */
private onConfigurationChange: ((config: CloudXRConfig) => void) | null = null;
/** Connect button click handler for cleanup */
private handleConnectClick: ((event: Event) => void) | null = null;
/** Array to store all event listeners for proper cleanup */
private eventListeners: Array<{
element: HTMLElement;
event: string;
handler: EventListener;
}> = [];
/** Cleanup function for certificate acceptance link */
private certLinkCleanup: (() => void) | null = null;
/**
* Creates a new CloudXR2DUI instance
* @param onConfigurationChange - Callback function called when configuration changes
*/
constructor(onConfigurationChange?: (config: CloudXRConfig) => void) {
this.onConfigurationChange = onConfigurationChange || null;
this.currentConfiguration = this.getDefaultConfiguration();
}
/**
* Initializes the CloudXR2DUI with all necessary components and event handlers
*/
public initialize(): void {
if (this.initialized) {
return;
}
try {
this.initializeElements();
this.setupLocalStorage();
this.setupProxyConfiguration();
this.setupEventListeners();
this.updateConfiguration();
this.setStartButtonState(false, 'CONNECT');
this.initialized = true;
} catch (error) {
// Continue with default values if initialization fails
this.showError(`Failed to initialize CloudXR2DUI: ${error}`);
}
}
/**
* Initializes all DOM element references by their IDs
* Throws an error if any required element is not found
*/
private initializeElements(): void {
this.startButton = this.getElement<HTMLButtonElement>('startButton');
this.serverIpInput = this.getElement<HTMLInputElement>('serverIpInput');
this.portInput = this.getElement<HTMLInputElement>('portInput');
this.proxyUrlInput = this.getElement<HTMLInputElement>('proxyUrl');
this.immersiveSelect = this.getElement<HTMLSelectElement>('immersive');
this.deviceFrameRateSelect = this.getElement<HTMLSelectElement>('deviceFrameRate');
this.maxStreamingBitrateMbpsSelect =
this.getElement<HTMLSelectElement>('maxStreamingBitrateMbps');
this.perEyeWidthInput = this.getElement<HTMLInputElement>('perEyeWidth');
this.perEyeHeightInput = this.getElement<HTMLInputElement>('perEyeHeight');
this.serverTypeSelect = this.getElement<HTMLSelectElement>('serverType');
this.appSelect = this.getElement<HTMLSelectElement>('app');
this.referenceSpaceSelect = this.getElement<HTMLSelectElement>('referenceSpace');
this.xrOffsetXInput = this.getElement<HTMLInputElement>('xrOffsetX');
this.xrOffsetYInput = this.getElement<HTMLInputElement>('xrOffsetY');
this.xrOffsetZInput = this.getElement<HTMLInputElement>('xrOffsetZ');
this.proxyDefaultText = this.getElement<HTMLElement>('proxyDefaultText');
this.errorMessageBox = this.getElement<HTMLElement>('errorMessageBox');
this.errorMessageText = this.getElement<HTMLElement>('errorMessageText');
this.certAcceptanceLink = this.getElement<HTMLElement>('certAcceptanceLink');
this.certLink = this.getElement<HTMLAnchorElement>('certLink');
}
/**
* Gets a DOM element by ID with type safety
* @param id - The element ID to find
* @returns The found element with the specified type
* @throws Error if element is not found
*/
private getElement<T extends HTMLElement>(id: string): T {
const element = document.getElementById(id) as T;
if (!element) {
throw new Error(`Element with id '${id}' not found`);
}
return element;
}
/**
* Gets the default configuration values
* @returns Default configuration object
*/
private getDefaultConfiguration(): CloudXRConfig {
const useSecure = typeof window !== 'undefined' ? window.location.protocol === 'https:' : false;
// Default port: HTTP → 49100, HTTPS without proxy → 48322, HTTPS with proxy → 443
const defaultPort = useSecure ? 48322 : 49100;
return {
serverIP: '127.0.0.1',
port: defaultPort,
useSecureConnection: useSecure,
perEyeWidth: 2048,
perEyeHeight: 1792,
deviceFrameRate: 90,
maxStreamingBitrateMbps: 150,
immersiveMode: 'ar',
app: 'generic',
serverType: 'manual',
proxyUrl: '',
referenceSpaceType: 'auto',
};
}
/**
* Enables localStorage persistence for form inputs
* Automatically saves and restores user preferences
*/
private setupLocalStorage(): void {
enableLocalStorage(this.serverTypeSelect, 'serverType');
enableLocalStorage(this.serverIpInput, 'serverIp');
enableLocalStorage(this.portInput, 'port');
enableLocalStorage(this.perEyeWidthInput, 'perEyeWidth');
enableLocalStorage(this.perEyeHeightInput, 'perEyeHeight');
enableLocalStorage(this.proxyUrlInput, 'proxyUrl');
enableLocalStorage(this.deviceFrameRateSelect, 'deviceFrameRate');
enableLocalStorage(this.maxStreamingBitrateMbpsSelect, 'maxStreamingBitrateMbps');
enableLocalStorage(this.immersiveSelect, 'immersiveMode');
enableLocalStorage(this.appSelect, 'app');
enableLocalStorage(this.referenceSpaceSelect, 'referenceSpace');
enableLocalStorage(this.xrOffsetXInput, 'xrOffsetX');
enableLocalStorage(this.xrOffsetYInput, 'xrOffsetY');
enableLocalStorage(this.xrOffsetZInput, 'xrOffsetZ');
}
/**
* Configures proxy settings based on the current protocol (HTTP/HTTPS)
* Sets appropriate placeholders and help text for port and proxy URL inputs
*/
private setupProxyConfiguration(): void {
// Update port placeholder based on protocol
if (window.location.protocol === 'https:') {
this.portInput.placeholder = 'Port (default: 48322, or 443 if proxy URL set)';
} else {
this.portInput.placeholder = 'Port (default: 49100)';
}
// Set default text and placeholder based on protocol
if (window.location.protocol === 'https:') {
this.proxyDefaultText.textContent =
'Optional: Leave empty for direct WSS connection, or provide URL for proxy routing (e.g., https://proxy.example.com/)';
this.proxyUrlInput.placeholder = '';
} else {
this.proxyDefaultText.textContent = 'Not needed for HTTP - uses direct WS connection';
this.proxyUrlInput.placeholder = '';
}
}
/**
* Sets up event listeners for form input changes
* Handles both input and change events for better compatibility
*/
private setupEventListeners(): void {
// Update configuration when form inputs change
const updateConfig = () => this.updateConfiguration();
// Helper function to add listeners and store them for cleanup
const addListener = (element: HTMLElement, event: string, handler: EventListener) => {
element.addEventListener(event, handler);
this.eventListeners.push({ element, event, handler });
};
// Add event listeners for all form fields
addListener(this.serverTypeSelect, 'change', updateConfig);
addListener(this.serverIpInput, 'input', updateConfig);
addListener(this.serverIpInput, 'change', updateConfig);
addListener(this.portInput, 'input', updateConfig);
addListener(this.portInput, 'change', updateConfig);
addListener(this.perEyeWidthInput, 'input', updateConfig);
addListener(this.perEyeWidthInput, 'change', updateConfig);
addListener(this.perEyeHeightInput, 'input', updateConfig);
addListener(this.perEyeHeightInput, 'change', updateConfig);
addListener(this.deviceFrameRateSelect, 'change', updateConfig);
addListener(this.maxStreamingBitrateMbpsSelect, 'change', updateConfig);
addListener(this.immersiveSelect, 'change', updateConfig);
addListener(this.appSelect, 'change', updateConfig);
addListener(this.referenceSpaceSelect, 'change', updateConfig);
addListener(this.xrOffsetXInput, 'input', updateConfig);
addListener(this.xrOffsetXInput, 'change', updateConfig);
addListener(this.xrOffsetYInput, 'input', updateConfig);
addListener(this.xrOffsetYInput, 'change', updateConfig);
addListener(this.xrOffsetZInput, 'input', updateConfig);
addListener(this.xrOffsetZInput, 'change', updateConfig);
addListener(this.proxyUrlInput, 'input', updateConfig);
addListener(this.proxyUrlInput, 'change', updateConfig);
// Set up certificate acceptance link and store cleanup function
this.certLinkCleanup = setupCertificateAcceptanceLink(
this.serverIpInput,
this.portInput,
this.proxyUrlInput,
this.certAcceptanceLink,
this.certLink
);
}
/**
* Updates the current configuration from form values
* Calls the configuration change callback if provided
*/
private updateConfiguration(): void {
const useSecure = this.getDefaultConfiguration().useSecureConnection;
const portValue = parseInt(this.portInput.value);
const hasProxy = this.proxyUrlInput.value.trim().length > 0;
// Smart default port based on connection type and proxy usage
let defaultPort = 49100; // HTTP default
if (useSecure) {
defaultPort = hasProxy ? 443 : 48322; // HTTPS with proxy → 443, HTTPS without → 48322
}
const newConfiguration: CloudXRConfig = {
serverIP: this.serverIpInput.value || this.getDefaultConfiguration().serverIP,
port: portValue || defaultPort,
useSecureConnection: useSecure,
perEyeWidth:
parseInt(this.perEyeWidthInput.value) || this.getDefaultConfiguration().perEyeWidth,
perEyeHeight:
parseInt(this.perEyeHeightInput.value) || this.getDefaultConfiguration().perEyeHeight,
deviceFrameRate:
parseInt(this.deviceFrameRateSelect.value) ||
this.getDefaultConfiguration().deviceFrameRate,
maxStreamingBitrateMbps:
parseInt(this.maxStreamingBitrateMbpsSelect.value) ||
this.getDefaultConfiguration().maxStreamingBitrateMbps,
immersiveMode:
(this.immersiveSelect.value as 'ar' | 'vr') || this.getDefaultConfiguration().immersiveMode,
app: this.appSelect.value || this.getDefaultConfiguration().app,
serverType: this.serverTypeSelect.value || this.getDefaultConfiguration().serverType,
proxyUrl: this.proxyUrlInput.value || this.getDefaultConfiguration().proxyUrl,
referenceSpaceType:
(this.referenceSpaceSelect.value as 'auto' | 'local-floor' | 'local' | 'viewer') ||
this.getDefaultConfiguration().referenceSpaceType,
// Convert cm from UI into meters for config (respect 0; if invalid, use 0)
xrOffsetX: (() => {
const v = parseFloat(this.xrOffsetXInput.value);
return Number.isFinite(v) ? v / 100 : 0;
})(),
xrOffsetY: (() => {
const v = parseFloat(this.xrOffsetYInput.value);
return Number.isFinite(v) ? v / 100 : 0;
})(),
xrOffsetZ: (() => {
const v = parseFloat(this.xrOffsetZInput.value);
return Number.isFinite(v) ? v / 100 : 0;
})(),
};
this.currentConfiguration = newConfiguration;
// Call the configuration change callback if provided
if (this.onConfigurationChange) {
this.onConfigurationChange(newConfiguration);
}
}
/**
* Gets the current configuration
* @returns Current configuration object
*/
public getConfiguration(): CloudXRConfig {
return { ...this.currentConfiguration };
}
/**
* Sets the start button state
* @param disabled - Whether the button should be disabled
* @param text - Text to display on the button
*/
public setStartButtonState(disabled: boolean, text: string): void {
if (this.startButton) {
this.startButton.disabled = disabled;
this.startButton.innerHTML = text;
}
}
/**
* Sets up the connect button click handler
* @param onConnect - Function to call when connect button is clicked
* @param onError - Function to call when an error occurs
*/
public setupConnectButtonHandler(
onConnect: () => Promise<void>,
onError: (error: Error) => void
): void {
if (this.startButton) {
// Remove any existing listener
if (this.handleConnectClick) {
this.startButton.removeEventListener('click', this.handleConnectClick);
}
// Create new handler
this.handleConnectClick = async () => {
// Disable button during XR session
this.setStartButtonState(true, 'CONNECT (starting XR session...)');
try {
await onConnect();
} catch (error) {
this.setStartButtonState(false, 'CONNECT');
onError(error as Error);
}
};
// Add the new listener
this.startButton.addEventListener('click', this.handleConnectClick);
}
}
/**
* Shows a status message in the UI with a specific type
* @param message - Message to display
* @param type - Message type: 'success', 'error', or 'info'
*/
public showStatus(message: string, type: 'success' | 'error' | 'info'): void {
if (this.errorMessageText && this.errorMessageBox) {
this.errorMessageText.textContent = message;
this.errorMessageBox.className = `error-message-box show ${type}`;
}
console[type === 'error' ? 'error' : 'info'](message);
}
/**
* Shows an error message in the UI
* @param message - Error message to display
*/
public showError(message: string): void {
this.showStatus(message, 'error');
}
/**
* Hides the error message
*/
public hideError(): void {
if (this.errorMessageBox) {
this.errorMessageBox.classList.remove('show');
}
}
/**
* Cleans up event listeners and resources
* Should be called when the component unmounts
*/
public cleanup(): void {
// Remove all stored event listeners
this.eventListeners.forEach(({ element, event, handler }) => {
element.removeEventListener(event, handler);
});
this.eventListeners = [];
// Remove CONNECT button listener
if (this.startButton && this.handleConnectClick) {
this.startButton.removeEventListener('click', this.handleConnectClick);
this.handleConnectClick = null;
}
// Clean up certificate acceptance link listeners
if (this.certLinkCleanup) {
this.certLinkCleanup();
this.certLinkCleanup = null;
}
}
}

View File

@@ -0,0 +1,288 @@
/**
* CloudXRComponent.tsx - CloudXR WebXR Integration Component
*
* This component handles the core CloudXR streaming functionality and WebXR integration.
* It manages:
* - CloudXR session lifecycle (creation, connection, disconnection, cleanup)
* - WebXR session event handling (sessionstart, sessionend)
* - WebGL state management and render target preservation
* - Frame-by-frame rendering loop with pose tracking and stream rendering
* - Server configuration and connection parameters
* - Status reporting back to parent components
*
* The component accepts configuration via props and communicates status changes
* and disconnect requests through callback props. It integrates with Three.js
* and React Three Fiber for WebXR rendering while preserving WebGL state
* for CloudXR's custom rendering pipeline.
*/
import { getConnectionConfig, ConnectionConfiguration, CloudXRConfig } from '@helpers/utils';
import { bindGL } from '@helpers/WebGLStateBinding';
import * as CloudXR from '@nvidia/cloudxr';
import { useThree, useFrame } from '@react-three/fiber';
import { useXR } from '@react-three/xr';
import { useRef, useEffect } from 'react';
import type { WebGLRenderer } from 'three';
interface CloudXRComponentProps {
config: CloudXRConfig;
onStatusChange?: (isConnected: boolean, status: string) => void;
onError?: (error: string) => void;
onSessionReady?: (session: CloudXR.Session | null) => void;
onServerAddress?: (address: string) => void;
}
// React component that integrates CloudXR with Three.js/WebXR
// This component handles the CloudXR session lifecycle and render loop
export default function CloudXRComponent({
config,
onStatusChange,
onError,
onSessionReady,
onServerAddress,
}: CloudXRComponentProps) {
const threeRenderer: WebGLRenderer = useThree().gl;
const { session } = useXR();
// React reference to the CloudXR session that persists across re-renders.
const cxrSessionRef = useRef<CloudXR.Session | null>(null);
// Disable Three.js so it doesn't clear the framebuffer after CloudXR renders.
threeRenderer.autoClear = false;
// Access Three.js WebXRManager and WebGL context.
const gl: WebGL2RenderingContext = threeRenderer.getContext() as WebGL2RenderingContext;
const trackedGL = bindGL(gl);
// Set up event listeners in useEffect to add them only once
useEffect(() => {
const webXRManager = threeRenderer.xr;
if (webXRManager) {
const handleSessionStart = async () => {
// Explicitly request the desired reference space from the XRSession to avoid
// inheriting a default 'local-floor' space that could stack with UI offsets.
let referenceSpace: XRReferenceSpace | null = null;
try {
const xrSession: XRSession | null = (webXRManager as any).getSession
? (webXRManager as any).getSession()
: null;
if (xrSession) {
if (config.referenceSpaceType === 'auto') {
const fallbacks: XRReferenceSpaceType[] = [
'local-floor',
'local',
'viewer',
'unbounded',
];
for (const t of fallbacks) {
try {
referenceSpace = await xrSession.requestReferenceSpace(t);
if (referenceSpace) break;
} catch (_) {}
}
} else {
try {
referenceSpace = await xrSession.requestReferenceSpace(
config.referenceSpaceType as XRReferenceSpaceType
);
} catch (error) {
console.error(
`Failed to request reference space '${config.referenceSpaceType}':`,
error
);
}
}
}
} catch (error) {
console.error('Failed to request XR reference space:', error);
referenceSpace = null;
}
if (!referenceSpace) {
// As a last resort, fall back to WebXRManager's current reference space
referenceSpace = webXRManager.getReferenceSpace();
}
if (referenceSpace) {
// Ensure that the session is not already created.
if (cxrSessionRef.current) {
console.error('CloudXR session already exists');
return;
}
const glBinding = webXRManager.getBinding();
if (!glBinding) {
console.warn('No WebGL binding found');
}
// Apply proxy configuration logic
let connectionConfig: ConnectionConfiguration;
try {
connectionConfig = getConnectionConfig(config.serverIP, config.port, config.proxyUrl);
onServerAddress?.(connectionConfig.serverIP);
} catch (error) {
onStatusChange?.(false, 'Configuration Error');
onError?.(`Proxy configuration failed: ${error}`);
return;
}
// Apply XR offset if provided in config (meters)
const offsetX = config.xrOffsetX || 0;
const offsetY = config.xrOffsetY || 0;
const offsetZ = config.xrOffsetZ || 0;
if (offsetX !== 0 || offsetY !== 0 || offsetZ !== 0) {
const offsetTransform = new XRRigidTransform(
{ x: offsetX, y: offsetY, z: offsetZ },
{ x: 0, y: 0, z: 0, w: 1 }
);
referenceSpace = referenceSpace.getOffsetReferenceSpace(offsetTransform);
}
// Fill in CloudXR session options.
const cloudXROptions: CloudXR.SessionOptions = {
serverAddress: connectionConfig.serverIP,
serverPort: connectionConfig.port,
useSecureConnection: connectionConfig.useSecureConnection,
perEyeWidth: config.perEyeWidth,
perEyeHeight: config.perEyeHeight,
gl: gl,
referenceSpace: referenceSpace,
deviceFrameRate: config.deviceFrameRate,
maxStreamingBitrateKbps: config.maxStreamingBitrateMbps * 1000, // Convert Mbps to Kbps
glBinding: glBinding,
telemetry: {
enabled: true,
appInfo: {
version: '6.0.0-beta',
product: 'CloudXR.js React Example',
},
},
};
// Store the render target and key GL bindings to restore after CloudXR rendering
const cloudXRDelegates: CloudXR.SessionDelegates = {
onWebGLStateChangeBegin: () => {
// Save the current render target before CloudXR changes state
trackedGL.save();
},
onWebGLStateChangeEnd: () => {
// Restore the tracked GL state to the state before CloudXR rendering.
trackedGL.restore();
},
onStreamStarted: () => {
console.debug('CloudXR stream started');
onStatusChange?.(true, 'Connected');
},
onStreamStopped: (error?: Error) => {
if (error) {
onStatusChange?.(false, 'Error');
onError?.(`CloudXR session stopped with error: ${error.message}`);
} else {
console.debug('CloudXR session stopped');
onStatusChange?.(false, 'Disconnected');
}
// Clear the session reference
cxrSessionRef.current = null;
onSessionReady?.(null);
},
};
// Create the CloudXR session.
let cxrSession: CloudXR.Session;
try {
cxrSession = CloudXR.createSession(cloudXROptions, cloudXRDelegates);
} catch (error) {
onStatusChange?.(false, 'Session Creation Failed');
onError?.(`Failed to create CloudXR session: ${error}`);
return;
}
// Store the session in the ref so it persists across re-renders
cxrSessionRef.current = cxrSession;
// Notify parent that session is ready
onSessionReady?.(cxrSession);
// Start session (synchronous call that initiates connection)
try {
cxrSession.connect();
console.log('CloudXR session connect initiated');
// Note: The session will transition to Connected state via the onStreamStarted callback
// Use cxrSession.state to check if streaming has actually started
} catch (error) {
onStatusChange?.(false, 'Connection Failed');
// Report error via callback
onError?.('Failed to connect CloudXR session');
// Clean up the failed session
cxrSessionRef.current = null;
}
}
};
const handleSessionEnd = () => {
if (cxrSessionRef.current) {
cxrSessionRef.current.disconnect();
cxrSessionRef.current = null;
onSessionReady?.(null);
}
};
// Add start+end session event listeners to the WebXRManager.
webXRManager.addEventListener('sessionstart', handleSessionStart);
webXRManager.addEventListener('sessionend', handleSessionEnd);
// Cleanup function to remove listeners
return () => {
webXRManager.removeEventListener('sessionstart', handleSessionStart);
webXRManager.removeEventListener('sessionend', handleSessionEnd);
};
}
}, [threeRenderer, config]); // Re-register handlers when renderer or config changes
// Custom render loop - runs every frame
useFrame((state, delta) => {
const webXRManager = threeRenderer.xr;
if (webXRManager.isPresenting && session) {
// Access the current WebXR XRFrame
const xrFrame = state.gl.xr.getFrame();
if (xrFrame) {
// Get THREE WebXRManager from the the useFrame state.
const webXRManager = state.gl.xr;
if (!cxrSessionRef || !cxrSessionRef.current) {
console.debug('Skipping frame, no session yet');
// Clear the framebuffer as we've set autoClear to false.
threeRenderer.clear();
return;
}
// Get session from reference.
const cxrSession: CloudXR.Session = cxrSessionRef.current;
// If the CloudXR session is not connected, skip the frame.
if (cxrSession.state !== CloudXR.SessionState.Connected) {
console.debug('Skipping frame, session not connected, state:', cxrSession.state);
// Clear the framebuffer as we've set autoClear to false.
threeRenderer.clear();
return;
}
// Get timestamp from useFrame state and convert to milliseconds.
const timestamp: DOMHighResTimeStamp = state.clock.elapsedTime * 1000;
// Send the tracking state (including viewer pose and hand/controller data) to the server, this will trigger server-side rendering for frame.
cxrSession.sendTrackingStateToServer(timestamp, xrFrame);
// Get the WebXR layer from THREE WebXRManager.
let layer: XRWebGLLayer = webXRManager.getBaseLayer() as XRWebGLLayer;
// Render the current streamed CloudXR frame (not the frame that was just sent to the server).
cxrSession.render(timestamp, xrFrame, layer);
}
}
}, -1000);
return null;
}

218
deps/cloudxr/react/src/CloudXRUI.tsx vendored Normal file
View File

@@ -0,0 +1,218 @@
/**
* CloudXRUI.tsx - CloudXR User Interface Component
*
* This component renders the in-VR user interface for the CloudXR application using
* React Three UIKit. It provides:
* - CloudXR branding and title display
* - Server connection information and status display
* - Interactive control buttons (Start Teleop, Reset Teleop, Disconnect)
* - Responsive button layout with hover effects
* - Integration with parent component event handlers
* - Configurable position and rotation in world space for flexible UI placement
*
* The UI is positioned in 3D space and designed for VR/AR interaction with
* visual feedback and clear button labeling. All interactions are passed
* back to the parent component through callback props.
*/
import { Container, Text, Image } from '@react-three/uikit';
import { Button } from '@react-three/uikit-default';
import React from 'react';
interface CloudXRUIProps {
onStartTeleop?: () => void;
onDisconnect?: () => void;
onResetTeleop?: () => void;
serverAddress?: string;
sessionStatus?: string;
playLabel?: string;
playDisabled?: boolean;
countdownSeconds?: number;
onCountdownIncrease?: () => void;
onCountdownDecrease?: () => void;
countdownDisabled?: boolean;
position?: [number, number, number];
rotation?: [number, number, number];
}
export default function CloudXR3DUI({
onStartTeleop,
onDisconnect,
onResetTeleop,
serverAddress = '127.0.0.1',
sessionStatus = 'Disconnected',
playLabel = 'Play',
playDisabled = false,
countdownSeconds,
onCountdownIncrease,
onCountdownDecrease,
countdownDisabled = false,
position = [1.8, 1.75, -1.3],
rotation = [0, -0.3, 0],
}: CloudXRUIProps) {
return (
<group position={position} rotation={rotation}>
<Container
pixelSize={0.001}
width={1920}
height={1584}
alignItems="center"
justifyContent="center"
pointerEvents="auto"
padding={40}
sizeX={3}
sizeY={2.475}
>
<Container
width={1600}
height={900}
backgroundColor="rgba(40, 40, 40, 0.85)"
borderRadius={20}
padding={60}
paddingBottom={80}
alignItems="center"
justifyContent="center"
flexDirection="column"
gap={36}
>
{/* Title */}
<Text fontSize={96} fontWeight="bold" color="white" textAlign="center">
Controls
</Text>
{/* Server Info */}
<Text fontSize={48} color="white" textAlign="center" marginBottom={24}>
Server address: {serverAddress}
</Text>
<Text fontSize={48} color="white" textAlign="center" marginBottom={48}>
Session status: {sessionStatus}
</Text>
{/* Countdown Config Row */}
<Container flexDirection="row" gap={24} alignItems="center" justifyContent="center">
<Text fontSize={40} color="white">
Countdown
</Text>
<Button
onClick={onCountdownDecrease}
variant="default"
width={105}
height={105}
borderRadius={52.5}
backgroundColor="rgba(220, 220, 220, 0.9)"
disabled={countdownDisabled}
>
<Text fontSize={48} color="black" fontWeight="bold">
-
</Text>
</Button>
<Container
width={180}
height={105}
alignItems="center"
justifyContent="center"
backgroundColor="rgba(255,255,255,0.9)"
borderRadius={12}
>
<Text fontSize={56} color="black">
{countdownSeconds}s
</Text>
</Container>
<Button
onClick={onCountdownIncrease}
variant="default"
width={105}
height={105}
borderRadius={52.5}
backgroundColor="rgba(220, 220, 220, 0.9)"
disabled={countdownDisabled}
>
<Text fontSize={48} color="black" fontWeight="bold">
+
</Text>
</Button>
</Container>
{/* Button Grid */}
<Container
flexDirection="column"
gap={60}
alignItems="center"
justifyContent="center"
width="100%"
>
{/* Start/reset row*/}
<Container flexDirection="row" gap={60} justifyContent="center">
<Button
onClick={onStartTeleop}
variant="default"
width={480}
height={120}
borderRadius={40}
backgroundColor="rgba(220, 220, 220, 0.9)"
hover={{
backgroundColor: 'rgba(100, 150, 255, 1)',
borderColor: 'white',
borderWidth: 2,
}}
disabled={playDisabled}
>
<Container flexDirection="row" alignItems="center" gap={12}>
{playLabel === 'Play' && <Image src="./play-circle.svg" width={60} height={60} />}
<Text fontSize={48} color="black" fontWeight="medium">
{playLabel}
</Text>
</Container>
</Button>
<Button
onClick={onResetTeleop}
variant="default"
width={480}
height={120}
borderRadius={40}
backgroundColor="rgba(220, 220, 220, 0.9)"
hover={{
backgroundColor: 'rgba(100, 150, 255, 1)',
borderColor: 'white',
borderWidth: 2,
}}
>
<Container flexDirection="row" alignItems="center" gap={12}>
<Image src="./arrow-uturn-left.svg" width={60} height={60} />
<Text fontSize={48} color="black" fontWeight="medium">
Reset
</Text>
</Container>
</Button>
</Container>
{/* Bottom Row */}
<Container flexDirection="row" justifyContent="center">
<Button
onClick={onDisconnect}
variant="destructive"
width={330}
height={105}
borderRadius={35}
backgroundColor="rgba(255, 150, 150, 0.9)"
hover={{
backgroundColor: 'rgba(255, 50, 50, 1)',
borderColor: 'white',
borderWidth: 2,
}}
>
<Container flexDirection="row" alignItems="center" gap={12}>
<Image src="./arrow-left-start-on-rectangle.svg" width={60} height={60} />
<Text fontSize={40} color="black" fontWeight="medium">
Disconnect
</Text>
</Container>
</Button>
</Container>
</Container>
</Container>
</Container>
</group>
);
}

576
deps/cloudxr/react/src/index.html vendored Normal file
View File

@@ -0,0 +1,576 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no, viewport-fit=cover">
<meta name="robots" content="noindex, nofollow">
<meta name="description" content="NVIDIA CloudXR.js React Three Fiber Example for VR/AR streaming">
<title>NVIDIA CloudXR.js React Three Fiber Example</title>
<link rel="icon" type="image/x-icon" href="favicon.ico">
<style>
:root {
--primary-green: #76b900;
--border-color: #e0e0e0;
--text-main: #111;
--background-main: #fff;
--input-radius: 0px;
--input-border: 2px solid var(--border-color);
}
* {
box-sizing: border-box;
-webkit-tap-highlight-color: transparent;
}
html,
body {
margin: 0;
padding: 0;
height: 100%;
overflow: hidden;
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', sans-serif;
background: var(--background-main);
color: var(--text-main);
touch-action: manipulation;
-webkit-font-smoothing: antialiased;
}
@supports (padding: max(0px)) {
body {
padding: max(0px, env(safe-area-inset-top)) max(0px, env(safe-area-inset-right)) max(0px, env(safe-area-inset-bottom)) max(0px, env(safe-area-inset-left));
}
}
/* Top Banner */
.top-banner {
width: 100vw;
height: 8px;
background: var(--primary-green);
margin: 0;
padding: 0;
}
/* Header */
header {
background: var(--background-main);
padding: 16px 0 16px 24px;
min-height: unset;
box-shadow: none;
border-bottom: 1px solid var(--border-color);
}
h1 {
font-size: 1.5rem;
font-weight: 600;
}
main {
display: flex;
height: calc(100vh - 64px);
}
aside {
width: 540px;
min-width: 400px;
padding: 32px 24px 24px;
display: flex;
flex-direction: column;
border-right: 1px solid var(--border-color);
overflow-y: auto;
}
.settings-title {
font-size: 1.5rem;
font-weight: 700;
margin-bottom: 24px;
}
/* Inputs and Selectors */
.ui-input,
select {
width: 100%;
min-width: 300px;
padding: 16px 12px;
font-size: 1rem;
border: var(--input-border);
border-radius: var(--input-radius);
background: var(--background-main);
min-height: 48px;
}
.ui-input {
margin-bottom: 16px;
}
.ui-input:focus,
select:focus {
outline: none;
border-color: var(--primary-green);
box-shadow: 0 0 0 3px rgba(118, 185, 0, 0.2);
}
.ui-input::placeholder {
color: #999;
}
.settings-table {
width: 100%;
margin-bottom: 24px;
border-collapse: collapse;
}
.settings-table td {
padding: 12px 0;
vertical-align: middle;
}
.settings-table label {
font-weight: 600;
display: inline-block;
min-width: 140px;
font-size: 1rem;
}
.settings-table select {
width: 200px;
padding: 12px 8px;
cursor: pointer;
}
button {
background: var(--primary-green);
color: #fff;
border: none;
border-radius: var(--input-radius);
padding: 16px 24px;
font-size: 1.1rem;
font-weight: 700;
cursor: pointer;
transition: all 0.3s ease;
min-height: 56px;
text-transform: uppercase;
letter-spacing: 0.5px;
}
button:hover {
background: #8ac800;
}
button:disabled {
background: #666;
cursor: not-allowed;
transform: none;
box-shadow: none;
}
.start-button {
margin-top: 16px;
width: auto;
font-size: 1rem;
padding: 12px 20px;
color: #000;
}
section {
flex: 1;
padding: 48px 32px 32px;
background: #f8f8f8;
overflow-y: auto;
}
.debug-title {
font-weight: 700;
margin-bottom: 16px;
font-size: 1.3rem;
}
.debug-list {
margin: 0;
padding: 0;
list-style: none;
color: #666;
}
.debug-list li {
padding: 8px 0;
border-bottom: 1px solid var(--border-color);
font-size: 0.95rem;
}
.debug-list li:last-child {
border-bottom: none;
}
@media (max-width: 768px) {
main {
flex-direction: column;
height: auto;
min-height: calc(100vh - 64px);
}
html,
body {
overflow: auto;
}
aside {
width: 100%;
min-width: auto;
border-right: none;
border-bottom: 1px solid #404040;
}
section {
padding: 24px 16px;
}
header {
font-size: 1.1rem;
padding: 12px 16px;
}
.settings-table select {
width: 100%;
max-width: 200px;
}
}
@media (prefers-contrast: high) {
body,
aside,
.ui-input,
select {
background: var(--background-main);
color: #000;
}
aside {
border-color: #000;
}
section {
background: #f0f0f0;
}
.ui-input,
select {
border-color: #000;
}
}
@media (prefers-reduced-motion: reduce) {
* {
animation-duration: 0.01ms !important;
animation-iteration-count: 1 !important;
transition-duration: 0.01ms !important;
}
}
.ui-input:focus-visible,
select:focus-visible,
button:focus-visible {
outline: 3px solid var(--primary-green);
outline-offset: 2px;
}
.input-label {
display: block;
font-size: 0.92rem;
color: #444;
margin-bottom: 5px;
margin-top: 10px;
font-weight: 500;
}
.input-label:first-child {
margin-top: 0;
}
/* Console Logs Section */
.console-logs-section {
margin-top: 24px;
}
.console-logs-textarea {
height: 300px;
font-family: 'Courier New', monospace;
resize: vertical;
overflow-y: auto;
}
.console-logs-buttons {
margin-top: 8px;
}
.console-logs-button {
background: #666;
font-size: 0.9rem;
padding: 8px 16px;
margin-right: 8px;
}
.console-logs-button:last-child {
margin-right: 0;
}
/* Configuration Section */
.config-section {
margin-top: 24px;
}
.config-input {
margin-bottom: 8px;
}
.config-text {
font-size: 0.85rem;
color: #666;
margin-bottom: 16px;
}
/* Exit Button */
.exit-button {
position: fixed;
top: 20px;
right: 20px;
background: #ff3300cc;
color: white;
border: none;
border-radius: 5px;
padding: 10px 15px;
font-weight: bold;
font-size: 16px;
cursor: pointer;
transition: all 0.3s ease;
display: none;
z-index: 10000;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.3);
}
.exit-button:hover {
background: #ff3300;
transform: scale(1.05);
box-shadow: 0 4px 8px rgba(0, 0, 0, 0.2);
}
/* Error Message Box */
.error-message-box {
display: none;
background: #ffebee;
border: 2px solid #f44336;
border-radius: 4px;
padding: 12px 16px;
margin: 16px 0;
color: #c62828;
font-size: 0.9rem;
font-weight: 500;
line-height: 1.4;
}
.error-message-box.show {
display: block;
}
/* Success state - green */
.error-message-box.success {
background: #e8f5e9;
border-color: var(--primary-green);
color: #2e7d32;
}
/* Info state - light blue */
.error-message-box.info {
background: #e3f2fd;
border-color: #2196f3;
color: #1565c0;
}
/* Error state - red (default) */
.error-message-box.error {
background: #ffebee;
border-color: #f44336;
color: #c62828;
}
.error-message-box .error-icon {
display: inline-block;
margin-right: 8px;
font-weight: bold;
}
/* Certificate Acceptance Link */
.cert-acceptance-link {
background: #e3f2fd;
border: 2px solid #2196f3;
border-radius: 4px;
padding: 12px 16px;
margin: 8px 0 16px 0;
font-size: 0.9rem;
line-height: 1.4;
}
.cert-acceptance-link .cert-icon {
display: inline-block;
margin-right: 8px;
}
.cert-acceptance-link a {
color: #1565c0;
text-decoration: none;
font-weight: 600;
border-bottom: 1px solid #1565c0;
}
.cert-acceptance-link a:hover {
color: #0d47a1;
border-bottom-color: #0d47a1;
}
/* Hide 2D UI when in XR mode */
body.xr-mode #2d-ui {
display: none;
}
</style>
</head>
<body>
<div id="2d-ui">
<div class="top-banner"></div>
<header>
<h1>NVIDIA CloudXR.js React Three Fiber Example</h1>
</header>
<main>
<aside>
<button id="startButton" class="start-button" type="button" aria-label="Connect"
disabled>CONNECT</button>
<div id="errorMessageBox" class="error-message-box" role="alert" aria-live="polite">
<span class="error-icon"></span>
<span id="errorMessageText"></span>
</div>
<h2 class="settings-title">Settings</h2>
<form id="settingsForm">
<label for="serverType" class="input-label">Select Server Backend</label>
<select id="serverType" class="ui-input">
<option value="manual">Manual Input IP:Port</option>
<option value="nvcf" disabled>NVCF</option>
</select>
<div id="manualFields">
<label for="serverIpInput" class="input-label">Server IP</label>
<input id="serverIpInput" class="ui-input" type="text" placeholder="Server IP"
spellcheck="false">
<label for="portInput" class="input-label">Port</label>
<input id="portInput" class="ui-input" type="number" placeholder="Port (default: 49100)"
spellcheck="false" min="1" max="65535">
<div id="certAcceptanceLink" class="cert-acceptance-link" style="display: none;">
<span class="cert-icon">🔒</span>
<a id="certLink" href="#" target="_blank" rel="noopener noreferrer"></a>
</div>
</div>
<label for="app" class="input-label">Application</label>
<select id="app" name="app" class="ui-input" aria-label="Select application">
<option value="generic">Generic Client</option>
<!-- TODO: Add other applications here -->
</select>
<label for="immersive" class="input-label">Immersive Mode</label>
<select id="immersive" name="immersive" class="ui-input" aria-label="Select immersive mode">
<option value="ar" selected>AR Immersive</option>
<option value="vr">VR Immersive</option>
</select>
</form>
</aside>
<section>
<h3 class="debug-title">Debug Settings</h3>
<div class="config-section">
<label class="input-label">Proxy Configuration</label>
<label for="proxyUrl" class="input-label">Proxy URL</label>
<input id="proxyUrl" class="ui-input config-input" type="text" placeholder="" spellcheck="false" autocapitalize="off">
<div class="config-text">
<span id="proxyDefaultText"></span>
</div>
</div>
<div class="config-section">
<label class="input-label">Frame Rate Configuration</label>
<label for="deviceFrameRate" class="input-label">Device Frame Rate</label>
<select id="deviceFrameRate" class="ui-input config-input">
<option value="72">72 FPS</option>
<option value="90" selected>90 FPS</option>
<option value="120">120 FPS</option>
</select>
<div class="config-text">
Select the target device frame rate for the XR session
</div>
</div>
<div class="config-section">
<label class="input-label">Maximum Streaming Bitrate</label>
<label for="maxStreamingBitrateMbps" class="input-label">Megabits per second</label>
<select id="maxStreamingBitrateMbps" class="ui-input config-input">
<option value="80">80 Mbps</option>
<option value="100">100 Mbps</option>
<option value="120">120 Mbps</option>
<option value="150" selected>150 Mbps</option>
<option value="180">180 Mbps</option>
<option value="200">200 Mbps</option>
</select>
<div class="config-text">
Select the maximum streaming bitrate (in Megabits per second) for the XR session
</div>
</div>
<div class="config-section">
<label class="input-label">Stream Resolution Configuration</label>
<label for="perEyeWidth" class="input-label">Per-Eye Width (default: 2048)</label>
<input id="perEyeWidth" class="ui-input config-input" type="number"
placeholder="Per-eye width in pixels" spellcheck="false" value="2048">
<label for="perEyeHeight" class="input-label">Per-Eye Height (default: 1792)</label>
<input id="perEyeHeight" class="ui-input config-input" type="number"
placeholder="Per-eye height in pixels" spellcheck="false" value="1792">
<div class="config-text">
Configure the per-eye resolution. Width and height must be multiples of 16.
</div>
</div>
<div class="config-section">
<label for="referenceSpace" class="input-label">Reference Space</label>
<select id="referenceSpace" name="referenceSpace" class="ui-input" aria-label="Select reference space">
<option value="auto">Auto (local-floor preferred)</option>
<option value="local-floor">local-floor</option>
<option value="local" selected>local</option>
<option value="unbounded">unbounded</option>
<option value="viewer">viewer</option>
</select>
<div class="config-text">
Select the preferred reference space for XR tracking. "Auto" uses fallback logic (local-floor → local → viewer). Other options will attempt to use the specified space only.
</div>
<label class="input-label">XR Reference Space Offset</label>
<label for="xrOffsetX" class="input-label">X Offset (centimeters): Horizontal</label>
<input id="xrOffsetX" class="ui-input config-input" type="number" placeholder="X offset in centimeters"
spellcheck="false" value="0" step="1" min="-1000" max="1000">
<label for="xrOffsetY" class="input-label">Y Offset (centimeters): Vertical</label>
<input id="xrOffsetY" class="ui-input config-input" type="number" placeholder="Y offset in centimeters"
spellcheck="false" value="-155" step="1" min="-1000" max="1000">
<label for="xrOffsetZ" class="input-label">Z Offset (centimeters): Depth (forward/backward)</label>
<input id="xrOffsetZ" class="ui-input config-input" type="number" placeholder="Z offset in centimeters"
spellcheck="false" value="10" step="1" min="-1000" max="1000">
<div class="config-text">
Configure the XR reference space offset in centimeters. These values will be applied to the reference space transform to adjust the origin position of the XR experience.
</div>
</div>
</section>
</main>
</div>
<div id="3d-ui"></div>
</body>
</html>

27
deps/cloudxr/react/src/index.tsx vendored Normal file
View File

@@ -0,0 +1,27 @@
import React from 'react';
import ReactDOM from 'react-dom/client';
import App from './App';
// Start the React app immediately in the 3d-ui container
function startApp() {
const reactContainer = document.getElementById('3d-ui');
if (reactContainer) {
const root = ReactDOM.createRoot(reactContainer);
root.render(
<React.StrictMode>
<App />
</React.StrictMode>
);
} else {
console.error('3d-ui container not found');
}
}
// Initialize the app when DOM is ready
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', startApp);
} else {
startApp();
}

27
deps/cloudxr/react/tsconfig.json vendored Normal file
View File

@@ -0,0 +1,27 @@
{
"compilerOptions": {
"target": "ES2020",
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "CommonJS",
"moduleResolution": "node",
"allowSyntheticDefaultImports": true,
"esModuleInterop": true,
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"forceConsistentCasingInFileNames": true,
"noEmit": false,
"outDir": "./dist",
"incremental": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noFallthroughCasesInSwitch": true,
"jsx": "react-jsx",
"baseUrl": ".",
"paths": {
"@helpers/*": ["../helpers/*"]
}
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}

View File

@@ -0,0 +1,109 @@
const fs = require('fs');
const path = require('path');
const https = require('https');
class DownloadAssetsPlugin {
constructor(assets) {
this.assets = assets;
this.hasRun = false;
this.createdDirs = new Set();
}
safeUnlink(filePath) {
try {
fs.unlinkSync(filePath);
} catch (err) {
// Ignore cleanup errors
}
}
apply(compiler) {
compiler.hooks.beforeCompile.tapAsync('DownloadAssetsPlugin', (params, callback) => {
// Only run once per webpack process
if (this.hasRun) {
callback();
return;
}
console.log('📦 Checking and downloading required assets...');
const downloadPromises = this.assets.map(asset => this.downloadFile(asset));
Promise.allSettled(downloadPromises)
.then((results) => {
const failed = results.filter(r => r.status === 'rejected');
if (failed.length > 0) {
console.warn(`⚠️ ${failed.length} asset(s) failed to download, continuing anyway...`);
}
console.log('✅ Asset check complete!');
this.hasRun = true;
callback();
});
});
}
downloadFile({ url, output }) {
return new Promise((resolve, reject) => {
// Ensure directory exists (only once per unique path)
if (!this.createdDirs.has(output)) {
fs.mkdirSync(output, { recursive: true });
this.createdDirs.add(output);
}
const filename = path.basename(url);
const filePath = path.join(output, filename);
// Skip if file already exists
if (fs.existsSync(filePath)) {
resolve();
return;
}
console.log(` Downloading ${filename}...`);
const file = fs.createWriteStream(filePath);
const downloadFromUrl = (downloadUrl) => {
https.get(downloadUrl, (response) => {
// Handle redirects
if (response.statusCode === 302 || response.statusCode === 301) {
file.close();
this.safeUnlink(filePath);
downloadFromUrl(response.headers.location);
return;
}
if (response.statusCode !== 200) {
file.close();
this.safeUnlink(filePath);
reject(new Error(`Failed to download ${filename}: HTTP ${response.statusCode}`));
return;
}
response.pipe(file);
file.on('finish', () => {
file.close();
console.log(` ✓ Downloaded ${filename}`);
resolve();
});
file.on('error', (err) => {
this.safeUnlink(filePath);
reject(err);
});
}).on('error', (err) => {
if (fs.existsSync(filePath)) {
this.safeUnlink(filePath);
}
reject(err);
});
};
downloadFromUrl(url);
});
}
}
module.exports = DownloadAssetsPlugin;

168
deps/cloudxr/react/webpack.common.js vendored Normal file
View File

@@ -0,0 +1,168 @@
const path = require('path');
const HtmlWebpackPlugin = require('html-webpack-plugin');
const CopyWebpackPlugin = require('copy-webpack-plugin');
const webpack = require('webpack');
const DownloadAssetsPlugin = require('./webpack-plugins/DownloadAssetsPlugin');
const WEBXR_ASSETS_VERSION = '1.0.19';
module.exports = {
entry: './src/index.tsx',
// Module rules define how different file types are processed
module: {
rules: [
{
test: /\.tsx?$/,
use: {
loader: 'ts-loader',
options: {
// Only transpile, don't type-check (faster builds)
transpileOnly: true,
},
},
exclude: /node_modules/,
},
{
test: /\.css$/,
use: ['style-loader', 'css-loader'],
},
],
},
// Resolve configuration for module resolution
resolve: {
extensions: ['.tsx', '.ts', '.js'],
alias: {
// @helpers can be used instead of relative paths to the helpers directory
'@helpers': path.resolve(__dirname, '../helpers')
}
},
// Output configuration for bundled files
output: {
filename: 'bundle.js',
path: path.resolve(__dirname, './build'),
},
// Webpack plugins that extend webpack's functionality
plugins: [
// Generates HTML file and automatically injects bundled JavaScript
new HtmlWebpackPlugin({
template: './src/index.html',
favicon: './favicon.ico'
}),
// Inject environment variables
new webpack.DefinePlugin({
'process.env.WEBXR_ASSETS_VERSION': JSON.stringify(WEBXR_ASSETS_VERSION),
}),
// Download external assets during build
new DownloadAssetsPlugin([
// HDRI environment map
{
url: 'https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/potsdamer_platz_1k.hdr',
output: path.resolve(__dirname, 'public/assets/hdri')
},
// WebXR controller profiles
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/profilesList.json`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles`)
},
// Generic hand profile
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-hand/profile.json`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-hand`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-hand/left.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-hand`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-hand/right.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-hand`)
},
// Generic trigger profile
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-trigger/profile.json`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-trigger`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-trigger/left.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-trigger`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-trigger/right.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/generic-trigger`)
},
// Oculus Touch v2
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v2/profile.json`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v2`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v2/left.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v2`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v2/right.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v2`)
},
// Oculus Touch v3
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v3/profile.json`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v3`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v3/left.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v3`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v3/right.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/oculus-touch-v3`)
},
// Meta Quest Touch Plus
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/meta-quest-touch-plus/profile.json`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/meta-quest-touch-plus`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/meta-quest-touch-plus/left.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/meta-quest-touch-plus`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/meta-quest-touch-plus/right.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/meta-quest-touch-plus`)
},
// Pico 4 Ultra
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/pico-4u/profile.json`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/pico-4u`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/pico-4u/left.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/pico-4u`)
},
{
url: `https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/pico-4u/right.glb`,
output: path.resolve(__dirname, `public/npm/@webxr-input-profiles/assets@${WEBXR_ASSETS_VERSION}/dist/profiles/pico-4u`)
},
]),
// Copies static assets from public directory to build output
new CopyWebpackPlugin({
patterns: [
{
from: 'public',
to: '.',
globOptions: {
// Don't copy index.html since HtmlWebpackPlugin handles it
ignore: ['**/index.html'],
},
},
{ from: './favicon.ico', to: 'favicon.ico' },
],
}),
],
};

46
deps/cloudxr/react/webpack.dev.js vendored Normal file
View File

@@ -0,0 +1,46 @@
const { merge } = require('webpack-merge')
const common = require('./webpack.common.js')
const path = require('path')
// Check if HTTPS mode is enabled via environment variable
const useHttps = process.env.HTTPS === 'true'
module.exports = merge(common, {
mode: 'development',
devtool: 'eval-source-map',
devServer: {
allowedHosts: 'all',
hot: true,
open: false,
// Enable HTTPS with self-signed certificate when HTTPS=true
...(useHttps && { server: 'https' }),
static: [
{
directory: path.join(__dirname, './build'),
},
{
directory: path.join(__dirname, './public'),
publicPath: '/',
},
],
watchFiles: {
paths: ['src/**/*', '../../build/**/*'],
options: {
usePolling: false,
ignored: /node_modules/,
},
},
client: {
progress: true,
overlay: {
errors: true,
warnings: false,
},
},
devMiddleware: {
writeToDisk: true,
},
compress: true,
port: 8080,
},
})

6
deps/cloudxr/react/webpack.prod.js vendored Normal file
View File

@@ -0,0 +1,6 @@
const { merge } = require('webpack-merge');
const common = require('./webpack.common.js');
module.exports = merge(common, {
mode: 'production'
});