yt lee
623e05f250
Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.
Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide
Usage:
# Start CloudXR Runtime + Isaac Lab
cd ~/IsaacLab && ./docker/container.py start \
--files docker-compose.cloudxr-runtime.patch.yaml \
--env-file .env.cloudxr-runtime
# Run teleop with CloudXR
~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
--task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr
# Serve web client
cd deps/cloudxr/isaac && npm run dev-server:https
NVIDIA CloudXR.js SDK - Getting Started
Note: For detailed documentation, see the
docs/folder (opendocs/index.htmlin browser).
The NVIDIA CloudXR.js SDK enables developers to build web clients that stream high-quality spatial content from CloudXR servers with powerful GPUs to web browsers on XR devices. It consists of:
- CloudXR.js SDK - a JavaScript client library
- WebGL-based simple sample client that uses core Web APIs (WebGL, WebXR)
- React-based sample client that uses the R3F (React-Three-Fiber) framework stack
It is strongly recommended that you work through this guide if you have never run CloudXR.js before.
The Pieces of a CloudXR.js Deployment
Even for development, you'll need all the pieces of a CloudXR.js deployment in order to build and test a client. These are:
- a CloudXR Server
- with a top-end NVIDIA GPU or 2 (e.g. dual RTX 6000 Ada)
- which will run
- the CloudXR Runtime
- an OpenXR application (the thing you want to render on the server but see on the client)
- a CloudXR.js development workstation
- with Node.js and
npm - which will run
- the CloudXR.js sample client build
- a Node-based development web server
- with Node.js and
- a CloudXR.js client
- which is one of:
- a Meta Quest 3 with its built-in Browser app
- a desktop browser: Google Chrome or Edge (IWER automatically loads for WebXR emulation)
- which will run...
- the CloudXR.js sample client served from the development web server.
- which is one of:
We recommend that for your first experience, all above run on the same computer to eliminate networking related issues.
High Level Workflow
You need both a working client and a working server in order to test. Typically we follow a startup flow where server starts before the client:
- CloudXR Runtime
- Server XR application
- Sample client build + web server
- Test from the same computer
- Test from Quest 3 or a different computer
Please refer to the CloudXR.js SDK documentations for details.
SDK Documentation
The docs/ folder contains comprehensive documentation including:
- Getting Started guides with example server applications
- API reference for building custom clients
- Common issues and solutions
Open docs/index.html in your browser for organized navigation with table of contents.
Support
For technical support and questions about this early access release:
- Review the troubleshooting sections in the SDK documentation
- Study the example implementations for integration patterns
- Contact the NVIDIA CloudXR team for additional assistance
License
Important
: This is an evaluation license for internal test and evaluation purposes only. For commercial use, please contact NVIDIA for appropriate licensing terms.
Note
: This is an early access release of CloudXR.js. Features and APIs may change in future releases.