Add CloudXR VR streaming support for PICO 4 Ultra (Early Access)
Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.
Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide
Usage:
# Start CloudXR Runtime + Isaac Lab
cd ~/IsaacLab && ./docker/container.py start \
--files docker-compose.cloudxr-runtime.patch.yaml \
--env-file .env.cloudxr-runtime
# Run teleop with CloudXR
~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
--task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr
# Serve web client
cd deps/cloudxr/isaac && npm run dev-server:https
This commit is contained in:
71
deps/cloudxr/README.md
vendored
Normal file
71
deps/cloudxr/README.md
vendored
Normal file
@@ -0,0 +1,71 @@
|
||||
# NVIDIA CloudXR.js SDK - Getting Started
|
||||
|
||||
> **Note:** For detailed documentation, see the `docs/` folder (open `docs/index.html` in browser).
|
||||
|
||||
[The NVIDIA CloudXR.js SDK](#) enables developers to build web clients that stream high-quality spatial content from CloudXR servers with powerful GPUs to web browsers on XR devices. It consists of:
|
||||
- **CloudXR.js SDK** - a JavaScript client library
|
||||
- **WebGL-based simple sample client** that uses core Web APIs (WebGL, WebXR)
|
||||
- **React-based sample client** that uses the R3F (React-Three-Fiber) framework stack
|
||||
|
||||
<!-- #region firstRunGuide -->
|
||||
|
||||
It is *strongly recommended* that you work through this guide if you have never run CloudXR.js before.
|
||||
|
||||
## The Pieces of a CloudXR.js Deployment
|
||||
|
||||
Even for development, you'll need all the pieces of a CloudXR.js deployment in order to build and test a client. These are:
|
||||
|
||||
- a CloudXR Server
|
||||
- with a _top-end_ NVIDIA GPU or 2 (e.g. dual RTX 6000 Ada)
|
||||
- which will run
|
||||
- the CloudXR Runtime
|
||||
- an OpenXR application (the thing you want to render on the server but see on the client)
|
||||
- a CloudXR.js development workstation
|
||||
- with Node.js and `npm`
|
||||
- which will run
|
||||
- the CloudXR.js sample client build
|
||||
- a Node-based development web server
|
||||
- a CloudXR.js client
|
||||
- which is one of:
|
||||
- a Meta Quest 3 with its built-in Browser app
|
||||
- a desktop browser: [Google Chrome](https://www.google.com/chrome) or Edge (IWER automatically loads for WebXR emulation)
|
||||
- which will run...
|
||||
- the CloudXR.js sample client *served from the development web server*.
|
||||
|
||||
We *recommend* that for your first experience, all above run on *the same computer* to eliminate networking related issues.
|
||||
|
||||
## High Level Workflow
|
||||
|
||||
You need both a working client and a working server in order to test. Typically we follow a startup flow where server starts before the client:
|
||||
|
||||
1. CloudXR Runtime
|
||||
2. Server XR application
|
||||
3. Sample client build + web server
|
||||
4. Test from the same computer
|
||||
5. Test from Quest 3 or a different computer
|
||||
|
||||
<!-- #endregion firstRunGuide -->
|
||||
|
||||
Please refer to [the CloudXR.js SDK documentations](#sdk-documentation) for details.
|
||||
|
||||
## SDK Documentation
|
||||
|
||||
The `docs/` folder contains comprehensive documentation including:
|
||||
- Getting Started guides with example server applications
|
||||
- API reference for building custom clients
|
||||
- Common issues and solutions
|
||||
|
||||
Open `docs/index.html` in your browser for organized navigation with table of contents.
|
||||
|
||||
## Support
|
||||
|
||||
For technical support and questions about this early access release:
|
||||
- Review the troubleshooting sections in the SDK documentation
|
||||
- Study the example implementations for integration patterns
|
||||
- Contact the NVIDIA CloudXR team for additional assistance
|
||||
|
||||
## License
|
||||
|
||||
> **Important**: This is an [evaluation license](LICENSE) for internal test and evaluation purposes only. For commercial use, please contact NVIDIA for appropriate licensing terms.
|
||||
|
||||
> **Note**: This is an early access release of CloudXR.js. Features and APIs may change in future releases.
|
||||
Reference in New Issue
Block a user