Add CloudXR VR streaming support for PICO 4 Ultra (Early Access)

Replaces manual H264/TCP stereo streaming with NVIDIA CloudXR for
higher-quality stereoscopic rendering and lower latency.

Changes:
- teleop_xr_agent.py: add --cloudxr flag (enables Isaac Sim XR mode,
  disables manual StreamingManager)
- deps/cloudxr/: NVIDIA CloudXR.js SDK (Early Access) with Isaac Lab
  teleop React web client
- deps/cloudxr/Dockerfile.wss.proxy: HAProxy WSS proxy for PICO 4 Ultra
  HTTPS mode (routes wss://48322 → ws://49100)
- deps/cloudxr/isaac/webpack.dev.js: disable file watching to avoid
  EMFILE errors with large node_modules
- deps/cloudxr/INSTALL.md: full setup guide

Usage:
  # Start CloudXR Runtime + Isaac Lab
  cd ~/IsaacLab && ./docker/container.py start \
      --files docker-compose.cloudxr-runtime.patch.yaml \
      --env-file .env.cloudxr-runtime

  # Run teleop with CloudXR
  ~/IsaacLab/isaaclab.sh -p teleop_xr_agent.py \
      --task Isaac-MindRobot-2i-DualArm-IK-Abs-v0 --cloudxr

  # Serve web client
  cd deps/cloudxr/isaac && npm run dev-server:https
This commit is contained in:
2026-03-26 14:29:03 +08:00
parent eef7ff838d
commit 623e05f250
133 changed files with 24869 additions and 2 deletions

View File

@@ -44,11 +44,18 @@ parser.add_argument("--stream-to", type=str, default=None, dest="stream_to")
parser.add_argument("--stream-port", type=int, default=12345, dest="stream_port")
parser.add_argument("--stream-bitrate", type=int, default=20_000_000, dest="stream_bitrate")
parser.add_argument("--debug-viewports", action="store_true", dest="debug_viewports", default=True)
parser.add_argument(
"--cloudxr", action="store_true", dest="cloudxr", default=False,
help="Enable CloudXR streaming mode (replaces --stream-to). "
"Requires CloudXR Runtime Docker container running on this workstation.",
)
AppLauncher.add_app_launcher_args(parser)
args_cli = parser.parse_args()
app_launcher_args = vars(args_cli)
app_launcher_args["xr"] = False
# Enable Isaac Sim XR extensions when using CloudXR; disable for normal streaming
app_launcher_args["xr"] = args_cli.cloudxr
app_launcher_args["enable_cameras"] = not args_cli.cloudxr # CloudXR renders its own stereo views
app_launcher = AppLauncher(app_launcher_args)
simulation_app = app_launcher.app
@@ -93,7 +100,8 @@ def main() -> None:
pos_sensitivity=pos_sens, rot_sensitivity=rot_sens,
base_speed=args.base_speed, base_turn=args.base_turn,
drive_speed=args.drive_speed, drive_turn=args.drive_turn,
stream_to=args.stream_to, stream_port=args.stream_port,
stream_to=None if args.cloudxr else args.stream_to,
stream_port=args.stream_port,
stream_bitrate=args.stream_bitrate,
debug_viewports=args.debug_viewports or bool(args.stream_to),
)