# Teleoperating on Meta Quest 3

## Prerequisites

Before building and hosting the web client, ensure you have the following installed:

- Node.js (v20.19.0 or later) and npm
  - Download and installation instructions: https://nodejs.org/en/download
  - Verify installation: `node --version` and `npm --version`

## 🔨 Build Client and Host Web Server

Once you have Isaac Lab and the CloudXR Runtime running, you can build and host the web server for the CloudXR.js client application.

### 1. Navigate to Example Directory

    cd isaac

### 2. Install Dependencies

    # For this early access release, install the SDK from the provided tarball
    # This step will not be required when the SDK is publicly accessible
    npm install /path/to/nvidia-cloudxr-6.0.0-beta.tgz

    # Install remaining dependencies
    npm install

### 3. Choose Hosting Mode

Select the appropriate server mode based on your deployment requirements:

For local development (HTTP):

    npm run dev-server

- Local: `http://localhost:8080/`
- Network: `http://<server-ip>:8080/`
- Supports both `ws://` (direct) and `wss://` (proxied) runtime connections

For local development or production (HTTPS):

    npm run dev-server:https

- Local: `https://localhost:8080/`
- Network: `https://<server-ip>:8080/`
- Generates self-signed certificates for development, use custom certificates for production
- See Client Setup - Trust SSL Certificates for trusting certs in your device

> Note: When using HTTPS mode, you must configure a WebSocket proxy. Browsers enforce mixed content policies that block insecure `ws://` connections from HTTPS pages. See Networking Setup - WebSocket Proxy for proxy configuration example.

## 🖥️ Test from the Same Computer (Optional but Recommended)

For local development, our web app automatically loads IWER (Immersive Web Emulator Runtime) to emulate a Meta Quest 3 headset when no physical XR device is detected.

1. Make sure you have:

   - Chromium browser or Google Chrome
   - IWER will automatically load when you start the application
     - If you have installed the Immersive Web Emulator Chrome extension, please disable it as it will conflict with IWER

2. Ensure your CloudXR Runtime and OpenXR application are running

3. Navigate to the web application based on your chosen hosting mode:

   - HTTP mode: `http://localhost:8080/`
   - HTTPS mode: `https://localhost:8080/`
     - Accept the cert (see Client Setup - Trust Web Cert)

4. Make sure you see the following messaging, indicating IWER is loaded

5. (HTTPS mode only) Fill in the `Proxy URL` if you have any, or click to accept the cert

6. Click `CONNECT`

If successful, you should see the green `CONNECT` button change to `CONNECT (STREAMING)`, and the streamed content from your server application will appear in your browser!

You could use the develop UI from IWER to emulate device position and input, for example:

## 🥽 Test from Meta Quest 3

Once you've validated the setup works locally, you can test with an actual Meta Quest 3 headset.

### Prerequisites

1. Configure Browser: Follow the Client Setup Guide - Meta Quest 3 Browser Configuration to:

   - Enable WebXR for HTTP origins (if using `npm run dev-server`)
   - Trust SSL certificates (if using `npm run dev-server:https`)

2. Configure Network: Ensure required ports are open:

   - Web server: Port 8080 (TCP) or your configured port
   - CloudXR Runtime: Port 49100 (TCP) and 47998-48012 (UDP)
   - WebSocket proxy (if using HTTPS): Port 48322 (TCP) or your configured port
   - See Networking Setup - Firewall Configuration for details

### Connection Configuration

1. Navigate to the web application on your Meta Quest 3 Browser:

   - HTTP mode: `http://<server-ip>:8080/`
   - HTTPS mode: `https://<server-ip>:8080/`

2. Configure runtime connection based on your hosting mode:

   If using HTTP client (`npm run dev-server`):

   - Direct: `ws://<server-ip>:49100`
   - Proxied: `wss://<server-ip>:48322`

   If using HTTPS client (`npm run dev-server:https`):

   - Proxied: `wss://<server-ip>:48322` (required)

3. Grant permissions:

   - Click `CONNECT`
   - Allow WebXR permissions when prompted

### Application-Specific Configuration

Before connecting, verify your Isaac Lab server is ready:

- CloudXR Runtime is running
- Isaac Lab is running with the desired task
- Isaac Lab is using `System OpenXR Runtime`
- Isaac Lab is in AR mode (`Start AR` button pressed)

Configure the following application settings in the client web page's Debug Settings section:

Setting | Recommended Value | Notes
--- | --- | ---
Preferred Reference Space | `local` | Required for proper tracking
X Offset (cm) | `0` | Horizontal positioning
Y Offset (cm) | `-155` | Vertical positioning
Z Offset (cm) | `10` | Depth positioning
Teleoperation Countdown Duration (s) | User preference | Time before recording starts (optional)

> Note: Offset values are task-dependent. Adjust based on robot height and workspace as needed.

### How to use

Once streaming starts, you will see a window to your right with teleoperation controls:

- `Countdown` - you can use the `-` (minus) and `+` (plus) buttons to adjust the countdown timer for starting teleoperation.
- `Play` - this will start the countdown. Once countdown is finished, the teleoperation session recording starts.
- `Reset` - this will both stop and reset the teleoperation session. After pressing this you can press `Play` to continue teleoperation (following the countdown timer).
- `Disconnect` - this will exit the whole streaming and WebXR session

To start teleoperating, click `Play`. It will start counting down in seconds as configured.

During the countdown, position your hands at the right place and wait for recording to start.

Whenever you complete a task, it will automatically start a new recording and generate the recording file.

To stop teleoperating and reset the position of robot arm, click the `Reset` button.

## Next

Troubleshooting Isaac Lab Teleoperation using CloudXR.js

## Previous

Running Isaac Lab with the CloudXR Runtime