# Running Isaac Lab with the CloudXR Runtime

> Note: For this early access release, make sure you have a valid subscription to the NVIDIA CloudXR Early Access program on NVIDIA NGC.

## Download CloudXR Runtime Container from NVIDIA NGC

1. Follow NGC API Key doc to get your NGC API Key (`nvapi-xxxx`) for your NGC Organization.
   - The key should have granted permission to access `NGC Catalog`
   - Save the key to a safe place as it will only appear once
2. Setup credential for docker (use username `$oauthtoken` as literal)

       docker login nvcr.io
       Username: $oauthtoken
       Password: <your-ngc-api-key>

3. Try to pull docker image and it should succeed

       docker pull nvcr.io/nvidia/cloudxr-runtime-early-access:6.0.1-webrtc

## 🖥️ Option 1: Running Isaac Lab and CloudXR Runtime Containers Together

On your Isaac Lab workstation:

1. From the root of the Isaac Lab repository, update `./docker/.env.cloudxr-runtime` to the following version tag

       # NVIDIA CloudXR Runtime base image
       CLOUDXR_RUNTIME_BASE_IMAGE_ARG=nvcr.io/nvidia/cloudxr-runtime-early-access
       # NVIDIA CloudXR Runtime version to use
       CLOUDXR_RUNTIME_VERSION_ARG=6.0.1-webrtc

2. Also update `./docker/docker-compose.cloudxr-runtime.patch.yaml` to use host network instead:

       services:
         cloudxr-runtime:
           image: ${CLOUDXR_RUNTIME_BASE_IMAGE_ARG}:${CLOUDXR_RUNTIME_VERSION_ARG}
           network_mode: host
           #ports:
           # - "48010:48010/tcp" # signaling
           # - "47998:47998/udp" # media
           # - "47999:47999/udp" # media
           # - "48000:48000/udp" # media
           # - "48005:48005/udp" # media
           # - "48008:48008/udp" # media
           # - "48012:48012/udp" # media
           healthcheck:
           ...
       ...

3. Start the Isaac Lab and CloudXR Runtime containers using the Isaac Lab container.py script

       ./docker/container.py start \
           --files docker-compose.cloudxr-runtime.patch.yaml \
           --env-file .env.cloudxr-runtime

   If prompted, elect to activate X11 forwarding, which is necessary to see the Isaac Lab UI.

   > Note: The `container.py` script is a thin wrapper around Docker Compose. The additional `--files` and `--env-file` arguments augment the base Docker Compose configuration to additionally run the CloudXR Runtime. For more details on `container.py` and running Isaac Lab with Docker Compose, see the Docker Guide.

4. Enter the Isaac Lab base container with:

       ./docker/container.py enter base

   From within the Isaac Lab base container, you can run Isaac Lab scripts that use XR.

5. Run an example teleop task with:

       ./isaaclab.sh -p scripts/environments/teleoperation/teleop_se3_agent.py \
           --task Isaac-PickPlace-Locomanipulation-G1-Abs-v0 \
           --teleop_device handtracking \
           --enable_pinocchio \
           --info

   > Note: You could also choose a different environment like `Isaac-PickPlace-GR1T2-Abs-v0`.

6. You'll want to leave the container running for the next steps. But once you are finished, you can stop the containers with:

       ./docker/container.py stop \
           --files docker-compose.cloudxr-runtime.patch.yaml \
           --env-file .env.cloudxr-runtime

## 🖥️ Option 2: Running Isaac Lab as Local Process and CloudXR Runtime Container

You can also run Isaac Lab as a local process that connects to the CloudXR Runtime Docker container. This method requires you to manually specify a shared directory for communication between the Isaac Lab instance and the CloudXR Runtime.

On your Isaac Lab workstation:

1. From the root of the Isaac Lab repository, create a local folder for temporary cache files:

    mkdir -p $(pwd)/openxr

2. Initiate the CloudXR Runtime Docker container, ensuring the previously created directory is mounted to the `/openxr` directory within the container for Isaac Lab visibility:

    docker run -dit --rm --name cloudxr-runtime \
        --user $(id -u):$(id -g) \
        --gpus=all \
        -e "ACCEPT_EULA=Y" \
        --mount type=bind,src=$(pwd)/openxr,dst=/openxr \
        --network host \
        nvcr.io/nvidia/cloudxr-runtime-early-access:6.0.1-webrtc

> If you choose a particular GPU instead of all, you need to make sure Isaac Lab also runs on that GPU. See Isaac Lab Teleop Troubleshooting how to do so.

3. In a new terminal where you intend to run Isaac Lab, export the following environment variables, which reference the directory created above:

    export XDG_RUNTIME_DIR=$(pwd)/openxr/run
    export XR_RUNTIME_JSON=$(pwd)/openxr/share/openxr/1/openxr_cloudxr.json

4. Run the example teleop task with:

    ./isaaclab.sh -p scripts/environments/teleoperation/teleop_se3_agent.py \
        --task Isaac-PickPlace-GR1T2-Abs-v0 \
        --teleop_device handtracking \
        --enable_pinocchio \
        --info

With Isaac Lab and the CloudXR Runtime running:

1. In the Isaac Lab UI: locate the Panel named AR.

2. Click Start AR

The Viewport should show two eyes being rendered, and you should see the status "AR profile is active".

> Above instructions are modified from these instructions, in particular, to provide the CloudXR Runtime with environment variables needed for streaming to the Meta Quest 3 Browser.

## Next

Teleoperating on Meta Quest 3

## Previous

Running Isaac Lab Teleoperation on Meta Quest 3