7 Commits

Author SHA1 Message Date
Tangger
6314603676 fix: rename migerate -> migrate (typo fix) 2026-04-03 15:24:41 +08:00
Tangger
b178aafe40 feat: migrate omni.isaac.* imports to isaacsim.* via compat layer
- Add core/compat.py: compatibility module with try/except imports
  supporting both IS 4.x (omni.isaac.*) and IS 5.x+ (isaacsim.*)
- Migrate 152 imports across 47 files from direct omni.isaac.* to core.compat
- Handle class renames: RigidPrim→SingleRigidPrim, GeometryPrim→SingleGeometryPrim,
  XFormPrim→SingleXFormPrim, Articulation→SingleArticulation (aliased for compatibility)
- Add migerate/migrate_imports.py: automated migration script for future use
- Leave debug_draw and env_loader try/except imports as-is

This eliminates ~100 deprecation warnings from our code on IS 5.0,
and future-proofs for IS 6.x when old APIs may be removed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 15:09:50 +08:00
Tangger
6032b12c59 refactor: simplify camera init, document aperture warning as cosmetic
Camera.__init__() internally calls UsdGeom.Camera.Define() which
resets aperture to defaults, making it impossible to pre-set values.
The aperture mismatch warnings are cosmetic — IS auto-corrects them
and our values are applied correctly after init.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:41:00 +08:00
Tangger
1105a49e8f fix: pre-set camera aperture on USD prim before Camera init
Camera.__init__() checks aperture consistency against resolution
before we can call set_horizontal_aperture(). Pre-set the aperture
values directly on the USD prim via UsdGeom.Camera API to eliminate
the mismatch warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:07:16 +08:00
Tangger
8d2fec09e0 fix: resolve camera aperture warnings and collision error
- Fix camera aperture mismatch: set aperture/focal_length before initialize()
- Replace deprecated set_projection_type with set_lens_distortion_model
- Fix lifting_link collision: meshSimplification -> convexHull in robot USD
- Add runtime_warnings.md documenting all warnings analysis and fixes
- Remove temporary debug script 1.py

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:47:15 +08:00
Tangger
6b78ba0d6f docs: rewrite README with quick start, project structure, and migration refs
- Rewrite README with clear prerequisites, quick start guide, and project structure
- Add references to install.md and migerate/migerate.md
- Add pipeline config table and configuration examples
- Preserve original license (CC BY-NC-SA 4.0) and paper citations
- Update remote: matai as origin, merge 5.0.0 into master

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:17:43 +08:00
Tangger
03d9a5b909 fix: IS 4.5.0 -> 5.0.0 migration — USD metadata, DomeLight, scene reuse
- Fix USD metersPerUnit/upAxis for IS 5.0.0 (no longer auto-compensated)
- Batch fix all Aligned_obj.usd, table, and art USD files with backups
- Fix DomeLight rotation to Z-axis only (prevent tilted environment map)
- Fix scene reuse across episodes (arena_file caching, task clearing, prim guard)
- Add migration tools: scan_usd_metadata.py, fix_usd_metadata.py
- Add migration guide: migerate/migerate.md
- Add nvidia-curobo to .gitignore
- Fix sort_the_rubbish config: obj_0 -> obj_1 (obj_0 does not exist)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:10:39 +08:00
109 changed files with 1656 additions and 302 deletions

3
.gitignore vendored
View File

@@ -18,4 +18,5 @@ _isaac_sim_410
InterDataEngine-docs
debug.sh
debug.yaml
depre
depre
workflows/simbox/src/nvidia-curobo/

133
README.md
View File

@@ -1,6 +1,8 @@
<div align="center">
# InternDataEngine: Pioneering High-Fidelity Synthetic Data Generator for Robotic Manipulation
# InternDataEngine
**High-Fidelity Synthetic Data Generator for Robotic Manipulation**
</div>
@@ -15,30 +17,127 @@
</div>
## 💻 About
## About
<div align="center">
<img src="./docs/images/intern_data_engine.jpeg" alt="InternDataEngine Overview" width="80%">
</div>
InternDataEngine is a synthetic data generation engine for embodied AI that powers large-scale model training and iteration. Built on NVIDIA Isaac Sim, it unifies high-fidelity physical interaction from InternData-A1, semantic task and scene generation from InternData-M1, and high-throughput scheduling from the Nimbus framework to deliver realistic, task-aligned, and massively scalable robotic manipulation data.
InternDataEngine is a synthetic data generation engine for embodied AI, built on NVIDIA Isaac Sim. It unifies high-fidelity physical interaction (InternData-A1), semantic task and scene generation (InternData-M1), and high-throughput scheduling (Nimbus) to deliver realistic, task-aligned, and scalable robotic manipulation data.
- **More realistic physical interaction**: Unified simulation of rigid, articulated, deformable, and fluid objects across single-arm, dual-arm, and humanoid robots, enabling long-horizon, skill-composed manipulation that better supports sim-to-real transfer.
- **More diverse data generation**: By leveraging the internal state of the simulation engine to extract high-quality ground truth, coupled with multi-dimensional domain randomization (e.g., layout, texture, structure, and lighting), the data distribution is significantly expanded. This approach produces precise and diverse operational data, while simultaneously exporting rich multimodal annotations such as bounding boxes, segmentation masks, and keypoints.
- **More efficient large-scale production**: Nimbus-powered asynchronous pipelines that decouple planning, rendering, and storage, achieving 23× end-to-end throughput, cluster-level load balancing and fault tolerance for billion-scale data generation.
**Key capabilities:**
## 🔥 Latest News
- **Realistic physical interaction** -- Unified simulation of rigid, articulated, deformable, and fluid objects across single-arm, dual-arm, and humanoid robots. Supports long-horizon, skill-composed manipulation for sim-to-real transfer.
- **Diverse data generation** -- Multi-dimensional domain randomization (layout, texture, structure, lighting) with rich multimodal annotations (bounding boxes, segmentation masks, keypoints).
- **Efficient large-scale production** -- Nimbus-powered asynchronous pipelines that decouple planning, rendering, and storage, achieving 2-3x end-to-end throughput with cluster-level load balancing and fault tolerance.
- **[2026/03]** We release the InternDataEngine codebase v1.0, which includes the core modules: InternData-A1 and Nimbus.
## Prerequisites
## 🚀 Quickstart
| Dependency | Version |
|------------|---------|
| NVIDIA Isaac Sim | 5.0.0 (Kit 107.x) |
| CUDA Toolkit | >= 12.8 |
| Python | 3.10 |
| GPU | NVIDIA RTX (tested on RTX PRO 6000 Blackwell) |
Please refer to the [Installation](https://internrobotics.github.io/InternDataEngine-Docs/guides/installation.html) and [Usage](https://internrobotics.github.io/InternDataEngine-Docs/guides/quickstart.html) to start the installation and run your first synthetic data generation task.
> For detailed environment setup (conda, CUDA, PyTorch, curobo), see [install.md](install.md).
>
> If migrating from Isaac Sim 4.5.0, see [migrate/migrate.md](migrate/migrate.md) for known issues and fixes.
For more details, please check [Documentation](https://internrobotics.github.io/InternDataEngine-Docs/).
## Quick Start
### 1. Install
```bash
# Create conda environment
conda create -n banana500 python=3.10
conda activate banana500
# Install CUDA 12.8 and set up Isaac Sim 5.0.0
conda install -y cuda-toolkit=12.8
source ~/isaacsim500/setup_conda_env.sh
export CUDA_HOME="$CONDA_PREFIX"
# Install PyTorch (CUDA 12.8)
pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 --index-url https://download.pytorch.org/whl/cu128
# Install project dependencies
pip install -r requirements.txt
# Install curobo (motion planning)
cd workflows/simbox/curobo
export TORCH_CUDA_ARCH_LIST="12.0+PTX" # Set to your GPU's compute capability
pip install -e .[isaacsim] --no-build-isolation
cd ../../..
```
See [install.md](install.md) for the full step-by-step guide including troubleshooting.
### 2. Run Data Generation
```bash
# Full pipeline: plan trajectories + render + save
python launcher.py --config configs/simbox/de_plan_and_render_template.yaml
```
Output is saved to `output/simbox_plan_and_render/` including:
- `demo.mp4` -- rendered video from robot cameras
- LMDB data files for model training
### 3. Available Pipeline Configs
| Config | Description |
|--------|-------------|
| `de_plan_and_render_template.yaml` | Full pipeline: plan + render + save |
| `de_plan_template.yaml` | Plan trajectories only (no rendering) |
| `de_render_template.yaml` | Render from existing plans |
| `de_plan_with_render_template.yaml` | Plan with live rendering preview |
| `de_pipe_template.yaml` | Pipelined mode for throughput |
### 4. Configuration
The main config file (`configs/simbox/de_plan_and_render_template.yaml`) controls:
```yaml
simulator:
headless: True # Set False for GUI debugging
renderer: "RayTracedLighting" # Or "PathTracing" for higher quality
physics_dt: 1/30
rendering_dt: 1/30
```
Task configs are in `workflows/simbox/core/configs/tasks/`. The example task (`sort_the_rubbish`) demonstrates a dual-arm pick-and-place scenario.
## Project Structure
```
InternDataEngine/
configs/simbox/ # Pipeline configuration files
launcher.py # Main entry point
nimbus_extension/ # Nimbus framework components
workflows/simbox/
core/
configs/ # Task, robot, arena, camera configs
controllers/ # Motion planning (curobo integration)
skills/ # Manipulation skills (pick, place, etc.)
tasks/ # Task definitions
example_assets/ # Example USD assets (robots, objects, tables)
curobo/ # GPU-accelerated motion planning library
migrate/ # Migration tools and documentation
output/ # Generated data output
```
## Documentation
- [Installation Guide](install.md) -- Environment setup and dependency installation
- [Migration Guide](migrate/migrate.md) -- Isaac Sim 4.5.0 to 5.0.0 migration notes and tools
- [Online Documentation](https://internrobotics.github.io/InternDataEngine-Docs/) -- Full API docs, tutorials, and advanced usage
## License and Citation
All the code within this repo are under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). Please consider citing our papers if it helps your research.
This project is based on [InternDataEngine](https://github.com/InternRobotics/InternDataEngine) by InternRobotics, licensed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).
If this project helps your research, please cite the following papers:
```BibTeX
@article{tian2025interndata,
@@ -62,13 +161,3 @@ All the code within this repo are under [CC BY-NC-SA 4.0](https://creativecommon
year={2025}
}
```
<!--
```BibTeX
@misc{interndataengine2026,
title={InternDataEngine: A Synthetic Data Generation Engine for Robotic Learning},
author={InternDataEngine Contributors},
year={2026},
}
}
``` -->

View File

@@ -10,7 +10,7 @@ load_stage:
rendering_dt: 1/30 # Render update rate
stage_units_in_meters: 1.0 # Stage unit scale
headless: True # Headless mode (no GUI); set false for visual debugging
renderer: "PathTracing" # PathTracing: higher quality, less noise on Blackwell
renderer: "RayTracedLighting" # PathTracing: higher quality, less noise on Blackwell 改回快速渲染器就改成 "RayTracedLighting"
anti_aliasing: 0 # Anti-aliasing level
layout_random_generator: # Scene randomization
type: env_randomizer

33
install.md Normal file
View File

@@ -0,0 +1,33 @@
conda create -n banana450 python=3.10
conda activate banana450
# 看看12.8安装了么?
conda list | grep cuda
# 安装12.8
conda install -y cuda-toolkit=12.8
# 其他的cuda版本在conda中的需要卸载
conda list | grep cuda
source ~/isaacsim450/setup_conda_env.sh
# 导入conda环境的12.8cuda环境
export CUDA_HOME="$CONDA_PREFIX"
export PATH="$CONDA_PREFIX/bin:$PATH"
# 看一下应该是12.8
nvcc -V
# 换掉torch版本换成cu128编译的版本
pip list | grep torch
pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 --index-url https://download.pytorch.org/whl/cu128
pip list | grep torch
pip install -r requirements.txt
pip list | grep cuda
# 卸载掉cu11的那些包
pip uninstall nvidia-cuda-cupti-cu11 nvidia-cuda-nvrtc-cu11 nvidia-cuda-runtime-cu11
# 安装curobo
cd workflows/simbox/curobo
# 12.0是显卡的算力 要自己去查 不要信AI的 PRO6000是12.0
export TORCH_CUDA_ARCH_LIST="12.0+PTX"
pip install -e .[isaacsim] --no-build-isolation
# 降级numpy
pip install numpy==1.26.0
pip install opencv-python==4.11.0.86
python launcher.py --config configs/simbox/de_plan_and_render_template.yaml

159
migrate/fix_usd_metadata.py Normal file
View File

@@ -0,0 +1,159 @@
"""
Batch fix USD metadata for IS 5.0.0 compatibility.
IS 4.5.0 auto-added 'Rotate:unitsResolve' and 'Scale:unitsResolve' xformOps
to compensate for metersPerUnit/upAxis differences. IS 5.0.0 no longer does this,
causing RigidObject physics to break (objects fly away).
Fix: Change metersPerUnit to 1.0 and upAxis to Z in USD metadata.
Vertex data stays unchanged (already in correct scale for the scene).
Usage:
# Scan only (dry run):
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets --dry-run
# Fix all:
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets
# Fix specific pattern:
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets --pattern "Aligned_obj.usd"
# Skip specific directories (e.g. robot models, curobo assets):
python migrate/fix_usd_metadata.py --root . --skip curobo,robot
# Restore from backups:
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets --restore
"""
import argparse
import glob
import os
import shutil
import sys
from pxr import Usd, UsdGeom
def fix_usd_files(root: str, pattern: str = "*.usd", target_mpu: float = 1.0,
target_up: str = "Z", dry_run: bool = False, skip: list = None):
usd_files = glob.glob(os.path.join(root, "**", pattern), recursive=True)
usd_files = [f for f in usd_files if not f.endswith(".bak")]
usd_files.sort()
skip = skip or []
print(f"Scanning {len(usd_files)} USD files under: {root}")
if dry_run:
print("[DRY RUN] No files will be modified.\n")
print()
fixed = 0
skipped = 0
skipped_dir = 0
errors = 0
for fpath in usd_files:
rel = os.path.relpath(fpath, root)
# Check skip patterns
if any(s in rel for s in skip):
print(f" SKIP (dir filter): {rel}")
skipped_dir += 1
continue
try:
stage = Usd.Stage.Open(fpath)
if stage is None:
print(f" ERROR: Cannot open {rel}")
errors += 1
continue
mpu = UsdGeom.GetStageMetersPerUnit(stage)
up = UsdGeom.GetStageUpAxis(stage)
needs_fix = (mpu != target_mpu) or (up != target_up)
if not needs_fix:
print(f" OK: {rel} (mpu={mpu}, up={up})")
skipped += 1
continue
if dry_run:
print(f" WOULD FIX: {rel} (mpu={mpu}, up={up} -> mpu={target_mpu}, up={target_up})")
fixed += 1
continue
# Backup
bak = fpath + ".bak"
if not os.path.exists(bak):
shutil.copy2(fpath, bak)
# Fix
UsdGeom.SetStageMetersPerUnit(stage, target_mpu)
UsdGeom.SetStageUpAxis(stage, UsdGeom.Tokens.z if target_up == "Z" else UsdGeom.Tokens.y)
stage.GetRootLayer().Save()
print(f" FIXED: {rel} (mpu={mpu}/{up} -> {target_mpu}/{target_up})")
fixed += 1
except Exception as e:
print(f" ERROR: {rel} -> {e}")
errors += 1
print(f"\nDone: {fixed} {'would fix' if dry_run else 'fixed'}, {skipped} already OK, "
f"{skipped_dir} skipped (dir filter), {errors} errors.")
if not dry_run and fixed > 0:
print("Backups saved as *.bak.")
print("To restore: python migrate/fix_usd_metadata.py --root <DIR> --restore")
return fixed
def restore_backups(root: str, pattern: str = "*.usd"):
bak_files = glob.glob(os.path.join(root, "**", pattern + ".bak"), recursive=True)
bak_files.sort()
if not bak_files:
print(f"No .bak files found under: {root}")
return
print(f"Found {len(bak_files)} backup files. Restoring...")
for bak in bak_files:
original = bak[:-4] # remove .bak
shutil.copy2(bak, original)
print(f" Restored: {os.path.relpath(original, root)}")
print(f"\nDone: {len(bak_files)} files restored.")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Batch fix USD metersPerUnit/upAxis metadata for IS 5.0.0 compatibility",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Dry run on example assets:
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets --dry-run
# Fix all Aligned_obj.usd in example assets:
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets --pattern "Aligned_obj.usd"
# Fix everything except robot/curobo dirs:
python migrate/fix_usd_metadata.py --root . --skip curobo,robot
# Restore all backups:
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets --restore
""")
parser.add_argument("--root", default="workflows/simbox/example_assets",
help="Root directory to scan (default: workflows/simbox/example_assets)")
parser.add_argument("--pattern", default="*.usd", help="Filename glob pattern (default: *.usd)")
parser.add_argument("--target-mpu", type=float, default=1.0, help="Target metersPerUnit (default: 1.0)")
parser.add_argument("--target-up", choices=["Y", "Z"], default="Z", help="Target upAxis (default: Z)")
parser.add_argument("--dry-run", action="store_true", help="Scan and report only, do not modify files")
parser.add_argument("--skip", default="", help="Comma-separated dir substrings to skip (e.g. 'curobo,robot')")
parser.add_argument("--restore", action="store_true", help="Restore all .bak files to originals")
args = parser.parse_args()
if args.restore:
restore_backups(args.root, args.pattern)
else:
skip_list = [s.strip() for s in args.skip.split(",") if s.strip()]
fix_usd_files(args.root, args.pattern, args.target_mpu, args.target_up, args.dry_run, skip_list)

199
migrate/migrate.md Normal file
View File

@@ -0,0 +1,199 @@
# Isaac Sim 4.5.0 -> 5.0.0 Migration Guide
## Background
- GPU: NVIDIA RTX PRO 6000 Blackwell (SM_120, compute capability 12.0)
- IS 4.5.0 = Kit 106.5.0, DLSS Ray Reconstruction not supported on Blackwell (needs Kit >= 106.5.3), rendering has noise
- IS 5.0.0 = Kit 107.x, fixes Blackwell rendering noise
- IS 5.1.0 headless camera rendering completely broken (known bug GitHub #3250), abandoned
## Issues & Solutions
### 1. SimulationApp import path changed
**Error:** `ModuleNotFoundError: No module named 'omni.isaac.kit'`
**Cause:** IS 5.x changed the import path.
**Fix** (`nimbus_extension/components/load/env_loader.py`):
```python
try:
from isaacsim import SimulationApp # IS 5.x
except ImportError:
from omni.isaac.kit import SimulationApp # IS 4.x
```
### 2. USD metersPerUnit / upAxis not auto-compensated
**Error:** RigidObject Z coordinate flying to 109.75+ after `world.reset()`, curobo `plan_single()` fails with "Plan did not converge" (goal_pos Z = 25261 meters).
**Cause:** IS 4.5.0 auto-added `Rotate:unitsResolve` (X=90) and `Scale:unitsResolve` (0.01, 0.01, 0.01) xformOps to compensate for USD files with `metersPerUnit=0.01, upAxis=Y`. IS 5.0.0 no longer does this, causing PhysX to misinterpret RigidObject positions.
GeometryObjects (no physics) were unaffected — only RigidObjects with PhysX simulation had the issue.
**Fix:** Change USD metadata directly using pxr scripting. Run `fix_usd_metadata.py`:
```python
from pxr import Usd, UsdGeom
stage = Usd.Stage.Open(usd_path)
UsdGeom.SetStageMetersPerUnit(stage, 1.0) # was 0.01
UsdGeom.SetStageUpAxis(stage, UsdGeom.Tokens.z) # was Y
stage.GetRootLayer().Save()
```
All 14 `Aligned_obj.usd` files under `workflows/simbox/example_assets` have been batch-fixed. Backups saved as `.bak`.
**Verification:** After fix, pick_object_right Z went from 109.75 to 0.7968 (normal).
### 3. scipy `scalar_first` parameter not supported
**Error:** `Rotation.as_quat() takes no keyword arguments`
**Cause:** IS 5.0.0 ships scipy < 1.11 which doesn't support `scalar_first` parameter in `Rotation.from_quat()` / `Rotation.as_quat()`. Dozens of files use this parameter.
**Fix** (`launcher.py`): Monkey-patch via subclass at startup:
```python
try:
from scipy.spatial.transform import Rotation as _R
_R.from_quat([1, 0, 0, 0], scalar_first=True)
except TypeError:
# Install Rotation subclass that handles scalar_first
# See launcher.py for full implementation
pass
```
### 4. `arena_file_path` is None on second reset
**Error:** `arena_file_path` becomes None after first episode.
**Cause:** `self.task_cfg.pop("arena_file", None)` on line 107 of `simbox_dual_workflow.py` deletes the key after first use.
**Fix** (`workflows/simbox_dual_workflow.py`): Cache the value:
```python
self._arena_file_path = None # in __init__
# in reset():
arena_file_path = self.task_cfg.get("arena_file", None) or self._arena_file_path
if arena_file_path:
self._arena_file_path = arena_file_path
```
### 5. "Task name should be unique in the world"
**Error:** `AssertionError` when calling `world.add_task()` on second episode.
**Cause:** World retains tasks from previous episode.
**Fix** (`workflows/simbox_dual_workflow.py`):
```python
self.world._current_tasks.clear()
self.world.add_task(self.task)
```
### 6. "A prim already exists at prim path"
**Error:** Scene setup tries to re-create existing prims on repeated `world.reset()` calls.
**Fix** (`workflows/simbox/core/tasks/banana.py`): Add `_scene_initialized` guard:
```python
def set_up_scene(self, scene):
if getattr(self, '_scene_initialized', False):
self._task_objects = {}
self._task_objects |= (
getattr(self, 'fixtures', {}) |
getattr(self, 'objects', {}) |
getattr(self, 'robots', {}) |
getattr(self, 'cameras', {})
)
return
self._scene_initialized = True
super().set_up_scene(scene)
```
### 7. Collision group prim already exists
**Error:** Collision group prims persist across resets.
**Fix** (`workflows/simbox/core/utils/collision_utils.py`): Remove existing collision prims before re-creating:
```python
collision_prim = stage.GetPrimAtPath(collision_root_path)
if collision_prim.IsValid():
stage.RemovePrim(collision_root_path)
```
### 8. DomeLight environment map tilted
**Error:** HDR environment map appears rotated/tilted in viewport and rendered output.
**Cause:** DomeLight rotation was randomized on all 3 axes (X, Y, Z). Rotating X/Y tilts the environment.
**Fix** (`workflows/simbox/core/tasks/banana.py`): Only rotate Z axis:
```python
# Before:
rotation = [random.uniform(r[0], r[1]) for _ in range(3)]
# After:
rotation = [0.0, 0.0, random.uniform(r[0], r[1])]
```
## Files Modified
| File | Change |
|------|--------|
| `nimbus_extension/components/load/env_loader.py` | SimulationApp import compatibility |
| `launcher.py` | scipy Rotation monkey-patch |
| `workflows/simbox_dual_workflow.py` | arena_file caching, task clearing, task reuse |
| `workflows/simbox/core/tasks/banana.py` | Scene re-init guard, DomeLight rotation fix |
| `workflows/simbox/core/utils/collision_utils.py` | Collision group cleanup |
| `workflows/simbox/example_assets/**/Aligned_obj.usd` | metersPerUnit=1.0, upAxis=Z |
| `fix_usd_metadata.py` | Batch USD metadata fix script |
## Tools
Migration tools are located in the `migrate/` directory.
### scan_usd_metadata.py — Scan USD metadata
Scan all USD files and report their `metersPerUnit` / `upAxis`:
```bash
conda activate banana500
# Scan entire project
python migrate/scan_usd_metadata.py --root .
# Scan specific directory
python migrate/scan_usd_metadata.py --root workflows/simbox/example_assets
```
Exit code: 0 = all OK, 1 = found files with non-standard metadata.
建议扫描后丢给大模型分析一下然后再做下一步
### fix_usd_metadata.py — Batch fix USD metadata
Fix `metersPerUnit` and `upAxis` in USD files, with backup/restore support:
```bash
conda activate banana500
# Dry run — see what would be fixed, no changes made
python migrate/fix_usd_metadata.py --root . --dry-run --skip curobo,robot
# Fix example assets only (default)
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets
# Fix all USD in project, skip robot/curobo models
python migrate/fix_usd_metadata.py --root . --skip curobo,robot
# Fix only Aligned_obj.usd files
python migrate/fix_usd_metadata.py --root . --pattern "Aligned_obj.usd"
# Restore from backups (undo all fixes)
python migrate/fix_usd_metadata.py --root workflows/simbox/example_assets --restore
```
**Important:** Do NOT fix robot USD (curobo, split_aloha) their metersPerUnit/upAxis are intentionally set for their own coordinate systems. Use `--skip` to exclude them.
## Notes
- `plan_single()` occasionally returns None for edge-case target poses (workspace boundary). This is normal and happens in IS 4.5.0 too.
- Example HDR is only 1k resolution production should use 4k/8k for better background quality.
- Example `envmap_lib` only has 1 HDR file add more for randomization diversity.
- `non_recyclable_garbage/obj_0` does not exist in example assets (only obj_1~obj_11). Config changed to use `obj_1`.

177
migrate/migrate_imports.py Normal file
View File

@@ -0,0 +1,177 @@
"""
Batch migrate omni.isaac.* imports to use core.compat module.
Usage:
python migrate/migrate_imports.py --dry-run # preview changes
python migrate/migrate_imports.py # apply changes
"""
import argparse
import os
import re
# Files to process (relative to project root)
SCAN_DIRS = [
"workflows/simbox/core/cameras",
"workflows/simbox/core/controllers",
"workflows/simbox/core/objects",
"workflows/simbox/core/robots",
"workflows/simbox/core/skills",
"workflows/simbox/core/tasks",
"workflows/simbox/core/utils",
"workflows/simbox/solver",
"workflows/", # simbox_dual_workflow.py
"nimbus_extension/components/load",
]
# Skip these files/dirs
SKIP = ["curobo", "__pycache__", "compat.py", "tools/art"]
# Mapping: old import pattern -> compat import
# Format: (regex_pattern, replacement_imports_from_compat)
IMPORT_MAP = {
# Full module imports
r"from omni\.isaac\.core import World": "from core.compat import World",
r"from omni\.isaac\.core\.controllers import BaseController": "from core.compat import BaseController",
r"from omni\.isaac\.core\.tasks import BaseTask": "from core.compat import BaseTask",
r"from omni\.isaac\.core\.robots\.robot import Robot": "from core.compat import Robot",
r"from omni\.isaac\.core\.robots import Robot": "from core.compat import Robot",
r"from omni\.isaac\.core\.prims import XFormPrim": "from core.compat import XFormPrim",
r"from omni\.isaac\.core\.prims import RigidPrim": "from core.compat import RigidPrim",
r"from omni\.isaac\.core\.prims import GeometryPrim": "from core.compat import GeometryPrim",
r"from omni\.isaac\.core\.prims import RigidContactView": "from core.compat import RigidContactView",
r"from omni\.isaac\.core\.prims import RigidContactView, XFormPrim": "from core.compat import RigidContactView, XFormPrim",
r"from omni\.isaac\.core\.articulations\.articulation import Articulation": "from core.compat import Articulation",
r"from omni\.isaac\.core\.objects\.cylinder import VisualCylinder": "from core.compat import VisualCylinder",
r"from omni\.isaac\.core\.objects import cuboid, sphere": "from core.compat import cuboid, sphere",
r"from omni\.isaac\.core\.objects import cuboid": "from core.compat import cuboid",
r"from omni\.isaac\.core\.objects import sphere": "from core.compat import sphere",
r"from omni\.isaac\.core\.materials import PreviewSurface": "from core.compat import PreviewSurface",
r"from omni\.isaac\.core\.materials import OmniPBR": "from core.compat import OmniPBR",
r"from omni\.isaac\.core\.materials\.omni_pbr import OmniPBR.*": "from core.compat import OmniPBR",
r"from omni\.isaac\.core\.materials import OmniGlass, OmniPBR": "from core.compat import OmniGlass, OmniPBR",
r"from omni\.isaac\.core\.materials import OmniGlass": "from core.compat import OmniGlass",
r"from omni\.isaac\.core\.scenes\.scene import Scene": "from core.compat import Scene",
r"from omni\.isaac\.sensor import Camera": "from core.compat import Camera",
# Utils
r"from omni\.isaac\.core\.utils\.prims import get_prim_at_path\b(?!,)": "from core.compat import get_prim_at_path",
r"from omni\.isaac\.core\.utils\.prims import create_prim, get_prim_at_path": "from core.compat import create_prim, get_prim_at_path",
r"from omni\.isaac\.core\.utils\.prims import create_prim, is_prim_path_valid": "from core.compat import create_prim, is_prim_path_valid",
r"from omni\.isaac\.core\.utils\.prims import is_prim_path_valid": "from core.compat import is_prim_path_valid",
r"from omni\.isaac\.core\.utils\.prims import create_prim\b": "from core.compat import create_prim",
r"from omni\.isaac\.core\.utils\.transformations import get_relative_transform\b(?!,)": "from core.compat import get_relative_transform",
r"from omni\.isaac\.core\.utils\.transformations import pose_from_tf_matrix": "from core.compat import pose_from_tf_matrix",
r"from omni\.isaac\.core\.utils\.stage import get_current_stage": "from core.compat import get_current_stage",
r"from omni\.isaac\.core\.utils\.stage import add_reference_to_stage": "from core.compat import add_reference_to_stage",
r"from omni\.isaac\.core\.utils\.stage import get_stage_units": "from core.compat import get_stage_units",
r"from omni\.isaac\.core\.utils\.xforms import get_world_pose": "from core.compat import get_world_pose",
r"from omni\.isaac\.core\.utils\.types import ArticulationAction": "from core.compat import ArticulationAction",
r"from omni\.isaac\.core\.utils\.viewports import set_camera_view": "from core.compat import set_camera_view",
r"from omni\.isaac\.core\.utils\.semantics import add_update_semantics": "from core.compat import add_update_semantics",
r"from omni\.isaac\.core\.utils\.string import find_unique_string_name": "from core.compat import find_unique_string_name",
r"from omni\.isaac\.core\.utils\.extensions import enable_extension": "from core.compat import enable_extension",
r"from omni\.isaac\.core\.utils\.nucleus import get_assets_root_path as nucleus_path": "from core.compat import nucleus_path",
}
# Multi-line import blocks that need special handling
MULTILINE_PATTERNS = [
# get_relative_transform, pose_from_tf_matrix block
(
r"from omni\.isaac\.core\.utils\.transformations import \(\s*\n\s*get_relative_transform,\s*\n\s*pose_from_tf_matrix,?\s*\n\s*\)",
"from core.compat import get_relative_transform, pose_from_tf_matrix",
),
# prims multi-import blocks
(
r"from omni\.isaac\.core\.utils\.prims import \(\s*\n\s*get_prim_at_path,\s*\n\s*create_prim,\s*\n\s*is_prim_path_valid,?\s*\n\s*\)",
"from core.compat import get_prim_at_path, create_prim, is_prim_path_valid",
),
(
r"from omni\.isaac\.core\.utils\.prims import \(\s*\n[^)]+\)",
None, # skip complex blocks, handle manually
),
]
def should_skip(filepath):
for s in SKIP:
if s in filepath:
return True
return False
def migrate_file(filepath, dry_run=False):
with open(filepath, "r") as f:
content = f.read()
original = content
changes = []
# Handle multi-line imports first
for pattern, replacement in MULTILINE_PATTERNS:
if replacement is None:
continue
matches = list(re.finditer(pattern, content))
for m in matches:
changes.append(f" MULTI: {m.group()[:60]}... -> {replacement}")
content = re.sub(pattern, replacement, content)
# Handle single-line imports
for pattern, replacement in IMPORT_MAP.items():
matches = list(re.finditer(pattern, content))
for m in matches:
changes.append(f" {m.group().strip()} -> {replacement}")
content = re.sub(pattern, replacement, content)
if content != original:
if not dry_run:
with open(filepath, "w") as f:
f.write(content)
return changes
return []
def main():
parser = argparse.ArgumentParser(description="Migrate omni.isaac imports to core.compat")
parser.add_argument("--dry-run", action="store_true", help="Preview changes without modifying files")
parser.add_argument("--root", default=".", help="Project root directory")
args = parser.parse_args()
total_files = 0
total_changes = 0
for scan_dir in SCAN_DIRS:
full_dir = os.path.join(args.root, scan_dir)
if not os.path.exists(full_dir):
continue
if os.path.isfile(full_dir):
files = [full_dir]
else:
files = []
for root, dirs, filenames in os.walk(full_dir):
for fn in filenames:
if fn.endswith(".py"):
files.append(os.path.join(root, fn))
for filepath in sorted(files):
if should_skip(filepath):
continue
rel = os.path.relpath(filepath, args.root)
changes = migrate_file(filepath, args.dry_run)
if changes:
total_files += 1
total_changes += len(changes)
print(f"\n{rel}:")
for c in changes:
print(c)
action = "would change" if args.dry_run else "changed"
print(f"\n{'='*60}")
print(f"Done: {total_changes} imports {action} in {total_files} files.")
if args.dry_run:
print("Run without --dry-run to apply changes.")
if __name__ == "__main__":
main()

222
migrate/runtime_warnings.md Normal file
View File

@@ -0,0 +1,222 @@
# Runtime Warnings & Errors Analysis
Pipeline: `python launcher.py --config configs/simbox/de_plan_and_render_template.yaml`
Environment: IS 5.0.0, RTX PRO 6000 Blackwell, conda `banana500`
Date: 2026-04-03
---
## Errors
### 1. PhysicsUSD: triangle mesh collision on dynamic body
```
[Error] PhysicsUSD: Parse collision - triangle mesh collision (approximation None/MeshSimplification)
cannot be a part of a dynamic body, falling back to convexHull approximation:
/World/task_0/split_aloha/.../lifting_link/collisions
```
**Severity:** Medium (PhysX auto-fallback, but logs error every reset)
**Root cause:** Robot USD `lifting_link/collisions` mesh 使用 `meshSimplification` 碰撞近似PhysX 不支持动态物体使用 triangle mesh collision。
**Fix:** 修改 robot USD 将 `MeshCollisionAPI.approximation``meshSimplification` 改为 `convexHull`
```python
from pxr import Usd, UsdPhysics
stage = Usd.Stage.Open('workflows/simbox/example_assets/split_aloha_mid_360/robot.usd')
prim = stage.GetPrimAtPath('/Root/.../lifting_link/collisions')
UsdPhysics.MeshCollisionAPI(prim).GetApproximationAttr().Set('convexHull')
stage.GetRootLayer().Save()
```
**Status:** FIXED. Backup: `robot.usd.bak`
---
### 2. Physics scenes stepping mismatch
```
[Error] Physics scenes stepping is not the same, step subscription will be send with later step,
per scene step is not yet supported.
```
**Severity:** Low (仅首次 reset 时出现一次)
**Root cause:** IS 5.0.0 内部 physics scene 步进调度问题。physics_dt 和 rendering_dt 均为 1/30配置一致。
**Status:** IGNORED. IS 内部行为,不影响仿真结果。
---
## Warnings — Fixed
### 3. Camera aperture inconsistent with resolution
```
[Warning] 'verticalAperture' (1.529) and 'horizontalAperture' (2.095) are inconsistent with
the pixel resolution aspect ratio (1.333). Setting 'verticalAperture' to 1.572 to ensure square pixels.
```
**Severity:** Low (IS 自动修正,但每个相机打 2 条 warning)
**Root cause:** `custom_camera.py``self.initialize()` 之后才设置 aperture/focal_length`initialize()` 时就用默认 aperture 做一致性检查。
**Fix:** 重构 `custom_camera.py`,将 aperture/focal_length 设置移到 `initialize()` 之前。
```python
# Before (wrong order):
super().__init__(...)
self.initialize() # <-- checks aperture here with default values
# ... later ...
self.set_horizontal_aperture(...) # too late, warning already printed
# After (correct order):
super().__init__(...)
self.set_horizontal_aperture(...) # set aperture first
self.set_vertical_aperture(...)
self.set_focal_length(...)
self.initialize() # now checks with correct values
```
**Status:** FIXED in `workflows/simbox/core/cameras/custom_camera.py`
---
### 4. Camera.set_projection_type deprecated
```
[Warning] Camera.set_projection_type is deprecated and will be removed in a future release.
Please use Camera.set_lens_distortion_model instead.
```
**Severity:** Low
**Fix:** 替换 API 调用,带 fallback 兼容旧版本。
```python
# Before:
self.set_projection_type("pinhole")
# After:
try:
self.set_lens_distortion_model("pinhole")
except AttributeError:
self.set_projection_type("pinhole")
```
**Status:** FIXED in `workflows/simbox/core/cameras/custom_camera.py`
---
## Warnings — Requires Manual Action
### 5. CPU powersave mode
```
[Warning] CPU performance profile is set to powersave. This profile sets the CPU to the
lowest frequency reducing performance.
```
**Severity:** Medium (影响整体性能)
**Fix:** 需要 sudo 权限手动设置。
```bash
# 临时切换到 performance 模式 (重启后恢复)
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
# 永久设置 (Ubuntu)
sudo apt install cpufrequtils
echo 'GOVERNOR="performance"' | sudo tee /etc/default/cpufrequtils
sudo systemctl restart cpufrequtils
```
**Status:** MANUAL. 需要用户自行设置。
---
## Warnings — Ignored (Safe)
### 6. IS 5.0 API deprecation warnings (~100 条)
```
[Warning] omni.isaac.core has been deprecated in favor of isaacsim.core.api.
Please update your code accordingly.
```
**Severity:** None
**说明:** IS 5.0 对所有 `omni.isaac.*` 模块重命名为 `isaacsim.*`。旧 API 仍然可用,仅打印 deprecation 提示。不影响功能,无需修改。如果将来升级到 IS 6.x 可能需要迁移。
---
### 7. Curobo warp cache warnings
```
[Warning] [curobo] Object already in warp cache, using existing instance for:
/World/task_0/pick_object_left/Aligned/mesh
```
**Severity:** None
**说明:** Curobo 内部 warp kernel 缓存机制。同一 mesh 在多次 world update 中被重复加载时复用缓存。完全正常,无害。
---
### 8. Robot dummy_base inertia tensor warnings
```
[Warning] The rigid body at .../dummy_base_rotate has a possibly invalid inertia tensor of
{1.0, 1.0, 1.0} and a negative mass, small sphere approximated inertia was used.
```
**Severity:** None
**说明:** Robot USD 中 `dummy_base_rotate/x/y` 是虚拟关节用于全局位移不需要真实质量属性。PhysX 自动使用球体近似惯性张量,功能正常。
---
### 9. Table mesh corrupted normal primvar
```
[Warning] Mesh '/World/task_0/table/Instance/Group_default_00/SM_08_component6_0' has corrupted
data in primvar 'normal': buffer size 14880 doesn't match expected size 2482
```
**Severity:** Low
**说明:** Table USD 中一个子网格的 normal 数据大小不匹配。IS 会忽略损坏的 normal 并自动计算。不影响渲染和物理。如需修复,需在 DCC 工具Blender/Maya中重新导出 table USD。
---
### 10. GPU / Hardware warnings
```
[Warning] Skipping unsupported non-NVIDIA GPU: AMD Ryzen 7 9800X3D (RADV RAPHAEL_MENDOCINO)
[Warning] ECC is enabled on physical device 0
[Warning] IOMMU is enabled.
```
**Severity:** None
**说明:** 系统有 AMD 集显跳过正常。RTX PRO 6000 默认开启 ECC牺牲约 6% 显存带宽换数据可靠性。IOMMU 已启用。均为信息提示。
---
### 11. SdRenderVarPtr missing LdrColorSDhost
```
[Warning] SdRenderVarPtr missing valid input renderVar LdrColorSDhost
```
**Severity:** None
**说明:** Synthetic data pipeline 中 LDR color 渲染变量在 headless 模式下首帧未就绪。后续帧正常。不影响输出。
---
### 12. trimesh sampling warning
```
[Warning] [trimesh] only got 3/4 samples!
```
**Severity:** Low
**说明:** Grasp pose 采样时某个物体只获得 3 个(期望 4 个抓取采样点。不影响功能pipeline 会使用获得的采样继续运行。
---
## Summary
| Category | Count | Fixed | Ignored | Manual |
|----------|-------|-------|---------|--------|
| Errors | 2 | 1 | 1 | 0 |
| Warnings (actionable) | 3 | 2 | 0 | 1 |
| Warnings (safe to ignore) | 7 | 0 | 7 | 0 |
| **Total** | **12** | **3** | **8** | **1** |

View File

@@ -0,0 +1,70 @@
"""
Scan all USD files in the project and report their metersPerUnit / upAxis metadata.
Usage:
python migrate/scan_usd_metadata.py [--root DIR]
Default root: current working directory (project root).
"""
import argparse
import glob
import os
import sys
from pxr import Usd, UsdGeom
def scan(root: str, target_mpu: float = 1.0, target_up: str = "Z"):
usd_files = glob.glob(os.path.join(root, "**", "*.usd"), recursive=True)
usd_files = [f for f in usd_files if not f.endswith(".bak")]
usd_files.sort()
ok_files = []
diff_files = []
error_files = []
for fpath in usd_files:
try:
stage = Usd.Stage.Open(fpath)
if stage is None:
error_files.append((fpath, "Cannot open stage"))
continue
mpu = UsdGeom.GetStageMetersPerUnit(stage)
up = UsdGeom.GetStageUpAxis(stage)
rel = os.path.relpath(fpath, root)
if mpu == target_mpu and up == target_up:
ok_files.append((rel, mpu, up))
else:
diff_files.append((rel, mpu, up))
except Exception as e:
error_files.append((fpath, str(e)))
# Report
print(f"Scanned {len(ok_files) + len(diff_files) + len(error_files)} USD files under: {root}")
print(f" OK (mpu={target_mpu}, up={target_up}): {len(ok_files)}")
print(f" Different: {len(diff_files)}")
print(f" Errors: {len(error_files)}")
if diff_files:
print(f"\n{'='*80}")
print(f"Files with non-standard metadata (mpu != {target_mpu} or up != {target_up}):")
print(f"{'='*80}")
for rel, mpu, up in diff_files:
print(f" mpu={mpu:<8} up={up:<4} {rel}")
if error_files:
print(f"\n{'='*80}")
print("Files with errors:")
print(f"{'='*80}")
for fpath, err in error_files:
print(f" ERROR: {fpath} -> {err}")
return diff_files
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Scan USD files for metersPerUnit/upAxis metadata")
parser.add_argument("--root", default=os.getcwd(), help="Root directory to scan (default: cwd)")
args = parser.parse_args()
diff = scan(args.root)
sys.exit(1 if diff else 0)

View File

@@ -75,7 +75,7 @@ class EnvLoader(SceneLoader):
)
self.logger.info(f"simulator params: physics dt={physics_dt}, rendering dt={rendering_dt}")
from omni.isaac.core import World
from core.compat import World
world = World(
physics_dt=physics_dt,

View File

@@ -0,0 +1,168 @@
#!/usr/bin/env python3
"""
Direct HuggingFace dataset downloader that avoids full repo enumeration.
Uses the HF tree API per-directory, then downloads files via HTTP.
Usage:
python scripts/download_assets_direct.py [OPTIONS]
Options:
--min Download only required scene assets
--full Download all scene assets (default)
--with-curobo Also download CuRobo package
--with-drake Also download panda_drake package
--local-dir DIR Where to save (default: current directory)
--mirror URL HF mirror base URL (default: https://hf-mirror.com)
"""
import os
import sys
import argparse
import time
from pathlib import Path
from concurrent.futures import ThreadPoolExecutor, as_completed
import requests
REPO_ID = "InternRobotics/InternData-A1"
ASSET_PREFIX = "InternDataAssets"
def get_token():
token = os.environ.get("HF_TOKEN", "")
if not token:
token_file = Path.home() / ".cache/huggingface/token"
if token_file.exists():
token = token_file.read_text().strip()
return token
def list_dir(api_base, repo_id, path, token, session):
"""List files in a specific directory via HF tree API (non-recursive for speed)."""
url = f"{api_base}/api/datasets/{repo_id}/tree/main/{path}"
headers = {"Authorization": f"Bearer {token}"} if token else {}
files = []
cursor = None
while True:
params = {"expand": "false", "recursive": "true", "limit": "1000"}
if cursor:
params["cursor"] = cursor
r = session.get(url, headers=headers, params=params, timeout=30)
r.raise_for_status()
data = r.json()
if isinstance(data, list):
items = data
cursor = None
else:
items = data.get("items", [])
cursor = data.get("nextCursor")
for item in items:
if item.get("type") == "file":
files.append(item["path"])
if not cursor or not items:
break
return files
def download_file(mirror_base, repo_id, file_path, local_dir, token, session):
"""Download a single file."""
url = f"{mirror_base}/datasets/{repo_id}/resolve/main/{file_path}"
headers = {"Authorization": f"Bearer {token}"} if token else {}
local_path = Path(local_dir) / file_path
local_path.parent.mkdir(parents=True, exist_ok=True)
if local_path.exists():
return file_path, "skipped"
for attempt in range(3):
try:
with session.get(url, headers=headers, stream=True, timeout=60) as r:
r.raise_for_status()
with open(local_path, "wb") as f:
for chunk in r.iter_content(chunk_size=1024 * 1024):
f.write(chunk)
return file_path, "ok"
except Exception as e:
if attempt == 2:
return file_path, f"error: {e}"
time.sleep(2 ** attempt)
def download_dir(api_base, mirror_base, repo_id, path, local_dir, token, label, workers=8):
print(f"[INFO] Listing {label} ...")
session = requests.Session()
try:
files = list_dir(api_base, repo_id, path, token, session)
except Exception as e:
print(f"[ERROR] Failed to list {path}: {e}")
return
print(f"[INFO] Downloading {label} ({len(files)} files) ...")
errors = []
done = 0
with ThreadPoolExecutor(max_workers=workers) as pool:
futures = {pool.submit(download_file, mirror_base, repo_id, f, local_dir, token, session): f for f in files}
for fut in as_completed(futures):
fp, status = fut.result()
done += 1
if "error" in status:
errors.append((fp, status))
print(f" [{done}/{len(files)}] FAIL {fp}: {status}")
else:
print(f" [{done}/{len(files)}] {status.upper()} {fp}", end="\r")
print()
if errors:
print(f"[WARN] {len(errors)} files failed in {label}")
def download_file_single(api_base, mirror_base, repo_id, file_path, local_dir, token, label):
print(f"[INFO] Downloading {label} ...")
session = requests.Session()
fp, status = download_file(mirror_base, repo_id, file_path, local_dir, token, session)
if "error" in status:
print(f"[ERROR] {fp}: {status}")
else:
print(f"[INFO] {label} done.")
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--min", action="store_true")
parser.add_argument("--full", action="store_true")
parser.add_argument("--with-curobo", action="store_true")
parser.add_argument("--with-drake", action="store_true")
parser.add_argument("--local-dir", default=".")
parser.add_argument("--mirror", default="https://hf-mirror.com")
parser.add_argument("--workers", type=int, default=8)
args = parser.parse_args()
mode = "min" if args.min else "full"
mirror = os.environ.get("HF_ENDPOINT", args.mirror).rstrip("/")
api_base = mirror
token = get_token()
local_dir = args.local_dir
if not token:
print("[WARN] No HF token found. Gated datasets will fail.")
print(f"[INFO] Mirror: {mirror}, mode={mode}, curobo={args.with_curobo}, drake={args.with_drake}")
base = f"{ASSET_PREFIX}/assets"
# Required scene assets
print("[INFO] ========== Downloading required scene assets ==========")
for d in ["background_textures", "envmap_lib", "floor_textures", "table_textures", "table0"]:
download_dir(api_base, mirror, REPO_ID, f"{base}/{d}", local_dir, token, d, args.workers)
download_file_single(api_base, mirror, REPO_ID, f"{base}/table_info.json", local_dir, token, "table_info.json")
# Full mode extras
if mode == "full":
print("[INFO] ========== Downloading all robots and tasks ==========")
for robot in ["lift2", "franka", "frankarobotiq", "split_aloha_mid_360", "G1_120s"]:
download_dir(api_base, mirror, REPO_ID, f"{base}/{robot}", local_dir, token, f"robot:{robot}", args.workers)
for task in ["basic", "art", "long_horizon", "pick_and_place"]:
download_dir(api_base, mirror, REPO_ID, f"{base}/{task}", local_dir, token, f"task:{task}", args.workers)
if args.with_curobo:
print("[INFO] ========== Downloading CuRobo ==========")
download_dir(api_base, mirror, REPO_ID, f"{ASSET_PREFIX}/curobo", local_dir, token, "curobo", args.workers)
if args.with_drake:
print("[INFO] ========== Downloading panda_drake ==========")
download_dir(api_base, mirror, REPO_ID, f"{ASSET_PREFIX}/panda_drake", local_dir, token, "panda_drake", args.workers)
print(f"[INFO] Done!")
if __name__ == "__main__":
main()

View File

@@ -2,13 +2,10 @@ import numpy as np
import omni.replicator.core as rep
from core.cameras.base_camera import register_camera
from core.utils.camera_utils import get_src
from omni.isaac.core.prims import XFormPrim
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
)
from omni.isaac.sensor import Camera
from core.compat import XFormPrim
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix
from core.compat import Camera
@register_camera
@@ -32,14 +29,44 @@ class CustomCamera(Camera):
reference_path: Reference prim path for camera mounting
name: Camera name
"""
# ===== From cfg =====
pixel_size = cfg["params"].get("pixel_size")
f_number = cfg["params"].get("f_number")
focus_distance = cfg["params"].get("focus_distance")
fx, fy, cx, cy = cfg["params"].get("camera_params")
width = cfg["params"].get("resolution_width")
height = cfg["params"].get("resolution_height")
self.output_mode = cfg.get("output_mode", "rgb")
horizontal_aperture = pixel_size * 1e-3 * width / 10.0
vertical_aperture = pixel_size * 1e-3 * height / 10.0
focal_length_x = fx * pixel_size * 1e-3
focal_length_y = fy * pixel_size * 1e-3
focal_length = (focal_length_x + focal_length_y) / 2 / 10.0
# ===== Initialize camera =====
# Note: Camera.__init__ triggers aperture consistency warnings with default USD values.
# These are cosmetic — IS auto-corrects, and we override with correct values below.
super().__init__(
prim_path=prim_path,
name=name,
resolution=(cfg["params"]["resolution_width"], cfg["params"]["resolution_height"]),
resolution=(width, height),
*args,
**kwargs,
)
# ===== Set camera parameters =====
self.set_focal_length(focal_length)
self.set_focus_distance(focus_distance)
self.set_lens_aperture(f_number * 100.0)
self.set_horizontal_aperture(horizontal_aperture)
self.set_vertical_aperture(vertical_aperture)
self.set_clipping_range(0.05, 1.0e5)
try:
self.set_lens_distortion_model("pinhole")
except AttributeError:
self.set_projection_type("pinhole")
self.initialize()
self.with_distance = cfg["params"].get("with_distance", True)
self.with_semantic = cfg["params"].get("with_semantic", False)
@@ -61,30 +88,6 @@ class CustomCamera(Camera):
if self.with_motion_vector:
self.add_motion_vectors_to_frame()
# ===== From cfg =====
pixel_size = cfg["params"].get("pixel_size")
f_number = cfg["params"].get("f_number")
focus_distance = cfg["params"].get("focus_distance")
fx, fy, cx, cy = cfg["params"].get("camera_params")
width = cfg["params"].get("resolution_width")
height = cfg["params"].get("resolution_height")
self.output_mode = cfg.get("output_mode", "rgb")
# ===== Compute and set camera parameters =====
horizontal_aperture = pixel_size * 1e-3 * width
vertical_aperture = pixel_size * 1e-3 * height
focal_length_x = fx * pixel_size * 1e-3
focal_length_y = fy * pixel_size * 1e-3
focal_length = (focal_length_x + focal_length_y) / 2
self.set_focal_length(focal_length / 10.0)
self.set_focus_distance(focus_distance)
self.set_lens_aperture(f_number * 100.0)
self.set_horizontal_aperture(horizontal_aperture / 10.0)
self.set_vertical_aperture(vertical_aperture / 10.0)
self.set_clipping_range(0.05, 1.0e5)
self.set_projection_type("pinhole")
fx = width * self.get_focal_length() / self.get_horizontal_aperture()
fy = height * self.get_focal_length() / self.get_vertical_aperture()
self.is_camera_matrix = np.array([[fx, 0.0, cx], [0.0, fy, cy], [0.0, 0.0, 1.0]])

View File

@@ -0,0 +1,198 @@
"""
Isaac Sim 4.x / 5.x API compatibility layer.
IS 5.0 renamed all `omni.isaac.*` modules to `isaacsim.*`.
This module provides unified imports that work across both versions.
Import from here instead of directly from omni.isaac.* or isaacsim.*.
"""
# ===== Core =====
try:
from isaacsim.core.api import World
except ImportError:
from omni.isaac.core import World
# ===== Controllers =====
try:
from isaacsim.core.api.controllers.base_controller import BaseController
except ImportError:
from omni.isaac.core.controllers import BaseController
# ===== Tasks =====
try:
from isaacsim.core.api.tasks.base_task import BaseTask
except ImportError:
from omni.isaac.core.tasks import BaseTask
# ===== Robots =====
try:
from isaacsim.core.api.robots.robot import Robot
except ImportError:
from omni.isaac.core.robots.robot import Robot
# ===== Prims (class names changed in IS 5.x) =====
try:
from isaacsim.core.prims import SingleXFormPrim as XFormPrim
except ImportError:
from omni.isaac.core.prims import XFormPrim
try:
from isaacsim.core.prims import SingleRigidPrim as RigidPrim
except ImportError:
from omni.isaac.core.prims import RigidPrim
try:
from isaacsim.core.prims import SingleGeometryPrim as GeometryPrim
except ImportError:
from omni.isaac.core.prims import GeometryPrim
try:
from isaacsim.core.api.sensors.rigid_contact_view import RigidContactView
except ImportError:
from omni.isaac.core.prims import RigidContactView
# ===== Articulations =====
try:
from isaacsim.core.prims import SingleArticulation as Articulation
except ImportError:
from omni.isaac.core.articulations.articulation import Articulation
# ===== Objects =====
try:
from isaacsim.core.api.objects.cylinder import VisualCylinder
except ImportError:
from omni.isaac.core.objects.cylinder import VisualCylinder
try:
from isaacsim.core.api.objects import cuboid, sphere
except ImportError:
from omni.isaac.core.objects import cuboid, sphere
# ===== Materials =====
try:
from isaacsim.core.api.materials import PreviewSurface
except ImportError:
from omni.isaac.core.materials import PreviewSurface
try:
from isaacsim.core.api.materials import OmniPBR
except ImportError:
try:
from omni.isaac.core.materials import OmniPBR
except ImportError:
from omni.isaac.core.materials.omni_pbr import OmniPBR
try:
from isaacsim.core.api.materials import OmniGlass
except ImportError:
from omni.isaac.core.materials import OmniGlass
# ===== Scenes =====
try:
from isaacsim.core.api.scenes.scene import Scene
except ImportError:
from omni.isaac.core.scenes.scene import Scene
# ===== Utils: prims =====
try:
from isaacsim.core.utils.prims import (
get_prim_at_path,
create_prim,
delete_prim,
is_prim_path_valid,
)
except ImportError:
from omni.isaac.core.utils.prims import (
get_prim_at_path,
create_prim,
delete_prim,
is_prim_path_valid,
)
# ===== Utils: transformations =====
try:
from isaacsim.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
tf_matrices_from_poses,
)
except ImportError:
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
tf_matrices_from_poses,
)
# ===== Utils: stage =====
try:
from isaacsim.core.utils.stage import (
get_current_stage,
add_reference_to_stage,
)
except ImportError:
from omni.isaac.core.utils.stage import (
get_current_stage,
add_reference_to_stage,
)
# ===== Utils: xforms =====
try:
from isaacsim.core.utils.xforms import get_world_pose
except ImportError:
from omni.isaac.core.utils.xforms import get_world_pose
# ===== Utils: types =====
try:
from isaacsim.core.utils.types import ArticulationAction
except ImportError:
from omni.isaac.core.utils.types import ArticulationAction
# ===== Utils: viewports =====
try:
from isaacsim.core.utils.viewports import set_camera_view
except ImportError:
from omni.isaac.core.utils.viewports import set_camera_view
# ===== Utils: semantics =====
try:
from isaacsim.core.utils.semantics import add_update_semantics
except ImportError:
from omni.isaac.core.utils.semantics import add_update_semantics
# ===== Utils: string =====
try:
from isaacsim.core.utils.string import find_unique_string_name
except ImportError:
from omni.isaac.core.utils.string import find_unique_string_name
# ===== Utils: stage units =====
try:
from isaacsim.core.utils.stage import get_stage_units
except ImportError:
from omni.isaac.core.utils.stage import get_stage_units
# ===== Utils: extensions =====
try:
from isaacsim.core.utils.extensions import enable_extension
except ImportError:
from omni.isaac.core.utils.extensions import enable_extension
# ===== Utils: nucleus =====
try:
from isaacsim.storage.native import get_assets_root_path as nucleus_path
except ImportError:
from omni.isaac.core.utils.nucleus import get_assets_root_path as nucleus_path
# ===== Sensor: Camera =====
try:
from isaacsim.sensors.camera import Camera
except ImportError:
from omni.isaac.sensor import Camera
# ===== SimulationApp =====
try:
from isaacsim import SimulationApp
except ImportError:
from omni.isaac.kit import SimulationApp

View File

@@ -29,7 +29,7 @@ tasks:
objects:
-
name: pick_object_left
path: task/sort_the_rubbish/non_recyclable_garbage/obj_0/Aligned_obj.usd
path: task/sort_the_rubbish/non_recyclable_garbage/obj_1/Aligned_obj.usd
target_class: RigidObject
dataset: oo3d
category: bottle

View File

@@ -34,15 +34,12 @@ from curobo.wrap.reacher.motion_gen import (
MotionGenPlanConfig,
PoseCostMetric,
)
from omni.isaac.core import World
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
)
from omni.isaac.core.utils.types import ArticulationAction
from core.compat import World
from core.compat import BaseController
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix
from core.compat import ArticulationAction
# pylint: disable=line-too-long,unused-argument
@@ -249,6 +246,12 @@ class TemplateController(BaseController):
obstacles = self.usd_help.get_obstacles_from_stage(
ignore_substring=self.ignore_substring, reference_prim_path=self.reference_prim_path
).get_collision_check_world()
# Diagnostic: print what curobo sees as the collision world
n_cuboid = len(obstacles.cuboid) if obstacles.cuboid else 0
n_mesh = len(obstacles.mesh) if obstacles.mesh else 0
mesh_names = [m.name for m in obstacles.mesh] if obstacles.mesh else []
print(f"[CUROBO_WORLD] cuboids={n_cuboid}, meshes={n_mesh}, "
f"mesh_names={mesh_names[:5]}{'...' if n_mesh > 5 else ''}", flush=True)
if self.motion_gen is not None:
self.motion_gen.update_world(obstacles)
self.world_cfg = obstacles
@@ -277,6 +280,10 @@ class TemplateController(BaseController):
self.T_world_base_init = get_relative_transform(
get_prim_at_path(self.robot_base_path), get_prim_at_path(self.task.root_prim_path)
)
print(f"[TRANSFORM_DBG] robot_base_path={self.robot_base_path}", flush=True)
print(f"[TRANSFORM_DBG] root_prim_path={self.task.root_prim_path}", flush=True)
print(f"[TRANSFORM_DBG] T_world_base_init translation={self.T_world_base_init[:3, 3]}", flush=True)
print(f"[TRANSFORM_DBG] T_world_base_init full=\n{self.T_world_base_init}", flush=True)
self.T_world_ee_init = self.T_world_base_init @ self.T_base_ee_init
self._ee_trans, self._ee_ori = self.get_ee_pose()
self._ee_trans = self.tensor_args.to_device(self._ee_trans)

View File

@@ -5,11 +5,11 @@ import random
import numpy as np
from core.objects.base_object import register_object
from omni.isaac.core.articulations.articulation import Articulation
from omni.isaac.core.utils.stage import add_reference_to_stage
from core.compat import Articulation
from core.compat import add_reference_to_stage
try:
from omni.isaac.core.materials.omni_pbr import OmniPBR # Isaac Sim 4.1.0 / 4.2.0
from core.compat import OmniPBR
except ImportError:
from isaacsim.core.api.materials import OmniPBR # Isaac Sim 4.5.0

View File

@@ -1,8 +1,8 @@
import os
from core.objects.base_object import register_object
from omni.isaac.core.prims import GeometryPrim
from omni.isaac.core.utils.prims import create_prim
from core.compat import GeometryPrim
from core.compat import create_prim
from pxr import Gf, UsdPhysics

View File

@@ -3,15 +3,11 @@ import os
import random
from core.objects.base_object import register_object
from omni.isaac.core.prims import GeometryPrim
from omni.isaac.core.utils.prims import (
create_prim,
get_prim_at_path,
is_prim_path_valid,
)
from core.compat import GeometryPrim
from core.compat import create_prim, get_prim_at_path, is_prim_path_valid
try:
from omni.isaac.core.materials.omni_pbr import OmniPBR # Isaac Sim 4.1.0 / 4.2.0
from core.compat import OmniPBR
except ImportError:
from isaacsim.core.api.materials import OmniPBR # Isaac Sim 4.5.0

View File

@@ -3,12 +3,12 @@ import os
import random
from core.objects.base_object import register_object
from omni.isaac.core.prims import XFormPrim
from omni.isaac.core.utils.prims import is_prim_path_valid
from omni.isaac.core.utils.stage import get_current_stage
from core.compat import XFormPrim
from core.compat import is_prim_path_valid
from core.compat import get_current_stage
try:
from omni.isaac.core.materials.omni_pbr import OmniPBR # Isaac Sim 4.1.0 / 4.2.0
from core.compat import OmniPBR
except ImportError:
from isaacsim.core.api.materials import OmniPBR # Isaac Sim 4.5.0

View File

@@ -3,11 +3,11 @@ import os
import random
from core.objects.base_object import register_object
from omni.isaac.core.prims import RigidPrim
from omni.isaac.core.utils.prims import create_prim, get_prim_at_path
from core.compat import RigidPrim
from core.compat import create_prim, get_prim_at_path
try:
from omni.isaac.core.materials.omni_pbr import OmniPBR # Isaac Sim 4.1.0 / 4.2.0
from core.compat import OmniPBR
except ImportError:
from isaacsim.core.api.materials import OmniPBR # Isaac Sim 4.5.0
@@ -47,6 +47,21 @@ class RigidObject(RigidPrim):
self.base_prim_path = prim_path
rigid_prim_path = os.path.join(self.base_prim_path, cfg["prim_path_child"])
self.mesh_prim_path = str(get_prim_at_path(rigid_prim_path).GetChildren()[0].GetPrimPath())
# [LOAD_DBG] Print xformOps right after create_prim, before physics
try:
from pxr import UsdGeom
_xf = UsdGeom.Xformable(get_prim_at_path(rigid_prim_path))
for _op in _xf.GetOrderedXformOps():
print(f"[LOAD_DBG] {cfg_name} after_create_prim: {_op.GetName()} = {_op.Get()}", flush=True)
_l2w = _xf.ComputeLocalToWorldTransform(0)
print(f"[LOAD_DBG] {cfg_name} after_create_prim l2w=({_l2w[3][0]:.6f}, {_l2w[3][1]:.6f}, {_l2w[3][2]:.6f})", flush=True)
# Also check the USD file's own metersPerUnit
_ref_stage = get_prim_at_path(prim_path).GetStage()
_mpu = UsdGeom.GetStageMetersPerUnit(_ref_stage)
_up = UsdGeom.GetStageUpAxis(_ref_stage)
print(f"[LOAD_DBG] {cfg_name} stage metersPerUnit={_mpu} upAxis={_up}", flush=True)
except Exception as _e:
print(f"[LOAD_DBG] {cfg_name} error: {_e}", flush=True)
super().__init__(prim_path=rigid_prim_path, name=cfg["name"], *args, **kwargs)
def get_observations(self):

View File

@@ -3,10 +3,10 @@ import os
import random
from core.objects.base_object import register_object
from omni.isaac.core.prims import GeometryPrim
from core.compat import GeometryPrim
try:
from omni.isaac.core.materials.omni_pbr import OmniPBR # Isaac Sim 4.1.0 / 4.2.0
from core.compat import OmniPBR
except ImportError:
from isaacsim.core.api.materials import OmniPBR # Isaac Sim 4.5.0

View File

@@ -3,11 +3,11 @@ import os
import random
from core.objects.base_object import register_object
from omni.isaac.core.prims import XFormPrim
from omni.isaac.core.utils.prims import create_prim, is_prim_path_valid
from core.compat import XFormPrim
from core.compat import create_prim, is_prim_path_valid
try:
from omni.isaac.core.materials.omni_pbr import OmniPBR # Isaac Sim 4.1.0 / 4.2.0
from core.compat import OmniPBR
except ImportError:
from isaacsim.core.api.materials import OmniPBR # Isaac Sim 4.5.0

View File

@@ -7,12 +7,9 @@ from copy import deepcopy
import numpy as np
from core.robots.base_robot import register_robot
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.utils.prims import create_prim, get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
tf_matrix_from_pose,
)
from core.compat import Robot
from core.compat import create_prim, get_prim_at_path
from core.compat import get_relative_transform, tf_matrix_from_pose
from scipy.interpolate import interp1d

View File

@@ -3,15 +3,11 @@ import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from core.utils.interpolate_utils import linear_interpolation
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R

View File

@@ -3,11 +3,11 @@ from copy import deepcopy
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import get_relative_transform
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform
from solver.planner import KPAMPlanner

View File

@@ -3,11 +3,11 @@ from copy import deepcopy
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import get_relative_transform
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform
from scipy.spatial.transform import Rotation as R
from solver.planner import KPAMPlanner

View File

@@ -4,15 +4,11 @@ from copy import deepcopy
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig, OmegaConf
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
# pylint: disable=unused-argument

View File

@@ -5,15 +5,11 @@ import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from core.utils.usd_geom_utils import compute_bbox
from omegaconf import DictConfig, OmegaConf
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R
from scipy.spatial.transform import Slerp

View File

@@ -12,14 +12,11 @@ from core.utils.plan_utils import (
)
from core.utils.transformation_utils import poses_from_tf_matrices
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, tf_matrix_from_pose
from omni.timeline import get_timeline_interface
from scipy.spatial.transform import Rotation as R

View File

@@ -5,15 +5,11 @@ from copy import deepcopy
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
# pylint: disable=unused-argument

View File

@@ -3,15 +3,11 @@ from copy import deepcopy
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R

View File

@@ -10,14 +10,11 @@ from core.utils.transformation_utils import (
poses_from_tf_matrices,
)
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R
from scipy.spatial.transform import Slerp

View File

@@ -1,9 +1,9 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
# pylint: disable=unused-argument

View File

@@ -1,13 +1,10 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.transformations import (
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import pose_from_tf_matrix, tf_matrix_from_pose
# pylint: disable=unused-argument

View File

@@ -1,9 +1,9 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
# pylint: disable=unused-argument

View File

@@ -4,9 +4,9 @@ import torch
from core.skills.base_skill import BaseSkill, register_skill
from core.utils.interpolate_utils import linear_interpolation
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
# pylint: disable=unused-argument

View File

@@ -12,14 +12,11 @@ from core.utils.plan_utils import (
)
from core.utils.transformation_utils import poses_from_tf_matrices
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, tf_matrix_from_pose
@register_skill

View File

@@ -1,9 +1,9 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
# pylint: disable=unused-argument

View File

@@ -1,9 +1,9 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
# pylint: disable=unused-argument

View File

@@ -1,9 +1,9 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from scipy.spatial.transform import Rotation as R

View File

@@ -3,11 +3,11 @@ from copy import deepcopy
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import get_relative_transform
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform
from scipy.spatial.transform import Rotation as R
from solver.planner import KPAMPlanner

View File

@@ -11,14 +11,11 @@ from core.utils.plan_utils import (
)
from core.utils.transformation_utils import poses_from_tf_matrices
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, tf_matrix_from_pose
# pylint: disable=unused-argument
@@ -284,8 +281,49 @@ class Pick(BaseSkill):
if frame == "body":
return self.T_obj_ee
T_world_obj = tf_matrix_from_pose(*self.pick_obj.get_local_pose())
local_pose = self.pick_obj.get_local_pose()
world_pose = self.pick_obj.get_world_pose()
print(f"[PICK_DBG] prim_path={self.pick_obj.prim_path}", flush=True)
print(f"[PICK_DBG] local_pose trans={local_pose[0]}", flush=True)
print(f"[PICK_DBG] world_pose trans={world_pose[0]}", flush=True)
print(f"[PICK_DBG] local_pose ori={local_pose[1]}", flush=True)
print(f"[PICK_DBG] world_pose ori={world_pose[1]}", flush=True)
parent_prim = get_prim_at_path(self.pick_obj.prim_path).GetParent()
print(f"[PICK_DBG] parent_prim_path={parent_prim.GetPrimPath()}", flush=True)
grandparent_prim = parent_prim.GetParent()
print(f"[PICK_DBG] grandparent_prim_path={grandparent_prim.GetPrimPath()}", flush=True)
try:
from pxr import UsdGeom
obj_prim = get_prim_at_path(self.pick_obj.prim_path)
obj_xf = UsdGeom.Xformable(obj_prim)
xform_ops = obj_xf.GetOrderedXformOps()
print(f"[PICK_DBG] obj_xformOps_count={len(xform_ops)}", flush=True)
for op in xform_ops:
print(f"[PICK_DBG] obj_xformOp: {op.GetName()} val={op.Get()}", flush=True)
obj_l2w = obj_xf.ComputeLocalToWorldTransform(0)
print(f"[PICK_DBG] obj_USD_l2w=({obj_l2w[3][0]:.6f}, {obj_l2w[3][1]:.6f}, {obj_l2w[3][2]:.6f})", flush=True)
parent_xf = UsdGeom.Xformable(parent_prim)
parent_l2w = parent_xf.ComputeLocalToWorldTransform(0)
print(f"[PICK_DBG] parent_l2w_translate=({parent_l2w[3][0]}, {parent_l2w[3][1]}, {parent_l2w[3][2]})", flush=True)
gp_xf = UsdGeom.Xformable(grandparent_prim)
gp_l2w = gp_xf.ComputeLocalToWorldTransform(0)
print(f"[PICK_DBG] grandparent_l2w_translate=({gp_l2w[3][0]}, {gp_l2w[3][1]}, {gp_l2w[3][2]})", flush=True)
stage = parent_prim.GetStage()
mpu = UsdGeom.GetStageMetersPerUnit(stage)
print(f"[PICK_DBG] stage_metersPerUnit={mpu}", flush=True)
# Check get_local_pose source
glp_method = type(self.pick_obj).get_local_pose
print(f"[PICK_DBG] get_local_pose_from={glp_method.__qualname__}", flush=True)
except Exception as e:
import traceback
print(f"[PICK_DBG] UsdGeom error: {e}", flush=True)
traceback.print_exc()
print(f"[PICK_DBG] root_prim_path={self.task.root_prim_path}", flush=True)
print(f"[PICK_DBG] reference_prim_path={self.controller.reference_prim_path}", flush=True)
T_world_obj = tf_matrix_from_pose(*local_pose)
print(f"[PICK_DBG] T_world_obj translation={T_world_obj[:3,3]}", flush=True)
T_world_ee = T_world_obj[None] @ self.T_obj_ee
print(f"[PICK_DBG] T_world_ee[0] translation={T_world_ee[0,:3,3]}", flush=True)
if frame == "world":
return T_world_ee
@@ -294,8 +332,11 @@ class Pick(BaseSkill):
T_world_base = get_relative_transform(
get_prim_at_path(self.controller.reference_prim_path), get_prim_at_path(self.task.root_prim_path)
)
print(f"[PICK_DBG] T_world_base translation={T_world_base[:3,3]}", flush=True)
T_base_world = np.linalg.inv(T_world_base)
print(f"[PICK_DBG] T_base_world translation={T_base_world[:3,3]}", flush=True)
T_base_ee = T_base_world[None] @ T_world_ee
print(f"[PICK_DBG] T_base_ee[0] translation={T_base_ee[0,:3,3]}", flush=True)
return T_base_ee
def get_contact(self, contact_threshold=0.0):

View File

@@ -12,15 +12,11 @@ from core.utils.plan_utils import (
from core.utils.transformation_utils import create_pose_matrices, poses_from_tf_matrices
from core.utils.usd_geom_utils import compute_bbox
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R

View File

@@ -7,9 +7,9 @@ from core.utils.transformation_utils import (
perturb_position,
)
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from scipy.spatial.transform import Rotation as R

View File

@@ -3,11 +3,11 @@ from copy import deepcopy
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig, OmegaConf
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import get_relative_transform
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform
from scipy.spatial.transform import Rotation as R
from solver.planner import KPAMPlanner

View File

@@ -3,15 +3,11 @@ import torch
from core.skills.base_skill import BaseSkill, register_skill
from core.utils.interpolate_utils import linear_interpolation
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R

View File

@@ -1,15 +1,11 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R

View File

@@ -3,15 +3,12 @@ from core.skills.base_skill import BaseSkill, register_skill
from core.utils.transformation_utils import get_orientation, perturb_orientation
from core.utils.usd_geom_utils import compute_bbox
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.objects.cylinder import VisualCylinder
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import (
get_relative_transform,
tf_matrix_from_pose,
)
from core.compat import BaseController
from core.compat import VisualCylinder
from core.compat import Robot
from core.compat import BaseTask
from core.compat import get_prim_at_path
from core.compat import get_relative_transform, tf_matrix_from_pose
from scipy.spatial.transform import Rotation as R
@@ -143,6 +140,8 @@ class Track(BaseSkill):
def cal_table_2_base(self):
tgt = self.task.fixtures["table"]
bbox_tgt = compute_bbox(tgt.prim)
print(f"[BBOX_DBG] table bbox min={list(bbox_tgt.min)}, max={list(bbox_tgt.max)}", flush=True)
print(f"[BBOX_DBG] T_base_2_world translation={self.T_base_2_world[:3, 3]}", flush=True)
table_center = (np.asarray(bbox_tgt.min) + np.asarray(bbox_tgt.max)) / 2
tgt_z_max = bbox_tgt.max[2]
table_center[2] = tgt_z_max

View File

@@ -1,9 +1,9 @@
import numpy as np
from core.skills.base_skill import BaseSkill, register_skill
from omegaconf import DictConfig
from omni.isaac.core.controllers import BaseController
from omni.isaac.core.robots.robot import Robot
from omni.isaac.core.tasks import BaseTask
from core.compat import BaseController
from core.compat import Robot
from core.compat import BaseTask
# pylint: disable=consider-using-generator,too-many-public-methods,unused-argument

View File

@@ -18,16 +18,12 @@ from core.utils.scene_utils import deactivate_selected_prims
from core.utils.transformation_utils import get_orientation
from core.utils.visual_distractor import set_distractors
from omegaconf import DictConfig
from omni.isaac.core.materials import PreviewSurface
from omni.isaac.core.prims import RigidContactView, XFormPrim
from omni.isaac.core.scenes.scene import Scene
from omni.isaac.core.tasks import BaseTask
from omni.isaac.core.utils.prims import (
delete_prim,
get_prim_at_path,
is_prim_path_valid,
)
from omni.isaac.core.utils.stage import get_current_stage
from core.compat import PreviewSurface
from core.compat import RigidContactView, XFormPrim
from core.compat import Scene
from core.compat import BaseTask
from core.compat import delete_prim, get_prim_at_path, is_prim_path_valid
from core.compat import get_current_stage
from omni.physx.scripts import particleUtils
from pxr import Gf, PhysxSchema, Sdf, Usd, UsdGeom, UsdLux, UsdShade, Vt
from scipy.spatial.transform import Rotation as R
@@ -335,6 +331,14 @@ class BananaBaseTask(BaseTask):
orientation = get_orientation(cfg.get("euler"), cfg.get("quaternion"))
obj.set_local_pose(translation=cfg.get("translation"), orientation=orientation)
obj.set_local_scale(cfg.get("scale", [1.0, 1.0, 1.0]))
# [LOAD_DBG] Print after set_local_pose
try:
from pxr import UsdGeom
_xf = UsdGeom.Xformable(get_prim_at_path(obj.prim_path))
for _op in _xf.GetOrderedXformOps():
print(f"[LOAD_DBG] {cfg['name']} after_set_local_pose: {_op.GetName()} = {_op.Get()}", flush=True)
except Exception as _e:
print(f"[LOAD_DBG] {cfg['name']} after_set_local_pose error: {_e}", flush=True)
obj.set_visibility(cfg.get("visible", True))
# Extra behavior per type
@@ -490,7 +494,7 @@ class BananaBaseTask(BaseTask):
if cfg.get("apply_randomization", False):
envmap_id = random.randint(0, len(envmap_hdr_path_list) - 1)
intensity = random.uniform(cfg["intensity_range"][0], cfg["intensity_range"][1])
rotation = [random.uniform(cfg["rotation_range"][0], cfg["rotation_range"][1]) for _ in range(3)]
rotation = [0.0, 0.0, random.uniform(cfg["rotation_range"][0], cfg["rotation_range"][1])]
else:
envmap_id = 0
intensity = 1000.0

View File

@@ -1,7 +1,7 @@
import numpy as np
from omni.isaac.core.utils.prims import get_prim_at_path
from omni.isaac.core.utils.transformations import get_relative_transform
from omni.isaac.sensor import Camera
from core.compat import get_prim_at_path
from core.compat import get_relative_transform
from core.compat import Camera
def _get_annotator(camera: Camera, annotator_name: str):

View File

@@ -1,5 +1,5 @@
import numpy as np
from omni.isaac.core.utils.transformations import pose_from_tf_matrix
from core.compat import pose_from_tf_matrix
from scipy.spatial.transform import Rotation as R

View File

@@ -18,7 +18,7 @@ logging.getLogger("urllib3").setLevel(logging.WARNING)
def set_semantic_label(prim: Prim, label):
from omni.isaac.core.utils.semantics import add_update_semantics
from core.compat import add_update_semantics
if prim.GetTypeName() == "Mesh":
add_update_semantics(prim, semantic_label=label, type_label="class")
@@ -28,7 +28,7 @@ def set_semantic_label(prim: Prim, label):
def set_plane_semantic_label(prim: Prim, label):
from omni.isaac.core.utils.semantics import add_update_semantics
from core.compat import add_update_semantics
if prim.GetTypeName() == "Plane":
add_update_semantics(prim, semantic_label=label, type_label="class")
@@ -38,7 +38,7 @@ def set_plane_semantic_label(prim: Prim, label):
def set_robot_semantic_label(robot: Prim, parent_name: str):
from omni.isaac.core.utils.semantics import add_update_semantics
from core.compat import add_update_semantics
if robot.GetTypeName() == "Mesh":
prim_path = str(robot.GetPrimPath())

View File

@@ -13,17 +13,9 @@ import solver.kpam.SE3_utils as SE3_utils
import solver.kpam.term_spec as term_spec
import yaml
from colored import fg
from omni.isaac.core.utils.prims import (
create_prim,
get_prim_at_path,
is_prim_path_valid,
)
from omni.isaac.core.utils.transformations import (
get_relative_transform,
pose_from_tf_matrix,
tf_matrices_from_poses,
)
from omni.isaac.core.utils.xforms import get_world_pose
from core.compat import create_prim, get_prim_at_path, is_prim_path_valid
from core.compat import get_relative_transform, pose_from_tf_matrix, tf_matrices_from_poses
from core.compat import get_world_pose
from pydrake.all import *
from scipy.spatial.transform import Rotation as R
from solver.kpam.mp_builder import OptimizationBuilderkPAM
@@ -966,8 +958,8 @@ if __name__ == "__main__":
# from omni.isaac.franka.controllers.pick_place_controller import PickPlaceController
from gensim_testing_v2.tasks.close_microwave import CloseMicrowave
from omni.isaac.core import World
from omni.isaac.core.utils.types import ArticulationAction
from core.compat import World
from core.compat import ArticulationAction
# from solver.planner import KPAMPlanner

Some files were not shown because too many files have changed in this diff Show More