tangger/issacdataengine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add test tube pick task with custom assets and grasp annotations

Browse Source 
- Add pick_test_tube task: USDC asset repackaging, grasp generation, task config
- Add tools: usdc_to_obj.py, repackage_test_tube.py, fix_test_tube_materials.py
- Add custom_task_guide.md: full Chinese documentation for creating custom tasks
- Add crawled InternDataEngine online docs (23 pages)
- Add grasp generation script (gen_tube_grasp.py) and pipeline config
This commit is contained in:
Tangger
2026-04-05 11:01:59 +08:00
parent 6314603676
commit 3d6b73753a
36 changed files with 18013 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

385
docs_crawled/custom_controller.md Normal file
View File

@@ -0,0 +1,385 @@
# Source: https://internrobotics.github.io/InternDataEngine-Docs/custom/controller.html
# New Controller [​](#new-controller)
This guide explains how to create a new robot controller for motion planning with CuRobo.
## Part 1: Create CuRobo Configuration Files [​](#part-1-create-curobo-configuration-files)
Our controller uses a separate CuRobo config file for each arm. You need to:
- **Prepare URDF files for each arm **- Extract single-arm URDF from your robot's full URDF
- **Create CuRobo config files **- Follow the official CuRobo tutorial
### Step 1.1: Prepare URDF Files [​](#step-1-1-prepare-urdf-files)
For dual-arm robots, create separate URDF files for left and right arms. Save them in:
```
workflows/simbox/curobo/src/curobo/content/assets/robot/
├── your_robot_left_arm.urdf
├── your_robot_right_arm.urdf
└── meshes/
```
1
2
3
4
Hint
Each arm URDF should be a standalone file containing only that arm's links and joints. Make sure mesh paths in the URDF are correct relative to the assets folder.
### Step 1.2: Create CuRobo Config Files [​](#step-1-2-create-curobo-config-files)
Follow the official CuRobo tutorial: [Configuring a New Robot](https://curobo.org/tutorials/1_robot_configuration.html)
Key steps:
- Convert URDF to USD using Isaac Sim
- Use Lula Robot Description Editor to generate collision spheres
- Configure self-collision parameters
Save config files in:
```
workflows/simbox/curobo/src/curobo/content/configs/robot/
├── your_robot_left_arm.yml
└── your_robot_right_arm.yml
```
1
2
3
Warning
Isaac Sim's Lula Robot Description Editor is only available in **Isaac Sim 4.0.0 **. Later versions have removed this feature. If you're using a newer version, you may need to manually create the collision sphere configuration or use Isaac Sim 4.0.0 for this step.
### Example CuRobo Config Structure [​](#example-curobo-config-structure)
yaml
```
# your_robot_left_arm.yml
robot_cfg:
kinematics:
usd_path: "robot/your_robot_left_arm.usd"
usd_robot_root: "/robot"
urdf_path: "robot/your_robot_left_arm.urdf"
asset_root_path: "robot"
base_link: "base_link"
ee_link: "link6" # End-effector link (must match fl_ee_path in robot.yaml)
collision_link_names:
- link1
- link2
# ...
collision_spheres:
link1:
- center: [0.0, 0.0, 0.0]
radius: 0.05
# ...
self_collision_ignore:
link1: ["link2", "link3"]
# ...
cspace:
joint_names: ["joint1", "joint2", "joint3", "joint4", "joint5", "joint6"]
retract_config: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
null_space_weight: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
cspace_distance_weight: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
max_jerk: 500.0
max_acceleration: 15.0
```
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
## Part 2: Update Robot YAML Configuration [​](#part-2-update-robot-yaml-configuration)
In your `new_robot.yaml `(see [New Robot](/InternDataEngine-Docs/custom/robot.html)), fill in the `robot_file `field:
### For Dual-Arm Robots [​](#for-dual-arm-robots)
yaml
```
# CuRobo kinematics config files (one per arm)
robot_file:
- workflows/simbox/curobo/src/curobo/content/configs/robot/your_robot_left_arm.yml
- workflows/simbox/curobo/src/curobo/content/configs/robot/your_robot_right_arm.yml
```
1
2
3
4
### For Single-Arm Robots [​](#for-single-arm-robots)
For single-arm robots, we conventionally name it as the "left" arm:
yaml
```
# CuRobo kinematics config file
robot_file:
- workflows/simbox/curobo/src/curobo/content/configs/robot/your_robot_left_arm.yml
```
1
2
3
## Part 3: Create Controller Python Class [​](#part-3-create-controller-python-class)
Create a new controller file in `workflows/simbox/core/controllers/ `. Here's a new controller template:
python
```
"""NewRobot controller implementation."""
import numpy as np
from core.controllers.base_controller import register_controller
from core.controllers.template_controller import TemplateController
@register_controller
class NewRobotController(TemplateController):
"""Controller for NewRobot."""
def _get_default_ignore_substring(self):
"""Return default collision ignore substrings."""
return ["material", "table", "floor", "scene"]
def _configure_joint_indices(self, robot_file: str) -> None:
"""
Configure joint names and indices for motion planning.
This method maps between CuRobo's joint names and the robot USD's joint names.
"""
# Raw joint names from CuRobo config (must match cspace.joint_names in yaml)
self.raw_js_names = ["joint1", "joint2", "joint3", "joint4", "joint5", "joint6"]
if "left" in robot_file:
# Command joint names in robot USD (with prefix)
self.cmd_js_names = ["fl_joint1", "fl_joint2", "fl_joint3",
"fl_joint4", "fl_joint5", "fl_joint6"]
# Joint indices from robot config
self.arm_indices = np.array(self.robot.cfg["left_joint_indices"])
self.gripper_indices = np.array(self.robot.cfg["left_gripper_indices"])
self.reference_prim_path = self.task.robots[self.name].fl_base_path
self.lr_name = "left"
self._gripper_state = 1.0 if self.robot.left_gripper_state == 1.0 else -1.0
elif "right" in robot_file:
# Command joint names in robot USD (with prefix)
self.cmd_js_names = ["fr_joint1", "fr_joint2", "fr_joint3",
"fr_joint4", "fr_joint5", "fr_joint6"]
self.arm_indices = np.array(self.robot.cfg["right_joint_indices"])
self.gripper_indices = np.array(self.robot.cfg["right_gripper_indices"])
self.reference_prim_path = self.task.robots[self.name].fr_base_path
self.lr_name = "right"
self._gripper_state = 1.0 if self.robot.right_gripper_state == 1.0 else -1.0
else:
raise ValueError("robot_file must contain 'left' or 'right'")
# Gripper joint position for open/close
# Adjust based on your gripper's joint values
self._gripper_joint_position = np.array([0.044]) # Example: 44mm for open
def get_gripper_action(self):
"""
Map gripper state to joint positions.
Returns:
np.ndarray: Joint positions for gripper.
State mapping:
- _gripper_state = 1.0 -> Gripper OPEN
- _gripper_state = -1.0 -> Gripper CLOSED
"""
# When _gripper_state is 1.0, gripper opens (joint value = positive)
# When _gripper_state is -1.0, gripper closes (joint value = 0 or negative)
return np.clip(self._gripper_state * self._gripper_joint_position, 0.0, 0.1)
```
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
_configure_joint_indices(self, robot_file: str )
Configure joint names and indices for motion planning. This method maps between CuRobo's joint names and the robot USD's joint names.
Parameters:
- **robot_file **( str ): Path to the CuRobo config file, used to determine left/right arm.
Key Variables:
- **raw_js_names **( list ): Joint names in CuRobo config (from `cspace.joint_names `in yaml).
- **cmd_js_names **( list ): Joint names in robot USD (must match actual USD joint names).
- **arm_indices **( np.ndarray ): Joint indices for arm in articulation.
- **gripper_indices **( np.ndarray ): Joint indices for gripper in articulation.
Warning
The order of `raw_js_names `and `cmd_js_names `must correspond! The i-th element in `raw_js_names `maps to the i-th element in `cmd_js_names `.
get_gripper_action(self)
Convert `_gripper_state `to actual gripper joint positions.
Returns:
- **np.ndarray **: Joint positions for gripper.
State Mapping:
- **_gripper_state = 1.0 **: Gripper OPEN (joint value that fingers open)
- **_gripper_state = -1.0 **: Gripper CLOSED (joint value that fingers close)
Code Example:
python
```
# Parallel gripper (single joint)
def get_gripper_action(self):
return np.clip(self._gripper_state * self._gripper_joint_position, 0.0, 0.1)
# Two-finger gripper (two joints, e.g., Franka Panda)
def get_gripper_action(self):
finger1 = np.clip(self._gripper_state * 0.04, 0.0, 0.04)
finger2 = np.clip(-self._gripper_state * 0.04, -0.04, 0.0)
return np.array([finger1, finger2])
```
1
2
3
4
5
6
7
8
9
### Register the Controller [​](#register-the-controller)
Add your controller to `workflows/simbox/core/controllers/__init__.py `:
python
```
from core.controllers.newrobot_controller import NewRobotController
__all__ = [
# ... existing controllers
"NewRobotController",
]
```
1
2
3
4
5
6
## Checklist [​](#checklist)
- [ ] Prepare single-arm URDF files for each arm
- [ ] Create CuRobo config YAML files using Isaac Sim 4.0.0 (Lula)
- [ ] Update `robot_file `in robot YAML configuration
- [ ] Create controller Python class
- [ ] Implement `_get_default_ignore_substring() `
- [ ] Implement `_configure_joint_indices() `with correct joint name mapping
- [ ] Implement `get_gripper_action() `with correct open/close mapping
- [ ] Register controller in `__init__.py `
- [ ] Test with task configuration
## References [​](#references)
- [CuRobo Robot Configuration Tutorial](https://curobo.org/tutorials/1_robot_configuration.html)
- [Controllers API Reference](/InternDataEngine-Docs/api/controllers.html)
- [Template Controller Source](https://github.com/your-org/DataEngine/blob/main/workflows/simbox/core/controllers/template_controller.py)