- Add pick_test_tube task: USDC asset repackaging, grasp generation, task config - Add tools: usdc_to_obj.py, repackage_test_tube.py, fix_test_tube_materials.py - Add custom_task_guide.md: full Chinese documentation for creating custom tasks - Add crawled InternDataEngine online docs (23 pages) - Add grasp generation script (gen_tube_grasp.py) and pipeline config
259 lines
7.4 KiB
Markdown
259 lines
7.4 KiB
Markdown
# Source: https://internrobotics.github.io/InternDataEngine-Docs/config/assets.html
|
||
|
||
# Assets [](#assets)
|
||
|
||
All simulation assets are organized under `workflows/simbox/assets `. This document describes the asset structure, required assets, and organization conventions.
|
||
|
||
## Asset Structure [](#asset-structure)
|
||
|
||
```
|
||
workflows/simbox/assets/
|
||
├── envmap_lib/ # HDR environment maps for scene lighting
|
||
├── background_textures/ # Background textures for domain randomization
|
||
├── floor_textures/ # Floor textures for domain randomization
|
||
├── table_textures/ # Table surface textures
|
||
├── table0/ # Default table USD model
|
||
├── lift2/ # ARX Lift-2 robot assets
|
||
├── G1_120s/ # Genie-1 robot assets
|
||
├── franka/ # Franka robot assets
|
||
├── frankarobotiq/ # Franka with Robotiq 2F-85 gripper assets
|
||
├── split_aloha_mid_360/ # Agilex Split ALOHA robot assets
|
||
├── basic/ # Basic task-specific assets
|
||
│ ├── arrange_the_tableware/
|
||
│ ├── hang_the_cup_on_rack/
|
||
│ ├── pour/
|
||
│ ├── store_the_eggs/
|
||
│ └── ... # Other task folders
|
||
├── art/ # Articulated object assets
|
||
│ ├── electriccooker/
|
||
│ ├── microwave_gr/
|
||
│ ├── laptop/
|
||
│ └── ...
|
||
├── long_horizon/ # Long-horizon task assets
|
||
└── pick_and_place/ # Pick-and-place task assets
|
||
```
|
||
1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
6
|
||
7
|
||
8
|
||
9
|
||
10
|
||
11
|
||
12
|
||
13
|
||
14
|
||
15
|
||
16
|
||
17
|
||
18
|
||
19
|
||
20
|
||
21
|
||
22
|
||
23
|
||
24
|
||
|
||
## Required Assets [](#required-assets)
|
||
|
||
To run a simulation task, the following assets are required:
|
||
|
||
- **Environment Maps **( `envmap_lib/ `): HDR lighting for scene illumination.
|
||
- **Background Textures **( `background_textures/ `): Domain randomization backgrounds.
|
||
- **Floor Textures **( `floor_textures/ `): Floor appearance variation.
|
||
- **Table Model **( `table0/ `): Default table fixture.
|
||
- **Robot Assets **(e.g., `lift2/ `): Robot USD and kinematics configs.
|
||
- **Task Assets **(e.g., `basic/<task_name>/ `): Objects specific to each task.
|
||
|
||
## Rigid Object Assets [](#rigid-object-assets)
|
||
|
||
This section describes the asset structure for rigid objects, using `workflows/simbox/example_assets/task/sort_the_rubbish `as an example.
|
||
|
||
### Directory Organization [](#directory-organization)
|
||
|
||
Different object categories are placed in separate folders at the same level. Objects of the same category are grouped together in a single folder to facilitate **category-level randomization **.
|
||
|
||
```
|
||
workflows/simbox/example_assets/task/sort_the_rubbish/
|
||
├── garbage_can/ # Container category
|
||
│ ├── recyclable_can/
|
||
│ └── nonrecyclable_can/
|
||
├── non_recyclable_garbage/ # Non-recyclable items category
|
||
│ ├── obj_1/
|
||
│ ├── obj_2/
|
||
│ └── ...
|
||
└── recyclable_garbage/ # Recyclable items category
|
||
├── bottle_0/
|
||
├── bottle_1/
|
||
└── ...
|
||
```
|
||
1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
6
|
||
7
|
||
8
|
||
9
|
||
10
|
||
11
|
||
12
|
||
|
||
### Object Instance Structure [](#object-instance-structure)
|
||
|
||
Each object instance folder contains the USD model, textures, and annotations:
|
||
|
||
```
|
||
workflows/simbox/example_assets/task/sort_the_rubbish/non_recyclable_garbage/obj_2/
|
||
├── Aligned_obj.usd # Object USD with physics properties (mass, collision, etc.)
|
||
├── Aligned_grasp_sparse.npy # Grasp pose annotations (for pickable objects)
|
||
└── textures/ # Texture files
|
||
└── baked_mesh_*.png
|
||
```
|
||
1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
|
||
**File Naming Conventions: **
|
||
|
||
- **Aligned_obj.usd **( file ): USD file containing the 3D model with complete physics properties (mass, collision mesh, etc.).
|
||
- **Aligned_grasp_sparse.npy **( file ): Grasp pose annotations for manipulation tasks.
|
||
- **textures/ **( directory ): Directory containing texture maps for the object.
|
||
|
||
## Articulated Object Assets [](#articulated-object-assets)
|
||
|
||
Articulated objects (e.g., doors, drawers, appliances) follow a similar organization pattern. This section uses `workflows/simbox/assets/art/electriccooker `as an example.
|
||
|
||
### Directory Organization [](#directory-organization-1)
|
||
|
||
Different articulated object categories are placed in separate folders. Objects within the same category share consistent orientation, category classification, and functionality after preprocessing.
|
||
|
||
```
|
||
workflows/simbox/assets/art/electriccooker/
|
||
├── electriccooker_0002/
|
||
├── electriccooker_0008/
|
||
├── electriccooker_0011/
|
||
├── electriccooker_0017/
|
||
├── electriccooker_0031/
|
||
└── ...
|
||
```
|
||
1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
6
|
||
7
|
||
|
||
### Articulated Object Instance Structure [](#articulated-object-instance-structure)
|
||
|
||
Each articulated object instance contains the USD model, materials, and keypoint annotations:
|
||
|
||
```
|
||
workflows/simbox/assets/art/electriccooker/electriccooker_0031/
|
||
├── instance.usd # Articulated object USD
|
||
├── instance.png # Preview image
|
||
├── Materials/ # Material definitions
|
||
└── Kps/ # Keypoint annotations
|
||
└── close_h/ # Keypoints for "close horizontally" action
|
||
├── keypoints.json
|
||
├── keypoints_final.json
|
||
└── info.json
|
||
```
|
||
1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
6
|
||
7
|
||
8
|
||
9
|
||
|
||
**File Naming Conventions: **
|
||
|
||
- **instance.usd **( file ): USD file for articulated objects (unlike rigid objects which use `Aligned_obj.usd `).
|
||
- **instance.png **( file ): Preview/thumbnail image.
|
||
- **Materials/ **( directory ): Material and texture definitions.
|
||
- **Kps/ **( directory ): Keypoint annotations for interaction points.
|
||
- **Kps/<action_type>/ **( directory ): Action-specific keypoints (e.g., `close_h `for closing horizontally).
|
||
|
||
### Keypoint Annotations [](#keypoint-annotations)
|
||
|
||
Keypoint annotations define interaction points for articulated objects:
|
||
|
||
```
|
||
Kps/
|
||
├── close_h/ # Close horizontally (e.g., laptop, pot, electric cooker)
|
||
├── close_v/ # Close vertically (e.g., microwave)
|
||
├── open_h/ # Open horizontally
|
||
├── open_v/ # Open vertically
|
||
├── pull/ # Pull (e.g., drawer)
|
||
├── push/ # Push
|
||
└── ...
|
||
```
|
||
1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
6
|
||
7
|
||
8
|
||
|
||
Each action folder contains:
|
||
|
||
- **keypoints.json **( file ): Initial keypoint positions.
|
||
- **keypoints_final.json **( file ): Final/processed keypoint positions.
|
||
- **info.json **( file ): Metadata about the keypoints.
|
||
|
||
## Asset Configuration [](#asset-configuration)
|
||
|
||
Assets are referenced in task YAML configurations:
|
||
yaml
|
||
```
|
||
objects:
|
||
- name: bottle_1
|
||
path: task/sort_the_rubbish/recyclable_garbage/bottle_1/Aligned_obj.usd
|
||
target_class: RigidObject
|
||
category: bottle
|
||
```
|
||
1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
|
||
For more details on object configuration, see [Objects](/InternDataEngine-Docs/concepts/objects.html).
|
||
|
||
## Best Practices [](#best-practices)
|
||
|
||
-
|
||
|
||
**Category Organization **: Group objects of the same category in a single folder for efficient domain randomization.
|
||
|
||
-
|
||
|
||
**Consistent Naming **: Use standardized naming conventions:
|
||
|
||
- `Aligned_obj.usd `for rigid objects
|
||
- `instance.usd `for articulated objects
|
||
- `Aligned_grasp_sparse.npy `for grasp annotations
|
||
|
||
-
|
||
|
||
**Complete Physics Properties **: Ensure USD files include:
|
||
|
||
- Accurate mass properties
|
||
- Collision meshes
|
||
- Appropriate friction coefficients
|
||
|
||
-
|
||
|
||
**Preprocessing **: For all objects, ensure consistent frame and alignment across instances in the same category. |