feat: add test tube pick task with custom assets and grasp annotations
- Add pick_test_tube task: USDC asset repackaging, grasp generation, task config - Add tools: usdc_to_obj.py, repackage_test_tube.py, fix_test_tube_materials.py - Add custom_task_guide.md: full Chinese documentation for creating custom tasks - Add crawled InternDataEngine online docs (23 pages) - Add grasp generation script (gen_tube_grasp.py) and pipeline config
This commit is contained in:
Tangger
259
docs_crawled/config_assets.md
Normal file
259
docs_crawled/config_assets.md
Normal file
@@ -0,0 +1,259 @@
|
||||
# Source: https://internrobotics.github.io/InternDataEngine-Docs/config/assets.html
|
||||
|
||||
# Assets [](#assets)
|
||||
|
||||
All simulation assets are organized under `workflows/simbox/assets `. This document describes the asset structure, required assets, and organization conventions.
|
||||
|
||||
## Asset Structure [](#asset-structure)
|
||||
|
||||
```
|
||||
workflows/simbox/assets/
|
||||
├── envmap_lib/ # HDR environment maps for scene lighting
|
||||
├── background_textures/ # Background textures for domain randomization
|
||||
├── floor_textures/ # Floor textures for domain randomization
|
||||
├── table_textures/ # Table surface textures
|
||||
├── table0/ # Default table USD model
|
||||
├── lift2/ # ARX Lift-2 robot assets
|
||||
├── G1_120s/ # Genie-1 robot assets
|
||||
├── franka/ # Franka robot assets
|
||||
├── frankarobotiq/ # Franka with Robotiq 2F-85 gripper assets
|
||||
├── split_aloha_mid_360/ # Agilex Split ALOHA robot assets
|
||||
├── basic/ # Basic task-specific assets
|
||||
│ ├── arrange_the_tableware/
|
||||
│ ├── hang_the_cup_on_rack/
|
||||
│ ├── pour/
|
||||
│ ├── store_the_eggs/
|
||||
│ └── ... # Other task folders
|
||||
├── art/ # Articulated object assets
|
||||
│ ├── electriccooker/
|
||||
│ ├── microwave_gr/
|
||||
│ ├── laptop/
|
||||
│ └── ...
|
||||
├── long_horizon/ # Long-horizon task assets
|
||||
└── pick_and_place/ # Pick-and-place task assets
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
6
|
||||
7
|
||||
8
|
||||
9
|
||||
10
|
||||
11
|
||||
12
|
||||
13
|
||||
14
|
||||
15
|
||||
16
|
||||
17
|
||||
18
|
||||
19
|
||||
20
|
||||
21
|
||||
22
|
||||
23
|
||||
24
|
||||
|
||||
## Required Assets [](#required-assets)
|
||||
|
||||
To run a simulation task, the following assets are required:
|
||||
|
||||
- **Environment Maps **( `envmap_lib/ `): HDR lighting for scene illumination.
|
||||
- **Background Textures **( `background_textures/ `): Domain randomization backgrounds.
|
||||
- **Floor Textures **( `floor_textures/ `): Floor appearance variation.
|
||||
- **Table Model **( `table0/ `): Default table fixture.
|
||||
- **Robot Assets **(e.g., `lift2/ `): Robot USD and kinematics configs.
|
||||
- **Task Assets **(e.g., `basic/<task_name>/ `): Objects specific to each task.
|
||||
|
||||
## Rigid Object Assets [](#rigid-object-assets)
|
||||
|
||||
This section describes the asset structure for rigid objects, using `workflows/simbox/example_assets/task/sort_the_rubbish `as an example.
|
||||
|
||||
### Directory Organization [](#directory-organization)
|
||||
|
||||
Different object categories are placed in separate folders at the same level. Objects of the same category are grouped together in a single folder to facilitate **category-level randomization **.
|
||||
|
||||
```
|
||||
workflows/simbox/example_assets/task/sort_the_rubbish/
|
||||
├── garbage_can/ # Container category
|
||||
│ ├── recyclable_can/
|
||||
│ └── nonrecyclable_can/
|
||||
├── non_recyclable_garbage/ # Non-recyclable items category
|
||||
│ ├── obj_1/
|
||||
│ ├── obj_2/
|
||||
│ └── ...
|
||||
└── recyclable_garbage/ # Recyclable items category
|
||||
├── bottle_0/
|
||||
├── bottle_1/
|
||||
└── ...
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
6
|
||||
7
|
||||
8
|
||||
9
|
||||
10
|
||||
11
|
||||
12
|
||||
|
||||
### Object Instance Structure [](#object-instance-structure)
|
||||
|
||||
Each object instance folder contains the USD model, textures, and annotations:
|
||||
|
||||
```
|
||||
workflows/simbox/example_assets/task/sort_the_rubbish/non_recyclable_garbage/obj_2/
|
||||
├── Aligned_obj.usd # Object USD with physics properties (mass, collision, etc.)
|
||||
├── Aligned_grasp_sparse.npy # Grasp pose annotations (for pickable objects)
|
||||
└── textures/ # Texture files
|
||||
└── baked_mesh_*.png
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
|
||||
**File Naming Conventions: **
|
||||
|
||||
- **Aligned_obj.usd **( file ): USD file containing the 3D model with complete physics properties (mass, collision mesh, etc.).
|
||||
- **Aligned_grasp_sparse.npy **( file ): Grasp pose annotations for manipulation tasks.
|
||||
- **textures/ **( directory ): Directory containing texture maps for the object.
|
||||
|
||||
## Articulated Object Assets [](#articulated-object-assets)
|
||||
|
||||
Articulated objects (e.g., doors, drawers, appliances) follow a similar organization pattern. This section uses `workflows/simbox/assets/art/electriccooker `as an example.
|
||||
|
||||
### Directory Organization [](#directory-organization-1)
|
||||
|
||||
Different articulated object categories are placed in separate folders. Objects within the same category share consistent orientation, category classification, and functionality after preprocessing.
|
||||
|
||||
```
|
||||
workflows/simbox/assets/art/electriccooker/
|
||||
├── electriccooker_0002/
|
||||
├── electriccooker_0008/
|
||||
├── electriccooker_0011/
|
||||
├── electriccooker_0017/
|
||||
├── electriccooker_0031/
|
||||
└── ...
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
6
|
||||
7
|
||||
|
||||
### Articulated Object Instance Structure [](#articulated-object-instance-structure)
|
||||
|
||||
Each articulated object instance contains the USD model, materials, and keypoint annotations:
|
||||
|
||||
```
|
||||
workflows/simbox/assets/art/electriccooker/electriccooker_0031/
|
||||
├── instance.usd # Articulated object USD
|
||||
├── instance.png # Preview image
|
||||
├── Materials/ # Material definitions
|
||||
└── Kps/ # Keypoint annotations
|
||||
└── close_h/ # Keypoints for "close horizontally" action
|
||||
├── keypoints.json
|
||||
├── keypoints_final.json
|
||||
└── info.json
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
6
|
||||
7
|
||||
8
|
||||
9
|
||||
|
||||
**File Naming Conventions: **
|
||||
|
||||
- **instance.usd **( file ): USD file for articulated objects (unlike rigid objects which use `Aligned_obj.usd `).
|
||||
- **instance.png **( file ): Preview/thumbnail image.
|
||||
- **Materials/ **( directory ): Material and texture definitions.
|
||||
- **Kps/ **( directory ): Keypoint annotations for interaction points.
|
||||
- **Kps/<action_type>/ **( directory ): Action-specific keypoints (e.g., `close_h `for closing horizontally).
|
||||
|
||||
### Keypoint Annotations [](#keypoint-annotations)
|
||||
|
||||
Keypoint annotations define interaction points for articulated objects:
|
||||
|
||||
```
|
||||
Kps/
|
||||
├── close_h/ # Close horizontally (e.g., laptop, pot, electric cooker)
|
||||
├── close_v/ # Close vertically (e.g., microwave)
|
||||
├── open_h/ # Open horizontally
|
||||
├── open_v/ # Open vertically
|
||||
├── pull/ # Pull (e.g., drawer)
|
||||
├── push/ # Push
|
||||
└── ...
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
6
|
||||
7
|
||||
8
|
||||
|
||||
Each action folder contains:
|
||||
|
||||
- **keypoints.json **( file ): Initial keypoint positions.
|
||||
- **keypoints_final.json **( file ): Final/processed keypoint positions.
|
||||
- **info.json **( file ): Metadata about the keypoints.
|
||||
|
||||
## Asset Configuration [](#asset-configuration)
|
||||
|
||||
Assets are referenced in task YAML configurations:
|
||||
yaml
|
||||
```
|
||||
objects:
|
||||
- name: bottle_1
|
||||
path: task/sort_the_rubbish/recyclable_garbage/bottle_1/Aligned_obj.usd
|
||||
target_class: RigidObject
|
||||
category: bottle
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
5
|
||||
|
||||
For more details on object configuration, see [Objects](/InternDataEngine-Docs/concepts/objects.html).
|
||||
|
||||
## Best Practices [](#best-practices)
|
||||
|
||||
-
|
||||
|
||||
**Category Organization **: Group objects of the same category in a single folder for efficient domain randomization.
|
||||
|
||||
-
|
||||
|
||||
**Consistent Naming **: Use standardized naming conventions:
|
||||
|
||||
- `Aligned_obj.usd `for rigid objects
|
||||
- `instance.usd `for articulated objects
|
||||
- `Aligned_grasp_sparse.npy `for grasp annotations
|
||||
|
||||
-
|
||||
|
||||
**Complete Physics Properties **: Ensure USD files include:
|
||||
|
||||
- Accurate mass properties
|
||||
- Collision meshes
|
||||
- Appropriate friction coefficients
|
||||
|
||||
-
|
||||
|
||||
**Preprocessing **: For all objects, ensure consistent frame and alignment across instances in the same category.
|
||||
Reference in New Issue
Block a user