Model Card for ACT/BananaPick (Distilled, Multi-Backend)
Action Chunking Transformer Policy (as per Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware) trained for banana pick-and-place on a 1-arm SO-101 robot. This checkpoint is a distilled model obtained via knowledge distillation.
This repo bundles multiple deployment backends so the same policy can run on GPU/CPU (PyTorch), Huawei Ascend (OM), and Rockchip RK3588 (RKNN) without per-backend glue code. All backends are described by a single inference_manifest.json routing table.
Repository Structure
├── config.json # torch 权重配置
├── model.safetensors # torch 权重 (340 MB)
├── policy_preprocessor.json
├── policy_postprocessor.json
├── policy_preprocessor_step_3_normalizer_processor.safetensors
├── policy_postprocessor_step_0_unnormalizer_processor.safetensors
├── train_config.json
├── inference_manifest.json # 部署路由表 (权威)
└── artifacts/
├── ascend/ascend_310b1/policy-1031c8ca5f1e.om # Ascend OM (123 MB)
└── rknn/rk3588/policy-cfcf62db3da4.rknn # Rockchip RKNN (146 MB)
The directory layout and every file path inside
inference_manifest.jsonmust stay in sync. Do not rename or relocate the weight/artifact files — the manifest's sha256 checks and path bindings depend on them.
Deployment Backends
Read inference_manifest.json → deployments[<target>] to route to the right backend. Each entry carries the artifact path, runtime, SoC, and full input/output tensor bindings (shape, dtype, semantic name).
| Target | Backend | Runtime | Artifact | Hardware |
|---|---|---|---|---|
torch-cuda | torch | PyTorch | model.safetensors (in bundle) | NVIDIA GPU |
cpu | torch | PyTorch | model.safetensors (in bundle) | CPU |
torch-npu | torch | PyTorch | model.safetensors (in bundle) | Huawei NPU |
ascend_310b1 | ascend | ACL | artifacts/ascend/ascend_310b1/policy-*.om | Ascend 310B1 |
rk3588 | rknn | rknn-lite2 | artifacts/rknn/rk3588/policy-*.rknn | Rockchip RK3588 |
Input tensors (all backends): observation.state [1,6] float32, observation.images.top [1,3,480,640] NCHW, observation.images.wrist [1,3,480,640] NCHW.
Output tensor: action [1,100,6] float32 (100-step action chunk, 6-dim).
How to Get Started with the Model
See the IB-Robot project (particularly the inference_service) for instructions on how to load and deploy this model with ROS 2.
To load the PyTorch backend directly in Python (weights are at the repo root):
from lerobot.common.policies.act.modeling_act import ACTPolicy
policy = ACTPolicy.from_pretrained("openEuler/IB_Robot_ACT_banana_pick_distill")
For Ascend (OM) or RKNN deployment, consume inference_manifest.json and load the artifact under the matching deployments[<target>].artifacts.policy.path.
Training Details
This model was trained via knowledge distillation (kd: true) within the IB-Robot framework. The OM/RKNN artifacts were converted from these torch weights (verified same-origin by sha256).
- Policy: ACT (Action Chunking with Transformers)
- Training method: Knowledge distillation
- Robot: 1-arm SO-101
- Cameras: top, wrist (480x640)
- Action dim: 6
- Training steps: 500,000 (configured)
- Checkpoint: step_160000
- Chunk size: 100
- Batch size: 32
- Vision backbone: ResNet18
- Optimizer: AdamW (lr=1e-5, weight_decay=1e-4)
- Latent dim (VAE): 32
- Dim model: 1024
Citation
@software{ib_robot,
title = {IB-Robot: Intelligence Boom Robot},
url = {https://gitcode.com/openeuler/IB_Robot},
license = {Apache-2.0}
}