Anonymous supplement
Language-Guided Static Grasp Synthesis
Prompt comparison
Toothpaste tube
- Detailed
Aim for mostly straight ring and pinky fingers and a bent index finger; the middle finger and thumb MCP should be partly bent.
- Conversational
On the tube, keep the thumb tip farther toward the nozzle than the ring fingertip. Leave the pinky tip some room.
- Brief
Pinky tip clear of the tube.
The conversational and brief prompts were written for the reference pose and checked against its measured geometry.
Method and evaluation
We translate a text instruction into five finger commands and condition the grasp generator on the commands and the object. Each command uses one of ten angle bins for the four proximal interphalangeal joints and the thumb metacarpophalangeal joint. Higher bins indicate greater extension. We measure the realized finger angles on the resulting mesh.
We compare text-only and text-plus-point-cloud control on 304 synthetic prompts with novel instruction chains across 19 objects held out from language-model fine-tuning. Both variants used the same grasp generator, trained on all 50 benchmark objects. The evaluation used fixed single-seed adapters.
| Evaluation stage | Text only | Text + PointNet |
|---|---|---|
| Predicted finger commands | 12.50% | 25.33% |
| Realized finger angles | 30.92% | 24.34% |
The separate refinement study evaluated 18 fixed candidates across six tasks. Four tasks passed all predefined geometric and task-specific checks. The elephant and banana refinement tasks failed.
The interactive viewer shows pre-rendered views of each pose at sampled angles.