Anonymous Supplement

Supplementary material Double-blind review

Language-Guided
Static Grasp Synthesis

We study how language can control the shape of a 3D hand around an object, using the bend of each finger as an intermediate representation.

Explore examples Read the method
STATIC 3D RENDER
A rendered hand pinches the near rim of a bowl. VIEW C
FIG. 01 “Pinch the rim of the bowl.”Example after geometric refinement

01 Method

From instruction to pose

The intermediate representation lets us evaluate language translation separately from the pose the generator actually produces.

  1. 01 / INPUT

    Text instruction

    Translate a natural-language request into a desired bend level for each of the five fingers.

  2. 02 / CONTROL

    Five finger commands

    Use ten angle bins for the four proximal interphalangeal joints and the thumb’s metacarpophalangeal joint. Higher bins indicate greater extension.

  3. 03 / OUTPUT

    Static hand mesh

    Condition the grasp generator on the object and the five commands, then measure the realized finger angles on the resulting mesh.

02 Prompt styles

Three prompt styles

Compare three ways to describe the same reference pose, from detailed finger states to a brief cue.

A rendered hand curves around a vertical toothpaste tube.
Reference pose · Toothpaste tube

  1. Tier 01Detailed

    Specifies the state of most or all fingers.

    Aim for mostly straight ring and pinky fingers and a bent index finger; the middle finger and thumb MCP should be partly bent.

  2. Tier 02Conversational

    Describes selected fingers or how they relate to each other or the object.

    On the tube, keep the thumb tip farther toward the nozzle than the ring fingertip. Leave the pinky tip some room.

  3. Tier 03Brief

    Gives one key constraint or a short cue for the whole hand.

    Pinky tip clear of the tube.

The conversational and brief prompts were written for the shown poses and checked against measured joint angles or fingertip geometry.

03 Qualitative examples

Grasps across objects

Explore selected text-to-grasp outputs, then inspect a separate set of poses produced with geometric refinement.

Direct generation

Each card shows the original instruction and a render of the corresponding object-conditioned mesh before geometric refinement.

Rendered hand above the fuselage of a model airplane.
Model airplaneKeep the index, ring and pinky mostly straight; let the middle PIP and thumb MCP be partly bent.
Rendered hand rests across the top of a twin-bell alarm clock.
Alarm clockKeep the middle mostly straight while the index, ring and pinky are bent. Leave the thumb MCP partly bent.
Rendered fingers wrap partway around a curved banana model.
BananaKeep the index mostly straight, with the ring bent. Let the middle, pinky, and thumb MCP be partly bent.
Rendered hand rests across the back of an elephant figurine.
Elephant modelUse a mostly straight index finger and a bent ring finger, with the middle and pinky partly bent and the thumb MCP partly bent.
Rendered hand wraps around the handle of a horizontal claw hammer.
Claw hammerLet the index and middle fingers be mostly straight while the ring and pinky fingers are bent, and keep the thumb MCP joint partly bent.
Rendered fingers curve over a light bulb with its screw base visible.
Light bulbKeep the index and pinky mostly straight, and leave the middle and ring partly bent. Keep the thumb MCP partly bent as well.
Rendered hand rests along the side of a rabbit figurine with upright ears.
Rabbit modelKeep the index, middle, ring, and thumb MCP mostly straight, with the pinky partly bent.
Rendered hand wraps around a stemmed wine glass above its base.
Wine glassKeep the index finger mostly straight while the middle finger is bent. The ring and pinky fingers, along with the thumb MCP, should be partly bent.
A rendered hand curls around a cylinder with the thumb near its top.
CylinderKeep the index, middle, ring and pinky PIP joints partly bent, and the thumb MCP partly bent too. The index PIP should be more bent than the pinky PIP.
A rendered hand curves around an eyeglass frame with the temple arms open.
EyeglassesUse a mostly straight pinky with the index, middle, and ring partly bent; keep the thumb MCP partly bent.
A rendered hand holds a flute horizontally, with its finger holes visible.
FluteKeep the middle bent and the ring and pinky partly bent. The index PIP and thumb MCP should be mostly straight.
A rendered hand curves around the upright fingers of a hand model.
Hand modelKeep the middle finger mostly straight and the thumb MCP partly bent. The index, ring, and pinky fingers should be partly bent, with the pinky more bent than the index and ring fingers.
A rendered hand wraps around a mug with the handle visible.
MugKeep the index and pinky mostly straight, with the middle and ring bent. Leave the thumb MCP partly bent.
A rendered hand curls around a toothbrush handle with the brush head exposed.
ToothbrushKeep the index, ring, and pinky bent at their middle knuckles, with the middle finger partly bent and the thumb MCP mostly straight.

These selected poses use known object classes and passed the finger-state and sampled geometry checks for this run. The checks allowed up to 3 mm of sampled penetration.

Geometric refinement

These poses use manually specified regions on seen objects. Change the object or camera to inspect each pose.

GEOMETRIC REFINEMENT A rendered hand pinches a small sphere, view C. VIEW C

Instruction

“Pinch the small sphere.”

The refinement targets opposing regions of the sphere with the thumb and index finger.

The refinement study evaluated 18 fixed candidates across six tasks. Four tasks passed all predefined geometric and task-specific checks. The elephant and banana refinement tasks failed.

04 Quantitative evaluation

Translation and realization

We compare text-only and text-plus-point-cloud control on 304 synthetic prompts with novel instruction chains across 19 objects held out from language-model fine-tuning.

Fraction of prompts satisfying all stated finger constraints
Evaluation stageText onlyText + PointNet
Predicted finger commands12.50%25.33%
Realized finger angles30.92%24.34%

Point-cloud conditioning improved command translation on this split, while strict satisfaction in the generated angles declined. Both variants used the same grasp generator, trained on all 50 objects in the benchmark. The evaluation used fixed single-seed adapters and measured finger constraints only.