Human-Centric Grasp State Assessment: Toward Transferring Subjective Evaluation to Robots

Ryohei Kobayashi, Kosei Isomoto, Yuga Yano, Yuichiro Tanaka, Hakaru Tamukoh · N/A · 2026

Summary

Abstract Summary

We propose a framework that transfers tacit human subjective criteria to robotic systems for the appropriate grasping of deformable objects. Achieving such behavior is challenging because a semantic gap exists between qualitative human expectations and quantitative robotic measurements. Conventional deep learning approaches for bridging this gap also require prohibitive amounts of manually annotated data for each newly encountered object. To address these challenges, our framework integrates a Vision-Language Model (VLM)-based semi-automated supervisor generator with a lightweight grasp state predictor, using a minimal set of human-annotated trials as contextual anchors to propagate subjective criteria to unannotated data. The prediction model then enables rapid online adaptation by sequentially estimating the grasp state from time-series tactile and grasping force measurements. Through experiments on three representative deformable objects and a human evaluation study with 25 participants, we demonstrate the feasibility of the proposed framework for adjusting grasping force according to human-perceived grasp appropriateness in the evaluated task setting.

Key Points

We propose a framework that transfers tacit human subjective criteria to robotic systems for the appropriate grasping of deformable objects.
Achieving such behavior is challenging because a semantic gap exists between qualitative human expectations and quantitative robotic measurements.
Conventional deep learning approaches for bridging this gap also require prohibitive amounts of manually annotated data for each newly encountered object.
To address these challenges, our framework integrates a Vision-Language Model (VLM)-based semi-automated supervisor generator with a lightweight grasp state predictor, using a minimal set of human-annotated trials as contextual anchors to propagate subjective criteria to unannotated data.
The prediction model then enables rapid online adaptation by sequentially estimating the grasp state from time-series tactile and grasping force measurements.

PROPOSED FRAMEWORK

Figure 2 illustrates the proposed framework. To support object-wise adaptation with limited annotation, the framework integrates two main modules: Supervisor Generator (SG) that generates training data starting from a limited number of human subjective annotations.

Grasp State Predictor (GSP) that predicts one-step-ahead the grasp state of sliding, appropriate, and excessive from time-series tactile and grasping force data. Rather than replacing human annotation, the SG reduces labeling costs.

PROPOSED FRAMEWORK — Human-Centric Grasp State Assessment: Toward Transferring Subjective Evaluation to Robots — Figure 3: Relationship between the grasp state and timestamp labels.

Experiments

To verify the effectiveness of the proposed framework in bridging the semantic gap and achieving rapid adaptation, we conduct evaluations structured as follows: A. Dataset and Setup: Details of multimodal data collection and ground truth definitions for three distinct objects to establish the experimental baseline. B.

SG Reliability: Evaluation of label accuracy versus example count to demonstrate the mitigation of manual data collection bottlenecks. C.

Experiments — Human-Centric Grasp State Assessment: Toward Transferring Subjective Evaluation to Robots — Figure 4: Concatenated video of the hand camera view and the tactile heatmap.

Sources and Demonstrations

Figures are reproduced from Kobayashi et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.