A Survey of 200 Papers Systematizes Affordance Learning in Embodied AI

Researchers have gathered nearly 200 papers on affordance learning in embodied intelligence into a single survey, Jiqizhixin reports. The topic sits in the gap between the phrase "the model sees a cup" and the action "the manipulator grabs it by the handle." An affordance answers three questions: what in a scene can be touched, where to do it, and how.

The gap between scene understanding and action

Recent years have given robotics several powerful tools at once: vision-language-action models (VLA), world models, multimodal foundation models, and claims of general-purpose robots. Machines already parse complex instructions and generate continuous trajectories across a growing number of tasks.

But recognizing an object does not tell a robot which part to grab. A pot's handle and its lid belong to the same "pot" class, yet the actions they call for are different. A knife has a safe side and a cutting side. Judging by the introduction, the survey's authors see this as a key unsolved problem: a robot needs not only to know what is in front of it, but also to understand what it can use in practice, where to apply force, and how.

What an affordance means in practice

The psychologist James Gibson introduced the term in 1979: an affordance is an opportunity for action that the environment offers a particular agent. In robotics, it is turned into a representation that can be trained and fed into a planner or policy.

Below are the typical forms such a representation takes in the field. This is a general picture of the area, not a retelling of the survey's taxonomy, which is not in the available excerpt.

• Object level: which objects are suitable for an action (a kettle for pouring, a door for opening).
• Spatial level: a point, mask, or heatmap on an image, or a 3D region in a point cloud.
• Interaction level: grasp poses, force direction, motion trajectory.
• Semantic level: linking a language command to a part of an object ("open the drawer" leads to the handle).
• Sources of training data: manual annotation, videos of human actions, simulation, prompts from VLMs and LLMs.
• Scale of the survey: about 200 papers.

Why the topic has moved to the foreground

Affordances appeared in robotics long before the current boom. In SayCan (2022), a language model proposed steps, while skill feasibility estimates filtered out what the robot could not do in its current environment. End-to-end VLA models such as RT-2, OpenVLA, and π0 take a different route: they learn to output actions directly from an image and text.

An explicit affordance as an intermediate layer has clear arguments in its favor. It can be visualized and checked. Many in the field believe it helps transfer skills between different objects and robot designs. And it gives a point to which geometry and physics can be attached. Whether that is enough against end-to-end learning on large volumes of demonstrations is an open question. A good survey is useful precisely for seeing where approaches converge and where they compete.

How to use the survey

The available excerpt contains only the problem statement, so it is too early to judge the classification, benchmarks, or the authors' conclusions. You need to read the original.

Teams building manipulation systems should check three things:

• Which affordance representation fits your task: a point and mask for tabletop manipulation, or a pose and trajectory for delicate operations.
• How quality is measured. Region localization accuracy and actual task success on a robot give different pictures, and choosing by a single metric is risky.
• Where the data comes from. Human videos scale cheaply, but transfer to a specific robot hand and the sim-to-real gap will need to be tested separately.

For those just entering the field, a survey like this saves weeks of searching: it shows which tasks are already solved and where embodied AI still has gaps between perception and action.