Towards human-like visual reasoning: an object-centric foundation

In brief

• Human perception is fundamentally object-centric: we parse scenes into persistent physical components that form the building blocks of causal reasoning.
• Unsupervised object representation models often suffer from unidentifiability, learning brittle correlations that collapse under distribution shift.
• This doctoral thesis establishes a unifying theoretical foundation for identifiable object-centric learning, introducing CoSA, PSA, and VISA.

This overview reflects the doctoral dissertation defended in May 2025 at Imperial College London: Towards human-like visual reasoning: theory and applications of object-centric learning by Avinash Kori, supervised by Ben Glocker and Francesca Toni.

When an infant observes a playroom, they do not perceive an unstructured grid of RGB pixels. They perceive a red ball, a wooden block, and a teddy bear. When the ball rolls behind the block, the infant understands that the ball continues to exist (object permanence), that its spherical shape has not changed, and that pushing the block will alter its position without modifying the teddy bear.

This ability—objectness—is the cornerstone of higher-level human cognition, counterfactual reasoning, and systematic generalization.

Yet despite dramatic advances in deep vision networks, modern foundation models frequently treat images as monolithic token sequences. When tested on compositional reasoning or out-of-distribution shifts, they struggle precisely because they lack stable, identifiable object representations that reflect real-world causal mechanisms.

This thesis tackles that challenge head-on, investigating both the mathematical theory and practical architectures required to build artificial visual reasoning systems that mirror human perception.

Why identifiability is non-negotiable

In representation learning, a model is identifiable if there is a unique mapping from data observations to the underlying latent factors (up to trivial ambiguities like permutation and coordinate scaling).

Without identifiability guarantees:

1. Shortcut learning: A model may achieve near-zero reconstruction loss by finding arbitrary entangled coordinates that fail completely on unseen test scenes.
2. Confounding and selection bias: Objects become entangled with lighting conditions, camera viewpoints, and background textures.
3. Symbolic incompatibility: Downstream symbolic reasoning modules (such as logical argumentation or graph neural networks) cannot assign stable semantic meanings to latent slot vectors.

The core thesis posits that for object-centric learning to enable true causal reasoning, the learned representations must be mathematically identifiable.

The tripartite framework: CoSA, PSA, and VISA

To systematically achieve identifiable object-centric representations across increasingly challenging environments, the dissertation introduces three complementary formalisms:

1. CoSA (Conditional Slot Attention)

Grounded Object-Centric Learning (ICLR 2024)
Standard Slot Attention samples slots from an uninformative isotropic Gaussian, meaning slots lack a stable identity across images. CoSA introduces a Grounded Slot Dictionary learned directly from data. Slots are initialized from category-specific distributions, anchoring representations to reusable physical concepts and providing a common vocabulary across scenes.

2. PSA (Probabilistic Slot Attention)

Identifiable Object-Centric Representation Learning (NeurIPS 2024)
PSA provides the rigorous generative foundation. By formulating object discovery as inference within a hierarchical Gaussian Mixture Model (local image GMMs aggregating into an aggregate global GMM), PSA establishes formal non-linear ICA theorems proving that latent object factors are identifiable up to permutation.

3. VISA (View-Invariant Slot Attention)

Identifiable Object Representations under Spatial Ambiguities (ICML 2025)
Real-world vision operates under severe spatial ambiguities: partial occlusions, perspective distortion, and changing viewpoints. VISA disentangles view-invariant intrinsic object properties (shape, material, class) from view-dependent extrinsic spatial transformations (pose, camera angle) using multi-view probabilistic aggregation without camera pose supervision.

Bridging object representations and causality

Beyond visual decomposition, the thesis bridges object-centric learning with Pearl’s structural causal framework:

\[\text{Raw Pixels } \mathbf{x} \xrightarrow[\text{PSA / VISA}]{\text{Identifiable Binding}} \text{Object Slots } \{O_1, \dots, O_K\} \xrightarrow[\text{Causal Discovery}]{\text{SCM Mechanisms}} \text{Causal Scene Graph } \mathcal{G}.\]

By treating object slots as autonomous nodes within a Structural Causal Model governed by Independent Causal Mechanisms (ICM), artificial agents can perform counterfactual simulation: predicting what a scene would look like if an object were moved, removed, or recolored without corrupting the rest of the visual world.

This synthesis establishes that object-centric learning is not merely an alternative segmentation technique, but a fundamental prerequisite for trustworthy, explainable, and human-aligned visual intelligence.

Paper and sources

• Avinash Kori (Supervisors: Ben Glocker and Francesca Toni). Towards human-like visual reasoning: theory and applications of object-centric learning. PhD Dissertation, Imperial College London (2025).