Resolving spatial ambiguities in object-centric representations

In brief

• Single-view object-centric learning struggles when objects are partially occluded or seen from ambiguous angles.
• A partially visible L-shaped block could be a cube behind a wall or a genuinely hollow shape.
• View-Invariant Slot Attention (VISA) aggregates multiple views to capture identifiable, invariant object content while isolating viewpoint parameters without camera annotations.

This retrospective explainer is dated to the June 2025 ICML conference. It discusses Identifiable Object Representations under Spatial Ambiguities by Avinash Kori, Francesca Toni and Ben Glocker.

Imagine peering through a fence and seeing what appears to be an orange rectangle resting beside a blue square. Is the orange shape actually a simple rectangle, or is it an L-shaped block whose bottom leg is hidden behind the blue square? And if you tilt your head or take two steps to the side, does your brain create an entirely new mental entity, or does it update its spatial estimate of the same underlying 3D object?

Human vision resolves these spatial ambiguities seamlessly. We understand that 2D silhouettes are transient projections of persistent 3D entities, and we use multi-view consistency to mentally “un-occlude” hidden geometry.

In unsupervised object-centric learning, however, single-view models (such as standard Slot Attention) have no mechanism to distinguish between genuine object geometry and viewpoint-induced occlusion. They frequently split occluded objects into disconnected fragments or assign identical objects completely different representations when viewed from different angles.

View-Invariant Slot Attention (VISA) solves this challenge by formulating a multi-view probabilistic framework that guarantees the theoretical identifiability of 3D object representations without requiring any camera pose annotations.

The nature of spatial ambiguities

Spatial ambiguity in visual perception arises from two distinct physical sources:

1. Occlusion Ambiguity: In panel (a) of the figure below, the visible pixels of the orange block could be produced by a vertical bar, a T-shape, or an L-shape hiding behind the blue cube. A single 2D image cannot uniquely identify the true object shape.
2. Viewpoint Ambiguity: In panel (b), rotating a 3D blue cube by 45 degrees transforms its 2D projection from a flat square into an isometric diamond with visible top and side facets. A model that lacks 3D awareness will treat these two projections as two completely different objects.

Figure from the paper. (a) Occlusion ambiguity: identical visible 2D pixels can belong to several distinct 3D object geometries hidden behind another object; (b) Viewpoint ambiguity: changing camera angle transforms object silhouettes and reveals new surfaces. Source: Avinash Kori, Francesca Toni and Ben Glocker. Extracted from the source manuscript; click to enlarge.

Disentangling content from perspective

To build identifiable representations under these conditions, VISA decomposes the latent representation of each object $k$ across views $v \in {1, \dots, V}$ into two disentangled components:

\[z_{k, v} = \left(\mathbf{c}_k, \boldsymbol{\theta}_v\right),\]

where:

• $\mathbf{c}_k \in \mathbb{R}^{D_c}$ is the invariant content vector representing intrinsic physical properties (3D shape, material, albedo, and semantic class). Crucially, $\mathbf{c}_k$ is invariant across all camera viewpoints.
• $\boldsymbol{\theta}v \in \mathbb{R}^{D\theta}$ is the extrinsic viewpoint vector representing camera angle, distance, and perspective transformations for view $v$.

Instead of treating each view independently, VISA uses a cross-view attention mechanism that aggregates view-specific slot queries into a shared canonical belief:

\[\mathbf{c}_k = \sum_{v=1}^V \alpha_{k, v} \, f_{\text{enc}}\left(s_{k, v}\right), \quad \alpha_{k, v} = \frac{\exp\left(w^T s_{k, v}\right)}{\sum_{u=1}^V \exp\left(w^T s_{k, u}\right)}.\]

The decoder then takes the canonical content vector $\mathbf{c}_k$, conditions it on the specific view parameters $\boldsymbol{\theta}_v$, and renders the 2D projection for that camera angle:

\[\hat{x}_v = \sum_{k=1}^K m_{k, v} \odot g_{\text{dec}}\left(\mathbf{c}_k, \boldsymbol{\theta}_v\right),\]

where $m_{k, v}$ is the rendered 2D alpha mask for object $k$ in view $v$.

Identifiability theorems without camera supervision

Most 3D reconstruction models require known camera matrices (extrinsic pose $[R \mid t]$) during training. VISA requires no camera labels.

The paper establishes a non-linear Independent Component Analysis (ICA) identifiability proof showing that:

1. When observing $V \ge 2$ views of a scene, the joint distribution $p(x_1, \dots, x_V)$ uniquely constrains the content space $\mathbf{c}_k$ up to coordinate permutation.
2. The intrinsic geometry $\mathbf{c}_k$ is mathematically separated from the extrinsic pose $\boldsymbol{\theta}_v$ because viewpoint changes act as a group transformation over spatial coordinates while preserving intrinsic topological features.

On challenging multi-view benchmarks featuring heavy occlusions, VISA achieves superior segmentation accuracy (Adjusted Rand Index > 0.92) and correctly reconstructs hidden object backsides that single-view Slot Attention fails to discover.

Paper and sources

• Avinash Kori, Francesca Toni and Ben Glocker. Identifiable Object Representations under Spatial Ambiguities. ICML 2025 / arXiv:2506.07806.