Where is that object? The answer depends on how you've been looking.
Macaque inferotemporal (IT) cortex encodes object position in coordinates that shift with the observer's recent visual history. After prolonged viewing of motion in one direction, neurons represent the position of a subsequently presented object as displaced — consistent with the motion aftereffect illusion, where a stationary object appears to drift opposite to adapted motion.
This means the brain's spatial representation is not anchored to retinal coordinates or to absolute world coordinates but to a perceptually adapted frame that incorporates the history of visual stimulation. The position code is plastic: what the neurons report as “here” depends on what they were processing moments ago.
Every tested artificial vision network fails to do this.
Standard convolutional networks, vision transformers, and self-supervised models all encode position in fixed coordinate systems. When given the same adaptation stimuli followed by the same test objects, none produce the perceptual displacement observed in IT cortex. The representations are rigid where the biology is flexible.
The gap is specific: it's not that artificial networks lack position encoding — they encode position well, sometimes better than IT cortex on objective metrics. The failure is in the adaptation. The biological system updates its coordinate frame in response to recent experience. The artificial systems don't. They treat position as a property of the current input, not a function of the input's context.
The implication is that temporal context — the sequence of recent stimulation — is not a nuisance variable to be factored out but a constitutive element of the representation. The brain doesn't encode where things are; it encodes where things are relative to where attention has been. No current architecture replicates this, which means no current model captures how biological vision actually constructs spatial awareness.