A vision-language model can describe a photograph of Tokyo in detail — the buildings, the street layout, the neon signs, the crowd density. Give it a map of Tokyo and it stumbles. It misreads labels. It cannot estimate distances from a scale bar. It fails at route reasoning that a first-year geography student would handle without thought.
Ung and colleagues built CartoMapQA, a benchmark of over 2,000 cartographic question-answering samples, and tested both open-source and proprietary vision-language models. The tasks ranged from basic symbol recognition (what does this icon represent?) to spatial reasoning (which route is shorter?) to scale interpretation (how far apart are these cities?). Current models struggle across all categories, with particularly poor performance on tasks requiring integration of visual symbols with spatial semantics.
The failure is instructive because it reveals a gap between two kinds of visual understanding. Photographs are dense with pixel-level information — textures, lighting, spatial depth cues — that models have learned to parse from billions of training images. Maps are sparse by design. A map deliberately discards most visual information and replaces it with a symbolic vocabulary: lines mean roads, colors mean elevation, a scale bar means “this distance on paper corresponds to that distance on earth.” Reading a map requires understanding the encoding, not the pixels.
This is the difference between seeing and reading. Vision-language models are trained primarily on photographs and their captions — images where what you see is roughly what there is. Maps are the opposite: what you see is a compression of what there is, and the decompression rules are conventional, not perceptual. The OCR errors that plague the models on map labels are symptomatic: the text on a map is not decoration but data, and misreading a label does not just miss a word — it misidentifies a city, a river, a boundary.
The through-claim is about a class of visual artifacts that are designed to be interpreted rather than perceived. Maps, circuit diagrams, sheet music, architectural plans — all share this property. They use visual space to represent non-visual structure through conventions that must be learned, not induced from pixels. A model that has seen a million photographs has learned what the world looks like. It has not learned what the world's representations of itself look like. The illegibility of maps to vision models is not a bug to be patched with more training data. It marks the boundary between perception and literacy.