The primary visual cortex contains a detailed map of the visual scene, which is represented according to multiple stimulus dimensions including spatial location, ocular dominance and stimulus orientation. The maps for spatial location and ocular dominance arise from the spatial arrangement of thalamic afferent axons in the cortex. However, the origins of the other maps remain unclear. Here we show that the cortical maps for orientation, direction and retinal disparity in the cat (Felis catus) are all strongly related to the organization of the map for spatial location of light (ON) and dark (OFF) stimuli, an organization that we show is OFF-dominated, OFF-centric and runs orthogonal to ocular dominance columns. Because this ON-OFF organization originates from the clustering of ON and OFF thalamic afferents in the visual cortex, we conclude that all main features of visual cortical topography, including orientation, direction and retinal disparity, follow a common organizing principle that arranges thalamic axons with similar retinotopy and ON-OFF polarity in neighbouring cortical regions.