After entering the cerebral cortex, sensory information spreads through six different horizontal neuronal layers that are interconnected by vertical axonal projections. It is believed that through these projections layers can influence each other's response to sensory stimuli, but the specific role that each layer has in cortical processing is still poorly understood. Here we show that layer six in the primary visual cortex of the mouse has a crucial role in controlling the gain of visually evoked activity in neurons of the upper layers without changing their tuning to orientation. This gain modulation results from the coordinated action of layer six intracortical projections to superficial layers and deep projections to the thalamus, with a substantial role of the intracortical circuit. This study establishes layer six as a major mediator of cortical gain modulation and suggests that it could be a node through which convergent inputs from several brain areas can regulate the earliest steps of cortical visual processing.