Lavie’s theory of attention proposes that the processing load in a relevant task determines the extent to which irrelevant distracters are processed. This theory was tested by asking participants in a study to perform linguistic tasks of low or high load while ignoring irrelevant visual motion in the periphery of the display. Although task and distracter were unrelated, both functional imaging of motion-related activity in cortical area V5 and psychophysical measures of the motion aftereffect showed reduced motion processing during high load in the linguistic task. These findings fulfill the prediction that perception of irrelevant distracters depends on the relevant processing load.