We have investigated specific lipid binding to the pore domain of potassium channels KcsA and chimeric KcsA-Kv1.3 on the structural and functional level using extensive coarse-grained and atomistic molecular dynamics simulations, solid-state NMR, and single channel measurements. We show that, while KcsA activity is critically modulated by the specific and cooperative binding of anionic nonannular lipids close to the channel's selectivity filter, the influence of nonannular lipid binding on KcsA-Kv1.3 is much reduced. The diminished impact of specific lipid binding on KcsA-Kv1.3 results from a point-mutation at the corresponding nonannular lipid binding site leading to a salt-bridge between adjacent KcsA-Kv1.3 subunits, which is conserved in many voltage-gated potassium channels and prevents strong nonannular lipid binding to the pore domain. Our findings elucidate how protein-lipid and protein-protein interactions modulate K+ channel activity. The combination of MD, NMR, and functional studies as shown here may help to dissect the structural and dynamical processes that are critical for the functioning of larger membrane proteins, including Kv channels in a membrane setting.