It is of significant biological interest and medical importance to develop class- and isoform-selective histone deacetylase (HDAC) modulators. The impact of the linker component on HDAC inhibition specificity has been revealed but is not understood. Using Born-Oppenheimer ab initio QM/MM MD simulations, a state-of-the-art approach to simulating metallo-enzymes, we have found that the hydroxamic acid remains to be protonated upon its binding to HDAC8, and thus disproved the mechanistic hypothesis that the distinct zinc hydroxamate chelation modes between two HDAC subclasses come from different protonation states of the hydroxamic acid. Instead, our simulations suggest a novel mechanism in which the chelation mode of hydroxamate with the zinc ion in HDACs is modulated by water access to the linker binding channel. This new insight into the interplay between the linker binding and the zinc chelation emphasizes its importance and gives guidance regarding linker design for the development of new class-IIa-specific HDAC inhibitors.