The paper focuses on the problem of the notch filter feedback control in the perturbed planar Hamiltonian systems. By Melnikov's method, a suitable range of parameters in the notch filter controller can be obtained to convert chaotic motions into desired low-period motions. The averaging method is introduced to analyze the stability of the low-period orbits, the subharmonic orbits inside the homoclinic loop, in the control systems. As a typical example, the design procedure for controlling Duffing's oscillator and its stability analysis are derived in detail, and the thorough simulation results are presented to demonstrate the effectiveness of the theoretical analysis. Finally. the further examples show the applicability of the notch filter controller in a wide range of chaotic systems and hyper-chaotic systems.