Six kinds of 1,3,4-oxadiazole-functionalized fibers were prepared to evaluate the adsorption capacity for Hg2+. Finally, a functionalized fiber (PAN(p-PMOD)F) with the best adsorption performance was selected for further research. The PAN(p-)(PMOD)F fiber was characterized by infrared (IR), elemental analysis (EA), scanning electron microscopy (SEM), X-ray diffraction (XRD), and X-ray photoelectron spectroscopy (XPS). The PAN(p-PMOD)F shows excellent selectivity for mercury ions in a mixed heavy metal ion solution (Hg-2(+), Ni2+, Pb2+, Zn2+, Ag+, Co2+, Cr3+, Cu2+, and Cd2+), and it can reduce the concentration of Hg2+ to 44 ppb in the actual sewage, which is of great practical value for the treatment of mercury pollution. Furthermore, the adsorbed Hg2+ can be desorbed completely by dilute HNO3 and the PAN(pPMOD)F can be reused many times with excellent recyclability.