Density functional calculations at the B3LYP/6-31+G(d,p) level are used to study the mechanism of 4a-hydroperoxyflavin oxidation of a series of heteroatom nucleophiles. The oxidation of xenobiotics catalyzed by flavin-containing monooxygenases (FMOs) is modeled by the oxidation of N-, S-, P-, and Se-containing nucleophiles. A mechanism for distinguishing oxygen atom versus hydroxyl transfer from RO-OH to (CH3)(3)N, (CH3)(2)S, (CH3)(3)P, and (CH3)(2)Se is presented. The nature of these oxidative processes is related to the magnitude of the single imaginary frequency for the transition state for oxygen atom transfer. Classical activation barriers for oxygen atom transfer from a tricyclic isoalloxazine C-4a-hydroperoxide 3 (FlHOOH) to dimethyl sulfide, trimethylamine, trimethylphosphine, and dimethylselenide suggest that the reactivity of this biologically important oxidizing agent is intermediate between that of tert-butyl hydroperoxide and a peracid. The gas-phase reactivity of FlHOOH toward these four nucleophiles is estimated to be 10(7)-10(12) greater than that of t-BuO-OH but 10(2)-10(6) less than that of peroxyformic acid. The intrinsic gas-phase barriers for the oxidation of (CH3)(3)N and (CH3)(2)S with 3 (FlHOOH) differ by only I kcal/mol. The experimentally observed 10(6) rate difference for amine oxidation with protein-bound FIHOOH versus synthetic flavinhydroperoxide (FlEtOOH) in solution is attributed simply to solvent effects; no special activation of the amine by the local enzymatic environment is required for this xenobiotic oxidation. A comparison of estimated O-O bond dissociation energies for CH3O-OH and CH3O-OCH3, at the AMI, Hartree-Fock, B3LYP, CBS-Q, and G3 levels of theory is presented. The performance of AMI, Hartree-Fock, versus CASSCF and QCISD(T) computational methods for the O-O bond elongation in CH3O-OH and CH3O-OCH3 is discussed. Our results provide an estimate of 40 kcal/mol for the O-O bond energy in 4a-flavinhydroperoxide (FIHOOH).