Forward osmosis (FO) polymer membrane filtration is an effective approach for the reclamation of oil sands process water (OSPW). Naphthenic acid (NAs), the major organic component in OSPW, can severely foul the FO membrane during wastewater treatment. Although much effort has been devoted to reducing the fouling of FO membrane, the fundamental understanding of fouling mechanism of NAs on membrane still remains incomplete. In this work, for the first time, the interaction forces between common FO membranes and major functional groups in NAs in aqueous solutions were directly measured using a surface forces apparatus (SFA) at nanoscale. The experimental force results could be well described by the classical Derjaguin-Landau-Verwey-Overbeek (DLVO) theory, indicating that the electrical double layer (EDL) repulsion plays a dominant role in inhibiting membrane fouling, and the adhesion leading to the membrane fouling is mainly driven by hydrophobic interaction. Addition of Ca2+ was found to induce the attractive bridging effect between the membranes and -COOH groups, thus aggravating the membrane fouling. Moreover, macroscale adsorption tests and OSPW fouling experiment results demonstrate that both surface potential and hydrophobicity significantly affect the membrane fouling/anti-fouling performance, in agreement with the nanoscale surface force results.