A novel organic rectorite (OREC) was prepared by treating the natural sodium-rectorite (Na-REC) with ionic liquid 1-hexadecyl-3-methylimidazolium bromide ([C16mim]Br). X-ray diffraction (XRD) analysis showed that the interlayer spacing of the OREC was expanded from 2.23nm to 3.14nm. Furthermore, two types of OREC/epoxy nanocomposites were prepared by using epoxy resin (EP) as matrix, 2-ethyl-4-methylimidazole (2-E-4-MI) and tung oil anhydride (TOA) as curing agents, respectively. XRD and transmission electron microscope (TEM) analysis showed that the intercalated nanocomposite was obtained with addition of the curing agent 2-E-4-MI, and the exfoliated nanocomposite was obtained with addition of the curing agent TOA when the OREC content was less than 2 wt %. For the exfoliated nanocomposite, the mechanical and thermal property tests indicated that it had the highest improvement when OREC content was 2 wt% in EP. Compared to pure EP, 60.3% improvement in tensile strength, 26.7% improvement in bending strength, 34% improvement in bending modulus, 14 degrees C improvement in thermal decomposition temperature (Td) and 5.7 degrees C improvement in glass transition temperature (Tg) were achieved. (c) 2012 Wiley Periodicals, Inc. J Appl Polym Sci, 2012