In this research, oil-based Fe3O4 nanoparticles were prepared by means of coprecipitation method followed by a surface modification using lauric acid. Oil-based Fe3O4 could disperse in styrene, and polystyrene/Fe3O4 (PS/Fe3O4) composite particles were prepared via miniemulsion polymerization in the presence of potassium peroxide (KPS) as an initiator, sodium dodecyl sulphate as a surfactant, hexadecane or sorbitan monolaurate(Span 20) as a costabilizer. The effects of Fe3O4 content, homogenization energy, amount of initiator, amount of surfactant and costabilizer on the conversion, size distributions of droplets and latex particles, nucleation mechanism and morphology of composite latex particles were investigated. The results showed that different nucleation mechanisms dominated during the course of reaction when polymerization conditions changed. The most important two key factors to influence the nucleation mechanism were homogenization energy and initiator. High homogenization energy provided critically stabilized size of droplets. Otherwise, secondary nucleation, including micellar and/or homogeneous nucleation, would take place rather than droplet nucleation when a water-soluble initiator, KPS, was used. It resulted in two populations of latex particles, pure PS particles in smaller size and PS/Fe3O4 composite particles in larger size. (C) 2007 Wiley Periodicals, Inc.