A novel nanocomposite, named as ARWZ, was fabricated using zero-valent iron (ZVI) nanoparticles supported by acid-modified waste rock wool (ARW). ARW displayed a porous rod-like structure and could load plenty of ZVI nanoparticles. Meanwhile, ARW could effectively inhibit the aggregation and oxidation of ZVI nanoparticles because of steric-hindrance effect. ARWZ exhibited a large specific surface area and high removal capacity (197.69 mg/g) in 30 min for hexavalent chromium (Cr(VI)) from water and soil through adsorption and reduction, and the resulting ARWZ-Cr could be easily collected under a magnetic field because of the superior magnetic property. Importantly, when ARWZ was loaded in filter cotton with a micro/nano networks structure, the resulting system could be conveniently used as an excellent filter layer to control the migration of Cr(VI). This work provides a low cost and promising approach to remove Cr(VI), which also promotes the reuse of waste rock wool.