The present work deals with the preparation of reduced graphene oxide-Fe3O4 (rGO-Fe3O4) nanocomposite and its nanofluid by an ultrasound assisted method for convective heat transfer study. Formation of the rGO-Fe3O4 nanocomposite with a uniform distribution of smaller sized (10-20 nm) Fe3O4 nanoparticles on graphene nanosheets was confirmed from UV/vis, transmission electron microscopy, Raman, X-ray diffraction, and X-ray photoelectron spectroscopy analysis. Thermal conductivity of prepared rGO-Fe3O4 nano composite based nanofluids with the aid of ultrasound showed an 83.44% enhancement for 0.2 vol % concentration of the rGO-Fe3O4 nanocomposite at 40 degrees C. Rheological study revealed non-Newtonian behavior of the nanofluids. Various viscosity models were used to predict the behavior of rGO-Fe3O4 nanocomposite based nanofluids. The estimated heat transfer coefficient with the use of 0.02 vol % rGO-Fe3O4 nanofluid at the exit of the test section was 4289.5 W/m(2).K for the Reynolds number equal to 7510 +/- 5. A new correlation for the estimation of Nusselt number has been proposed for the rGO-Fe3O4 nanofluid which fits the experimental data very well.