We developed highly aligned graphene nanosheets (GNSs) in epoxy composites with incorporating magnetic GNS-Fe3O4 hybrids under a magnetic field with the aim to take full advantage of the high inplane thermal conductivity of graphene. GNS-Fe3O4 hybrids were fabricated by a simple coprecipitation method, and their morphology, chemistry, and structure were characterized. GNS-Fe3O4 hybrids were found to be homogenously dispersed and well aligned through the direction of the magnetic field in the epoxy matrix, as confirmed by SEM observation and Raman spectra analysis. The resulting epoxy/GNS-Fe3O4 composites possessed high thermal conductivity in a parallel magnetic-alignment direction at low GNS-Fe3O4 loadings, which greatly outperformed the composites with randomly dispersed bare GNSs. The obtained results indicated that the magnetic alignment of magnetic-functionalized GNSs is an effective way for greatly improving the thermal conductivity of the graphene-based composites.