Display omitted •Graphene-Fe3O4 nanoparticles coated by oleic acid were synthesized.•Ultra-stable kerosene based nanofluid was produced from Graphene-Fe3O4 nanohybrid.•Rheological and thermal properties of nanofluids were investigated.•Significant enhancement in thermal characteristics of nanofluid was observed. This research deals with a novel synthesis method for preparation of Fe3O4 decorated Graphene and its application as a kerosene-based nanofluid with the purpose of heat transfer enhancement. In order to stabilize the Graphene-Fe3O4 nanoparticles, oleic acid was grafted on the surface of nanoparticles by chemisorption. Synthesized nanohybrid was characterized by X-ray powder diffraction (XRD), Fourier transform infrared spectroscopy (FTIR), Transmission electron microscopy (TEM), Dynamic light scattering (DLS) and UV–Vis spectrophotometer. The nanofluid remains stable for more than five months without any sedimentation. Moreover, rheological and thermal properties of nanofluids were measured. Viscosity increment, especially in low concentrations of nanoparticles was negligible enough for industrial applications and density enhancement is completely negligible even in high concentrations. The thermal conductivity was improved 31% at 50°C, compared to the base fluid. In order to examine convective heat transfer enhancement, an experimental setup was designed and measurements were examined in Reynolds numbers between 2166 and 4553. The maximum heat transfer enhancement was 66% at Reynolds number of 4553 and 0.3wt.% of nanoparticle. Increase in particles loading and Reynolds number causes convective heat transfer coefficient to improve.