•Flow boiling heat transfer of MgO/therminol 66 was investigated.•Fouling thermal resistance of nanoparticles was modeled.•A new model for flow boiling heat transfer of nanofluid was developed.•The boiling surface was a copper made disc. An experimental investigation was performed on the flow boiling heat transfer characteristics of MgO/therminol 66 nanofluid as a potential coolant on a copper-made disc. Nanofluids were prepared using two step method at wt.% = 0.1, and wt.% = 0.3. Results showed that the presence of MgO/therminol 66 increases the flow boiling heat transfer coefficient in comparison with the base fluid. However, with an increase in the mass concentration of nanoparticles, the heat transfer coefficient decreased. Results also revealed that bubble formation induces a pressure drop within the test section. Heat flux had no influence on the pressure drop, while an increase in the fluid flow rate caused an increase in the pressure drop. It was also found that the heat transfer coefficient decreased with operating time due to the presence of nanoparticles on the boiling surface resulting in the creation of thermal resistance on the surface. Also, an asymptotic behavior for the fouling thermal resistance over the time was registered. Two correlations were re-developed to accurately predict the heat transfer coefficient and fouling thermal resistance of the system. Experiments proved that MgO nanoparticles offer a potential to be used in thermal engineering systems with boiling heat transfer mechanism. The maximum enhancement for the heat transfer coefficient was 23.7% at wt.% = 0.1. For wt.% = 0.2 and wt.% = 0.3, the maximum enhancement of 16.2% and 13.3%, were achieved, respectively.