The purpose of this research was to achieve the best performance, combustion and emission characteristics of a marine engine fueled with hydrogen (5%, 10% and 15% energy fraction), water (2, 4 and 6 wt%) and rapeseed methyl ester (RME) blend fuel by the multi-objective optimization. Based on the experimental results of the engine at 50%, 75% and 100% loads, the computational fluid dynamics (CFD) model was developed, and an improved chemical kinetic mechanism was developed to simulate the fuel combustion process. The response surface methodology (RSM) was used to optimize the combustion and emission characteristics of the marine diesel engine. The result showed that the hydrogen and water can improve the combustion and emission characteristics of the marine engine. The best solution was to select the desirable value of 0.632, and the engine achieves the best state by fueling R + 15H + 2.5W at 74.69% load. At the point, the optimum values of brake specific fuel consumption (BSFC), brake thermal efficiency (BTE), nitrogen oxide (NOx), hydrocarbon (HC) and carbon monoxide (CO) emissions were 208.31 g/kW·h, 39.22%, 941.21 ppm, 325.86 ppm and 1073.4 ppm, respectively. For this study, adding appropriate hydrogen and water to biodiesel can achieve efficient, low carbon and clean combustion in marine engine. •Effects of hydrogen/biodiesel/water blend fuels on combustion and emission characteristics are compared.•Optimum blend fuel is obtained for diesel engine.•An improved 3D model and chemical kinetics mechanism are developed.•The performance optimization was carried out by the response surface methodology.