•Plasma-enhanced chemical synthesis of molybdenum selenide nanoflake is demonstrated.•Inexpensive precursors in the low-temperature reactive plasma environment were used.•Self-organized growth mechanizm of molybdenum selenide nanoflakes is proposed. The role of reactive environment and hydrogen specifically in growth and structure of molybdenum selenide (MoSe2) nanomaterials is presently debated, and it is not clear whether hydrogen can promote the growth of MoSe2 sheets and alter their electronic properties. To find efficient, convenient methods for controlling the nucleation, growth and resultant properties of MoSe2 nanomaterials, MoSe2 nanoflakes were synthesized on silicon substrates by hot filament chemical vapor deposition using molybdenum trioxide and selenium powders in pure hydrogen, nitrogen gases and hydrogen-nitrogen mixtures. The structures and composition of synthesized MoSe2 nanoflakes were studied using the advanced characterization instruments including field emission scanning electron microscopy, micro-Raman spectroscopy, X-ray photoelectron spectroscopy, transmission electron microscopy and energy dispersive X-ray spectrometry. The analysis of the growth process indicates that hydrogen can improve the formation of MoSe2 nanoflakes and significantly alter their properties due to the high reduction capacity of hydrogen and the creation of more nucleation centers of MoSe2 nanoflakes on the silicon surface. The study of photoluminescent (PL) properties reveals that the MoSe2 nanoflakes can generate a strong PL band at about 631nm, differently from the plain MoSe2 nanoflakes. The major difference in the PL properties may be related to the edges of MoSe2 nanoflakes. These results can be used to control the growth and structure of MoSe2-based nanomaterials and contribute to the development of advanced MoSe2-based optoelectronic devices.