An accelerator based BNCT has been desired because of its therapeutic convenience. However, optimal design of a neutron moderator system is still one of the issues. Therefore, detailed studies on materials consisting of the moderator system are necessary to obtain the optimal condition. In this study, the epithermal neutron flux and the RBE dose have been calculated as the indicators to look for optimal materials for the filter and the moderator. As a result, it was found that a combination of MgF2 moderator with Fe filter gave best performance, and the moderator system gave a dose ratio greater than 3 and an epithermal neutron flux over 1.0×109cm−2s−1. •We performed design study based on RBE estimation for a BNCT moderator system driven by an accelerator.•A moderator system consisting of MgF2 combined with a Fe filter gives the best performance.•The proposed model gives an epithermal neutron flux of more than 1.0×109cm−2s−1 and a dose ratio of 3.07.