We have been investigating applications of a topology optimisation method with the level set method. In this study, to further enhance the applicability of the method, we investigate a topology optimisation method for three-dimensional scalar wave scattering problems which can be defined in an unbounded domain. To this end, the fast multipole boundary element method (FMBEM), which can deal with the unbounded domain accurately and efficiently, is implemented in the proposed optimisation method. A detail of the algorithm of the topology optimisation with the level set method and the FMBEM is presented. Also, a rigorous derivation of the topological derivative, which characterises the sensitivity of the objective function when an infinitely small spherical object appears, using spherical functions is presented. After validating the topological derivatives with approximated ones, we show the efficiency of the proposed optimisation method with a numerical benchmark. Through these numerical experiments, we conclude that the proposed topological optimisation with the level set method and the FMBEM can be applied to scattering problems in acoustics.