In this paper, we propose a novel people counting algorithm exploiting convolutional neural network (CNN) using a low radiation impulse radio ultra-wide bandwidth (IR-UWB) radar. Because of the ever-changing signals caused by the various cases of human motion scales, superposition and obstruction of signals as well as the attenuate of signal's strength along the distance and the angle, it is not easy to handle the people counting task by directly detecting targets for each range bin. Thus, we hope to excavate the information of targets' patterns, including their densities and forms of patterns' distributions in the detecting region to execute the counting task. To achieve this, the multi-scale range-time maps are extracted from the received data and further used to classify the number of people using the CNN. Finally, the experiments are conducted to show the priority of the proposed algorithm.