•We extend the successful RPN+BF framework to combine handcrafted features and CNN features for pedestrian detection.•RoI-pooling is used to extract features from handcrafted and CNN channels, and with larger output resolution for the former.•We explore several handcrafted channels such as HOG+LUV, Checkerboards, and RotatedFilters in our framework.•Experimental results on Caltech pedestrian dataset demonstrate the effectiveness of the proposed framework.•When using a more advanced RPN, our approach can be further improved and get competitive results on the benchmarks. Pedestrian detection has achieved great improvements with the help of Convolutional Neural Networks (CNNs). CNN can learn high-level features from input images, but the insufficient spatial resolution of CNN feature channels (feature maps) may cause a loss of information, which is harmful especially to small instances. In this paper, we propose a new pedestrian detection framework, which extends the successful RPN+BF framework to combine handcrafted features and CNN features. RoI-pooling is used to extract features from both handcrafted channels (e.g. HOG+LUV, CheckerBoards or RotatedFilters) and CNN channels. Since handcrafted channels always have higher spatial resolution than CNN channels, we apply RoI-pooling with larger output resolution to handcrafted channels to keep more detailed information. Our ablation experiments show that the developed handcrafted features can reach better detection accuracy than the CNN features extracted from the VGG-16 net, and a performance gain can be achieved by combining them. Experimental results on Caltech pedestrian dataset with the original annotations and the improved annotations demonstrate the effectiveness of the proposed approach. When using a more advanced RPN in our framework, our approach can be further improved and get competitive results on both benchmarks.