Despite the recent rapid development of neural style transfer, existing style transfer methods are still somewhat inefficient or have a large model size, which limits their application on computational resource limited devices. The major problem lies in that they usually adopt a pre-trained VGG-19 backbone which is relatively large or the feature transformation module is computationally heavy. To address above problems, we propose a DIstillation based Style Transfer framework (called DIST) in conjunction with an efficient feature transformation module for arbitrary image and video style transfer. The distillation module can lead to a highly compressed backbone network, which is 15.95× smaller than the VGG-19 based backbone. The proposed feature transformation is capable of transforming the content features in an extremely efficient feed forward pass. For video style transfer, the above framework is further combined with a temporal consistency regularization loss. Extensive experiments show that the proposed method is superior over the state-of-the-art image and video style transfer methods, even with a much smaller model size. •A knowledge distillation method to compress VGG-19 based backbone is proposed.•A light weight feature transformation module for flexible style transfer is proposed.•A temporal consistency loss to maintain video style transfer stability is proposed.•A current smallest style transfer model is derived, only 2.67 MB.•The final model can perform style transfer at 167 FPS on a 2080Ti GPU.