Despite the success of data-driven converter fault diagnosis methods, interpretability and generalizability limit the further promotion of data-driven methods in industrial applications. Therefore, to improve the accuracy in face of out-of-distribution problems and increase the confidence of power converter fault diagnosis, it is essential to understand the change and decision mechanism inside the deep model. First, we construct a general temporal convolutional network (TCN) to visualize the diagnostic process, which has been proven effective in power converter fault diagnosis. Then, the effect of hyperparameters on generalizability is analyzed under typical power converter disturbances. Finally, the concern area of the model for the current in the fault decision is interpreted intuitively by gradient-weighted class activation mapping (Grad-CAM), and the feature maps generated by the different channels are analyzed from multiple perspectives. The visualization results help to understand the complex structure of neural networks and can support the design of model to improve generalizability.