As a branch of dictionary learning (DL), analysis dictionary learning has been widely used for pattern classification, which achieves outstanding performance. However, it is still a challenging to learn a more compact and discriminative analysis dictionary to ensure that the coding coefficient matrix of training samples presents a more discriminative block diagonal structure. To address this issue, we propose a self-eliminating discriminant analysis dictionary learning (SeDADL) method to learn a discriminant analysis dictionary that makes the coding coefficient matrix have an approximate block diagonal structure. Specifically, we first design a novel analysis dictionary regularization term to improve the discrimination capability of analysis dictionary by eliminating repeated and linearly dependent atoms in the analysis dictionary while preventing the generation of trivial solutions. Then, we design a self-eliminating coding coefficient constraint term to enhance the discrimination capability of spare codes by forcing the coding coefficient matrix to achieve an approximate block diagonal structure. In order to further improve the classification efficiency of SeDADL model, we introduce a linear classification error term into SeDADL model to learn a linear classifier, which constructs the links between spare codes and class labels. Moreover, an efficient iterative algorithm is designed to solve the optimization problem of SeDADL. Extensive experimental results on six datasets demonstrate that SeDADL can achieve satisfactory classification performance compared with some state-of-the-art methods.