Speaker verification models have achieved good results on the single genre data. But the performance degrades when model training and testing are not in the same domain. The adversarial training method is proposed to solve this problem by minimizing domain distribution differences. However, the adversarial training ignores domain‐specific information for the domain‐invariant speaker representations. In this paper, an improved collaborative adversarial network for domain adaptation in speaker verification is performed. Compared to the adversarial training, a collaborative discriminator is newly incorporated that learns domain‐specific information at the lower layers. Further, the projection block is added to the collaborative discriminator. It reduces the noise introduced by the collaborative discriminator. Experiments are conducted in different mismatch scenarios and using different speaker encoders. All the experimental results show that the performance of this method is better than the baseline and previous work using adversarial training. This work can extract better speaker representations that are both domain‐ invariant and domain‐specific. The proposed collaborative discriminator enables the speaker encoder to learn domain‐specific information, which is beneficial for adversarial training. Further, the projection block is designed to reduce the noise introduced by the collaborative discriminator.