Stigma poses considerable challenges to the mental health of people living with HIV who use drugs (PLHWUD). In this study, we explored factors related to different types of stigma (perceived and internalized) attached to layered stigmatizing characters (HIV and drug use) and their mental health influences on PLHWUD. The study used baseline data of an ongoing randomized controlled trial among 241 PLHWUD recruited between March and December 2018 in Vietnam. A structural equation model was used to assess the relationships among different types and layers of stigma and mental health status. Both perceived and internalized drug-related stigma measures were significantly higher than their corresponding HIV-related stigma. HIV-related stigma was negatively associated with mental health status; however, we did not find a significant relationship between drug-related stigma and mental health. Tailored intervention strategies in consideration of types and layers of stigma are needed to address stigma-related challenges faced by PLHWUD.