In a longitudinal two-wave study we examined the effects of positive and negative intergroup contact on outgroup attitudes in participants who perceived positive, negative, or ambivalent group stereotypes. We focused on stereotype-consistent contact, occurring when the valence of participants’ contact matches the valence of the perceived group stereotype (e.g., negative–negative), and on stereotype-inconsistent contact, occurring when the valence of contact contradicts the valence of the group stereotype (e.g., positive–negative). In relations of the Czech majority (N = 890) with two distinctly stereotyped minorities, the Roma and the Vietnamese, stereotype-inconsistent contact predicted changes in attitudes better than stereotype-consistent contact. In the case of negatively stereotyped groups, positive intergroup contact is a viable way to improve attitudes. For positively stereotyped groups, negative contact can worsen attitudes, while positive contact does not have any attitude-improving effect. Interventions aimed at improving outgroup attitudes need to be applied with caution, considering the valence of group stereotypes.