To better understand the social norms surrounding fantasizing behavior, the current research aimed to assess how acceptable various types of fantasizing (romantic or sexual) are perceived. Understanding and abiding by social norms helps people avoid criticism, social sanctions, and ostracism. Thus, better understanding the social norms surrounding various types of fantasies can help people better navigate their social worlds, especially with respect to sexuality, dating, and relationships. Participants (n = 828) reported how acceptable, violating, and bothersome they perceived sexual and romantic fantasizing to be towards themselves and others. Results suggest that despite the current sentiment on socially and morally unacceptable physical acts, mental acts of fantasizing are not perceived as unacceptable or violating. No gender differences arose between men and women’s perceptions of fantasy acceptability. Demographic differences in perceived fantasy acceptability by race, sexual orientation, relationship status, and age are discussed. These findings deepen the understanding of how society views fantasizing behavior and help begin to define boundaries for acceptable versus violating thoughts.