The mammary gland undergoes dramatic post-natal growth beginning at puberty, followed by full development occurring during pregnancy and lactation. Following lactation, the alveoli undergo apoptosis, and the mammary gland reverses back to resemble the nonparous gland. This process of growth and regression occurs for multiple pregnancies, suggesting the presence of a hierarchy of stem and progenitor cells that are able to regenerate specialized populations of mammary epithelial cells. Expansion of epithelial cell populations in the mammary gland is regulated by ovarian steroids, in particular estrogen acting through its receptor estrogen receptor alpha (ERα) and progesterone signaling through progesterone receptor (PR). A diverse number of stem and progenitor cells have been identified based on expression of cell surface markers and functional assays. Here we review the current understanding of how estrogen and progesterone act together and separately to regulate stem and progenitor cells within the human and mouse mammary tissues. Better understanding of the hierarchal organization of epithelial cell populations in the mammary gland and how the hormonal milieu affects its regulation may provide important insights into the origins of different subtypes of breast cancer.