Pancreatic ductal adenocarcinoma (PDAC) is an aggressive disease with a notably poor prognosis, in urgent need of improved treatment strategies. The desmoplastic PDAC tumor microenvironment (TME), marked by a high concentration of cancer-associated-fibroblasts (CAFs), is a dynamic part of PDAC pathophysiology which occasions a variety of effects throughout the course of pancreatic tumorigenesis and disease evolution. A better understanding of the desmoplastic TME and CAF biology in particular, should provide new opportunities for improving therapeutics. That CAFs have a tumor-supportive role in oncogenesis is well known, yet research evidence has shown that CAFs also have tumor-repressive functions. In this review, we seek to clarify the intriguing heterogeneity and plasticity of CAFs and their ambivalent role in PDAC tumorigenesis and progression. Additionally, we provide recommendations to advance the implementation of CAF-directed PDAC care. An improved understanding of CAFs’ origins, spatial location, functional diversity, and marker determination, as well as CAF behavior during the course of PDAC progression and metastasis will provide essential knowledge for the future improvement of therapeutic strategies for patients suffering from PDAC.