This work is the most comprehensive study on the quantitative behavior of olive fruit phenols during olive oil processing, providing insight into their transfer, transformation, and partition trail. In total, 69 phenols were quantified in 6 olive matrices from a three-phase extraction line employing ultra high pressure liquid chromatography–diode array detection analysis. Crushing had a larger effect than malaxation in terms of phenolic degradation and transformation, resulting in several new evolutions of respective derivatives. The peel and pulp together confined 95% of total fruit phenols, while stone only 5%. However, only 0.53% of all ended-up in olive oil, nearly 6% in wastewater, and 48% in pomace. Secoiridoids were the predominant class in all matrices, though represented by different individuals. Their partition behavior was rather similar to other phenolic classes, where with few minor exceptions only aglycones were partitioned to the oil, while other glycosides were lost with the wastes.