Purpose The aim was to evaluate the prognostic role of sub-categories of ISUP 4 prostate cancer (PCa) on final pathology, and assess the tumor architecture prognostic role for predicting biochemical recurrence (BCR) after radical prostatectomy. Methods From a prospectively-maintained database, we included 370 individuals with ISUP 4 on final pathology. The main outcomes were to evaluate the relationship between different ISUP patterns within the group 4 with pathological and oncological outcomes. Binary logistic regression and Kaplan–Meier estimator were used to evaluate the role of the different categories (3 + 5, 4 + 4, 5 + 3) and tumor architecture (intraductal and/or cribriform) on pathological and oncological outcomes. Results Among the 370 individuals with ISUP considered for the study, 9, 85 and 6% had grade 3 + 5, 4 + 4 and 5 + 3 PCa, respectively. Overall, 74% had extracapsular extension, while lymph node invasion (LNI) was documented in 9%. A total of 144 patients experienced BCR during follow-up. After adjusting for PSA, pT, grade group, LNI and positive surgical margins (PSM), grade 3 + 5 was a protective factor (HR: 0.30, 95% CI: 0.13,0.68, p  = 0.004) in predicting BCR relative to grade 4 + 4. Intraductal or cribriform architecture was correlated with BCR (HR: 5.99, 95% CI: 2.68, 13.4, p  < 0.001) after adjusting for PSA, pT, grade group, LNI and PSM. Conclusions Patients with tumor grade 3 + 5 had better pathological and prognostic outcomes compared to 4 + 4 or 5 + 3. When accounting for tumor architecture, the sub-stratification into subgroups lost its prognostic role and tumor architecture was the sole predictor of poorer prognosis in terms of biochemical recurrence.