Background Adrenal lipid-poor adenomas (LPA) are defined by high unenhanced density (≥ 10 HU), and absolute and relative contrast medium washout > 60% and > 40%, respectively, at computerized tomography (CT). To date, no thorough histopathological characterization has been performed in those frequent lesions (one-third of adrenal adenomas). Our aim was to analyze the histopathological characteristics of adrenal LPA. Methods Patients with LPA ( n  = 57) were selected among consecutive subjects referred for an adrenal incidentaloma or ACTH-independent Cushing syndrome. FluoroDeoxyGlucose-Positron Emission Tomography (FDG-PET) was performed in 37 patients. In patients treated by adrenalectomy ( n  = 17), Weiss score and Lin–Weiss–Bisceglia score (in tumors composed entirely or predominantly of oncocytes) were calculated. Results Radiological parameters did not differ among patients with ACTH-independent Cushing syndrome ( n  = 6) and those with adrenal incidentalomas associated with primary aldosteronism ( n  = 2), autonomous cortisol secretion ( n  = 14), or non-functioning ( n  = 35). Patients treated by adrenalectomy had larger tumors (28.9 ± 11.2 vs 17.3 ± 8.4 mm, P < 0.001), higher CT unenhanced density (29.1 ± 11.0 vs 23.1 ± 9.0 HU, P  = 0.043), and FDG-PET adrenal uptake (9.0 ± 6.4 vs 4.4 ± 2.3 SUV, P  = 0.003) than non-operated ones. Oncocytic features > 75% of the tumor were detected in 12/17 cases (70.6%). Five of those showed borderline-malignant histopathological characteristics by Lin–Weiss–Bisceglia score. Among remaining non-oncocytic tumors, 1/5 had a Weiss score ≥ 3. Overall, 6/17 tumors (35.3%) had borderline-malignant potential. Radiological parameters were similar between patients with benign and borderline-malignant tumors. Conclusions Adrenal LPA are a heterogeneous group of tumors, mostly composed of oncocytomas. Up to 1/3 of those tumors may have a borderline-malignant potential at histopathology.