•A very low rate of 3 ± 3 FPs per patient was maintained for all brain metastasis sizes.•Automated segmentation performance for brain metastases depends on metastasis size.•For metastases ≥6 mm, DSC was 87%, sensitivity was 99%, and PPV was 67%.•For metastases ≥3 mm and <6 mm, DSC was 64%, sensitivity was 87%, and PPV was 49%.•For metastases <3 mm, DSC was 17%, sensitivity was 25%, and PPV was 28%. Brain metastases are manually contoured during stereotactic radiosurgery (SRS) treatment planning, which is time-consuming, potentially challenging, and laborious. The purpose of this study was to develop and investigate a 2-stage deep learning (DL) approach (MetNet) for brain metastasis segmentation in pre-treatment magnetic resonance imaging (MRI). We retrospectively analyzed postcontrast 3D T1-weighted spoiled gradient echo MRIs from 934 patients who underwent SRS between August 2009 and August 2018. Neuroradiologists manually identified brain metastases in the MRIs. The treating radiation oncologist or physicist contoured the brain metastases. We constructed a 2-stage DL ensemble consisting of detection and segmentation models to segment the brain metastases on the MRIs. We evaluated the performance of MetNet by computing sensitivity, positive predictive value (PPV), and Dice similarity coefficient (DSC) with respect to metastasis size, as well as free-response receiver operating characteristics. The 934 patients (mean ±standard deviation age 59 ± 13 years, 474 women) were randomly split into 80% training and 20% testing groups (748:186). For patients with metastases 1–52 mm (n = 766), 648 (85%) were detected and segmented with a mean segmentation DSC of 81% ± 15%. Patient-averaged sensitivity was 88% ± 19%, PPV was 58% ± 25%, and DSC was 85% ± 13% with 3 ± 3 false positives (FPs) per patient. When considering only metastases ≥6 mm, patient-averaged sensitivity was 99% ± 5%, PPV was 67% ± 28%, and DSC was 87% ± 13% with 1 ± 2 FPs per patient. MetNet can segment brain metastases across a broad range of metastasis sizes with high sensitivity, low FPs, and high segmentation accuracy in postcontrast T1-weighted MRI, potentially aiding treatment planning for SRS.