Purpose To develop a new method for high‐fidelity, high‐resolution 3D multi‐slab diffusion MRI with minimal distortion and boundary slice aliasing. Methods Our method modifies 3D multi‐slab imaging to integrate blip‐reversed acquisitions for distortion correction and oversampling in the slice direction (kz) for reducing boundary slice aliasing. Our aim is to achieve robust acceleration to keep the scan time the same as conventional 3D multi‐slab acquisitions, in which data are acquired with a single direction of blip traversal and without kz‐oversampling. We employ a two‐stage reconstruction. In the first stage, the blip‐up/down images are respectively reconstructed and analyzed to produce a field map for each diffusion direction. In the second stage, the blip‐reversed data and the field map are incorporated into a joint reconstruction to produce images that are corrected for distortion and boundary slice aliasing. Results We conducted experiments at 7T in six healthy subjects. Stage 1 reconstruction produces images from highly under‐sampled data (R = 7.2) with sufficient quality to provide accurate field map estimation. Stage 2 joint reconstruction substantially reduces distortion artifacts with comparable quality to fully‐sampled blip‐reversed results (2.4× scan time). Whole‐brain in‐vivo results acquired at 1.22 mm and 1.05 mm isotropic resolutions demonstrate improved anatomical fidelity compared to conventional 3D multi‐slab imaging. Data demonstrate good reliability and reproducibility of the proposed method over multiple subjects. Conclusion The proposed acquisition and reconstruction framework provide major reductions in distortion and boundary slice aliasing for 3D multi‐slab diffusion MRI without increasing the scan time, which can potentially produce high‐quality, high‐resolution diffusion MRI.