Recently, stereo matching algorithms based on end-to-end convolutional neural networks achieve excellent performance far exceeding traditional algorithms. Current state-of-the-art stereo matching networks mostly rely on full cost volume and 3D convolutions to regress dense disparity maps. These modules are computationally complex and high consumption of memory, and difficult to deploy in real-time applications. To overcome this problem, we propose multilevel disparity reconstruction network, MDRNet, a lightweight stereo matching network without any 3D convolutions. We use stacked residual pyramids to gradually reconstruct disparity maps from low-level resolution to full-level resolution, replacing common 3D computation and optimization convolutions. Our approach achieves a competitive performance compared with other algorithms on stereo benchmarks and real-time inference at 30 frames per second with 4×10 4 resolutions.