Capacitive deionization (CDI) is a competent water desalination technique offering an appropriate route to obtain clean water. However, a rational designed structure of the electrode materials is essentially required for achieving high CDI performance. Here, a novel sponge‐templated strategy is developed for the first time to prepare graphene sheets with high specific surface area and suitable pore size distribution. Sponge is used as the support of graphene oxide to prevent the restack of graphene sheets, as well as to suppress the agglomerate during the annealing process. Importantly, the as‐fabricated graphene sheets possess high specific surface area of 305 m2 g−1 and wide pore size distribution. Ultrahigh CDI performance, a remarkable electrosorptive capacity of 4.95 mg g−1, and siginificant desorption rate of 25 min, is achieved with the sponge‐templated prepared graphene electrodes. This work provides an effective solution for the synthesis of rational graphene architectures for general applications in CDI, energy storage and conversion. Sponge‐templated graphene sheets (STGS) with high specific surface area, wide pore size distribution, and low internal resistance are prepared through a simple annealing method. The as‐prepared STGS exhibits promising CDI performance with an ultrahigh electrosorptive capacity of 4.95 mg g−1 and fast desorption rate of 25 min.