Electrochemical synthesis of valuable chemicals and feedstocks through carbon dioxide (CO2) reduction in acidic electrolytes can surmount the considerable CO2 loss in alkaline and neutral conditions. However, achieving high productivity, while operating steadily in acidic electrolytes, remains a big challenge owing to the severe competing hydrogen evolution reaction. Here, we show that vertically grown bismuth nanosheets on a gas-diffusion layer can create numerous cavities as electrolyte reservoirs, which confine in situ–generated hydroxide and potassium ions and limit inward proton diffusion, producing locally alkaline environments. Based on this design, we achieve formic acid Faradaic efficiency of 96.3% and partial current density of 471 mA cm−2 at pH 2. When operated in a slim continuous-flow electrolyzer, the system exhibits a full-cell formic acid energy efficiency of 40% and a single pass carbon efficiency of 79% and performs steadily over 50 h. We further demonstrate the production of pure formic acid aqueous solution with a concentration of 4.2 weight %.