Load restoration coordinating transmission grid, distribution grid, and microgrids is an effective measure that is taken into consideration while improving the power system resilience in extreme weather conditions. An online decision-making method is proposed to deal with the unexpected nature of power supply issues regarding the re-energization of microgrids and transmission grids. In this research work, an online multi-agent interaction technique is used for coordinated load restoration. The main algorithm comprises of two subsections, namely, a resilience index and a multi-agent-based decision-making system which are used to administer the coordination among the transmission grid, distribution grid, and microgrids. A distributionally robust optimization model is used to evaluate the power supply capability of microgrids on the basis of load restoration parameters. Finally, a step-by-step decision-making method, based on a deep Q-network, is proposed for distribution network reconfiguration considering the uncertainty of power supply capabilities of transmission grid and microgrids. Simulation results demonstrated that the proposed method can perform the online decision-making of substation load restoration, which significantly improves the load restoration efficiency.