This study aimed to explore why urban exodus occurred in Tokyo during the COVID-19 pandemic. Using panel-level data from 203 municipalities, we estimated the impact of epidemic, medical, economic, and industrial factors on the movement of people across cities in the Tokyo metropolitan area during the pandemic. We found that fear of infection was one of the main factors driving the urban exodus in Tokyo and that people chose to live in reasonably priced habitable areas during the pandemic. We also found that the population increased when the municipality had an essential sector in which remote work was difficult but decreased when it had an industry in which remote work was relatively feasible. However, medical factors had contrasting effects on out-migration and in-migration in each municipality. Increases in hospitals and medical doctors may not have increased the attractiveness of the city because of the malfunctioning of the healthcare system during the pandemic.