Owing to the rapid spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic worldwide, individuals experience considerable psychological distress daily. The present study aimed to clarify the prevalence of psychological distress and determine the population most affected by risk factors such as the pandemic, socioeconomic status (SES), and lifestyle-related factors causing psychological distress in the early phases of the pandemic in Japan. This study was conducted via a web-based survey using quota sampling to ensure representativeness of the Japanese population aged 20-64 years. A cross-sectional study of 11,342 participants (5,734 males and 5,608 females) was conducted using a self-administered questionnaire that included the Japanese version of the Kessler 6 Psychological Distress Scale (K6) and questions related to the pandemic, SES, and lifestyle. The prevalence of psychological distress, represented by a K6 score of 5 or more, was 50.3% among males and 52.6% among females. Both males and females with annual household incomes less than 2 million yen and males aged in their twenties had significantly higher K6 scores than those with annual household incomes above 2 million yen and males aged over 30 years. Binary logistic regression analyses found pandemic-related factors such as medical history, inability to undergo clinical tests immediately, having trouble in daily life, unavailability of groceries, new work style, and vague anxiety; SES-related factors such as lesser income; and lifestyle-related factors such as insufficient rest, sleep, and nutritious meals to be significantly related to psychological distress. Psychological distress was more prevalent among people with low income and in younger generations than among other groups. There is an urgent need to provide financial, medical, and social support to those affected by the coronavirus disease 2019 (COVID-19) pandemic.