The SARS-CoV-2 virus emerged in December 2019 and has caused a worldwide pandemic due to the lack of any pre-existing immunity. Accurate serology testing is urgently needed to help diagnose infection, determine past exposure of populations and assess the response to a future vaccine. The landscape of antibody responses to SARS-CoV-2 is unknown. In this study, we utilized the luciferase immunoprecipitation system to assess the antibody responses to 15 different SARS-CoV-2 antigens in patients with COVID-19. We identified new targets of the immune response to SARS-CoV-2 and show that nucleocapsid, open reading frame (ORF)8 and ORF3b elicit the strongest specific antibody responses. ORF8 and ORF3b antibodies, taken together as a cluster of points, identified 96.5% of COVID-19 samples at early and late time points of disease with 99.5% specificity. Our findings could be used to develop second-generation diagnostic tests to improve serological assays for COVID-19 and are important in understanding pathogenicity.