Efforts to contain the spread of SARS-CoV-2 have spurred the need for reliable, rapid, and cost-effective diagnostic methods which can be applied to large numbers of people. However, current standard protocols for the detection of viral nucleic acids while sensitive, require a high level of automation and sophisticated laboratory equipment to achieve throughputs that allow whole communities to be tested on a regular basis. Here we present Cap-iLAMP (capture and improved loop-mediated isothermal amplification) which combines a hybridization capture-based RNA extraction of gargle lavage samples with an improved colorimetric RT-LAMP assay and smartphone-based color scoring. Cap-iLAMP is compatible with point-of-care testing and enables the detection of SARS-CoV-2 positive samples in less than one hour. In contrast to direct addition of the sample to improved LAMP (iLAMP), Cap-iLAMP prevents false positives and allows single positive samples to be detected in pools of 25 negative samples, reducing the reagent cost per test to ~1 Euro per individual.