The COVID-19 pandemic provides an urgent example where a gap exists between availability of state-of-the-art diagnostics and current needs. As assay protocols and primer sequences become widely known, many laboratories perform diagnostic tests using methods such as RT-PCR or reverse transcription loop mediated isothermal amplification (RT-LAMP). Here, we report an RT-LAMP isothermal assay for the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus and demonstrate the assay on clinical samples using a simple and accessible point-of-care (POC) instrument. We characterized the assay by dipping swabs into synthetic nasal fluid spiked with the virus, moving the swab to viral transport medium (VTM), and sampling a volume of the VTM to perform the RT-LAMP assay without an RNA extraction kit. The assay has a limit of detection (LOD) of 50 RNA copies per μL in the VTM solution within 30 min. We further demonstrate our assay by detecting SARS-CoV-2 viruses from 20 clinical samples. Finally, we demonstrate a portable and real-time POC device to detect SARS-CoV-2 from VTM samples using an additively manufactured three-dimensional cartridge and a smartphone-based reader. The POC system was tested using 10 clinical samples, and was able to detect SARS-CoV-2 from these clinical samples by distinguishing positive samples from negative samples after 30 min. The POC tests are in complete agreement with RT-PCR controls. This work demonstrates an alternative pathway for SARS-CoV-2 diagnostics that does not require conventional laboratory infrastructure, in settings where diagnosis is required at the point of sample collection.