Material that can emit broad spectral wavelengths covering deep ultraviolet, visible, and near-infrared is highly desirable. It can lead to important applications such as broadband modulators, photodetectors, solar cells, bioimaging, and fiber communications. However, there is currently no material that meets such desirable requirement. Here, we report the layered structure of nitrogen-doped graphene quantum dots (N-GQDs) which possess broadband emission ranging from 300 to >1000 nm. The broadband emission is attributed to the layered structure of the N-GQDs that contains a large conjugated system and provides extensive delocalized π electrons. In addition, a broadband photodetector with responsivity as high as 325 V/W is demonstrated by coating N-GQDs onto interdigital gold electrodes. The unusual negative photocurrent is observed which is attributed to the trapping sites induced by the self-passivated surface states in the N-GQDs.