Simultaneous recordings of large populations of neurons in behaving animals allow detailed observation of high-dimensional, complex brain activity. However, experimental approaches often focus on singular behavioral paradigms or brain areas. Here, we recorded whole-brain neuronal activity of larval zebrafish presented with a battery of visual stimuli while recording fictive motor output. We identified neurons tuned to each stimulus type and motor output and discovered groups of neurons in the anterior hindbrain that respond to different stimuli eliciting similar behavioral responses. These convergent sensorimotor representations were only weakly correlated to instantaneous motor activity, suggesting that they critically inform, but do not directly generate, behavioral choices. To catalog brain-wide activity beyond explicit sensorimotor processing, we developed an unsupervised clustering technique that organizes neurons into functional groups. These analyses enabled a broad overview of the functional organization of the brain and revealed numerous brain nuclei whose neurons exhibit concerted activity patterns. •Sensory input drives behavior via distributed circuits in larval zebrafish•Activity from nearly all neurons in the brain was recorded in behaving animals•Convergent representations of diverse visual stimuli inform behavioral choices•Unsupervised clustering reveals patterns of brain-wide functional organization Chen et al. examine brain-wide functional organization in larval zebrafish under diverse visual stimulus conditions. They systematically characterize neurons related to convergent sensorimotor processing as well as extract concerted brain-wide activity patterns beyond sensorimotor contexts.