Neuronal computations critically depend on the connectivity rules that govern the convergence of excitatory and inhibitory synaptic signals onto individual neurons. To examine the functional synaptic organization of a distributed memory network, we performed voltage clamp recordings in telencephalic area Dp of adult zebrafish, the homolog of olfactory cortex. In neurons of posterior Dp, odor stimulation evoked large, recurrent excitatory and inhibitory inputs that established a transient state of high conductance and synaptic balance. Excitation and inhibition in individual neurons were co-tuned to different odors and correlated on slow and fast timescales. This precise synaptic balance implies specific connectivity among Dp neurons, despite the absence of an obvious topography. Precise synaptic balance stabilizes activity patterns in different directions of coding space and in time while preserving high bandwidth. The coordinated connectivity of excitatory and inhibitory subnetworks in Dp therefore supports fast recurrent memory operations. •Odor responses in zebrafish olfactory cortex support a “balanced state”•Synaptic conductances are balanced on fast timescales and in coding space•This implies co-tuning of excitatory and inhibitory synaptic inputs•Precise balance may stabilize memory patterns and support fast classification Rupprecht and Friedrich find that a transient balanced state is established in the zebrafish homolog of olfactory cortex during an odor response. The balance is maintained on short timescales (tight balance) and in stimulus space (detailed balance) and can stabilize activity patterns in coding space while preserving a high temporal bandwidth.