As our cities become more complex and traffic demand grows, managing such traffic efficiently becomes challenging. Hence, solutions that allow building upon the current traffic light systems and that can be readily deployed are of global interest. In this work, we address the challenge of improving traffic light management at intersections. We propose an agent-based traffic light control system where an agent, one per intersection, dynamically regulates the light’s phase cycle depending on the current traffic conditions. To this end, we will rely on Deep Networks to adequately train agents to make good decisions. Simulation results in a realistic scenario using SUMO show that our proposed approach can significantly reduce waiting times, improving transit times by 44% compared to the standard fixed-timing method. Additionally, to assess the effectiveness and reliability of our control algorithm, we introduce new performance metrics. Display omitted •A distributed neural network-based agent system optimizes traffic in superblocks.•Every agent manages one intersection using deep Q-learning with experience replay.•Each agent optimizes its intersection and reduces vehicle waiting times.•Compared with fixed time, our algorithm achieves a 44% reduction in waiting times.•Thanks to deep Q-networks, changes in agents’ actions allow traffic light adaptation.