Artificial deep neural networks (DNNs) initially inspired by the brain enable computers to solve cognitive tasks at which humans excel. In the absence of explanations for such cognitive phenomena, in turn cognitive scientists have started using DNNs as models to investigate biological cognition and its neural basis, creating heated debate. Here, we reflect on the case from the perspective of philosophy of science. After putting DNNs as scientific models into context, we discuss how DNNs can fruitfully contribute to cognitive science. We claim that beyond their power to provide predictions and explanations of cognitive phenomena, DNNs have the potential to contribute to an often overlooked but ubiquitous and fundamental use of scientific models: exploration. Neurally inspired deep neural networks (DNNs) have recently emerged as powerful computer algorithms tackling real-world tasks on which humans excel, such as object recognition, speech processing, and cognitive planning. In the absence of scientific explanations regarding how humans solve such tasks, some cognitive scientists have turned to DNNs as models of human brain responses and behaviour. In visual and auditory processing, DNNs were found to predict human brain responses and behaviour better than other models. The use of DNNs as models in cognitive science has created a heated debate about their scientific value: in particular, are DNNs only valuable as predictive tools or do they also offer useful explanations of the phenomena investigated?