Load forecasting is a vital part of smart grids for predicting the required electrical power using artificial intelligence (AI). Deep learning is broadly used for load forecasting in the smart grid using the artificial neural network (ANN). Generally, computing the deep learning in the smart grid requires massive data aggregation or centralization and significant computational time. This paper presents a survey of deep learning-based load forecasting techniques from 2015 to 2020. This survey discusses the studies based on their deep learning techniques, Distributed Deep Learning (DDL) techniques, Back Propagation (BP) based works, and non-BP based works in the load forecasting process. Consequent to the survey, it was determined that data aggregation dependency would be beneficial for reducing computational time in load forecasting. Therefore, a conceptual model of DDL for smart grids has been presented, where the HSIC (Hilbert-Schmidt Independence Criterion) Bottleneck technique has been incorporated to provide higher accuracy.