This research introduces the Boosted Ensemble deep Multi-Layer Layer Perceptron (EdMLP) architecture with multiple output layers, a novel enhancement for the traditional Multi-Layer Perceptron (MLP). By adopting a layer-wise training approach, EdMLP enables the integration of boosting techniques within a single model, treating each layer as a weak learner, resulting in substantial performance gains. Additionally, the inclusion of layer-wise hyperparameter tuning allows optimization of individual layers thereby reducing the tuning time. Furthermore, the ensemble deep architecture’s versatility can be extended to other neural network-based models, such as the Self Normalized Network (SNN) where experiments demonstrate substantial performance enhancements yielded by the EdSNN compared to the standard original SNN model. This research underscores the potential of the EdMLP, and the Ed architecture in general as a powerful tool for improving the performance of various multilayer feedforward neural network models. The source code of this work is publicly accessible from the authors GitHub. •Proposed EdMLP architecture with multiple output layers to enhance conventional MLP.•Layer-wise training integrates boosting in the second and subsequent output layers.•Layer-wise tuning cuts hyperparameter tuning time significantly.•Training a deep neural network with multiple output layers and boosting can improve other models.