•Extensive experimental field investigations of track degradation were carried out.•Heterogeneity of sub-layers and geometrical irregularities influences degradation.•Favorable features of wooden sleepers (lower stiffness, imprinting of ballast).•CAE ANN reveals nonlinear influence of influential parameters on track degradation.•Weights of influential parameters indicate welds as the most important parameter. Several various examples of anomalies on ballast railway tracks resulting in accelerated degradation of railway sleepers and ballast layer are described in the paper. Initially, some locations on the Slovenian railway network (SRN) where the degradation occurs were identified. Afterwards, various experimental field investigations (georadar, geometry measurements, measurements of displacements and accelerations on the track, visual assessment of ballast degradation) have been used to measure the characteristics of the railway track and the parameters of its behavior. Based on the experimentally obtained data a simplified numerical model which interconnects the individual measured and estimated parameters of the railway track and also the processes of further degradation has been developed using the artificial neural network. Influential factors for the model’s individual parameters influencing the degradation were identified. The proposed model is able to assist in the evaluation of critical areas on the railway infrastructure and further enables a better understanding of the process and prediction/estimation of degradation of railway sleepers and ballast.