In heavy-ion collisions at large particle colliders, such asLHC or RHIC, heavy-flavour (charm and beauty) quarks are producedmainly through initial hard scatterings. Therefore, they can serveas the probes of properties of the hot medium created in suchcollisions. Additionally, in small collision systems, such asd/p+Au collisions, cold nuclear matter effects can also affect thecharm quark production with respect to p+p collisions.Hadrons, that contain heavy-flavour quarks, could not be directlydetected, thus they are measured via reconstruction of their decayproducts. However, due to a large number of particles produced insuch collisions, separation of the decay products from combinatorialbackground is challenging and advanced statistical analysis isneeded.In this article, we exploitD0 (D0)→K-π+ (K+ π-) decay in order to investigateperformance of several machine learning algorithms with differentimplementation approaches to find the most effective way how toseparate signal from random combinatorial background. For thisstudy, we use HIJING and STAR detector simulation of d+Au collisionsat √(sNN)=200 GeV embedded to the collisions recorded with theSTAR. In this paper we compare deep neural network implementedusing Keras with TensorFlow backend, random forest model implementedusing scikit-learn and boosted decision trees implemented by meansof the Toolkit for Multivariate Data Analysis with ROOT. Describedmethods might be applied on reconstruction of any two-body decay inhigh-energy physics experiments.