With the recent advances of experimental techniques, the nonlinear ultrafast optical response of metal nano-objects can now be investigated both on ensembles and on single nanoparticles. Its connection with the metal electronic and lattice kinetics is studied on the basis of a model describing the wavelength and time-dependent modifications of the object material dielectric function. Its application is illustrated in the case of single silver nanospheres and gold nanorods, as well as on ensembles of noble metal nanoparticles and metal-semiconductor nano-hybrids. This quantitative analysis also permits to elucidate the physical mechanisms at the origin of ultrafast nonlinearities in confined metals at different timescales.