Purpose This work proposes new low rank approximation approaches with significant memory savings for large scale MR fingerprinting (MRF) problems. Theory and Methods We introduce a compressed MRF with randomized singular value decomposition method to significantly reduce the memory requirement for calculating a low rank approximation of large sized MRF dictionaries. We further relax this requirement by exploiting the structures of MRF dictionaries in the randomized singular value decomposition space and fitting them to low‐degree polynomials to generate high resolution MRF parameter maps. In vivo 1.5T and 3T brain scan data are used to validate the approaches. Results T1, T2, and off‐resonance maps are in good agreement with that of the standard MRF approach. Moreover, the memory savings is up to 1000 times for the MRF‐fast imaging with steady‐state precession sequence and more than 15 times for the MRF‐balanced, steady‐state free precession sequence. Conclusion The proposed compressed MRF with randomized singular value decomposition and dictionary fitting methods are memory efficient low rank approximation methods, which can benefit the usage of MRF in clinical settings. They also have great potentials in large scale MRF problems, such as problems considering multi‐component MRF parameters or high resolution in the parameter space. Magn Reson Med 79:2392–2400, 2018. © 2017 International Society for Magnetic Resonance in Medicine.