We present constraints on Ks-band emission from one of the nearest short hard gamma-ray bursts, GRB 160821B, at z = 0.16, at three epochs. We detect a red relativistic afterglow from the jetted emission in the first epoch but do not detect any excess kilonova emission in the second two epochs. We compare upper limits obtained with Keck I/MOSFIRE to multi-dimensional radiative transfer models of kilonovae, that employ composition-dependent nuclear heating and LTE opacities of heavy elements. We discuss eight models that combine toroidal dynamical ejecta and two types of wind and one model with dynamical ejecta only. We also discuss simple, empirical scaling laws of predicted emission as a function of ejecta mass and ejecta velocity. Our limits for GRB 160821B constrain the ejecta mass to be lower than 0.03 M for velocities greater than 0.1 c. At the distance sensitivity range of advanced LIGO, similar ground-based observations would be sufficiently sensitive to the full range of predicted model emission including models with only dynamical ejecta. The color evolution of these models shows that I-K color spans 7-16 mag, which suggests that even relatively shallow infrared searches for kilonovae could be as constraining as optical searches.