Holographic stereograms produce multiple parallax images that are seen from multiple viewpoints. Because random phase distributions are added to the parallax images to remove areas where images cannot be seen in the viewing area, speckles are generated in the reconstructed images. In this study, virtual viewpoints are inserted between the original viewpoints (real viewpoints) to make the interval of the viewpoints smaller than the pupil diameter of the eyes in order to remove the areas without images. In this case, regular interference patterns appear in the reconstructed images instead of the speckle patterns. The proper phase modulation of the parallax images displayed to the real and virtual viewpoints increases the spatial frequencies of the regular interference patterns on retinas so that the eyes cannot perceive them. The proposed technique was combined with the multiview-based holographic stereogram calculation technique and was experimentally verified.