In this paper, we present a study of the realism of color image composites. Assessing the realism of image composites has emerged as a new field of image processing due to the advances in digital ...imaging and communications. However, when making image composites, users often suffer from color incompatibility between the inserted object and the background. We observe two properties that help make an image composite look realistic. The first property is related to the color similarity between different segments of the image, and the second one is related to the consistency of color deviation between the segments. These two properties only require information available from a single image. An algorithm based on these two properties is proposed for assessment of image realism. Effectiveness of the algorithm is demonstrated.
We present a method for estimating Neural Radiance Fields (NeRF) from a single headshot portrait. While NeRF has demonstrated high-quality view synthesis, it requires multiple images of static scenes ...and thus impractical for casual captures and moving subjects. In this work, we propose to pretrain the weights of a multilayer perceptron (MLP), which implicitly models the volumetric density and colors, with a meta-learning framework using a light stage portrait dataset. To improve the generalization to unseen faces, we train the MLP in the canonical coordinate space approximated by 3D face morphable models. We quantitatively evaluate the method using controlled captures and demonstrate the generalization to real portrait images, showing favorable results against state-of-the-arts.
Compared to DSLR cameras, smartphone cameras have smaller sensors, which limits their spatial resolution; smaller apertures, which limits their light gathering ability; and smaller pixels, which ...reduces their signal-to noise ratio. The use of color filter arrays (CFAs) requires demosaicing, which further degrades resolution. In this paper, we supplant the use of traditional demosaicing in single-frame and burst photography pipelines with a multiframe super-resolution algorithm that creates a complete RGB image directly from a burst of CFA raw images. We harness natural hand tremor, typical in handheld photography, to acquire a burst of raw frames with small offsets. These frames are then aligned and merged to form a single image with red, green, and blue values at every pixel site. This approach, which includes no explicit demosaicing step, serves to both increase image resolution and boost signal to noise ratio. Our algorithm is robust to challenging scene conditions: local motion, occlusion, or scene changes. It runs at 100 milliseconds per 12-megapixel RAW input burst frame on mass-produced mobile phones. Specifically, the algorithm is the basis of the Super-Res Zoom feature, as well as the default merge method in Night Sight mode (whether zooming or not) on Google's flagship phone.
We present Steadiface, a new real-time face-centric video stabilization method that simultaneously removes hand shake and keeps subject's head stable. We use a CNN to estimate the face landmarks and ...use them to optimize a stabilized head center. We then formulate an optimization problem to find a virtual camera pose that locates the face to the stabilized head center while retains smooth rotation and translation transitions across frames. We test the proposed method on fieldtest videos and show it stabilizes both the head motion and background. It is robust to large head pose, occlusion, facial appearance variations, and different kinds of camera motions. We show our method advances the state of art in selfie video stabilization by comparing against alternative methods. The whole process runs very efficiently on a modern mobile phone (8.1 ms/frame).
Panoramic imaging system for camera phones Pulli, K.; Tico, M.; Yingen Xiong ...
2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE),
2010-Jan.
Conference Proceeding
We introduce a mobile system for high resolution panoramic image creation. In our system the user can rotate the camera arbitrarily and see the preview panorama in real-time. The system automatically ...captures high resolution images and generates a high-quality result with unlimited viewing angle. We employ a coarse-to-fine method for high-quality registration, and a seam-finding method to remove the ghosting effect due to moving objects. The proposed system has been tested on several types of camera phones, and the tests reveal that the system can efficiently provide a high-quality panoramic image in spite of the low computational power and memory available in such devices.
Depth Detection of Light Field Yi-Hao Kao; Chia-Kai Liang; Li-Wen Chang ...
2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07,
2007-April, Letnik:
1
Conference Proceeding
We propose an algorithm to detect depths in a light field. Specifically, given a 4D light field, we find all planes at which objects are located. Although the exact depth of each pixel in the space ...is left unknown, the partial information obtained is very useful for many applications, such as synthetic aperture photography and all-focused rendering. Our algorithm measures the degree of focus of different planes by calculating the ratio of high frequencies to the low frequencies. To handle different depth distributions, we reformulate the maximum detection problem to a maximum-cover problem that can be solved efficiently by dynamic programming. Compared with auto-focusing and per-pixel depth estimation, our algorithm is much faster yet sufficiently accurate.
Reducing LCD backlight saves power consumption of a portable device, but it also decreases the contrast and brightness of the displayed image. Previous approaches adjust the backlight level frame by ...frame to reach a specified image quality level without optimizing the image quality. In contrast, the proposed method adjusts the backlight to meet the target power level while maintaining the image quality. This is achieved by performing brightness compensation and local contrast enhancement in accordance with the given backlight level. Experimental results show that the proposed algorithm outperforms previous methods.
The electronic image stabilizer and video codec are two important components of a digital video camera. Both require motion information of the captured image sequence to perform their respective ...tasks. Since motion estimation is a computationally intensive operation, we propose three schemes for integrating the electronic image stabilizer with the video codec. The technical issues involved in the integration are discussed, and simulation results are shown to illustrate the effectiveness of the integration schemes.
This paper introduces 3D cinematography principles to the field of multimedia and illustrates their usage in stereoscopic media processing applications. These principles include (1) maintaining ...coordination among views, (2) having a continuous depth chart, (3) placing rest areas between strong 3D shots, (4) using a shallow depth of field for shots with excessive depth brackets, and (5) being careful about the stereoscopic window. Taking these principles into account, we propose designs for stereoscopic extensions of two popular 2D media applications---video stabilization and photo slideshow---to provide a better 3D viewing experience. User studies show that by incorporating 3D cinematography principles, the proposed methods yield more comfortable and enjoyable 3D viewing experiences than those delivered using naive extensions of conventional 2D methods.
Belief propagation has become a popular technique for solving computer vision problems, such as stereo estimation and image denoising. However, it requires large memory and bandwidth, and hence naive ...hardware implementation is prohibitive. In this paper, we first analyze the memory and bandwidth requirements of the technique from the hardware perspective. Then, we propose a tile-based belief propagation algorithm that works with existing data reuse schemes and achieves bandwidth reduction by a factor of 10 to 400. We apply the proposed algorithm to stereo estimation and show that its performance is comparable to the original algorithm.
PDF
