Image manipulation localization (IML), which seeks to accurately segment tampered regions that are artfully fastened into a normal image, is a fundamental yet challenging computer vision task. ...Despite that impressive results have been achieved by some progressive deep learning methods, they usually fail in capturing the subtle manipulation artifacts at different object scales, which are not competent to generate a perfect segmentation mask with complete and fine object structures. Besides, the problem of coarse boundaries also occurs frequently. To this end, in this paper, we propose a Transformer-Auxiliary by operator-induced neural Network (TANet) to localize forged regions for IML. Specifically, a stacked multi-scale transformer (SMT) branch is first introduced as a compensation for feature representations of the mainstream convolutional neural network branch. SMT can detect structured abnormalities of the input image at multi-levels by operating on patches of different sizes. Then TANet explicitly exploits an operator induction module (OIM) to excavate valuable and manipulated region-related boundary semantics to guide the representative learning of the mainstream branch. The OIM encourages the network to generate features that highlight object structure, thereby promoting precise boundary localization of forged regions. We conduct extensive experiments on various datasets and settings to validate the effectiveness of TANet. Results show that TANet outperforms the state-of-the-art methods by a large margin under widely-used evaluation metrics.
Fast motion deblurring Cho, Sunghyun; Lee, Seungyong
ACM SIGGRAPH Asia 2009 papers,
12/2009
Conference Proceeding
Peer reviewed
This paper presents a fast deblurring method that produces a deblurring result from a single image of moderate size in a few seconds. We accelerate both latent image estimation and kernel estimation ...in an iterative deblurring process by introducing a novel prediction step and working with image derivatives rather than pixel values. In the prediction step, we use simple image processing techniques to predict strong edges from an estimated latent image, which will be solely used for kernel estimation. With this approach, a computationally efficient Gaussian prior becomes sufficient for deconvolution to estimate the latent image, as small deconvolution artifacts can be suppressed in the prediction. For kernel estimation, we formulate the optimization function using image derivatives, and accelerate the numerical process by reducing the number of Fourier transforms needed for a conjugate gradient method. We also show that the formulation results in a smaller condition number of the numerical system than the use of pixel values, which gives faster convergence. Experimental results demonstrate that our method runs an order of magnitude faster than previous work, while the deblurring quality is comparable. GPU implementation facilitates further speed-up, making our method fast enough for practical use.
Color composition (or color theme) is a key factor to determine how well a piece of art work or graphical design is perceived by humans. Despite a few color harmony models have been proposed, their ...results are often less satisfactory since they mostly neglect the variations of aesthetic cognition among individuals and treat the influence of all ratings equally as if they were all rated by the same anonymous user. To overcome this issue, in this paper we propose a new color theme evaluation model by combining a back propagation neural network and a kernel probabilistic model to infer both the color theme rating and the user aesthetic preference. Our experiment results show that our model can predict more accurate and personalized color theme ratings than state of the art methods. Our work is also the first-of-its-kind effort to quantitatively evaluate the correlation between user aesthetic preferences and color harmonies of five-color themes, and study such a relation for users with different aesthetic cognition.
Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively ...manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.
Objetivo. A pesquisa a seguir tem como objetivo analisar os novos conteúdos de disseminação de informações falsas, assim buscando conceituar o que seriam as “deepfake” e as “fakenews” e as ...consequências dessas manipulações de imagem e veiculação de noticias.Metodo. Desenvolveu-se uma pesquisa exploratória com levantamento bibliográfico de materiais previamente publicados, como artigos de periódicos científicos, em que foi realiza uma revisão de literatura apropriada de maneira a esclarecer e apresentar o assunto sobre as “deepfakes” ainda pouco conhecida, além disso, apresentar de que maneira ela é criada, manipulada a ponto de se dissemina de mananeira desenfreada frente às redes sociais.Resultados. Sabe-se que hoje os softwares e aplicativos para smatphones proporcionam, de modo muito hábil, que usuários manipulem imagem com extrema facilidade, os chamados fakeApp,que são ferramentas que permitem alteração e manipulação de imagens de maneira que não deixem pistas visuais de sua alteração, podendo ser indistinguíveis aos autênticos. Exemplo do resultado destas manipulações é a “deepfake”, é a técnica que substitui o rosto de uma pessoa por outra em um vídeo, no panorama atual é comum visualizar vídeos falsos, o que geram as “fakenews”, ou notícias falsas, muito veiculadas nas redes sociais e que cabem por ganhar certa credibilidade devido à dificuldade de distinguir a veracidade na imagem ali mostrada, tão quão são imperceptíveis as marcas de adulteração. O que facilita a produção das “fakenews”, é que qualquer usuário com conhecimento limitado de programação e pouca aprendizagem tecnológica pode criar “deepfakes” e esse tipo de produção tem desencadeado desafios aos profissionais forenses, que ainda encontram significativa diferença entre as “deepfakes” e os vídeos autênticos.
The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of ...trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the benchmark is based on Deep-Fakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size. The benchmark is publicly available and contains a hidden test set as well as a database of over 1.8 million manipulated images. This dataset is over an order of magnitude larger than comparable, publicly available, forgery datasets. Based on this data, we performed a thorough analysis of data-driven forgery detectors. We show that the use of additional domain-specific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.
Generative adversarial network (GAN) studies have grown exponentially in the past few years. Their impact has been seen mainly in the computer vision field with realistic image and video ...manipulation, especially generation, making significant advancements. Although these computer vision advances have garnered much attention, GAN applications have diversified across disciplines such as time series and sequence generation. As a relatively new niche for GANs, fieldwork is ongoing to develop high-quality, diverse, and private time series data. In this article, we review GAN variants designed for time series related applications. We propose a classification of discrete-variant GANs and continuous-variant GANs, in which GANs deal with discrete time series and continuous time series data. Here we showcase the latest and most popular literature in this field—their architectures, results, and applications. We also provide a list of the most popular evaluation metrics and their suitability across applications. Also presented is a discussion of privacy measures for these GANs and further protections and directions for dealing with sensitive data. We aim to frame clearly and concisely the latest and state-of-the-art research in this area and their applications to real-world technologies.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, SAZU, UL, UM, UPUK
Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called
seam carving
that supports content-aware ...image resizing for both reduction and expansion. A seam is an optimal 8-connected path of pixels on a
single
image from top to bottom, or left to right, where optimality is defined by an image energy function. By repeatedly carving out or inserting seams in one direction we can change the aspect ratio of an image. By applying these operators in both directions we can retarget the image to a new size. The selection and order of seams protect the content of the image, as defined by the energy function. Seam carving can also be used for image content enhancement and object removal. We support various visual saliency measures for defining the energy of an image, and can also include user input to guide the process. By storing the order of seams in an image we create
multi-size
images, that are able to continuously change in real time to fit a given size.
Estimating 6D poses of objects from images is an important problem in various applications such as robot manipulation and virtual reality. While direct regression of images to object poses has ...limited accuracy, matching rendered images of an object against the input image can produce accurate results. In this work, we propose a novel deep neural network for 6D pose matching named DeepIM. Given an initial pose estimation, our network is able to iteratively refine the pose by matching the rendered image against the observed image. The network is trained to predict a relative pose transformation using a disentangled representation of 3D location and 3D orientation and an iterative training process. Experiments on two commonly used benchmarks for 6D pose estimation demonstrate that DeepIM achieves large improvements over state-of-the-art methods. We furthermore show that DeepIM is able to match previously unseen objects.
Full text
Available for:
CEKLJ, DOBA, EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, IZUM, KILJ, KISLJ, MFDPS, NLZOH, NUK, OBVAL, OILJ, PILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, SIK, UILJ, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
During the last three decades, a series of key technological improvements turned atomic force microscopy (AFM) into a nanoscopic laboratory to directly observe and chemically characterize molecular ...and cell biological systems under physiological conditions. Here, we review key technological improvements that have established AFM as an analytical tool to observe and quantify native biological systems from the micro- to the nanoscale. Native biological systems include living tissues, cells, and cellular components such as single or complexed proteins, nucleic acids, lipids, or sugars. We showcase the procedures to customize nanoscopic chemical laboratories by functionalizing AFM tips and outline the advantages and limitations in applying different AFM modes to chemically image, sense, and manipulate biosystems at (sub)nanometer spatial and millisecond temporal resolution. We further discuss theoretical approaches to extract the kinetic and thermodynamic parameters of specific biomolecular interactions detected by AFM for single bonds and extend the discussion to multiple bonds. Finally, we highlight the potential of combining AFM with optical microscopy and spectroscopy to address the full complexity of biological systems and to tackle fundamental challenges in life sciences.
Full text
Available for:
IJS, KILJ, NUK, PNG, UL, UM
PDF
