Automatically describing the content of an image is a challenging task in computer vision that connects the machine learning and natural language processing. In this paper, we present a framework, ...based on modeling image context, to generate natural sentences describing an image, which consists of two parts: relation modeling and description generating. By modeling the mapping from image spatial context to the logical relationship between objects, the former is trained to maximize the likelihood of the target linguistics phrase describing the relationship between object given the training image. By taking the the advantages of the syntactic-tree based method, the latter takes the predicted relationships as key ingredients to facilitate the image description generation within tree-growth process. We conduct extensive experimental evaluations on MS COCO dataset. Our framework outperforms the state-of-the-art methods. The results demonstrates that our framework provides robust and significant improvements for the relationship prediction between objects and the image description generation.
How do global warming and agriculture influence each other? It is possible to answer the question by searching knowledge about the relationship between global warming and agriculture. As exemplified ...by this question, strong demands exist for searching relationships between objects. Mining knowledge about relationships on Wikipedia has been studied. However, it is desired to search more diverse knowledge about relationships on the Web. By utilizing the objects constituting relationships mined from Wikipedia, we propose a new method to search images with surrounding text that include knowledge about relationships on the Web. Experimental results show that our method is effective and applicable in searching knowledge about relationships. We also construct a relationship search system named “Enishi” based on the proposed new method. Enishi supplies a wealth of diverse knowledge including images with surrounding text to help users to understand relationships deeply, by complementarily utilizing knowledge from Wikipedia and the Web.
Abstract
A syntactic analysis of Gen 1:27 shows that this verse not only speaks about a sexual distinction in humanity, it also clearly indicates that God consists of both male and female genders.
PDF
