•An image-based poetry generation model with the unsupervised method.•The proposed model can generate diverse poetry from images.•The model combines reinforcement learning and contrastive learning ...methods.
Automatic poetry generation represents a typical exhibition of artificial intelligence creativity, and the cross-modal generation methods reveal a promising direction for improvement. Although previous methods have made some progress, they still suffer from the following challenges: (1) lack of annotated multimodal Chinese poetry datasets; (2) insufficient diversity of generated poetry; (3) inadequate semantic consistency between images and poems. In this paper, we propose a novel Unsupervised Image to Poetry Model (UI2P) with a newly designed generative adversarial network to address the above issues. Specifically, the unsupervised learning framework eliminates the dependence on annotated multimodal poetry datasets. We present a contrastive learning approach to optimize the diversity of generated poems. Furthermore, a consistency strategy is developed, including constructing a modern-classical concept dictionary to ensure semantic coherence between poems and images. Extensive experiments are conducted on the CCPC dataset, and the results with both automatic and manual evaluations demonstrate the superiority of our model compared with several state-of-the-art baselines.
PoeTryMe is a poetry generation system that autonomously produces poems from a set of initial parameters. After using it, creative writers and other users of this system expressed the desire to ...partake in the creative process by directly interacting with PoeTryMe. This paper is motivated by the previous impressions. It illustrates some of the challenges of automatic poetry generation, highlights limitations of PoeTryMe, and reports on recent efforts to address user feedback and provide alternative ways of using this system. First, making some functionalities available via a web API enabled their exploitation by other systems. Those include the generation of single lines as well as retrieval of related words, allowing for additional constraints on the number of syllables, sentiment and rhyme. On top of this API, a co-creative interface, Co-PoeTryMe, has been developed. Co-PoeTryMe allows users to either start from scratch or from a generated draft. Extensive editing functionality has been incorporated, in particular, allowing lines and words to be switched or replaced by newly generated alternatives. Co-PoeTryMe, its interface and the co-creative process for producing poetry are described, along with a user study that provided useful feedback. Users pointed out the strengths of this new system, including its capacities in providing inspiration, and giving the user the ability to customize an existing draft and visualize the changes, but also pointed out weaknesses, primarily by identifying potential improvements in the user interface.
In this paper, firstly, we start from the imagery in the poem, excavate the resources of the ideology and politics in the university language poetry, and put forward the feasible path of the ...integration of the ideology and politics of the university language course and the poetry culture of the family training. The research on poetry culture in university language is conducted using a deep learning model. Based on the deep learning model, keywords in the poetry text are extracted and extended, taking into account the issue of word splitting and lexical annotation. The sequence coding of word vectors is needed so as to construct a poetry generation model based on RNNPG, and then the research and analysis on the concept of ideology and politics of the poetry culture in the university language are carried out. The results show that in terms of model performance, the RNNPG model is 1.17 overall, and the overall generation effect is more ideal compared to other single discriminator generation models. The acceptance level of sophomores is generally above 0.29, which is better than that of freshmen, and the teaching of ancient poems under the perspective of “curriculum politics” is especially necessary to improve their cultural taste and aesthetic interest. This study is conducive to the personalized development of students and the formation of correct concepts.
The paper presents the Bairon system that supports the automatic generation of poetry. The proposed system allows generating a poem in the literary style of the selected writer using the user's input ...as the first line, and translating the given text into Shakespearean English. To accomplish that, GPT-2 and T5 language models were fine-tuned. We also propose easy to understand metrics to evaluate the quality of the generated poems and their similarity to the corresponding poet's original work, and to present the results. Additionally, the Poetry Turing Test with human participants was conducted to get another measure of quality of the generated poetry.
Chinese poetry has been a favorite literary genre for thousands of years. Chinese ancient poetry is still being read and practiced, and many famous ancient Chinese poets are honored and adorned. ...Recently, deep learning has been widely adopted for poetry generation. In this paper, we present a new context-aware Chinese poetry generation method based on sequence-to-sequence framework. We generate a new concept called keyword team, which is a combination of all the keywords to capture the context of the Chinese poetry. Then we use the keyword, the keyword team and the previously generated lines to generate the present line in the poetry. We find that, by including keyword teams into the generation of the poetry, it can additionally perceive the keywords of preceding and succeeding lines to generate the present line, which can effectively improve the adhesion among the overall lines. The comprehensive evaluation results show that our proposed model outperforms many of the state-of-the-art poetry generation models.
In the paper we present an approach for automatic lyrics generation. From the American National Corpus of written texts we build a Word Network, which encodes word sequences. Lyrics are then ...generated by performing a constrained random walk over the Word Network. The constraints include the structure of the generated sentence, the rhythm of the lines of the stanza or the rhymes of the stanza itself. Lyrics are generated using each constraint individually and also using all three constraints at the same time. We tested the single constraint strategies using a toy example, while the results of the joint strategy were subject to human review. While the given properties of the toy example, were kept in the results, replicating the toy example perfectly proved a difficult task. The results of the questionnaire showed that lack of a deeper meaning and strange capitalization were the main reasons that our results did not appear as though they were written by a human.
At present, most of the poetry generation models use keywords provided by users to generate poems that conform to the rules of rhythm and fluctuations in pitch. Because keywords contain less semantic ...information, it is difficult to guarantee the quality of generated poems, and the phenomenon of contextual theme shift is likely to occur. In response to this problem, this paper proposes a generative model based on conditional variational autoencoders, which can generate poems that are more in line with keyword descriptions and user satisfaction under the guidance of richer semantic information. By sampling human poetry and introducing additional semantic information related to keywords, the model effectively estimates the prior probability distribution of the conditional variational autoencoder, and generates a prior probability that more closely matches the true distribution. Because this model automatically expands keyword information, it narrows the gap between input and output semantic information, and alle
Machine poetry generation has been studied for decades, among which ancient Chinese poetry is still challenging in the field of poetry generation due to its unique regularity and rhythm. The quality ...improvement of ancient Chinese poetries is one of the most promising research areas of ancient Chinese Natural Language Processing. This paper proposes an ancient Chinese poetry polishing model, which is used for polishing to obtain high-quality ancient Chinese poetry. The model consists of a detection network and a correction network. The detection network based on BiLSTM and CRF is used to detect different types of low-quality words in poems. The correction network based on the BERT model is used to modify the detected low-quality words in the global context. The polishing process is iteratively performed until the model judges that there are no low-quality words in the poem. The results show that the polished poems are improved in multiple evaluations. Compared with existing polishing models, the model proposed in this paper performs better in both automatic evaluation and human evaluation when the number of parameters is reduced.
Images2Poem in different contexts with Dual‐CharRNN Yan, Jie; Xie, Yuxiang; Luan, Xidao
CAAI Transactions on Intelligence Technology,
December 2022, 2022-12-00, 2022-12-01, Letnik:
7, Številka:
4
Journal Article
Recenzirano
Odprti dostop
Image to caption has attracted extensive research attention recently. However, image to poetry, especially Chinese classical poetry, is much more challenging. Previous works mainly focus on ...generating coherent poetry without taking the contexts of poetry into account. In this paper, we propose an Images2Poem with the Dual‐CharRNN model which exploits images to generate Chinese classical poems in different contexts. Specifically, we first extract a few keywords representing elements from the given image based on multi‐label image classification. Then, these keywords are expanded to related ones with the planning‐based model. Finally, we employ Dual‐CharRNN to generate Chinese classical poetry in different contexts. A comprehensive evaluation of human judgements demonstrates that our model achieves promising performance and is effective in enhancing poetry's semantic consistency, readability, and aesthetics. We present an Images2Poem with the Dual‐CharRNN model exploiting images to generate Chinese classical poems in different contexts, which effectively improves the semantic consistency, readability and aesthetics of the generated poetry.
