Display omitted
•We survey methods of representing clinical text using neural networks.•We provide a “how-to” guide for training these representations on clinical text.•We describe word models, ...corpora, evaluation methods, and applications.
Representing words as numerical vectors based on the contexts in which they appear has become the de facto method of analyzing text with machine learning. In this paper, we provide a guide for training these representations on clinical text data, using a survey of relevant research. Specifically, we discuss different types of word representations, clinical text corpora, available pre-trained clinical word vector embeddings, intrinsic and extrinsic evaluation, applications, and limitations of these approaches. This work can be used as a blueprint for clinicians and healthcare workers who may want to incorporate clinical text features in their own models and applications.
Over the last several years, the field of natural language processing has been propelled forward by an explosion in the use of deep learning models. This article provides a brief introduction to the ...field and a quick overview of deep learning architectures and methods. It then sifts through the plethora of recent studies and summarizes a large assortment of relevant contributions. Analyzed research areas include several core linguistic processing issues in addition to many applications of computational linguistics. A discussion of the current state of the art is then provided along with recommendations for future research in the field.
Figurative language generation (FLG) is the task of reformulating a given text to include a desired figure of speech, such as a hyperbole, a simile, and several others, while still being faithful to ...the original context. This is a fundamental, yet challenging task in Natural Language Processing (NLP), which has recently received increased attention due to the promising performance brought by pre-trained language models. Our survey provides a systematic overview of the development of FLG, mostly in English, starting with the description of some common figures of speech, their corresponding generation tasks, and datasets. We then focus on various modelling approaches and assessment strategies, leading us to discussing some challenges in this field, and suggesting some potential directions for future research. To the best of our knowledge, this is the first survey that summarizes the progress of FLG including the most recent development in NLP. We also organize corresponding resources, e.g., article lists and datasets, and make them accessible in an open repository. We hope this survey can help researchers in NLP and related fields to easily track the academic frontier, providing them with a landscape and a roadmap of this area.
Attention in Natural Language Processing Galassi, Andrea; Lippi, Marco; Torroni, Paolo
IEEE transaction on neural networks and learning systems,
10/2021, Volume:
32, Issue:
10
Journal Article
Open access
Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced ...advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data. We propose a taxonomy of attention models according to four dimensions: the representation of the input, the compatibility function, the distribution function, and the multiplicity of the input and/or output. We present the examples of how prior information can be exploited in attention models and discuss ongoing research efforts and open challenges in the area, providing the first extensive categorization of the vast body of literature in this exciting domain.
Explainable Natural Language Processing by Anders Søgaard ( University of Copenhagen). Morgan & Claypool (Synthesis Lectures on Human Language Technologies, edited by Graeme Hirst, volume 51), 2021, ...xvi + 107 pp; paperback, ISBN: 9781636392134; ebook, ISBN: 9781636392141; hardcover, ISBN: 9781636392158 DOI:10.2200/S01118ED1V01Y202107HLT051.
This open access book provides an overview of the recent advances in representation learning theory, algorithms and applications for natural language processing (NLP). It is divided into three parts. ...Part I presents the representation learning techniques for multiple language entries, including words, phrases, sentences and documents. Part II then introduces the representation techniques for those objects that are closely related to NLP, including entity-based world knowledge, sememe-based linguistic knowledge, networks, and cross-modal entries. Lastly, Part III provides open resource tools for representation learning techniques, and discusses the remaining challenges and future research directions. The theories and algorithms of representation learning presented can also benefit other related domains such as machine learning, social network analysis, semantic Web, information retrieval, data mining and computational biology. This book is intended for advanced undergraduate and graduate students, post-doctoral fellows, researchers, lecturers, and industrial engineers, as well as anyone interested in representation learning and natural language processing.
In recent years, artificial intelligence (AI) and machine learning have been transforming the landscape of scientific research. Out of which, the chatbot technology has experienced tremendous ...advancements in recent years, especially with ChatGPT emerging as a notable AI language model. This comprehensive review delves into the background, applications, key challenges, and future directions of ChatGPT. We begin by exploring its origins, development, and underlying technology, before examining its wide-ranging applications across industries such as customer service, healthcare, and education. We also highlight the critical challenges that ChatGPT faces, including ethical concerns, data biases, and safety issues, while discussing potential mitigation strategies. Finally, we envision the future of ChatGPT by exploring areas of further research and development, focusing on its integration with other technologies, improved human-AI interaction, and addressing the digital divide. This review offers valuable insights for researchers, developers, and stakeholders interested in the ever-evolving landscape of AI-driven conversational agents. This study explores the various ways ChatGPT has been revolutionizing scientific research, spanning from data processing and hypothesis generation to collaboration and public outreach. Furthermore, the paper examines the potential challenges and ethical concerns surrounding the use of ChatGPT in research, while highlighting the importance of striking a balance between AI-assisted innovation and human expertise. The paper presents several ethical issues in existing computing domain and how ChatGPT can invoke challenges to such notion. This work also includes some biases and limitations of ChatGPT. It is worth to note that despite of several controversies and ethical concerns, ChatGPT has attracted remarkable attentions from academia, research, and industries in a very short span of time.
Readmission within 30 days is a prevalent issue among elderly patients, linked to unfavorable health outcomes. Our objective was to develop and validate multimodal machine learning models for ...predicting 30-day readmission risk in elderly patients discharged from internal medicine departments.
This was a retrospective cohort study which included elderly patients aged 75 or older, who were hospitalized at the Hadassah Medical Center internal medicine departments between 2014 and 2020. Three machine learning algorithms were developed and employed to predict 30-day readmission risk. The primary measures were predictive model performance scores, specifically area under the receiver operator curve (AUROC), and average precision.
This study included 19,569 admissions. Of them, 3258 (16.65%) resulted in 30-day readmission. Our 3 proposed models demonstrated high accuracy and precision on an unseen test set, with AUROC values of 0.87, 0.89, and 0.93, respectively, and average precision values of 0.76, 0.78, and 0.81. Feature importance analysis revealed that the number of admissions in the past year, history of 30-day readmission, Charlson score, and admission length were the most influential variables. Notably, the natural language processing score, representing the probability of readmission according to a textual-based model trained on social workers' assessment letters during hospitalization, ranked among the top 10 contributing factors.
Leveraging multimodal machine learning offers a promising strategy for identifying elderly patients who are at high risk for 30-day readmission. By identifying these patients, machine learning models may facilitate the effective execution of preventive actions to reduce avoidable readmission incidents.
PDF
