The COVID-19 pandemic has resulted in a surge of fake news, creating public health risks. However, developing an effective way to detect such news is challenging, especially when published news ...involves mixing true and false information. Detecting COVID-19 fake news has become a critical task in the field of natural language processing (NLP). This paper explores the effectiveness of several machine learning algorithms and fine-tuning pre-trained transformer-based models, including Bidirectional Encoder Representations from Transformers (BERT) and COVID-Twitter-BERT (CT-BERT), for COVID-19 fake news detection. We evaluate the performance of different downstream neural network structures, such as CNN and BiGRU layers, added on top of BERT and CT-BERT with frozen or unfrozen parameters. Our experiments on a real-world COVID-19 fake news dataset demonstrate that incorporating BiGRU on top of the CT-BERT model achieves outstanding performance, with a state-of-the-art F1 score of 98%. These results have significant implications for mitigating the spread of COVID-19 misinformation and highlight the potential of advanced machine learning models for fake news detection.
This review emphasizes the critical need for accurate integration of solar energy into power grids. It meticulously examines the advancements in transformer models for solar forecasting, representing ...a confluence of renewable energy research and cutting-edge machine learning. It evaluates the effectiveness of various transformer architectures, including single, hybrid, and specialized models, across different forecasting horizons, from short to medium term. This review unveils substantial improvements in forecasting accuracy and computational efficiency, highlighting the models' proficiency in handling complex and diverse solar data. A key contribution is the emphasis on the crucial role of hyperparameters in refining model performance, balancing precision against computational demands. Importantly, the research also identifies critical challenges, such as the significant computational resources required and the need for expansive, high-quality datasets, which limit the broader application of these models. In response, this review advocates for future research directions focused on standardizing model configurations, venturing into longer-term forecasting, and fostering innovations to enhance computational economy. These proposed pathways aim to surmount current challenges, steering the domain towards more accurate, adaptable, and sustainable solar forecasting solutions that can contribute to achieving global renewable energy and climate objectives. This review not only maps the present landscape of transformer models in solar energy forecasting but also charts a trajectory for future advancements. It serves as a pivotal guide for researchers and practitioners, delineating the current advancements and future directions in navigating the complexities of solar data interpretation and forecasting, thereby significantly contributing to the development of reliable and efficient renewable energy systems.
•Addressing the critical need for accurate forecasting of solar irradiance and photovoltaic generation.•Examining transformer advancements in solar forecasting, critical for integrating solar energy into power grids.•Analyzing model capabilities in diverse environments, highlighting their role in forecasting accuracy and deployment issues.•Identifying future research in model standardization and computational efficiency to improve solar forecasting.
This pioneering study introduces the use of transformer-based machine learning models and explainable AI approaches to explore the impact of nutrition on Alzheimer's disease (AD) mortality. Using ...data from the Third National Health and Nutrition Examination Survey (Nhanes iii 1988 to 1994) and the NHANES III Mortality-Linked File (2019) databases, we investigate the intricate relationship between various nutritional factors and AD mortality. Our approach features a novel application of transformer models, which are then benchmarked against established methods like random forests and support vector machines. This comparison not only underscores the strengths of transformer models in handling complex medical datasets but also highlights their potential for providing deeper insights into disease progression. Key findings, such as the significant roles of Platelet distribution width in AD mortality in transformer and Serum Vitamin B12 in random forest, are enhanced by the use of Explainable Artificial Intelligence (XAI), particularly the Shapley Additive Explanations (SHAP) and the integrated gradient methods. This study serves as a vital step forward in applying advanced AI techniques to medical research, offering new perspectives in understanding and combating Alzheimer's Disease.
The total electricity consumption (TEC) reflects the operation condition of the national economy, and the prediction of the total electricity consumption can help track the economic development ...trend, as well as provide insights for macro policy making. Nowadays, novel neural networks provide a new perspective to predict the total electricity consumption. In this paper, a time series forecasting method based on Transformer model, Trans-T2V model, is proposed and applied to TEC forecasting. Transformer is the first network structure that completely relies on self-attention to calculate input and output. In this paper, the Time2vec method is used to improve the existing Transformer model, as embedding the month sequence more efficiently in the Transformer model. By comparing with the existing Transformer models and other intelligent algorithm models, the robustness and superiority of the proposed method framework are verified, and the highest accuracy reaches 97.36%. The method presented in this paper provides valuable insights in the field of time series prediction.
Abstract
Background
Social determinants of health are non-medical factors that influence health outcomes (SDOH). There is a wealth of SDOH information available in electronic health records, clinical ...reports, and social media data, usually in free text format. Extracting key information from free text poses a significant challenge and necessitates the use of natural language processing (NLP) techniques to extract key information.
Objective
The objective of this research is to advance the automatic extraction of SDOH from clinical texts.
Setting and data
The case reports of COVID-19 patients from the published literature are curated to create a corpus. A portion of the data is annotated by experts to create ground truth labels, and semi-supervised learning method is used for corpus re-annotation.
Methods
An NLP framework is developed and tested to extract SDOH from the free texts. A two-way evaluation method is used to assess the quantity and quality of the methods.
Results
The proposed NER implementation achieves an accuracy (F1-score) of 92.98% on our test set and generalizes well on benchmark data. A careful analysis of case examples demonstrates the superiority of the proposed approach in correctly classifying the named entities.
Conclusions
NLP can be used to extract key information, such as SDOH factors from free texts. A more accurate understanding of SDOH is needed to further improve healthcare outcomes.
Coronavirus disease (COVID-19) is an infectious disease, which is caused by the SARS-CoV-2 virus. Due to the growing literature on COVID-19, it is hard to get precise, up-to-date information about ...the virus. Practitioners, front-line workers, and researchers require expert-specific methods to stay current on scientific knowledge and research findings. However, there are a lot of research papers being written on the subject, which makes it hard to keep up with the most recent research. This problem motivates us to propose the design of the COVID-19 Search Engine (CO-SE), which is an algorithmic system that finds relevant documents for each query (asked by a user) and answers complex questions by searching a large corpus of publications. The CO-SE has a retriever component trained on the TF–IDF vectorizer that retrieves the relevant documents from the system. It also consists of a reader component that consists of a Transformer-based model, which is used to read the paragraphs and find the answers related to the query from the retrieved documents. The proposed model has outperformed previous models, obtaining an exact match ratio score of 71.45% and a semantic answer similarity score of 78.55%. It also outperforms other benchmark datasets, demonstrating the generalizability of the proposed approach.
•We propose a search engine design to search COVID-19 publications.•We curate recent COVID-19 literature to mine COVID-19 information.•We prepare a dataset in SQuAD format to fine-tune the DistilBERT for reading task.•We present a retriever component trained on TF–IDF vectorizer to retrieve relevant documents.•We present a reader component to find a short answer in response to a query from retrieved articles.
Attention-based transformer language models have shown significant performance gains in various natural language tasks. In this work, we explore the impact of transformer language models on the task ...of source code suggestion. The core intention of this work is to boost the modeling performance for the source code suggestion task and to explore how the training procedures and model architectures impact modeling performance. Additionally, we propose a transformer-based self-supervised learning technique called Transformer Gated Highway that outperforms recurrent and transformer language models of comparable size. The proposed approach combines the Transformer language model with Gated Highway introducing a notion of recurrence. We compare the performance of the proposed approach with transformer-based BERT (CodeTran), RoBERTa (RoBERTaCode), GPT2 (TravTrans), CodeGen and recurrent neural language-based LSTM (CodeLSTM) models. Moreover, we have experimented with various architectural settings for the transformer models to evaluate their impact on modeling performance. The extensive evaluation of the presented approach exhibits better performance on two programming language datasets; Java and C#. Additionally, we have adopted the presented approach for the syntax error correction task to predict the correct syntax token to render its possible implications for other source code modeling tasks.
•Transformer Gated Highway with two different self-supervised strategies.•Shown training procedures impact significantly on modeling performance.•Similar learning procedures with different model architectures, marginal difference•Extensive evaluation with two datasets (Java & C#) with five different baselines.
•Climate movements can permeate discourses around global climate change negotiations.•Youth climate activists have demonstrated moral power, shifting debates.•Computational analysis of Twitter data ...2014–2021 suggests normative change.•Activists as norm entrepreneurs rely on norm champions for global normative change.•Global norms such as anti-fossil fuel norms and climate justice are on the rise.
Youth Climate Activists are important norm entrepreneurs as humanity is increasingly awakening to the realities of accelerating climate change. They push for seeing climate change not merely through cost-benefit analysis frames but through frames of multiple climate justices. But how successful have these activists been in shifting perspectives in the context of international climate politics? This paper aims to investigate (1) to what extent the normative framework advanced by this movement is increasingly penetrating the international public climate debate, changing arguments, priorities, and frames around the annual UNFCCC COP conferences and (2) the key actors pushing for normative change. Using a unique and comprehensive Twitter dataset for the period between 2014 and 2021 revolving around the annual UNFCCC COP conferences and major youth climate protest events we combine various computational methods, including transformers-based topic modelling and social network analysis in this study. We find that indeed the normative framework advanced by the movement has successfully penetrated the discourse around UNFCCC and that youth climate activists were able gain support from central actors outside the movement, which is further contributing to the diffusion of their normative framework. We conclude that while these results demonstrate the moral power of youth climate activists, more research is needed to understand the influence on the actual negotiations outcomes
Previous research on emotional language relied heavily on off-the-shelf sentiment dictionaries that focus on negative and positive tone. These dictionaries are often tailored to nonpolitical domains ...and use bag-of-words approaches which come with a series of disadvantages. This paper creates, validates, and compares the performance of (1) a novel emotional dictionary specifically for political text, (2) locally trained word embedding models combined with simple neural network classifiers, and (3) transformer-based models which overcome limitations of the dictionary approach. All tools can measure emotional appeals associated with eight discrete emotions. The different approaches are validated on different sets of crowd-coded sentences. Encouragingly, the results highlight the strengths of novel transformer-based models, which come with easily available pretrained language models. Furthermore, all customized approaches outperform widely used off-the-shelf dictionaries in measuring emotional language in German political discourse.
