Social media applications, such as Twitter and Facebook, allow users to communicate and share their thoughts, status updates, opinions, photographs, and videos around the globe. Unfortunately, some ...people utilize these platforms to disseminate hate speech and abusive language. The growth of hate speech may result in hate crimes, cyber violence, and substantial harm to cyberspace, physical security, and social safety. As a result, hate speech detection is a critical issue for both cyberspace and physical society, necessitating the development of a robust application capable of detecting and combating it in real-time. Hate speech detection is a context-dependent problem that requires context-aware mechanisms for resolution. In this study, we employed a transformer-based model for Roman Urdu hate speech classification due to its ability to capture the text context. In addition, we developed the first Roman Urdu pre-trained BERT model, which we named BERT-RU. For this purpose, we exploited the capabilities of BERT by training it from scratch on the largest Roman Urdu dataset consisting of 173,714 text messages. Traditional and deep learning models were used as baseline models, including LSTM, BiLSTM, BiLSTM + Attention Layer, and CNN. We also investigated the concept of transfer learning by using pre-trained BERT embeddings in conjunction with deep learning models. The performance of each model was evaluated in terms of accuracy, precision, recall, and F-measure. The generalization of each model was evaluated on a cross-domain dataset. The experimental results revealed that the transformer-based model, when directly applied to the classification task of the Roman Urdu hate speech, outperformed traditional machine learning, deep learning models, and pre-trained transformer-based models in terms of accuracy, precision, recall, and F-measure, with scores of 96.70%, 97.25%, 96.74%, and 97.89%, respectively. In addition, the transformer-based model exhibited superior generalization on a cross-domain dataset.
Radical-right campaigns commonly employ three discursive elements: anti-elite populism, exclusionary and declinist nationalism, and authoritarianism. Recent scholarship has explored whether these ...frames have diffused from radical-right to centrist parties in the latter’s effort to compete for the former’s voters. This study instead investigates whether similar frames had been used by mainstream political actors prior to their exploitation by the radical right (in the U.S., Donald Trump’s 2016 and 2020 campaigns). To do so, we identify instances of populism, nationalism (i.e., exclusionary and inclusive definitions of national symbolic boundaries and displays of low and high national pride), and authoritarianism in the speeches of Democratic and Republican presidential nominees between 1952 and 2020. These frames are subtle, infrequent, and polysemic, which makes their measurement difficult. We overcome this by leveraging the affordances of neural language models—in particular, a robustly optimized variant of bidirectional encoder representations from Transformers (RoBERTa) and active learning. As we demonstrate, this approach is more effective for measuring discursive frames than other methods commonly used by social scientists. Our results suggest that what set Donald Trump’s campaign apart from those of mainstream presidential candidates was not the invention of a new form of politics, but the combination of negative evaluations of elites, low national pride, and authoritarianism—all of which had long been present among both parties—with an explicit evocation of exclusionary nationalism, which had been articulated only implicitly by prior presidential nominees. Radical-right discourse—at least at the presidential level in the United States—should therefore be characterized not as a break with the past but as an amplification and creative rearrangement of existing political-cultural tropes.
The COVID-19 pandemic and its associated setbacks have significantly impacted human mental health. Depression of various intensities has resulted due to a wide variety of losses that people have ...experienced. However, unlike physical illness, mental illness is still underestimated by patients themselves and by society due to various factors, such as the societal stigma of visiting a psychotherapist and being diagnosed with a mental health disorder. On the other hand, general practitioners can recognize signs of depression using the Patient Health Questionnaire (PHQ-9), which is used as a screening test for depression. The PHQ-9 questionnaire comprises nine questions that correspond to the nine symptoms of depression outlined in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). In this paper, we aim to detect the nine depression symptoms stated by DSM-5 from Arabic tweets, as recognizing the type of depression symptom is crucial in diagnosing depression. We used AraBERT and MARBERT pretrained transformers to classify tweets with depression symptoms. We also performed data augmentation using ChatGPT to balance the training set. The model was applied to a dataset consisting of 1,290 samples labeled with nine different symptoms, in addition to a 'normal' class which was also generated using ChatGPT. This work used four performance metrics to evaluate the models' performance, which are accuracy, precision, recall, and F1 scores. The AraBERT and MARBERT transformers have yielded promising results, achieving accuracy, precision, recall, and F1 scores of 99.3%, 99.1%, 98.8%, and 98.9%, respectively, using the AraBERT transformer. While using the MARBERT transformer achieved accuracy, recall, precision, and F1 scores of 98.3%, 97.9%, 98.2%, and 98%, respectively.
The advent of attention-based architectures in medical imaging has ushered in an era of precision diagnostics, particularly in the detection and classification of brain tumors. This study introduced ...an innovative knowledge distillation framework employing a tripartite attention mechanism within transformer encoder models, specifically tailored for the identification of multiple brain tumor classes through magnetic resonance imaging (MRI). The proposed methodology synergistically harnesses the capabilities of large, highly parameterized teacher models to train more compact, efficient student models suitable for deployment in resource-constrained environments such as the internet of medical things and smart healthcare devices. Utilizing a diverse array of MRI sequences—including T1, contrast-enhanced T1, and T2—this study accounts for the nuanced variations across brain tumor classes derived from three extensive datasets. The tripartite attention mechanism addresses the limitation of traditional attention models by innovatively integrating temperature-softening neighborhood attention, global attention, and cross-attention layers. This sophisticated approach allows for a richer and more nuanced feature representation, capturing both local and global contextual information and intricate tumor features within MRI scans. This is supplemented by a unique augmentation pipeline and shifted patch tokenization technique, which enrich the model's input representation, especially for underrepresented classes. Through meticulous experimentation and ablation studies, the study demonstrates that the proposed model not only retains the robustness of its larger counterparts but also delivers enhanced performance metrics. When juxtaposed with benchmarking models—including traditional deep CNNs and various transformer-based architectures—the proposed model consistently showcases superior results. Its effectiveness is reflected in its lower teacher and student losses, commendable Brier scores, and noteworthy top-1 and top-5 accuracies, as well as AUC metrics across all datasets. This paper not only validates the efficacy of knowledge distillation in complex medical image analysis tasks but also provides a promising pathway for the integration of cutting-edge AI techniques in real-world clinical applications, potentially revolutionizing the early detection and treatment of brain tumors.
The COVID-19 pandemic and its associated setbacks have significantly impacted human mental health. Depression of various intensities has resulted due to a wide variety of losses that people have ...experienced. However, unlike physical illness, mental illness is still underestimated by patients themselves and by society due to various factors, such as the societal stigma of visiting a psychotherapist and being diagnosed with a mental health disorder. On the other hand, general practitioners can recognize signs of depression using the Patient Health Questionnaire (PHQ-9), which is used as a screening test for depression. The PHQ-9 questionnaire comprises nine questions that correspond to the nine symptoms of depression outlined in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). In this paper, we aim to detect the nine depression symptoms stated by DSM-5 from Arabic tweets, as recognizing the type of depression symptom is crucial in diagnosing depression. We used AraBERT and MARBERT pretrained transformers to classify tweets with depression symptoms. We also performed data augmentation using ChatGPT to balance the training set. The model was applied to a dataset consisting of 1,290 samples labeled with nine different symptoms, in addition to a ‘normal’ class which was also generated using ChatGPT. This work used four performance metrics to evaluate the models’ performance, which are accuracy, precision, recall, and F1 scores. The AraBERT and MARBERT transformers have yielded promising results, achieving accuracy, precision, recall, and F1 scores of 99.3%, 99.1%, 98.8%, and 98.9%, respectively, using the AraBERT transformer. While using the MARBERT transformer achieved accuracy, recall, precision, and F1 scores of 98.3%, 97.9%, 98.2%, and 98%, respectively.
While scientific knowledge of post-COVID-19 condition (PCC) is growing, there remains significant uncertainty in the definition of the disease, its expected clinical course, and its impact on daily ...functioning. Social media platforms can generate valuable insights into patient-reported health outcomes as the content is produced at high resolution by patients and caregivers, representing experiences that may be unavailable to most clinicians.
In this study, we aimed to determine the validity and effectiveness of advanced natural language processing approaches built to derive insight into PCC-related patient-reported health outcomes from social media platforms Twitter and Reddit. We extracted PCC-related terms, including symptoms and conditions, and measured their occurrence frequency. We compared the outputs with human annotations and clinical outcomes and tracked symptom and condition term occurrences over time and locations to explore the pipeline's potential as a surveillance tool.
We used bidirectional encoder representations from transformers (BERT) models to extract and normalize PCC symptom and condition terms from English posts on Twitter and Reddit. We compared 2 named entity recognition models and implemented a 2-step normalization task to map extracted terms to unique concepts in standardized terminology. The normalization steps were done using a semantic search approach with BERT biencoders. We evaluated the effectiveness of BERT models in extracting the terms using a human-annotated corpus and a proximity-based score. We also compared the validity and reliability of the extracted and normalized terms to a web-based survey with more than 3000 participants from several countries.
UmlsBERT-Clinical had the highest accuracy in predicting entities closest to those extracted by human annotators. Based on our findings, the top 3 most commonly occurring groups of PCC symptom and condition terms were systemic (such as fatigue), neuropsychiatric (such as anxiety and brain fog), and respiratory (such as shortness of breath). In addition, we also found novel symptom and condition terms that had not been categorized in previous studies, such as infection and pain. Regarding the co-occurring symptoms, the pair of fatigue and headaches was among the most co-occurring term pairs across both platforms. Based on the temporal analysis, the neuropsychiatric terms were the most prevalent, followed by the systemic category, on both social media platforms. Our spatial analysis concluded that 42% (10,938/26,247) of the analyzed terms included location information, with the majority coming from the United States, United Kingdom, and Canada.
The outcome of our social media-derived pipeline is comparable with the results of peer-reviewed articles relevant to PCC symptoms. Overall, this study provides unique insights into patient-reported health outcomes of PCC and valuable information about the patient's journey that can help health care providers anticipate future needs.
RR2-10.1101/2022.12.14.22283419.
Large Language Models (LLMs) such as General Pretrained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT), which use transformer model architectures, have ...significantly advanced artificial intelligence and natural language processing. Recognized for their ability to capture associative relationships between words based on shared context, these models are poised to transform healthcare by improving diagnostic accuracy, tailoring treatment plans, and predicting patient outcomes. However, there are multiple risks and potentially unintended consequences associated with their use in healthcare applications. This study, conducted with 28 participants using a qualitative approach, explores the benefits, shortcomings, and risks of using transformer models in healthcare. It analyses responses to seven open-ended questions using a simplified thematic analysis. Our research reveals seven benefits, including improved operational efficiency, optimized processes and refined clinical documentation. Despite these benefits, there are significant concerns about the introduction of bias, auditability issues and privacy risks. Challenges include the need for specialized expertise, the emergence of ethical dilemmas and the potential reduction in the human element of patient care. For the medical profession, risks include the impact on employment, changes in the patient-doctor dynamic, and the need for extensive training in both system operation and data interpretation.
