How Can We Know What Language Models Know? Jiang, Zhengbao; Xu, Frank F.; Araki, Jun ...
Transactions of the Association for Computational Linguistics,
01/2020, Volume:
8
Journal Article
Peer reviewed
Open access
Recent work has presented intriguing results examining the knowledge contained in language models (LMs) by having the LM fill in the blanks of prompts such as “
”. These prompts are usually manually ...created, and quite possibly sub-optimal; another prompt such as “
__ ” may result in more accurately predicting the correct profession. Because of this, given an inappropriate prompt, we might fail to retrieve facts that the LM
know, and thus any given prompt only provides a lower bound estimate of the knowledge contained in an LM. In this paper, we attempt to more accurately estimate the knowledge contained in LMs by automatically discovering better prompts to use in this querying process. Specifically, we propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. Extensive experiments on the LAMA benchmark for extracting relational knowledge from LMs demonstrate that our methods can improve accuracy from 31.1% to 39.6%, providing a tighter lower bound on what LMs know. We have released the code and the resulting LM Prompt And Query Archive (LPAQA) at
.
Corticosteroids are commonly used as adjuvant therapy for acute respiratory distress syndrome by many clinicians because of their perceived anti-inflammatory effects. However, for patients with ...severe viral pneumonia, the corticosteroid treatment is highly controversial.
The purpose of this review is to systematically evaluate the effect and potential mechanism of corticosteroid administration in pandemic viral pneumonia.
We comprehensively searched all manuscripts on corticosteroid therapy for influenza, severe acute respiratory syndrome (SARS), Middle East respiratory syndrome (MERS) and SARS coronavirus 2 (SARS-CoV-2) viral pneumonia from the PubMed, EMBASE, Web of Science and Cochrane Library databases.
We systematically summarized the effects of corticosteroid therapy for pandemic viral pneumonia and the potential mechanism of action for corticosteroids in coronavirus disease 2019 (COVID-19).
Observational studies showed that corticosteroid treatment was associated with increased mortality and nosocomial infections for influenza and delayed virus clearance for SARS-CoV and MERS-CoV. Limited data on corticosteroid therapy for COVID-19 were reported. Corticosteroids were used in about a fifth of patients (670/2995, 22.4%). Although clinical observational studies reported the improvement in symptoms and oxygenation for individuals with severe COVID-19 who received corticosteroid therapy, case fatality rate in the corticosteroid group was significantly higher than that in the non-corticosteroid group (69/443, 15.6% versus 56/1310, 4.3%). Compared individuals with non-severe disease, those with severe disease were more likely to receive corticosteroid therapy (201/382, 52.6% versus 201/1310, 15.3%). Although there is no evidence that corticosteroid therapy reduces mortality in people with COVID-19, some improvements in clinical symptoms and oxygenation were reported in some clinical observational studies. Excessive inflammatory response and lymphopenia might be critical factors associated with severity of and mortality from COVID-19. Sufficiently powered randomized controlled trials with rigorous inclusion/exclusion criteria and standardized dose and duration of corticosteroids are needed to verify the effectiveness and safety of corticosteroid therapy.
During eukaryotic evolution, ribosomes have considerably increased in size, forming a surface-exposed ribosomal RNA (rRNA) shell of unknown function, which may create an interface for yet ...uncharacterized interacting proteins. To investigate such protein interactions, we establish a ribosome affinity purification method that unexpectedly identifies hundreds of ribosome-associated proteins (RAPs) from categories including metabolism and cell cycle, as well as RNA- and protein-modifying enzymes that functionally diversify mammalian ribosomes. By further characterizing RAPs, we discover the presence of ufmylation, a metazoan-specific post-translational modification (PTM), on ribosomes and define its direct substrates. Moreover, we show that the metabolic enzyme, pyruvate kinase muscle (PKM), interacts with sub-pools of endoplasmic reticulum (ER)-associated ribosomes, exerting a non-canonical function as an RNA-binding protein in the translation of ER-destined mRNAs. Therefore, RAPs interconnect one of life’s most ancient molecular machines with diverse cellular processes, providing an additional layer of regulatory potential to protein expression.
Display omitted
•A mammalian ribosome affinity approach reveals ribosome-associated proteins (RAPs)•A multitude of RAPs link the ribosome to diverse cellular and molecular functions•Ribosomes are modified by metazoan-specific ufmylation•A metabolic enzyme, PKM, is at ER ribosomes and translates ER-destined mRNAs
Functionally diverse proteins associate with mammalian ribosomes, and this repertoire differs with the subcellular localization of ribosomes and guides transcript-specific translation.
Each year, a number of typhoons in the western North Pacific pass through the Luzon Strait into South China Sea (SCS). Although the storms remain above a warm open sea, the majority of them weaken ...due to atmospheric and oceanic environments unfavorable for typhoon intensification in SCS, which therefore serves as a natural buffer that shields the surrounding coasts from potentially more powerful storms. This study examines how this buffer has changed over inter-decadal and longer time scales. We show that the buffer weakens (i.e. greater potential for more powerful typhoons) in negative Pacific Decadal Oscillation (PDO) years, as well as with sea-level-rise and surface warming, caused primarily by the deepening of the ocean's 26 °C isotherm Z
. A new Intensity Change Index is proposed to describe the typhoon intensity change as a function of Z
and other environmental variables. In SCS, the new index accounts for as high as 75% of the total variance of typhoon intensity change.
Topological Dirac and Weyl semimetals not only host quasiparticles analogous to the elementary fermionic particles in high-energy physics, but also have a non-trivial band topology manifested by ...gapless surface states, which induce exotic surface Fermi arcs1,2. Recent advances suggest new types of topological semimetal, in which spatial symmetries protect gapless electronic excitations without high-energy analogues3–11. Here, using angle-resolved photoemission spectroscopy, we observe triply degenerate nodal points near the Fermi level of tungsten carbide with space group \P\bar{6}m2\ (no. 187), in which the low-energy quasiparticles are described as three-component fermions distinct from Dirac and Weyl fermions. We further observe topological surface states, whose constant-energy contours constitute pairs of ‘Fermi arcs’ connecting to the surface projections of the triply degenerate nodal points, proving the non-trivial topology of the newly identified semimetal state.
Schizophrenia (SCZ) and bipolar disorder (BPD) are severe mental disorders with high heritability. Clinicians have long noticed the similarities of clinic symptoms between these disorders. In recent ...years, accumulating evidence indicates some shared genetic liabilities. However, what is shared remains elusive. In this study, we conducted whole transcriptome analysis of post-mortem brain tissues (cingulate cortex) from SCZ, BPD and control subjects, and identified differentially expressed genes in these disorders. We found 105 and 153 genes differentially expressed in SCZ and BPD, respectively. By comparing the t-test scores, we found that many of the genes differentially expressed in SCZ and BPD are concordant in their expression level (q⩽0.01, 53 genes; q⩽0.05, 213 genes; q⩽0.1, 885 genes). Using genome-wide association data from the Psychiatric Genomics Consortium, we found that these differentially and concordantly expressed genes were enriched in association signals for both SCZ (P<10(-7)) and BPD (P=0.029). To our knowledge, this is the first time that a substantially large number of genes show concordant expression and association for both SCZ and BPD. Pathway analyses of these genes indicated that they are involved in the lysosome, Fc gamma receptor-mediated phagocytosis, regulation of actin cytoskeleton pathways, along with several cancer pathways. Functional analyses of these genes revealed an interconnected pathway network centered on lysosomal function and the regulation of actin cytoskeleton. These pathways and their interacting network were principally confirmed by an independent transcriptome sequencing data set of the hippocampus. Dysregulation of lysosomal function and cytoskeleton remodeling has direct impacts on endocytosis, phagocytosis, exocytosis, vesicle trafficking, neuronal maturation and migration, neurite outgrowth and synaptic density and plasticity, and different aspects of these processes have been implicated in SCZ and BPD.
HERV-K (human endogenous retrovirus type K) type 1-encoded Np9 is a tumor-specific biomarker, but its oncogenic role and targets in human leukemia remain elusive. We first identified Np9 as a potent ...viral oncogene in human leukemia. Silencing of Np9 inhibited the growth of myeloid and lymphoblastic leukemic cells, whereas expression of Np9 significantly promoted the growth of leukemia cells in vitro and in vivo. Np9 not only activated ERK, AKT and Notch1 pathways but also upregulated β-catenin essential for survival of leukemia stem cells. In human leukemia, Np9 protein level in leukemia patients was substantially higher than that in normal donors (56% vs 4.5%). Moreover, Np9 protein level was correlated with the number of leukemia stem/progenitor cells but not detected in normal CD34(+) hematopoietic stem cells. In addition, Np9-positive samples highly expressed leukemia-specific pol-env polyprotein, env and transmembrane proteins as well as viral particles. Thus, the viral oncogene Np9 is a critical molecular switch of multiple signaling pathways regulating the growth of leukemia stem/progenitor cells. These findings open a new perspective to understand the etiology of human common leukemia and provide a novel target for treating leukemia.
The design of efficient gene delivery vectors is a challenging task in gene therapy. Recent progress in living/controlled radical polymerizations (LRPs), in particular atom transfer radical ...polymerization (ATRP) and reversible addition-fragmentation chain transfer (RAFT) polymerization providing a means for the design and synthesis of new polymeric gene vectors with well-defined compositions, architectures and functionalities is reviewed here. Polymeric gene vectors with different architectures, including homopolymers, block copolymers, graft copolymers, and star-shaped polymers, are conveniently prepared
via ATRP and RAFT polymerization. The corresponding synthesis strategies are described in detail. The recent research activities indicate that ATRP and RAFT polymerization have become essential tools for the design and synthesis of advanced, noble and novel gene carriers.
Recent progress in controlled radical polymerizations, in particular atom transfer radical polymerization (ATRP), has provided a unique means for the design and synthesis of bioactive surfaces and ...functional biomaterials. This review summarizes such recent research activities. The synthesis strategies of bioactive surfaces and biomaterials via ATRP are described in detail. The highly robust and versatile ATRP technique is particularly suited for the preparation of functional bioactive surfaces, including antifouling, antibacterial, stimuli-responsive, biomolecule-coupled and micropatterned surfaces. In addition to bioactive surfaces, ATRP has also been widely used for the preparation of well-structured functional biomaterials, such as micellar delivery systems, hydrogels, cationic gene carriers and polymer–protein conjugates. The research activities in the last decade indicate that ATRP has become an essential tool for the design and synthesis of advanced, noble and novel biomaterials.
PDF
