Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies. Even then they ...struggle to predict rare or unseen words even if the context makes the prediction unambiguous. We introduce the pointer sentinel mixture architecture for neural sequence models which has the ability to either reproduce a word from the recent context or produce a word from a standard softmax classifier. Our pointer sentinel-LSTM model achieves state of the art language modeling performance on the Penn Treebank (70.9 perplexity) while using far fewer parameters than a standard softmax LSTM. In order to evaluate how well language models can exploit longer contexts and deal with more realistic vocabularies and larger corpora we also introduce the freely available WikiText corpus.
Existing approaches to neural machine translation condition each output word on previously generated outputs. We introduce a model that avoids this autoregressive property and produces its outputs in ...parallel, allowing an order of magnitude lower latency during inference. Through knowledge distillation, the use of input token fertilities as a latent variable, and policy gradient fine-tuning, we achieve this at a cost of as little as 2.0 BLEU points relative to the autoregressive Transformer network used as a teacher. We demonstrate substantial cumulative improvements associated with each of the three aspects of our training strategy, and validate our approach on IWSLT 2016 English-German and two WMT language pairs. By sampling fertilities in parallel at inference time, our non-autoregressive model achieves near-state-of-the-art performance of 29.8 BLEU on WMT 2016 English-Romanian.
Widespread Local House-Sparrow Populations Hole, David G; Whittingham, Mark J; Bradbury, Richard B ...
Nature (London),
08/2002, Letnik:
418, Številka:
6901
Journal Article
Recenzirano
Evidence indicates that populations of house sparrows (Passer domesticus) have declined radically throughout western Europe over the past decades. A study is described in which researchers utilized a ...combination of field experiments, genetic analysis, and demographic information to conclude that a reduction in winter food supply resulting from agricultural intensification is probably the primary reason behind widespread local extinctions of rural house sparrow populations in southern England. Findings from the study indicated that farmland populations are characterized by fine-level genetic structuring. Certain population are unable to sustain themselves (sinks), while other populations serve as sources. Implications of these findings are discussed.
In the macronucleus of the ciliate Oxytricha nova, telomeres end with single-stranded (T4G4)2 DNA bound to a heterodimeric telomeVe protein (αβ). Both the α and β subunits (α-TP and β-TP) were ...phosphory-lated in asynchronously growing Oxytricha; β-TP was phosphorylated to a much higher degree. In vitro, mouse cyclIn-dependent kineses (Cdks) phosphory-lated β-TP In a lyslne-rlch domain that Is not required for specific DNA binding but is implicated in higher order structure formation of telomeres. Therefore, phosphorylation of β-TP could modulate a function of the telomere protein that is separate from specific DNA binding. Phosphoamino acid analysis revealed that the mouse Cdks modify predominantly threonlne residues in β-TP, consistent with the observation that β-TP contains two consensus Cdk recognition sequences containing threonlne residues. In Xenopus egg extracts that undergo cell cycling, β-TP was phos-phorylated In M phase and dephosphorylated In interphase. This work provides the first direct evidence of phosphorylation at telomeres in any organism, as well as indirect evidence for cell cycle regulation of telomere phosphorylation. The Cdc2/cyclin A and Cdc2/cyclin B kinases are required for major m It otic events. An attractive model is that phosphorylation of β-TP by these kinases is required for the breakdown of telomere associations with each other and/or with nuclear structures prior to nuclear division.
Review Bradbury, Richard B; Hill, Ross A; Mason, David C ...
Ibis (London, England),
07/2005, Letnik:
147, Številka:
3
Journal Article
Recenzirano
Airborne LiDAR (Light Detection and Ranging) is a remote sensing technology that offers the ability to collect high horizontal sampling densities of high vertical resolution vegetation height data, ...over larger spatial extents than could be obtained by field survey. The influence of vegetation structure on the bird is a key mechanism underlying bird-habitat models. However, manual survey of vegetation structure becomes prohibitive in terms of time and cost if sampling needs to be of sufficient density to incorporate fine-grained heterogeneity at a landscape extent. We show that LiDAR data can help bridge the gap between grain and extent in organism-habitat models. Two examples are provided of bird-habitat models that use structural habitat information derived from airborne LiDAR data. First, it is shown that data on crop and field boundary height can be derived from LiDAR data, and so have the potential to predict the distribution of breeding Sky Larks in a farmed landscape. Secondly, LiDAR-retrieved canopy height and structural data are used to predict the breeding success of Great Tits and Blue Tits in broad-leaved woodland. LiDAR thus offers great potential for parameterizing predictive bird-habitat association models. This could be enhanced by the combination of LiDAR data with multispectral remote sensing data, which enables a wider range of habitat information to be derived, including both structural and compositional characteristics.
The quantity of small scale solar photovoltaic (PV) arrays in the United States has grown rapidly in recent years. As a result, there is substantial interest in high quality information about the ...quantity, power capacity, and energy generated by such arrays, including at a high spatial resolution (e.g., counties, cities, or even smaller regions). Unfortunately, existing methods for obtaining this information, such as surveys and utility interconnection filings, are limited in their completeness and spatial resolution. This work presents a computer algorithm that automatically detects PV panels using very high resolution color satellite imagery. The approach potentially offers a fast, scalable method for obtaining accurate information on PV array location and size, and at much higher spatial resolutions than are currently available. The method is validated using a very large (135 km^2) collection of publicly available 1 aerial imagery, with over 2,700 human annotated PV array locations. The results demonstrate the algorithm is highly effective on a per-pixel basis. It is likewise effective at object-level PV array detection, but with significant potential for improvement in estimating the precise shape/size of the PV arrays. These results are the first of their kind for the detection of solar PV in aerial imagery, demonstrating the feasibility of the approach and establishing a baseline performance for future investigations.
PDF
