With the growth of large photometric surveys, accurately estimating photometric redshifts, preferably as a probability density function (PDF), and fully understanding the implicit systematic ...uncertainties in this process, has become increasingly important. In this paper, we present a new, publicly available, parallel, machine learning algorithm that generates photometric redshift PDFs by using prediction trees and random forest techniques, which we have named TPZ.1 This new algorithm incorporates measurement errors into the calculation while also dealing efficiently with missing values in the data. In addition, our implementation of this algorithm provides supplementary information regarding the data being analysed, including unbiased estimates of the accuracy of the technique without resorting to a validation data set, identification of poor photometric redshift areas within the parameter space occupied by the spectroscopic training data, a quantification of the relative importance of the variables used to construct the PDF, and a robust identification of outliers. This extra information can be used to optimally target new spectroscopic observations and to improve the overall efficacy of the redshift estimation. We have tested TPZ on galaxy samples drawn from the Sloan Digital Sky Survey (SDSS) main galaxy sample and from the Deep Extragalactic Evolutionary Probe-2 (DEEP2) survey, obtaining excellent results in each case. We also have tested our implementation by participating in the PHAT1 project, which is a blind photometric redshift contest, finding that TPZ performs comparable to if not better than other empirical photometric redshift algorithms. Finally, we discuss the various parameters that control the operation of TPZ, the specific limitations of this approach and an application of photometric redshift PDFs.
ABSTRACT
We present a novel method of robust probabilistic cosmic web particle classification in three dimensions using a supervised machine learning algorithm. Training data were generated using a ...simplified ΛCDM toy model with pre-determined algorithms for generating haloes, filaments, and voids. While this framework is not constrained by physical modelling, it can be generated substantially more quickly than an N-body simulation without loss in classification accuracy. For each particle in this data set, measurements were taken of the local density field magnitude and directionality. These measurements were used to train a random forest algorithm, which was used to assign class probabilities to each particle in a ΛCDM, dark matter-only N-body simulation with 2563 particles, as well as on another toy model data set. By comparing the trends in the ROC curves and other statistical metrics of the classes assigned to particles in each data set using different feature sets, we demonstrate that the combination of measurements of the local density field magnitude and directionality enables accurate and consistent classification of halo, filament, and void particles in varied environments. We also show that this combination of training features ensures that the construction of our toy model does not affect classification. The use of a fully supervised algorithm allows greater control over the information deemed important for classification, preventing issues arising from arbitrary hyperparameters and mode collapse in deep learning models. Due to the speed of training data generation, our method is highly scalable, making it particularly suited for classifying large data sets, including observed data.
ABSTRACT
The next generation of wide-field deep astronomical surveys will deliver unprecedented amounts of images through the 2020s and beyond. As both the sensitivity and depth of observations ...increase, more blended sources will be detected. This reality can lead to measurement biases that contaminate key astronomical inferences. We implement new deep learning models available through Facebook AI Research’s detectron2 repository to perform the simultaneous tasks of object identification, deblending, and classification on large multiband co-adds from the Hyper Suprime-Cam (HSC). We use existing detection/deblending codes and classification methods to train a suite of deep neural networks, including state-of-the-art transformers. Once trained, we find that transformers outperform traditional convolutional neural networks and are more robust to different contrast scalings. Transformers are able to detect and deblend objects closely matching the ground truth, achieving a median bounding box Intersection over Union of 0.99. Using high-quality class labels from the Hubble Space Telescope, we find that when classifying objects as either stars or galaxies, the best-performing networks can classify galaxies with near 100 per cent completeness and purity across the whole test sample and classify stars above 60 per cent completeness and 80 per cent purity out to HSC i-band magnitudes of 25 mag. This framework can be extended to other upcoming deep surveys such as the Legacy Survey of Space and Time and those with the Roman Space Telescope to enable fast source detection and measurement. Our code, deepdisc, is publicly available at https://github.com/grantmerz/deepdisc.
In this paper, we explore the applicability of the unsupervised machine learning technique of self-organizing maps (SOM) to estimate galaxy photometric redshift probability density functions (PDFs). ...This technique takes a spectroscopic training set, and maps the photometric attributes, but not the redshifts, to a two-dimensional surface by using a process of competitive learning where neurons compete to more closely resemble the training data multidimensional space. The key feature of a SOM is that it retains the topology of the input set, revealing correlations between the attributes that are not easily identified. We test three different 2D topological mapping: rectangular, hexagonal and spherical, by using data from the Deep Extragalactic Evolutionary Probe 2 survey. We also explore different implementations and boundary conditions on the map and also introduce the idea of a random atlas, where a large number of different maps are created and their individual predictions are aggregated to produce a more robust photometric redshift PDF. We also introduced a new metric, the I-score, which efficiently incorporates different metrics, making it easier to compare different results (from different parameters or different photometric redshift codes). We find that by using a spherical topology mapping we obtain a better representation of the underlying multidimensional topology, which provides more accurate results that are comparable to other, state-of-the-art machine learning algorithms. Our results illustrate that unsupervised approaches have great potential for many astronomical problems, and in particular for the computation of photometric redshifts.
The estimation and utilization of photometric redshift probability density functions (photo-z PDFs) have become increasingly important over the last few years and currently there exist a wide variety ...of algorithms to compute photo-z's, each with their own strengths and weaknesses. In this paper, we present a novel and efficient Bayesian framework that combines the results from different photo-z techniques into a more powerful and robust estimate by maximizing the information from the photometric data. To demonstrate this, we use a supervised machine learning technique based on random forest, an unsupervised method based on self-organizing maps, and a standard template-fitting method but can be easily extended to other existing techniques. We use data from the DEEP2 survey and the Sloan Digital Sky Survey to explore different methods for combining the predictions from these techniques. By using different performance metrics, we demonstrate that we can improve the accuracy of our final photo-z estimate over the best input technique, that the fraction of outliers is reduced, and that the identification of outliers is significantly improved when we apply a naïve Bayes classifier to this combined information. Our more robust and accurate photo-z PDFs will allow even more precise cosmological constraints to be made by using current and future photometric surveys. These improvements are crucial as we move to analyse photometric data that push to or even past the limits of the available training data, which will be the case with the Large Synoptic Survey Telescope.
ABSTRACT
During the last decade, there has been an explosive growth in survey data and deep learning techniques, both of which have enabled great advances for astronomy. The amount of data from ...various surveys from multiple epochs with a wide range of wavelengths, albeit with varying brightness and quality, is overwhelming, and leveraging information from overlapping observations from different surveys has limitless potential in understanding galaxy formation and evolution. Synthetic galaxy image generation using physical models has been an important tool for survey data analysis, while deep learning generative models show great promise. In this paper, we present a novel approach for robustly expanding and improving survey data through cross survey feature translation. We trained two types of neural networks to map images from the Sloan Digital Sky Survey (SDSS) to corresponding images from the Dark Energy Survey (DES). This map was used to generate false DES representations of SDSS images, increasing the brightness and S/N while retaining important morphological information. We substantiate the robustness of our method by generating DES representations of SDSS images from outside the overlapping region, showing that the brightness and quality are improved even when the source images are of lower quality than the training images. Finally, we highlight images in which the reconstruction process appears to have removed large artefacts from SDSS images. While only an initial application, our method shows promise as a method for robustly expanding and improving the quality of optical survey data and provides a potential avenue for cross-band reconstruction.
There exist a variety of star–galaxy classification techniques, each with their own strengths and weaknesses. In this paper, we present a novel meta-classification framework that combines and fully ...exploits different techniques to produce a more robust star–galaxy classification. To demonstrate this hybrid, ensemble approach, we combine a purely morphological classifier, a supervised machine learning method based on random forest, an unsupervised machine learning method based on self-organizing maps, and a hierarchical Bayesian template-fitting method. Using data from the CFHTLenS survey (Canada–France–Hawaii Telescope Lensing Survey), we consider different scenarios: when a high-quality training set is available with spectroscopic labels from DEEP2 (Deep Extragalactic Evolutionary Probe Phase 2 ), SDSS (Sloan Digital Sky Survey), VIPERS (VIMOS Public Extragalactic Redshift Survey), and VVDS (VIMOS VLT Deep Survey), and when the demographics of sources in a low-quality training set do not match the demographics of objects in the test data set. We demonstrate that our Bayesian combination technique improves the overall performance over any individual classification method in these scenarios. Thus, strategies that combine the predictions of different classifiers may prove to be optimal in currently ongoing and forthcoming photometric surveys, such as the Dark Energy Survey and the Large Synoptic Survey Telescope.
We present GHOST, a database of 16,175 spectroscopically classified supernovae (SNe) and the properties of their host galaxies. We have constructed GHOST using a novel host galaxy association method ...that employs deep postage stamps of the field surrounding a transient. Our gradient ascent method achieves fewer misassociations for low-z hosts and higher completeness for high-z hosts than previous methods. Using dimensionality reduction, we identify the host galaxy properties that distinguish SN classes. Our results suggest that the host galaxies of superluminous SNe, Type Ia SNe, and core-collapse SNe can be separated by brightness and derived extendedness measures. Next, we train a random forest model to predict SN class using only host galaxy information and the radial offset of the SN. We can distinguish Type Ia SNe and core-collapse SNe with ∼70% accuracy without any photometric or spectroscopic data from the event itself. Vera C. Rubin Observatory will usher in a new era of transient population studies, demanding improved photometric tools for rapid identification and classification of transient events. By identifying the host features with high discriminatory power, we will maintain SN sample purities and continue to identify scientifically relevant events as data volumes increase. The GHOST database and our corresponding software for associating transients with host galaxies are both publicly available through the astro_ghost package.
AGNet: weighing black holes with deep learning Lin, Joshua Yao-Yu; Pandya, Sneh; Pratap, Devanshi ...
Monthly Notices of the Royal Astronomical Society,
02/2023, Letnik:
518, Številka:
4
Journal Article
Recenzirano
Odprti dostop
ABSTRACT
Supermassive black holes (SMBHs) are commonly found at the centres of most massive galaxies. Measuring SMBH mass is crucial for understanding the origin and evolution of SMBHs. Traditional ...approaches, on the other hand, necessitate the collection of spectroscopic data, which is costly. We present an algorithm that weighs SMBHs using quasar light time series information, including colours, multiband magnitudes, and the variability of the light curves, circumventing the need for expensive spectra. We train, validate, and test neural networks that directly learn from the Sloan Digital Sky Survey (SDSS) Stripe 82 light curves for a sample of 38 939 spectroscopically confirmed quasars to map out the non-linear encoding between SMBH mass and multiband optical light curves. We find a 1σ scatter of 0.37 dex between the predicted SMBH mass and the fiducial virial mass estimate based on SDSS single-epoch spectra, which is comparable to the systematic uncertainty in the virial mass estimate. Our results have direct implications for more efficient applications with future observations from the Vera C. Rubin Observatory. Our code, AGNet, is publicly available at https://github.com/snehjp2/AGNet.
ABSTRACT
Measurements of large-scale structure are interpreted using theoretical predictions for the matter distribution, including potential impacts of baryonic physics. We constrain the feedback ...strength of baryons jointly with cosmology using weak lensing and galaxy clustering observables (3 × 2pt) of Dark Energy Survey (DES) Year 1 data in combination with external information from baryon acoustic oscillations (BAO) and Planck cosmic microwave background polarization. Our baryon modelling is informed by a set of hydrodynamical simulations that span a variety of baryon scenarios; we span this space via a Principal Component (PC) analysis of the summary statistics extracted from these simulations. We show that at the level of DES Y1 constraining power, one PC is sufficient to describe the variation of baryonic effects in the observables, and the first PC amplitude (Q1) generally reflects the strength of baryon feedback. With the upper limit of Q1 prior being bound by the Illustris feedback scenarios, we reach $\sim 20{{\ \rm per\ cent}}$ improvement in the constraint of $S_8=\sigma _8(\Omega _{\rm m}/0.3)^{0.5}=0.788^{+0.018}_{-0.021}$ compared to the original DES 3 × 2pt analysis. This gain is driven by the inclusion of small-scale cosmic shear information down to 2.5 arcmin, which was excluded in previous DES analyses that did not model baryonic physics. We obtain $S_8=0.781^{+0.014}_{-0.015}$ for the combined DES Y1+Planck EE+BAO analysis with a non-informative Q1 prior. In terms of the baryon constraints, we measure $Q_1=1.14^{+2.20}_{-2.80}$ for DES Y1 only and $Q_1=1.42^{+1.63}_{-1.48}$ for DESY1+Planck EE+BAO, allowing us to exclude one of the most extreme AGN feedback hydrodynamical scenario at more than 2σ.
PDF
