A Survey on Deep Learning Pouyanfar, Samira; Sadiq, Saad; Yan, Yilin ...
ACM computing surveys,
09/2019, Volume:
51, Issue:
5
Journal Article
Peer reviewed
The field of machine learning is witnessing its golden era as deep learning slowly becomes the leader in this domain. Deep learning uses multiple layers to represent the abstractions of data to build ...computational models. Some key enabler deep learning algorithms such as generative adversarial networks, convolutional neural networks, and model transfers have completely changed our perception of information processing. However, there exists an aperture of understanding behind this tremendously fast-paced domain, because it was never previously represented from a multiscope perspective. The lack of core understanding renders these powerful methods as black-box machines that inhibit development at a fundamental level. Moreover, deep learning has repeatedly been perceived as a silver bullet to all stumbling blocks in machine learning, which is far from the truth. This article presents a comprehensive review of historical and recent state-of-the-art approaches in visual, audio, and text processing; social network analysis; and natural language processing, followed by the in-depth analysis on pivoting and groundbreaking advances in deep learning applications. It was also undertaken to review the issues faced in deep learning such as unsupervised learning, black-box models, and online learning and to illustrate how these challenges can be transformed into prolific future research avenues.
Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. GPs have received increased attention in the machine-learning community over the past ...decade, and this book provides a long-needed systematic and unified treatment of theoretical and practical aspects of GPs in machine learning. The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics.The book deals with the supervised-learning problem for both regression and classification, and includes detailed algorithms. A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective. Many connections to other well-known techniques from machine learning and statistics are discussed, including support-vector machines, neural networks, splines, regularization networks, relevance vector machines and others. Theoretical issues including learning curves and the PAC-Bayesian framework are treated, and several approximation methods for learning with large datasets are discussed. The book contains illustrative examples and exercises, and code and datasets are available on the Web. Appendixes provide mathematical background and a discussion of Gaussian Markov processes.
In a layer house, the COsub.2 (carbon dioxide) concentration above the upper limit can cause the oxygen concentration to be below the lower limit suitable for poultry. This leads to chronic COsub.2 ...poisoning in layers, which manifests as listlessness, reduced appetite, weak constitution, decreased production performance, and weakened resistance to disease. Regulating ventilation may ensure a suitable COsub.2 concentration in layer houses. Predicting the changes in COsub.2 concentration and regulating the COsub.2 concentration in advance are key to ensuring healthy large-scale breeding of layers. In recent years, machine learning and deep learning methods have been increasingly applied to this field. A COsub.2 prediction model for layer house is proposed based on a GRU (gated recurrent unit) and LSTM (long short-term memory). The temperature, humidity, and COsub.2 were determined as inputs to the model by the correlation coefficient. The datasets of the experimental layer house were continuously measured during June–July 2023, using a self-developed environmental monitor, and the monitored data were used as samples for model inputs. There were 22,000 time series data in the datasets. In this study, multivariate time series data were standardized via data pre-processing to improve model training. GRU and LSTM models were constructed. The models were trained using a training set. Then, these trained models were used to provide predictions on a test set. The prediction errors were calculated using the true values of the test set and the predicted values provided by the models. To test the performance of the model and accuracy of the predictions, predictions were made for different numbers of datasets. The results demonstrated that the combined prediction model had good generalization, stability, and convergence with high prediction accuracy. Due to the structure of the model, the stability of the LSTM model was higher than that of the GRU model, and its prediction accuracy and speed were lower than those of the GRU model. When the datasets of the GRU model were 15,000 to 17,000, The MAE of the GRU was 70.8077 to 126.7029 ppm, and the prediction time of the GRU is 16 to 24 ms. When the LSTM model’s datasets were 15,000–20,000, the MAE of LSTM was 78.8596 to 136.0896 ppm, and the prediction time of the GRU was 17 to 26 ms.
Self-Checking Deep Neural Networks in Deployment Xiao, Yan; Beschastnikh, Ivan; Rosenblum, David S. ...
2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)
Conference Proceeding
Open access
The widespread adoption of Deep Neural Networks (DNNs) in important domains raises questions about the trustworthiness of DNN outputs. Even a highly accurate DNN will make mistakes some of the time, ...and in settings like self-driving vehicles these mistakes must be quickly detected and properly dealt with in deployment. Just as our community has developed effective techniques and mechanisms to monitor and check programmed components, we believe it is now necessary to do the same for DNNs. In this paper we present DNN self-checking as a process by which internal DNN layer features are used to check DNN predictions. We detail SelfChecker, a self-checking system that monitors DNN outputs and triggers an alarm if the internal layer features of the model are inconsistent with the final prediction. SelfChecker also provides advice in the form of an alternative prediction. We evaluated SelfChecker on four popular image datasets and three DNN models and found that SelfChecker triggers correct alarms on 60.56% of wrong DNN predictions, and false alarms on 2.04% of correct DNN predictions. This is a substantial improvement over prior work (SelfOracle, Dissector, and ConfidNet). In experiments with self-driving car scenarios, SelfChecker triggers more correct alarms than SelfOracle for two DNN models (DAVE-2 and Chauffeur) with comparable false alarms. Our implementation is available as open source.
Machine learning (ML) is a growing field in medicine. This narrative review describes the current body of literature on ML for clinical decision support in infectious diseases (ID).
We aim to inform ...clinicians about the use of ML for diagnosis, classification, outcome prediction and antimicrobial management in ID.
References for this review were identified through searches of MEDLINE/PubMed, EMBASE, Google Scholar, biorXiv, ACM Digital Library, arXiV and IEEE Xplore Digital Library up to July 2019.
We found 60 unique ML-clinical decision support systems (ML-CDSS) aiming to assist ID clinicians. Overall, 37 (62%) focused on bacterial infections, 10 (17%) on viral infections, nine (15%) on tuberculosis and four (7%) on any kind of infection. Among them, 20 (33%) addressed the diagnosis of infection, 18 (30%) the prediction, early detection or stratification of sepsis, 13 (22%) the prediction of treatment response, four (7%) the prediction of antibiotic resistance, three (5%) the choice of antibiotic regimen and two (3%) the choice of a combination antiretroviral therapy. The ML-CDSS were developed for intensive care units (n = 24, 40%), ID consultation (n = 15, 25%), medical or surgical wards (n = 13, 20%), emergency department (n = 4, 7%), primary care (n = 3, 5%) and antimicrobial stewardship (n = 1, 2%). Fifty-three ML-CDSS (88%) were developed using data from high-income countries and seven (12%) with data from low- and middle-income countries (LMIC). The evaluation of ML-CDSS was limited to measures of performance (e.g. sensitivity, specificity) for 57 ML-CDSS (95%) and included data in clinical practice for three (5%).
Considering comprehensive patient data from socioeconomically diverse healthcare settings, including primary care and LMICs, may improve the ability of ML-CDSS to suggest decisions adapted to various clinical contexts. Currents gaps identified in the evaluation of ML-CDSS must also be addressed in order to know the potential impact of such tools for clinicians and patients.
We present a physics‐informed deep neural network (DNN) method for estimating hydraulic conductivity in saturated and unsaturated flows governed by Darcy's law. For saturated flow, we approximate ...hydraulic conductivity and head with two DNNs and use Darcy's law in addition to measurements of hydraulic conductivity and head to train these DNNs. For unsaturated flow, we approximate unsaturated conductivity function and capillary pressure with DNNs and train these DNNs using measurements of capillary pressure and the Richards equation. Because it is difficult to measure unsaturated conductivity in the field, we assume that no measurements of unsaturated conductivity are available. The proposed approach enforces the partial differential equation (PDE) (Darcy or Richards equation) constraints by minimizing the PDE residual at select points in the simulation domain. We demonstrate that physics constraints increase the accuracy of DNN approximations of sparsely observed functions and allow for training DNNs when no direct measurements of the functions of interest are available. For the saturated conductivity estimation problem, we show that the physics‐informed DNN method is more accurate than the state‐of‐the‐art maximum a posteriori probability method. For the unsaturated flow in homogeneous porous media, we find that the proposed method can accurately estimate the pressure‐conductivity relationship based on the capillary pressure measurements only, even in the presence of measurement noise.
Key Points
Physics constraints improve the accuracy of machine learning methods, especially when learning from sparse data
Physics constraints allow learning constitutive relationships without direct observations of the quantities of interest
For considered examples, the proposed physics‐informed neural networks provide a more accurate parameter estimation than the maximum a posteriori probability method
The Logic of Graph Neural Networks Grohe, Martin
2021 36th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS),
2021-June-29
Conference Proceeding
Open access
Graph neural networks (GNNs) are deep learning architectures for machine learning problems on graphs. It has recently been shown that the expressiveness of GNNs can be characterised precisely by the ...combinatorial Weisfeiler-Leman algorithms and by finite variable counting logics. The correspondence has even led to new, higher-order GNNs corresponding to the WL algorithm in higher dimensions.The purpose of this paper is to explain these descriptive characterisations of GNNs.
PDF
