We propose a technique for producing `visual explanations' for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent. Our approach - ...Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say logits for `dog' or even a caption), flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad- CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g. VGG), (2) CNNs used for structured outputs (e.g. captioning), (3) CNNs used in tasks with multi-modal inputs (e.g. visual question answering) or reinforcement learning, without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are more faithful to the underlying model, and (d) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show even non-attention based models can localize inputs. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a `stronger' deep network from a `weaker' one even when both make identical predictions. Our code is available at https: //github.com/ramprs/grad-cam/ along with a demo on CloudCV 2 and video at youtu.be/COjUB9Izk6E.
Visual Question Answering (VQA) has attracted much attention in both computer vision and natural language processing communities, not least because it offers insight into the relationships between ...two important sources of information. Current datasets, and the models built upon them, have focused on questions which are answerable by direct analysis of the question and image alone. The set of such questions that require no external information to answer is interesting, but very limited. It excludes questions which require common sense, or basic factual knowledge to answer, for example. Here we introduce FVQA (Fact-based VQA), a VQA dataset which requires, and supports, much deeper reasoning. FVQA primarily contains questions that require external information to answer. We thus extend a conventional visual question answering dataset, which contains image-question-answer triplets, through additional image-question-answer-supporting fact tuples. Each supporting-fact is represented as a structural triplet, such as < Cat,CapableOf,ClimbingTrees> . We evaluate several baseline models on the FVQA dataset, and describe a novel model which is capable of reasoning about an image on the basis of supporting-facts.
Data Analytics is being deployed to predict the dissolved nitrous oxide (N2O) concentration in a full-scale sidestream sequence batch reactor (SBR) treating the anaerobic supernatant. On average, the ...N2O emissions are equal to 7.6% of the NH4–N load and can contribute up to 97% to the operational carbon footprint of the studied nitritation-denitritation and via-nitrite enhanced biological phosphorus removal process (SCENA). The analysis showed that average aerobic dissolved N2O concentration could significantly vary under similar influent loads, dissolved oxygen (DO), pH and removal efficiencies. A combination of density-based clustering, support vector machine (SVM), and support vector regression (SVR) models were deployed to estimate the dissolved N2O concentration and behaviour in the different phases of the SBR system.
The results of the study reveal that the aerobic dissolved N2O concentration is correlated with the drop of average aerobic conductivity rate (spearman correlation coefficient equal to 0.7), the DO (spearman correlation coefficient equal to −0.7) and the changes of conductivity between sequential cycles. Additionally, operational conditions resulting in low aerobic N2O accumulation (<0.6 mg/L) were identified; step-feeding, control of initial NH4+ concentrations and aeration duration can mitigate the N2O peaks observed in the system. The N2O emissions during aeration shows correlation with the stripping of accumulated N2O from the previous anoxic cycle. The analysis shows that N2O is always consumed after the depletion of NO2− during denitritation (after the “nitrite knee”). Based on these findings SVM classifiers were constructed to predict whether dissolved N2O will be consumed during the anoxic and anaerobic phases and SVR models were trained to predict the N2O concentration at the end of the anaerobic phase and the average dissolved N2O concentration during aeration. The proposed approach accurately predicts the N2O emissions as a latent parameter from other low-cost sensors that are traditionally deployed in biological batch processes.
Display omitted
•S.C.E.N.A N2O emissions contribute up to 97.3% to the operational carbon footprint.•Stripping of accumulated N2O from the previous anoxic cycle increases emissions.•Aerobic dissolved N2O concentration is correlated with DO and conductivity.•Conductivity can be used to control SBR NH4+ concentration and N2O.•N2O can be estimated as a latent parameter from other low-cost sensors.
Visual7W: Grounded Question Answering in Images Yuke Zhu; Groth, Oliver; Bernstein, Michael ...
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
2016-June
Conference Proceeding
We have seen great progress in basic perceptual tasks such as object recognition and detection. However, AI models still fail to match humans in high-level vision tasks due to the lack of capacities ...for deeper reasoning. Recently the new task of visual question answering (QA) has been proposed to evaluate a model's capacity for deep image understanding. Previous works have established a loose, global association between QA sentences and images. However, many questions and answers, in practice, relate to local regions in the images. We establish a semantic link between textual descriptions and image regions by object-level grounding. It enables a new type of QA with visual answers, in addition to textual answers used in previous work. We study the visual QA tasks in a grounded setting with a large collection of 7W multiple-choice QA pairs. Furthermore, we evaluate human performance and several baseline models on the QA tasks. Finally, we propose a novel LSTM model with spatial attention to tackle the 7W QA tasks.
We introduce Interactive Question Answering (IQA), the task of answering questions that require an autonomous agent to interact with a dynamic visual environment. IQA presents the agent with a scene ...and a question, like: "Are there any apples in the fridge?" The agent must navigate around the scene, acquire visual understanding of scene elements, interact with objects (e.g. open refrigerators) and plan for a series of actions conditioned on the question. Popular reinforcement learning approaches with a single controller perform poorly on IQA owing to the large and diverse state space. We propose the Hierarchical Interactive Memory Network (HIMN), consisting of a factorized set of controllers, allowing the system to operate at multiple levels of temporal abstraction. To evaluate HIMN, we introduce IQUAD V1, a new dataset built upon AI2-THOR 35, a simulated photo-realistic environment of configurable indoor scenes with interactive objects. IQUAD V1 has 75,000 questions, each paired with a unique scene configuration. Our experiments show that our proposed model outperforms popular single controller based methods on IQUAD V1. For sample questions and results, please view our video: https://youtu.be/pXd3C-1jr98.
This open access book constitutes revised selected papers from the International Workshops held at the Third International Conference on Process Mining, ICPM 2021, which took place in Eindhoven, The ...Netherlands, during October 31–November 4, 2021. The conference focuses on the area of process mining research and practice, including theory, algorithmic challenges, and applications. The co-located workshops provided a forum for novel research ideas. The 28 papers included in this volume were carefully reviewed and selected from 65 submissions. They stem from the following workshops: 2nd International Workshop on Event Data and Behavioral Analytics (EDBA) 2nd International Workshop on Leveraging Machine Learning in Process Mining (ML4PM) 2nd International Workshop on Streaming Analytics for Process Mining (SA4PM) 6th International Workshop on Process Querying, Manipulation, and Intelligence (PQMI) 4th International Workshop on Process-Oriented Data Science for Healthcare (PODS4H) 2nd International Workshop on Trust, Privacy, and Security in Process Analytics (TPSA) One survey paper on the results of the XES 2.0 Workshop is included.
Much of vision-and-language research focuses on a small but diverse set of independent tasks and supporting datasets often studied in isolation; however, the visually-grounded language understanding ...skills required for success at these tasks overlap significantly. In this work, we investigate these relationships between vision-and-language tasks by developing a large-scale, multi-task model. Our approach culminates in a single model on 12 datasets from four broad categories of task including visual question answering, caption-based image retrieval, grounding referring expressions, and multimodal verification. Compared to independently trained single-task models, this represents a reduction from approximately 3 billion parameters to 270 million while simultaneously improving performance by 2.05 points on average across tasks. We use our multi-task framework to perform in-depth analysis of the effect of joint training diverse tasks. Further, we show that finetuning task-specific models from our single multi-task model can lead to further improvements, achieving performance at or above the state-of-the-art.
Web Mining Kumbhar, V. S.; Oza, K. S.; Kamat, R. K.
2016, 20220901, 2022-09-01, 2017-01-31
eBook
Web mining is the application of data mining strategies to excerpt learning from web information, i.e. web content, web structure, and web usage data. With the emergence of the web as the predominant ...and converging platform for communication, business and scholastic information dissemination, especially in the last five years, there are ever increasing research groups working on different aspects of web mining mainly in three directions. These are: mining of web content, web structure and web usage. In this context there are good number of frameworks and benchmarks related to the metrics of the websites which is certainly weighty for B2B, B2C and in general in any e-commerce paradigm. Owing to the popularity of this topic there are few books in the market, dealing more on such performance metrics and other related issues. This book, however, omits all such routine topics and lays more emphasis on the classification and clustering aspects of the websites in order to come out with the true perception of the websites in light of its usability. In nutshell, Web Mining: A Synergic Approach Resorting to Classifications and Clustering showcases an effective methodology for classification and clustering of web sites from their usability point of view. While the clustering and classification is accomplished by using an open source tool WEKA, the basic dataset for the selected websites has been emanated by using a free tool site-analyzer. As a case study, several commercial websites have been analyzed. The dataset preparation using site-analyzer and classification through WEKA by embedding different algorithms is one of the unique selling points of this book. This text projects a complete spectrum of web mining from its very inception through data mining and takes the reader up to the application level. Salient features of the book include: Literature review of research work in the area of web mining Business websites domain researched, and data collected using site-analyzer to
Objetivo: O artigo propõe a realização de um estudo descritivo sobre a participação da mulher na produção científica, em termos de produção bibliográfica e de orientações, no âmbito da Ciência da ...Informação. Metodologia: Utiliza um percurso descritivo, baseado no processo Knowledge Discovery in Databases, com o emprego de soluções de mineração de dados para a análise da participação da mulher na produção científica, especificamente na Ciência da Informação. Resultados: Os resultados obtidos apresentam indicadores em termos da produtividade no contexto da Ciência da Informação, abordando diferentes visões de acordo com tipos específicos de produção. Conclusão: As contribuições alcançadas pelas pesquisadoras na área de Ciência são destacadas, principalmente para produções em periódicos especializados e formação em nível de pós-graduação, demonstrando a representatividade e importância da mulher para o crescimento da área.
Objective: The article proposes a descriptive study on women's participation in scientific production, in terms of bibliographical production and guidelines, within the scope of Information Science. Methodology: Uses a descriptive path, based on the Knowledge Discovery in Databases process, using data mining solutions to analyze women's participation in scientific production, specifically in Information Science. Results: The results obtained present indicators in terms of productivity in the context of Information Science, addressing different views according to specific types of production. Conclusion: The contributions made by researchers in the area of Science are highlighted, mainly for productions in specialized journals and training at postgraduate level, demonstrating the representation and importance of women for the growth of the area.
Objetivo: El artículo propone realizar un estudio descriptivo sobre la participación de las mujeres en la producción científica, en términos de producción bibliográfica y lineamientos, en el ámbito de las Ciencias de la Información. Metodología: Utiliza un camino descriptivo, basado en el proceso de Descubrimiento del Conocimiento en Bases de Datos, con el uso de soluciones de minería de datos para analizar la participación de las mujeres en la producción científica, específicamente en las Ciencias de la Información. Resultados: Los resultados obtenidos presentan indicadores en términos de productividad en el contexto de las Ciencias de la Información, abordando diferentes miradas según tipos específicos de producción. Conclusión: Se destacan los aportes realizados por investigadores del área de Ciencias, principalmente para producciones en revistas especializadas y formación a nivel de posgrado, demostrando la representación e importancia de las mujeres para el crecimiento del área.
PDF
