•In-model approach for producing visual explanations for CNN-based classifiers.•Custom loss functions that require no supervision or further annotations of the data.•Explanation assessment metric - ...Percentage of Meaningful Pixels Outside the Mask.•Thorough quantitative analysis on a subset of ImageNet and a medical dataset.
With the outstanding predictive performance of Convolutional Neural Networks on different tasks and their widespread use in real-world scenarios, it is essential to understand and trust these black-box models. While most of the literature focuses on post-model methods, we propose a novel in-model joint architecture, composed by an explainer and a classifier. This architecture outputs not only a class label, but also a visual explanation of such decision, without the need for additional labelled data to train the explainer besides the image class. The model is trained end-to-end, with the classifier taking as input an image and the explainer’s resulting explanation, thus allowing for the classifier to focus on the relevant areas of such explanation. Moreover, this approach can be employed with any classifier, provided that the necessary connections to the explainer are made. We also propose a three-phase training process and two alternative custom loss functions that regularise the produced explanations and encourage desired properties, such as sparsity and spatial contiguity. The architecture was validated in two datasets (a subset of ImageNet and a cervical cancer dataset) and the obtained results show that it is able to produce meaningful image- and class-dependent visual explanations, without direct supervision, aligned with intuitive visual features associated with the data. Quantitative assessment of explanation quality was conducted through iterative perturbation of the input image according to the explanation heatmaps. The impact on classification performance is studied in terms of average function value and AOPC (Area Over the MoRF (Most Relevant First) Curve). For further evaluation, we propose POMPOM (Percentage of Meaningful Pixels Outside the Mask) as another measurable criteria of explanation goodness. These analyses showed that the proposed method outperformed state-of-the-art post-model methods, such as LRP (Layer-wise Relevance Propagation).
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
The remarkable success of deep learning has prompted interest in its application to medical imaging diagnosis. Even though state-of-the-art deep learning models have achieved human-level accuracy on ...the classification of different types of medical data, these models are hardly adopted in clinical workflows, mainly due to their lack of interpretability. The black-box nature of deep learning models has raised the need for devising strategies to explain the decision process of these models, leading to the creation of the topic of eXplainable Artificial Intelligence (XAI). In this context, we provide a thorough survey of XAI applied to medical imaging diagnosis, including visual, textual, example-based and concept-based explanation methods. Moreover, this work reviews the existing medical imaging datasets and the existing metrics for evaluating the quality of the explanations. In addition, we include a performance comparison among a set of report generation–based methods. Finally, the major challenges in applying XAI to medical imaging and the future research directions on the topic are discussed.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, SAZU, UL, UM, UPUK
Human–Robot Collaboration is a critical component of Industry 4.0, contributing to a transition towards more flexible production systems that are quickly adjustable to changing production ...requirements. This paper aims to increase the natural collaboration level of a robotic engine assembly station by proposing a cognitive system powered by computer vision and deep learning to interpret implicit communication cues of the operator. The proposed system, which is based on a residual convolutional neural network with 34 layers and a long-short term memory recurrent neural network (ResNet-34 + LSTM), obtains assembly context through action recognition of the tasks performed by the operator. The assembly context was then integrated in a collaborative assembly plan capable of autonomously commanding the robot tasks. The proposed model showed a great performance, achieving an accuracy of 96.65% and a temporal mean intersection over union (mIoU) of 94.11% for the action recognition of the considered assembly. Moreover, a task-oriented evaluation showed that the proposed cognitive system was able to leverage the performed human action recognition to command the adequate robot actions with near-perfect accuracy. As such, the proposed system was considered as successful at increasing the natural collaboration level of the considered assembly station.
•Implicit communication cues are crucial for effortless Human–Robot Collaboration.•Human actions contain task-focused information that provide operation context.•A Deep Learning cognitive system was proposed to interpret task-focused human actions.•The proposed cognitive system was applied in a real collaborative assembly scenario.•Great accuracy was achieved at recognizing human actions and commanding robot tasks.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Mesenchymal stem cells (MSCs) have been used for cell-based therapies in regenerative medicine, with increasing importance in central and peripheral nervous system repair. However, MSCs grafting ...present disadvantages, such as, a high number of cells required for transplantation and low survival rate when transplanted into the central nervous system (CNS). In line with this, MSCs secretome which present on its composition a wide range of molecules (neurotrophins, cytokines) and microvesicles, can be a solution to surpass these problems. However, the effect of MSCs secretome in axonal elongation is poorly understood. In this study, we demonstrate that application of MSCs secretome to both rat cortical and hippocampal neurons induces an increase in axonal length. In addition, we show that this growth effect is axonal intrinsic with no contribution from the cell body. To further understand which are the molecules required for secretome-induced axonal outgrowth effect, we depleted brain-derived neurotrophic factor (BDNF) from the secretome. Our results show that in the absence of BDNF, secretome-induced axonal elongation effect is lost and that axons present a reduced axonal growth rate. Altogether, our results demonstrate that MSCs secretome is able to promote axonal outgrowth in CNS neurons and this effect is mediated by BDNF.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
Nowadays, video surveillance systems are taking the first steps toward automation, in order to ease the burden on human resources as well as to avoid human error. As the underlying data distribution ...and the number of concepts change over time, the conventional learning algorithms fail to provide reliable solutions for this setting. In this paper, we formalize a learning concept suitable for multi-camera video surveillance and propose a learning methodology adapted to that new paradigm. The proposed framework resorts to the universal background model to robustly learn individual object models from small samples and to more effectively detect novel classes. The individual models are incrementally updated in an ensemble-based approach, with older models being progressively forgotten. The framework is designed to detect and label new concepts automatically. The system is also designed to exploit active learning strategies, in order to interact wisely with operator, requesting assistance in the most ambiguous to classify observations. The experimental results obtained both on real and synthetic data sets verify the usefulness of the proposed approach.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
The scarcity of balanced and annotated datasets has been a recurring problem in medical image analysis. Several researchers have tried to fill this gap employing dataset synthesis with adversarial ...networks (GANs). Breast magnetic resonance imaging (MRI) provides complex, texture-rich medical images, with the same annotation shortage issues, for which, to the best of our knowledge, no previous work tried synthesizing data. Within this context, our work addresses the problem of synthesizing breast MRI images from corresponding annotations and evaluate the impact of this data augmentation strategy on a semantic segmentation task. We explored variations of image-to-image translation using conditional GANs, namely fitting the generator’s architecture with residual blocks and experimenting with cycle consistency approaches. We studied the impact of these changes on visual verisimilarity and how an U-Net segmentation model is affected by the usage of synthetic data. We achieved sufficiently realistic-looking breast MRI images and maintained a stable segmentation score even when completely replacing the dataset with the synthetic set. Our results were promising, especially when concerning to Pix2PixHD and Residual CycleGAN architectures.
Torsional strength is related with one of the most critical failure types for the design and assessment of reinforced concrete (RC) members due to the complexity of the associated stress state and ...low ductility. Previous studies have shown that reliable methods to predict the torsional strength of RC beams are still needed, namely for over-reinforced and high-strength RC beams. This research aims to offer a novel set of models to predict the torsional strength of RC beams with a wide range of design attributes and geometries by using advanced M5P tree and nonlinear regression models. For this, a broad database with 202 experimental tests is used to generate highly reliable and resilient models. To build the models, three independent variables related with the properties of the RC beams are considered: concrete cross-section area (area enclosed within the outer perimeter of the cross-section), concrete compressive strength, and torsional reinforcement factor (which accounts for the type—longitudinal or transverse—amount, and yielding strength of the torsional reinforcement). In contrast to multiple nonlinear regression approaches, the findings show that the M5P tree approach has the best estimation in terms of both accuracy and safety. Furthermore, M5P model predictions are far more accurate and safer than the most prevalent design equations. Finally, sensitivity and parametric studies are used to confirm the robustness of the presented models.
The inactivation of
Escherichia coli using moderate electric fields (MEF) below 25
°C, was investigated. Keeping the temperature always below 25
°C demonstrated that electric fields are involved in ...the inactivation of
E. coli, without possible synergistic temperature effects. Electric fields above 220
V
cm
−1 promoted death rates of 3
log
10 cycles of
E. coli in less than 6
min, and even higher rates at greater electric fields, while presumably overcoming the thermal degradation caused by conventional high temperature treatments. A non-thermal model was proposed that successfully describes the
E. coli death kinetics under this treatment. SEM observations of
E. coli cells after the exposure to the MEF treatment, revealed changes at the cell membrane level, indicating a possible cause for the cell death rates. These results show that this treatment holds potential for sterilization of thermolabile products (e.g. serum and other physiological fluids, food products), by itself or as a complement of the traditional heat-dependent techniques.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
Design codes provide the necessary tools to check the torsional strength of reinforced concrete (RC) members. However, some researchers have pointed out that code equations still need improvement. ...This study presents a review and a comparative analysis of the calculation procedures to predict the torsional strength of RC beams from some reference design codes, namely the Russian, American, European, and Canadian codes for RC structures. The reliability and accuracy of the normative torsional strengths are checked against experimental results from a broad database incorporating 202 RC rectangular beams tested under pure torsion and collected from the literature. The results show that both the readability and accuracy of the codes' equations should be improved. Based on a correlation study between the experimental torsional strengths, and geometrical and mechanical properties of the beams, refined yet simple equations are proposed to predict torsional strength. It is demonstrated that the proposed formulation is characterized by a significant improvement over the reference design codes. The efficiency of the proposed formulae is also assessed against another equation earlier proposed in the literature, and an improvement is noted as well. From the results, it can be concluded that the proposed equations in this study can contribute to a more accurate and economical design for practice.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
The still prevalent use of paper conformity lists in the automotive industry has a serious negative impact on the performance of quality control inspectors. We propose instead a hybrid quality ...inspection system, where we combine automated detection with human feedback, to increase worker performance by reducing mental and physical fatigue, and the adaptability and responsiveness of the assembly line to change. The system integrates the hierarchical automatic detection of the non-conforming vehicle parts and information visualization on a wearable device to present the results to the factory worker and obtain human confirmation. Besides designing a novel 3D vehicle generator to create a digital representation of the non conformity list and to collect automatically annotated training data, we apply and aggregate in a novel way state-of-the-art domain adaptation and pseudo labeling methods to our real application scenario, in order to bridge the gap between the labeled data generated by the vehicle generator and the real unlabeled data collected on the factory floor. This methodology allows us to obtain, without any manual annotation of the real dataset, an example-based F1 score of 0.565 in an unconstrained scenario and 0.601 in a fixed camera setup (improvements of 11 and 14.6 percentage points, respectively, over a baseline trained with purely simulated data). Feedback obtained from factory workers highlighted the usefulness of the proposed solution, and showed that a truly hybrid assembly line, where machine and human work in symbiosis, increases both efficiency and accuracy in automotive quality control.
PDF
