Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that ...involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships
riding(man, carriage)
and
pulling(horse, carriage)
to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of
35
objects,
26
attributes, and
21
pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.
Regardless of how much data artificial intelligence agents have available, agents will inevitably encounter previously unseen situations in real-world deployments. Reacting to novel situations by ...acquiring new information from other people—socially situated learning—is a core faculty of human development. Unfortunately, socially situated learning remains an open challenge for artificial intelligence agents because they must learn how to interact with people to seek out the information that they lack. In this article, we formalize the task of socially situated artificial intelligence—agents that seek out new information through social interactions with people—as a reinforcement learning problem where the agent learns to identify meaningful and informative questions via rewards observed through social interaction. We manifest our framework as an interactive agent that learns how to ask natural language questions about photos as it broadens its visual intelligence on a large photo-sharing social network. Unlike active-learning methods, which implicitly assume that humans are oracles willing to answer any question, our agent adapts its behavior based on observed norms of which questions people are or are not interested to answer. Through an 8-mo deployment where our agent interacted with 236,000 social media users, our agent improved its performance at recognizing new visual information by 112%. A controlled field experiment confirmed that our agent outperformed an active-learning baseline by 25.6%. This work advances opportunities for continuously improving artificial intelligence (AI) agents that better respect norms in open social environments.
BACKGROUND AND PURPOSE:The diverse causes of right-sided heart failure (RHF) include, among others, primary cardiomyopathies with right ventricular (RV) involvement, RV ischemia and infarction, ...volume loading caused by cardiac lesions associated with congenital heart disease and valvular pathologies, and pressure loading resulting from pulmonic stenosis or pulmonary hypertension from a variety of causes, including left-sided heart disease. Progressive RV dysfunction in these disease states is associated with increased morbidity and mortality. The purpose of this scientific statement is to provide guidance on the assessment and management of RHF.
METHODS:The writing group used systematic literature reviews, published translational and clinical studies, clinical practice guidelines, and expert opinion/statements to summarize existing evidence and to identify areas of inadequacy requiring future research. The panel reviewed the most relevant adult medical literature excluding routine laboratory tests using MEDLINE, EMBASE, and Web of Science through September 2017. The document is organized and classified according to the American Heart Association to provide specific suggestions, considerations, or reference to contemporary clinical practice recommendations.
RESULTS:Chronic RHF is associated with decreased exercise tolerance, poor functional capacity, decreased cardiac output and progressive end-organ damage (caused by a combination of end-organ venous congestion and underperfusion), and cachexia resulting from poor absorption of nutrients, as well as a systemic proinflammatory state. It is the principal cause of death in patients with pulmonary arterial hypertension. Similarly, acute RHF is associated with hemodynamic instability and is the primary cause of death in patients presenting with massive pulmonary embolism, RV myocardial infarction, and postcardiotomy shock associated with cardiac surgery. Functional assessment of the right side of the heart can be hindered by its complex geometry. Multiple hemodynamic and biochemical markers are associated with worsening RHF and can serve to guide clinical assessment and therapeutic decision making. Pharmacological and mechanical interventions targeting isolated acute and chronic RHF have not been well investigated. Specific therapies promoting stabilization and recovery of RV function are lacking.
CONCLUSIONS:RHF is a complex syndrome including diverse causes, pathways, and pathological processes. In this scientific statement, we review the causes and epidemiology of RV dysfunction and the pathophysiology of acute and chronic RHF and provide guidance for the management of the associated conditions leading to and caused by RHF.
Image retrieval using scene graphs Johnson, Justin; Krishna, Ranjay; Stark, Michael ...
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
06/2015
Conference Proceeding
This paper develops a novel framework for semantic image retrieval based on the notion of a scene graph. Our scene graphs represent objects ("man", "boat"), attributes of objects ("boat is white") ...and relationships between objects ("man standing on boat"). We use these scene graphs as queries to retrieve semantically related images. To this end, we design a conditional random field model that reasons about possible groundings of scene graphs to test images. The likelihoods of these groundings are used as ranking scores for retrieval. We introduce a novel dataset of 5,000 human-generated scene graphs grounded to images and use this dataset to evaluate our method for image retrieval. In particular, we evaluate retrieval using full scene graphs and small scene subgraphs, and show that our method outperforms retrieval methods that use only objects or low-level image features. In addition, we show that our full model can be used to improve object localization compared to baseline methods.
Researchers in areas as diverse as computer science and political science must increasingly navigate the possible risks of their research to society. However, the history of medical experiments on ...vulnerable individuals influenced many research ethics reviews to focus exclusively on risks to human subjects rather than risks to human society. We describe an Ethics and Society Review board (ESR), which fills this moral gap by facilitating ethical and societal reflection as a requirement to access grant funding: Researchers cannot receive grant funding from participating programs until the researchers complete the ESR process for their proposal. Researchers author an initial statement describing their proposed research's risks to society, subgroups within society, and globally and commit to mitigation strategies for these risks. An interdisciplinary faculty panel iterates with the researchers to refine these risks and mitigation strategies. We describe a mixed-method evaluation of the ESR over 1 y, in partnership with a large artificial intelligence grant program at our university. Surveys and interviews of researchers who interacted with the ESR found 100% (95% CI: 87 to 100%) were willing to continue submitting future projects to the ESR, and 58% (95% CI: 37 to 77%) felt that it had influenced the design of their research project. The ESR panel most commonly identified issues of harms to minority groups, inclusion of diverse stakeholders in the research plan, dual use, and representation in datasets. These principles, paired with possible mitigation strategies, offer scaffolding for future research designs.
Behavior change systems help people manage their time online and achieve many other goals. These systems typically consist of a single static intervention, such as a timer or site blocker, to ...persuade users to behave in ways consistent with their stated goals. However, static interventions decline in effectiveness over time as users begin to ignore them. In this paper, we compare the effectiveness of static interventions to a rotation strategy, where users experience different interventions over time. We built and deployed a browser extension called HabitLab, which features many interventions that the user can enable across social media and other web sites to control their time spent browsing. We ran three in-the-wild field experiments on HabitLab to compare static interventions to rotated interventions. We found that rotating between interventions increased effectiveness as measured by time on site, but also increased attrition: more users uninstalled HabitLab. To minimize attrition, we introduced a just-in-time information design about rotation. This design reduced attrition rates by half. With this research, we suggest that interaction design, paired with rotation of behavior change interventions, can help users gain control of their online habits.
•Latino adults are disproportionately impacted by dementia due to high exposure to adverse social determinants of health (SDOH).•“Brain health literacy” in middle adulthood may be a modifiable factor ...that could help mitigate later life risk of cognitive decline.•The proposed Brain Health Literacy framework can guide health promotion efforts, with the ultimate goal of increasing brain health equity.
Learning Perceptual Kernels for Visualization Design Demiralp, Cagatay Demiralp; Bernstein, Michael S.; Heer, Jeffrey
IEEE transactions on visualization and computer graphics,
12/2014, Letnik:
20, Številka:
12
Journal Article
Recenzirano
Odprti dostop
Visualization design can benefit from careful consideration of perception, as different assignments of visual encoding variables such as color, shape and size affect how viewers interpret data. In ...this work, we introduce perceptual kernels: distance matrices derived from aggregate perceptual judgments. Perceptual kernels represent perceptual differences between and within visual variables in a reusable form that is directly applicable to visualization evaluation and automated design. We report results from crowd-sourced experiments to estimate kernels for color, shape, size and combinations thereof. We analyze kernels estimated using five different judgment types-including Likert ratings among pairs, ordinal triplet comparisons, and manual spatial arrangement-and compare them to existing perceptual models. We derive recommendations for collecting perceptual similarities, and then demonstrate how the resulting kernels can be applied to automate visualization design decisions.
High-grade osteosarcoma is a primary malignant bone tumour mainly affecting children and young adults. The European and American Osteosarcoma Study (EURAMOS)-1 is a collaboration of four study groups ...aiming to improve outcomes of this rare disease by facilitating randomised controlled trials.
Patients eligible for EURAMOS-1 were aged ≤40 years with M0 or M1 skeletal high-grade osteosarcoma in which case complete surgical resection at all sites was deemed to be possible. A three-drug combination with methotrexate, doxorubicin and cisplatin was defined as standard chemotherapy, and between April 2005 and June 2011, 2260 patients were registered. We report survival outcomes and prognostic factors in the full cohort of registered patients.
For all registered patients at a median follow-up of 54 months (interquartile range: 38–73) from biopsy, 3-year and 5-year event-free survival were 59% (95% confidence interval CI: 57–61%) and 54% (95% CI: 52–56%), respectively. Multivariate analyses showed that the most adverse factors at diagnosis were pulmonary metastases (hazard ratio HR = 2.34, 95% CI: 1.95–2.81), non-pulmonary metastases (HR = 1.94, 95% CI: 1.38–2.73) or an axial skeleton tumour site (HR = 1.53, 95% CI: 1.10–2.13). The histological subtypes telangiectatic (HR = 0.52, 95% CI: 0.33–0.80) and unspecified conventional (HR = 0.67, 95% CI: 0.52–0.88) were associated with a favourable prognosis compared with chondroblastic subtype. The 3-year and 5-year overall survival from biopsy were 79% (95% CI: 77–81%) and 71% (95% CI: 68–73%), respectively. For patients with localised disease at presentation and in complete remission after surgery, having a poor histological response was associated with worse outcome after surgery (HR = 2.13, 95% CI: 1.76–2.58). In radically operated patients, there was no good evidence that axial tumour site was associated with worse outcome.
In conclusion, data from >2000 patients registered to EURAMOS-1 demonstrated survival rates in concordance with institution- or group-level osteosarcoma trials. Further efforts are required to drive improvements for patients who can be identified to be at higher risk of adverse outcome. This trial reaffirms known prognostic factors, and owing to the large numbers of patients registered, it sheds light on some additional factors to consider.
•Osteosarcoma is a rare disease, and treatment can only improve with international collaboration.•We have assembled prospectively collected data from treatments of all the patients in the intercontinental European and American Osteosarcoma Study-1 protocol.•These consistently treated patients provided a strong data set for reporting survival outcomes and reporting on prognostic factors.•The trial reaffirms known prognostic factors, and the most adverse factors were metastases and tumours in the axial skeleton.•Owing to the large numbers of patients registered, light is shed on some additional factors to be considered. Around seven in ten patients were still alive five years after diagnosis.
It has long been known that many pulse oximeters function less accurately in patients with darker skin. Reasons for this observation are incompletely characterized and potentially enabled by ...limitations in existing regulatory oversight. Based on decades of experience and unpublished data, we believe it is feasible to fully characterize, in the public domain, the factors that contribute to missing clinically important hypoxemia in patients with darkly pigmented skin. Here we propose 5 priority areas of inquiry for the research community and actionable changes to current regulations that will help improve oximeter accuracy. We propose that leading regulatory agencies should immediately modify standards for measuring accuracy and precision of oximeter performance, analyzing and reporting performance outliers, diversifying study subject pools, thoughtfully defining skin pigmentation, reporting data transparently, and accounting for performance during low-perfusion states. These changes will help reduce bias in pulse oximeter performance and improve access to safe oximeters.
PDF
