Learning to Rank Using Privileged Information Sharmanska, Viktoriia; Quadrianto, Novi; Lampert, Christoph H.
2013 IEEE International Conference on Computer Vision,
12/2013
Conference Proceeding, Journal Article
Odprti dostop
Many computer vision problems have an asymmetric distribution of information between training and test time. In this work, we study the case where we are given additional information about the ...training data, which however will not be available at test time. This situation is called learning using privileged information (LUPI). We introduce two maximum-margin techniques that are able to make use of this additional source of information, and we show that the framework is applicable to several scenarios that have been studied in computer vision before. Experiments with attributes, bounding boxes, image tags and rationales as additional information in object classification show promising results.
Transitional peri-urban contexts are frontiers for sustainable development where land-use change involves negotiation and contestation between diverse interest groups. Multiple, complex trade-offs ...between outcomes emerge which have both negative and positive impacts on progress towards achieving Sustainable Development Goals (SDGs). These trade-offs are often overlooked in policy and planning processes which depend on top-down expert perspectives and rely on course grain aggregate data which does not reflect complex peri-urban dynamics or the rapid pace of change. Tools are required to address this gap, integrate data from diverse perspectives and inform more inclusive planning processes. In this paper, we draw on a reinterpretation of empirical data concerned with land-use change and multiple dimensions of food security from the city of Wuhan in China to illustrate some of the complex trade-offs between SDG goals that tend to be overlooked with current planning approaches. We then describe the development of an interactive web-based tool that implements deep learning methods for fine-grained land-use classification of high-resolution remote sensing imagery and integrates this with a flexible method for rapid trade-off analysis of land-use change scenarios. The development and potential use of the tool are illustrated using data from the Wuhan case study example. This tool has the potential to support participatory planning processes by providing a platform for multiple stakeholders to explore the implications of planning decisions and land-use policies. Used alongside other planning, engagement and ecosystem service mapping tools it can help to reveal invisible trade-offs and foreground the perspectives of diverse stakeholders. This is vital for building approaches which recognise how trade-offs between the achievement of SDGs can be influenced by development interventions.
Adaptation finance addresses the effects of climate variability and change on development and physical insecurity. Yet, adaptation has proven difficult to systematically measure and assess, resulting ...in a lack of coherent and comparable evidence to learn about good adaptation practice. Measurement has become even more challenging since the integration of resilience into policy framings. Measuring climate resilience requires evidence of resistance to, and recovery from, shocks and stresses. But resilience of what, to what seldom guides the design of assessments. Researchers tend to use proxy indicators and aggregated units of analysis that cloud the relationships under study, and crucially, leave unclear the interactions between climate and development. This viewpoint documents these common barriers to progress in the field. It then outlines two methods for practitioners and researchers to link climate shocks and stresses to climate-sensitive development outcomes as a key first step to research and evaluation design. Both methods enable prediction of expected levels of development outcomes, given the extent of climate shocks and stresses, which is then comparable with actual levels achieved under climate resilience interventions. The product is standardized and comparable metrics to learn about the performance of climate adaptation policymaking and resilient development programming.
The issue of fairness in machine learning models has recently attracted a lot of attention as ensuring it will ensure continued confidence of the general public in the deployment of machine learning ...systems. We focus on mitigating the harm incurred by a biased machine learning system that offers better outputs (e.g., loans, job interviews) for certain groups than for others. We show that bias in the output can naturally be controlled in probabilistic models by introducing a latent target output. This formulation has several advantages: first, it is a unified framework for several notions of group fairness such as Demographic Parity and Equality of Opportunity; second, it is expressed as a marginalization instead of a constrained problem; and third, it allows the encoding of our knowledge of what unbiased outputs should be. Practically, the second allows us to avoid unstable constrained optimization procedures and to reuse off-the-shelf toolboxes. The latter translates to the ability to control the level of fairness by directly varying fairness target rates. In contrast, existing approaches rely on intermediate, arguably unintuitive, control parameters such as covariance thresholds.
A Very Simple Safe-Bayesian Random Forest Quadrianto, Novi; Ghahramani, Zoubin
IEEE transactions on pattern analysis and machine intelligence,
2015-June-1, 2015-Jun, 2015-6-1, 20150601, Letnik:
37, Številka:
6
Journal Article
Recenzirano
Odprti dostop
Random forests works by averaging several predictions of de-correlated trees. We show a conceptually radical approach to generate a random forest: random sampling of many trees from a prior ...distribution, and subsequently performing a weighted ensemble of predictive probabilities. Our approach uses priors that allow sampling of decision trees even before looking at the data, and a power likelihood that explores the space spanned by combination of decision trees. While each tree performs Bayesian inference to compute its predictions, our aggregation procedure uses the power likelihood rather than the likelihood and is therefore strictly speaking not Bayesian. Nonetheless, we refer to it as a Bayesian random forest but with a built-in safety. The safeness comes as it has good predictive performance even if the underlying probabilistic model is wrong. We demonstrate empirically that our Safe-Bayesian random forest outperforms MCMC or SMC based Bayesian decision trees in term of speed and accuracy, and achieves competitive performance to entropy or Gini optimised random forest, yet is very simple to construct.
Developing data-driven solutions that address real-world problems requires understanding of these problems' causes and how their interaction affects the outcome-often with only observational data. ...Causal Bayesian Networks (BN) have been proposed as a powerful method for discovering and representing the causal relationships from observational data as a Directed Acyclic Graph (DAG). BNs could be especially useful for research in global health in Lower and Middle Income Countries, where there is an increasing abundance of observational data that could be harnessed for policy making, program evaluation, and intervention design. However, BNs have not been widely adopted by global health professionals, and in real-world applications, confidence in the results of BNs generally remains inadequate. This is partially due to the inability to validate against some ground truth, as the true DAG is not available. This is especially problematic if a learned DAG conflicts with pre-existing domain doctrine. Here we conceptualize and demonstrate an idea of a "Causal Datasheet" that could approximate and document BN performance expectations for a given dataset, aiming to provide confidence and sample size requirements to practitioners. To generate results for such a Causal Datasheet, a tool was developed which can generate synthetic Bayesian networks and their associated synthetic datasets to mimic real-world datasets. The results given by well-known structure learning algorithms and a novel implementation of the OrderMCMC method using the Quotient Normalized Maximum Likelihood score were recorded. These results were used to populate the Causal Datasheet, and recommendations could be made dependent on whether expected performance met user-defined thresholds. We present our experience in the creation of Causal Datasheets to aid analysis decisions at different stages of the research process. First, one was deployed to help determine the appropriate sample size of a planned study of sexual and reproductive health in Madhya Pradesh, India. Second, a datasheet was created to estimate the performance of an existing maternal health survey we conducted in Uttar Pradesh, India. Third, we validated generated performance estimates and investigated current limitations on the well-known ALARM dataset. Our experience demonstrates the utility of the Causal Datasheet, which can help global health practitioners gain more confidence when applying BNs.
Discovering Fair Representations in the Data Domain Quadrianto, Novi; Sharmanska, Viktoriia; Thomas, Oliver
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
2019-June
Conference Proceeding
Odprti dostop
Interpretability and fairness are critical in computer vision and machine learning applications, in particular when dealing with human outcomes, e.g. inviting or not inviting for a job interview ...based on application materials that may include photographs. One promising direction to achieve fairness is by learning data representations that remove the semantics of protected characteristics, and are therefore able to mitigate unfair outcomes. All available models however learn latent embeddings which comes at the cost of being uninterpretable. We propose to cast this problem as data-to-data translation, i.e. learning a mapping from an input domain to a fair target domain, where a fairness definition is being enforced. Here the data domain can be images, or any tabular data representation. This task would be straightforward if we had fair target data available, but this is not the case. To overcome this, we learn a highly unconstrained mapping by exploiting statistics of residuals -- the difference between input data and its translated version -- and the protected characteristics. When applied to the CelebA dataset of face images with gender attribute as the protected characteristic, our model enforces equality of opportunity by adjusting the eyes and lips regions. Intriguingly, on the same dataset we arrive at similar conclusions when using semantic attribute representations of images for translation. On face images of the recent DiF dataset, with the same gender attribute, our method adjusts nose regions. In the Adult income dataset, also with protected gender attribute, our model achieves equality of opportunity by, among others, obfuscating the wife and husband relationship. Analyzing those systematic changes will allow us to scrutinize the interplay of fairness criterion, chosen protected characteristics, and prediction performance.
The impact of automated decision-making systems on human lives is growing, emphasizing the need for these systems to be not only accurate but also fair. The field of algorithmic fairness has expanded ...significantly in the past decade, with most approaches assuming that training and testing data are drawn independently and identically from the same distribution. However, in practice, differences between the training and deployment environments exist, compromising both the performance and fairness of the decision-making algorithms in real-world scenarios. A new area of research has emerged to address how to maintain fairness guarantees in classification tasks when the data generation processes differ between the source (training) and target (testing) domains. The objective of this survey is to offer a comprehensive examination of fair classification under distribution shift by presenting a taxonomy of current approaches. The latter is formulated based on the available information from the target domain, distinguishing between adaptive methods, which adapt to the target environment based on available information, and robust methods, which make minimal assumptions about the target environment. Additionally, this study emphasizes alternative benchmarking methods, investigates the interconnection with related research fields, and identifies potential avenues for future research.
To reduce the inference cost of large language models, model compression is increasingly used to create smaller scalable models. However, little is known about their robustness to minority subgroups ...defined by the labels and attributes of a dataset. In this paper, we investigate the effects of 18 different compression methods and settings on the subgroup robustness of BERT language models. We show that worst-group performance does not depend on model size alone, but also on the compression method used. Additionally, we find that model compression does not always worsen the performance on minority subgroups. Altogether, our analysis serves to further research into the subgroup robustness of model compression.
PDF
