Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement ...learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.
Semantic interoperability for the Internet of Things (IoT) is enabled by standards and technologies from the Semantic Web. As recent research suggests a move towards decentralised IoT architectures, ...we have investigated the scalability and robustness of RDF (Resource Description Framework)engines that can be embedded throughout the architecture, in particular at edge nodes. RDF processing at the edge facilitates the deployment of semantic integration gateways closer to low-level devices. Our focus is on how to enable scalable and robust RDF engines that can operate on lightweight devices. In this paper, we have first carried out an empirical study of the scalability and behaviour of solutions for RDF data management on standard computing hardware that have been ported to run on lightweight devices at the network edge. The findings of our study shows that these RDF store solutions have several shortcomings on commodity ARM (Advanced RISC Machine) boards that are representative of IoT edge node hardware. Consequently, this has inspired us to introduce a lightweight RDF engine, which comprises an RDF storage and a SPARQL processor for lightweight edge devices, called RDF4Led. RDF4Led follows the RISC-style (Reduce Instruction Set Computer) design philosophy. The design constitutes a flash-aware storage structure, an indexing scheme, an alternative buffer management technique and a low-memory-footprint join algorithm that demonstrates improved scalability and robustness over competing solutions. With a significantly smaller memory footprint, we show that RDF4Led can handle 2 to 5 times more data than popular RDF engines such as Jena TDB (Tuple Database) and RDF4J, while consuming the same amount of memory. In particular, RDF4Led requires 10%-30% memory of its competitors to operate on datasets of up to 50 million triples. On memory-constrained ARM boards, it can perform faster updates and can scale better than Jena TDB and Virtuoso. Furthermore, we demonstrate considerably faster query operations than Jena TDB and RDF4J.
A 19-Year-Old Man with a Cavitating Lung Lesion Hayes, Conor J; O'Farrell, Naoimh J; Fabre, Aurelie ...
Annals of the American Thoracic Society,
11/2022, Letnik:
19, Številka:
11
Journal Article
We propose a novel multi-objective reinforcement learning algorithm that successfully learns the optimal policy even for non-linear utility functions. Non-linear utility functions pose a challenge ...for SOTA approaches, both in terms of learning efficiency as well as the solution concept. A key insight is that, by proposing a critic that learns a multi-variate distribution over the returns, which is then combined with accumulated rewards, we can directly optimize on the utility function, even if it is non-linear. This allows us to vastly increase the range of problems that can be solved compared to those which can be handled by single-objective methods or multi-objective methods requiring linear utility functions, yet avoiding the need to learn the full Pareto front. We demonstrate our method on multiple multi-objective benchmarks, and show that it learns effectively where baseline approaches fail.
The recent paper “Reward is Enough” by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial, and ...provides a suitable basis for the creation of artificial general intelligence. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In this paper we explain why scalar rewards are insufficient to account for some aspects of both biological and computational intelligence, and argue in favour of explicitly multi-objective models of reward maximisation. Furthermore, we contend that even if scalar reward functions can trigger intelligent behaviour in specific cases, this type of reward is insufficient for the development of human-aligned artificial general intelligence due to unacceptable risks of unsafe or unethical behaviour.
In many risk-aware and multi-objective reinforcement learning settings, the utility of the user is derived from a single execution of a policy. In these settings, making decisions based on the ...average future returns is not suitable. For example, in a medical setting a patient may only have one opportunity to treat their illness. Making decisions using just the expected future returns–known in reinforcement learning as the value–cannot account for the potential range of adverse or positive outcomes a decision may have. Therefore, we should use the distribution over expected future returns differently to represent the critical information that the agent requires at decision time by taking both the future and accrued returns into consideration. In this paper, we propose two novel Monte Carlo tree search algorithms. Firstly, we present a Monte Carlo tree search algorithm that can compute policies for nonlinear utility functions (NLU-MCTS) by optimising the utility of the different possible returns attainable from individual policy executions, resulting in good policies for both risk-aware and multi-objective settings. Secondly, we propose a distributional Monte Carlo tree search algorithm (DMCTS) which extends NLU-MCTS. DMCTS computes an approximate posterior distribution over the utility of the returns, and utilises Thompson sampling during planning to compute policies in risk-aware and multi-objective settings. Both algorithms outperform the state-of-the-art in multi-objective reinforcement learning for the expected utility of the returns.
Prostate-specific antigen (PSA) is the best serum marker currently available for the detection of prostate cancer and is the forensic marker of choice for determining the presence of azoospermic ...semen in some sexual assault cases. Most current assays for PSA detection are processed on large analyzers at dedicated testing sites, which require that samples be sent away for testing. This leads to delays in patient management and increased administration costs. The recent emphasis placed on the need for point-of-care patient management has led to the development of novel biosensor detection strategies that are suitable for the miniaturization of assays for various targets including PSA. This review highlights the current and novel analytical technologies used for PSA detection, which will benefit clinicians, patients and forensic workers in the future.
Abstract
Neoantigens derived from somatic mutations are specific to cancer cells and are ideal targets for cancer immunotherapy.
KRAS
is the most frequently mutated oncogene and drives the ...pathogenesis of several cancers. Here we show the identification and development of an affinity-enhanced T cell receptor (TCR) that recognizes a peptide derived from the most common KRAS mutant, KRAS
G12D
, presented in the context of HLA-A*11:01. The affinity of the engineered TCR is increased by over one million-fold yet fully able to distinguish KRAS
G12D
over KRAS
WT
. While crystal structures reveal few discernible differences in TCR interactions with KRAS
WT
versus KRAS
G12D
, thermodynamic analysis and molecular dynamics simulations reveal that TCR specificity is driven by differences in indirect electrostatic interactions. The affinity enhanced TCR, fused to a humanized anti-CD3 scFv, enables selective killing of cancer cells expressing KRAS
G12D
. Our work thus reveals a molecular mechanism that drives TCR selectivity and describes a soluble bispecific molecule with therapeutic potential against cancers harboring a common shared neoantigen.
Malignant pleural disease Piggott, Laura M; Hayes, Conor; Greene, John ...
Breathe (Lausanne, Switzerland),
12/2023, Letnik:
19, Številka:
4
Journal Article
Recenzirano
Odprti dostop
Malignant pleural disease represents a growing healthcare burden. Malignant pleural effusion affects approximately 1 million people globally per year, causes disabling breathlessness and indicates a ...shortened life expectancy. Timely diagnosis is imperative to relieve symptoms and optimise quality of life, and should give consideration to individual patient factors. This review aims to provide an overview of epidemiology, pathogenesis and suggested diagnostic pathways in malignant pleural disease, to outline management options for malignant pleural effusion and malignant pleural mesothelioma, highlighting the need for a holistic approach, and to discuss potential challenges including non-expandable lung and septated effusions.
PDF
