Using a game engine, we have developed a virtual environment which models important aspects of critical incident scenarios. We focused on modelling phenomena relating to the identification and ...gathering of key forensic evidence, in order to develop and test a system which can handle chemical, biological, radiological/nuclear or explosive (CBRNe) events autonomously. This allows us to build and validate AI-based technologies, which can be trained and tested in our custom virtual environment before being deployed in real-world scenarios. We have used our virtual scenario to rapidly prototype a system which can use simulated Remote Aerial Vehicles (RAV s) to gather images from the environment for the purpose of mapping. Our environment provides us with an effective medium through which we can develop and test various AI methodologies for critical incident scene assessment, in a safe and controlled manner.
We recently conducted a randomized double-blind study in which we demonstrated that moderate/severe chronic graft-versus-host disease (cGVHD) but not cGVHD-free survival was reduced in patients ...receiving anti-T lymphocyte globulin (ATLG) versus placebo. In a companion study we performed immunophenotypic analysis to determine the impact of ATLG on immune reconstitution (IR) and to correlate IR with clinical outcomes. The randomized study (n = 254) included patients (aged 18 to 65 years) who underwent myeloablative transplants for acute myeloid leukemia, myelodysplastic syndrome, or acute lymphoblastic leukemia from HLA-matched unrelated donors. Ninety-one patients consented for the companion IR study (ATLG = 44, placebo = 47). Blood samples were collected on days 30, 100, 180, and 360 after hematopoietic cell transplantation (HCT), and multiparameter flow cytometry was performed in a blinded fashion. Reconstitution of CD3+ and CD4+ T cells was delayed up to 6 months post-HCT in the ATLG arm, whereas absolute regulatory T cell (Treg) (CD4+25+127-) numbers were lower only in the first 100 days. Analysis of the CD4+ Treg and conventional T cells (Tconv) (CD4+25–127+) compartments showed a profound absence of naive Tregs and Tconv in the first 100 days post-HCT, with very slow recovery for 1 year. B cell and natural killer cell recovery were similar in each arm. Higher absolute counts of CD3+, CD4+, CD8+ T, Tregs, and Tconv were associated with improved overall survival, progression-free survival, and nonrelapse mortality but not moderate/severe cGVHD. Although ATLG delays CD3+ and CD4+ T cell recovery post-transplant, it has a relative Treg sparing effect after the early post-HCT period, with possible implications for protection from cGVHD. ATLG severely compromises the generation of naive CD4+ cells (Treg and Tconv), potentially affecting the diversity of the TCR repertoire and T cell responses against malignancy and infection.
This paper presents a new approach to classification of high dimensional spectroscopy data and demonstrates that it outperforms other current state-of-the art approaches. The specific task we ...consider is identifying whether samples contain chlorinated solvents or not, based on their Raman spectra. We also examine robustness to classification of outlier samples that are not represented in the training set (negative outliers). A novel application of a locally-connected neural network (NN) for the binary classification of spectroscopy data is proposed and demonstrated to yield improved accuracy over traditionally popular algorithms. Additionally, we present the ability to further increase the accuracy of the locally-connected NN algorithm through the use of synthetic training spectra and we investigate the use of autoencoder based one-class classifiers and outlier detectors. Finally, a two-step classification process is presented as an alternative to the binary and one-class classification paradigms. This process combines the locally-connected NN classifier, the use of synthetic training data, and an autoencoder based outlier detector to produce a model which is shown to both produce high classification accuracy, and be robust to the presence of negative outliers.
In this paper, we introduce a skill-balancing mechanism for adversarial non-player characters (NPCs), called Skilled Experience Catalogue (SEC). The objective of this mechanism is to approximately ...match the skill level of an NPC to an opponent in real-time. We test the technique in the context of a First-Person Shooter (FPS) game. Specifically, the technique adjusts a reinforcement learning NPC's proficiency with a weapon based on its current performance against an opponent. Firstly, a catalogue of experience, in the form of stored learning policies, is built up by playing a series of training games. Once the NPC has been sufficiently trained, the catalogue acts as a timeline of experience with incremental knowledge milestones in the form of stored learning policies. If the NPC is performing poorly, it can jump to a later stage in the learning timeline to be equipped with more informed decision-making. Likewise, if it is performing significantly better than the opponent, it will jump to an earlier stage. The NPC continues to learn in real-time using reinforcement learning but its policy is adjusted, as required, by loading the most suitable milestones for the current circumstances.
Multi-class classification algorithms are very widely used, but we argue that they are not always ideal from a theoretical perspective, because they assume all classes are characterized by the data, ...whereas in many applications, training data for some classes may be entirely absent, rare, or statistically unrepresentative. We evaluate one-sided classifiers as an alternative, since they assume that only one class (the target) is well characterized. We consider a task of identifying whether a substance contains a chlorinated solvent, based on its chemical spectrum. For this application, it is not really feasible to collect a statistically representative set of outliers, since that group may contain \emph{anything} apart from the target chlorinated solvents. Using a new one-sided classification toolkit, we compare a One-Sided k-NN algorithm with two well-known binary classification algorithms, and conclude that the one-sided classifier is more robust to unexpected outliers.
In current state-of-the-art commercial first person shooter games, computer controlled bots, also known as non player characters, can often be easily distinguishable from those controlled by humans. ...Tell-tale signs such as failed navigation, "sixth sense" knowledge of human players' whereabouts and deterministic, scripted behaviors are some of the causes of this. We propose, however, that one of the biggest indicators of non humanlike behavior in these games can be found in the weapon shooting capability of the bot. Consistently perfect accuracy and "locking on" to opponents in their visual field from any distance are indicative capabilities of bots that are not found in human players. Traditionally, the bot is handicapped in some way with either a timed reaction delay or a random perturbation to its aim, which doesn't adapt or improve its technique over time. We hypothesize that enabling the bot to learn the skill of shooting through trial and error, in the same way a human player learns, will lead to greater variation in game-play and produce less predictable non player characters. This paper describes a reinforcement learning shooting mechanism for adapting shooting over time based on a dynamic reward signal from the amount of damage caused to opponents.
While reinforcement learning (RL) has been applied to turn-based board games for many years, more complex games involving decision-making in real-time are beginning to receive more attention. A ...challenge in such environments is that the time that elapses between deciding to take an action and receiving a reward based on its outcome can be longer than the interval between successive decisions. We explore this in the context of a non-player character (NPC) in a modern first-person shooter game. Such games take place in 3D environments where players, both human and computer-controlled, compete by engaging in combat and completing task objectives. We investigate the use of RL to enable NPCs to gather experience from game-play and improve their shooting skill over time from a reward signal based on the damage caused to opponents. We propose a new method for RL updates and reward calculations, in which the updates are carried out periodically, after each shooting encounter has ended, and a new weighted-reward mechanism is used which increases the reward applied to actions that lead to damaging the opponent in successive hits in what we term "hit clusters".
This paper describes an architecture for controlling non-player characters (NPC) in the First Person Shooter (FPS) game Unreal Tournament 2004. Specifically, the DRE-Bot architecture is made up of ...three reinforcement learners, Danger, Replenish and Explore, which use the tabular Sarsa({\lambda}) algorithm. This algorithm enables the NPC to learn through trial and error building up experience over time in an approach inspired by human learning. Experimentation is carried to measure the performance of DRE-Bot when competing against fixed strategy bots that ship with the game. The discount parameter, {\gamma}, and the trace parameter, {\lambda}, are also varied to see if their values have an effect on the performance.
Dynamic Difficulty Adjustment (DDA) is a mechanism used in video games that automatically tailors the individual gaming experience to match an appropriate difficulty setting. This is generally ...achieved by removing pre-defined difficulty tiers such as Easy, Medium and Hard; and instead concentrates on balancing the gameplay to match the challenge to the individual's abilities. The work presented in this paper examines the implementation of DDA in a custom survival game developed by the author, namely Colwell's Castle Defence. The premise of this arcade-style game is to defend a castle from hordes of oncoming enemies. The AI system that we developed adjusts the enemy spawn rate based on the current performance of the player. Specifically, we read the Player Health and Gate Health at the end of each level and then assign the player with an appropriate difficulty tier for the proceeding level. We tested the impact of our technique on thirty human players and concluded, based on questionnaire feedback, that enabling the technique led to more enjoyable gameplay.
PDF