Reinforcement Learning (RL) agents are commonly thought of as adaptive decision procedures. They work on input/output data streams called “states”, “actions” and “rewards”. Most current research ...about RL adaptiveness to changes works under the assumption that the streams signatures (
i.e.
arity and types of inputs and outputs) remain the same throughout the agent lifetime. As a consequence, natural situations where the signatures vary (
e.g.
when new data streams become available, or when others become obsolete) are not studied. In this paper, we relax this assumption and consider that signature changes define a new learning situation called Protean Learning (PL). When they occur, traditional RL agents become undefined, so they need to restart learning. Can better methods be developed under the PL view? To investigate this, we first construct a stream-oriented formalism to properly define PL and signature changes. Then, we run experiments in an idealized PL situation where input addition and deletion occur during the learning process. Results show that a simple PL-oriented method enables graceful adaptation of these arity changes, and is more efficient than restarting the process.
When we make rapid reaching movements, we have to trade speed for accuracy. To do so, the trajectory of our hand is the result of an optimal balance between feed-forward and feed-back control in the ...face of signal-dependant noise in the sensorimotor system. How far do these principles of trajectory formation still apply after a stroke, for persons with mild to moderate sensorimotor deficits who recovered some reaching ability? Here, we examine the accuracy of fast hand reaching movements with a focus on the information capacity of the sensorimotor system and its relation to trajectory formation in young adults, in persons who had a stroke and in age-matched control participants. We find that persons with stroke follow the same trajectory formation principles, albeit parameterized differently in the face of higher sensorimotor uncertainty. Higher directional errors after a stroke result in less feed-forward control, hence more feed-back loops responsible for segmented movements. As a consequence, movements are globally slower to reach the imposed accuracy, and the information throughput of the sensorimotor system is lower after a stroke. The fact that the most abstract principles of motor control remain after a stroke suggests that clinicians can capitalize on existing theories of motor control and learning to derive principled rehabilitation strategies.
In this paper, we propose a game adaptation technique that seeks to improve the training outcomes of stroke patients during a therapeutic session. This technique involves the generation of customized ...game levels, which difficulty is dynamically adjusted to the patients’ abilities and performance. Our goal was to evaluate the effect of this adaptation strategy on the training outcomes of post-stroke patients during a therapeutic session. We hypothesized that a dynamic difficulty adaptation strategy would have a more positive effect on the training outcomes of patients than two control strategies, incremental difficulty adaptation and random difficulty adaptation. To test these strategies, we developed three versions of PRehab, a serious game for upper-limb rehabilitation. Seven stroke patients and three therapists participated in the experiment, and played all three versions of the game on a graphics tablet. The results of the experiment show that our dynamic adaptation technique increases movement amplitude during a therapeutic session. This finding may serve as a basis to improve patient recovery.
In this paper, we propose a dynamic difficulty adaptation approach for serious games dedicated to upper-limb rehabilitation after stroke. The proposed approach aims to provide a personalized ...rehabilitation session in which the training intensity and challenges can be adapted to patient's abilities and training needs. The objective is to increase the rehabilitation volume by getting the patient engaged to the therapy session. This approach has been implemented and tested through a point and click game. Finally, we present a pilot experiment that we have conducted with healthy people in order to explain how this approach has been implemented and integrated to post-stroke therapeutic games.
PDF
