Past research has revealed cognitive improvements resulting from engagement with both traditional action video games and newer action-like video games, such as action real-time strategy games (ARSG). ...However, the cortical dynamics elicited by different video gaming genres remain unclear. This study explored the temporal dynamics of cortical networks in response to different gaming genres. Functional magnetic resonance imaging (fMRI) data were obtained during eye-closed resting and passive viewing of gameplay videos of three genres: life simulation games (LSG), first-person shooter games (FPS), and ARSG. Data analysis used a seed-free Co-Activation Pattern (CAP) based on Regions of Interest (ROIs). When comparing the viewing of action-like video games (FPS and ARSG) to LSG viewing, significant dynamic distinctions were observed in both primary and higher-order networks. Within action-like video games, compared to FPS viewing, ARSG viewing elicited a more pronounced increase in the Fraction of Time and Counts of attentional control-related CAPs, along with an increased Transition Probability from sensorimotor-related CAPs to attentional control-related CAPs. Compared to ARSG viewing, FPS viewing elicited a significant increase in the Fraction of Time of sensorimotor-related CAPs, when gaming experience was considered as a covariate. Thus, different video gaming genres, including distinct action-like video gaming genres, elicited unique dynamic patterns in whole-brain CAPs, potentially influencing the development of various cognitive processes.
•Four pairs of co-activation patterns represented distinct functional correlations.•Dynamic structures of co-activation patterns induced by different gaming viewing.•Action-like video games and LSG induced extensive dynamic differences.•FPS and ARSG induced distinct dynamic patterns in co-activation patterns.
We investigate the use of artificial intelligence (AI)-based techniques in learning to play a 2-player, real-time strategy (RTS) game called Hunting-of-the-Plark. The game is challenging to play for ...both humans and AI-based techniques because players cannot observe each other's moves while playing the game and one player is at a disadvantage due to the asymmetric nature of the game rules. We analyze the performance of different deep reinforcement learning algorithms to train software agents that can play the game. Existing reinforcement learning techniques for RTS games enable players to converge towards an equilibrium outcome of the game but usually do not facilitate further exploration of techniques to exploit and defeat the opponent. To address this shortcoming, we investigate techniques including self-play and strategy diversity that can be used by players to improve their performance beyond the equilibrium outcome. We observe that when players use self-play, their number of wins begins to cycle around an equilibrium value as each player quickly learns to outwit and defeat its opponent and vice-versa. Finally, we show that strategy diversity could be used as an effective means to alleviate the performance of the disadvantaged player caused by the asymmetric nature of the game.
Professional StarCraft game players are likely to focus on the management of the most important group of units (called the main force) during gameplay. Although macro-level skills have been observed ...in human game replays, there has been little study of the high-level knowledge used for tactical decision-making, nor exploitation thereof to create AI modules. In this paper, we propose a novel tactical decision-making model that makes decisions to control the main force. We categorized the future movement direction of the main force into six classes (e.g., toward the enemy’s main base). The model learned to predict the next destination of the main force based on the large amount of experience represented in replays of human games. To obtain training data, we extracted information from 12,057 replay files produced by human players and obtained the position and movement direction of the main forces through a novel detection algorithm. We applied convolutional neural networks and a Vision Transformer to deal with the high-dimensional state representation and large state spaces. Furthermore, we analyzed human tactics relating to the main force. Model learning success rates of 88.5%, 76.8%, and 56.9% were achieved for the top-3, -2, and -1 accuracies, respectively. The results show that our method is capable of learning human macro-level intentions in real-time strategy games.
Research showed that action real-time strategy gaming (ARSG) experience is related to cognitive and neural plasticity, including visual selective attention and working memory, executive control, and ...information processing. This study explored the relationship between ARSG experience and information transmission in the auditory channel. Using an auditory, two-choice, go/no-go task and lateralized readiness potential (LRP) as the index to partial information transmission, this study examined information transmission patterns in ARSG experts and amateurs. Results showed that experts had a higher accuracy rate than amateurs. More importantly, experts had a smaller stimulus-locked LRP component (250 – 450 ms) than amateurs on no-go trials, while the response-locked LRP component (0 – 300 ms) on go trials did not differ between groups. Thus, whereas amateurs used an asynchronous information transmission pattern, experts used a reduced asynchronous information transmission pattern or a synchronous pattern where most of processing occurred prior to response execution – an information transmission pattern that supports rapid, error-free performance. Thus, experts and amateurs may use different information transmission patterns in auditory processing. In addition, the information transmission pattern used by experts is typically observed only after long-term auditory training according to past research. This study supports the relationship between ARSG experience and the development of information processing patterns.
Invalid action masking is a practical technique in deep reinforcement learning to prevent agents from taking invalid actions. Existing approaches rely on action masking during policy training and ...utilization. This study focuses on developing reinforcement learning algorithms that incorporate action masking during training but can be used without action masking during policy execution. The study begins by conducting a theoretical analysis to elucidate the distinction between naive policy gradient and invalid action policy gradient. Based on this analysis, we demonstrate that the naive policy gradient is a valid gradient and is equivalent to the proposed composite objective algorithm, which optimizes both the masked policy and the original policy in parallel. Moreover, we propose an off-policy algorithm for invalid action masking that employs the masked policy for sampling while optimizing the original policy. To compare the effectiveness of these algorithms, experiments are conducted using a simplified real-time strategy (RTS) game simulator called Gym-μRTS. Based on empirical findings, we recommend utilizing the off-policy algorithm for addressing most tasks while employing the composite objective algorithm for handling more complex tasks.
Real-time strategy games are one of the important scenarios for studying multi-agent reinforcement learning, and there have been some researchers who have achieved some results in the field. However, ...limited by problems such as the complexity of the environment, these methods not only take up a large number of computational resources but also require a long training time. We based on the idea of curriculum Learning, the training process of real-time strategy game models is turned into an incremental level training. It can reduce the time cost on model training and the amount of resources required. In this study, we use StarCraft 2 as a simulation environment for real-time strategy games, and use PPO as the base algorithm to design a reinforcement learning model training method that incorporates the idea of curriculum Learning. We hope that this study is a guide to improve the efficiency of multi-agent reinforcement learning.
Real-time strategy (RTS) game is a kind of strategy game, in which the players compete for resources on 2D terrain by establishing the economy, training army, and guiding them into battle in real ...time. The winner prediction of the RTS games often involves studying a highly uncertain problem in an adversarial environment. In addition, the limit of the number of samples restricts on the application and performance of the prediction models. To obtain better winner prediction accuracy and maintain the prediction uncertainty under an adversarial environment, this paper proposes a neural network-based prediction method incorporated probability inference dealing with a small set of samples. This paper uses a dataset released based on SC2LE, a reinforcement learning environment released jointly by Blizzard Entertainment and DeepMind, and then employed the proposed neural processes model to build a winner prediction model. To verify, this paper implemented different features types' grouping and different game length grouping experiments for demonstrating better adaptability to such problems. Furthermore, this paper also implemented the SVM model experiments and compared the proposed method with the SVM model. Finally, when making predictions on a 1000 size testing data, the results show that the proposed prediction model achieves an accuracy of 0.811 at 200 and 0.821 at 1000 sizes of training sets, which is better than the SVM model with small training datasets.
PDF
