This study investigates the strategic decision-making abilities of large language models (LLMs) via the game of Tic-Tac-Toe, renowned for its straightforward rules and definitive outcomes. We ...developed a mobile application coupled with web services, facilitating gameplay among leading LLMs, including Jurassic-2 Ultra by AI21, Claude 2.1 by Anthropic, Gemini-Pro by Google, GPT-3.5-Turbo and GPT-4 by OpenAI, Llama2-70B by Meta, and Mistral Large by Mistral, to assess their rule comprehension and strategic thinking. Using a consistent prompt structure in 10 sessions for each LLM pair, we systematically collected data on wins, draws, and invalid moves across 980 games, employing two distinct prompt types to vary the presentation of the game’s status. Our findings reveal significant performance variations among the LLMs. Notably, GPT-4, GPT-3.5-Turbo, and Llama2 secured the most wins with the list prompt, while GPT-4, Gemini-Pro, and Mistral Large excelled using the illustration prompt. GPT-4 emerged as the top performer, achieving victory with the minimum number of moves and the fewest errors for both prompt types. This research introduces a novel methodology for assessing LLM capabilities using a game that can illuminate their strategic thinking abilities. Beyond enhancing our comprehension of LLM performance, this study lays the groundwork for future exploration into their utility in complex decision-making scenarios, offering directions for further inquiry and the exploration of LLM limits within game-based frameworks.
Reading behaviour is an important factor in building a society that is ready to face the global competition era. Reading behaviour has become important since it helps people to develop their mind; ...expand creativity and imagination; and discover new things. However, the average reading behaviour of Indonesians, especially for the young generation, is still low. It is evidenced by most of the youth nowadays prefer using their smartphones rather than reading books. With the available technology of smartphones and tablets, it could become one of the many potential ways in addressing the lack of their reading behaviours f. The utilization of a text-based game could encourage and motivate people to be involved in more reading activities. This paper describes a study to develop and enhance a text-based game that has a potential feature to motivate the reading behaviour of youths. The outcome of this study is to evaluate the possibility of text-based games as an alternative method in encouraging the young generation to improve their reading habits and behaviours.
Material artifacts included in video game packaging, referred to in the industry as feelies, operate as media paratexts that are both extensions of and separate from the video games that inspired ...them. Although most discourses on video game feelies are centered on 1980s text-based adventure games, feelies have continually been included in contemporary games, albeit primarily in collector's or special editions. To explore the diversity of feelies and how they are able to generate their own texts away from the digital game itself, I identify two specific types of feelies: artifact feelies, which are life-size reproductions of objects from within the game space, and collectible feelies, which serve as extensions of the game space into the physical realm but tend to include objects more frequently associated with fan collecting activities. Taking an interdisciplinary approach that includes material culture studies and media studies, I show how feelies allow scholars to gain further insight into how screen media operate away from the screens themselves, how the accumulation of material objects in the digital age encourages us to reevaluate our notions of the material and the immaterial, and how the concept of play is crucial to understanding how these objects are reappropriated in ways that move beyond their originally intended use.
First aid skills are an important part of medical's competency. The set of instructions for first aid operations are officially approved by the state. These instruction texts are the algorithms. ...Medical students are studying these algorithms in special course. First of all, we convert the instructions from text to the graphical flowcharts (according to ISO 5807-85 standard) for checking the ambiguity and possible misunderstanding. The execution process of such algorithms is one of typical "complex openended assignments". We have the classification of typical user errors. On the base of these errors we construct the set of alternative choices for all steps of algorithm. Every such set will convert to the answers for multiple choice questions (MCQ). There are repeated cyclic question for the student (executor): "what you will do?" or "what is your next operation?" We plan to build a special environment for gamification of the learning process. The short version will have only one right way (sequence of answers). Every wrong answer will lead to the error message - "your patient is dead" and explanation why it happened. In the more complex model, we evaluate the patient state and students can read the comments and the errors list only after the end of the algorithm execution. This year (2020) we plan to make the first iteration: text-based online adventure game, one content set, based on the first aid instructions that are approved in Ukraine. The next iteration will use first aid instructions that are accepted in other countries, starting from countries of the EU. We suppose the future development of this game will be like a well-known history of the evolution of computer games. This project will be part of the second co-author PhD. thesis.
Within the tradition of Digital Game courses, the use and teaching of programming is seen as essential for students to work on skills and gain knowledge to develop an artifact. However, for students ...at a more basic level, this can be a major difficulty considering the complexity of understanding a language they have never seen before. Narrative, sound, and music are critical elements in game creation and can be used to teach programming concepts in a more intuitive and engaging way. Recently, the "Creative Experience: Text-Based Game Development" module has strengthened the use of narrative as a way of teaching programming concepts. Narrative creates contexts and stories for players, while sound provides feedback and guidance during the game. According to Kelleher (2006), the use of games to teach programming is an effective form of learning because it allows students to experiment with concepts in a fun and playful way and develops important skills such as problem-solving, critical thinking, and teamwork. The use of narrative, sound, and music in games can be an effective way to teach complex programming concepts clearly and engagingly. This is an article that addresses the strategies created in the digital game module "Creative Experience: Text-Based Game Development" exploring narrative, sound, and music as learning elements for programming and comparing the results within the period of two course offerings, one with only programming teaching and another with the introduction of narrative and sound as the basis for constructing this knowledge. It is hoped that these teaching strategies can contribute to the development of valuable skills in the field of education and programming.
There has been an active movement towards fun learning, using games in education. This article introduces the text-based serious game “Rise of the Java Emperor” that aims to support students in ...learning basic object-oriented programming concepts using Java. Information concerning the analysis, the design and the pilot evaluation of the game is presented. Thirty-three undergraduate and postgraduate students of an Applied Informatics Department voluntarily played and answered a questionnaire based on the MEEGA+ model, in order to investigate the perceived player experience and short-term learning as well as the acceptance of a text-based programming game by students. The results of the evaluation show that text based games can be both fun and instructional for the field of computer programming. An important issue that requires further research is how this or other programming games can be successfully combined with traditional teaching methods for enhancing the learning of programming.
Communication via natural language is a key aspect of machine intelligence, and it requires computational models to learn and reason about world concepts, with varying levels of supervision. ...Significant progress has been made on fully-supervised non-interactive tasks, such as question-answering and procedural text understanding. Yet, various sequential interactive tasks, as in text-based games, have revealed limitations of existing approaches in terms of coherence, contextual awareness, and their ability to learn effectively from the environment. In this paper, we propose a knowledge-injection framework for improved functional grounding of agents in text-based games. Specifically, we consider two forms of domain knowledge that we inject into learning-based agents: memory of previous correct actions and affordances of relevant objects in the environment. Our framework supports two representative model classes: reinforcement learning agents and language model agents. Furthermore, we devise multiple injection strategies for the above domain knowledge types and agent architectures, including injection via knowledge graphs and augmentation of the existing input encoding strategies. We experiment with four models on the 10 tasks in the ScienceWorld text-based game environment, to illustrate the impact of knowledge injection on various model configurations and challenging task settings. Our findings provide crucial insights into the interplay between task properties, model architectures, and domain knowledge for interactive contexts.
In this paper, we study transformers for text-based games. As a promising replacement of recurrent modules in Natural Language Processing (NLP) tasks, the transformer architecture could be treated as ...a powerful state representation generator for reinforcement learning. However, the vanilla transformer is neither effective nor efficient to learn with a huge amount of weight parameters. Unlike existing research that encodes states using LSTMs or GRUs, we develop a novel lightweight transformer-based representation generator featured with reordered layer normalization, weight sharing and block-wise aggregation. The experimental results show that our proposed model not only solves single games with much fewer interactions, but also achieves better generalization on a set of unseen games. Furthermore, our model outperforms state-of-the-art agents in a variety of man-made games.
In order to train a computer agent to play a text-based computer game, we must represent each hidden state of the game. A Long Short-Term Memory (LSTM) model running over observed texts is a common ...choice for state construction. However, a normal Deep Q-learning Network (DQN) for such an agent requires millions of steps of training or more to converge. As such, an LSTM-based DQN can take tens of days to finish the training process. Though we can use a Convolutional Neural Network (CNN) as a text-encoder to construct states much faster than the LSTM, doing so without an understanding of the syntactic context of the words being analyzed can slow convergence. In this paper, we use a fast CNN to encode position-and syntax-oriented structures extracted from observed texts as states. We additionally augment the reward signal in a universal and practical manner. Together, we show that our improvements can not only speed up the process by one order of magnitude but also learn a superior agent.
PDF
