Roman Palace: A Videogame for Foreign-Language Vocabulary Retention

allarcos@doctor.upv.es Abstract— The goal of this work was to develop a serious game which would aid in the retention of foreign-language non-contextualized vocabulary items in different age categories. To this end, the loci method was used to enhance visual and spatial memory among the game’s players. The novelty of this proposal lies in combining mnemonic strategies within an exploratory video game environment. Structurally, this project consists of three main pillars: pedagogical foun-dations, the development of the serious game, and the evaluation of the game's efficiency through qualitative and quantitative study. The 3D game was developed through the Unity game engine in its version 2020.3. The prototype’s efficiency was assessed quantitatively in an experimental group of 21 participants who were compared with a control group of 27 participants. All the data were displayed in R Studio using the ggplot2 library. The study was complemented with qualitative findings collected through semi-structured interviews with a group of 7 participants. The experimental method proved to be more efficient and more entertaining than the conventional method used in the control


Background
The use of educational videogames is constantly expanding, together with the fast development of information technologies. The methodological progress of the last decades has led to the development and adaptation to the demands of the 21st century of technologies such as Computer Assisted Language Learning (CALL), started by Levy [1], Technology-Enhanced Language Learning (TELL), investigated by Bush and Roberts [2], and the Serious Games Initiative, introduced by Sawyer [3]. While various authors [4], [5], [6], [7] have reported the benefits of videogames in education in general, the didactic core of serious games distinguishes them from commercially distributed products that were initially designed for entertainment only [8]. As mentioned by Nazry [9], the educational element of serious games is thoroughly integrated into their design, thereby offering players both learning content and an entertaining structure.
The Roman Palace prototype presented in this study is an example of a serious game developed to enhance the retention of foreign-language (L2) vocabulary. Retention is obtained through the use of virtual flashcards and the implementation of spatial memory. In linguistics, a flashcard is typically a card that has a word on one side and its translation on the other, thus establishing the connection [10], [11]. The main function of the game Roman Palace is to do this by establishing spatial connections between the words on the cards and the objects in the Roman villa (through association, organisation, and the loci method). The players are told to imagine what every new word reminds them of, as well as to remember where in the game it appears (for example, the room or a nearby object). All these elements create extra memory support. From the point of view of psychology, mnemonic techniques use combinations, relations, and associations of ideas while at the same time exploring our capabilities [12].
One of the methods used, the loci method, was described as far back as 55 BC by Cicero [13]. "Loci" in Latin is the plural of "locus", which means place or location. This concept is also known by other names, such as the Roman Room or the Memory Palace. They all refer to a mnemonic strategy in which a familiar environment activates a mechanism of association [14] that facilitates memorisation. The number of vocabulary items and the amount of time allotted for retention in Roman Palace in the pilot study is based on the results of a study published by Larchen Costuchen, Darling and Uytman [15] which showed that retention of 10 new vocabulary items in 15 minutes was significantly better with the experimental method (flashcards linked to 3D models and viewed in a familiar environment on mobile devices) than with the conventional method consisting of flashcards with images presented through the application Quizlet for Windows, Android and iOS. The serious game Roman Palace aims to serve for different age groups as well as a variety of Indo-European languages. The pilot study focused on the 18-to-60 age group. The language used was not a real one but rather a group of pseudowords generated by an external tool. The effectiveness of vocabulary retention was measured in-game by testing the participants' translation and spelling of the words. The game also stored the time used to complete each level, the failed words in each test, and the scores obtained. Even though mnemonic strategies have been in use since Cicero's age [13] and are often mentioned in the academic literature [13], [16]- [24], they are not widely used in immersive environments for specific uses (such as L2 vocabulary acquisition), and scientific publications in this field are rare.
According to Levin and Levin [25] and Jiménez [12], mnemonic techniques can have positive effects not only on language learning, but on any type of learning. They do require a specific systematic approach. The loci method, for instance, involves selecting different elements within a familiar environment and then creating a mental image for every element that will be remembered together with each associated word [15], [26]- [29]. Recent studies show that visual immersive environments are more efficient for vocabulary memorisation than traditional methods. These studies include virtual reality [28], [29], augmented reality [15] , and the program Memory Palace Beta [30]. The latter is a multiplatform application available for Windows, Android and iOS devices that enables the construction of memory palaces using the loci method (https://www.memorypalace.com/). These palaces are composed of digital images of interconnected rooms where objects are placed on. These objects are associated with the information that is to be memorised. Routes are then drawn connecting all the rooms and objects. The present study has gone one step further, designing an exploratory game centred around its educational objectives.

Method and game development
This study aimed to design a serious game for non-contextual vocabulary learning in a foreign language (L2) using the loci mnemonic method. In this instance, the game used a set of pseudowords for L2, but future studies will collect data from different combinations of existing Indo-European Latin-alphabet languages. At start-up, the game first asks players to consent to the use of their data for statistical purposes. Once the permission is given, the app collects certain demographic data (gender, age, mother language, and country of residence). The game has specific training and entertainment objectives. The entertainment objectives include familiarising players with the Romanera-inspired setting, locating the cards hidden among the objects in the rooms, and overcoming the obstacles and time limits. The educational objectives include memorising the words on the cards and passing the tests. The main research question is whether the retention of new L2 vocabulary is enhanced by the serious game Roman Palace (the experimental method) in comparison with an alternative conventional method (a list of words presented in the form of a table on screen devices). The experimental hypothesis is that the serious game Roman Palace is more efficient than the screen-device tables for L2 vocabulary retention. Both qualitative and quantitative data were collected to analyse the results. The ages of the study participants ranged anywhere from 18 to 60 years.
This project used the videogame engine Unity (v. 2020.3), the database-management system SQLite; and the graphical user interface (GUI) LeanTween, an animation tool developed by Dented Pixel. The game was written in C#, a multi-paradigm programming language developed by Microsoft, who also created the integrated-development environment (IDE) used, Visual Studio 2019, which was chosen for its wide range of extensions and programming facilities available for Unity. Most of the game's Romanera-inspired models came from Unity Asset Store, Blend Swap and 3D Warehouse. We used the modelling tool Blender for creating some objects and model the Roman villa. As for the textures, most were obtained from 3D Textures, a free online texture library. The remainder was created using Adobe Photoshop CC 2020, a photo editor developed by Adobe Systems Incorporated that was also used to create the graphical user interfaces. The first version of Roman Palace's prototype was evaluated by two experts on the Roman era and ancient history whose suggestions improved the design of the game.
In order to determine the influence of Roman Palace on vocabulary learning in L1 and L2, this study enlisted 48 participants divided into two groups: an experimental group and a control one, both of whom used the same words. The L2 was not an existing language but a series of pseudo-words created by an online word generator (https://randomwordgenerator.com/). The experimental group used Roman Palace to memorise a set of 10 vocabulary items and another of 20, with the game itself calculating the scores and the time used ( Figure 1). As for the control group, they were given two lists ( Figure  2) containing the words that needed to be learned for each test (10 and 20 respectively). They also recorded the time used in the vocabulary-learning and problem-solving phases themselves. http://www.i-jet.org  In addition, individual semi-structured interviews were conducted with the participants from both groups to obtain confirmation about their experiences. Quantitative results The graphics showing the results obtained by the 21 participants from the experimental group (the Roman Palace video game) and the 27 controls that used the traditional method (lists of words in L1 and L2) were created using R Studio's ggplot2 library. Figure 3 shows the scores obtained on the final test by the participants in the experimental and control groups, the latter using the traditional method to memorise the 10 and 20 vocabulary items. The scores are ordinal variables expressed in round numbers: 0, 10, 20, 30, 40, 60, 70, 80, 90, and 100. The graph shows the percentage of each score obtained by each group on each test (10 and 20 items). In conclusion, the 10-word list yielded maximum test results (100 points) in a similar proportion (more than 50%) in both groups. The 20-item list, however, yielded a notably lower test result in the control group (around 30%) compared with the experimental group (46%). On the other hand, the experimental group obtained similar proportions (80%) of high results (90 points or more) on both tests, almost doubling the proportions of high test results obtained by the control group on both tests. Finally, the only failing grades (<60) occurred among control-group participants on the 20-word test.  Figure 4) shows that 7.1% more women than men obtained the highest possible score (100%) on the 10-word test. In the 20-word test, however, this advantage increased to the point where twice as many women as men obtained the highest possible score. Overall, however, the percentage of participants that obtained high scores (>80) was similar for both genders (80%).   (Figure 6) shows that the traditional method proved to be the fastest one for the 10-item list, with an average of 6 minutes (7 minutes in the experimental group), but it was also the slowest one for the 20-item list, needing an average of 12 minutes compared with the experimental group's 10 minutes. The box diagrams of the times needed for word memorisation and test completion show that the two groups perform in opposite ways. Thus, as the grouping of the control group's participants around low time values shows, the control group generally needed less time to memorise the 10item list than the experimental group. When it came to the 20 vocabulary items, however, the control group needed more time than their experimental counterparts, with 50% of the controls requiring more than 12 minutes, compared with only 25% of the experimental group.  Figure 7 shows the average time needed by both genders for both tests. In both the control and experimental groups the women needed an average of between 12 and 24 more seconds to learn the 10 words. In the 20-item test the women in the control group show an even more marked gender difference, needing almost a minute more. The results are similar for the experimental group, although it is the only group where the men took slightly more than the women (6 seconds on average).  Figure 8 shows how long it took both groups to complete the two tests by age and gender. Both graphics show the linear regression accompanied by the 95% confidence interval. There appears to be no relation between age and time needed in the experimental group, while the control group shows a clear linear relation in both genders, i.e. the older the person, the more minutes needed. Qualitative results A total of 7 people (experimental group/EG = 4 and control group/CG = 3) were interviewed about their experiences with the methods used. The interviews were individual and semi-structured, conducted on the platform Zoom and lasting between 15 and 20 minutes. No group interviews were conducted because they were considered to entail the risk of participants influencing and interfering with one another. The data were analysed using a four-category semantic scale: "Question 1. Opinion of the method used", "Question 2. How did I feel during the tests", "Question 3. Highlights", and "Question 4. Prior experience with the method used." The results are shown below.
─ Quantitative Profile EG-14: man, 30 years, time 10 items: 7 minutes; time 20 items: 11 minutes; score level 1: 90%, score level 2: 100%. Opinion of the method (semantic methods used): "very easy", "fun". How they felt during the tests: "Actually, I was more focused on finding the objects than on memorising the words, but it worked. When I got to the tests, I did get it right. I remembered almost everything." "When I started playing, I first did a little tour towards the fountain to see how many rooms there were. Then I returned to the starting point and started entering the rooms, looking at the type of rooms and the modern objects. Some rooms looked quite similar. What set them apart was their location with reference to the fountain. I tried not to get lost and to take into account the number of objects I had already found. Even so I entered some rooms more than once". Highlights: "Some combinations of objects and their positions in level 2 were funny. The helmet that was on the statue and the glasses that, instead of being on the table, were on a bust." Prior experience with the method used: the participant has no prior experience with mnemonics, but has played video games before. "When I was young I played Doom and as a kid some 2D and 3D games." "I haven't played in years." ─ Quantitative Profile EG-9: man; 38 years; time 10 items: 9 minutes; time 20 items: 16 minutes; score level 1: 100%; score level 2: 90%. Opinion of the method (semantic methods used): "easy", "more fun than a notebook with translations", "I would try again with other words, I liked it." How they felt during the tests: "I had to enter some rooms more than two or three times. Things got especially complicated at the end of level 1 with my last object (a rocket in the water). In level 2 there was this new room (kitchen/pantry) that I just couldn't find." Highlights: "There were some funny combinations. For example, in the kitchen there was a modern milk carton with a drawing of a cow and on the card it said "horsetheoscorian." Prior experience with the method used: the participant has no prior experience with mnemonics, but has played video games before. "I used to play Monkey Island and other graphic adventures." "My favourite now: MMOG." ─ Quantitative Profile EG-7: woman; 26 years; time 10 items: 9 minutes; time 20 items: 9 minutes; score level 1: 90%; score level 2: 100%. Opinion of the method (semantic methods used): "it takes getting used to", "I liked the idea", "it's not a hard method at all". How they felt during the tests: "Curiously, level 1 was harder for me than level 2. I had to learn to understand the system and how to use the mouse and keys. I went more slowly in level 1, but level 2 was really fast. I was very aware of where I was going and of whether I had been there before. I tried not to lose any time. I found the translation tests easier than the ones where you had to fill in letters. In any case, I had no trouble remembering the vocabulary. When I saw the list in the beginning, I thought it was going to be much harder." Highlights: "It took me a while to find a modern object in the vegetable pantry. I got stuck, and then I realised it was the potatoes." Prior experience with the method used: the participant has no prior experience with mnemonics, but has played video games before. "I used to play Super Mario and some other games with my brother. Right now I'm not playing anything. A couple of years ago I tried Angry Birds on my phone, and I was hooked for a few weeks." ─ Quantitative Profile EG-8: woman; 24 years; time 10 items: 10 minutes; time 20 items: 10 minutes; score level 1: 100%; score level 2: 100%. Opinion of the method (semantic methods used): "fantastic", "very user-friendly". How they felt during the tests: "I would have preferred to play on my phone or tablet, but I understand that this type of game has been designed for computers. I used the 34KB version because with the 64KB version the character would move a bit slowly on my laptop. The game looked nice and fun to me in spite of it being an educational tool. Even now I remember many words. I would use it again to improve my English." Highlights: "In general, I remember not only the words but also where I found the cards, in which rooms, and near which objects. Sometimes some element escapes me. If I have to go over all the words, I first remember the room, then the object, and then the word in Spanish with its translation into the made-up language. When I did the tests, it was the other way around: first the word, then the object, then the room." Prior experience with the method used: the participant has no prior experience with mnemonics, but has played video games before. "I used to play when I was little, but I don't remember the names. I tried Pokémon Go a few years ago, but I didn't like it. Sometimes I do Sudoku on my cell." ─ Quantitative Profile CG-45: woman; 37 years; time 10 items: 5 minutes; time 20 items: 12 minutes; score level 1: 100%; score level 2: 90%. Opinion of the method (semantic methods used): "difficult", "requires a lot of patience". How they felt during the tests: "I looked at the screen and repeated the combination in order: first column and second column." "The words that were more than two syllables were more difficult to memorise than the monosyllabic ones or those that were two syllables." "One thing that I found very complicated was the absence of vowels between some combinations of consonants." Highlights: "The 10-item test felt much easier to me than the 20-item test. Spelling tests 1.1 and 1. 2 were more complicated than tests 2.1 and 2.2. Maybe because the second time I knew what to expect and I was better prepared." Prior experience with the method used: the participant has no prior experience with mnemonics, but has used similar methods for learning vocabulary before. "I had often taken notes and made diagrams to memorise formulas and words in English, French, and Valencian. It's an easy method but very monotonous." ─ Quantitative Profile CG-45: woman; 49 years; time 10 items: 6 minutes; time 20 items: 15 minutes; score level 1: 100%; score level 2: 100%. Opinion of the method (semantic methods used): "It's not very hard, but it's repetitive", "The more words you have to memorise, the more tiring it gets." How they felt during the tests: "I realised that I got very easily distracted by anything else (cell phone, my cat, etc.). I found it impossible to stay concentrated during the whole exercise." Highlights: "Some of the words in the test were rather easy, and I remembered them almost immediately (chairstraline, gamestranians, and unionunelind in part 1, and shopservit, bearnocobot, dogsmulaps and italianomilato in part 2)." Prior experience with the method used: the participant has no prior experience with mnemonics, but has used similar methods for learning vocabulary before. "I take notes both on my phone and in a notebook, and I also like to use cork wallpaper with reminders and schedules." ─ Quantitative Profile CG-39: woman; 36 years; time 10 items: 7 minutes; time 20 items: 13 minutes; score level 1: 100%; score level 2: 100%. Opinion of the method (semantic methods used): "complicated", "requires a lot of dedication", "might have been easier on a printed sheet". How they felt during the tests: "To better memorise I had to repeat the words out loud. The language looked like Latin or some Nordic language. It might have been easier for me with another language (English, German, French, or Italian)." Highlights: "The 20 words were more complicated than the 10. As for the tests, I found translating easier than the spelling test. The most complicated words for me were brinercionized and kisheanicirch." Prior experience with the method used: the participant has no prior experience with mnemonics, but has used similar methods for learning vocabulary before. "I'm used to memorising texts, sentences, words, and telephone numbers at my job (office clerk). I use paper notes, Post-Its next to the computer, Samsung Notes, and electronic files on the computer."

Results
The experimental method proved to be more efficient (according to the numerical data) and more entertaining (according to the opinions of the participants) than the conventional method used in the control group. There were more men in the experimental group but fewer in the control group. The highest scores were obtained by women in the experimental groups, but in the control group they obtained the lowest scores. This leads us to believe that future studies should explore these differences and record how men and women in both experimental and control groups memorise, play, and behave, as well as whether any gender pays more attention to the play elements of the game than to the learning tasks. It would also be worthwhile to analyse how many seconds and how often participants spend looking at the word cards and whether this bears any relation to their results both in the intermediate and final tests.

Conclusion: Limitations and future prospects
This project is the prototype of a much more complex serious game. It is meant to be the base on which will be built a mnemonic system that will enrich the field of newlanguage learning by offering an alternative to conventional study methods. However, time restraints prevented some useful improvements from being incorporated: customisation of the character's appearance (this could increase immersion and thus enhance retention), more interactive user interfaces, the possibility of saving progress in a nonfinished level (currently the program can only save one's progress at the completion of a level), internationalising the application (the availability of multiple languages would make the game accessible to a much wider audience, allowing data to be collected from disparate regions), offering a more complex and memorable story, adding the specified levels and tests, using a translation service (the user would only have to complete the L1 word list and choose the L2, and the system would automatically translate everything else; this would also make it possible to add auditory and pronunciation tests, creating a more all-round, complete learning experience), adapting to virtual reality (generating a version of the game that can be played in a VR environment, greatly enhancing player immersion), improving the game's performance, and, finally, replacing the game's database-management system-SQLite-with one that is cloud-based, centralising and thus facilitating the collection of data. We believe the modern world is changing us and that more attention needs to be paid to the possibilities that new technologies can offer us [32]- [34]. Our objective with this project is to turn the memorisation of non-contextualised vocabulary items or any other pieces of information into a more exciting and immersive task.