Interactive Science Fiction Prototyping in Virtual Worlds: Fundamentals and Applications

Science Fiction Prototyping (SFP) is a powerful tool to imagine, explore, and exploit future technologies, science, and environments and it can be a valuable asset for education, multimedia, and research. This article explores SFP in interactive, flexible, immersive, and collaborative settings, which introduced as Interactive Science Fiction Prototyping (ISFP) and allows new forms of experiencing and reflecting on prototypes. We investigate how to integrate the ISFP process into an existing collaborative virtual world platform and outline required integration steps. Furthermore, we discuss different application scenarios for ISFP in educational, artistic, and business domains and conclude with future ideas and challenges to explore modern scientific ideas.


I INTRODUCTION
Let us think about a future city. What kind of transport will we use? Will we still drive by ourselves? What role would robots have in our future? And how would these things influence our society? If one tries to imagine those scenarios as well as think of consequences and implications, already Science Fiction Prototyping (SFP) is experienced. Johnson describes a science fiction prototype as "a short story, movie or comic based specifically on a science fact for the purpose of exploring the implications, effects and ramifications of that science or technology". [12] Typically, SFP involves creative processes such as writing, drawing, or story-telling and combines it with reflective aspects such as the exploration of implications and ramifications. While these activities may serve as an excellent starting point for sharing ideas in a narrative way from a single point of view, the aim of Interactive Science Fiction Prototyping (ISFP) is the integration of a flexible and collaborative component. ISFP uses SFP strategies and augments these with adaptive settings, interactive and collaborative features, and immersive experiences. 3D Virtual World (VW) environments provide a framework to support such social, collaborative activities within an adaptable and immersive environment.
Possible applications of ISFP in flexible virtual worlds include social exploration and studies of future scientific environments and setups, imaginative and creative uses in art and entertainment, as well as collaborative educational and training scenarios. Future cities and societal setups can be simulated in a flexible way, which can be explored and adapted. Researchers, learning communities or even interested users can then investigate, observe, and change the scenarios. For art and entertainment, the environment gives users not only the possibility to watch the scene (contrary to movies or books), but also to experience the setting together with peers. For education and training, the users' awareness of implications, problems, and issues of future scenarios can be raised by virtualizing the future environment.
The aim of this article is to discuss the single steps of Interactive Science Fiction Prototyping process in the context of three application scenarios which are integrated into a virtual world framework. This paper is an extended version of work published in [17]. We extend our previous work discussing different application scenarios for ISFP in the context of education, research, and entertainment.

II RELATED WORK AND BACKGROUND
"Science fiction writers foresee the inevitable, and although problems and catastrophes may be inevitable, solutions are not." (Isaac Asimov) [9] Science Fiction has always been fascinating to people. Imagining and predicting what scientific inventions of the future can bring did inspire not only the intellectual worlds, but also the media, such as literature, film, and now video games. [1,18] While books and films only allow passive consummation of the scientific innovations, users in video games can immerse themselves into future scenarios and be part of it. People also describe this feeling as "being in the game". [11] Games such as Valve's Half-Life series 1 , Halo 2 , or Deus Ex 3 allow players to use modern technologies and transport systems, live in alien environments, and experience different societal structures. [15] Many science fiction creations are considered as predecessors or inspirations for modern technologies. A famous example was Motorola's first cell phone. According to the inventor, Star Trek's communicator inspired the first design. [18] Science fiction media explores potential inventions and innovations of future worlds in a static and invariable way. In SFP, people are encouraged to invent fictional scenarios on their own and prospect implications, problems, and controversies. "We use these [science fiction] prototypes to envision technologies in the lives of real people all over the world. SF prototyping allows us to explore and iterate how technology can shape and be

PAPER INTERACTIVE SCIENCE FICTION PROTOTYPING IN VIRTUAL WORLDS: FUNDAMENTALS AND APPLICATIONS
shaped by the people who use it." [12] The output of SFP is science fiction media, like the one described above, to enable other people to experience (read, watch, or play) the invented ideas. According to Johnson, SF prototypes have the potential to not only demonstrate fictional futures, but also to change the future with valuable innovative ideas. "Creating SF prototypes, writing stories, making movies and drawing comics about the future are one of those things that people can do to actually change the future." [12] SFP allows users to exploit scenarios and technologies, not only to explore them, but also to deliver feedback back into the current development process. SFP can be used in different application domains such as education, innovation, or research. Examples can be found in [6,14,19].
To create SF prototypes Johnson describes five major steps: (1) world building, (2) scientific infliction point, (3) technological ramification on society, (4) human inflection point, and (5) reflection. First, the science or technology is picked and the world, locations, and personas are introduced. In the second step the science is placed into the world, which will affect the world and its inhabitants. The third step investigates the ramification and effects of the science on the personas and the society. How will the science affect the personas, their lives, and their world? In the fourth step, conflicts between the science and the personas and the environment are created. It is investigated how characters will react on the changes and how they will change their behavior and their lives due to the ramifications of the science on their lives. Problems and issues are investigated and modification on the science and technology are deliberated. The last step is used to reflect on what was learned during the previous steps. How has the environment, the people, or the system changed? What can be improved, what should be different? [12] Figure 1 illustrates the five steps of SFP.
Traditional SF prototypes such as short stories, comics, or movies often lack of interactivity, immersion, collaboration and adaptability.
More immersive forms of prototypes are threedimensional video games, which allow users not only to imagine the scenarios, but also to experience the scenarios and feel part of the environment. This kind of a prototype can also be described as Explorative Science Fiction Prototype. However the design and development of such games is demanding and usually requires many professional skills (e.g. programming and 3D-modelling). Also, traditional games are designed for experiencing scenarios and do not involve the possibilities for adaptation or collaboration. A more social and collaborative form of immersion can be experienced in 3D virtual worlds. A 3D virtual world is defined as "a synchronous, persistent network of people, represented as avatars, facilitated by networked computers" [1]. Like traditional digital games, VWs also immerse users in their environments. They also integrate different applications for communication and collaboration what intensifies their social and collaborative character. [4] Users can communicate through different channels, like VoIP, text-chat, or gestures. Different tools such as whiteboards or document sharing allow them to collaborate or brainstorm. VWs enable users to interact with the environment and the objects, as well as with other users. VW Figure 1. The five steps of science fiction prototyping defined by Johnson [10] toolkits such as Open Wonderland 4 , Open Simulator 5 , or RealXtend 6 enable users to create their own worlds, scenarios, and environments. Summarizing, the following advantages of using virtual worlds for SFP can be identified. VWs support different kinds of content which can be used to simulate futuristic scenarios (e.g. 3D-objects, avatars, images, sounds, videos, applications, or flexible physics). Having different applications and media at one common space allows users to create environments and scenarios together. Users experience a feeling of immersion and a feeling of presence as a result of their awareness. VWs facilitate discussions, collaboration, and are able to share different kinds of content. Their virtual nature enables collaboration of geographically dispersed groups. [5,7,16,17] Considering these advantages, VWs allow an advanced form of science fiction prototyping, which integrates flexible, collaborative, and interactive features. We introduce this form of SFP as Interactive Science Fiction Prototyping (ISFP).

III INTERACTIVE SCIENCE FICTION PROTOTYPING
ISFP is an agile approach to design and develop an explorative, interactive, and immersive science fiction prototype in a collaborative way. ISFP involves five major steps in the development process, which are revised iteratively. (1) First, users design and plan the environment, the scenarios, and the science or technology. In the design and planning phase the users can either think of interesting future technologies or can also start by picking an existing technology or science and exploit it by provoking ques-  [14] (2) As a result of the first step, the users have received a list of required objects, items, and models. In the second step, creative content creation, it is necessary to either create or retrieve those objects and models. (3) This content will be used in step three, flexible settings, to arrange, setup, and adapt the environment. (4) In the fourth phase, users explore the prototype. (5) The fifth phase is the reflection phase, in which users discuss the prototype and think about implications, issues, or improvements.
Since ISFP is designed as an iterative process, users should adapt and redesign the environment after the reflection phase to experience the prototype again with alignments for potential implications and issues (see Figure 2).

A. Interactive Science Fiction Prototyping in Open Wonderland
Summarizing the five steps, four major requirements for the integration into a 3D virtual world environment can be defined: creativity, flexibility, interactivity, and collaboration. Figure 3 illustrates the requirements and their connection to the applications like external 3D modeling tools and virtual world toolkits. The single elements will be discussed in the next sections.
Based on these requirements, ISFP can be integrated exemplarily into the virtual world toolkit Open Wonderland (OWL). Its design goals are described as collaboration, extensibility, and federation. [13] The standard setup already supports different communications methods (e.g. VoIP, text-chat, or gestures) and synchronous collaboration possibilities (see next sub-section). In the next section we will identify which of the four main requirements of ISFP are already covered by OWL's built-in features and which require further developments and extensions.

1) Collaboration
In the phases 'design and planning' and 'reflection' it is necessary that users are able to discuss, collaborate, and brainstorm. Figure 3 shows a brainstorming scenario with a whiteboard, uploaded documents and an ongoing discussion to design a first prototype together. In figure 4 the users explore and discuss aspects such as implications, issues, and further ideas together.
OWL supports different forms of synchronous collaboration. Users can share content such as images, videos, documents, or 3D models simply by dragging it from the desktop or the web browser into the application. Users can work together on whiteboards or document writers and share applications or remote desktops.
Creative aspects and possibilities are especially necessary in the design phase and during the content creation. Users should be able to visualize and virtualize their ideas in the 3D space. In the virtual environment this is done by adding objects and media to the worlds.
To add 3D objects users can either retrieve existing objects from databases, or create their own models with 3D modeling tools such as Google SketchUp 7 , Blender 8 , or Maya 9 . Compared to Blender and Maya, Google Sketch-Up enables intuitive 3D object creation also to users without 3D modeling knowledge.

2) Interactivity
To develop an explorative science fiction prototype, which allows users not only to observe, but to experience the scenarios as well, it is necessary to add interactivities with the environment, the technology, objects, and also other avatars. OWL supports scripting capabilities, which enables users to add interactive behavior to objects and the environment. Scripting is a simplified version of programming and allows users without advanced programming skills to add behavior to objects. More complex interactions, visualizations, and behaviors can be added to OWL with modules. Implementing modules, however, requires programming skills in Java.

3) Flexibility
The iterative structure of ISFP requires users to be able to change the prototype according to updated requirements. It is necessary that users can adapt the environments and its objects without great effort.
OWL offers possibilities for creative people to build and adapt their own worlds. Unfortunately it has very few build-in tools to support the creation of entire environments and scenarios in a more abstract level such as a 2D editor. In general, worlds have to be crafted manually on a low abstraction level, which complicates in particular the construction and adaptation of complex worlds. To overcome this, the integration of an abstract 2D editor into OWL is outlined in more detail in the next section.

B. Enhancing Flexibility in Open Wonderland
The traditional in-world tools to move or duplicate objects are adequate to change one item at a time or to edit single objects. Figure 5 demonstrates how to move, or copy objects with the in-worlds tools. Duplicating an object enables the user create one copy at a time. Furthermore it does not allow users to set a new position for the new object; it just appears near the original. Users then have to move it manually to the desired position. When they need to copy the item multiple times it can get fairly tedious. An editor tool, in comparison, may allow users to not only move more than one item at a time, or select multiple items at once for copy, but also offer the possibility to place them before starting the paste process. (see figure 6) The missing selection and change options in OWL for multiple objects also makes it difficult to do quick changes, like rotating a group of objects or setting the rights for them to a specific user group. In-world, the user has to modify every single object at a time and add the changes, whereas in an editor tool he could just select all the objects and make the change with one entry. Therefore using a tool would help the user to save a lot of time and tedium.
Another big problem is the lack of undo options, which makes changes permanent and hard to revert. Looking at other creative construction tools, like text editing software or 2D/3D graphic programs, undo and redo actions are necessities when it comes to usability. Of course implementing such actions on a platform with multiple users could be difficult, but would allow users to undo their mistakes easily.
To add the required flexibility to OWL we started developing an according 2D editing application. Following we describe technical details of the editor, which should not only be applicable to OWL, but with the integration of adapters also for other worlds, such as Open Simulator.   [17] The editor for OWL has three main parts, as seen in Figure 7. Because the editor is built with interoperability in mind, it has an adapter module, if the editor should be used with different virtual world platforms. The adapter forwards messages from the virtual world (e.g. OWL or Open Simulator) to the editor's data module and sends update messages back to the virtual world server. If the editor should be used with a different platform, the only part that needs to be replaced is the adapter. As its name suggests, it converts the data from the virtual world and from the editor, before sending out an update message.
The data package receives update messages from the adapter and stores a slim version of objects currently ex-

PAPER INTERACTIVE SCIENCE FICTION PROTOTYPING IN VIRTUAL WORLDS: FUNDAMENTALS AND APPLICATIONS
isting in the virtual world. It also stores environmental data such as the approximated world size, calculated from the data objects and their position. The data packages second purpose is to update the editor itself, when changes arise. The editor is the part where user interactions take place. When a user makes changes, the editor sends updates to the adapter, which converts the message and forwards it to the server. In order to stay in sync with the server, the editor only changes objects when the adapter receives update messages, not when the user does his changes. This is possible, due to the fact that OWL sends update messages anyway, whether the current user has triggered it or not.
When the user makes a change in the editor, the adapter is notified and sends an update message to the server. There the changes of the object are made and update message are sent to all users, including the user who made the change. The adapter receives the message and forwards it to the data part which forwards it to the editor. There, the change can now be seen.
Revising the single requirements, Open Wonderland is a good fit to create first scenarios for interactive and explorative science fiction prototypes. In the next section we will discuss different application scenarios.

IV APPLICATION SCENARIOS IN VIRTUAL WORLDS
Science fiction prototyping is used for different application domains. In the next subsection we will outline a step-by-step approach on fictive stories using ISFP to create an interactive and explorative prototype. Other scenarios are addressed briefly to explore the diversity of application domains. The first scenario introduces the single steps of ISFP introducing a fictional meeting of experts (e.g. in a company) to explore and create innovative products. The second prototype is created in the context of an educational program, the third for research activities.

C. Eric and OLEDs
Eric and OLED's is a SF prototype, which is developed by a group of international researchers to create and test new and innovative products. Following, the different process steps are outlined.
Design and Planning. In the first step, the international team meets in the virtual worlds and starts brainstorming about potential technologies. They search the web together for interesting new ideas, talk about future newspapers, future media, and finally find an article in a journal about the OLED technology. They start by raising questions such as: "If this technology were in an average home how would it actually work?" [14] . They start to build an innovative product around this technology, thinking about future scenarios which can make use of OLEDs and finally create the following scenario: It is a story about the daily life of a teenage boy who lives with his family in the year 2030. The newspapers are no longer on paper but on OLED displays, the homework is written on OLED paper and can be uploaded to the school system and even the mirrors of the bathroom are made of OLED.
After finalizing the first iteration of the story, the researchers make a list of the objects, environmental aspects, and activities which they need in order to virtualize the idea.

Creative Content Creation & Reuse.
In this phase, the team retrieves and creates the content which is necessary for the prototype. The team downloads simple models from Google 3D warehouse and adapts them according to their requirements in Google SketchUp (see Figure 8). In the figure an OLED display is integrated in the mirror of the bathroom.
Flexible Setting in VW. After the content is created it can be integrated into OWL. This happens in the flexible settings phase. Object behavior can be scripted, so the display is able to react to user input. The researchers add different behavior models to the mirror, to simulate an interactive bathroom experience.

Experience in VW
In the experience phase, the researchers immerse themselves into the world they have created and experience the scenario first alone, then as a group. In a role-playing scenario they can slip into the role of the school boy, to experience the interactive mirror with the eyes of a young male, who can revise his homework in the evening or learn new vocabulary while brushing his teeth. Figure 9 shows an avatar interacting with the bathroom mirror.

Reflection
After all objects are at their right position the reflection phase of ISFP can begin. The researchers exploit the scenario and talk together about implications, issues, improvements, and further ideas and application scenarios. One interesting idea a researcher had while exploring the bathroom scenario was to use of the mirror for reading and learning while taking a shower. Hence, they want to edit the position and size of the mirror, to make it also visible from other points in the bathroom. The flexibility of the environment makes it possible to easily adapt the mirror to the revised requirements. The researchers start over again with the first phase of the iterative process, to enhance the current idea.

D. Mira and the cars
Mira and her classmates got a new exercise from their teacher to think about future means of transportation. Together, they decide to develop the following scenario: This story is about a young journalist in the year 230 who has to drive daily several miles. By that time cars still cannot drive without a human driver, but they are able to connect to each other. So if there is a person driving the same direction as Mira, she can simply hook up electronically to the other car. This means that her car follows the leading car automatically.
In the content creation phase, two different cars and the highway from Google Warehouse were downloaded. In the next phase they were integrated into OWL. In figure  10 the group of cars can be seen. As they are all connected to each other they do not need to have a minimum safety distance. In the reflection phase of the ISFP process the students started discussing about advantages, implications, and in particular safety issues. One important point they experienced while trying out their scenario: What happens if the leading driver makes a human mistake? Will the following cars make the same mistakes or are there mechanisms that prevent this? Due to the iterative process, the students are able to revise their prototype and add additional safety aspects to explore the environment again.

E. Steve and his Farm
A group of environmental experts and life scientists convoke a meeting in the virtual world to discuss possibilities and implications of future farming scenarios. They start by describing the potential daily life of farmer in 2060.
Steve was born in 2000, is now 60 years old and is a farmer. As a young adult he started his education and finished a few years later. Now he is responsible for 2 farm buildings with 10 floors each. At this time farmers cultivate their crops in buildings, no longer on fields. Therefore his workstation is in front of a computer where he controls his tractors. The work tasks for a farmer changed to an IT specialist as he controls his machines remotely in order to produce crops. Such cultivating buildings are necessary, because the population density increases every year. More and more people need to be fed, unexploited land is very rare. The machines in these buildings are standardized to harvest as much crops as possible. Fertilizers were chemically improved, so it became unhealthy for farmers to be in direct contact with huge amounts of it.
The experts designed and arranged the environment, with suitable buildings and objects (see Figure 11) to explore the scenario on their own and to get a first impression of problems or issues.

DISCUSSION AND FUTURE WORK
This article investigates the potential of ISFP in virtual worlds, which demonstrates a collaborative, flexible and adaptable form of science fiction prototyping and supports the creation of immersive and explorative SF prototypes. Within the context of potential future scenarios, we showed the capabilities of ISFP in application domains Figure 10. Group of cars with leading car (grey) and followers (blue) Figure 11. Farm by the year 2060 such as education, innovation, and research. These prototypes have much educational and innovative potential. Users can express their ideas in a more visual way. The social and collaborative character enables users to develop and to explore the scenarios together. The flexible character and the iterative process motivate users to constantly change the scenario to enhance or to push the scenario to its extreme. These characteristics make ISFP particularly interesting for domains, with an innovative, research or educational character.
For future work we intend to enhance virtual worlds with the necessary features, such as advanced flexible adaptability and to integrate a set of predefined models into the environment to allow users to easily create scenarios and worlds without the necessity of designing and developing new models. After that, we can start evaluating the scenario with different user groups in different application domains.