Mass Effect: A Chemical Engineering Application of Virtual Reality Simulator Technology

Damian Schofield
Associate Professor and Director of Human-Computer Interaction
School of Computer Science
State University of New York at Oswego
Oswego, NY 13126 USA
schofield@cs.oswego.edu

Abstract Advanced three-dimensional (3D) virtual reality (VR) technology similar to that used by the film and computer game industries can allow educational developers to rapidly create realistic online virtual environments. This technology has been used to generate a range of online VR-based learning environments across a broad spectrum of industries and educational application areas. This idea is not new; flight simulators have been used for decades to train pilots for both commercial and military aviation. There are a number of lessons that can be learned from the industries that have successfully utilized virtual training and learning systems. Generic rules of thumb regarding the specification, development, application, and operation of these learning environments can be garnered from these industrial training systems and examined in an educational context. In this paper, an online VR-based system developed by the author, ViRILE (Virtual Reality Interactive Learning Environment), is introduced. This software is designed for use by undergraduate chemical engineers and simulates the configuration and operation of a polymerization plant. During the implementation of this and other visual learning environments, a number of complex operational problems were encountered that have required a number of innovative solutions and management procedures to be developed. The implementation of this and other similar systems is also discussed in this paper, and the lessons learned are extrapolated into general pedagogical guidelines to be considered for the development of VR-based online educational learning resources. Keywords: virtual reality, simulation, chemical engineering education, learning technology, training, guidelines

Introduction

Inevitably, the future will be digital. The continuing digital revolution has had an enormous impact on the way learning is undertaken and information is disseminated. A wide range of online digital media will end up being used, to varying degrees in multifaceted educational applications throughout the world. However, in many places around the globe, technology can be slow to become fully accepted; this is often due to financial and budgetary constraints. It is fair to say that, in general, utilization of digital learning media in many educational institutions can lag behind the technological development (Lester, Schofield, & Chapman, 2006; Roussos et al., 1997; Schofield, Noond, & Burton, 2002).

Advanced three-dimensional (3D) computer graphics and virtual environment technology, similar to that used by the film and computer games industry, has been used to generate interactive learning environments that will allow learners to undertake a range of simulated experiences. Other "virtual" teaching and training applications from a range of industries (flight, surgery, and driving simulators to name a few) have proved the value of this technology, however these VR-based learning environments are currently not widely utilized in the education sector. Early attempts at "virtual" education simulators, particularly those which tried to apply 3D computer graphics based technology, were often constrained by lack of realism detail in their graphical interfaces and crude level of simulation (Denby & Schofield, 1999; Roussos et al., 1997; Wilson, 1996). However, it has been also noted that even given these limitations, these virtual environments had the potential to allow users to experience situations which would not readily exist within the real world, for example, to see "into" a chemical reaction or to cause a major catastrophe through their actions (Psotka, 1995; Roussos et al., 1997; Schofield, 2007).

There are many examples of VR-based learning environments in the field of engineering education, including online science education laboratories (Shin, 2002) and virtual engineering laboratories (Bresciani, Morsi, Tucker, Siprut, Stewart, & Duncan, 2010). There are also specific examples in the fields of mining engineering (Stothard et al., 2008), construction engineering (Messner, Yerrapathruni, Baratta, & Whisker, 2003), and manufacturing (Fernandes, Raja, & Eyre, 2003; Mujber, Szecsi, & Hashmi, 2004). Of particular interest is the Pharmatopia project run by the Faculty of Pharmacy and Pharmaceutical Sciences at Monash University in Australia, in which students undertake simulated tasks in a virtual tablet manufacturing plant within the online VR-based environment of Second Life (see Monash University, 2011).

Technology

At this point, it is perhaps appropriate to define and describe the technologies under discussion in this paper. Virtual reality (VR) involves interactive, real-time, 3D graphical environments that respond to user input and action, such as moving around in the virtual world or operating virtual equipment. This paper in intended to provide information for VR system designers (particularly Internet and web-based clients) but will not deal with immersive environment technology (such as the CAVE, head-mounted display, or desk). An important aspect of such a VR system is its underlying processes, simulations, behavior, and reactions, and the way in which a user can interact with objects within the virtual world. A VR user could, for example, sit in a virtual vehicle and drive it, with obvious potential for a constructionist learning experience (Loyens & Gijbels, 2008; Woo & Reeves, 2007). Popular cultural examples of this technology and interface metaphor include modern VR-based computer games such as the Halo and Grand Theft Auto series. These games have morphed into the massively multiplayer online games (MMOGs) enjoyed by millions of players every day, the most popular and pervasive of these currently being World of Warcraft.

Novel applications of this technology have emerged due to the rapid developments in personal computer technology, especially for desktop VR. In particular, the home computer games market has encouraged the development of software tools together with specialist graphics accelerator boards and peripheral products. Whilst much of the development is aimed at the home and leisure industry, there are many applications that have been developed for a range of commercial sectors. This has consequently also had an effect on the education sector, and educators have seen examples of this technology being introduced into VR-based learning environments around the world over the past few years (Gee, 2003; Gibson, Aldrich, & Prensky, 2007; Zyda, 2005). Pedagogically, these types of interactive VR display systems can offer major advantages over other visualization media, because of the interactive (active rather than passive) nature of the learning experience they create (Balamuralithara & Woods, 2009; Gutiérrez, Vexo, & Thalmann, 2008; Piccoli, Ahmad, & Ives, 2001).

Second Life is a prime example of a multi-user 3D virtual environment that has been used to provide an online VR-based learning experience by a number of educational institutions. The aim is to provide an engaging interactive sensory experience while increasing social interactions (student-teacher and student-student) and hence deepen the learning among users. Second Life supports virtual learning within a 3D learning space and a unique human-computer interaction (HCI) style with the potential to promote a sense of presence, which engages remote users in the learning activities. Initial published feedback about the experience of using this environment has been fairly positive, frequently quoting increased learning and engagement among the participants (Baker et al., 2009; De Lucia, Francese, Passero, & Tortora, 2008; Jarmon, Traphagan, Mayrath, & Trivedi, 2009).

It is important to realize that the use of such computer-generated, 3D interactive learning media in the education sector is only the current manifestation of a long history of visual training and education systems (Gutiérrez et al., 2008; Psotka, 1995; Roussos et al., 1999; Schofield, 2007; Zyda, 2005). However VR-based learning environments are unparalleled in their capabilities for presenting complex information. The use of such enabling visualization technology can affect the manner in which data is assimilated and correlated by the viewer; in many instances, it can potentially help make the information more relevant and easier to understand (Machado, Moraes, Souza, Souza, & Cunha, 2009; Piccoli et al., 2001).

Virtual Reality Interactive Learning Environment (ViRILE)

A problem identified by academic staff among a chemical engineering cohort of students at the University of Nottingham in the United Kingdom was the students' lack of awareness about "real" process equipment (Schofield, 2004, 2005). Question-and-answer sessions indicated that the undergraduates were often not only unfamiliar with full-scale industrial plant, but were also unable to identify some of the main components even though they had been studied (theoretically) in the classroom. To address this issue an online VR-based environment was developed to replicate a "real" industrial process. This allowed the students to experiment with large-scale equipment to which they would not normally have access. It was decided to build a large-scale virtual simulation where students are able to design and build a particular processing plant and then operate the major equipment over a period of time. Key features of this project included the design-orientated nature of the task (constructivist) facilitated by the interactive character (active rather than passive engagement) of the technology (Lester et al., 2006; Schofield & Lester, 2010).

To solve this problem, a continuous polymerization plant was modeled, consisting of a reactor section and three distillation columns (Figure 1). It is important not to underestimate the work involved in developing commercial quality, VR-based training simulators to a professional standard. To a generation weaned on animated movies and computer games, the level of expectation of the student cohort is usually high. Previous VR-based learning environments developed have provided experience of the quality of the software required to gain a level of acceptance among the learners (Psotka, 1995; Roussos et al., 1997, Schofield, 2007; Schofield & Lester, 2010).

Figure 1. The Virtual Processing Plant (ViRILE)

In the ViRILE system, large amounts of process plant simulation data was generated using steady state chemical flow sheet simulation software. A complex, real-time, mathematical model was programmed (using C++) to reference the data generated. The final learning environment contains over a billion discrete configurable states, allowing the students unlimited scope for experimentation and configuration. This also allows the educator to set individual tasks within the learning environment for particular students. A full economics and costing model has also been integrated into the ViRILE chemical plant simulation, giving learners an insight into the constraints facing engineers in the real world. Figure 2 shows a student calculating the rates of propane production and associated costs of this process (Schofield et al., 2005; Schofield & Lester, 2010). It should be noted that even though this VR-based system represents a processing plant, the learning framework embedded within the system is primarily based on the retention and application of declarative rather than procedural knowledge.

Figure 2. Interacting with the Virtual Processing Plant (ViRILE)

VR worlds (such as the ViRILE system) can provide a giant laboratory for educators to experiment and play and explore new possibilities and alternative configurations. There are some instructional designers who can extrapolate from their experiences with other technologies and immediately seize on using virtual worlds for what they are best at (co-presence, simulation, collaboration, prototyping) and leave the quizzes and notes and document repositories on their course management system, which delivers those types of content better than virtual worlds currently can. However, it is unrealistic to expect that all academics will have the technical skill required to develop professional, photorealistic virtual environments. Nevertheless, there are a number of compromises that can be made, and the use of different levels of reality in these environments is discussed later in this paper. It is imperative that academics and developers continue to push the boundaries and not get locked into habits or practices in the virtual world that mirror those of the real world – but to instantly dismiss every replica of a traditional learning space in a virtual world without understanding the context in which it was created, the purpose and intent with which it was to be used, is not only unproductive, it may even be harmful (Collins, 2012).
The ViRILE system allows a student to easily understand complex material and interact with a detailed processing simulation. The pedagogical benefits of this interactive environment when it is used as an online educational tool have been well researched and investigated (Calongne, 2008; Dickey, 2003; Eschenbrenner, Nah, & Siau, 2008; Jarmon et al., 2009). Students build their own solutions to problems that are individually assigned, and hence gain a deeper understanding of the complex chemical processes involved. However, there are a number of issues and questions that appear when such a simulation model is examined in further detail (Schofield, 2007; Schofield & Lester, 2010).

Issues Arising

Analyses of VR-based learning environments show that they can be advantageous in many situations, providing they are used appropriately (Gibson et al., 2007; Jarmon et al., 2009; Piccoli et al., 2001; Roussos et al., 1999; Schofield & Lester, 2010; Tromp & Schofield, 2004). However, potential difficulties can occur from the application of this technology; when these learning environments are examined in further detail, a number of issues and questions can arise. The consequences of these problems cannot be underestimated (Grunwald & Corsbie-Massay, 2006; Schofield & Mason, 2010).

Viewpoint

One issue is how to correlate the viewpoint of a user in a "virtual" environment with the view from their physical position in the "real-world" environment. Consider, for example, driving in a computer game (such as Gran Turismo) where the user's real-world view is represented by a camera view in the virtual world. When driving in the real world the user will move their head around within the car to improve their vision and ability to better see the surrounding environment. This movement is often not replicated within the virtual environment (Weinberg & Harsham, 2009).

Compare the "physical-world" view of a worker on a real chemical plant with the field of view of a camera in the ViRILE online learning simulator. A number of researchers have discussed the problems of correlating the experience between a user's view of a virtual world and its real world counterpart (Anquetil & Jeannerod, 2007; Kankaanranta & Neittaanmäki, 2009; Nakamura et al., 2010). However, there are also a number of benefits to the use of virtual camera viewpoints in these virtual worlds. Unlike the real world, in the ViRILE simulator (as in many other virtual worlds) it is possible to rapidly switch between views of the world from multiple angles. For example, it is possible to examine and inspect different items of chemical plant within the ViRILE system rapidly and efficiently (Schofield & Lester, 2010).

Popular computer game titles provide good examples of distinct viewing configurations through their playing styles. Referring to the computer games previously mentioned in this paper, the Halo series belongs to a genre known as the first-person shooter (FPS); distinguished by a first person perspective (egocentric) that renders the game world from the visual perspective of the player character. The Grand Theft Auto series is a third-person shooter (TPS); this is a genre of video game in which an avatar of the player character is seen at a distance from a number of different possible perspective angles (exocentric). Driving provides an interesting example of comparing FPS and TPS viewpoints, in the real world humans drive using a first-person perspective, whereas in many computer games vehicles are easier to control using a third-person perspective (Nakamura et al., 2010; Weinberg & Harsham, 2009). Most collaborative online virtual environments (such as Second Life) and MMOGs (such as World of Warcraft) tend to rely on third-person perspectives. The ViRILE online virtual learning environment used a first-person perspective, the aim was to allow the learner to feel immersed and engaged within the environment as if they were controlling the process equipment, rather than performing the actions vicariously through an avatar (Schofield et al., 2004, 2005).

Camera angles and viewpoints are used in film to position the viewer so that they can understand the relationships between the characters. These are very important for shaping meaning in film as well as in other visual media (Nelmes, 1999). In any 3D virtual learning environment (as in any computer game), the choice of the viewing perspective may have a significant effect on the way an image is interpreted by the viewer. Changing the viewing perspective can potentially alter which "character" in a learning scenario that a viewer identifies with, or aligns him/herself with (Bryce & Rutter, 2002; Scharrer & Leone, 2008).

Spatial Location

There is also an issue regarding the correlation of the locations of learners when they are positioned in a VR-based environment, such as ViRILE, in comparison to actual positions in the real world. It is a reasonable assumption to make that most people would be better able to correlate their actual spatial location from a 3D "virtual" simulation, than they might be able to on a two-dimensional (2D) plan or map. It is interesting to note that research has indeed shown that a significant proportion of the general public has problems relating and correlating 2D (e.g., maps and plans) and 3D (e.g., real and virtual) spatial information (Schnabel & Kvan, 2003). In practice, this means that some learners may find it easier to specify their physical position by referring to a virtual environment (relating "physical" 3D space to "virtual" 3D space) rather than to a 2D plan or map of the scene.

One of the main pedagogical advantages of the use of an online VR-based simulation, such as ViRILE, for learning over a passive computer-generated animation is the ability to dynamically control the virtual camera movement within the environment (Ware & Osborne, 1990; Scharrer & Leone, 2008). This permits the learner to "interactively" adjust the view of any selected digital object. For example, a user could move a camera around the chemical plant to allow them to examine individual items of process equipment. It should be noted that how humans position themselves and correlate spatial information between 3D views of virtual worlds and the physical world are not fully understood (Arthur, Hancock, & Crysler, 1997; Montello, Waller, Hegarty, & Richardson, 2004). However, consistent with dual-coding theory, spatial ability allows high-spatial-ability learners to devote more cognitive resources to building referential connections to the presented material, whereas low-spatial-ability learners must devote more cognitive resources to building representation connections between visually presented material and its visual representation (Mayer & Sims, 1994; Paivio, 1990).

Realism

Items relating to specific learning objectives are usually the most important objects built and represented within the virtual world. However, additional environment features surrounding the primary interactive elements may be included within the online VR-based learning simulation to provide context. For example, the ViRILE learning simulation not only shows the location of the main items of chemical plant, but also the position of nearby buildings and other peripheral chemical plant features. Any of these items may be placed and animated within a chronology of events or react to user interaction. As this technology develops (driven by the computer games industry), the realism of virtual environments continues to improve. As computer-processing power increases and software tools develop, it is natural to assume that it will be possible to achieve computer game style levels of photorealism within the computer-generated environments used in online learning environments. Two recent, popular films demonstrate two distinct animation and representation styles. The first, Shrek, relies on a cartoon-like, abstract approach to present its narrative. The second, Beowulf, relies on a more realistic representational form. A number of researchers have noted an interesting observable fact relating to the realism in such animated imagery (as shown in Figure 3), where many viewers become "unnerved" by images of humans which are close to, but not quite real. This phenomenon (experienced by a number of viewers of the Beowulf movie) has become known as the "uncanny valley" because of the sharp dip seen in a graph of familiarity against the perception of reality (MacDorman, 2006).

Figure 3. Demonstrating varying levels of realism in the representation of a ViRILE virtual plant worker

Objects in a VR-based learning environment can be modeled with varying degrees of accuracy to explain and visualise the certainty, believability and veracity of the information related to that object. For example, in the ViRILE system, the colour of particular items of plant varied based on the temperatures and pressures of liquids flowing through the equipment at any particular moment during the life of the simulation. However, this mixing of visual metaphors and modes may be potentially disorientating to some viewers (Fiedler, 2003; Schofield, 2007).

In a virtual learning context, many online systems currently in use rely on fairly abstract representations (Kankaanranta & Neittaanmäki, 2009; Schofield, 2007). However, as technology progresses, the development of increasingly photorealistic VR-based learning environments becomes ever more likely. Combining abstract data representations in photorealistic environments and expecting a viewer to draw additional information from a number of abstract representations in the virtual environment may overload the viewer, create an unatural experience and potentially add to their confusion rather than increasing their comprehension of the information that is presented (Fiedler, 2003).

Also the mixing of levels of detail and different modes of abstraction may be potentially disorientating to some viewers (Gibson et al., 2007; Gutiérrez et al., 2008). A number of researchers have reported on the way learners can be misled by the use of confusing visual metaphors and abstract representations in learning environments (Bystrom, Barfield, & Hendrix, 1999; Pederson & Sokoler, 1997).

Media Mode

It is rare that one form of media will be sufficient to fully explain every facet of a complex process or complicated learning objective to a viewer. A number of educators in the past have seen 3D graphics technology as a universal solution, and it can be "over-applied" or "misapplied" in many learning applications. It is important to choose an appropriate representation mode (photographs, text, video, graphics etc.) for the material that needs to be presented (Woo & Reeves, 2007; Zyda, 2005). Additional data may be included and displayed within any virtual environment. For example, within the ViRILE system, location-based statistical or analytical data relating to the chemical process can be displayed through interaction with objects in the VR-based learning environment. Experimental results are displayed and calculations are presented in a visual format, all linked to form 3D virtual chemical plant objects (Schofield & Lester, 2010).
The linking of learning material (often using relevant Internet hyperlinks) to spatially contextualized hotspots in a virtual environment has the potential to provide an effective mechanism to help the learner to better understand spatial relationships between individual elements of the material being studied. Consider, for example, learning anatomy using an online VR-based model, such as the one developed by Google Labs. Google Body, which has been renamed Zygote Body (http://www.zygotebody.com/) following the shutdown of Google Labs, is a web application that renders manipulable 3D anatomical models of the human body. For many learners the most crucial thing to be gained from an online educational tool such as this is to visually understand the spatial location and integration of the body's complex organs and systems – the ability to link to further information on each body organ or system using context-sensitive hotspots is an invaluable addition to the learning experience. Such a multimodal approach can be very effective, and different media may also be used as a device to help to retain the attention of the learner and thereby increase engagement and hence, understanding (Mastaglio & Callahan, 1995; Ravet & Layte, 1998). The success of this approach correlates well with work that has been done modeling cognitive differences in individual students and their relationships and learning capacity related to different forms of media (Aspinwall & Taylor, 1992; Austin, 2009).

Audio

The integration of physical-world audio within learning environments has been successfully implemented in many educational software packages. Spatialized audio-rendering systems capable of rendering a number of dynamically moving sound sources in multi-speaker, immersive, VR-based environments have been available in computer games for many years. This technology can be easily integrated into online VR-based learning environments using off-the-shelf audio hardware and software. Based on simplified physics-based models, developers can achieve a good trade off between audio quality, spatial precision, and performance in any virtual environment (Cruz-Neira, Sandin, DeFanti, Kenyon, & Hart, 1992; Frécon & Stenius, 1998).

The ViRILE system extensively used audio cues for the learners in a variety of different contexts: from audible sounds whenever process selections were made to spatially contextualised audio allowing the learner to hear when equipment was operating. Feedback from use of the ViRILE system suggested that the audio was a significant contributing factor to the level of engagement with the online simulation (Lester et al., 2006). Other research has reported that adding audio to a computer-generated visual can have a significant effect on the level of engagement of the learner, and hence may potentially affect their understanding and interpretation of the material being viewed (Hendrix & Barfield, 1996; Tsingos, Gallo, & Drettakis, 2004). This effect on the level of engagement by the addition of audio can be explained using a dual-processing model of working memory, which has implications for the design of VR-based learning systems (Mayer & Anderson, 1992; Moreno & Mayer, 2002).

Resolution

One difficulty is to correlate the resolution of a virtual scene with that subjectively perceived by the viewer in the physical world. In this instance, resolution not only refers to screen image dimensions (specifically, the pixel count), but also to the field of view and the device on which the simulation is viewed (Brooks, 1999). A user interacting with an online VR-based learning environment on a mobile device such as a mobile telephone or an iPod has a very different experience to one who watches it on a computer monitor. In addition, a viewer interacting with a VR simulation on a 2D computer monitor or screen may not have the same experience (depth of field, motion parallax, peripheral vision, etc.) as a viewer watching a real event. The introduction of 3D screen technology may go some way toward minimizing these differences, but there is very little research on this in the context of learning simulations (Blais, Brutzman, Horner, & Nicklaus, 2001; Bronack, Sanders, Cheney, Riedl, Tashner, & Matzen, 2008; Ko, Chang, Chen, & Hua, 2011).

The ViRILE system was primarily designed for use on a computer monitor, although tests were also undertaken using projected images on large-scale screens (Figure 4). The way the technology is to be used (in what educational context) and for what purpose, as well as the medium and technology through which it will be displayed and operated, need to be considered at the design stage of any project (Balamuralithara & Woods, 2009; Kanade, Rander, & Narayanan, 1997; Tromp & Schofield, 2004).

Figure 4. Extensive testing of the ViRILE online simulation was undertaken on a range of devices and screens

Accuracy

The creation of any VR environment begins with data collection; accuracy is crucial, because this data serves as the foundation for the visual forms and behaviors of the objects to be created. The technology used for collecting data and measurements varies depending on the type of environment to be created. The ViRILE system was based on both manufacturer's equipment specifications and measurements and photographs taken on site at real chemical plants. This data provided a reliable numerical data set for the creation of the geometry that is the foundation of any credible computer model or reconstruction of a scene.

If a VR environment is created to a sufficient level of accuracy, then it may potentially be used to test hypotheses. For example, in the ViRILE system, the simulation data were accurate enough to allow the learner to verify the orientation or placement of specific equipment within the plant and determine operating tolerances for that specific item of equipment (Schofield et al., 2005).

Simulation

It should never be forgotten that a VR simulation, is by its very definition a "simulation" of reality. In the context of an online VR-based learning environment, it is necessary to understand the nature of the simulation and the veracity of the representation, that is, how close is it to the original data from which it was derived (Kankaanranta & Neittaanmäki, 2009).

The behavior of the chemical plant within the ViRILE online learning simulation was based on the same equations that would be used by a chemical engineer in calculating or predicting how the plant would behave under any particular set of conditions. However, questions that arise include whether the simulation applies the equations and formulae in the same way, whether the simulation works to the same level of accuracy, whether the simulation make the same assumptions as the chemical engineer, and whether the visual representation of the chemical plant in the virtual environment provides a realistic and relevant portrayal of the simulation data (Lester et al., 2006).

Narrative

The narrative structure of story is the way in which the story is told. Traditional books, plays, and movies have a linear narrative structure. Although the contents of the story may be nonlinear (flashbacks, foreshadowing, etc.), the telling of the story is usually linear, from page-to-page or frame-to-frame in a fixed order. Similarly, most traditional educational experiences follow a linear path. An online VR-based learning simulation, in contrast, can have a non-linear narrative structure; there are multiple potential paths that the educational experience can take, depending on the actions of the learner (Heise, 1997).

Nonlinear narratives extend the benefits of linear narratives. The learner is given options as to what he or she chooses to do. Historically, these options were presented as predetermined choices, such as choosing open one door or another. Yet, nonlinear narrative is much more than merely offering choices. It also includes online simulations of physical reality, such as ViRILE, that dynamically generate learning objectives and alter potential process outcomes based on the actions of the learner (Kinder, 2008; Schofield et al., 2005).

However, giving the learner the ability to move through time and along a chronology of events in the virtual environment may be potentially disorientating to many viewers. Most members of the general public are used to linear narratives, and may struggle to follow multiple narrative threads when faced with such a non-linear approach (Craven et al., 2001).

Lighting

Consideration needs to be given as to how it is possible to correlate the lighting in the VR world with that available in the real environment. It has to be determined whether an approximation is acceptable in the simulation. Arguably, this might not be crucial in some online VR-based learning environments, so long as all objects are sufficiently illuminated so as to be clearly visible (Walter, Alppay, Lafortune, Fernandez, & Greenberg, 1997).

Recommendations

By their very nature, any recommendations and guidelines formulated are likely to be broadly defined and generic. Many of the recommendations offered below are little more than general suggestions that users of the technology be aware of these issues when involved in developing the types of online VR-based learning systems described in this paper.

Field of View

It is recommended that designers of VR-based learning environments study film-making techniques, for two reasons:

• The first is to aim to achieve the same effects as a filmmaker; perhaps getting the viewer to emotively identify with a particular character in a learning environment to enhance the power of the message.
• Alternatively, an animator or modeler may wish to eliminate these effects and to remove the emotive content to provide an objective, understandable view of a data set, with no distracting emotive attachment.

An awareness of the ways the emotions and engagement of the viewer can be manipulated (for example through the use of egocentric and exocentric viewpoints) is essential.

Interaction and Resolution

Careful thought needs to be given to the enabling technology; it is necessary to know he device to be used and how the user will interact with, and connect to, any VR-based simulation created. For example, the best mechanism to teach a specific learning objective could be to deliver a spatially contextualized data visualisation to a user's personal digital assistant (PDA) or mobile telephone screen as they traverse a real environment (such as a chemical plant). Alternatively, a complex data set may be best viewed as a shared, multi-user, collaborative experience on the learners own (remote) computer screen.

Modes of Representation

Developers need to be aware that VR-based virtual systems are not a panacea solution for all learning requirements. They are not ideal for representing every learning objective. Any learning simulation developer should adopt a holistic, multimodal visualisation approach using appropriate technology, delivering relevant media as and when required. This may involve linking to, and integrating with, pertinent text, diagrams, photography, video, computer graphics, and so on; whatever is deemed the most suitable for the particular type of material and learning content.

Effect of the Media

Most online VR-based learning environments have the capacity to allow the user to interact with a range of digital media (often using spatially, context sensitive hotspots – which usually consist of clickable hyperlinks connecting objects in the virtual world to other web based media such as text, diagrams, photographs and video). It is necessary to be not only aware of the effect of the particular form of media used, but also to have an appreciation of the context in which it will be experienced by the user. The pedagogical effect of transitions between the forms of media should be considered. For example, switching between a virtual, rendered image of a chemical plant and a real chemical plant photograph may cause confusion in the learner as they attempt to correlate information between the different media forms and levels of detail.

Audio

The integration of sound into a VR system is often overlooked or added as an afterthought. Very few virtual developers are also qualified as, or competent as, sound engineers. However, effective audio soundtracks can add new dimensions to the learner's media experience.

Abstraction

Careful use of visual metaphors is essential. Thought needs to be given to each abstract data representation in the virtual learning environment and how that will be perceived by the potential audience. Experience and literature from disciplines such as psychology, cultural and critical theory, visual media, art history, education, and such like can inform how abstract (and realist) representations are interpreted by the viewer. These representations in turn influence the information what the viewer remembers and understands from the visual simulation that has been presented to him/her (Spiesel, 2006).

Navigation and Interface

Many VR-based systems have complicated interaction and navigation systems (often based on computer game-style controls), which may add an extra layer of complexity to the data users are trying to comprehend rather than augmenting their understanding. Careful thought should be given to the options available to the user. If control is to be passed to the learners, then it may be better to restrict their movement and control in the online virtual environment (e.g., between set points) rather than allow them to potentially become "lost" in the data or environment. The ability to professionally manipulate, operate, and utilize the technology needed to navigate through complex 3D data sets is a skill that many of the general population do not possess.

Behavior

It is important that the developers of any VR-based system have an understanding of the processes and events being simulated (whether this is a chemical plant, vehicle movement, or human anatomy). The developers must be aware of the veracity and realism of the simulation, that is, the accuracy of the model. Also, it is important that if decisions are to be made based on the simulation, it is necessary that information is made available that explains how the simulation works (at a range of levels) to the learner.

Narrative

In an online VR-based system, users may be given the ability to take control of the narrative, altering the chronological presentation of information and choosing which information they see at which time. This can easily become confusing to learners, particularly those used to linear narratives in other media (e.g., novels and films). Developers should provide a guide to the interactions within their online learning environments and be aware of how the users are able to interact with the chronology of the data and any possible interpretations that may result.

Lighting

It is very rare that light meters would be installed in a real world location, measuring the intensity of the illumination at a particular moment, thus allowing the designer of a virtual world to replicate exactly the luminosity in the virtual environment. In many cases, it is possible to argue that this is not an issue, because the lighting may not be crucial to the viewing of the information. However, considering the amount of money and time spent in a major motion picture on lighting and the emotional effect this can have on the viewer, one can see how the effect of the VR system lighting might be significant, and perhaps more consideration should be given to this aspect of development.

Testing

It is axiomatic that an online VR system should be tested before it is released. However, it is common knowledge that a number of broadly distributed learning environments have received limited user testing before their release (Balamuralithara & Woods, 2009; Gibson et al., 2007; Grunwald & Corsbie-Massay, 2006). The use of this type of technology requires perceptive construction, because a number of issues only come to light when the technology makes contact with the users. For example, the brightness may be too low, or the colors of the image on the user's personal device may be different to how they appeared on a the large, high-resolution monitor used by the developer, or the resolution of the display may make some objects difficult to see. As with any technology, it is important to be aware that it has the potential to fail, and it is only through field testing that most of these potential failure mechanisms will be identified.

Recommendation Summary

While each of the individual issues discussed earlier in this paper are important, an appreciation of the holistic nature of the user's learning experience is what will give rise to real pedagogical benefits. Each of the individual technological aspects may help the user to understand a concept or achieve a particular learning outcome. However, as with any other online learning experience, each component in an online VR-based learning experience must be carefully designed, developed, and evaluated to ensure it contributes to a better learning experience for the user.

Conclusion

VR technology advances rapidly and the public, who regularly see photorealistic computer graphics on television, expect to see their TV experience duplicated in the workplace and specifically in the modern-day online training and educational tools they use. Society now functions using a variety of online mechanisms. Our culture is dominated with images whose value may be simultaneously over determined and indeterminate, whose layers of significance can only be teased apart with difficulty. Different academic disciplines (including critical theory, psychology, education, media studies, art history, semiotics, etc.) help explain how audiences interpret visual imagery. Learners expect professional visual representations illustrating complex information, polished digital media displays demonstrating the location of spatially distributed data and dynamic animated graphics showing event chronologies. However, the analysis of educational imagery and its interpretation by learners is often overlooked (Spiesel, 2006).

Around the world, a number of educational organizations are already beginning to utilize "slick" computer-generated visuals to replace oral/text-based learning materials and depend on their audience adapting a visual learning style. Whether one likes it or not, in the future, 3D, interactive, multi-user, collaborative virtual environments (based on computer games technology) are going to be increasingly used to generate online educational experiences around the world. It is imperative that researchers and practitioners start to examine the implications of this technology, evaluate its potential advantages and disadvantages and assess its impact on learners. However, it should be noted that there is potentially significant development costs in the use of this technology, which may involve the assistance of additional staff or technology vendors and consultants.

This paper has, hopefully, been fairly positive about the future and the benefits that can arise through the introduction of this technology into the education sector. However, there are a number of issues and concerns that arise through the use of online VR-based learning environments. These are not reasons in themselves for abandoning the use of this technology, but rather aspects that need to be investigated further and safeguards and guidelines put in place to avoid any possible misuse of the technology.

References

Anquetil, T., & Jeannerod, M. (2007). Simulated actions in the first and in the third person perspectives share common representations. Brain Research, 1130(1), 125-129. doi:10.1016/j.brainres.2006.10.091

Arthur, E. J., Hancock, P. A., & Crysler, S. T. (1997). The perception of spatial layout in real and virtual worlds. Ergonomics, 40(1), 69-77. doi:10.1080/001401397188387

Aspinwall, L. G., & Taylor, S. E. (1992). Modeling cognitive adaptation: A longitudinal investigation of the impact of individual differences and coping on college adjustment and performance. Journal of Personality and Social Psychology, 63(6), 989-1003. doi:10.1037/0022-3514.63.6.989

Austin, K. A. (2009). Multimedia learning: Cognitive individual differences and display design techniques predict transfer learning with multimedia learning modules. Computers & Education, 53(4), 1339-1354. doi:10.1016/j.compedu.2009.06.017

Baker, S. C., Wentz, R. K., & Woods, M. M. (2009). Using virtual worlds in education: Second Life as an educational tool. Teaching of Psychology, 36(1), 59-64. doi:10.1080/00986280802529079

Balamuralithara, B., & Woods, P. C. (2009). Virtual laboratories in engineering education: The simulation lab and remote lab. Computer Applications in Engineering Education, 17(1), 108-118. doi:10.1002/cae.20186

Blais, C., Brutzman, D., Horner, D., & Nicklaus, S. (2001, November). Web-based 3D technology for scenario authoring and visualization: The SAVAGE project. Paper presented at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), Orlando, FL. Retrieved from http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA422086

Bresciani, M. J., Morsi, K., Tucker, M., Siprut, M., Stewart, K., & Duncan, A. (2010). Designing and implementing an assessment plan for a virtual engineering lab. Eludamos: Journal for Computer Game Culture, 4(2), 277-285. Retrieved from http://www.eludamos.org/index.php/eludamos/article/view/vol4no2-10/189

Brooks, F. P. (1999). What's real about virtual reality? IEEE Computer Graphics and Applications, 19(6), 16-27. doi:10.1109/38.799723

Bronack, S., Sanders, R., Cheney, A., Riedl, R., Tashner, J., & Matzen, N. (2008). Presence pedagogy: Teaching and learning in a 3D virtual immersive world. International Journal of Teaching and Learning in Higher Education, 20(1), 59-69. Retrieved from http://www.isetl.org/ijtlhe/pdf/IJTLHE453.pdf

Bryce, J., & Rutter, J. (2002). Spectacle of the deathmatch: Character and narrative in first-person shooters. In G. King & T. Krzywinska (Eds.), ScreenPlay: Cinema/videogames/interfaces (pp. 66-80). London: Wallflower.

Bystrom, K. E., Barfield, W., & Hendrix, C. A. (1999). A conceptual model of the sense of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 8(2), 241-244. doi:10.1162/105474699566107

Calongne, C. M. (2008). Educational frontiers: Learning in a virtual world. EDUCAUSE Review, 43(5), 36-48. Retrieved from http://net.educause.edu/ir/library/pdf/ERM0852.pdf

Collins, C. (2012, May). Should virtual environments be realistic? Hypergrid Business Magazine. Retrieved from http://www.hypergridbusiness.com/2012/05/should-virtual-environments-be-realistic/

Craven, M., Taylor, I., Drozd, A., Purbrick, J., Greenhalgh, C., Benford, S., ... Hoch, M. (2001). Exploiting interactivity, influence, space and time to explore non-linear drama in virtual worlds. In M. Beaudouin-Lafon & R. J. K. Jacob (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01) (pp. 30-37). New York: Association for Computing Machinery. doi:10.1145/365024.365032

Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., & Hart, J. C. (1992). The CAVE: Audio visual experience automatic virtual environment. Communications of the ACM, 35(6), 64-72. doi:10.1145/129888.129892

De Lucia, A., Francese, R., Passero, I., & Tortora, G. (2008). Development and evaluation of a virtual campus on Second Life: The case of SecondDMI. Computers & Education, 52(1), 220-233. doi:10.1016/j.compedu.2008.08.001

Denby, B., & Schofield, D. (1999, October). The role of virtual reality in the safety training of mine personnel. Mining Engineering, 59-64.

Dickey, M. D. (2003). Teaching in 3D: Pedagogical affordances and constraints of 3D virtual worlds for synchronous distance learning. Distance Education, 24(1), 105-121. doi:10.1080/01587910303047

Eschenbrenner, B., Nah, F. F.-H., & Siau, K. (2008). 3-D virtual worlds in education: Applications, benefits, issues, and opportunities. Journal of Database Management, 19(4), 91-110. doi:10.4018/jdm.2008100106

Fernandes, K. J., Raja, V. H., & Eyre, J. (2003). Immersive learning system for manufacturing industries. Computers in Industry, 51(1), 31-40. doi:10.1016/S0166-3615(03)00027-7

Fiedler, B. S. (2003). Are your eyes deceiving you? The evidential crisis regarding the admissibility of computer-generated evidence. New York Law School Law Review, 48(1-2), 295-321. Retrieved from http://www.nyls.edu/user_files/1/3/4/17/49/v48n1-2p295-322.pdf

Frécon, E., & Stenius, M. (1998). DIVE: A scaleable network architecture for distributed virtual environments. Distributed Systems Engineering, 5(3), 91-100. doi:10.1088/0967-1846/5/3/002

Gee, J. P. (2003). What video games have to teach us about learning and literacy. Computers in Entertainment, 1(1), 20. doi:10.1145/950566.950595

Gibson, D., Aldrich, C., & Prensky, M. (Eds.). (2007). Games and simulations in online learning: Research and development frameworks. Hershey, PA: Information Science Reference. doi:10.4018/978-1-59904-304-3

Grunwald, T., & Corsbie-Massay, C. (2006). Guidelines for cognitively efficient multimedia learning tools: Educational strategies, cognitive load, and interface design. Academic Medicine, 81(3), 213-223. doi:10.1097/00001888-200603000-00003

Gutiérrez, M. A., Vexo, F., & Thalmann, D. (2008). Stepping into virtual reality. London: Springer. doi:10.1007/978-1-84800-117-6

Heise, U. K. (1997). Chronoschisms: Time, narrative, and postmodernism. Cambridge, UK: Cambridge University Press.

Hendrix, C. M., & Barfield, W. (1996). The sense of presence within auditory virtual environments. Presence: Teleoperators and Virtual Environments, 5(3), 290-301.

Jarmon, L., Traphagan, T., Mayrath, M., & Trivedi, A. (2009). Virtual world teaching, experiential learning, and assessment: An interdisciplinary communication course in Second Life. Computers & Education, 53(1), 169-182. doi:10.1016/j.compedu.2009.01.010

Kanade, T., Rander, P., & Narayanan, P. J. (1997). Virtualized reality: Constructing virtual worlds from scenes. IEEE Multimedia, 4(1), 34-47. doi:10.1109/93.580394

Kankaanranta, M. H., & Neittaanmäki, P. (Eds.). (2009). Design and use of serious games. Dordrecht, Netherlands: Springer. doi:10.1007/978-1-4020-9496-5

Kinder, M. (2008). Hot spots, avatars, and narrative fields forever – Buñuel's legacy for new digital media and interactive database narrative. Film Quarterly, 55(44), 2-15. Retrieved from doi:10.1525/fq.2002.55.4.2

Ko, C.-H., Chang, T.-C., Chen, Y.-H., & Hua, L.-H. (2011).  The application of augmented reality to design education. In M. Chang, W.-Y. Hwang, M.-P. Chen, & W. Mueller (Eds.), Edutainment technologies. Educational games and virtual reality/augmented reality applications: Proceedings of the 6th International Conference on E-learning and Games, Edutainment 2011. Vol. 6872 Lecture Notes in Computer Science (pp. 20-24). Heidelberg, Germany: Springer. doi:10.1007/978-3-642-23456-9

Lester, E., Schofield, D., & Chapman, P. (2006). The interaction of engineering "types": A study of group dynamics and its relationship to self and peer assessment during computer-based exercises. Engineering Education, 1(1), 39-49. Retrieved from http://www.heacademy.ac.uk/assets/documents/subjects/engineering/1-1-lester.pdf

Loyens, S. M. M., & Gijbels, D. (2008). Understanding the effects of constructivist learning environments: Introducing a multi-directional approach. Instructional Science, 36(5-6), 351-357. doi:10.1007/s11251-008-9059-4

MacDorman, K. F. (2006). Subjective ratings of robot video clips for human likeness, familiarity, and eeriness: An exploration of the uncanny valley. In Toward social mechanisms of android science: Proceedings of the ICCS/CogSci-2006 Long Symposium (pp. 26-29). Retrieved from http://www.androidscience.com/proceedings2006/8MacDorman2006Subjective.pdf

Machado, L. S., Moraes, R. M., Souza, D. F. L., Souza, L. C., & Cunha, I. L. L. (2009). A framework for development of virtual reality-based training simulators. In J. D. Westwood, S. W. Westwood, R. S. Haluck, H. M. Hoffman, G. T. Mogel, R. Phillips, R. A. Robb, K. G. Vosburgh (Eds.), Medicine meets virtual reality 17 – NextMed: Design for/the well being (pp. 174-176). Amsterdam: IOS Press.

Mastaglio, T. W., & Callahan, R. A. (1995). Large-scale complex virtual environment for team training. Computer, 28(7), 49-56. doi:10.1109/2.391041

Mayer, R. E., & Anderson, R. B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84(4), 444-452. doi:10.1037/0022-0663.84.4.444

Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86(3), 389-401. doi:10.1037/0022-0663.86.3.389

Messner, J. I., Yerrapathruni, S. C. M., Baratta, A. J., & Whisker, V. E. (2003). Using virtual reality to improve construction engineering education. In Proceedings of the 2003 American Society for Engineering Education (ASEE) Annual Conference & Exposition. Washington, DC: American Society for Engineering Education. Retrieved from http://search.asee.org/search/click?query=title=file://localhost/E:/search/conference/27/AC 2003Paper602.pdf

Monash University. (2011). Pharmatopia – a pharmacy education "oasis". Retrieved from http://www.pharm.monash.edu.au/education/epharm/pharmatopia.html

Montello, D. R., Waller, D., Hegarty, M., & Richardson, A. E. (2004). Spatial memory of real environments, virtual environments, and maps. In G. L. Allen (Ed.), Human spatial memory: Remembering where (pp. 251-285). Mahwah, NJ: Lawrence Erlbaum.

Moreno, R., & Mayer, R. E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94(1), 156-163. doi:10.1037/0022-0663.94.1.156

Mujber, T. S., Szecsi, T., & Hashmi, M. S. J. (2004). Virtual reality applications in manufacturing process simulation. Journal of Materials Processing Technology, 155-156, 1834-1838. doi:10.1016/j.jmatprotec.2004.04.401

Nakamura, R., Lago, L. L. M., Carneiro, A. B., Cunha, A. J. C., Ortega, F. J. M., Bernardes, J. L., Jr., & Tori, R. (2010). 3PI experiment: Immersion in third-person view. In S. N. Spencer (Ed.), Proceedings of the Fifth ACM SIGGRAPH Symposium on Video Games (Sandbox '10) (pp. 43-48). New York: Association for Computing Machinery. doi:10.1145/1836135.1836142

Nelmes, J. (Ed.). (1999). An introduction to film studies (2nd ed.). London: Routledge.

Paivio, A. (1990). Mental representations: A dual coding approach. Oxford, UK: Oxford University Press. doi:10.1093/acprof:oso/9780195066661.001.0001

Pedersen, E. R., & Sokoler, T. (1997). AROMA: Abstract representation of presence supporting mutual awareness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '97) (pp. 51-58). New York: Association for Computing Machinery. doi:10.1145/258549.258584

Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quarterly, 25(4), 401-426. doi:10.2307/3250989

Psotka, J. (1995). Immersive training systems: Virtual reality and education and training. Instructional Science, 23(5-6), 405-431. doi:10.1007/BF00896880

Ravet, S., & Layte, M. (1998). Technology-based training: A comprehensive guide to choosing, implementing, managing, and developing new technologies in training. Houston, TX: Gulf Publishing Company.

Roussos, M., Johnson, A. E., Leigh, J., Vasilakis, C. A., Barnes, C. R., & Moher, T. G. (1997). NICE: Combining constructionism, narrative, and collaboration in a virtual learning environment. Computer Graphics, 31(3), 62-63. doi:10.1145/262171.262264

Roussos, M., Johnson, A., Moher, T., Leigh, J., Vasilakis, C. A., & Barnes, C. (1999). Learning and building together in an immersive virtual world. Presence: Teleoperators and Virtual Environments, 8(3), 247-263. doi:10.1162/105474699566215

Scharrer, E., & Leone, R. (2008). First-person shooters and the third-person effect. Human Communication Research, 34(2), 210-233. doi:10.1111/j.1468-2958.2008.00319.x

Schnabel, M. A., & Kvan, T. (2003). Spatial understanding in immersive environments. International Journal of Architectural Computing, 1(4), 435-448. doi:10.1260/147807703773633455

Schofield, D. (2007). Animating and interacting with graphical evidence: Bringing courtrooms to life with virtual reconstructions. In E. Banissi, M. Sarfraz, & N. Dejdumrong (Eds.), Proceedings of the IEEE Conference on Computer Graphics, Imaging and Visualisation (CGIV '07) (pp. 321-328). Los Alamitos, CA: IEEE Computer Society. doi:10.1109/CGIV.2007.18

Schofield, D., & Lester, E. (2010). Learning in a virtual environment: Should there be guidelines? In J. Herrington & B. Hunter (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2010 (pp. 3848-3860). Chesapeake, VA: Association for the Advancement of Computing in Education.

Schofield, D., Lester, E., & Wilson, J. A. (2004). ViRILE: Virtual reality interactive laboratory experiments. In A. Halstead & P. Lister (Eds.), Proceedings of the International Conference on Innovation, Good Practice and Research in Engineering Education (EE '04) (pp. 225-231). Wolverhampton, UK: University of Wolverhampton.

Schofield, D., Lester, E., & Wilson, J. A. (2005). Virtual reality based, interactive e-learning for chemical engineers. In Proceedings of the Seventh World Congress of Chemical Engineering [CD-ROM]. Rugby, UK: Institution of Chemical Engineers.

Schofield, D., & Mason, S. (2010). Using graphical technology to present evidence. In S. Mason (Ed.), Electronic evidence (2nd ed., pp. 135-168). New York, NY: LexisNexis Butterworths.

Schofield, D., Noond, J., & Burton, A. (2002). Reconstructing accidents: Simulating accidents using virtual reality. In S. Bandopadhyay (Ed.), Proceedings of the 30th International Symposium on the Application of Computers and Operations Research in the Minerals Industry (APCOM '02) (pp. 559-568). Littleton, CO: The Society for Mining, Metallurgy, and Exploration.

Shin, Y.-S. (2002). Virtual reality simulations in web-based science education. Computer Applications in Engineering Education, 10(1), 18-25. doi:10.1002/cae.10014

Spiesel, C. (2006). Reflections on reading: Words and pictures and law. International Journal of Law in Context, 2(3), 305-320. doi:10.1017/S1744552306003077

Stothard, P., Squelch, A., van Wyk, E., Schofield, D., Fowle, K., Caris, C., … Schmidt, M. (2008). Taxonomy of interactive computer-based visualisation systems and content for the mining industry: Part One. In S. Saydam (Ed.), Proceedings of the First International Future Mining Conference and Exhibition 2008 (pp. 201-210). Melbourne, Australia: The Australasian Institute of Mining and Metallurgy.

Tromp, J., & Schofield, D. (2004). Practical experiences of building virtual reality systems. In Proceedings of the Workshop on Designing and Evaluating Virtual Reality Systems. Nottingham, UK: The University of Nottingham.

Tsingos, N., Gallo, E., & Drettakis, G. (2004). Perceptual audio rendering of complex virtual environments. In J. Marks (Ed.), Proceedings of the 31st International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '04) (pp. 249-258). New York: Association for Computing Machinery. doi:10.1145/1186562.1015710

Walter, B., Alppay, G., Lafortune, E., Fernandez, S., & Greenberg, D. P. (1997). Fitting virtual lights for non-diffuse walkthroughs. In T. Whitted (Ed.), Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '97) (pp. 45-48). New York: Association for Computing Machinery. doi:10.1145/258734.258766

Ware, C., & Osborne, S. (1990). Exploration and virtual camera control in virtual three dimensional environments. In M. Zyda (Ed.), Proceedings of the 1990 Symposium on Interactive 3D Graphics (I3D '90) (pp. 175-183). New York: Association for Computing Machinery. doi:10.1145/91385.91442

Weinberg, G., & Harsham, B. (2009). Developing a low-cost driving simulator for the evaluation of in-vehicle technologies. In (Eds.), Proceedings of the First International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '09) (pp. 51-54). New York: Association for Computing Machinery. doi:10.1145/1620509.1620519

Wilson, B. G. (1996). Constructivist learning environments: Case studies in instructional design. Englewood Cliffs, NJ: Educational Technology Publications.

Woo, Y., & Reeves, T. C. (2007). Meaningful interaction in web-based learning: A social constructivist interpretation. The Internet and Higher Education, 10(1), 15-25. doi:10.1016/j.iheduc.2006.10.005

Zyda, M. (2005). From visual simulation to virtual reality to games. Computer, 38(9), 25-32. doi:10.1109/MC.2005.297