Third Dimension

Virtual reality as we know it today has some of its earliest roots in the 19th century. In 1838 scientist and inventor Sir Charles Wheatstone theorized that we perceive the world in depth because we have two eyes, set slightly apart and seeing from two different points of view. He surmised that the parallax difference between what our eyes see is interpreted into depth, and proved this by designing a device to allow the viewing of two images drawn from different perspectives, one for each eye. His device, which he called the stereoscope (from the Greek meaning “to see solid”) proved his theory to be correct. Wheatstone’s stereoscope, a cumbersome device using mirrors to combine the two views, was the first mechanical means of viewing a reproduced three-dimensional image. With the invention of photography in the decade that followed, we finally had a method of capturing multiple still images from real life and creating stereograms: immersive images for stereoscopic viewing.

Figure 1.2 Wheatstone’s stereoscope

Another scientist, Sir David Brewster, refined the design of the stereoscope into a handheld device. Brewster’s “lenticular stereoscope” placed optical lenses onto a small box to allow the viewing of pairs of backlit glass photographic slides and front-lit photo prints. His viewer was introduced to the public at the Great Exhibition of 1851 in London, England, where it became very popular, helped greatly by an endorsement by Queen Victoria herself. An industry quickly developed to produce stereoscopes and 3D photographic images for viewing.

Figure 1.3 Brewster’s stereoscope

In the United States, author and physician Oliver Wendell Holmes, Sr. saw a need to produce a simpler, less expensive stereoscope for the masses. In 1861 he designed a stereoscope that could be manufactured easily, and specifically chose not to file a patent in order to encourage their mass production and use.

Figure 1.4 Holmes’ stereoscope

Throughout the latter half of the 19th century and until the 1920s, stereoscopes became a ubiquitous form of home entertainment. Companies such as the London Stereoscopic Company, the Keystone View Company, and Underwood & Underwood sent photographers around the globe to capture stereoscopic images, and millions of images were produced and sold. Stereo cards depicted all manner of subjects, from travel to exotic locations, to chronicles of news and current events, to entertaining comedy and dramatic narratives. Stereo viewers were also used in education, finding their way into the classroom to supplement lessons on geography, history, and science. Much of the appeal of these stereoscopic photographs was the ability of a 3D image to immerse the viewer and virtually transport them to faraway places that they would never be able to visit in person.

Figure 1.5 Stereoscopes in use in a classroom, 1908

Immersive Presentations

Another popular form of immersive entertainment in the 18th and 19th centuries was the panorama. The term panorama, from the Greek meaning “to see all,” was first used by artist Robert Barker in the 1790s to describe his patented large-scale cylindrical paintings which were viewed from within and surrounded the viewer. Barker constructed a rotunda building in London in 1793 specifically for the exhibition of his panoramic paintings. The popularity of Barker’s panorama led to competition and many others being constructed throughout the 1800s. Historically, a panorama (sometimes referred to as a cyclorama) consisted of a large 360° painting, often including a three-dimensional faux terrain and foreground sculptural elements to enhance the illusion of depth and simulated reality, and building architecture designed to surround the spectator in the virtual environment.

Figure 1.6 A panorama

Figure 1.7 Illustrated London News. “Grand Panorama of the Great Exhibition of All Nations,” 1851

Figure 1.8 Cinéorama, invented by Raoul Grimoin-Sanson for the Paris Exposition of 1900

The grand panoramas of the period created the illusion for the audience of standing in the middle of a landscape and scene, while the depicted events were happening. These paintings in the round served both to entertain and to educate, often depicting grandiose locations or great historical events. Panoramas proved to be very successful venues, with over 100 documented locations in Europe and North America. Some notable installations include the Gettysburg and Atlanta Cycloramas, painted in 1883 and 1885, which depicted scenes from those American Civil War battles, and the Racławice Panorama in Poland, a massive painting 49 feet high and 374 feet long, painted by artists Jan Styka, Wojciech Kossakover, and a team of assistants over the course of nine months from 1893 to 1894 to commemorate the 100th anniversary of the Polish Battle of Racławice. During their heyday in the Victorian period, the panoramas saw hundreds of thousands of guests each year.

The post-Industrial Revolution brought about a new age of technological advances. The birth of cinema in the 1890s brought to the public a new form of media – the moving image. The apocryphal story of early filmmakers Auguste and Louis Lumière’s 1896 film “L’arrivée d’un train à La Ciotat” sending audiences screaming out of the theater, believing the on-screen train was going to hit them, may be an often-told myth, but it still demonstrates the sense of reality that early motion picture attendees reported experiencing.

Throughout the 20th century, cinematic developments such as color, sound, widescreen, and 3D added to the content creation toolset, as inventors sought new methods and technologies, first analog then digital, to build realistic immersive experiences. One early attempt was the Cinéorama, devised by Raoul Grimoin-Sanson for the Paris Exposition of 1900, which combined a panorama rotunda with cinema projection to simulate a ride in a hot-air balloon over Paris. Footage was first filmed with ten cameras mounted in a real hot-air balloon, and then presented using ten synchronized projectors, projecting onto screens arranged in a full 360° circle around the viewing platform. The platform itself was large enough that 200 spectators were able to experience the exhibit at the same time.

The 1927 French silent film “Napoleon” by director Abel Gance also used a multi-camera widescreen process. Gance wanted to bring a heightened impact to the climatic final battle scene and devised a special widescreen format, given the name Polyvision, which used three stacked cameras to shoot a panoramic view. Exhibition required three synchronized projectors to show the footage as a triptych on three horizontally placed screens to ultimately display an image that was four times wider than it was high. While the impact of such a wide-screen picture was dramatic, it was technically very difficult to display properly, as projector synchronization was complicated, and there was no practical method to hide the seams between the three projected frames.

In 1939, at the World’s Fair in New York, filmmaker and special effects expert Fred Waller introduced an immersive theater for the Petroleum Industry exhibit, called Vitarama, which used an array of 11 projectors to project a giant image onto a dome-like spherical screen, designed by architect Ralph Walker.

In 1941, Waller took elements of his Vitarama immersive theater and invented a multi-projection simulator for the military. Called the Waller Flexible Gunnery Trainer, its purpose was to train airplane gunners under realistic conditions. They would learn to estimate quickly and accurately the range of a target, to track it, and to estimate the correct point of aim using non-computing sights. To create the footage for the machine, five cameras were mounted in the gun turret position of a bomber, and filmed during flight. Combat situations were depicted by having “enemy” planes fly past the cameras. The Waller Flexible Gunnery Trainer itself used a special spherical screen designed by Ralph Walker, and five projectors to create a large tiled image that surrounded four gunner trainees. It featured a 150° horizontal field of view and a 75° vertical. The trainees were seated at mock gun turrets and engaged in simulated combat with moving targets. In addition to the visual feedback on the spherical screen, the trainees also received audio via headphones, and vibration feedback through the dummy guns. A mechanical system kept score of their hits and misses and provided a final tally. Waller’s system was used by the US military during World War II, and the first installation was in Honolulu, Hawaii, following the Japanese attack on Pearl Harbor.

Figure 1.9 Waller Flexible Gunnery Trainer

Following World War II, Waller devised another multi-camera/multi-projector system for entertainment, and named it Cinerama (for cinema-panorama). Similar to the projection used for Gance’s “Napoleon,” Cinerama used a triptych of three projectors, projecting a seamed three-panel image onto a giant curved screen. A special system was designed to shoot for Cinerama, with three cameras mounted together at 48° angles to each other. The interlocked cameras each photographed one-third of the full wide scene, which filled a 146° arc. The Cinerama theaters used three synchronized projectors spread out across three projection booths, each projecting one-third of the full image onto a curved screen made of forward-facing vertical strips. A mechanical system of vibrating combs in the projectors was meant to limit the light output in the overlapping areas of the three panels, effectively blending the three images into one. In practice, this method did not prove to be overly effective, as parallax differences at the seams still made them apparent. Cinerama was also one of the first theater experiences to utilize stereo surround sound, with a total of seven tracks – five behind the screen and two in the back of the auditorium.

Figure 1.10 Waller’s Cinerama

The First Head-Mounted Displays (HMDs)

The View-Master was conceived by organ maker and stereo photographer William Gruber as a 20th-century redesign of the Victorian stereoscope. The View-Master provided a sequence of seven stereoscopic slide views on a backlit circular reel that could be manually advanced by the viewer. The View-Master was put into mass-production by the Sawyers Company of Portland, Oregon, which also hired photographers to create content. As with the stereocards that preceded it, the stereo reels consisted of all manner of subject matter, from travel and scenic wonder, to popular culture and entertainment. Thanks in part to its small form factor and ease of use, it proved to be a very popular consumer product. The View-Master also found a market in education, being used extensively in medical training where it could provide realistic views of human anatomy and disease. View-Masters have remained in production continuously since their launch, and while the size and shape of the viewers have changed over the years, the design of the reels themselves has changed very little, allowing nearly 80 years of content to continue to be viewed even in modern viewers.

Figure 1.11 The View-Master

Figure 1.12

In 1936, science-fiction writer and inventor Hugo Gernsback (after whom the Hugo Awards for sci-fi literature were named) conceived of what he called Tele-Eyeglasses. He described the future device as a pair of eyeglasses utilizing two miniature cathode ray tubes (CRTs) to provide television images to each eye. His device was intended to be connected by wire to a television set, which would receive and feed the signals to the wearer. Gernsback created mock-ups of his invention, but never produced a working prototype.

Inventor Morton Heilig may have been the first person actually to use the term “virtual reality,” which he used to describe “a simulation of reality.” Heilig designed a head-mounted CRT-based viewing system, which he patented in 1960, and he went a step further than Gernsback, actually building working prototypes of his invention. His device, which he named the Telesphere Mask, offered stereoscopic viewing of moving images by positioning two miniature televisions in front of the viewer’s eyes. Heilig used a special lens arrangement to bend the light from the TV tubes to be visible in the viewer’s peripheral vision, and effectively provided a 140° angle of view both horizontally and vertically. The Telesphere also included built-in headphones to deliver binaural sound, and it had a pair of air discharge nozzles which, according to Heilig’s patent, could be used to “convey to the head of the spectator, air currents of varying velocities, temperatures and odors.”

Heilig also developed and built an immersive experiential device he called the Sensorama. Rather than a wearable viewer like the Telesphere, his Sensorama was essentially an immersive theater for an individual viewer who would be seated inside the machine. Again, Heilig relied on optics to fill the viewer’s field of view with a moving 3D image, multiple speakers to provide surround sound, and air valves to simulate wind and deliver scents. The Sensorama also added the ability to vibrate the seat for a tactile element. Heilig designed his own 3D camera to capture content for the Sensorama, and produced five demonstration films, four of which were ride films depicting the point of view from a bicycle, motorcycle, go-kart, and helicopter. The fifth demo film was an interaction with a human subject, a belly dancer, and featured a fragrant perfume smell as she appeared to approach the viewer. Heilig’s intention was that his devices could be used for training and education, similar to how Waller’s flight simulators had been utilized, but Heilig was never able to make his inventions commercially viable.

Figure 1.13 Morton Heilig’s invention

Figure 1.14 Morton Heilig’s Sensorama

Ivan Sutherland is often referred to as “the father of computer graphics” thanks to his development of the first graphical user interface, Sketchpad, programmed while earning his PhD at the Massachusetts Institute of Technology in 1962. By 1968, Sutherland was an associate professor of electrical engineering at Harvard University, and he and his student Bob Sproull constructed what is considered the first true VR head-mounted display. Nicknamed the “Sword of Damocles” because the weight of the helmet required that it be suspended from a ceiling-mounted arm, Sutherland’s system was able to track the wearer’s head to determine where they were looking and then render a real-time stereoscopic image as a computer-generated vector wireframe. The HMD was somewhat see-through, so the user could see some of the room, making this a very early augmented reality system as well. Sutherland went on to teach at the University of Utah, where he partnered with another professor, David Evans, to found the computer graphics company Evans and Sutherland, which specialized in creating high-end computer-generated imagery for, among other purposes, flight simulators and planetariums. The company is still one of the largest distributors of media for full-dome immersive presentations.

Figure 1.15 The “Sword of Damocles”

Figure 1.16 Nintendo’s Virtual Boy

The 1980s saw the rise of the video game, and several manufacturers ventured into early digital attempts at interactive immersive games. The Vectrex system in 1981 offered a stereoscopic imager that used a head-mounted spinning mechanical shutter to display stereoscopic graphics on its vector display screen. Sega released a stereoscopic option for its gaming system which used liquid-crystal active shutter glasses. In 1993 Sega announced, but never actually released, an HMD-based VR system for its home video game console. Nintendo also released the ill-fated Virtual Boy game system, an attempt at a portable standalone stereoscopic VR game which ultimately failed due to its limited graphics and gameplay capabilities.

1991 saw a first attempt at a VR arcade with the Virtuality system and its game “Dactyl Nightmare.” Some 350 of these coin-operated game systems were produced and placed in shopping malls. Customers would pay their fee and step up onto a large round platform, surrounded by a magnetic ring. The player would don the VR visor and gun, and was required to wear a heavy belt at their waist, containing the wires that tethered the devices to the computer controller (either an Amiga 300 or 486 computer). The graphics were very primitive polygons, but the tracking worked rather well for its time. Sega’s VR-1 motion simulator arcade attraction was introduced in 1994 in SegaWorld amusement arcades. It had the ability to track head movement and also featured stereoscopic 3D polygon graphics. In 2011, Nintendo released the 3DS handheld game, which included a built-in stereoscopic camera, positional tracking, and a glasses-free autostereoscopic screen for AR gaming.

Theme Parks

Theme parks and entertainment venues have also made attempts at bringing VR to the public. The Disney VR lab developed a prototype of Aladdin’s magic carpet ride in 1994 and installed it at EPCOT in Orlando, Florida. They refined their HMD and in 1998 opened DisneyQuest, a theme park specifically for VR which featured the Aladdin VR.

Circarama (later renamed Circle-Vision 360°) was a 360° panoramic theater, introduced at the Disneyland theme park in 1955, which used nine projectors and nine huge screens arranged in a circle around the audience.

Disney theme parks have also used stereoscopic films and moving theater seats to create immersive movies, such as their “Honey I Shrunk the Audience,” which simulated the entire theater changing in scale. The “Soarin’ over California” ride (now remade as “Soarin’ over the World”) at Disney’s California Adventure Park and EPCOT features large-format projection onto a giant dome screen to simulate flight for riders who are suspended in front of it. That attraction also releases various scents into the audience to enhance the realism of the environments. Universal Studios theme parks also used stereoscopic film and immersive practices with many of their attractions. Their “T2 3-D: Battle across Time,” directed by James Cameron, combined live performance, environmental effects, and 3D film. The show culminated in a climactic scene projected in 3D across three giant surround screens. Universal’s “Back to the Future” ride, later replaced by “The Simpsons” ride, is another ride that suspends the audience in front of an 85-foot IMAX dome screen. Recently Six Flags amusement parks, in a partnership with Samsung and Oculus, have even introduced VR rollercoaster rides to their parks, where the riders wear Samsung Gear VR HMDs to experience an interactive simulated adventure while actually riding in a moving rollercoaster car.

Figure 1.17 Circle-Vision 360°

Computer Power and Tracking

Figure 1.18 A CAVE system

By the 1990s, workstation computers were becoming powerful enough to render computer graphics in real time, and researchers found new ways to track users’ movements to create interactive immersive experiences, utilizing both HMDs and immersive rooms. In 1977, Daniel Sandin, Tom DeFanti, and Rich Sayre designed the Sayre Glove, the first wired glove which allowed hand gesture control of data by measuring finger flexion. The CAVE (Cave Automatic Virtual Environment) is an immersive room environment invented in 1991 by Sandin at the Electronic Visualization Laboratory at the University of Illinois, and developed by Sandin, David Pape, and Carolina Cruz. The CAVE is a cubic room using rear-projection material for walls. Real-time computer graphics were projected on the walls, and sometimes also on the floor and ceiling. The user would wear 3D glasses to view the stereoscopic projections, and the glasses were tracked within the space so that the projected images could be manipulated to match the viewer’s perspective as one moved around. This gave the illusion of actually standing in a real environment, with objects appearing to be floating in the virtual room. The tracking information was also used to create three-dimensional audio through multiple surround speakers. A control wand allowed the user to virtually interact with objects and to navigate through larger environments. A more recent iteration, the CAVE2 developed in 2012, uses liquid crystal displays (LCDs) instead of projection.

Figure 1.19

One of the most advanced immersive VR rooms is the StarCAVE, installed at the University of California, San Diego, under the supervision of Tom DeFanti. It utilizes 17 individual screens to create a pentagon-shaped five-walled space, each wall being three screens high, and includes two screens of floor projection. Some 34 individual projectors provide high-resolution left and right stereoscopic video for the screens. The upper and lower wall screens tilt toward the viewer at 15° angles, and the entryway wall rolls into place to completely enclose the user for a true 360° immersive VR experience.

Further development of data gloves was carried out by Thomas Zimmerman at the company VPL Research. VPL was founded by Jaron Lanier, who popularized the use of the term virtual reality in the late 1980s. VPL’s data glove added the ability to track hand position in addition to finger flex. This led to the development of the DataSuit, a full-body tracking system for VR. VPL became the first company to sell a commercial head-tracking HMD, and the DataGlove was licensed to Mattel in 1989 and released as the PowerGlove accessory for the Nintendo Entertainment System home video game console.

Figure 1.20 Fakespace’s “BOOM” system

In 1988, Mark Bolas, Ian McDowall, and Eric Lorimer founded the company Fakespace to engineer VR hardware and software for government and scientific use. Fakespace’s innovations included their own version of a data glove, the Pinch Glove, and a VR imaging and tracking system called the BOOM (Binocular Omni-Orientation Monitor), which placed a high-resolution computer monitor inside a stereoscope on the end of a boom arm. The user would look through the wide-angle optics and view stereoscopic real-time computer-generated imagery (CGI), and the arm sensors tracked the stereoscope’s position through six axes of motion.

The “VR for Everyone” Revolution

A “do-it-yourself” handheld VR viewer, called the FOV2GO, leveraged the computing and graphics power of available smartphones, and was developed in 2012 at the University of Southern California (USC) by a team of students and their instructor, multimedia innovator and educator Perry Hoberman, in conjunction with USC’s Institute for Creative Technologies. The FOV2GO led directly to Google’s open-source design for their Cardboard VR viewer, an inexpensive, paper-based VR headset that is compatible with practically any Android or iOS phone.

As computing power has continued to increase and computing devices have become miniaturized, the feasibility of a self-contained handheld device with high-resolution displays and a full complement of positional and rotational tracking sensors has become a reality. A new wave of public and corporate interest has spawned a new period of heavy development in virtual reality. Companies like Sony and HTC have released their own proprietary virtual reality systems. Oculus Rift, a consumer head-mounted display VR device based on inexpensive cellular phone parts and originally developed in 2012 through a crowdfunding campaign on Kickstarter by its developer, Palmer Luckey, was purchased by Facebook for billions of dollars. It seems that new immersive devices and experiences, from personal VR HMDs to dedicated VR theaters, gaming centers, and theme parks, are being announced every day. None of these new inventions and innovations would be happening if it were not for

Figure 1.21

Figure 1.22

their predecessors. In many ways things have come full circle: Google’s Cardboard as an inexpensive mass-produced VR viewer, is almost identical in purpose and design to its analog ancestor, Holmes’s patent-free stereoscope from over 150 years ago.