179
12
AGenericMultiviewRendering
EngineArchitecture
M. Adil Yalçın
Tolga Çapın
Department of Computer Engineering, Bilkent University
12.1Introduction
Conventional monitors render a single image, which is generally observed by the
two eyes simultaneously. Yet, the eyes observe the world from slightly different
positions and form different images. This separation between the eyes provides
an important depth cue in the real world. Multiview rendering aims to exploit this
fundamental feature of our vision system for enhanced 3D rendering.
Technologies that allow us to send different images to the two eyes have
been around for years, but it is only now that they can reach the consumer level
with higher usability [Bowman et al. 2004]. The existing technologies vary
among the different types of 3D displays, and they include shutter glasses, binoc-
ular head-mounted displays, and the more recent and popular autostereoscopic
displays that require no special glasses.
Recent techniques for multiview rendering differ in terms of visual character-
istics, fidelity, and hardware requirements. Notably, multiview rendering engines
should be able to support more than two simultaneous views, following recent
3D display technologies that can mix a higher number of simultaneous views
than traditional stereo view [Dodgson 2005].
Currently, many available multiview applications are configured for the ste-
reo-view case, and the routines that manage stereo rendering are generally im-
plemented as low-level features targeted toward specific APIs and displays. We
present a higher-level multiview rendering engine architecture that is generic,
robust, and easily configurable for various 3D display platforms, as illustrated in
180 12.AGenericMultiviewRenderingEngineArchitecture
Figure 12.1. A torus rendered for different basic 3Ddisplay platforms.
Figure 12.1. The architecture simplifies the use of multiview components of a
rendering engine and it solves the problems of rendering separate views and
merging them for the target display. It also includes support for multiview ren-
dering optimization techniques, such as view-specific level-of-detail systems and
pipeline modifications. Additional discussions include insights on further exten-
sions, design guidelines, and other possible uses of the presented architecture.
Most of the discussions, terminologies, and guidelines in this chapter follow
OpenGL conventions. The implementation of the architecture is included in the
OpenREng library,
1
an open-source rendering engine based on modern desktop
and mobile OpenGL specifications with multiview rendering support, and the
sample applications are built using this library.
12.2AnalyzingMultiviewDisplays
At its core, rendering for 3D displays aims to generate separate images (views)
for separate eyes. All types of 3D displays, summarized below, create the illusion
of 3D space based on multiplexing the different images for each eye, whether it
be temporally, spatially, or in color. It is a challenge to support all types of dis-
plays transparently to application implementations in a way that hides the details
of the low-level resource management required for multiview rendering.
Displays that require wearing special eyeglasses can be subdivided into the
following categories (as also shown in Figure 12.2):
1
See http://openreng.sourceforge.net/.
Mono
Parallax-based
Lenticular-based
Anaglyph-based
12.2AnalyzingMultiviewDisplays 181
Figure 12.2. Stereo rendering techniques that require wearing glasses. From left to right, anaglyph
glasses, polarized glasses, shutter glasses, and head-mounted displays.
■ Anaglyph glasses. These are based on multiplexing color channels. The two
views are filtered with different colors and then superimposed to achieve the
final image.
■ Head-mounted displays (HMDs). These are based on displaying both views
synchronously to separate display surfaces, typically as miniaturized LCD,
organic light-emitting diode (OLED), or CRT displays.
■ Shutter glasses. These are based on temporal multiplexing of the two views.
These glasses work by alternatively closing the left or right eye in sync with
the refresh rate of the display, and the display alternately displays a different
view for each eye.
■ Polarized glasses. With passive and active variants, these glasses are based
on presenting and superimposing the two views onto the same screen. The
viewer wears a special type of eyeglasses that contain filters in different ori-
entations.
While these displays require special hardware, another type of 3D display,
called an autostereoscopic display, creates the 3D effect without any special eye-
glasses. Autostereoscopic displays operate by emitting a different image toward
each eye of the viewer to create the binocular effect. This is achieved by aligning
an optical element on the surface of the screen (normally an LCD or OLED) to
redirect light rays for each eye. A composite image that superimposes the two
views is rendered by the display subpixels, but only the correct view is directed
to the corresponding eye.
There are two common types of optical filter, a lenticular sheet and a parallax
barrier. A lenticular sheet consists of small lenses having a special shape that re-
fract the light in different directions. A parallax barrier is essentially a mask with
GB
R
GB
R
GB
R
GB
R
GB
R
Right(GB)
Colorfiltering
glasses
Screen
RGB
RGB
RGB
RGB
RGB
RGB
RGB
RGB
RGB
RGB
Right
Screen
−
+
−
+
−
+
−
+
−
+
Right(−)
filteringglasses
Polarizedscreen
RGB
RGB
RGB
RGB
RGB
RGB
RGB
RGB
RGB
RGB
Right
HMDs
182 12.AGenericMultiviewRenderingEngineArchitecture
Figure 12.3. Autostereoscopic displays: a lenticular sheet (left) and a parallax barrier
(right).
openings that allow light to pass through only in certain directions. These two
technologies are illustrated in Figure 12.3. In both cases, the intensity of the light
rays passing through the filter changes as a function of the viewing angle, as if
the light is directionally projected. The pixels for both eyes are combined in a
single rendered image, but each eye sees the array of display pixels from a differ-
ent angle and thus sees only a fraction of the pixels, those precisely conveying
the correct left or right view.
The number of views supported by autostereoscopic displays varies. The
common case is two-view, which is generally called stereo-view or stereo-
rendering. Yet, some autostereoscopic 3D displays can render 4, 8, or 16 or more
views simultaneously. This allows the user to move his head side to side and ob-
serve the 3D content from a greater number of viewpoints. Another basic varia-
ble is the size of the display. Three-dimensional TVs, desktop LCD displays, and
even mobile devices with multiview support are becoming popular and accessible
to the mass market.
As a result, it is a challenge to build applications that run on these different
types of devices and 3D displays in a transparent way. There is a need for a mul-
tiview rendering architecture that hides the details of multiplexing and displaying
processes for each type of display.
12.3TheArchitecture
The architecture we present consists of the following:
■ An extensible multiview camera abstraction.
■ An extensible multiview compositor abstraction.
L
R
L
R
L
R
L
R
L
R
Screen
L
R
L
R
L
R
L
R
L
R
Right
Parallax
barrier
lenses
Screen
Right
12.3TheArchitec
t
■
A
r
■
R
■
L
T
the
m
disp
l
on t
h
vie
w
pict
u
gra
p
it o
n
Brie
mul
t
Fig
u
M
b
uf
fe
of t
h
disti
os.
A
Fi
gu
conf
i
and
o
t
ure
A
configura
b
r
endering res
u
R
endering pi
p
L
evel-of-det
a
T
o be able t
o
m
ultiview sy
s
l
ay surfaces.
h
e screen, e
v
w
por
t
, as sho
w
u
re multivie
w
p
hical user in
t
n
ly a single ti
fly, with this
t
iview conte
n
u
re 12.5.
M
ultiview re
n
fer
, and a mul
t
h
ese compon
e
nct set of fe
a
At
render ti
m
u
re 12.4. The
i
gurations: an
a
o
n-target (sten
c
b
le multivie
w
u
lts.
p
eline modif
i
a
il implemen
t
o
support bot
h
s
tem archite
c
This further
v
en with diff
e
w
n in Figure
w
regions to
y
t
erface (GUI
)
me after the
approach, y
o
n
t to be rend
e
n
dering is en
a
t
iview comp
o
e
nts are confi
g
a
tures, the sy
s
m
e, the multi
v
same scene r
e
a
glyph using
c
c
iling), on-tar
g
w
buffer tha
t
i
cations to su
p
t
ations for m
u
h
single-vie
w
c
ture is integ
r
allows mult
i
e
rent multivi
e
12.4. With
t
y
our screen,
o
)
elements o
v
m
ultiview c
o
o
u have cont
r
e
red. An ove
r
a
bled by atta
c
o
sitor to a vi
e
g
urable on th
s
tem can be a
d
v
iew renderi
n
e
ndered to dif
f
c
olo
r
-mode (b
o
g
e
t
-wiggle, an
d
t
holds inte
r
p
port multivi
e
u
ltiview rend
e
w
and multivi
r
ated into a
v
i
view render
i
e
w configur
a
t
his approac
h
o
r you can s
h
v
e
r
multivie
w
o
ntent is mer
g
r
ol over wher
e
rview of ou
r
c
hing a multi
v
e
wport in a re
h
eir own and
p
djusted to fit
n
g pipeline i
s
f
erent viewpo
r
o
tto
m
-left), p
a
d
on-target-sep
r
mediate vie
w
e
w rendering
e
ring.
i
ew renderin
g
v
iewport abs
t
i
ng in multip
a
tions for ea
c
h
, you can ad
h
ow your sin
w
3D content
b
g
ed into the
f
e
and how y
o
r
architecture
v
iew camera,
e
nder windo
w
p
rovide abstr
a
into many t
a
s
activated if
r
ts with differ
e
a
rallax using
b
arated.
w
-dependent
.
g
seamlessly,
t
raction over
le viewports
c
h multiview
d picture-in-
gle-view 2D
b
y rendering
f
rame buffer.
o
u want your
is shown in
a multiview
w
. Since most
a
ction over a
a
rget scena
r
i-
the attached
e
nt multiview
b
oth off-target
183
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.