i
i
i
i
i
i
i
i
9.3. Mosaicing and Making Panoramic Images 229
Of course, getting the iterative algorithm started is no easy matter. When
building a panorama, it is likely that image I
n
will have a lot of overlap with
I
n−1
, and so starting with the R that placed I
n−1
in the panorama is probably
the best way to generate an initial guess. However, if the overlap between two
successive frames is small (less than 50%), an alternative strategy of determin-
ing R, or at least getting an initial guess for it, must be followed. The two
most successful appr oaches are:
1. Hierarchical matching. Small subsampled and smoothed versions of I
n
and I
c
are made, e.g., images may be downsampled from 512 ×512 to
32 × 32 as a first guess. In these smaller images, the overlap appears
greater and, therefore, easier to determine R.OnceanR has been
determined, a larger subsampled image (e.g., 256 × 256) is used and
the least squares algorithm applied so as to refine the best estimate
for R.
2. Phase correlation [10]. This is most useful when the overlap is very
small. It is generally used to estimate the displacement between two
planar images. For example, two aerial photos can be montaged using
this approach. In it, a 2D Fourier transform of the pair of images is
made. The phase difference between them at each frequency is deter-
mined. An inverse transform is applied to the difference signal. Back
in the spatial domain, the location of a peak in the magnitude shows
where the images overlap.
In cases where I
n
has no overlap with I
c
, I
n
will have to be discarded
(temporarily) and reconsidered later after other images which may overlap
with it have been added into the mosaic. All in all, producing panoramic
mosaic images from video footage shot on a basic camcorder is no easy task.
Once panoramic images have been obtained, they can be used in a num-
ber of different ways. For example, McM i llan and Bishop [13] have demon-
strated how two panoramic images acquired from camera centers located
about 20 inches apart can generate parallax motion in the viewer.
9.3.1 QuickTime VR
Apple’s QuickTime movie file format was introduced in Chapter 5 in the
context of a container for storing compressed video and movies. Within a
QuickTime movie, there can be multiple tracks, with each one storing some
i
i
i
i
i
i
i
i
230 9. Image-Based Rendering
type of linear media indexed with time, video and audio being the most well
known. A track can contain compressed data that requires its own player. The
QuickTime VR (QTVR) standard adds two new types of track. The first type
of track provides an inward-looking (or object) view which allows an object
to be looked at from any orientation. The second track holds a collection
of outward-looking environment panorama views. Appropriate interactive
software gives the user the impression of control over the view and viewpoint.
In the object format, the movie looks conventional, with each frame con-
taining a different view of the photographed object. It is the duty of the player
to find and display the correct frame in the movie given the required viewing
direction. Extra frames can be added to allow for animation of the object.
In the panoramic format (see Figure 9.10), a number of panoramic images
are stored in a conventional movie track. These views give a 360
◦
view from
several locations; it could be in different rooms of a virtual house or the same
view at different times. The player can pan around these and zoom in and
out. To allow the user to select a specific image, possibly by clicking on a
hotspot in the image, additional data can be added to the tracks. For example,
the panoramic track may contain data called nodes which correspond to points
in space. The nodes identify which panoramic image is to be used at that
point in space and identify how other panoramic images are selected as the
user moves from node to node. Hotspot images may also be mixed in to
give the appearance of being attached to objects in the panoramic view, say to
provide some information about an object, for example exhibits in a virtual
museum.
Figure 9.10. QuickTime VR, panoramic display process.
i
i
i
i
i
i
i
i
9.4. Summary 231
The panoramic images, which tend to be rather large, are usually divided
into smaller sub-images that are nearer the siz e of conventional video frames.
When playing the QTVR movie, there is no need for the frames to be stored
in any particular order. On playback, only those sub-images that lie in the
field of view need to be loaded from the file, so little RAM storage is usually
needed to interact with a QTVR movie. In some players, image caching may
be used to aid accelerated display. The final stage of rendering is accomplished
by a patented cylindrical to planar image warp [4] which renders the view
onto the flat screen.
QTVR is not the only panoramic image viewer, but it is the only one
tightly integrated into a major operating system. Other panoramic VR view-
ing image preparation software packages are relatively common and commer-
cially available. Some have been particularly designed for use on the Web. As
mentioned before, realtor websites abound with virtual tours of their property
portfolios.
9.4 Summary
In this chapter, w e looked at the three classical image-based rendering tech-
niques: rendering with no geometry, rendering with implicit geometry and
finally rendering with explicit geometry. Ho wever, r egardless of which tech-
nique is used, the main advantage that IBR has to offer over traditional ren-
dering techniques based on polygonal models is its ability to render a scene
or object in great detail. The fact that the rendering time is independent of
the complexity of the scene is another significant plus. Before 3D-accelerated
graphics pr ocessors became commonplace, IBR was the only way to achieve
high-quality real-time graphics. Parallax effects, virtual tours and varying
lighting conditions can all now be simulated with IBR. This has great utility
in many different types of applications, one of the most common being ar-
chaeological walkthroughs where it is imperative to maintain the detail of the
real environment. Here, many people will get to experience a virtual archae-
ological site, where only a limited number of people would have access to the
real site.
The only real negative in IBR is the loss of flexibility in choosing viewing
conditions. Where IBR really excels is in producing environmental back-
grounds and maps for use as part of a scene described by polygons and ren-
dered in real time with a GPU. So, even if QuickTime VR fades from use,
the production of panoramic image maps will still be an essential element of
VR work, and as such, IBR will retain its importance to the VR community.
i
i
i
i
i
i
i
i
232 9. Image-Based Rendering
Bibliography
[1] E. Adelson and J. Bergman. “The Plenoptic Function and the Elements of
Early Vision”. In C omputational Models of Visual Processing, M. Landy and J.
Movshon (editors), pp. 3–20. Cambridge, MA: The MIT Press, 1991.
[2] Be Here Corp. “TotalView”. http://www.behere.com/.
[3] S. Chen. “QuickTime VR: An Image-Based Approach to Virtual Environment
Navigation”. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings,
Annual Conference Series, edited by R. Cook, pp. 29–38. Reading, MA: Addison
Wesley, 1995.
[4] S. Chen and G. Miller. “Cylindrical to Planar Image Mapping Using Scanline
Choerence”. United States Patent no. 5,396,583. 1995.
[5] S. Chen and L. Williams. “View Interpolation for Image Synthesis”. In Pro-
ceedings of SIGGRAPH 93, Computer Graphics Proceedings, Annual Conference
Series, edited by J. Kajiya, pp. 279–288. New York: ACM Press, 1993.
[6] Columbia University, Computer Vision Laboratory. “Applications of 360 De-
gree Cameras”. http://www1.cs.columbia.edu/CAVE//projects/app
cam 360/
app
cam 360.php.
[7] S. Gortler et al. “The Lumigraph”. In Proceedings of SIGGRAPH 96, Computer
Graphics Proceedings, Annual Conference Series, edited by H. Rushmeier, pp. 43–
54. Reading, MA: Addison Wesley, 1996.
[8] P. Heckbert. “Filtering by Repeated Integration”. Computer Graphics 20:4
(1986) 317–321.
[9] K. Huang and M. Trivedi. “Driver Head Pose and View Estimation with Single
Omnidirectional Video Stream”. In Proceedings of the 1
st
International Workshop
on In-Vehicle Cogn itive Computer Vision Systems , pp. 44–51. Washington, D.C.:
IEEE Computer Society, 2003.
[10] C. Kuglin and D. Hine. “The Phase Correlation Image Alignment Method”.
In Proceedings of the IEEE Conference on Cybernetics and Society, pp. 163–165.
Washington, D.C.: IEEE Computer Society, 1975.
[11] W. Matusik et al. “Image-Based 3D Photography using Opacity Hulls”. Proc.
SIGGRAPH ’02, Transactions on Graphics 21:3 (2002) 427–437.
[12] L. McMillan. “An Image-Based Approach to Three-Dimensional Computer
Graphics”. Ph.D. Thesis, University of Nor th Carolina at Chapel Hill, TR97-
013, 1997.
i
i
i
i
i
i
i
i
Bibliography 233
[13] L. McMillan and G. Bishop. “Plenoptic Modelling: An Image-Based Render-
ing System”. I n Proceedings of SIGGRAPH 95, Computer Graphics Proceedings,
Annual Conference Series, edited by R. Cook, pp. 39–46. Reading, MA: Addi-
son Wesley, 1995.
[14] J. Meehan. Panoramic Photography. New York: Amphoto, 1990.
[15] W. Press et al. (editors). Numerical Recipes in C++: The Art of Scientific Com-
puting, Second Edition. Cambridge, UK: Cambridge University Press, 2002.
[16] S. Seitz a nd C. Dyer. “View Morphing”. In Proceedings of SIGGRAPH 96, Com-
puter Graphics Proceedings, Annual Conference Series, edited by H. Rushmeier,
pp. 21–30. Reading, MA: Addison Wesley, 1996.
[17] H. Shum and S Kang. “A Review of Image-Based Rendering Techniques”. In
Proceedings of IEEE/SPIE Visual Communications and Image Processing (VCIP)
2000, pp. 2–13. Wa shington, D.C.: IEEE Computer Society, 2000.
[18] H. Shum and R. Szeliski. “Panoramic Image Mosaics”. Technical Repor t MSR-
TR-97-23, Microsoft Research, 1997.
[19] R. Szeliski. “Image Mosaicing for Tele-Reality Applications”. Technical Report
CRL 94/2, Digital Equipment Corp., 1994.
[20] R. Szeliski et al. “Creating Full View Panoramic Image Mosaics and Texture-
Mapped Models”. In Proceedings of SIGGRAPH 97, Computer Graphics Proceed-
ings, Annual Conference Series, edited by T. Whitted, pp. 251–258. Reading,
MA: Addison Wesley, 1997.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.