i
i
i
i
i
i
i
i
216 8. Computer Vision in VR
Figure 8.24. A distinctive marker in the field of view of the camera provides a frame of
reference that is used to place virtual objects in the scene with a size and orientation
calculated to make them look like they are attached to the marker.
provides a neat library of functions for use in a C++/C application program
to overlay 3D virtual objects on the video from a digital camera in real time,
as illustrated in Figure 8.24. To be able to calculate a transformation that
will place the 3D virtual object correctly over the marker, the software has to
recognize the marker in the first place and then find its four corners. This
requires four steps:
1. Every pixel in the camera’s image raster is converted to a binary value,
i.e., set to black or white. Above a given threshold, the pixel’s intensity
is set to “1”; below it, the pixel value is “0”.
2. Using the binary image, identify a set of contour lines in the image.
3. Try to find the places where these lines meet to form the corners of the
possible image of a marker.
4. Using the known pattern of the markers, attempt to match it to the re-
gion bounded b y the possible corners and edges of the possible marker.
Once the marker has been correctly identified, the 3D virtual object will be
transformed so this it sits on top of it. More details of this process are given
in Chapter 18, where we document a project using the ARToolKit.
8.5 Summary
In this chapter, we have given y ou a quick run through of some of the con-
cepts of computer vision and how they might be applied to enhance and
i
i
i
i
i
i
i
i
Bibliography 217
facilitate the VR experience. In particular, a number of the key algorithms
which emerged during the development of the geometric theory of computer
vision have great utility in the implementation and delivery of practical VR
applications.
Two of the most significant applications of computer vision in the context
of VR are in augmented reality (for example, in-place videoconferencing) and
image-based rendering, which is the subject of the next chapter.
Of course, being able to do all this means that the images you take must
be geometrically accurate. Therefore, we need to remove any lens distortion
and perspective distortion from the images before we try to match them to-
gether. We also need to know the geometrical model of the camera i.e., the
camera model. This model is difficult to determine, and to do so we entered
into the mathematical complexities of the fundamental matrix etc. Essentially,
we work backwards by identifying similar points from multiple images of the
same scene to recover the camera model. This is non-trivial and requires
the use of minimization algorithms, all of which w ere described within this
chapter.
Thus,wehopewehavegivenyouaninsightintothetheorythatsur-
rounds computer vision and how that theory can be utilized to enhance the
VR experience.
Bibliography
[1] The Augmented Reality Toolkit. http://www.hitl.washington.edu/artoolkit/.
[2] M. Billinghurst and H. Kato. “Real World Teleconferencing”. In CHI ’99 Ex-
tended Abstracts on Human Factors in Computing Systems, pp. 194–195. New
York: ACM Press, 1999.
[3] O. Faugeras. Three-Dimen s ional Computer Vision: A Geom etric Viewpoint.
Cambridge, MA: The MIT Press, 1993.
[4] O. Faugeras and T. Papadopoulo. “A Nonlinear Method for Estimating the
Projective Geometry of 3 Views”. In Proceedings of the Sixth International Con-
ference on Computer Vision, pp. 477–484. Wshington D.C.: IEEE Computer
Society, 1998.
[5] M. Fischler and R. Bolles. “Random Sample Consensus: A Paradigm for Model
F itting with Applications to Image Analysis and Automated Cartography”.
Communications of the ACM 24:6 (1981) 381–395.
i
i
i
i
i
i
i
i
218 8. Computer Vision in VR
[6] R. Hartley. “Lines and Points in Three Views: An Integrated Approach”. In
Proceedings of the ARPA Image Understanding Workshop, pp. 1009–1016. San
Francisco, CA: Morgan Kaufmann, 1994.
[7] R. Har tley and A. Zisserman. Multiple View Geometry in Computer Vision, Sec-
ond Edition. Cambridge, UK: Cambridge University Press, 2004.
[8] Intel Corp. “Open Source Computer Vision Library”. http://www.intel.com/
technology/computing/opencv/.
[9] T. Jebara et al. 3D Structure from 2D Motion. IEEE Signal Processing Magazine
16:3 (1999) 66–84.
[10] Microsoft Research, Vision Technology Group. “The Micorsoft Vision SDK,
Version 1.2”. http://research.microsoft.com/research/vision, 2000.
[11] M. Nixon and A. Aguado. Feature Extraction and Image Processing. Boston, MA:
Newnes, 2002.
[12] W. Press et al. (editors). Numerical Recipes in C++: The Art of Scientific Com-
puting, Second Edition. Cambridge, UK: Cambridge University Press, 2002.
[13] A. Shashua and M. Werman. “On the Trilinear Tensor of Three Perspective
Views and Its Underlying Geometry”. In Proceedings of the International Con-
ference on Computer Vision, pp. 920–925. Washington, D.C.: IEEE Computer
Society, 1995.
[14] A. Vijayan, S. Kumar and D. Radhakrishnan. “A New Approach for Nonlinear
Distortion Correction in Endoscopic Images Based on Least Squares Etima-
tion”. IEEE Transactions on Medical Imaging 18:4 (1999) 345–354.
[15] C. Zhang, J. Helferty, G. McLennan and W. Higgins. “Non Linear Distortion
Correction In Endoscopic Video Images”. In Proceedings of International Con-
ference on Image Processing, pp. 439–442. Washington, D.C.: IEEE Computer
Society, 2000.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.