5.8 MPEG-4 BIFS and AFX

Standardization of representation formats can facilitate content exchange and encourage the popularity of consumer advanced display devices. An early attempt to standardize 3D representation is conducted by MPEG-4 via binary format for scenes (BIFS) [57]. BIFS is specified as an extension of virtual reality model language (VRML) and inherits all features from VRML. VRML is a text format and defines a set of elements to transmit 3D graphics, including the geometric primitives, 3D meshes, textures, and appearance. The VRML data can be rendered by a VRML player. The interactivity between users and the virtual world, including light sources and collision, is supported. BIFS enriches VRML by providing a better representation, delivery, and rendering for interactive and streamable rich-media services, including 2D and 3D graphics, images, text, and audiovisual material. In place of the text format and the download-and-play of VRML, BIFS adopts binary representation and has a streaming format already adopted in MPEG-4 design. MPEG-4 BIFS extends VRML to include the following new features. First, MPEG-4 BIFS supports integration and management of different audio and video objects seamlessly in a scene. Besides, graphical constructs for 2D and 3D graphics, authoring of complex face and body animation, tools for 3D mesh encoding, and representation of 2D and 3D natural and synthetic sound models are also provided in MPEG-4 BIFS. BIFS enables interactivity between the client-side scene and remote servers by defining input elements such as keyboard and mouse. MPEG-4 scenes equip two types of scripted behavior: a Java API can control and manipulate the scene graph and the built-in ECMA script (JavaScript) support that can be used to create complex behaviors, animations, and interactivity. The BIFS also provides streamable scene description to detail the spatial and temporal graphics in a BIFS-Command stream. With commands, a stream can edit the scene graph to replace, delete, and insert elements in the scene graph. With MPEG-4 system layer, the audio and visual content can be tightly synchronized with other A/V content.

However, BIFS cannot fulfill the emerging requirements of the interactivity and navigability of 3D worlds. An extension to MPEG-4, animation framework extension (AFX) [57], is proposed to equip with the better ability to handle computer games and animation. The AFX defines six components in a hierarchical way to accomplish the designed goal. The first component is the geometric component which describes the appearance and the form of an object. An extension of the geometric models with additional linear or nonlinear deformations is included in the modeling component to allow transformation. The third component is the physical component describing the objects' physics in the virtual world, such as motion and gravity. Based on the physical component, a biomechanical component is defined to describe the muscles on animals. There are two other higher level components, namely, behavior component and cognitive component, which define objects' reactive behavior and learning ability.

Four different categories of tools are included in AFX:

1. Shape: NURBS, subdivision surfaces, MeshGrid, particle systems, solid representation.
2. Texturing: procedural texture, depth image based rendering, light-field mapping.
3. Deformation: nonlinear deformer, freeform deformer.
4. Animation: skeleton/muscle/skin-based animator.

We have noticed that MPEG has defined many audiovisual standards, a vendor may want to choose different 3D/visual coding tool with different audio capability from one of the MPEG-x family to meet its unique market. To facilitate this great flexibility, MPEG-A [58] specifies multimedia application formats (MAFs) to assemble the existing MPEG audio/video coding tools to achieve the targeted immersive 3D applications.