This book describes the protocols, standards, and architecture of systems that deliver real-time voice, music, and video over IP networks, such as the Internet. These systems include voice-over-IP, telephony, teleconferencing, streaming video, and webcasting applications. The book focuses on media transport: how to deliver audio and video reliably across an IP network, how to ensure high quality in the face of network problems, and how to ensure that the system is secure.
The book adopts a standards-based approach, based around the Real-time Transport Protocol (RTP) and its associated profiles and payload formats. It describes the RTP framework, how to build a system that uses that framework, and extensions to RTP for security and reliability.
Many media codecs are suitable for use with RTP—for example, MPEG audio and video; ITU H.261 and H.263 video; G.711, G.722, G.726, G.728, and G.729 audio; and industry standards such as GSM, QCELP, and AMR audio. RTP implementations typically integrate existing media codecs, rather than developing them specifically. Accordingly, this book describes how media codecs are integrated into an RTP system, but not how media codecs are designed.
Call setup, session initiation, and control protocols, such as SIP, RTSP, and H.323, are also outside the scope of this book. Most RTP implementations are used as part of a complete system, driven by one of these control protocols. However, the interactions between the various parts of the system are limited, and it is possible to understand media transport without understanding the signaling. Similarly, session description using SDP is not covered, because it is part of the signaling.
Resource reservation is useful in some situations, but it is not required for the correct operation of RTP. This book touches on the use of resource reservation through both the Integrated Services and the Differentiated Services frameworks, but it does not go into details.
That these areas are not covered in this book does not mean that they are unimportant. A system using RTP will use a range of media codecs and will employ some form of call setup, session initiation, or control. The way this is done depends on the application, though: The needs of a telephony system are very different from those of a webcasting application. This book describes only the media transport layer that is common to all those systems.
The book is logically divided into four parts: Part I, Introduction to Networked Multimedia, introduces the problem space, provides background, and outlines the properties of the Internet that affect audio/video transport:
Chapter 1, An Introduction to RTP, gives a brief introduction to the Real-time Transport Protocol, outlines the relationship between RTP and other standards, and describes the scope of the book.
Chapter 2, Voice and Video Communication over Packet Networks, describes the unique environment provided by IP networks, and how this environment affects packet audio/video applications.
The next five chapters, which constitute Part II, Media Transport Using RTP, discuss the basics of the Real-time Transport Protocol.
Road Map for This Book
You will need this information to design and build a tool for voice-over-IP, streaming music or video, and so on.
Chapter 3, The Real-time Transport Protocol, introduces RTP and related standards, describes how they fit together, and outlines the design philosophy underpinning the protocol.
Chapter 4, RTP Data Transfer Protocol, gives a detailed description of the transport protocol used to convey audiovisual data over IP networks.
Chapter 5, RTP Control Protocol, describes the control protocol, which provides reception quality feedback, membership control, and synchronization.
Chapter 6, Media Capture, Playout, and Timing, explains how a receiver can reconstruct the audiovisual data and play it out to the user with correct timing.
Chapter 7, Lip Synchronization, addresses a related problem: how to synchronize media streams—for example, to get lip synchronization.
Part III, Robustness, discusses how to make your application reliable in the face of network problems, and how to compensate for loss and congestion in the network. You can build a system without using these techniques, but the audio will sound a lot better, and the pictures will be smoother and less susceptible to corruption, if you apply them.
Chapter 8, Error Concealment, addresses the issue of concealing errors caused by incomplete reception, describing several techniques a receiver can use to hide network problems.
Chapter 9, Error Correction, describes techniques that can be used to repair damaged media streams, where the sender and receiver cooperate in repairing the media stream.
Chapter 10, Congestion Control, discusses the way the Internet responds to congestion, and how this affects the design of audio/video applications.
Finally, Part IV, Advanced Topics, describes various techniques that have more specialized use. Many implementations do not use these features, but they can significantly improve performance in some cases:
Chapter 11, Header Compression, outlines a technique that can significantly increase the efficiency of RTP on low-speed network links, such as dial-up modems or cellular radio links.
Chapter 12, Multiplexing and Tunneling, describes how several media streams can be combined into one. The intent is to improve efficiency when many similar streams are to be transported between gateway devices.
Chapter 13, Security Considerations, describes how encryption and authentication technology can be used to protect RTP sessions; it also describes common security and privacy issues.
This book describes audio/video transport over IP networks in considerable detail. It assumes some basic familiarity with IP network programming and the operation of network protocol stacks, and it builds on this knowledge to describe the features unique to audio/video transport. An extensive list of references is included, pointing readers to additional information on specific topics and to background reading material.
Several classes of readers might be expected to find this book useful:
Engineers. The primary audience is those building voice-over-IP applications, teleconferencing systems, and streaming media and webcasting applications. This book is a guide to the design and implementation of the media engine of such systems. It should be read in conjunction with the relevant technical standards, and it builds on those standards to show how a system is built. This book does not discuss signaling (for example, SIP, RTSP, or H.323), which is a separate subject worthy of a book in its own right. Instead it talks in detail about media transport, and how to achieve good-quality audio and smooth-motion video over IP networks.
Students. The book can be read as an accompaniment to a course in network protocol design or telecommunications, at either a graduate or an advanced undergraduate level. Familiarity with IP networks and layered protocol architectures is assumed. The unique aspects of protocols for real-time audio/video transport are highlighted, as are the differences from a typical layered system model. The cross-disciplinary nature of the subject is highlighted, in particular the relation between the psychology of human perception and the demands of robust media delivery.
Researchers. Academics and industrial researchers can use this book as a source of information about the standards and algorithms that constitute the current state of the art in real-time audio/video transport over IP networks. Pointers to the literature are included in the References section, and they will be useful starting points for those seeking further depth and areas where more research is needed.
Network administrators. An understanding of the technical protocols underpinning the common streaming audio/video applications is useful for those administering computer networks—to show how those applications can affect the behavior of the network, and how the network can be engineered to suit those applications better. This book includes extensive discussion of the most common network behavior (and how applications can adapt to it), the needs of congestion control, and the security implications of real-time audio/video traffic.
In summary, this book can be used as a reference, in conjunction with the technical standards, as a study guide, or as part of an advanced course on network protocol design or communication technology.