User agents are audio/video endpoints and call control servers in a SIP network. These endpoints have a client element, the user agent client (UAC), and a server element called the user agent server (UAS). A SIP user agent (UA) has both a client and a server. The client initiates the requests, and the server initiates the responses.

A SIP conference server is sometimes also referred to as Back-2-Back UA(B2BUA). It is a SIP element that has two UAs working back to back and thus can control the SIP dialogs as they go through it. Note that the endpoints are not considered B2BUAs.Figure 5-1 shows a SIP UA (video phone) initiating a SIP call to another user agent.

Figure 5-1. SIP UAs in a SIP Session

A proxy server receives SIP requests, determines where to send them, and passes them to the next server (the next server could be a user agent). The proxy server can provide functions such as authentication, authorization, routing of messages, security, and so on.

There are two types of proxy servers: stateful and stateless. A stateful proxy server stores incoming requests it receives, along with the responses it sends back and the outgoing requests it sends to other UAs and proxies. A stateless proxy server maintains no proxy information after it services a request. A stateful proxy server can keep track of active sessions, which allows it to load-balance the sessions across multiple SIP control servers.

Figure 5-2 shows two UAs (IP phones) involved in a SIP session through a single proxy server. Both phones are registered with the proxy server. IPP1 sends a SIP INVITE message to the proxy. The proxy finds the location of the called party (IPP2) and forwards the INVITE. The proxy forwards all responses (100 Trying/200 OK) from IPP2 to the calling party (IPP1). In this example, the ACK response goes directly from IPP2 to IPP1. This can happen if the proxy does not insert a Record-Route SIP header in the SIP messages. Record-Route headers are explained in detail in the section “SIP Record Routing.”

Figure 5-2. SIP Session Through Proxy Server

A redirect server accepts a SIP request directed toward a particular UA and returns an alternate address to the initiating UA, thus providing forwarding services.

Figure 5-3 shows a SIP redirection of a call initiation.

Figure 5-3. SIP Redirect Server

A registrar processes the registration requests from UACs. The registration request contains the current location (typically, IP addresses) of the UA. The registrar maintains a location database to associate SIP URIs to IP addresses. Note that the location could also be a different phone number or URL that is used by the UA for receiving calls. Registrars are usually colocated with the proxy servers.

Figure 5-4 shows an example of SIP registration from a UA to a registrar.

Figure 5-4. Registration Sequence

The following are the different types of SIP requests:

Example 5-1 shows a SIP request message.

!The following line is the request line
INVITE sip:[email protected] SIP/2.0
!The following lines are the SIP headers
Via: SIP/2.0/UDP 172.27.14.4:5070;branch=8dJXAX9MDw

Max-Forwards: 70
To: <sip:[email protected]>
From: <sip:[email protected]>;tag=ds17aa9bd4
Call-ID: [email protected]
CSeq: 1 INVITE
Content-Length: 251
Content-Type: application/sdp

!The following lines are the SDP body


v=0
o=Sam 1549546120 0 IN IP4 10.10.10.26
s=-
c=IN IP4 10.10.10.26
t=0 0
m=audio 49220 RTP/AVP 0
a=rtpmap:0 PCMU/8000

The following sections describe the different components of the SIP request message.

SIP responses are associated with a SIP request. Example 5-2 shows a typical response message.

SIP/2.0 200 OK
Via: SIP/2.0/UDP
172.27.14.4:5070; branch=z9hG4bKhWn9PFlB2yaZbsvp36
From: <sip:[email protected]>;tag=ds15cee408
To: <sip:[email protected]>;tag=E2CE8-87E
Date: Fri, 01 Mar 2002 00:15:28 GMT
Call-ID: [email protected]
Server: Cisco-conferenceserver
CSeq: 1 INVITE
Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, UPDATE, REFER, SUBSCRIBE, NOTIFY, INFO,
REGISTER, PUBLISH
Contact: <sip:[email protected]:5060>
Reason: Q.850; cause=47
Content-Length: 0

The first line of the response contains the protocol version (SIP/2.0) and the status code (such as 200), along with a text description (such as OK). The status code is an integer from 100 to 699. RFC 3261 defines the following classes of responses:

Table 5-1 summarizes the commonly used response codes.

The request to which a particular response belongs is identified using the CSeq header field. This header field contains the sequence number and the method of the corresponding request. In Example 5-2, it is an INVITE request.

A conference in a SIP framework is identified through a conference URI. The conference URI is the destination where all the SIP requests are sent and created/managed by the conference server. An example of the conference URI is sip:[email protected]. Users can enter these URIs manually in their SIP client to dial into the conference system. Alternatively, the conference system embeds this in a web link and sends the link to the user through e-mail or instant messenger. If the user dials in from the public switched telephone network (PSTN), the PSTN gateway determines the destination conference URI, typically by prompting the user to enter touch tones on the telephone pad. This prompting functionality is provided by the IVR system. The gateway is configured to either forward signaling to a conference URI or forward these DTMF tones to the conference server, which looks up the conference URI and instructs the gateway to forward the call.

Endpoints establish connections on the media plane by first negotiating media properties such as codec types, packetization periods, media IP address/RTP port numbers, and so on. This information is transmitted with SIP messages using SDP. An endpoint may use two methods of exchanging SDP information:

Figure 5-7 shows examples of early and delayed offers. A delayed offer is typically used by call control entities that do protocol translation (for example, an H.323 endpoint dialing into a call agent that has a SIP trunk to the conference server). In this case, the call agent may send a delayed offer INVITE to the conference server as soon as it receives an H.323 SETUP from the endpoint. The endpoint media capabilities are not known to the call agent until the H.245 OLC/OLC-ACK is done.

Figure 5-7. Early and Delayed Offer

Endpoints that connect to a conference server via a PSTN gateway often must navigate through an IVR using DTMF tones, and therefore DTMF support in the endpoints and the conference server is important to the conferencing support. Endpoints can use three methods to send DTMF digits:

Voice-band DTMF tones are modulated as actual tones in the media. Endpoints that dial into a PSTN gateway must play DTMF tones in the media stream so that the PSTN gateway can “hear” the tones. Endpoints connecting via an IP network send DTMF information in-band or out-of-band. RFC 2833 is a special way of sending DTMF in-band, and Key Press Markup Language (KPML) provides a way for the endpoints to send DTMF out-of-band.

m=audio 19008 RTP/AVP 0 8 116
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:116 telephone-event/8000

KPML

KPML is an out-of-band mechanism to collect DTMF digits. A KPML flow is established using SUBSCRIBE/NOTIFY. An endpoint can advertise that it supports KPML by including the string kpml in the Allow-Events header in the initial offer message. Similarly, the conference server may indicate the same in the response. The conference server then sends a SUBSCRIBE to the endpoint with the event package kpml. If the subscription succeeds, the digits are sent in the NOTIFY messages. Figure 5-8 shows the KPML subscription between an endpoint and the conference server.

Figure 5-8. KPML Sequence

The common video codecs used in video conferencing are H.261, H.263, and H.264. This section explains the syntax and semantics for describing parameters related to video codecs. Currently, no standard method exists to specify certain video-related parameters in the SDP offer/answer. These parameters include the following:

Video endpoints and conference servers use the a=fmtp attribute to carry codec-specific parameters. Note that SDP extensions for supporting video are still going through the standards process, and many endpoints and conference servers are using proprietary attributes in the SDP offer/answer. These variations might lead to interoperability issues between endpoints/conference systems.

a = fmtp:<rtp payload type> <options>
<rtp payload type> = 31
<options>= "Size | Annex" where
 Size ="QCIF = MPI" or "CIF = MPI"
 MPI = 1 or 2
 Annex = D

m=video 49222 RTP/AVP 31
a=rtpmap:31 H261/90000
a=fmtp:31 CIF=1 QCIF=1

a = fmtp:<rtp payload type> <options>

<options>= size | annex 
Size = "SQCIF=MPI" or "QCIF=MPI" or "CIF=MPI" or "CIF4=MPI" or "CIF16=MPI" or 
"XMAX=xmax, ymax,mpi" 
MPI = 1 or 2
Annex = F/J/T

m=video 22334 RTP/AVP 96
a=rtpmap:96 H263-1998/90000
a=fmtp:96 QCIF=1 SQCIF=1 CIF=2 CIF4=2 CUSTOM=352,240,1

m=video 22334 RTP/AVP 96
a=rtpmap:96 H263-2000/90000
a=fmtp:96 profile=0; level= 10;

m=video 49198 RTP/AVP 96
a=fmtp:96 SQCIF=1 QCIF=1 CIF=1 CIF4=2 CUSTOM=352,240,1 CUSTOM=704,480,2 F J T
a=rtpmap:96 H263-1998/90000

m=video 6010 RTP/AVP 96
b=AS:768
a=fmtp:96 CIF=1 QCIF=1/MaxBR=7680 profile-level-id=428014
a=framerate:25.0
a=rtpmap:96 H264/90000
a=sendrecv

m=video 49222 RTP/AVP 109 31
b=TIAS:128000
a=rtpmap:109 H264/90000                           
a=fmtp:109 profile-level-id=42800c max-mbps=10000
a=rtpmap:31 H261/90000
a=fmtp:31 CIF=1 QCIF=1

Bandwidth usage is specified with the attribute b: <modifier> <bandwidth value>.Modifier should be application-specific (AS), conference type (CT), or transport-independent application-specific (TIAS), as defined in RFC 3890. The AS bandwidth includes the bandwidth that the RTP data traffic will consume, including the lower layers, down to the IP layer. Therefore, the bandwidth is in most cases calculated by considering the entire IP packet, which includes RTP payload, RTP header, UDP header, and IP header. TIAS indicates the actual bandwidth in bits per second without the lower-layer overhead. If present at the session level, this entry indicates the bandwidth for all the media lines in the SDP offer/answer.

Although a session-level AS parameter should indicate the bandwidth needed for all the media lines in the SDP offer/answer (including audio), you might find that some of the endpoint implementations indicate just the video bandwidth needed.

Some endpoints may send a session-level AS parameter in addition to media-level TIAS to maintain backward compatibility with endpoints that do not support TIAS. In those cases, the session-level AS indicates the transport-independent rate for the call, and the TIAS parameter in the video session is the maximum bit rate that the endpoint can receive. Example 5-12 shows the presence of the bandwidth attribute in the SDP.

a=video 18664 96
b=TIAS:320000
a=rtpmap:96 H263-1998/90000
a=fmtp:96 QCIF=1 SQCIF=1 CIF=4

The b= parameter indicates that the endpoint expects to receive 320 kbps of video, which represents only the H.263 payload without the protocol headers. Upon receiving this SDP information, the conference server allocates CPU resources, and possibly network resources, to accommodate this bandwidth.

Advanced video endpoints may ask the conference server to send multiple video streams. The initial INVITE has one audio m-line (media line) and multiple video m-lines. Multiple video stream capability requires the ability to group the media lines so that the conference server knows which audio stream and video streams are tied together for lip-sync purposes. RFC 3388 defines some attributes (group) for the grouping. The syntax is as follows:

a=group:LS
a=mid:<identification tag>

Consider Example 5-13.

v=0
o=Ron 276544 2887654 IN IP4 10.10.10.18
t=0 0
c=IN IP4 10.10.10.18
a=group:LS 1 2
m=audio 20000 RTP/AVP 0
a=mid:1
m=video 20002 RTP/AVP 31
a=mid:2
m=video 20004 RTP/AVP 34
a=mid:3

The sender of this SDP offer would like to receive three media streams: one audio stream (identified as mid:1) and two video streams (mid:2 and mid:3). Audio stream 1 and video stream 2 are grouped, which indicates that these need to be lip-synced. RFC 3388 mandates that all media streams have a prefix of mid, whether or not they are included in the group, which is why the third stream has a prefix of mid even though it is not included in the group.

Escalation is a process that allows a video-capable endpoint to join the conference in the audio-only mode and later establish a video stream. This process occurs in response to one of two scenarios:

Figure 5-10. Escalation to Video

In the RE-INVITE, the endpoint uses the previous SDP offer/answer information but adds another m-line that includes the video properties. Note that the endpoint can take this opportunity to renegotiate the audio properties, but this type of renegotiation is uncommon.

In Example 5-14, an endpoint sends a RE-INVITE to the conference server to add video to the existing audio session.

v=0
o=bob 2808844564 2808844564 IN IP4 ep.xyz.com
s=
c=IN IP4 ep.xyz.com
t=0 0
m=audio 49172 RTP/AVP 0
a=rtpmap:0 PCMU/8000
m=video 44172 RTP/AVP 31
a=rtpmap:34 H261/90000

In Example 5-15, the conference server responds with the SDP answer confirming that video will be streamed to the endpoint.

v=0
o=alice 2890844526 2890844526 IN IP4 meetings.xyz.com
s=
c=IN IP4 meetings.xyz.com
t=0 0
m=audio 49174 RTP/AVP 0
a=rtpmap:0 PCMU/8000
m=video 44172 RTP/AVP 31
a=rtpmap:34 H261/90000

De-escalations are scenarios in which the endpoint tears down a video stream in an audio/video conference. The SIP flow for de-escalation is the same as that for escalation. The endpoint repeats the offer but removes the video information in the SDP offer section by setting the port number of the video stream to 0.

In Example 5-16, an endpoint that initially joined the conference in audio/video mode is now removing video from the session.

v=0
o=bob 2808844564 2808844564 IN IP4 ep.xyz.com
s=
c=IN IP4 ep.xyz.com
t=0 0
m=audio 49172 RTP/AVP 0
a=rtpmap:0 PCMU/8000
m=video 0 RTP/AVP 31
a=rtpmap:34 H261/90000

In Example 5-17, the conference server sends the answer SDP, acknowledging the removal of video for the endpoint.

v=0
o=alice 2890844526 2890844526 IN IP4 meetings.xyz.com
s=
c=IN IP4 meetings.xyz.com
t=0 0
m=audio 49174 RTP/AVP 0
a=rtpmap:0 PCMU/8000
m=video 0 RTP/AVP 31
a=rtpmap:34 H261/90000

The following describes the implementation details of the escalation and de-escalation scenarios:

Escalation and de-escalation present resource-allocation challenges to the conference server. When the endpoint sends the initial INVITE to the conference server to set up an audio-only connection, there is no standard way to indicate in the SIP header or in the SDP offer that this endpoint is video-capable. Therefore, the conference server cannot know at this point whether it needs to allocate a video port to handle a possible escalation. There are a few possible solutions to address this issue:

Two primary video-specific media control operations need to be supported on video conferences:

An endpoint issues a VFU if its decoder requires an I-frame to continue decoding the video stream. When the encoder receives the VFU, it encodes the next frame as an I-frame. The decoder can request a full update or can ask the encoder to update only a part of the frame. The frame is divided into smaller parts, each called a group of blocks (GOB). GOBs are further subdivided into smaller units called macroblocks (MB). An endpoint can specify a video refresh for certain GOBs or MBs. However, endpoints typically request full frames.

Video freeze picture is sent from an encoder to a decoder. In case the encoder is aware of a change in the transmitted picture that would cause loss of synchronization, it requests the decoding side to freeze the picture—that is, to stop presenting the changes until a new stable image is encoded and transmitted. When an endpoint receives the video freeze picture request, it stops decoding the video until it receives a release signal that is part of the video bitstream. This release signal is sent by the encoder when it generates a new I-frame. A typical conferencing application of picture freeze occurs when the active speaker in a multipoint video conference changes. In this case, the conference server can send picture freeze to the endpoints and then send a release signal (the I-frame from the active speaker).

SIP currently has no standard method to support these operations, and competing proposals have emerged for adding these capabilities. However, most of the SIP endpoints and conference servers use the SIP INFO message with an XML document to communicate the codec control primitives. The XML body is carried in an INFO message and is sent by either the endpoint or the conference server. Example 5-18 shows an XML description of an I-frame request sent from a conference server to an endpoint.

INFO sip:[email protected]:5060 SIP/2.0
Via: SIP/2.0/UDP 10.10.10.2:5060;branch=z9hG4bK123F
From: confserver <sip:[email protected]>;tag=1F144-1EEF
To: endpoint <sip:[email protected]>;tag=50a0a0a-13c4-418c31c9-5cb8215-d53
Date: Sun, 08 Jan 2006 00:02:08 GMT
Call-ID: [email protected]
User-Agent: Cisco-meetingplaceexpress
Max-Forwards: 70
Timestamp: 1014941016
CSeq: 101 INFO
Contact: <sip:[email protected]:5060>                                                
Content-Type: application/media_control+xml
Content-Length: 168
<?xml version="1.0" encoding="utf-8"?>
<media_control>
    <vc_primitive>
      <to_encoder>
       <picture_fast_update>
       </picture_fast_update>
      </to_encoder>
    </vc_primitive>
</media_control>

Note that the Content-Type header is set to application/media_control+xml.

Even though using INFO for FVU is commonly supported among the video endpoints and conference server vendors, the solution is not approved by the IETF for a number of reasons:

IETF is currently defining a standard for codec control. It is not SIP-based but rather RTCP-based.

In the centralized conferencing model, the conference server is the central entity and terminates all the SIP signaling. After the media has been connected, the conference server performs IVR functionality, including DTMF collection, in the media plane, without further actions on the signaling plane, as shown in Figure 5-11.

Figure 5-11. Initial IVR Session

In a distributed conferencing model, however, one central, logical conference server is composed of many individual servers. An endpoint might need to be moved from one physical server to another.

In Figure 5-12, endpoint EP dials into the entry IVR associated with the conference server, enters the meeting ID, and goes through the name-recording process. Centralized logic then moves the endpoint to another entity in the conference server that hosts the conference itself. Note that the name recording could be done after the endpoint moves to the conference.

Figure 5-12. Conference Join Through REFER

The entry IVR uses the REFER method to redirect the endpoint to the conference service component. The Refer-To header in the REFER provides the contact URI of the conference server.

The endpoint then sends an INVITE to that URI. The entry IVR sends a BYE to the endpoint immediately after the REFER message, because the INVITE dialog between the entry IVR and the endpoint is no longer meaningful.

Some implementations use INVITE with the Replaces header rather than REFER, as defined in RFC 3891 and as shown in Figure 5-13. The Replaces header is used to logically replace an existing SIP dialog with a new dialog.

Figure 5-13. Conference Through INVITE with the Replaces Header

In this scenario, the destination conference server sends an INVITE with a Replaces header directly to the endpoint, while the endpoint still has a connection to the IVR. The IVR must provide the conference server with the information it may need from the IVR dialog. This information can be exchanged between the conference service and initial IVR through some out-of-band mechanism.

The rationale behind using INVITE with Replaces over REFER has to do with security considerations. The endpoint is authenticated by the conference system when it establishes a dialog relationship with the initial IVR. If the endpoint has to send another INVITE to the conference service (as a result of the REFER), it needs to be authenticated again by the conference service. With INVITE with Replaces, because the INVITE is initiated by the conference system, the conference system can reuse the credentials already established for the endpoint.

While in a conference, the endpoint can initiate a number of in-conference features. These features include roll call request, mute, unmute, hold/resume the call, transfer, and so on. The following sections address some of the most common features briefly.

A roll call as shown in Figure 5-14, asks the conference system to announce the names of all users in the conference. The announcement can be a public announcement, played to all participants in a conference, or it can be a private announcement, played to only a single participant. An end user who requests a private roll call enters the request by pressing a special digit (typically, the pound sign followed by a digit). The conference server detects the DTMF digit(s) and plays the names of all the callers in the system. During a private announcement, the conference server mutes the requesting endpoint while it plays the recorded names of participants. In Figure 5-14, the key sequence #4 notifies the conference server that the endpoint wants a roll call.

Figure 5-14. Roll Call

The conference servers get the DTMF digits and play the prerecorded participant names to the endpoint. Note that the conference server mutes and unmutes the receive side of the stream before playing the names.

The user presses the Hold button on the phone to place the conference call on hold. The endpoint initiates a RE-INVITE and puts the audio stream in sendonly mode, as shown in Figure 5-15.

Figure 5-15. Call Hold

In the following SDP offer/answer exchange, note that the endpoint adds the attribute line a=sendonly, causing audio to flow only from the EP to the conference server. The conference server responds with a=recvonly.

The EP sends RE-INVITE with the offer shown in Example 5-19.

v=0
o=bob 2808844564 2808844564 IN IP4 ep.xyz.com
s=
c=IN IP4 ep.xyz.com
t=0 0
m=audio 49172 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=sendonly

The conference server responds with the answer shown in Example 5-20.

v=0
o=alice 2890844526 2890844526 IN IP4 meetings.xyz.com
s=
c=IN IP4 meetings.xyz.com
t=0 0
m=audio 49174 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=recvonly

When the user resumes the conference call, the endpoint sends a RE-INVITE again with the SDP offer a=sendrecv, and the conference server responds with the same.

An endpoint can mute itself using one of two methods:

An endpoint can instruct a conference server to ignore audio or video media packets by sending proper DTMF tones. In Figure 5-16, the key sequence #5 notifies the conference server that the endpoint wants to be muted. In response, the conference server plays an announcement to the endpoint and proceeds to mute the participant. Unmute works the same way as described for mute.

Figure 5-16. Participant Mute

Some conference systems can dial out to a participant when the conference is about to start. The conference server obtains the participant outdial information from a directory through Lightweight Directory Access Protocol (LDAP) or from a presence server. Regardless, after the conference server knows the SIP URL of the participant, it initiates an INVITE with the SDP offer. In Figure 5-17 the conference server initiates outdial to a participant. The conference server sends an early offer in the INVITE and gets back an answer SDP in the “200 OK” response message.

Figure 5-17. Outdial from the Conference Server