CHAPTER 5

Business—Technology Interface

5.1. When Technology Meets Business

Technology, especially information technology, often involves numbers and formulas, machinery and components, and so forth. This is certainly true when it comes to MPEG-4, and while you have read and learned quite a lot about this technology by this point, you may be asking yourself, do I actually need this, and if so, how and where would I use it? Well, as we all know, technology is a key ingredient in any product, no matter whether it creates nice moving pictures for you, long-lasting concrete, fast cars, or wonderful textiles—it is always there. The trouble starts when the technical experts come along and propose—and in many cases rightly so— the development, introduction, adoption, or deployment of new technologies to improve the quality of products. Any and all of these proposals demand a certain level of investment of the usual resources, which, at the end of the day, all come down to money.

Here we are, right in the middle of an MPEG-4 book, talking about the common influences in business life from the point of view of an investor—someone who must decide whether or not to invest and deploy new technologies in order to improve the overall performance, standing, and indeed value of his or her company.

The concerns of an investor often lie, to a great extent, in the skepticism toward this new MPEG-4 technology and whether it is good enough, and whether your company’s products and services will be successful or not. Technically, there is more than proof in the pudding that MPEG-4 will be a dominant and key player in the multimedia landscape, and that it is an exceptionally well-developed technology. But as is not uncommon in business, there are a few obstacles in terms of adoption, deployment, usage, and, indeed, migration to this new technology, which we will address later on. Economically, the prerequisites are also ready to be listed. Amongst them are obviously the availability of respective products and services offered under business models that make sense and the commercial basics surrounding standards-based technologies, such as licensing. But in every worry there is also a challenge and, with a diligent approach, risks can certainly be minimized and controlled. As a matter of fact, if you are a realistic optimist you will see the opportunity in the difficulty; if you are, however, a pessimist you sadly see the difficulty in the opportunity, which is of course not very helpful in business at all. The next question is related to desire for profit—how much can we make with this? How much can we save with this? And so forth. The kind of strategies centered around profitability, affordability, revenue, and market share growth, and, at some stage, in most cases exit strategies or end games, come to most people’s mind. Exit- and end-games in the MPEG-4 world are not really dissimilar to any other technologies or enterprises. Either a company gets acquired or gets into a merger, competes a trade sale, or attracts major strategic partners, delivering more revenues. Obviously, this chain of potential events can be continued for quite some time. In addition, we are talking about an investment in the future, which can, if managed properly, certainly create long-term impacts on your revenues. After all, you are not interested in this technology because your mission is to utilize new technologies in your business; your mission is to successfully create revenues and profits. This is not meant to be a sales pitch, but clearly the main objective in creating, developing, and deploying new technologies is for our own and your organization’s benefit.

Here is the true interface of technology and business. The important bit is obviously to understand and to differentiate between what you want to, should, and must have in order to stay competitive. Technology in some way determines your destiny in this respect, and you must carefully look at the potential business benefits, and indeed risks, associated with various technologies. Equally important is the decision of whether to go a “standard” or “non-standard” route. This is where coherent decisions need to be made, and it is very important to look at all the pros and cons from as many perspectives as possible. It also, of course, depends on which side you are on (i.e., whether you are a vendor or a user). It may well be that a vendor will go down the opportunistic path and offer products, services, or solutions for both variants. Or the vendor might exclusively commit herself to one direction. A user will always go down the path of least resistance in the sense that as long as products and services are affordable and offer satisfactory quality, they are what he or she will opt for.

5.2. Arguments for MPEG-4

The beauty of MPEG-4 is that it is not only an extremely advanced and versatile technology, it also delivers new and exciting business opportunities.

Already at this point you may be thinking, technologists come up with more and more solutions to all our media problems by the hour. What’s so special about MPEG-4? The answer lies in the simple fact that MPEG-4 is the multimedia standard that enables content to be created, delivered, and consumed in various qualities, for various devices, and is the only standard that gives you the opportunity to create interactive content. The other good news is that MPEG-4 is an international, open standard, which brings with it many advantages, as we saw earlier on. One major advantage, which plays an important role for business decision makers, is choice. To put it in simple terms, buyers are in charge. The many technology providers (i.e., the vendors that are offering their products) strive to deliver best-of-breed products to the marketplace, corresponding to their buyers’ specifications and requirements. Flexibility, transparency, competitiveness, interoperability, and creation of larger markets are the key words surrounding this aspect of open standards.

With “non-standard,” better known as proprietary, technologies or formats, buyers may have a choice of a variety of vendors, but the technology provider is ultimately always the same organization. Buyers run the risk of becoming dependent on the vendor and being captured in a monopolistic environment where many obstacles may need to be overcome over time. In other words, in addition to worrying about your business and pricing models, you will have to do the same with respect to your supplier, who may change, amend, or add to her model without telling you—not a pleasant thought. You may end up having to guess the roadmaps and plans of your supplier, which will divert your attention away from the essential parts of your business. Or you have to take technology licenses, which include much more than you require (for which you will end up paying, too). You may even find yourself in the tricky situation that your supplier becomes a competitor of yours, which might defeat the purpose of the whole relationship.

From a user’s perspective, another important consideration in the global discussion of whether or not to use MPEG-4 is that it was developed by the Moving Picture Experts Group (MPEG). Emmy award–winning MPEG also developed MPEG-1 and MPEG-2, the latter of which currently represents a market worth double-figure billions of dollars.

As a technology user, without a doubt your demands and requirements for technology and services are increasing over time, and you are most interested in availability, reliability, scalability, and price. Standards deliver you all of these. This was only the technology side, we will get to the content issues a little later.

With this brief explanation of the key advantages of an open standard, and specifically MPEG-4, in mind, the next step is to appreciate what this means in the business context. Put simply, the use of MPEG-4 can reduce costs and increase revenues.

Let us boil this down to something even simpler. There are two main benefits, which have a direct impact on business:

5.2.1. MPEG-4 Is a Technical Evolution

Let us look at the technical evolution of MPEG. It is creating new, alternative business models and embracing a broad variety of markets. In fact, it is pretty simple to explain and we have seen the beginning already in the MPEG-2 arena. If one looks at audio/video coding, it goes something like this: “We have something to encode, we have something to deliver, we have something to decode. Plus we serve both the professional and the retail segments.” The rest, to a large extent, is strategy.

However, the situation has changed slightly in recent years. MPEG-2 has reached its limits to some extent, and there are other uses that need to be covered besides digital television and packaged media, for example, DVDs. With MPEG-4 having such vast possibilities for usage, like interactivity (which sadly so far has not really taken off), audio, video (in virtually all networks you can think of), etc., it is the prime technology to close these gaps.

A few chapters earlier, we were able to familiarize ourselves more with the technical advantages and finesse vested in MPEG-4, and a translation of this into business terms is not very difficult. If you are involved in this kind of market and happen to own or distribute some form of content, your prime aim obviously is to reach as wide an audience as possible. Until recently, you were able to work with MPEG-2. Your exposure was kind of limited to digital TV and distribution on DVDs. Certainly, you were able to utilize the additional channel of the Internet, but you actually had to change formats in order to do that, due to issues of bandwidth.

In order to better understand, let us look at aspects of content creation and delivery, which represent the most important parts of the process and are therefore a paramount consideration when it comes to acquisition and running costs.

For the purpose of simplifying, let us break this whole delivery scenario down into networks. Remember the orange juice example? We were talking about the transport of the OJ in the “concentrated” form, say by air, rail, or road—using any or all of these different delivery methods. The most popular transport networks in our multimedia scenario are broadband (Internet streaming), broadcast (digital television), and mobile networks (handheld video)—all different types of universal data-highways.

The delivery is a very crucial aspect of your business considerations. In this respect, there is no difference between you and the OJ producer—all you want is to make sure that your product arrives on time and in good condition. But this does not only depend on the actual delivery or its method. In today’s technology climate, delivery requires that two main factors be taken into consideration. The first is that with broadband connections becoming more widely available, there is an accompanying increase in user/viewer expectations—in fact, many viewers are only interested in and driven by their demand for personally relevant information and content.

Second, where more bandwidth connections are available, more content can, in theory, be streamed or downloaded by viewers. Over the past few years, indisputably, the supply of content offered has increased considerably, as has the demand for it by customers. More and more people are enhancing their “daily dose of news,” for example, by watching news clips on their PC, or they check out the trailers for the newest hit movies.

As a rule of thumb, the easier it is for consumers to receive and acquire content, the more interested and eager they are to get more and diversified content. This can develop into a very cost-intensive exercise for the content owner which is why better compression must be a major consideration. We will look into the cost aspect a bit later; however, one example would be that football enthusiasts will always choose the channel or network that delivers the broadest coverage (i.e., many different camera angles, etc.), and certainly the content owner charges a respective amount of money for the privilege. The fact remains, however, that the underlying risk is never really on the content consumer side, but rather on the content provider side, because the latter has to make an initial investment in the hope that some pays off. But while we are still on the subject of different usages, we should comment on what content actually will be consumed by users and in what circumstances.

In reality, the likelihood of watching a full-length feature film on a handheld device is extremely remote. Handheld devices are ideal media for the delivery of “quick content” such as trailers, news, music clips, stock quotes, or short personal videos. However, with devices like laptop computers, packaged media such as DVDs are already enjoying tremendous popularity—who hasn’t watched a DVD on a plane flight or in a hotel room somewhere. And of course, there is the PC with its big, flat screen, where much longer streamed content can be viewed than on handheld devices. But so far nothing has come along that beats the experience of watching movies, sports, etc., on a quality TV set.

As stated earlier, the market must utilize a variety of formats in order to cover the aforementioned market segments. In today’s technology landscape, with an exponentially growing jungle of content, market participants are becoming more and more aware of the necessity to re-purpose content. Thus, the ideal situation would be to create content for multiple usage scenarios right off the bat. In other words, stick to one format for all media deliveries. This is possible with MPEG-4, because as the multimedia standard for low, intermediate, and high bit-rate coding, there are solutions available for all of these types of content. By a “solution,” we do not mean separate machinery for each of these purposes. One machine will do, and most codecs available in the marketplace today offer a variety of flavors when it comes to encoding. From straightforward, PC-based applications to highly sophisticated encoding platforms able to run parallel encoding jobs, there are enough options to choose from. It does not stop there though. As this technology evolves, progresses, and becomes more efficient, there will be a variety of new features and functionalities becoming available.

Actually, the equation is very simple. With better compression of data, less bandwidth is required, which keeps the cost down. So, with MPEG-4, we can eliminate inconvenient multi-format content creation for multi-network delivery—including IP—and can thereby reduce the actual delivery cost. Also, MPEG-4 can easily be integrated into existing MPEG delivery environments, which is extremely valuable in terms of saving costs. Not bad for a start! If one considers Figure 5.1, quite a lot is immediately evident—the broad spectrum as to where MPEG-4 and MPEG-4 AVC can be utilized. Interestingly enough, if one considers the various usage scenarios like DVD, digital TV, or HDTV and takes into account the unarguably large market in this area, then the conclusion that MPEG-4 will be widely successful can be theoretically drawn. Business-wise, it is extremely important for content owners and providers to be able to turn to one format that can cover all possible exploitation scenarios.

Figure 5.1 This figure shows the various video coding standards and their corresponding bit rates and networks/usages.

5.2.2. MPEG-4 Is an International Open Standard

Let’s now look at the benefits of MPEG-4 as an international open standard. From the standpoint of continuity and stability, the fact that MPEG-4 is an international standard coming from a well-reputed organization is very important, as are its previous successes with the introduction of MPEG-1 and MPEG-2.

Figure 5.2 The benefits of the open standard MPEG-4 at a glance. Competition pulls cost per bit south as interoperability reduces risk. Greater resources drive up the quality per bit.

Market Maturity: In theory, markets actually function quite simply, based on concepts like supply and demand. But there are also underlying issues that make it possible for markets to function for the benefit of all concerned. One of them is the element of time. Although maturity does not necessarily come with time, in the technology world there are many examples where the use of standards has proven to be a key factor in establishing a continuous market, which has brought maturity to the technology itself as well as the products and services offered by market participants. Maturity is, of course, less bound to time, and more affected by the impact a technology has on a marketplace, or on a whole industry. Think about the impact DVDs (and, along with them, MPEG-2) have had since their introduction, or plain old CDs.

Extreme Durability: Another positive aspect of standards is durability. Also an element established by long-term use, durability is founded in robustness and completion. If one considers TV standards like NTSC and PAL, it is apparent that both have stood the test of time and have provided billions of users with moving pictures. Since watching is one thing, and hearing is another, the availability of MP3 (also an MPEG standard) is another good example of a durable standard, which has been successfully deployed all over the world.

Format Continuity: Once a format is agreed on and deployed, the markets require that it not become obsolete in the foreseeable future. The invention of CDs has delivered exactly that—a format that has stood the test of time. Over more than a decade, CDs have become part of our lives in many respects. Initially, we bought these small, silver discs to improve upon the sound quality of our good old LPs (another great example of format continuity), and now we are loading most of our computer programs from them, or utilizing them to store data. Or consider VHS video tapes—another format that has had a significant impact on our entertainment lives for a long period of time, and which is now being changed, slowly but surely, to DVDs.

Innovation: GSM, WLAN, 3GPP. Besides the factor of time, there is also the element of innovation, which is naturally present through the ambition of technologists to develop new technologies and improve existing ones. In order for these innovations to really work, and to deliver the other three aforementioned benefits of maturity, durability, and continuity, all of these elements are packed into agreed-on standards. A wonderful example of this is GSM. We all remember running around with a shoe box-sized mobile telephone, which we now easily slip into our shirt pockets and which we can use virtually anywhere in the world—all because of great efforts by technologists to provide a common standard. WLAN is no different in the sense that “wireless” is the latest industry buzzword, and judging from the overall spread and availability of this technology, one can rest assured it will have a prosperous future (even broadband in this context is not far behind!). Besides GSM, there is also 3GPP, another great standard initiative, which comes into play in next-generation mobile communications like 3G. As it happens, MPEG-4 Visual Simple Profile as well as MPEG-4 AAC audio are optional video and audio codecs within the 3GPP specification. MPEG-4 AVC Baseline Profile will become another optional video codec sometime during 2004.

Figure 5.3 puts the variety of standards into a timeline. It is very interesting to see what sort of quantum leaps the underlying technology has been going through over a relatively brief period of time. From still pictures to video pictures requiring less and less bandwidth while more and more bandwidth is being available is quite a journey. Plus, bear in mind the different kind of lifestyle today compared with the early 1990s. Today, you can get content and information anywhere, anytime, without a problem, which automatically means a bigger audience and therefore potentially more revenues. MPEG-4 is an enabler of the digital future.

Figure 5.3 The evolution and progress of multimedia standards over more than a decade at a glance: from still images to multimedia framework.

What many people do not realize is the fact that open standards are part of our everyday life, as mentioned in some of the examples above. Just make a call on your mobile telephone and you are taking advantage of an open standard, i.e., GSM. Switch on your television and enjoy your favorite films, delivered to you in PAL or NTSC, both open standards. Consider the dominance and popularity of MP3s, also an open standard. There are countless other examples of open standards, many of which have survived the test of time and still play a significant role in today’s technology landscape. In addition, open standards help bring the markets to maturity, as seen with MPEG-2, DVD, and CD technology.

5.2.3. Business Arguments for Open Standards

The discerning reader will agree that it is almost impossible to be best at everything. To illustrate, it’s like having a decathlon athlete run 100 meters against the world-record holder in that specific discipline. The decathlon athlete has proven he is world class at all 10 events, but compared to the 100-meter runner, who is focused on his one specialty, he will always come in second.

In the open standard context, developers and manufacturers excel in their specific areas and not in others. The good news here is that “you never walk alone.” That is, in the “standards ecosystem,” many developments occur through joint activities, so you can partner up with the best in the business—pick and choose your partners, so as to deliver best-of-breed products. This very fact significantly lowers risk and exposure, making it easier to operate, including for new entrants to the field.

MPEG is really all about interoperability. Here lies the first theory, which in a wider sense has to do with the overall issue of how to make/save money with MPEG-4. It is of paramount importance that all market participants sit on one side of the table, i.e., technology providers/participants/vendors, and specialize in and focus on the particular discipline where they excel. With the existence of interoperability and specialization in certain distinct fields, market participants not only interoperate on the technical side, they also cooperate on the commercial side.

Competition is obviously still ensured, because the streaming server software vendors, for example, can still compete amongst themselves, as can the codec providers, which is a good thing for any economic ecosystem. In essence, a climate of cooperation and competition can be created, which are both ingredients for a healthy economic climate in any industry.

Ultimately, of course, the customers sitting on the other side of the table decide whose products they will buy. With MPEG-4, they have freedom of choice since competition will “control” prices, which would not be the case if they choose proprietary solutions, where they would in effect be locked into the strategies and plans of a monopoly. So the first task is to create and foster the climate described above.

We could bring in many other examples where standards make life easier, from the lamp on your bedside table to the paper in your printer, fax, or copy-machine, but we will come back to this point later on. Let us now consider the most important business arguments for open standards, all of which, of course, apply to MPEG-4.

Competition: Competition is actually the key element in this whole equation, because to a very large extent your business decision will depend on how much this adventure will cost. Competition is actually a wonderful market mechanism, because it forces the market and its participants to operate within more or less realistic price boundaries. Many roadmaps, philosophies, and strategies collide here, which can result in a variety of different actions, but all of them are orientated toward being competitive at all times. What it boils down to is that competition within this standard ecosystem drives the cost per bit down, which benefits the customer. This is a logical process that has been clearly demonstrated, for example, in the MPEG-2 marketplace. It is far more convenient for a user to keep informed about the market, which guarantees her freedom of choice, rather than having to keep watch over one vendor, hoping that nothing changes.

Interoperability: One really odd aspect of the standard world— including MPEG-4—is interoperability. It’s odd for the simple reason that vendors have to make sure amongst themselves that their tools and equipment do actually work together. So, on one hand you might find bitter rivals competing for valuable market share and revenues, but on the other hand, these rivals may be cooperating to make sure their work fits together. But the fact is that although many vendors compete with each other, they are all working toward the goal of being interoperable, because this is what a standard requires, and it is what makes standards work. So the motivation for a vendor to achieve interoperability with other market participants is clearly and directly linked to being competitive. Being competitive in today’s technology landscape is extremely important. A product that might be offered at a great price, but does not work with other related products, is not of much use. Having said that, interoperability ensures that all vendors are working towards a common principle—they all strive to be best with their particular product and this is exactly what delivers the beauty of standards in that the actual, underlying technology improves over time. So, interoperability is tied to some large extent also into a continuous improvement of standard-based technology. In practice, this works very well, as can be seen from the highly successful and popular interoperability program initiated by the MPEG Industry Forum (MPEGIF). In fact, MPEGIF is currently expanding its reach in this respect by initiating a Logo Qualification Program, which will be available to its members who have successfully completed specific interoperability testing. They will be able to apply the logo issued by MPEGIF, which will gain industrywide recognition that products bearing this logo have successfully proven to be interoperable. In other words, it will in all possibility be established as a seal of quality. It goes without saying that this will also become a strong sales argument amongst vendors.

Greater Resources: We have just learned about the competition driving the cost per bit down, and the coexistence of “rival” vendors through interoperability efforts. To these ends, many resources are deployed, be they on the business development side or the technical side. Focusing for a moment on the technical side, we should realize the following: Over time, the coding efficiency will drastically improve. The same thing happened, as we already know, with MPEG-2, with bit rates having been reduced by over 50% since the standard was frozen in 1996—all, by the way, without upgrading decoders. MPEG specifies and standardizes the decoding side only, which makes these encoding improvements possible. If this fact is paired with the huge amount of specialists from all over the world, it will be clearly apparent that these greater resources actually drive up the quality per bit.

So overall, competition, interoperability, and greater resources provide for an economically healthy, workable environment and deliver an extremely persuasive argument for going with open standards. An additional piece of good news is the fact that not one element in an open standard can be controlled by a single vendor—so no monopolies, but many shops for you to walk into and choose what you like most. That’s the beauty of an open standard and what makes it work!

5.3. MPEG-4 in Action

5.3.1. Broadcast Industry

Bearing in mind that broadcasters need to be able to serve their content on multiple delivery platforms such as TV, Internet, broadband, DVD, wireless, VHS, and so on, they are in need of solutions that provide them with the utmost flexibility, scalability, functionality, speed, and reliability. Many of the systems currently in situ are not likely to be able to manage this additional workload in the most efficient way possible. This is exactly where MPEG-4 comes in. While MPEG-2 is currently regarded as the de facto video coding standard in digital broadcasting, MPEG-4 offers improved coding efficiency resulting in higher quality, particularly at low bit rates, of coded video and audio. With utilization spread across low, intermediate, and high bitrates, MPEG-4 offers a significant advantage compared to other video standards, enabling encoded data to be accessible over a wide range of media in various qualities. MPEG-4 is an extremely interesting format due to its coverage of many types of applications and wide ranges of resolutions, qualities, bit rates, and services. In addition, taking the latest developments into consideration, MPEG-4 will eventually succeed MPEG-2 as the dominant broadcast format.

With this diversity of usage, MPEG-4 can definitely help broadcasters satisfy their requirement for a coherent media asset-management solution and, at the same time, serve as the ideal format for their content delivery and storage. All of this combined offers a great potential to increase the value and return on investment of the broadcasters’ media assets. We will look into this media asset management in more detail further on.

One goal of a broadcaster is to have a solution for easy and accurate search and speedy retrieval within a sophisticated digital broadcast chain, supporting indexing, browsing, and archiving. Content description (as specified in MPEG-7) and Digital Rights Management (with tools from MPEG-4 and MPEG-21) complement this picture.

Broadcasters also face the challenge of being multipurpose-oriented, i.e., their content nowadays is distributed not just via the traditional means, but also is delivered over broadband, IP, and mobile networks. MPEG-4 helps broadcasters meet this challenge by delivering the ideal multimedia standard to broadcasters. In fact, this cross-media exploitation is a unique selling point of MPEG-4. Broadcasters are interested in distributing their content using all the means described above, in order to maximize the value of their assets and use their resources and technology efficiently. Here is where MPEG-4, with its high coding efficiency for low bit rates, comes into its own. This is achieved, for example, through a variety of professional encoding products by numerous companies in the marketplace, which encode real-time and high-quality live or taped content at multiple bit rates on a hard disc, where content can then be archived and stored in different qualities for a variety of uses (IPTV, Internet streaming, mobile phone video).

MPEG-4 video coding is extremely flexible—it can be utilized for low, intermediate, and high bit rates, which is a great advantage compared to other video standards. MPEG-4 covers a wide range of applications, bit-rates, resolutions, qualities, and services, making it the ideal format for media delivery over different types of transmission or storage technologies.

Undeniably, the broadcast industry has changed dramatically over the past few years with the dawning of the digital age. Certainly, most broadcasters still follow the traditional methods of producing and creating, for example, television content, but there is a significant move toward the utilization of digital formats. Currently the most commonly used format is MPEG-2, which is known to all market participants. Also, regarding existing coding standards like MPEG-1, MPEG-2, or H.263, the fact is that MPEG-4 offers a better coding efficiency, which improves the quality of coded video and audio. Therefore, MPEG-4 will play a significant role in the interactive, digital television arena.

One of the characteristics of the broadcast application scenario is the delivery of television service over a unidirectional broadcast channel using a one-to-many communication method. The broadcaster’s goal is to provide a media service to the customer that satisfies a minimum quality requirement for which a customer is willing to pay. In order to be cost effective, it is important to minimize the transmission bandwidth (via satellite, cable, or terrestrial means) that is necessary for the delivery of the service, that is, to minimize the cost for the service as each Megahertz of physical bandwidth has an associated price tag. Therefore, coding efficiency of audio and video codecs is of prime importance. Alternatively, the broadcaster can book a fixed amount of transmission bandwidth from a satellite service provider or cable carrier for a good price. In order to increase the revenue, the broadcaster can increase the number of offered programs if the coding efficiency of the audio and video codecs can be improved while preserving the minimum quality for the visual or audio service as perceived by the end user.

There is a certain asymmetry of the architecture in the broadcast scenario, which requires keeping the complexity and cost on the receiving end low. However, in order to achieve improved compression, it is feasible to accept some moderate increase in cost/complexity for a future decoder or set-top box. A receiver must be able to access programming content without the use of a return path. Therefore, all pertinent information needed to understand the basic organization of a channel or channel multiplex and the data contained therein must be available in the broadcast data on a periodic basis. The acceptable delay from the time a user requests a change in programs (channel surfing) until an “acceptable” quality rendition of the requested material is presented is less than 300 to 500 ms.

Since there is a relatively low number of encoders necessary at the broadcaster’s playout facility, additional complexity and the associated cost for the encoder is less of an issue. The encoding must be able to function in real time for delivering live broadcast of sports or similar live events. However, the broadcast scenario is not overly sensitive to delay incurred by the overall encoding-transmission-decoding signal path. As the consumer has no absolute reference to the actual time instance of events to be broadcast, a delay of up to several seconds is acceptable for the service without a negative impact in the perceived service quality. However, for very brief events, the latency should not exceed the length of the event.

The bit rates that can be expected for a broadcast channel range from hundreds of bits per second up to hundreds of megabits per second. Typical audio-visual broadcast services today use broadcast channels with bit rates in the range of 1.5 Mbit/s to 6 Mbit/s.

Media types to be used in broadcasting include text, still pictures, moving pictures, audio, and graphics. Current examples of multimedia broadcasting include TV station logos (watermarking), graphical overlays used in sporting events and stock tickers, and multi-window screen formats such as those used by Bloomberg Information Television or the sports and stock market crawls used by CNN Headline News.

Besides the more conventional form of media broadcast, there is a new business emerging called “data casting.” This service is characterized by sending information that one assumes many receivers need or wish to receive. This is achieved, for example, by combining multiple data sets in a carousel structure, where each dataset is broadcast repeatedly. The repeat interval of such a carousel is based on the bit rate allocated to the service and the volume of data in the service. All forms of digital information can be multiplexed into such a digital data-casting environment. Not all data services require low latency. Some services, however, will require very low latency, such as downloads of small software or data snippets that run in set-top boxes (applets) or other end-user terminals. These can be accommodated in a large carousel that can conceivably contain vast amounts of data. This could be a way to distribute large chunks of operating software updates. Currently this mechanism is used in digital television services to update the operating system of TV set-top boxes over the air. The customer doesn’t even notice that his or her box has received a new software version.

Typical video resolutions required are CCIR-601 (Standard Definition resolution) but may migrate to High Definition resolution in the near future. Audio is currently stereo but may migrate toward a multi-channel format (5.1 surround sound), where the expected quality is comparable to or better than that of an audio CD.

Virtual production techniques are becoming increasingly popular in TV production studios. These techniques are based on the well-known chroma-key method that has been in use for many years. Chroma-keying means that the actors perform in front of a colored background (usually blue or green), and a key signal is derived from a chroma-key unit, indicating which parts of the image contain the actors. A mixer then overlays the actors on another virtual background image, using the key signal to control a “soft” switch between foreground and background.

With traditional chroma-keying, the studio camera cannot be moved, since the registration between actors and background would be lost. One of the new features of virtual production is that the camera can be moved, because its position and orientation are measured, and the background image is adjusted to keep the correct registration. This is usually achieved by rendering the background on a graphics computer, and updating the position of the virtual camera to match that of the studio camera. In situations where the camera is allowed to pan, tilt, and zoom, but not translate, the background image can be stored as a 2D image, which is transformed to match the current camera angle. If the camera is allowed to translate, true 3D models of all virtual set elements are generally required. Virtual objects can also be inserted in front of the live action, by generating a key signal for each object that forces the mixer to switch to the virtual background signal within the object, regardless of the presence of the actor’s key signal. This can be achieved by giving every object (including the actors) a depth value, which determines the way in which objects are overlaid.

MPEG-4 offers the possibility of broadcasting programs produced in this way, using object-based techniques. The final image composition then takes place in the decoder rather than in the studio mixer. This is likely to offer advantages both in coding efficiency and increased functionality (such as user interaction and stereoscopic viewing of the scene).

It is also important to ensure that the audio signal presents the same spatial “story” as the pictures that go with it. As an actor who is talking moves within a space, the proportion of direct-to-reverberant voice sound changes and indeed even the spectral content of the sound may change. This is normally accomplished manually, but in the context of MPEG-4, where spatial information may be rendered at the consumer’s end of the chain, it is important to generate this feature automatically, using tools such as those provided in structured audio effects (SAFX).

Virtual actors are sometimes incorporated into virtual productions. These are 3D models of people, usually animated by motion data captured from a real actor. The face and body animation features of MPEG-4 could be used here, as long as they offer sufficient control over the appearance of the virtual actor to satisfy the program producer.

Integrated Service Digital Broadcast (ISDB) is a concept for constructing a complete digital broadcasting system, which offers a great variety of services with high-spectrum efficiency, flexibility, and extendibility. ISDB provides not only existing basic broadcasting services such as SDTV and HDTV, but also new services, such as multimedia TV, the TV newspaper (multimedia information services), and two-way information services. It enables viewers to make better use of television and offers services with multiple functions. It is controlled by a CPU, thus enabling the viewer to enjoy programs through personal filters and an intelligent agent for broadcasting. Figure 5.4 shows an example of the program menu the user can see just after turning on the switch.

Figure 5.4 Example of the program menu of a terminal for receiving ISDB services.

5.3.2. MPEG-4 and Media Asset Management

Since we learned how to move pictures, there has been a constant and continuously growing necessity to archive content. In today’s technology environment, where digital technology rules, broadcasters are faced with the requirement of optimizing efficiency, increasing revenues and market share, and reducing cost, while at the same time improving quality and, of course, staying competitive.

Businesses must be managed, but so must content! Now that we know a little bit about creation and delivery, let us look at another interesting and crucial aspect of the digital media arena—Content Management, which we will call Media Asset Management; after all, contents are revenue-generating assets. MPEG-4 directly impacts the production, archiving, and programming chain, where for example it permits broadcasters to browse through video archives with ease and speed, which is of paramount importance. There are browsing systems available on the market, where Standard Definition MPEG-2 to MPEG-4 smart file transcoding is used to produce a frame-accurate proxy and/or browse a version of the high-quality MPEG-2 archive. The bit rates utilized in these kind of scenarios are usually 300 Kbps and 800 Kbps. The displayed video is of high quality, particularly when the MPEG-4 Visual Advanced Simple Profile is used. From a business perspective, this is very attractive since, in essence, the content is “purposed” in two directions: storage of a high-quality video, and archiving. The latter is particularly attractive given that there is an extremely large amount of content available, which is growing every day, and content owners must utilize the principles of archiving not only for their own purposes, but as a contemporary content-selling tool, which clients can access through a number of different means.

Another benefit of an MPEG-4 media asset management system is that not only the broadcaster but also the viewers can utilize the system to locate, preview (at a very low bit rate), and choose content for viewing (in an appropriate bit rate and quality that fits the user’s needs and environment).

It is certainly fair to say that the broadcast industry is constantly evolving, which requires market participants to deploy flexible, standards-based environments and management systems that can be easily accessed both internally and externally, taking into account the fact that content, and the way it is managed and presented, will always rule and determine the overall value proposition. MPEG-4 can deliver this important piece in the value chain and is on the way to establishing itself not only as an exciting professional add-on, but as the new de facto standard for these applications.

Let’s now look at elements of content-based storage and retrieval as part of media asset management. The term “content-based” refers to systems whose objective is to provide access to content, or rather archived/stored content, based on attributes associated with the video or audio content. These attributes may be keywords (often a semantic description) but they can also be numerical attributes. Generally, the purpose of such libraries is to assist in the management of large collections of digital video assets. As an example, many of these systems rely on temporal and/or spatial segmentation of an audio-visual stream. Although the segmentation process itself is not part of MPEG standardization, the provision of mechanisms to efficiently access such temporal or spatial segments is within the scope of MPEG-4. The result is that in addition to accessing content using traditional methods such as fast-forwarding to a particular temporal value, it is also very beneficial to be able to browse through these segments based on associated textual and numeric attributes.

Many content-based media asset management applications will need to be able to speedily compare attributes associated with the assets stored in the database with a representative set of attributes defining a query. Often, in the case of browsing, the query is defined by one or more examples and their associated attributes. When it comes to rather large collections, the way to make this exercise feasible is to make sure the method does not involve the decoding of the entire audio/video stream. A fundamental component of such systems for this purpose is the use of a “decision support representative” (DSR). A DSR can be used to represent a large audio-visual asset in a very condensed form, allowing the user to decide on the appropriateness of an asset for his or her purposes. The exact nature of the DSR can vary from application to application (for example, single representative frames, or a small mosaic of frames, icons, etc.); however, it is desirable to have support for the efficient storage and access to the DSRs. What is important to realize is that MPEG-4 is the enabler for media asset management, providing the ability to browse through and/or access content in order to identify and choose specific content, for specific purposes at any point in time. MPEG-4 is not intended to be involved in the delivery of assets, since these can be on a variety of media, such as on film or tape.

5.3.3. Studio and Television Post-Production

When it comes to post-production, before source material ends up in its final form—for example, as a television program—it passes through various processing steps, which involve editing and multiple copying from one storage medium to another.

The ease of creating complex effects, such as color separation, overlaying (or chroma-keying), cross-fading, cutting-and-pasting, and a variety of other digital video effects, is increasing because of the transparency of digital recording operations and the automation using edit controllers as well as offline edit systems. The need for versions in different languages favors separation of program elements and requires separate storage of all intermediate versions. Regional variations of a program may also have similar requirements.

In reality, a pretty broad spectrum of operations, as the ones described above, is needed to produce the final product and caption it. This may involve several passes, for technical reasons, and may therefore require the intermediate storage of results to an appropriate medium (typically tape). It is entirely possible that as many as 10 to 20 passes may be needed on a single project, with 4 or 5 versions retained.

For TV production and post-production, the advantage of choosing a standard offering the possibility of content-based video manipulation is attractive for the following reasons:

In post-production, it is not uncommon that each pass is made on a different, separated platform, each of which has quite different hardware and software features (CPU power, 3D-CG accelerator, real-time grabber, mixing capabilities, etc.). This especially applies to 3D-CG workstations with both fast rendering capability and fast mixers for online 2D layer composition. This clearly requires interoperable exchange formats supporting the import and export of extended video as well as scripting data. In addition, a common scripting language to use when supervising the different “post-production islands” from a central editing suite is strongly desired.

Continuing along this road, we come to the concept of content-driven manipulation of coded video in the studio. Although the ambition is to keep each video in its compressed form for as long as possible, some applications may require transcoding, which means that the compression used should be very efficient in this respect.

Thanks to the fact that digital uncompressed data can be copied straight (dubbing), it is possible to reproduce distortion-free video data as many times as required. Where lossy compression is utilized, however, the accumulation of coding errors limits the number of times that encoding/decoding operations can be undertaken before the quality becomes visually unacceptable.

An additional difficulty arises if further processing is needed between the aforementioned encode/decode operations. Very simple effects can be applied to simulate intergenerational processing, such as small spatial/temporal shifts and fades. However, when these kind of effects are combined with traditional compression algorithms at television transmission rates (i.e., MPEG-2 MP@[email protected] Mbit/s), the output video quickly becomes unusable (i.e., after 3 to 4 generations).

5.3.4. Digital TV

By using digital data, the viewing experience of consumers can be improved through more interactivity, for example:

MPEG-4 is designed to deliver the above-mentioned functionalities. Typically, these sorts of functions will be available through set-top boxes, which means that they can be classified as MPEG-4 applications. As you can see, there are far more interesting uses for set-top boxes than simply being a receiving device for moving pictures. In fact, there is a very strong tendency that STBs develop themselves into multimedia gateways; not only is the receiving of content on the agenda, but the storage of content, whether that may be temporary or permanent, is becoming prevalent. In addition, if one considers the various aspects of conditional access and the related possibilities, which we do not want to list here, then it becomes pretty much apparent what sort of impact the existence, evolution and progress of digital media technology can have right in the living rooms of millions of users. Surely, let’s not get overly excited and assume this all happens overnight. The fact remains that there are millions of STBs deployed and operators do not tend to simply switch everybody over and hand out new STBs. It will take sometime and it is a specific transition and migration path to be followed, which is not only characterized by technology considerations, but also very much so by corporate and commercial considerations. One must very soberly distinguish between the availability of technology and its deployment—as in many things in life, it is all about timing, but the great news is that we can lean back and look forward of taking advantage of these technologies in the not-too-distant future.

As a rule of thumb, a set-top box will receive MPEG-2 transport streams carrying MPEG-2 video and audio. The decoded audio and video will then be rendered on a set of speakers and a screen, respectively, and can be described as the standard digital TV experience. In an MPEG-4 scenario, the MPEG-2 transport stream can also carry MPEG-4 streams, which have been specified in the standard to enable such an operation. In the conventional setup, the de-multiplexer would simply discard this information. This makes the bit stream carrying the additional MPEG-4 data fully backward compatible. On the other hand, on an MPEG-4-enabled set-top box, the MPEG-4 stream will be recognized by the receiver as such, and passed on to the appropriate MPEG-4 decoder. The MPEG-2 video and audio will be rendered through the MPEG-4 rendering mechanism, together with the additionally received MPEG-4 objects.

Use Case—Home-Shopping: There are many countries, where home-shopping channels are successfully in operation, giving the consumer at home the opportunity to purchase all sorts of products. This is not a talking-heads environment, but an object-rich scenery, which is perfectly suited to MPEG-4. For example, besides the moderator, there is a background (mainly a picture), there are logos, information about the products (text), and of course the product itself (as a still picture from a variety of perspectives).

Orders are typically placed via the telephone but they can also be placed through the configuration of a back channel (which may well be a telephone line), which makes the ordering more user-friendly, simply by interacting with the TV screen using the remote control. This makes shopping easier and more impulse driven, and therefore it becomes an interesting feature for shopping channel operators. This can all be achieved through sending a small MPEG-J application program to check if a back channel is configured, and if so, product-ordering information can be taken with special pop-up menus, and, via remote control, information about the number of items, the item specifics (size, color, options, etc.), and credit card number (note that this will involve security issues) can be taken. The back channel is opened (possibly by dialing a specific phone number), the order information is sent, and the back channel is then closed.

Use Case—Conditional Access: Subscribing to specific channels or pay-per-view services has been possible for quite a while now with analog set-top boxes. Using the same technology as described above, movies can be ordered and paid for. Conditional access can be set up to require a password, which will be verified via a back channel or compared with information stored on the set-top box hardware. Alternatively, a specially installed device like Smart Cards (used for access verification and/or decryption) can be utilized to control access.

5.3.5. Video Over IP

One buzzword that has come out of the woodwork recently is “triple play” in the telecoms industry, which can be translated, in essence, to mean the delivery of broadband Internet access, voice transmission/telephony, and video over IP, the last of which is a new addition to the telco service spectrum. However, the “video” part of this equation is becoming more and more of a necessity in order to stay competitive and to keep their subscribers. Overall, it is the same old song of attracting subscribers, keeping them, in turning them into revenue generators. Let’s be clear on the dimensions of this—we are not only talking about big telcos, but telcos of all sizes that are embracing this new business and service-offering initiative.

The fact is, the infrastructure is in place and the underlying technology is as well, so many market participants are deploying these kinds of services, bringing motion pictures directly to your PCs, laptops, or mobile devices, utilizing your broadband connection, over wireless—on demand, scheduled … you name it.

It is a great opportunity for telcos to get involved in this field. On one hand, they have been forced to do so to some extent by deregulations in their original field of activity, but on the other hand, they clearly understand that consumers are demanding even more content, everywhere and all the time, which gives them a real chance to deploy and sell more services. The good news of course for the telcos is that this all goes down a tiny cable, which is already present in their subscribers’ homes. Plus, from a consumer’s viewpoint, it is simple to get and manage—three services out of one shop!

So how does this actually work and where does MPEG-4 come into play? The short version of the technical side is that there is a so-called operator head-end (the point where content is “injected”), a regional center (a kind of distribution point not too far from the subscriber), and then there is what is called “the last mile” (i.e., the distance to the actual subscriber). MPEG-4, and especially MPEG-4 AVC, can be perfectly utilized to encode/transcode the respective content at the head end, which is then transmitted over the backbone via the regional center to the subscriber, where the content is decoded on a set-top box or on the PC. There are already quite a number of sophisticated MPEG-4 encoder and streaming products available on the market, which will become even more efficient, functional, and versatile than they are today, once MPEG-4 AVC is being adopted and introduced in the market place.

As we learned earlier, MPEG-4 can be utilized here in a very efficient way, since it works for ostensibly all bit rates—whether it is for a standard broadband connection or a satellite connection.

The benefit of using MPEG-4 in this triple-play scenario—and it is anticipated that video over DSL will be one of the key drivers for MPEG-4 over the next 12 months—is that it is easy to integrate into the existing infrastructure and that it offers cross-media exploitation by making content available for a variety of different uses. Without a shadow of a doubt, video over IP is a valuable alternative to the traditional ways of broadcasting, since it gives even more consumers the opportunity to receive more content, more services, and more value for their money.

5.3.6. Mobile Communications Industry

We have already briefly touched upon the significant involvement of MPEG-4 in the mobile industry, mainly through the adoption as an optional video and audio codec with the 3GPP and 3GPP2 standards. Without going much deeper into these standards, it is important to understand the basic definitions in order to fully appreciate the impact of MPEG-4 within the mobile industry.

First, let us define yet another buzzword in the technology land-scape—3G. It stands for third generation and is in fact a generic industry term describing the high-speed delivery of data over mobile networks. To put this into context, it all started with the first generation, analog, leading into 2G and 2.5G (GSM, GPRS, and EDGE, respectively), and now 3G. 3G, which is currently being rolled out worldwide, enables the sending and receiving of bandwidth-intensive data, such as video (streamed/on demand), high-quality audio, and all sorts of other rich multimedia data.

3GPP and 3GPP2 have been specifically designed for this purpose, enabling the creation, delivery, and playback of multimedia content within a unified framework and delivered over high-speed mobile connections to new, state-of-the-art mobile devices. While MPEG-4 has proven to work very well in the Internet arena, the technologists behind 3GPP decided to take advantage of this knowledge and have based their standard on MPEG-4. Therefore, as in MPEG-4, in 3GPP you can mix text, video, and audio in a single file and deliver it over your chosen network.

It should be mentioned at this point that MPEG-4 took a lot of input and basics from Apple Quicktime, which has been extremely successful over the course of many years. In fact, if you look at the overall impact of Quicktime, and consider its support for MPEG-4, you do not need to be a rocket scientist to work out how hundreds of existing applications can easily adopt and utilize MPEG-4, and indeed 3GPP. The availability is guaranteed; the various uses are commonly recognized, and content owners are able to create their content easily within their “usual” environment but in a new, more contemporary format to deliver said content via new, contemporary delivery channels. With millions of users currently, Quicktime is a prime example of MPEG-4 in action. The number is growing weekly, and with more content being available via all manner of transmission tools, MPEG-4 has found a prominent position in the mobile industry.

Of course, in the mobile industry it is of paramount importance to ensure interoperability, and virtually all the leading infrastructure companies providing content owners with the “engine room” for their services deliver solutions that work with devices and software available in the marketplace. For this reason, all the big names in mobile handsets, whether from Japan, the United States, or Europe, have been spending significant resources in order to remain or become interoperable.

But what will drive MPEG-4, and later MPEG-4 AVC, in the mobile markets? Here, a very interesting picture emerges in which content is not only created by professional content providers, but by the actual users themselves. Mainly this will entail short video clips with talking heads as in news, sports, music videos, and so on, but it will also include privately created videos from the last holiday, etc. For the operators, that means that they need to be able to serve both the “traditional” networks but also mobile networks. In fact, operators do like this scenario, from the simple point of view that it presents them with a sort of a Greenfield opportunity to get involved in a market that is currently being developed and expected to properly take off soon. Operators must be supported in terms of the content creation in order to provide a service with value additions to deliver more subscribers, or enable subscribers to upgrade. There are visible trends in the industry for this, with a number of 3G services being launched and quite a large number of video-enabled mobile handsets becoming available.

There are of course a number of obstacles, which are addressed through the utilization of MPEG-4. If one subscriber likes to send a video to a friend, utilizing her video-enabled handset and her operator’s network, and this friend happens to not have a video-enabled handset, the subscriber can email the video to her friend (video mail), who can then watch the respective movie using MPEG-4-compliant player software.

5.3.7. Mobile Multimedia

The mobile industry is obviously not only characterized by mobile telephones, but also includes mobile computing—the use of a portable computer capable of wireless communication, which is not only used for local, stand-alone data processing, but also for wireless communication situations of a mobile user in motion.

In a typical mobile computing scenario, a mobile user communicates with a remote computer system using, for example, a laptop/notebook or a Personal Digital Assistant (PDA) via wireless communication links, which follow the same principles in terms of the delivery as described earlier.

Multimedia applications in the mobile sector are faced with challenges that do not exist in this form for desktop applications. The reason for this is that mobile technologies and applications must deal with less computational power, low bandwidths, and at times the poor reliability of the chosen transmission mechanism.

In addition to requiring a high compression performance, the ability to adapt is very important for mobile applications. The reasons for this are as follows:

MPEG-4 is very appropriate for mobile multimedia applications since, because of its ability to accommodate very low bit-rate applications. More precisely and in summary,

5.4. MPEG-4 in Business

5.4.1. Pricing

If one considers the current pricing for MPEG-4 products, it is evident that some market participants on the vendor side apparently believe they are in a business that sells power plants in volume, or believe that MPEG-4 products need to be priced for early adopters, who are sometimes willing to pay more than the average customer would. Many of these companies have been taking on a considerable amount of investment, which in turn, being faced with significant monthly costs for their operation per month, seems to drive prices up, never mind their expectations. In economic terms, they need to charge and present these sorts of prices to sustain their operation and deliver some form of return back to their investors. Of course, this is something that is apparent in any industry, however, and as always, timing is the important factor. Usually what happens is that when the market becomes more mature and the possible volumes are higher, prices will automatically come down. Obviously, growing competition contributes to that too. The bad news in this regard is that there may be more than a few corporate casualties along the way, which to some extent is normal in any industry that is experiencing change or consolidation in some shape or form. Critics may take this as an opportunity to judge MPEG-4 as failing, which is of course their right; however it is rather misguided to draw conclusions about the success or failure of a technology that is just now getting in gear. Certainly, this “gearing up” has taken some time. In terms of “age,” MPEG-4 Video (Part 2), which was finalized in 2000, may sound like it’s “old” considering the speed of technological progress these days. But look at it instead from the perspective of maturity, and the ability to improve over time. It can take a number of years for a technology to “mature.” Just like MPEG-2, which has only now reached its limits, the quality of MPEG-4 is slowly improving over time. And, of course, with MPEG-4 AVC having been finalized in early 2003, the story of developing and making available the best, most flexible and scalable compression technology continues.

5.4.2. Specialization

With specialization, and staying focused on which market segments to serve, the likelihood of companies succeeding is much higher than it would be otherwise. The MPEG-4 market requires a healthy, coherent, competitive, and reasonable pricing for products in order to function well and deliver commercial satisfaction.

The intent here is to illustrate how the market could function possibly much better with the help of MPEG-4, generating more revenues for all concerned. The great advantage of MPEG-4 in contributing a “revenue-increasing element” to the business of content creators is the fact that consumers’ requirements and expectations grow constantly. There is the never-ending demand for personal, anticipated, and relevant information.

Let us consider the evolution of media. Going back to the emergence of the TV set, besides the fact that the consumer could view only one program in the beginning, he or she needed to get out of a seat to interact with his/her TV—switch it on and off, turn the volume up and down, etc. The first improvement to this situation (besides more programs to choose from) came with the introduction, or rather the innovation, of remote control—the consumer could now interact from a much more convenient position than before. In the years that followed, there were video players/recorders that enabled consumers to record and watch content whenever desired. The evolution of this interaction has gone yet another step with Internet and DVD technology bringing information to consumers. But consumers want more, continuously. New technologies have always made it possible to go a step further; the same applies to MPEG-4. While MPEG-2, the currently dominant format in the digital media arena, has brought a lot of new functionalities to the consumer, e.g., DVD as mentioned above, it is impossible to create truely interactive content with it and deploy it on DVDs and a variety of interactive broadcast networks. This function, which MPEG-4 offers, is extremely attractive to content creators since it enables them to deliver the type of content required and demanded by users. So MPEG-4 makes money primarily because of technological superiority.

This same principle can then be deployed in another major growth market for MPEG-4. After having spent billions on obtaining the respective licenses from governments, the operators of 3G/UMTS networks are also taking advantage of MPEG-4 (or at least have the freedom of choice to do so through the 3GPP specification, as we learned earlier, that contains either MPEG-4 or H.263), as have the manufacturers of appliances supplying the relevant gadgets, since, the demand by customers has risen dramatically, sometimes even exponentially, in recent years. Where consumers at first were happy to speak to one another on a phone they could carry in their pocket (OK, initially it was more like a shoe box), soon they were able to send text messages, followed by still pictures, and now they can even send each other videos over their mobile phones. This emerging market, with a number of networks in place and others being launched in the months to come, particularly in Europe, will, by its very nature, deliver a considerable revenue to the MPEG-4 community—particularly to hardware manufacturers but ultimately to all participants in the production chain if one considers the growth potential for “home-cooked” videos. The “personal creation market” has potentially the biggest growth potential among all perceived opportunities including news, sports, music, etc.

5.4.3. How to Make Money

The “how to make money” aspect comes into play through the simplistic and everlasting principle of satisfying demand. However, a demand can only be satisfied if the cost of doing so is within reason, and above all within reach, of the consumers’ and customers’ wallets, which leads us back to where we were a few lines earlier. In other words, MPEG-4 makes money when vendors provide content creators with the opportunity to capitalize upon the technical innovation MPEG-4 brings along by offering multiple narrowband as well as broadband platforms like the Internet, digital TV, and wireless, for the distribution of content.

In addition, as we have learned above, MPEG-4 and especially MPEG-4 AVC vest a significant bit rate saving in itself. Bit rate savings can be immediately equated to monetary savings. Also, one has to look at MPEG standards as such, and it is clearly visible that they were structured from day one with economics in mind— simple decoders, complicated encoders. Or, in other words—cheap decoders, expensive encoders. Certainly, this analogy is slightly exaggerated, but at the core remains a fact—MPEG standards are made to make money. Also, it created a whole new industry from which no one would have probably expected this sort of performance in terms of quality improvement.

So clearly, as in MPEG-2, we will be seeing the same sort of curve as in Figure 5.5 happening over the next few years with coding efficiency improving. Technology vendors and certainly content owners as well, will benefit from the fact that more bandwidth will be available, while offering new products requiring the use of less bandwidth—a very nice situation to be in and to take advantage of. MPEG-4 as a proven technology delivers much better economics and is a feasible facilitator for content owners to achieve their objective and provide big audiences with great quality content and to generate continuous revenues in doing so.

Figure 5.5 Improvement of MPEG–2 video performance over time from 1995 to 2004, as seen by Harmonic. The graph shows the bit rate that is necessary to offer broadcast quality television services. This is probably the most popular graph in the industry to illustrate the development and progress in performance terms of MPEG standards.

5.4.4. Key Factors for Migration

The many uses of MPEG-4 could fill hundreds of pages, if we were to cover all possible business models. However, there are certain key factors evident that make a migration to or introduction of state-of-the-art compression technology, i.e., MPEG-4, viable for your business, some of which we have already described above.

The critical reader may now comment that all these wonderful things that we have been describing over the last few pages can also be provided through proprietary technologies It goes without saying that there is excellent multimedia technology being offered; however, as we mentioned before, the use of a single technology provider/vendor comes with a lot of disadvantageous elements for market participants. In addition, there are a few more considerations to be taken into account when looking at the past few years of digital media. For example, the open standard MPEG-2 is dominating the broadcast industry at the moment, and the likelihood of a proprietary format taking over is, frankly, extremely remote. Plus, MPEG-4 can offer interactivity, which will be playing a major role in years to come. So, let us look further into the various potential uses of MPEG-4 and start with some thoughts about compression technologies in the MPEG-2 arena.

One of the most discussed topics today is obviously which compression technology will become a true successor to MPEG-2. First, the departure of MPEG-2 is certainly not imminent. Second, with MPEG-2 being a true open standard, it can, from this perspective, only be succeeded by another true open standard. The good news is that this successor, MPEG-4 AVC, exists, and not as some kind of a futuristic, conceptual, and revolutionary idea, trying to capture market attention while being light-years away from reality, but is a highly sophisticated compression standard, available today. MPEG-4 AVC advances many techniques utilized in MPEG-2 and adds new tools that dramatically improve coding efficiencies with up to 40% to 50% gains over today’s state-of the-art MPEG-2.

In business terms, all this can be characterized quite simply. If your existing business was technically a success while using MPEG-2, the likelihood of this continuing with MPEG-4 is a logical and reasonable conclusion. The very fact that you have been relying on MPEG-2 in the past makes a migration over to MPEG-4 easier, in large part because the same organization is behind it.

There are certainly implications to be considered here, since technology alone will never ever be the sole factor that makes your business a success. Technology is certainly a facilitator enabling businesses to be successful; however it will always boil down to your proposition, unique selling points, and overall offering. Just a few paragraphs earlier, we discussed some of the money-making aspects. Making money can obviously also be achieved through saving money. If you consider the gains in coding efficiency mentioned above, you will realize that this percentage can also be illustrated as your potential savings if the respective costs are taken into account. We will look into this interesting aspect in the next section, where we will deal with costs.

There is of course also Microsoft’s Windows Media 9 (WM9) compression technology, which is without a doubt a very advanced, multipurpose technology. In addition, in a move toward some of the benefits of standardization, the Windows Media 9 specification was submitted to SMPTE for ratification in September 2003. Certainly, Windows Media technology is extremely well recognized, is commonly available, and is now moving more into the broadcast arena with (a) the decoupling of the transport stream and (b) the new Advanced Profile with interlace support and other essential elements required for professional-grade broadcast. (The previous WM9 Main Profile was PC-centric and clearly not targeting solutions for the broadcast community.)

The discerning critic will ask, where does all this leave an open standard like MPEG-4. The simple answer is in an interesting competition. In addition, it should be noted that currently, as far as the broadcast world in early 2004 is concerned, MPEG-4 AVC is ahead of Windows Media 9, since the aforementioned Advanced Profile has not yet been released.

Customers in the broadcast industry will soon have the choice between these two technologies, which in the end is good for competition and perhaps allows broadcasters to pursue slightly diverging business models. So there will be natural competition within MPEG-4 as to which vendor can actually offer the best implementation technically or the best acquisition model, paired with the competition on the overall technology level, i.e., proprietary vs standard-based technology. This attitude is already clearly demonstrated by a number of vendors, who, while being supporters of MPEG-4, have also now added Windows Media 9 support to some of their products. You may ask yourself, how is this coherently possible and viable to actually support two different technologies. Well, if one considers the current situation where we also have a major element of coexistence of various formats serving the industry, this will, to some extent, continue. This may sound like a contradiction, since MPEG-4 is being advocated throughout this book, but then consider the size of the market we are talking about. The multimedia, digital market is a multi-billion dollar industry, and therefore there are many corporate considerations, cross-company interests, and a variety of other factors that must be thought of

The name of the game is not for a technology vendor to decide what its customers want, but to offer them, and let them choose from, a variety of “flavors.” Without a doubt, there is room for MPEG-4 and Windows Media 9 (Advanced Profile) to coexist in the huge digital media marketplace and ecosystem, which can therefore deliver the key elements of a demanding economic environment: advanced offerings, choice, and competition. The advantages and benefits of this are quite evident with respect to MPEG-4. But no matter how much one believes in MPEG-4, it has to be acknowledged that there are alternatives around, and it will be quite interesting to see the industry move forward.

In summary, the business aspects, especially regarding cross-media exploitation, are mainly tied to the creation element where an automated flow of encoding procedures will deliver a cost-effective solution. But there is much more to it, which leads us to the exciting opportunities MPEG-4 opens up in a variety of sectors. We cannot consider all of them in this book, but let us take a closer look at some of the issues related to content.

5.4.5. The Chicken and the Egg—How to Roll Out Coding Technology

We all know about the chicken-and-egg scenario, which has kept mankind pondering for many centuries. However, no matter how you look at it, all parties have their “logical” explanations, which, strangely enough, all seem to make sense. The trouble is that no one solution has emerged and been agreed upon, so mankind will no doubt have this interesting topic to talk about for years to come.

In the media world, we are faced with a similar kind of question. What was there first, the content or the players? First we have to consider what is actually content. “Content” is effectively an ongoing fully flexible event, whether in the past, present, or future, and whether it is real or fictional. A “player” is the medium through which content is shown and presented to an audience. In order to consider this sort of question properly, we need to go back in time.

In the early days, content was painted/written on cave walls, which, research has shown, was for information and entertainment purposes (please do not get overly excited and mark this point in time as the birth of pre-historic television). Transmission at that time was almost exclusively through word-of-mouth within communities, since at that time everything was pretty micro-cosmic. But already at that early stage, we are not able to answer coherently whether in fact the content was there first or the player.

We could play this little game through the thousands of years of mankind’s existence and would not find an answer. However, if we start from this pre-historic beginning of content creation and play forward to today’s media landscape, there are a few considerations about content and players, at which we want to look a bit more closer now.

First, let’s step back into the present.

When the Internet became more popular, it was logical that pictures would start to move on the screen that everybody until then only thought of as a projector of numbers, letters and still pictures. Quite boring stuff, if we compare this to what is possible in today’s technology landscape. All of a sudden, it seemed, the world community possessed the ability not only to watch TV but also videos, clips, etc., on their PC screens. Yet again, content had found a way of presenting itself.

The problem now became a pragmatic one (i.e., with broadband in its very early stages and nowhere near being deployed and the overall concept of transmitting moving pictures over IP networks was also still a dream). But the very fact of getting a PC monitor to be an acceptable medium for video content was pretty progressive. Critics may now say that, overall, the whole idea of watching TV on a PC screen is nonsense, but here are a few things to consider. Going back to our little timeline, users initially had to live with video CDs or the like until the Internet was able to offer an on-demand scenario. “Video-on-demand” was a pretty widely used buzzword at the time for a variety of reasons. For technicians, it was a logical consequence of their progressive technology developments, and for the business development people, it was a nice key word for securing finance and marketing appeal. With content now available online, the player machinery really gained momentum and began to roll. The big players on the circuit, i.e., Apple, Microsoft, and Real Networks, started to flood the market with very good products, and the market has done quite well since then, offering consumers a wide variety of choices.

Standards were not really an issue at the time, since in the early days of content, it did not really matter, as everybody was “doing their own thing.” In today’s content world, it is absolutely imperative to have some sort of focus as to which formats can or should be used. The standards issue came to confront broadcasters as well, when, after scratching their heads for a while, they decided to utilize the standards PAL and NTSC. To give you an idea of what might have happened if that had not been the case, in theory, it would have meant that users would have been required to have a different TV set for every different format a broadcaster was using to air respective programs—quite an irritating prospect if one considers the amount of channels available today.

The same situation is apparent with the players when it comes to Internet content, with one slight difference: Some of the companies offering players team up directly with content owners and providers, effectively blocking out users, who may not have that particular player. A standard would help a lot here. Having said that, it should be noted that two of the “big three” player providers (Apple and Real Networks) do support MPEG-4 in one way or another.

5.5. MPEG-4 and the Costs

Let’s have a closer look at the cost of transmitting data by jumping into the IT money jungle. In order to make this exercise a little easier and illustrative, let us consider the cost in an Internet streaming environment at the server end.

Fact: Broadband adoption will dramatically increase Internet server costs. There will be a tremendous cost impact on video streaming Internet servers as broadband use grows. Much has been documented on the topic of consumer adoption of broadband Internet connections and the resulting requirement for increased video content. However, there has been no discussion of the impact of this paradigm shift on the cost to the Internet servers streaming the video. The increase in bandwidth utilization will require better and better compression techniques in order to meet the expectations of users and the cost requirements for bandwidth.

Internet Server Costs Impacted by Video: Every video played by a viewer must have a server streaming the video to it. The server must pay for every bit of the video that is streamed to a viewer. Streaming digital video requires a significant amount of bandwidth. The number of concurrent video streams from a server determines the amount of bandwidth consumed by the server.

There are two general types of digital video streams. The first, narrowband, is designed for viewers connected via standard phone-based modems—typically 27 to 50 kilobits per second (Kbps). The second, for viewers using a broadband connection, such as DSL or a cable modem, is typically 100 to 300 Kbps (and projected by futurists to someday be as high as 10,000 Kbps).

The calculations that follow will show that bandwidth is the single highest cost component for video servers. Ignoring dedicated line fees and other peripheral costs, currently 1 Megabit per second (1 Mbps = 1024 Kbps) costs approximately $18,000 per year. This means that top video streaming sites may have to pay as much as $24 million per year for server bandwidth, which is a considerable expense that somehow needs to be earned back.

Current Site Bandwidth Costs: Broadband users place a much larger burden on video servers than narrowband users. As demonstrated in the following calculations, even a simple site streaming only 100 concurrent users can expect a hundredfold cost increase as users convert to higher bandwidth streams.

Table 5.1 Costs for narrow and broadband connections for 100 concurrent streams

This increase in cost will be outside the budget for many companies who wish to stream video and action will need to be taken in order to stay competitive and maintain a viable business. Two of these required actions are

It is reasonable to expect that managers will reduce their costs as far as possible while retaining their required quality. Estimates using current compression techniques result in acceptable streaming rates of approximately 300 Kbps for base commercial usage and as high as 3Mbps for entertainment. As shown in the previous example, even the minimal 300 Kbps rate will not be within most budgets.

Increase in Broadband Internet Users Will Increase Demand For Video: Many marketing strategy and analyst firms track user Internet profiles. Combining their estimates, it is reasonable to expect that by 2004, more than 25% of American households (more than 16 million) will connect to the Internet via a broadband connection. In addition, the vast majority of businesses will connect to the Internet via broadband (this is estimated to be about 3 times the household access). Not surprisingly, surveys of users with access to broadband show that their expectation for more and better graphic and video content matches their added investment in connection speed. This means that in addition to the increases in the cost per user, the number of video users will increase. If the example is extended to grow from 100 users to 1000 simultaneous video viewers, the picture becomes even more desperate in terms of the costs involved for the provider.

Table 5.2 Server bandwidth costs for 1000 simultaneous video streams

Bandwidth Costs May Drop, But Increased Usage Will Offset: It is likely that bandwidth costs will drop over time. How fast? Traditionally, they have not dropped nearly as fast as semiconductor costs (which follow “Moore’s Law”). But even assuming that bandwidth breaks with traditional trends and starts falling at the dramatic rates of Moore’s Law, a quick calculation shows that because of the increase in Internet usage, even this aggressive cost drop will not solve the problem of server costs.

Current projections by the major marketing firms state that broadband utilization in U.S. households will potentially grow by around 80% per year over the next five years. As shown in the table below, this growth outweighs even a generous projected reduction in bandwidth costs. In fact, the huge increase in users will continue to increase the costing pressure on sites delivering streaming video.

Table 5.3 Increases in Internet usage will more than offset drop in bandwidth costs

Compression Improvements Will Be Very Valuable: What is the value of compression improvement in this scenario? A quick calculation shows that the value is immense. In 2001, the example above shows a site with approximately 1,800 concurrent broadband users generating about 16.5 Terabits of streamed video at a cost of just under $7 million. Using these figures, the table below shows the dramatic cost savings that can be realized by better compression.

The example above is for a single site. Assuming 1000 of these large sites streaming video in 2001, these sites would have a combined bandwidth cost of $7 billion per year. An advanced compression that reduces this bandwidth by only 10% could save these 1000 customers $700 million per year! This figure already sounds extremely good, but if one applies even a conservative estimate as to how much MPEG-4, and specifically MPEG-4 AVC, delivers as bit rate savings compared to MPEG-2—say 30%—the resulting monetary saving would be a staggering USD 2.1 billion!

Table 5.4 Improved compression reduces server bandwidth cost

There are more than 15 million commercial sites on the Internet today, so it is easy to project that there will be more than 1000 sites serving video in the coming years. Some will be large as in the example above, and many will be small, but it is clear that broadband will make bandwidth costs a concern for all.

Analysis Assumptions: For the sake of simplifying the examples, the following assumptions were followed: There is no accurate count of the current video servers in the United States or worldwide. It is cumbersome to do comparisons using bandwidth due to arguments of appropriate utilization (during peak hours this might be 80% while off hours are 10%), proximity to the backbone, etc. As such, we have converted bandwidth to a per-Gigabit download price using current bandwidth costs and assuming 100% utilization (our estimate is $0.63 per Gigabit). This defines a solid lowest cost data point for transmission. Calculations are focused on bandwidth used by broadband users for two reasons. First, there are accurate projections for the number of such users in the United States, and their interest in video has been documented. Second, the impact of narrowband streams is negligible when compared to broadband.

5.6. MPEG-4 Licensing

Most certainly, the next argument from MPEG-4 critics would be the acquisition cost and licensing aspects. Well, it’s true, nothing comes for free; proprietary formats also have their price tag. In addition, one must clearly and categorically differentiate between patent licensing and technology licensing.

In an open standard, a number of patents are utilized to achieve a particular result. But the technologies or procedures underlying these patents first had to be created by someone. Creation means that research, developing, and testing were undertaken by a number of organizations, which in turn could also be characterized in terms of resources. Resources come at a price. Therefore, it is only logical that the patent holders should get their share of the action. In MPEG-2, this is illustrated quite well and indeed has proven to be a scheme that works. In addition to patent fees in open standards, the technology provider also would like to see some return on his investment in developing standard-conforming products. Thus, open standards come with a combination of patent and technology licensing.

A bit later, the licensing aspects of MPEG-4, which have been widely criticized, will be considered in much more detail, but one fact to keep in mind at this stage is that with such a vast number of possible uses for MPEG-4, it is virtually impossible to create an “all-encompassing” licensing scenario that takes on all possible business models.

With proprietary formats, it is a bit different since only technology licensing fees apply. However, they very often come without patent infringement indemnification, which is where we once again encounter a disadvantage of these kinds of technologies. Another factor is operating system independence, which is not always delivered by proprietary technologies; whereas MPEG-4, the open standard, can be obtained in all flavors that a market participant may want to utilize: Windows, Linux, OSX, Solaris, Pocket PC, Symbian, and so on.

A little earlier, we talked about patents and intellectual property rights. ISO, which we’re familiar with by now, does not deal with licensing at all and operates under firm rules. MPEG, being part of ISO, also does not get involved in licensing, but takes technology proposals from companies to be adopted into the standard and requires contributors to make their patents and technology available on reasonable and non-discriminatory terms.

But how does the patent licensing of standards, or more precisely, of MPEG standards work? Very simply. There are so-called “patent pools” in which companies and institutions that have contributed to a standard bundle their interests and are represented by an administrator. In the MPEG world, there are actually two main players. MPEG LA, an independent license administration company, acts on behalf of the patent holders of MPEG-2 visual and MPEG-4 Visual as well as MPEG-4 Systems. The audio part for both is administered by Via Licensing, which is an independent subsidiary of Dolby Laboratories and which develops as well as administers patent licensing programs or patent pools on behalf of patent owners/licensors. An important fact to realize in this context is that patent pools are effectively a private initiative from the patent holders for the benefit of the market and are not conducted on behalf of either ISO or MPEG, which also do not endorse or support these efforts, the reasons for which is found in their rules of operation.

Patent pools, in essence, operate quite simply. The patent holders meet, agree on a joint strategy and mutually acceptable terms, and these are then announced with the administrator collecting the respective patent license fees from the market where applicable.

There have been quite a few critical voices regarding patent licensing who feel that all or part of the standard should be royalty free. If one really thinks about it, this would be akin to Chrysler giving away cars for free, which is highly unlikely. Let’s stay with the car scenario for a minute. Car manufacturers are paying patent license fees to companies that have designed particular parts of the cars they manufacture. Similarly, it is perfectly reasonable that patent license fees are payable to the organizations that have contributed to a standard like MPEG-4, because, after all, they have made the whole thing possible. In addition, there is a legitimate desire for every industry participant to be remunerated for their efforts.

Another trouble with MPEG-4 and patent licensing is that, in some people’s minds, it took quite a while to get the whole licensing scenario structured, announced, and released. While this may be true, it should be noted that the MPEG-2 market was already buzzing even before the patent licensing was in place. In reality, however, the delay of licensing terms being available has prevented the industry from deploying even more MPEG-4 products and solutions. Regarding MPEG-4 AVC, the good news is that the fundamental licensing terms were already released by the end of 2003 by two organizations who had organized a patent pool, and respective agreements are expected to follow in early 2004. By that time, the finalization of new products and services supporting MPEG-4 AVC will be well advanced with deployments starting sometime in 2004. So, from this perspective, MPEG-4 AVC holds the promise of great opportunities and potential. In addition to the “break” in deployments due to the licensing situation, at a certain point the industry seemed to hold back slightly and wait for MPEG-4 AVC to be available. As a result, 2004 will be a very important and interesting year for MPEG-4.

Overall, patent licensing by its very nature is reasonable and appropriate and contributes to an appropriate climate for technology being used by all market participants. Patents are considered important from an investment point of view and are revenue generators, just as regular products and services are.

On the following pages, we will take a schematic look at the various licensing terms for MPEG-4 Visual, Audio, and Systems as well as the brand-new MPEG-4 AVC.

5.6.1. MPEG-4 Visual

In this section, we will deal with both the visual parts in MPEG-4. The first is MPEG-4 Part 2, which is widely deployed in today’s technology landscape, and the second is MPEG-4 Part 10, mostly referred to as MPEG-4 AVC (Advanced Video Coding), which has recently been finalized and is beginning to be deployed throughout the industry.

MPEG-4: The administrator for MPEG-4 Part 2 is MPEG LA, and the respective license covers MPEG-4 visual products from January 1, 2000, on (with an initial term until December 31, 2008). In a nutshell, the deployment of an MPEG-4 video encoder amounts to USD 0.25 per encoder, and the deployment of a decoder sets you back USD 0.25 as well. In addition, there are use fees applicable for different scenarios, and a variety of licenses are available from “pay as you go” to “all you can eat” licenses. For several usage scenarios, there are also thresholds with no license fee payments available.

Coverage and Patent Holders: The license for MPEG-4 visual covers all the current MPEG-4 visual profiles as defined in ISO/IEC 14496-2:2001 (Part 22 Visual), 14496-2:2001/Amd.1:2002 (Studio Profile), or 14496-2:2001/Amd.2:2002 (Streaming Video Profile), which obviously includes the popular Simple and Advanced Simple Profiles. To put it more simply, it covers the deployment of MPEG-4 video encoders and video decoders from January 1, 2000, until December 31, 2008.

Here is the list of patent holders, whose patents are essential for the MPEG-4 Visual standard: Canon; Competitive Technologies; Curitel Communications; France Telecom; Fujitsu; GE Technology Development; General Instrument; Hitachi; KDDI; Matsushita Electric Industrial; Microsoft; Mitsubishi Electric; Oki Electric Industry; Philips Electronics; Samsung Electronics; Sanyo Electric; Sharp Kabushiki Kaisha; Sony; Telenor AS; Toshiba; and Victor Company of Japan.

License Structure and Fees: The structure of the MPEG-4 Visual license is actually quite simple and organized in a “pick-and-choose” manner with six categories in order to respond and cater to a variety of business models.

Category 1—Consumer-Recorded Video, which means MPEG-4 Video encoded by a consumer (with an appropriate device) but not Internet, mobile, stored, or unique-use video. Examples are devices like PVRs (personal video recorders) and camcorders.

Category 2—Internet Video, meaning MPEG-4 Video received by or transmitted to a device, for example a PC, using the Internet.

Category 3—Mobile Video. This means MPEG-4 Video transmitted to and received by a personal or portable wireless device, for example a mobile telephone.

Category 4—Unique Use Video, which means MPEG-4 Video sold or transmitted to a subscriber who is neither classified in Category 2 (Internet) nor Category 3 (Mobile Video). Examples are cable, satellite, and other conditionally accessed television or video.

Category 5—Stored Video, meaning MPEG-4 Video that is paid for on a title-by-title basis and is stored either on physical media, like a DVD, or transmitted electronically in a form that allows end users to view it at least 20 times for at least one year.

Category 6—Enterprise (ultimate parent entity and its 50% or greater owned subsidiaries)

As time and the market as well of course, have gone on, there has already been a revision in the licensing terms. This happened at the end of April 2004. A revision in the sense that licensees that have already signed a license agreement based on the terms outlined above can freely elect if they wish to continue with their current agreement or switch to the new arrangement.

The process or the terms of patent licensing are also some sort of a moving target because market conditions may change or develop further. In the case with MPEG-4, it actually is designed by the licensors to make things easier and provide a more compact licensing scheme in addition to responding to the voice of the industry, which has been calling for changes in the terms, especially on the use fee front. What has happened here and what we will see again later on when we are looking into the Licensing of MPEG-4 AVC is that MPEG LA have effectively synchronized the terms for both MPEG-4 and MPEG-4 AVC in certain areas, which is actually a good thing.

Under the revised License, there are the following changes:

5.6.2. MPEG-4 Part 10 (MPEG-4 AVC)

The licensing for MPEG-4 AVC is not very transparent at first glance. Instead of one patent pool—as we have seen for MPEG-4 Visual Part 2, for example—there are in fact two patent pools offering licensing terms to the public. Or to be precise, they are in the process of finalizing licensing terms and agreements in order to have them available sometime in early 2004.

MPEG LA and MPEG-4 AVC Licensing: Let us start with the patent pool in the process of being organized by MPEG LA, since this organization was the first one of the two to issue a call for submission of patents in September 2002. After just over a year, in November 2003, MPEG LA announced, on behalf of the licensors, the terms of a joint patent license for MPEG-4 AVC and issued the final terms on May 18, 2004. At the time of writing this book, these terms, which we will illustrate below, can only be considered for information purposes since no formal and final agreement has yet been published, although some are expected very shortly during the course of 2004. Also, as one can assume, companies can still apply to MPEG LA to have their patents (issued patents only) evaluated to determine whether they are essential for the MPEG-4 AVC standard, and therefore should be included in the respective patent pool.

Figure 5.6 Qualitative relationship between the competing licensing pools for MPEG–4 visual in the context of essential patents. Note that there are patents that may be considered essential, but are not covered by either patent pool.

What can, however, be said is that that the initial term of the license will be until December 31, 2010. Royalties for decoder-encoder manufacturer Sublicenses will commence on January 1, 2005; participation fees on January 1, 2006. Before these dates, no royalty payments will apply.

Coverage and Patent Holders: The proposed terms published by MPEG LA cover all of MPEG-4 AVC (ISO/IEC 14496-10), which means Baseline, Extended, and Main Profile. A number of organizations and companies have submitted their patents to MPEG LA whose applications were subsequently determined to be essential to MPEG-4 AVC. The following is a list of essential patent holders that are expected to participate: Columbia Innovation Enterprises; Electronics and Telecommunications Research Institute; France Te´le´com, S.A.; Fujitsu Limited; General Instrument Corporation, d/b/a Motorola’s Broadband Communications Sector; Matsushita Electric Industrial Co., Ltd.; Microsoft Corporation; Mitsubishi Electric Corporation; Koninklijke Philips Electronics, N.V.; Robert Bosch GmbH; Samsung Electronics Co., Ltd.; Sharp Kabushiki Kaisha; Sony Corporation; Toshiba Corporation; and Victor Company of Japan, Limited. In addition, the following companies with essential patent applications have participated in the deliberations concerning the licensing terms that will be described below and are also expected to participate in the License subject to the determination that they have an essential patent: LG Electronics, Inc., LSI Logic Corporation, and Polycom, Inc.

License Structure and Fees: The licensing terms are classified into two main categories. The first is called Decoder-Encoder Royalties and deals with the product side of things, whereas the second category, Participation Fees, deals with services. Let us consider both categories in more detail:

Decoder-Encoder Royalties

End-product manufacturers will have to pay a royalty for what is referred to as a “unit,” i.e., an encoder, a decoder, or both. The first 100,000 units each year are free. After that, for each unit, a royalty of USD 0.20 has to be paid. This royalty applies up to 5 million units per year being deployed. Above this figure, a royalty of USD 0.10 per unit must be paid.

As already practiced in the licensing model for MPEG-4 Visual (Part 2), the licensors have provisioned for an “all inclusive” license for companies (and their subsidiaries above 50% ownership). This right comes at a royalty fee of USD 3,500,000 per year in 2005/2006, USD 4,250,000 in 2007/2008, and USD 5,000,000 in 2009/2010. Furthermore, considering existing distribution channels, a legal entity selling branded AVC products OEM for PC OS may pay for its customers as follows:

The above includes the right to make and sell encoders and decoders as well as to their personal use by or between end users such as in connection with a video teleconference or mobile messaging. However, it does not include participation fees.

Participation Fees

The participation fees come in two main flavors and cover a number of possible applications for MPEG-4 AVC.

Where the end user pays directly for AVC Video

Title-by-title—For an MPEG-4 AVC video that is either on a physical medium, like a DVD, or ordered and paid for on a title-by-title basis—for example, pay-per-view (PPV), video-on-demand (VOD), or digital download, where the viewer determines titles to be viewed, or number of viewable titles are otherwise limited—there are no royalties payable for videos up to a length of 12 minutes. For videos longer than 12 minutes, royalties are either 2% of the price paid to the licensee from the licensee’s first arms-length sale or USD 0.02 per title, whichever is lower. Categories of licensees include replicators of physical media, and service/content providers (such as cable, satellite, video DSL, Internet, and mobile service providers and operators) of VOD, PPV, and electronic downloads to end users.

Subscription—Systems (such as, for example satellite, Internet, local mobile, or local cable franchise) consisting of 100,000 or fewer subscribers (MPEG-4 AVC subscribers of course) in a year and providing MPEG-4 AVC video on a subscription basis (not ordered title-by-title) do not have to pay royalties. For systems with more than 100,000 subscribers, the annual participation fee is USD 25,000 per year up to 250,000 subscribers, USD 50,000 per year for 250,001 to 500,000 subscribers, USD 75,000 per year for 500,001 to 1,000,000 subscribers, and USD 100,000 per year for more than 1,000,000 subscribers.

Where remuneration is derived from other sources

Free Television AVC Video—Licensees will have two options here. Either they will pay a one-time fee of $2,500 per transmission encoder or they will pay an annual fee per broadcast market starting at $2,500 per annum, per broadcast market of at least 100,000 households, but no more than 499,999 households. For a broadcast market that includes at least 500,000 households, but no more than 999,999 households, $5,000 per annum will be payable, and $10,000 per annum per broadcast market that includes 1,000,000 or more households.

Internet broadcast—Since this market is still developing, there will be no royalties payable for Internet broadcast services (non-subscription, not title-by-title) during the initial term of the license running until December 31, 2010. After that, royalties shall not exceed the over-the-air free broadcast TV encoding fee during the renewal term.

As seen above in the decoder/encoder royalty description, as well as for the participation fee scenario, an “all inclusive” scheme is stipulated, amounting to a royalty fee of USD 3,500,000 per year in 2005/2006, USD 4,250,000 in 2007/2008, and USD 5,000,000 in 2009/2010.

Via Licensing and MPEG-4 AVC Licensing: Via Licensing has a very strong track record especially in administering the patent pools for a variety of audio standards. It came out much later (June 2003) with a call for submission of essential patents with respect to MPEG-4 AVC, and announced proposed licensing terms in October 2003. Final terms were announced on April 20, 2004. The current licensors in the pool hope to have the respective license agreement available in early 2004, so by the time you read this, there might already be an agreement available.

Currently, there is no information as to how long the initial term of this license will be, however there is a planned incentive for early adoption: Companies that execute a license agreement before September 30, 2004, will not have to pay royalties through December 31, 2004. This is based on the expectation of a license being available in early 2004.

Coverage and Patent Holders: The proposed terms published by Via Licensing cover all of MPEG-4 AVC (ISO/IEC 14496-10), which means Baseline, Extended, and Main Profile. The companies who have participated in the development of the terms are Apple Computer, Dolby Laboratories, FastVDO, Fraunhofer-Gesellschaft eV, IBM, LSI Logic, Microsoft, Motorola, Polycom, and RealNetworks.

License Structure and License Fees: First, let us consider what sort of products or services will be covered by this license. These will be end-use products in the event that they are sold directly or through distribution channels to end users. Not end-user products (for example, software development kits), or similar products are not subject to the license. In these cases, the party integrating or incorporating the products into its own end-user products is required to execute a respective license. In addition, the services of content provision, where content is replicated and sold/rented on a title-by-title basis to end-users, i.e., consumers, at their request and purchased/rented by them on a title-by-title basis, are also covered by this license.

It also should be noted which companies do not have to sign a license. Implementation providers (companies providing chipsets, board assemblies other than to end users, firmware code, software development kits, reference design hardware, and similar non-end-user products) are not required to execute a license. Users of end-user products (not limited to consumers), such as broadcasters, cable service providers, Internet service providers, and any other entity who only utilizes an MPEG-4 AVC-enabled product and does not replicate content or manufacture and/or distribute products containing MPEG-4 AVC technology, are also not required to execute a license. On the “who has to sign” side of things, companies engaged in the manufacturing of end-user products, replicating (for physical media), and those companies conducting transactions (for non-physical distribution) with regard to content that is subject to the replication fees, have to execute licenses.

The proposed license structure includes an initial license fee payment of USD 15,000, which will not be deducted from any royalty payments in the future. For companies with less than USD 2,000,000 per year in revenues, there are hardship terms available.

Also, there is a so-called “Threshold Royalty Free Program,” which is designed for companies that manufacture and/or distribute less than 50,000 devices/products per year (that would be subject to the payment of royalties) and who, in this context, generate less than USD 500,000 of annual revenues from the combination of the aforementioned distributed devices/products and revenues generated through the sale/rent of content (that is subject to replication fees). For companies that would qualify under these provisions, no initial fee nor payment of royalties for devices/products or replication fees apply. Although this program does not require specific product or revenue reports, it may require an annual certification to verify that the use remained under the stipulated limits. In addition, companies participating in this program need to notify the administrator when these limits are being exceeded.

In addition, there are two categories of royalties—the first one is for encoders/decoders and the second for what is referred to as replications.

Encoder/Decoder Royalties

Encoder/Decoder, permanent end-user product—The royalty fee amounts to USD 0.25 per licensed product. The number of encoders and/or decoders with any single product is not limited, provided that they are not purchased separately. A permanent end-user product is defined as being able to decode content or use the decoder for not more than 20 decodes and/or 30 days after the first use.

Decoder, temporary end-user product—The royalty fee amounts to USD 0.0025 per licensed product. A temporary decoder is defined as being provided with content and intended to solely support decoding of that particular content. The temporary decoder must cease to function after not more than 20 decodes and not more than 30 days after the first use of the decoder.

Encoder/Decoder royalty cap—There is a provision for companies called an “enterprise cap” relating to licensed product royalties in the amount of USD 2,500,000 per year for the sale of all products. Excluded from this provision is PC software that is only sold through PC OEMs (Original Equipment Manufacturer) and which is installed by these OEMs on the storage medium of a PC. For companies selling its PC software through a PC OEM distribution channel, the enterprise cap for licensed product royalties is USD 4,000,000 per year. Alternatively, PC OEMs have the option to pay royalties for their hardware product, rather than the company that supplies the PC software paying separately for the installed software products. In this case, the standard enterprise cap as mentioned above would apply. In the event that a company opts for the enterprise cap at the beginning of the annual licensing period, it does not need to provide specific royalty reports.

Companies wishing to opt for the enterprise cap can do so for all parent, subsidiary, and affiliated companies that are under its common control, where control is defined as an ownership of more than 50% of the respective entity’s voting stock or similar controlling interest that may exist under local jurisdiction.

Replication Fees

Content provision without replication fees—For content that is intended to be sold on a title-by-title basis, no matter whether it is to be distributed via a physical or non-physical medium, so-called replication fees apply. However, if content is provided on a title-by-title basis free of charge and not provided for sale and or rent, then no replication fees apply. As long as a user does not select, request, or authorize, on a title-by-title basis, the provision of content by any means or method, and pays on a title-by-title basis, no replication fees are payable.

Title-by-title, permanent basis—The maximum replication fee payable amounts to USD 0.025 per title replicated/sold on a permanent basis. The rates per title replicated and sold on a permanent basis are as follows (There are no caps fro replication fees of this kind.):

Title-by-title, temporary basis—Disregarding the length of the title, the maximum replication fee per title replicated/sold on a temporary title-by-title basis is USD .0025. There are no caps for replication fees of this kind. “Temporary” means that the end user can use the content or end-user product for no more than 20 decodes and no more than 30 days after the first use.

For all of the above, “title” is defined as a single video work, or a group of related video works (such as episodes of a television series), sold or provided to an end user bundled together, either as part of a digitally delivered bundle or replicated together on a single physical media carrier.

5.6.3. MPEG-4 Audio

Having learned about the licensing situation with respect to MPEG-4 Video (Part 2 and MPEG-4 AVC), let us now look into the licensing aspects of MPEG-4 Audio, which happens to be Part 3 of the MPEG-4 nomenclature. We will look into two aspects—the licensing of MPEG-4 AAC and MPEG-4 AAC HE (High Efficiency).

MPEG-4 AAC: In December 2002, Via Licensing was appointed as administrator for the MPEG-4 AAC licensing and the terms were finalized. Licensing began, with a license agreement in place, in July 2003.

Coverage and Patent Holders: The MPEG-4 AAC patent license grants rights for a variety of MPEG-4 AAC object types, including AAC LC (Low Complexity), AAC Scalable, and ER AAC LD (Low Delay). A number of companies hold essential patents with respect to the MPEG-4 Audio standard, including AT&T; Dolby Laboratories; Electronics and Telecommunications Research Institute (ETRI); France Telecom; Fraunhofer IIS; Fujitsu Ltd.; Nokia Corp.; Nippon Telegraph and Telephone Corp. (NTT); Philips Electronics; Samsung Electronics Co., Ltd.; and Sony Corp.

License Structure and Fees: The several licensing conditions specifically for encoder and decoder use will be outlined later, but before that, a few more important points and stipulations of the license will be covered.

For MPEG-4AVC, no use fees apply and also no fees are collected for the distribution of content in the MPEG-4 AAC format (no license is required for the distribution of MPEG-4 AAC bit streams). Only on the sale of encoders and decoders are royalties payable. Companies that are engaged in the manufacturing or developing of complete (or virtually complete) end-use encoder and/or decoder products, or of component encoder and/or decoder products that are directly provided to end-users, have to execute the license.

It should also be noted that the licensing of MPEG-4 AAC is more expensive than the licensing of MPEG-2 AAC. What must, however, be considered is the fact that companies can practice, and of course deploy and sell, MPEG-2 AAC products, provided they have executed an MPEG-4 Audio license. The reason for this is that the MPEG-2 AAC Low Complexity Profile is a compatible subset of MPEG-4 AAC, and therefore covered by the license.

On executing a license agreement, an initial, one-time fee of USD 15,000 is payable and is not deductible from future royalty payments.

There are three categories to consider in licensing MPEG-4 AAC— standard rates, PC-based software pricing for consumers, and professional products. There is a provision in the license that allows for the deployment of trial products. For decoder trial products, this trial is limited in terms of time for a period of up to 30 days. For encoder trial products, the trial is limited both in terms of time (up to 30 days) and use (the number of encoded items must not exceed 50). In order to better illustrate the respective fees, please have a look at the following tables.

The term “Channels” in these tables refers to an audio output channel. As you may know, stereo is a two-channel implementation, whereas mono is only one channel. The classification and breakdown into channels is pretty advantageous considering the fact that in MPEG-4 AAC, there are up to 48 full-frequency range audio channels. Pricing on a per-channel basis allows a much better differentiation between multi-channel (such as home theatre products) and simple mono or stereo products. The term “reset” basically means that the number of units sold or deployed in a previous quarter is disregarded, and every quarter is counted on its own in terms of units deployed, i.e., there is no roll-over or annual consideration.

Table 5.5 Standard rates

Table 5.6 PC-Based software pricing—consumer

Table 5.7 PC-Based software pricing—professional

In order to better understand the above and put it into context, the difference between “consumer” and “professional” products should be explained. Consumer products include devices like DVD players, TV receivers, set-top boxes, and portable music players, for example, and are non–revenue-generating products (from the perspective of the user). Professional products are those that are specifically used for the purpose of revenue generation, such as broadcast encoders and high-end audio applications, and are utilized in a production environment. The difference in price, and one reason for the non-existence of a volume cap in the licensing program, comes from the fact that consumer products potentially tend to be on the high volume, lower price side (which are therefore well served by the per-channel approach) and conversely, professional products tend to come in at lower volumes and much higher prices.

MPEG-4 AAC HE: Licensing terms for MPEG-4 AAC HE will have been released in January 2004 by its administrator, Via Licensing, and are illustrated in the following tables. Since, technically speaking, MPEH-4 AAC HE is a superset of MPEG-4 AAC, it is anticipated that the same license structure conditions will apply as in the present MPEG-4 AAC license.

Table 5.8 Standard rates

*If greater than two channels, total per-product fees are not to exceed 3% of end-user price, but not be less than $65 per product or more than $2,000 per product.

Table 5.9 PC-Based software pricing—consumer

Table 5.9 PC-Based software pricing—professional

5.6.4. MPEG-4 Systems

Interestingly enough, the plan for creating a patent pool for MPEG-4 Systems was announced all the way back in September 2000, with final terms being released in July 2002 (alongside the final terms for MPEG-4 Part 2), and the actual licenses became available in February 2003.

Coverage and Patent Holders: The patent administrator for MPEG-4 Systems is MPEG LA. The license provides coverage for the MPEG-4 Systems standard is defined in ISO/IEC 14496-1 (Part 1 Systems) and currently includes patents owned by Apple-Computer, Electronics and Telecommunications Research Institute (ETRI), France Telecom, Koninklijke Philips Electronics, Mitsubishi Electric Corp., Samsung Electronics, and Sun Microsystems. The initial term of the license is from January 1, 2000, until December 31, 2008 (as with MPEG-4 Part 2; as commonly practiced in patent licensing, new licensors and essential patents may be added at no additional royalty during the current, initial term).

License Structure and Fees: The licensing for MPEG-4 Systems is structured in a similar way to what we have already seen for MPEG-4 Part 2.

Category 1—Consumer-Recorded Data, which means MPEG-4 Systems Data encoded by a consumer but not Internet, mobile, stored or unique use (for example, PVR or camcorder)

Category 2—Internet Data, which means MPEG-4 Systems Data received by or transmitted to a device or product using the Internet

Category 3—Mobile Data, which means MPEG-4 Systems Data transmitted to and received by a personal or portable wireless device

Category 4—Unique-Use Data, which means MPEG-4 Systems Data that is sold or transmitted to a subscriber, and that is neither MPEG-4 Systems Internet Data nor MPEG-4 Systems Mobile Data (e.g., cable or satellite)

Category 5—Stored Data, which means MPEG-4 Systems Data that is paid for on a title-by-title basis and that is either (i) stored, replicated, or encoded onto physical media or (ii) transmitted to an end user in a form that allows the end user to view, hear, or use such data at least 20 times and for a period of at least 365 days from the date of transmission

Category 6—Enterprise (ultimate parent entity and its subsidiaries that own 50% or greater)

5.7. Bibliography

dicas digital image coding GmbH www.dicas.de

MPEGIF White Paper “The Media Standard” 2002. download from: http://www.mpegif.org.

Popwire Technology www.popwire.com

Report “Global Digital TV Technology & Markets” by Nick Flaherty, Senior Analyst InsideChips.com

VBrick Systems www.vbrick.com

Web site 3GPP www.3gpp.org

Web site Apple www.apple.com

Web site DVB www.dvb.org

Web site Harmonic Inc. www.harmonicinc.com

Web site Internet Streaming Media Alliance www.isma.tv

Web site ISO www.iso.ch

Web site Microsoft www.microsoft.com

Web site MPEG Industry Forum (MPEGIF) www.mpegif.org

Web site MPEG LA www.mpegla.com

Web site MPEG www.chiariglione.org/mpeg

Web site Nokia www.nokia.com

Web site Via Licensing www.vialicenisng.com