Chapter 5

Managing Your Media Assets and Workflows

Section Editors: Jay Veloso Batista

The next workflow steps media assets follow can be convoluted and specialized but increase the monetization of the material at every step. To assist your understanding, this chapter focuses on the criticality of unique identifications for your content, which logically extends to the fundamentally important area of metadata management, and business management implications, as monetization of assets requires juggling multiple media formats to maximize their applications. This chapter explores new forms of essence, such as Virtual Reality, Augmented Reality, and haptics. And to fully understand the workflows involved in the media chain, we conclude with a look at the various business systems involved, including Quality Control and repurposing of content. To increase the financial viability and profitability of the media assets, today’s business systems focus on automating workflows, and this chapter is devoted to explaining the applications of the many diverse systems involved.

There is a well-worn advertising description that has become so widespread and applicable in the media industry that it now commands market research as its own category by the IABM (The International Trade Association for the Broadcast & Media Industry) – that catch phrase is “Best of Breed.” Media production companies demand the top, best performing systems, and this market indicator can be measured year to year. Product vendor companies use this portrayal to indicate their product is the best in its category positioned versus competitors. As with many marketing axioms, the phrase contains a germ of truth – the truly best in class products are very complicated and leverage unique design attributes that allow them to be integrated into a complete system that performs better than the overarching all-in-one tools sold by some vendors.

In order to provide you a deep understanding of these various paths media assets can take, we have assembled a panel of expert contributors, each with industry credentials or leading product expertise. Each section highlights the expert contributor and, as much as possible employs their words to describe the impact and importance of their topic to the track of the assets through the modern workflows in today’s media industry.

Content Identification and Its Own Metadata

Advertising

When we discuss making money with our assets, a fundamental topic is advertising. Identifying advertisements to ensure proper use in broadcasts has been an important issue since the very beginnings of broadcast Radio and Television, and today those requirements include streaming media, Over-The-Top services, and Video-On-Demand playback. An advertiser only wants to pay for the proper advertisement in the proper program location. Harold S Geller, the Executive Director of Ad-ID LLC, a leader in the advertising industry and an expert regarding the history, systems and methodology, explains the basis and applications of metadata to advertising assets:

“If you can’t identify it, you can’t operationalize or measure it; if you can’t measure it you can’t monetize it,” Clyde Smith, SVP Advanced Technology, FOX has said. Unique identifiers and metadata for advertisements must be universally recognizable and employed across the industry to see the full benefits, in Operational systems, in Administration and Finance, and of course in Measurement. There are two registration authorities that supply the industry with advertising identifier metadata, and they are Ad-ID in the United States and Clearcast in the United Kingdom: Both provide unique identifiers, metadata, and APIs for validation and retrieval. By using IDs and data, and the associated program integration tools, media companies can maximize interoperability.

The benefits and cost savings of interoperability are most apparent when technologies work together so that the data they exchange prove most useful at the other end of the transaction.

We need highly interoperable systems at the technology, data, human, and institutional layers. See Figure 5.1.

In the United States, Advertising Digital Identification, LLC (Ad-ID) is a joint venture of the American Association of Advertising Agencies (4As) and the Association of National Advertisers (ANA). Ad-ID was launched in 2002 and replaced the ISCI (Industry Standard Coding Identification), which was in place since 1969, established by the Broadcast Administration Technical Developments committee of the 4As. ISCI was withdrawn from the market in 2007. As of 2014, SAG-AFTRA requires that all ads using union talent be coded with Ad-ID.

All Ad-ID codes are 11 characters except for HD or 3D codes, which have an H or D in the twelfth character. See Figure 5.2.

Ad-ID provides the Complete External Access (CEA) API to approved companies (generally media outlets, online publishers, measurement companies, and their industry vendors) with access to all Ad-ID codes and selected metadata in the Ad-ID system. Companies using CEA must know the Ad-ID code they are looking for in order to validate the code or read the associated metadata.

Ad-ID advocates and supports embedding the Ad-ID code into files, using file native metadata format, and subsequently using the CEA API to retrieve the necessary metadata. In file-based workflows, the objective is to automate as much of the process of moving digital files through the supply chain by integrating different business systems. Having the identifier embedded in the file, and having that identifier survive the various transcoding processes is of critical importance, as ads are used on multiple platforms, and multiple consumption devices.

Industry Benefits of Ad-ID include:

  • Guarantee of Unique Codes: This foundation is vital for the digital era and needed by the entire advertising ecosystem, including advertisers, agencies, vendors, and media;

    Figure 5.1 Interoperability Layers

    Figure 5.2 What Is Advertising Identification & Metadata?

  • Improved Reporting Accuracy: The Ad-ID system allows for enhanced evaluation of advertising assets;

  • Removal of Re-keying and Duplication of Efforts: Without Ad-ID, an asset’s identifying information must be re-entered into systems of record up to 20 times from when an advertiser gives approval to create an ad to the time the ad is actually published and invoiced. This is currently a huge duplication of effort, resulting in human error and increased costs; and

  • Streamline of Workflow via Integration across vendor products: When advertising systems work together, the whole industry benefits. Ad-ID is integrated with a growing list of industry vendors driving advertising interoperability and simplifying processes throughout the industry.

We have learned from history – the fact is that non-standard, home-grown IDs for advertisements are bad for the Industry. These kinds of IDs do not account for the operational steps that follow the point where the ad leaves the advertiser’s domain. Unregistered identifier codes cause confusion among the rest of the advertising, broadcasting, and media community, and lead to wasted time and money. When everyone has their own methods to track, execute, and measure, it takes everyone involved increased time and effort to move things forward. See Figure 5.3.

Figure 5.3 Interoperability and Advertising

As the advertising requirements in our media channels grow more complicated, the Media and Entertainment industry needs greater transparency and standardization. The industry has to be able to efficiently identify, manage, and track creative assets. Using the Ad-ID metadata and enforcing its persistence in your monetization workflows provides a reliable business platform, providing better digital video asset data, coding, and distribution in an industry that is becoming extremely fragmented.

Content Identification for Programs

EIDR

The entertainment industry has always had its own unique media identification difficulties. Different companies and software management systems introduced wide variations in categorizing assets, metadata formats, and even version confusion, such as how to define the difference between an original theatrical release and a “director’s cut,” or how to manage the different language translations. A committee of industry executives founded a non-profit organization, the Entertainment Identification Registry (EIDR), to provide a shared universal identifier system for theatrical and broadcast television assets. According to their own promotional description, “From top level titles, edits, and DVDs, to encodings, clips and mashups, EIDR provides global unique identifiers for the entire range of audiovisual object types that are relevant to entertainment commerce. The Registry’s flexible data model supports complex relationships that exist between various assets and is interoperable with other standard and proprietary identifier schemes.”

EIDR was designed to supplement and support cross reference the application of existing asset identification systems for business to business media distribution needs as well as the requirements of the post-production community. The centralized registry is based on the industry standard Digital Object Identifier (DOI) technology and is available to all participants. It is easy to register media and create and ID and the IDs are “immutable” in respect to rights, or location of metadata or physical asset. The system was architected to prevent duplicates, both to support episodic series, and the ability to create complete sets of video assets from an original abstracted work, such as all the versions of a theatrical release.

The registry is built on a collection of records sub-divided into fields, with each record referenced externally by a DOI and once established every identifier is immutable. Underneath the DOIs is a Handle System, and each native EIDR ID is handle formatted, so the structure is increasing in its specificity, to handle, then DOI then EIDR standards. The resulting “IDs” follow a particular format:

10.5240/XXXX-XXXX-XXXX-XXXX-XXXX-C

where 10.5240 is the DOI prefix: The “10” indicates the handle is a DOI and the “5240” is assigned to the EIDR Association. The remaining string after the “/” is the DOI suffix where each “X” denotes a hexadecimal digit (A-F), and “C” is an ISO check digit. The EIDR registry can also supply a 96-bit Compact Binary form of the ID that is intended for embedding in payloads such as watermarks and a form for “web use.”

EIDR supports four kinds of content records each with its own reserved prefix: “Content” for Movie or Television series assets; “Party IDs” to identify partners, media producers, and distribution companies; “Video Service IDs” to identify channels or networks; and “User IDs” to address the need to manage companies that are subordinate to the parties yet inherit the media access rights. Content records are objects categorized by their types and relationships with the expectation that the Content Record will be permanent, but the organization did plan for human errors and corrupted records – there is a mechanism for linked records called “aliasing” and a manner to designate a record a “tombstone” or dead record.

Another key aspect of the EIDR identifiers is that they support a rich set of Alternate IDs. This allows EIDR IDs to be integrated everywhere in content workflows. Alternate IDs are stored in the metadata of the EIDR ID and support the inclusion of proprietary as well as other standard (e.g., ISAN) references, and these additional identifiers can be added by any party that needs a new ID to support a new workflow. More details on the specification and the ID formats can be found at www.eidr.org.

Metadata

We mention metadata often, it seems encompassing and ubiquitous, so, what is this metadata? For our purposes of moving digital media assets through multiple workflows, our media must be annotated with additional data that enhances its applications and interoperability. For an in-depth analysis of this important aspect of managing our media workflows, we turn to Bruce Devlin, long time industry expert and the Founder/CEO of Mr. MXF and SMPTE’s Standards Vice President – Bruce starts with a definition:

Metadata: plural noun(computing): information that is held as a description of stored data.

This seemingly innocuous definition hides an immense amount of complexity that has arrived over recent years in the definition, processing, distribution, and transformation of descriptive data. To really understand metadata (or data about data) in the timeline media world, it’s worth looking back in time and seeing how we got to where we are today. This will help explain some of the mismatches between theory and practice that become evident as we discuss this topic.

Back in the days when physical media was interchanged, the metadata was generally helpful information that was easily interpreted by humans. Sure, there were databases of scripts and cast lists and media formats, but essentially all workflows were governed by the tape cassette or the spool of film. Metadata might travel with the physical media in the form of paper or a label, or maybe an ID or bar code that could be related to a database somewhere. The actual processing of the media was dependent on humans managing physical machines that read the tapes and connected various hardware devices for processing. The concept of dynamic reconfiguration of workflows was pretty rare. It was the era when we were still fighting physics and silicon technology was the limitation.

Fast forward by a decade and most workflows involved digitized files. Yet, the workflows themselves were predominantly replicas of the file-based workflows and databases full of records about media were predominantly used for searching, locating, and feeding the business pipelines of media organizations. We copied our manual processes into machine-driven processes. Although there was a lot of metadata in use, its interpretation was still very much dependent on humans reading the metadata and taking action or learning from it.

With recent developments and standards, we are entering an era where the metadata is now being used to drive and automate workflows. There is an implicit assumption that the metadata is correct and that concrete workflow decisions can be based on metadata and not interpreted via human operators. This new, high efficiency way of working can deliver great commercial benefit, but only if the metadata is good. So, what is good metadata and when does it go wrong?

Metadata – the good, the bad, and the ugly! Traditional metadata has often been split crudely into technical and descriptive. This crude delineation is no longer helpful because it tells us nothing about what is creating or consuming the metadata, or whether the metadata itself is reliable. Both types of metadata can be used for descriptive purposes or as an operational trigger or tool in file-based workflows driven by business logic.

Good metadata is typically immutable (can never be changed), accurate, and specific. Take, for example, some of the technical metadata in an SMPTE Material eXchange Format (MXF) file. We don’t just say “the video codec format is JPEG2000,” we take great care to say exactly what sort of JPEG2000 and how it’s been wrapped in MXF. This specific and accurate metadata is generally made by software and consumed by software.

Bad metadata is typically vague, and often woefully inaccurate. In an archive migration project that I had an opportunity to oversee, there was a lot of metadata in the database but only a few key fields had been historically used to drive workflows. The rest of the name-value fields in the database were simply annotations for a human reader. One particular field was frequently used to indicate a portion of the timeline that would be removed for a compliance edit. This annotation would be great metadata to improve automation, if only the field were used in a reliable way. Regretfully, in this particular library, the database field sometimes contained a timecode reference, sometimes a frame count and other times, a reference with no discernable use to anyone on the current operational team! Unfortunately, some key metadata was missing – what territory has this edit been made to support? This “meta-metadata” would be required to get the best automation out of the system. In the end, human interaction was required to validate the field and the cost of doing this additional validation step meant that the database field was never used. Management could not support the additional expense.

Ugly metadata can start off looking good, but then turn bad. A recent example was in an SMPTE Interoperable Master Format (IMF) workflow. The request was to have the name of the producer stored in the IMF Composition as embedded metadata. The local system designer applied sensible rules from his perspective, “There is only one producer for the title, so obviously storing the name of the producer in a static metadata area is easiest.” No problem was discovered until much later when a promotional short was made with content from three different producers. The metadata that was previously considered static will now only apply to a portion of the timeline. Historically, this morphing of metadata type was never a problem because there was always a human to analyze the notes and “figure it out,” but as we more and more automate our workflows with software, good timeline models for the data make life much easier and make the metadata immensely more useful. To future proof our databases, the initial data model design may seem ugly and unnecessarily complicated.

Mighty morphing metadata: During a metadata experiment conducted to design the descriptive metadata for MXF, we asked a group of German Public Service Broadcaster to choose their must have metadata set from a collection of 300 terms. We expected a basic unity of responses because when you view the collection of German Public Service Broadcasters from an international perspective, they all have very similar habits and models when compared to North American or Asian broadcasters. It came as a shock that each broadcaster chose a different set of best metadata elements. Many years later the moral of the story was discovered. Each of the broadcasters is fiercely protective of their own regional uniqueness and business model. The metadata chosen reflected that distinction. For this reason, in-house metadata models will nearly always be different and dynamic. The metadata must reflect the unique business if it is to drive the business. We therefore need to look at what is involved when exchanging metadata between different models.

A simple example to demonstrate the complexity of this situation is described as the “Address Book problem.” Many data models for timeline media include the idea of roles such as producer, director, grip, etc. These roles tend to be fulfilled by people. An address book of people tends to be ordered by the person’s name and will often contain a field that has their title. In a simple world, linking these two sets of data together should be an easy, static mapping. In the real world, however, there may be many such roles that apply to a particular title, and a person may have had several different roles. This many-to-many mapping problem requires active code and some careful logic to work in a reliable and automated way.

This morphing of data models is key to being able to interchange metadata between systems in a reliable, repeatable, and predictable way. Very often the interchange format will be different to the native format in either system. This helps improve test ability and interoperability of the metadata.

There are two leading methods for defining and representing metadata: XML vs JSON. Examples of these two metadata formats are supplied in the following Figure 5.4:

Many software developers treat the XML vs JSON question as a religious cause and will argue until someone buys a round in the pub over which is best. Pragmatically, they have different strengths and weaknesses. Here’s my personal list:

XMLJSON

Strict methodology for defining data types, data relationships, and namespaces

Flexible, ad-hoc data based on numbers, strings, arrays, and generic objects

General purpose tools can validate an XML document using the right static schema

All the validation is done in custom code (but there are some good libraries)

Human readable but painful to do so

Very human readable

Great for rigid datasets such as interchange standards

Great for ad-hoc datasets such as the return values from REST API calls

Takes a while to learn, especially for namespace rich documents

Quick to learn

Difficult to make invalid documents if validating tools are used during creation

Easy to make invalid documents unless you are strict about the document creation

In the media world, we have traditionally used mostly XML and will continue to do so because software systems will need to exchange strict, complex data models without different design teams ever interacting personally. In the future, our workflow systems should be prepared to see more JSON appearing (and maybe some YAML too – “YAML Ain’t Mark Up Language”).

Metadata interchange formats are on-going in many parts of the world. Many expert industry teams have worked very hard to develop metadata interchange models. Some examples you may have heard of are the DublinCore, EBUCore, SMPTECore, and the SMPTE Broadcast eXchange Format or BXF. Each of these has a slightly different application space. It’s also worth understanding that the MXF specification is just a metadata model for representing file-based media and the IMF specification is a metadata model for handling and managing mastering workflows. Both can be fully expressed in XML. For more in-depth information, search Google for regxml.

Figure 5.4 XML Example

We have all this metadata, where do I stick it? Now that you know how to represent your metadata and the pitfalls of different representations, you usually end up with the tough decision of putting the metadata in with the media or in a separate associated package called a “sidecar.” There is a fairly simple test which nearly always works to determine if the metadata should be embedded in the media file or separated into a sidecar: Ask yourself this question, “If I wait several years and then sell the media to someone else, does the metadata need to change?” If the answer is yes, then the metadata should go in a sidecar. If the answer is no, then it’s safe to put the metadata in the media file.

Business Systems

DAM/MAM Systems

Turning attention to business systems, the libraries for the media assets and their associated metadata are our first concern. As a newcomer to the industry, the DAM and MAM jargon seem to be used interchangeably, although they actually have very different applications.

Digital Asset Management is a document library focused on controlling, securing, finding, and leveraging content across an organization. DAM systems are typically used to enforce company governance, managing inter- and intra-office communications and templates, while making assets available to groups and individuals based on security clearance. DAM systems manage digital rights, enforce metadata and taxonomy, and optimize business processes, especially the interaction between individuals and departments, such as marketing communication, illustration, and graphics projects. DAM systems offer particular operational features, such as the ability to “check out” a project file and remove it from the library like a book from the shelf during editing or processing, and then “check in” the file when it is returned for general use. DAM libraries often manage complicated review and approval processes, mapped to the business logic of company requirements. A subset of the DAM system is the PIM, Product Information Manager, dedicated to the marketing department as an assistance to displaying the proper message to the correct channel in a timely and management-approved method. In addition to providing marketing communications, PIM systems manage data import and data model formatting and often interface to a customer facing “portal.” Portals can be used for many different purposes, such as customer support, distributor communications and management, as well as media exchange between partners.

DAM systems can manage video and audio assets as a part of their library, but their tools and workflows are directed across all the internal assets and are primarily designed for managing internal and external written communications or training media. Media Asset Managers (MAM) on the other hand are often paired with DAM systems to extend their functionality to address media-specific chores, such as multiple audio translations, technical versions such as standard definition, high definition and ultra-high definition media, subtitles, edited content versions, and distribution versions. MAM systems employ internal playback tools with more professional video and audio features, extended data models to cover the many industry-specific data resources, and workflows dedicated to media and entertainment business chores. MAM systems usually include hierarchical storage manager tools to control massive amounts of storage in tiers as the media is often in huge files, and by design are more complicated and video specific in their focus.

So, a MAM is a MAM is a MAM, right? As Shakespeare says, “A rose by any name would smell as sweet,” but in today’s Media and Entertainment Industry, it is important to recognize that all MAMs are not the same, and while the language the marketers use to promote the products is often rich, expansive, and all encompassing, broadcasters and media professionals can save themselves a lot of pain, time, and money by understanding the various types of MAMs and workflow tools available, and where a particular product provides the best solution.

Some MAMs are optimized for linear play-out: Often utilized for enhancing or extending a Master Control automation system, these dedicated MAMs support a linear channel and effectively automate a number of repetitive station tasks such as dub lists and preparing media for air. They are architected to be workhorses and have solid integrations with the local automation and traffic systems. The “best of breed” systems employ the SMPTE BXF interface for this communication channel and come with ready third-party tool integrated workflows to handle the standard chores a Television broadcast facility addresses on a daily basis. There are a number of these cost-effective systems available, and typically their drawback is a lack of flexibility, business analysis options, or limits on their upgrade applications.

Traffic Extended MAMs: A subset of the MAM for linear playout operations are library management tools that are sold as embedded solutions or packaged with Traffic and Billing Systems, Content Management Software for OTT or VOD distribution, or Work Order systems. These extend a particular tool within an operation, and often suffer from the limits of the playout MAMs, as well as the limits of some vendors to devote development and support resources to more than their core product line.

Production Asset Managers (PAM): Years ago, we would call them PAMs, but vendors have tried to blur the lines of their products’ abilities to widen their market reach. Library systems that prepare production schedules and manage workflows to create versions of media, track version relationships, and provide automated or manual quality control focused on a particular editing platform meet this essential need. Managing editing projects, “parent media,” and all various “child” versions is of prime importance to production support software. These systems are often maximized to a specific editing toolset and have limited expansion capabilities, but what they do, they do extremely well.

Business Process Managers (BPM): Marrying your library to workflow engines for a network operation can bring some solid benefits, especially in task management for employees, automation of repetitive tasks, and business analysis for the executive team. These systems are more flexible, customizable, and provide more insight into large operations with reports and dashboards. BPM MAMs often integrate into traffic and work order systems, all internal systems including custom-built products, and external systems for supplier input and distribution capabilities. If your organization is looking to measure and manage increasing throughput and add new operations or functionalities, a BPM orchestration system may be the best solution for you. If your goal is managing your employees better, understanding their workload and watching the bottom line, BPM MAMs can provide the task management by individual and user groups to control all segments of your company. Key to finding a good BPM MAM is their support third-party vendor list. How many common products do they already support? These systems are usually customized to a particular operation, and they can be expensive to build due to the unique requirements of a particular broadcaster.

While many vendors will promote their software as an “enterprise” solution, only a very few actually have a system designed to support multi-site, multi-department, multi-tenant operations. Enterprise solutions build on BPM MAM tools and add the ability to limitlessly scale throughput, manage millions of assets, workflows and users, and support unique, geographically dispersed operations. PAMs and specialized MAMs are often built into these solutions. These are expensive, multi-year projects with major benefits. The resulting automation of workflows, oversight on task management and deep business analytics provide measurable reduction in overhead and labor costs.

There are also specialized MAMs built as sports and live events logging tools, archive management systems, distributed production applications for TV Group operations, native automatically scaling cloud-based systems, systems with internal programmable playlists – there are a number of specialized options available, each with particular benefits that may meet a defined organizational requirement. Yet, specialization comes with a price, usually a limitation in one or more areas of the product.

There are fundamentals of MAM technology that support automated and efficient workflows and are found in most, if not all, of the MAM products available on the market today. Architecturally speaking, most modern systems supply a browser-based user interface which allows quick deployment of updates and the ability to “skin” the interface in a user’s language or save user-specific screen controls. The system should be ready to be virtualized, in other words the underlying architecture is ready to deploy on-premises, or in cloud infrastructure or in a hybrid combination of both. Frankly, in today’s world with the proliferation of cloud service providers, an all cloud infrastructure is an acceptable alternative to a flexible architecture. Most systems are prepared with active Cluster design options and disaster recovery features. Object relational grid databases provide a fundamental library design that will allow true scaling of your metadata and the system and leverage the functionality of modern database software such as the rich reporting tools now available.

The core of a MAM is its search engine, the system that quickly seeks and displays the media according to user requests. Most companies have adopted “google-like” search tools that automatically offer suggestions as the user types, drawing on an index of search criteria. More recently vendors have added support for faceted search and “elastic search.” The Graphical User Interface (GUI) can define special search criteria across millions of assets, often employing Boolean logic and wild cards, and many systems offer the user the ability to save unique searches for use at a later time. Some search engines have been enhanced with departmentalization strategies for assets, which is of great importance if the operation will manage millions of assets and associated files. Asset storage can be arranged in logical or physical storage, and systems can control media access with security measures.

The primary function of the MAM system is to bring media into the library in a controlled manner, allow management and enrichment of the assets by the library users, and then publish or distribute that media in many different ways in order to increase the monetization of the materials. The incoming process is called Ingest, and these capture workflows can be quite simple or multi-stepped with branching business logic to automate all possible acceptance of media deliveries. Typically ingest workflows will generate a reference “proxy” file, usually a frame accurate representation to the original media that allows annotations and edit decision lists to be applied by a user to the original media. Some systems will normalize incoming media to a house standard video format, but many media executives prefer to keep the incoming asset in its most pure form, and they save the original file as it was received, using the proxy to drive editing and versioning workflows.

Enriching media assets in a MAM includes adding metadata from multiple sources. Artificial Intelligence solutions can automatically annotate every frame of a file or program, convert the audio tracks into scripts, automatically mark points of interest such as product placements and on-screen graphics, etc. Automated quality control systems can annotate the proxy file with technical and visual data based on the testing criteria. Descriptive metadata can be translated into many languages and added to the database referencing to the original media file. Audio versions, such as stereo pairs, 5.1 surround sound and Dolby ATMOS, and foreign language translations can be associated with the original media, and the video can be marked for edits to align to the new translations as well as subtitles. High Dynamic Resolution (HDR) technical data can be added to the media asset and referenced in the MAM. Unique business requirements drive the enrichment process, and the MAM is receptacle for these enhancements and the maintenance of their connections to the original media files.

Publishing, or export of the assets from the library, is another fundamental function of the MAM. Leveraging integrations with third-party tools such as watermarking software, Digital Rights Management, video format transcoders, etc., allows MAM technology to maximize the numbers of versions the assets can take upon export, and most of these functions are automatable in current technology. The publish function of MAMs has been an industry focus for years, driving innovation to reduce labor and increase speed and efficiency for servicing social media, streaming services, video-on-demand and over-the-top services. Publishing can be as simple as a “drag-and-drop” of a file to a new folder, or as complicated as managing thousands of profiles in third-party tools.

In the newest technological advances, cloud-based tools designed as microservices in dockerized modules permit the scaling up and down of operations that allows cost management for unpredictable workloads. This is a boon for start-up businesses or companies that want to “pop-up” a channel or deploy an increase in capacity for a short-term production project. These MAM systems often tout their “native” cloud operation and link with a particular public cloud service provider. Recently there have been some highly publicized network projects to move MAM operations to the cloud to support cost effective, and rapid deployment of linear channel broadcasts. Cloud infrastructure offers MAM systems new flexibility and scalability.

Media has been stored and prepared by the MAM, and in some cases that media is even made “rights aware” by connecting it to the program and rights management tools. This is the next process that media workflows address.

Programming Workflow

Before we discuss the programming workflow, it would make sense to define Rights and Rights management. The rights to programming seem simple to understand, but in reality, rights are difficult to manage. There are two types of “rights,” rights for material that your company owns or manages and rights for acquired content. An easy way to describe these two types are as “rights in” or media acquired by your company for use, and “rights out” for product you own and distribute.

“Rights out” management systems are of key importance to asset owners and are often directly connected to an enterprise MAM to control the library access or manage/limit distribution. They track the media asset and its constituent parts and elements, as well as the contractual obligations the company may have in regard to those components. For example, music may require ASCAP and BMI license payments, or be restricted in some regions. Actors, directors, and writers’ guilds, etc., require percentage payments based on the distribution receipts. Distribution management includes coverage of territories and transmission format such as over the air broadcasts, IP streams, On-Demand file distribution, watermarking requirements, and Digital rights management encryption. These software systems manage the legal distribution contracts and support financial tools. “Rights out” workflows are typically managed as MAM extensions in conjunction with Content Management Systems (CMS) for packaging, and these operations will be discussed later in this chapter in section on “Automating Workflows.”

For an analysis of the workflows for “Rights in,” acquired content systems and the typical workflows associated with them, Peter Storer, Founder and CEO of the industry leading STORER TV Program Management software, has supplied a detailed description:

Programming, or for many in the industry simply “content,” covers everything from local live productions, like news and sports productions, to syndicated content purchased from large content distributors, such as “Wheel of Fortune” from King World Productions, Inc. Networks, like, CBS, NBC, ABC, and Fox act as both creator and producer as well as the purchaser of lots of content. In return, they license the use of this content through affiliation agreements to their local stations. Cable Networks also both produce and purchase lots of content in order to fill their 168 hours of programming every week or 8,736 hours every year. Based on a 2014 report by Nielsen, the typical American has access to almost 200 channels of content of which they watch on average only 17 channels.1 Think about what it takes to manage that 1.7 million hours of mostly unduplicated content every year.

For every hour of non-live content, there is often hundreds of hours of video/audio captured via various devices that must be reviewed, color corrected, audio enhanced, edited, close-captioned, and then delivered to your viewing device, which also has evolved significantly from the days of the standard-definition TV. In short, the process, the workflows, are staggeringly complex and diverse depending on what you are recording, how it will be shown, and who will be watching it.

Let’s consider a relatively simple workflow, say for a live news program. This program airs every day, at the same time slot for exactly the same length of time with the same format or structure (more on this later). Out of the 30 minutes total, you subtract out the number of commercials, let’s say 8 1/2 minutes along with 10 seconds of promotional material, and you are left with exactly 21:20 minutes of time to fill every show. You plan on 3 minutes for your standup weather, 4 minutes of sports (these might vary significantly depending on the market), and 20 seconds for your open and closing graphics and introduction, and that leaves you with 14 minutes for actual news stories. As a result, you might need fourteen 1-minute stories, or five 1-minute pieces and three 3-minute pieces, or some combination that fills the time. Your newsroom assignment editor hands out the stories to their reporters based on what they think will happen that day. In addition, there are also a number of “timeless” pieces that can be used as filler. All of this comes together as the day progresses with last minute changes until airtime. When the final pieces have been chosen, their order decided, the program’s director cues for the program to start. It is a time compressed, intensive, and unforgiving process that happens every day to deliver the news, yet the workflow is pretty straight forward and well established.

Separate from linear television and network broadcast channels is the News gathering workflows which constantly adapt to changing technology and vendor innovation. Over the last several years, technology has evolved to make the tagging and movement of video simpler and today it is practical to do live feeds from the front lines of virtually any story direct to the station and then on to the viewer. These workflows are defined by the key vendors in the space and are quite competitive in features and operations – news gathering organizations should investigate the product defined workflows from each vendor and source the model that best suits their business operation.

Turning back to the more traditional content workflows and how a television station or cable network decides what to put on the air every day and the process of actually making it happen. The first step is the same whether you are writing a book, giving a speech, or making any type of presentation: It is to know your audience. For a traditional broadcaster, there is a long history of surveying their audience using research companies like Nielsen, Arbitron, and others that provided a customized look into who, when, and why people watch what they do. These preferences are why the news is on when it is, and network programming starts when it does. Over any particular day, the TV audience changes its demographic makeup, minute by minute, but interestingly, when averaged out, the actual viewer levels stay pretty stable for any particular time period. Hence the decision to place any particular program in your inventory will be influenced by the demographics that you determine are available at that time. It will also be influenced by the competitive content already available in the timeslot. You might own a really strong game show that targets people 35+ in age, but if there are already three game shows airing in the marketplace, you might consider counter-programing them with news or alternative series content and placing the game show elsewhere. In short, what you put on in each slot is a very complex decision process that many consider more art than science.

In fact, it is a little bit of both. You have to consider what you have purchased or otherwise have available, what it costs to air in the time period, what it may generate in terms of audience performance and hence its attractiveness to your sales staff to be able to sell advertising in and around it. If a show costs $10,000 to air each day and commercials will sell for $1,000 per 30 second and if there are 6 1/2 minutes of time to sell based on the shows format, then your profit will be $3,000 each day. If you can only average $500 per 30, then you will be losing $3,500 each time it airs.

Of course, you sell advertising on a future prediction that it will perform at a certain level (“an estimate”) and what it actually delivers in audience is only known once it starts airing. Hence the reason networks often produce lots of short-run series (7–8 episodes) then run them and see how the audience likes them. Poor performance results in quick cancelations.

Once content has been selected, acquired, and scheduled to air, you often also have to arrange for delivery to your location. This process can range from standard mail delivery of an analog tape to digital files being streamed via private satellite hookups. Once your content arrives, it will often need to be ingested into your MAM, and sometimes it must be transcoded from the production format into the format you have standardized for linear playout within your own operational environment. As part of this process, it may also need to be viewed by someone to determine that it will play back correctly and that there are no objectionable elements based on your predetermined standards. Once approved for air, the show may need some editing to prepare it properly. There is a huge range of options depending on whether the content will air with or without commercial interruptions or many other considerations. Building in appropriate fades to black at designated transition points often need to be done and, if there is any objectionable material, this may also have to be edited out. The final edited version is given an identification code, often referred to as a “house number,” that is then used to reference its use by all the operational in-house systems: typically, the programing software, the traffic software, the MAM, and Master Control automation systems.

Once an asset is selected to air, the house number will ultimately appear on an automation log that controls the order of playback. The structure of the program (the exact timing of each of the program’s segments between the black fade points) must also be entered so the automation system knows exactly how long each of the segments will play before they switch to other content. As the content airs, a record of its exact playback is maintained in order to determine if anything aired improperly. More important, these records act as a proof of airing for commercials, and also serve as an accurate record of what each station aired during every day to comply with any Federal Government investigations. The program management system will often then be updated to reflect that the specific content has been aired and this may be used for various financial reports. As a result of it being aired, the cost of that program has been incurred and its remaining value has been reduced or amortized. If the contract for the use of the content indicates a limited number of airings, the use of the program will also be reduced by one. A history of each airing of every piece of content owned or produced by the station is also typically maintained in the program management system.

This discussion concentrates mostly on content workflows that are used within a traditional TV ecosystem. Today, an alternate content delivery process overshadows this linear broadcasting methodology by delivering huge amounts of video content via the Internet. YouTube, Hulu, Netflix, and the hundreds of other Over-the-Top content sources provide both traditional series, feature and short length films, as well as thousands of other different formats that are consumed by today’s viewer in so many ways. Much of this content does not follow the traditional Network or TV “workflow” process, but it has become a significant challenger to the traditional video content delivery systems.

Media Planning and Promotion in Linear Network and Broadcast Television

Assets are scheduled but no one will watch them if they don’t know about their broadcast. For understanding the next steps media takes as it works through the operations of a linear broadcast, we turn to the President/CEO of Effective Media Systems, the supplier of the industry leading GRIP IT! Software, Brendan Kehoe.

Success in Television should often be viewed as a three-legged stool. Acquiring or making compelling Programs can be viewed as the first leg. Without compelling Programs that attract and keep your viewer interested, and engage them in the plot line, characters, and the outcome, success will be fleeting. The second leg of the stool is to make Promo materials, those 60, 30, 15 second clips that outline what you are offering as a product to the viewer, and interest them in “purchasing the product” – getting them to tune in long enough to be willing to sample and come back again for more.

But having those two legs of the stool is not enough, unless you can get those Promos to the viewers most interested in that type of Genre/Program, and get to those viewers enough times such that what you are offering is “TOP OF MIND” – they remember your offering, and when it is being offered to tune in.

That is the art of Media Planning and Promotion Scheduling. This art is no different than what advertisers spend so much time and effort in planning, executing, and validating results of their own commercials on all channels. Channels need to treat their own advertising with as much seriousness and planning as those who are willing to pay to advertise their products on your channel.

If you fail at this art for yourself, advertisers will not see the ratings success they expect in your programs, and your revenue stream will never meet expectations. There have been many wonderful programs that have been produced at great cost but have failed because this last leg of the stool (and the second leg of the stool) was executed poorly. There have been many a “why did we fail meeting” on any channel, where audience has been polled on why they did not watch, and often the results are:

  1. The Promos I saw did not really tell me what the show was about, or did not interest me

  2. I’m interested in only Sports or News, so why show me a promo for a Soap Opera on in the afternoon

  3. What Program – never saw one but wish I did

Well executed Media Planning and Promotion can overcome this. In fact, well executed Media Planning can even get viewers to tune in to a terrible program, at least for the first few minutes.

So, what is involved in this art? It involves research, some industry math, and an understanding of what drives viewers. Let’s discuss these at a high level.

Being the Bridge: As someone engaged in the role of Media Planning, the first thing to recognize is you must be the bridge between many groups at your channel. Programming will have their own ideas about how to promote a program, Research will weigh in on what they think may be successful, and Advertising Sales will weigh in on what they hope will happen, and what they hope to sell the Program at and to what advertisers. The Media Planner will have to take all these opinions into account, and yet be the singular entity to understand everything else going on at the proposed launch time and push back where necessary.

This involves understanding the “Media Math” of your channel and audience. Much like your Mother may have told you to first eat your vegetables off your plate, lets deal with the math part first of Media Planning. The key terms of this are: GRPs or TRPs, Gross Ratings Points or Target Ratings Points. When a promo spot airs, each spot will reach a certain number of viewers. In each demographic, the numbers will vary greatly. In a sporting event, you may reach more Men 18–49 (M1849) than Women 18–49. In a reality program (think “the Housewives of…”) it will be the reverse. For each Demo, a number will be assigned by the reality of who watched. This number is expressed as a percentage of that Demo audience in your “Universe.” A 1.0 Rating for the demo will count as one GRP or TRP for that demo. The raw count is called Impressions. All Media Planning is done only by GRP/TRP achieved. As well as Reach and Frequency (R&F).

Many times, both Programming and Senior Management will ask “how many spots ran?” The job of the Media Planner is to re-direct these questions to “how many viewers did we reach?” or as you see in a bit “How many viewers did we reach enough times to get them to tune in?”

The “number of spots ran” question is immaterial. A spot in Prime Time may reach 600,000 viewers. A spot at 3AM may only reach 10,000 viewers. So how does the number of spots run make a difference?

Reach and Frequency (R&F) is important: Reach is defined as “How many of my audience have I reached?” You can never hope to reach 100% of them with limited Promo Time and current viewing patterns of viewers. To reach all viewers who are in your Universe, you would need to run a 30-second spot, 24×7 in every break position to achieve that. Unfortunately, many viewers will have abandoned your channel out of being fed up seeing the same spot so many times.

The goal of any media plan should be to reach as many of the target audience as possible, without entering “negative frequency.” Negative frequency is the point where viewers have seen your promo so many times that when it comes on, they leave your channel, to ill effects. Starting out a Media Plan will require at least picking a Demographic (Demo – M1849) to which you hope to achieve effective frequency.

Typically, a good media plan will try and reach approximately 70% of the Universe of viewers who WATCH your channel, and to reach them enough times in a “purchase cycle” to influence their viewing behavior. This is called Effective Frequency. That number will typically be between a minimum of three, and a maximum of seven. So, most Media/promo plans will look to achieve this over a period of four weeks, four weeks considered the Buy cycle for new shows.

There are systems that will project these numbers for you, as it requires the ingest of huge volumes of research data to make these calculations. These measurements use Audience Targeting/Affinity. Just getting to an effective frequency against a specific demo is usually not enough. A Media Plan will need to target specific groups of viewers within a demographic. For instance, if you are launching a Situation Comedy, you want to reach those viewers in the demo that view Comedy programs, on either your own channel, or on competitor channels.

There is research data that allows for systems building a Media Plan, with your input targeting information, that will track viewers in that category, and will direct your promo spots to where you can reach those exact groupings effectively, without wastage.

As well, those systems need to understand when the program is running, taking into account the viewers that it will try to reach are even available at that day/time to view the program.

A critical consideration is Spot Length: The Media Planner also needs to take into account the length of the Promo spots available, and when they should be run. Promo spots in the beginning of the Media Plan execution should be long enough to tell the story of what you are promoting to attract viewers. This usually requires a spot of at least 30 or 60 seconds. Many Media planners make the mistake of running shorter spots (15) at the onset of the Promo Campaign and scheduling the longer length spots closer to air.

Audience testing has proven that initially running the longer spot at the start of the campaign, then using the shorter “cut-down” of that spot closer to premier, causes viewers who have seen the longer spot first to immediately recall that when they view the shorter length spot. It’s the way the human mind works. Have you ever picked up a book, read the first two pages and realize how it ends because you read it years ago? Promotion spots have the same effect.

There are added tools available to the Media Planner in reaching audience. Especially for reinforcing the messages they received earlier. These are the in-show graphics you see run on many channels. They go by many terms – Lower thirds, Snipes, Violators, Ghost Bugs – but serve a very distinct purpose for audience targeting.

There is a great deal of clutter on most channels. Break durations seem longer every year and have more commercials in them (especially since the advent of the 15 second commercial). Your Promo spot runs the risk of being lost in this clutter. An effective mechanism is to use these in show graphics as “reminders.” In the show segment following a break where your Promo spot ran, scheduling a 12-second graphic as a “reminder” – Premiers Tuesday at 10PM – will assist in getting the target audience to remember what you are offering. They also help reach viewers who may not watch any breaks at all. The planning of these should be just as sophisticated as the rest of the Media Plan.

Your own Promotions, when added up, make you the single biggest advertiser on your channel. The use of these tools will help you generate success. Many a failed, expensive program has resulted when effective Promo Planning is skipped.

Sales/Traffic/Scheduling

Karyn Reid, the Vice President for Broadcast Systems at Fox Corporation, has been a Product Manager at both broadcast traffic/automation and program management software vendors and has been involved in the SMPTE BXF schema committee since its inception. In this section she provides insight into the systems that monetize the program channels, the advertising sales and scheduling tools.

Ad Sales and Traffic systems have taken different evolutionary paths depending on the geographies and businesses that are supported by the software design. Different terminology has also been adopted to refer to similar processes, depending on the business or region. The system that modules are contained within can also vary, so for this review we will focus on the functional module versus the system that contains them. Most of these systems now include integration between their elements, and much of the data that in the past was manually entered in multiple systems can be entered once and flow through the systems at the required time.

Ad Sales

Ad Sales systems typically contain modules that support advertising campaign planning, proposal creation, maintenance, and reconciliation.

In the United States, ad campaigns are typically referred to as Contracts (or Orders) in Local TV/MVPD sales and Deals in network sales. Network buys are typically negotiated using a Cost-Per-Thousand (CPM) currency, while local buys use Cost-Per-(Rating) Point (CPP). Deals or Contracts are typically planned and proposed to buyers, and once accepted create commercial “spots” that are scheduled on the Traffic log. As the spots are aired, there is a matching process between spots scheduling parameters and how they actually were broadcast, referred to as Reconciliation (Recon). The Recon process is usually contained in the Traffic module and when complete, the Ad Sales module is updated with the actual airing time.

There are two workflows for spots once Recon is complete: one to manage spots that have not aired according to the Contract parameters, and a second that compares the estimated viewer ratings measurement against the actual measurement. If a spot does not air, or does not air as scheduled, then a “Makegood” or “Re-Expression” is required to resolve the discrepancy (the second spot “makes good” the missed spot or spots). This can be another spot in the same program or time period or another spot or spots in a program or time period with similar viewer demographics to the originally purchased spot. This second process is called Stewardship or Posting. Once the audience delivery data is received and processed by the seller, the campaign delivery is compared to the estimated delivery. If there is a shortfall in delivery, then additional spots are scheduled at no charge to the client to make up the deficiency.

In other regions, campaigns can be sold on an impression basis, which is much more like digital advertising. Buyers are sold a number of impressions, and as campaigns air and audience measurements are received, the campaigns and/or spot placements are modified to assure the promised impressions are delivered.

There is also a move to include digital campaigns within linear campaigns, though there are limitations on the extent of the implementation, especially around the electronic billing function for the omni-platform campaigns.

Content Module

Traffic Content modules are a repository for limited content metadata and primarily support short-form content such as commercials, promos, and IDs. Secondary Event content can also be managed though the scheduling parameters are typically managed in the log module. Some Traffic systems can also manage program content metadata, while in some regions this is typically maintained in Program Management and Rights systems. Some systems are now supporting frame level time content metadata and can import technical metadata from MAM/DAM and Playout/Automation systems.

Traffic/Copy Instructions

While the Ad Sales system manages the request for airtime, what content is requested in that time is managed by a Traffic or Copy Instruction module. Many Traffic Instructions are simple, such as rotating two pieces of commercial content equally for the duration of the campaign. Others can be extremely complex, such as defining a detailed rotation pattern or changing what content airs by day, and/or by time period. For Networks the commercial content can be defined on a spot-by-spot basis. Traffic Instructions are also often very volatile and can be revised by the buyer multiple times during a campaign. Examples of frequently revised commercial content include movie trailers, and in the US, political commercial content which can be changed multiple times a day.

Log Module

The Log module is where all the details come together to create the schedule for a linear channel. Programs are scheduled well in advance and include the program structure (Format) including the expected number of program segments and commercial breaks. Formats can also include secondary event scheduling details for both technical commands and promotional or ad sales content. Spots can be scheduled in the breaks many months in advance of the actual log date and are moved and updated as more information is added to the Log. As the air date comes nearer, more details are added to the Log such as Program Episode segment timings and Secondary Event Detail, either by integrations between systems or modules or entered manually by Traffic staff. Then, the spot scheduling process consisting of scheduling, moving, and optimizing spot placement begins as new or revised ad campaigns are processed by the system. There is typically a point in the Log processing workflow were the Traffic staff lock out external processes, and manually complete the log. They ensure that all open time is filled and that all spots are scheduled in compliance with their campaign parameters. The Copy Instructions are applied to all spots, and any spots missing content are removed from the log and replaced with either another spot or by promotional or filler content. When the log is completed, then typically the playlist is created and sent to the Playout system. Changes that need to be made after this point can either be handled manually, or by a Live Log mechanism provided by the Traffic and Playout vendors.

Scheduling Engine

The process to schedule and optimized ad spots is a very complex algorithm and continually updates the Log as ad campaigns are entered and modified or Log events are revised. Scheduling Engines ensure spots are placed as details in the campaign, while also ensuring protection from competing advertisers or restricted content. The Scheduling Engine should also provide an even rotation of a campaign’s spots across days and within a time period ordered. An example would be a campaign that ordered ten spots a week, ordered Monday–Friday, from 06:00 to 10:00. A good rotation would ensure the spots are evenly scheduled across all days ordered, and within the time period. An example of a bad rotation would be most spots being scheduled on Monday and Tuesdays, only most within the nine to ten hour. Spots are also given a priority within the Ad Sales module, and spots with a higher priority (and hopefully spot cost) will have a better chance of receiving a preferred schedule.

Reconciliation

Once a playlist or event has completed airing, the airing details are returned to one or more business system modules to be processed. The Reconciliation (Recon) process compares how an event was scheduled or ordered, and how it aired. If there is a significant variance, such as a spot not airing, or airing out of the time period requested, then the event is flagged and will require resolution. The Recon process is typically automated, with a manual evaluation of the events marked as out of requested parameters. Each business system can process the Recon details from the playout system, or a single system can process the output from the playout system and then send the relevant details to the other business systems.

Invoicing and Accounts Receivable

After the Reconciliation process is completed, invoices are produced, typically on a monthly or weekly basis. These invoices can be sent to the buyer either electronically or in printed format. Many Traffic systems include an Accounts Receivable module though many networks use their General Ledger system instead.

Reporting

Ad Sales, Traffic, and Scheduling systems include detailed reporting modules to track projected and actual revenue, commercial inventory available to be sold, and regulatory compliance. As Ad spots are scheduled and revised and Logs are edited there is an impact to the projected revenue and inventory, so these reports can be run during the business day to get the most current data.

Master Control Automation / Playback

With years of experience in broadcast operations and as VP/GM for iHeart radio’s Florical Master Control Automation software division, Shawn Maynard provides a thoughtful analysis of the next workflows that media follows on its way to transmission, Master Control Automation and playback:

Since the beginning of television, various electronic machinery has been in use to produce the product that viewers have enjoyed for ages. As the industry evolved, the electronic machinery became more and more intelligent and eventually allowing external control. This external control, once used for more conveniently placed buttons for Master Control operators, birthed the dawn of automation. The inception of automation began with very basic capabilities to manage tape/reel machines for recording live content for later playback, then quickly evolving into full automated control that manages the entire Master Control workflow chain. It would be very difficult to operate a modern television Master Control environment without some level of automation governing the process. Automation moved from a luxury to a necessity for the future of television content management and playout.

Television automation can be divided into three basic functions within Master Control: Acquisition and Preparation, Traffic and Management, and On-Air Playout. Each of these various functions will be explored from a high-level point of view providing the reader with a general understanding of the role of automation within a broadcasting environment. Before diving into each individual area of function we must first provide an overall description of the workflow of Master Control to understand the environment that demands the need for automation.

A viewer is sitting on their sofa enjoying their favorite television program without any thought of how it was delivered to them in the full package in which they see. The fact that the process is invisible to the viewer is a good thing and a testament of the success of automation. A broadcast company includes many different departments that are responsible for various aspects of the product that eventually is delivered to the viewer. From a very high level, a typical facility will have a Sales department that is responsible for selling commercial time (i.e., commercials/spots/ interstitial content), a Programming department for tailoring what “shows” will air, a Creating Services/ Marketing department for promoting the product to the consumer through various means, Production department for creating unique and live content, and a Traffic department that is responsible for aggregating all the business needs and create a schedule that contains the detailed instructions on how each element will play out for a given day.

The very end of this television “sausage factory” process chain that is responsible for what the viewer will see is Master Control. They are the tip of the funnel that receives, manages, and plays out the content that comes from all the various departments to fulfill the instructions of the schedule from Traffic. The strongest relationship and acting liaison between Master Control and the other business units is Traffic. In today’s automated environment the responsibility of how content will air has shifted from the Master Control operator to Traffic. The automation system precisely executes the schedule provided.

The first steps media takes in these workflows is in the acquisition and preparation phase. Traffic may be responsible for the schedule of instructions of how and when the precise content will play back but they do not generally provide the content itself. The responsibility of Master Control is to acquire the content needed to fulfill the Traffic schedule. In today’s operating environment most content arrive via Internet delivery into a local storage device as electronic files with metadata instructions attached and, although rarer, some content arrives via satellite stream that must be recorded into a local storage device. Automation has evolved over the years to manage the entire acquisition chain by monitoring “watch folders” for the delivery of electronic files and matching the inherent metadata provided with the local copy instructions of the asset from Traffic.

The automation will automatically retrieve the file, run it through a transcoding process as necessary to create a localized version, update its database, and move the finalized copy into the on-air video playback system. Additionally, any satellite or live video feed content is handled in a similar way. The automation will control the various machinery needed for receiving the feed to convert the content into an electronic file for the process described above. In general, all of the machinery (equipment) necessary for acquisition of the content is managed and controlled by the automation system. The operator interfaces with the system through a GUI (Graphic User Interface) in order to verify or manipulate the content once it is acquired within the system.

The main responsibility of the Master Control operator is to verify that all content is in the system and ready for on-air playback. By using various applications provided by automation the operator will manage the content to verify the quality and ensure that it will playback as instructed within the Traffic schedule. For long-form programs (i.e., shows) the operator will verify that each segment matches the format that is scheduled. Most Internet-provided content will contain format metadata that the automation system will automatically incorporate within the on-air database and operators mainly use the automation GUIs to verify or make any necessary adjustments to the timing details of each segment. The ultimate goal is to ensure that all content is prepared and ready for on-air playback.

It the symbiotic relationship between Traffic and Master Control cannot be understated. It is imperative that these two departments “dance to the same rhythm” and work closely together. Master Control is at the mercy of the broadcast schedule and metadata instructions for interstitial content that is provided Traffic. Whereas Traffic is dealing with sales, programming, creative services, News, and advertising agency instructions, Master Control is utilizing electronic equipment to manage the essence (“physical representation”) of the content itself. They have at their disposal all the equipment and software necessary to ensure the integrity of the on-air product.

Advancements in technology standards has significantly helped to facilitate the integrity of communications between these two departments such as SMPTE’s ST-2021 (Broadcast eXchange Format). BXF has standardized the electronic communications between the Traffic software system and the Master Control automation system to ensure they are speaking a common “language.” The growing use of BXF within television is helping to maintain metadata workflows from programming and commercial content that carry through the Traffic business system on to the on-air Master Control automation system.

Asset Management is also a large part of the overall responsibility of Master Control to maintain content inventory for playback. Storage architecture varies from one business to another based on the amount of content that can be accessable by the automation system. MAM (Media Asset Management) and DAM (Digital Asset Management) system integrations are becoming more prevalent and necessary as business requirements become more complex and move beyond simple playback instructions. The integration of these systems within automation has advanced the capabilities of the playback system significantly. The storage management can now be maintained in more efficient technology mediums like off premise solutions (cloud) or complex archive management solutions that are seamless to the on-air automation system.

As already detailed previously, the schedule instructions provided by the Traffic system are what the Master Control automation system utilizes to manage the on-air product. Those instructions account for every second of the 24-hour broadcast business day. Depending on the type of on-air product, the complexity of those scheduled instructions vary: for simple on-air products, it could be playing back pre-recorded content in a back-to-back linear fashion requiring very little interaction of Master Control operators outside of maintenance issue; for more complex operations a schedule may include a variation of live programming, pre-recorded content, graphics, and dynamic interaction (e.g., sports, breaking news, etc.).

Master Control automation should require little to no operator intervention to perform the duties found within the Traffic system. The instructions should be clear and concise for the automation system to fully execute them consistently and flawlessly. In a traditional Master Control environment, the automation system will be managing and controlling various equipment needed for playout.

This equipment includes but is not limited to a video router, Master Control switch, video servers, keyers, graphics systems, and more. More modern systems are using software versions of traditional hardware that run on COTS (Commercial-Off-The-Shelf) computers called Integrated Playout or “Channel-in-a-Box.” In the early adoption stage of the always evolving Master Control automation is moving to more advanced technology in virtualized environments that move away from monolithic software applications to micro services that handle the functional requirements in a more efficient and economical workflow.

What does the future hold? It is impossible to accurately predict details of the future of Master Control but certainly the broadcast industry is moving quickly in the footsteps of the IT industry by using more software-centric solutions to provide greater quality that is more adaptable to an ever-changing consumer expectation.

There have been three main technology shifts over the past 15 years that has radically changed the landscape of Master Control. Moore’s Law has remained faithful by accurately predicting computing capabilities that operate at speeds to enable more software-centric tools to meet and exceed hardware equivalents birthing the use of COTS computers in place of purpose-built hardware. The shift from SDI (Serial Digital Interface) video standards to a wider-industrial IP (Internet Protocol) standard allows more IT industry tools and deeper manipulation of the video workflow without damaging the integrity of the product. Finally, the expanding use of cloud technology has moved the product out of the purpose-built facility to a more virtual environment that can be controlled and managed throughout software solutions.

These shifts of technology demand more and more need for intelligent Master Control automation solutions that are reliable, flexible, and simple to integrate. The mantra of doing more with less is remaining true. To add to that, there is a desperate need for automation solutions to simplify the inherently complex environment of disparate technologies.

In conclusion, Master Control operations are like snowflakes; to the casual observer they are all the same. However, the devil is in the details. No two operations are ever exactly the same. They each have unique requirements based upon the on-air product they control. Automation is key to keep advancing the capabilities of the on-air product by continuing to make the process more and more simple to manage.

How BXF Can Help

Again, drawing on Karyn Reid for her expertise, here is a description of BXF and how it applies to our media workflows:

The SMPTE Broadcast eXchange Format (BXF) standard was originally planned as a way for business systems and playout vendors to reduce the number of software interfaces they were maintaining. Each time a vendor released a new update, each of the existing interfaces had to be regression tested to ensure that there would be no impact on TV stations’ critical schedule functions. As the group began its work, it became clear that there were many other workflows that could be improved with an agreed upon standard, and the work also expanded to include transactional as well as file-based communication between systems.

Early in the adoption of BXF, there was a common misconception that BXF was a “plug-and-play” standard. It is a very expansive schema and a single transaction can be expressed in multiple ways, so it is key that all systems in a transaction agree on the structure of the data. BXF is also extensible, so any data missing in the schema can be added to an integration as long as the vendors agree to the schema expansion.

One commonly known application of BXF is Live Log, which allows changes made in a Traffic Log to be updated in Playout in real-time. Live Log replaces a very manual process for traffic schedule revisions made after a Log is completed. The benefit of Live Log is that changes can be made in the Log module, where the revision can be validated against ad sales, promotion scheduling, and Program Rights parameters, ensuring a late change will not violate any of the many constraints managed by these systems. A caution to this is that the Log module may not know the actual “live” time, and changes made too close to live, or even past the actual time, may impact the on-air product. Any implementation of Live Log should include at least one gating mechanism to ensure changes are made far enough in the future that issues, such as missing content, can be resolved before the event is scheduled to air.

BXF also supports Schedule and Program Format communication between Program Management and Traffic systems, Content updates including dubbing and purging commands, and Reconciliation (As-Run) data.

The ability to communicate real-time Recon data can have the greatest impact on a media organization’s revenue. Traditionally, the Recon process is completed on the morning of the first business day after the schedule log was completed. If a spot was missed on the last day of an ad campaign, but the traffic staff was not notified until the next business day, the result can be a credit for the missed spot and lost revenue. If the Traffic system was made aware of the issue as soon as it occurs, the staff, or the system itself, could resolve the missed spot and save the revenue.

Being able to add or remove content records or update the status of content is another powerful feature of BXF. Many times, content is received via a Content Delivery Service, and then the metadata is manually entered in the business systems. This can result in keying errors or mismatches in metadata, which can flow through the various workflows to the invoice and delay payment. By using BXF to pass a single source of truth between systems in the media workflow, processes should be smoother with less confusion and error. Real-time content deletion can resolve a common issue, where content is deleted by Traffic staff and the Content ID is reused shortly afterwards. If the deletion of the original content is delayed in the playout system, the old content may air in place of the new content. The use of BXF to support Content IDs such as Ad-ID in place of legacy House Numbers can help reduce the risk of incorrect content being aired.

BXF has been used to pass inventory availabilities between Traffic and Promo Optimization systems, Promo schedules including secondary events, program schedules between Networks to Broadcast stations, and has supported Metadata communication.

As media workflows have expanded to include systems such as MAM/DAM, Content Delivery Systems, and Enterprise Service Buses (ESB), BXF has found a place in the communication workflow.

The BXF committee continues to refine and expand the schema which now includes Traffic/Copy Instructions and enhanced Secondary Event commands along with many small updates that support new technologies.

Quality Control

Quality control in a broadcast operation is different from the QC operations used in post-production. To understand the workflows, we have tapped Frans De Jong, Senior Engineer, European Broadcasting Union to supply an understanding of its wide-ranging impact:

Quality Control (QC) is essential for making sure content is technically useable and editorially correct. A lack of proper QC can translate itself in delays, viewer dissatisfaction, and potentially hefty costs for recommissioning and penalties.

The Desire for Automation

QC processes have developed slowly over many years, based on linear workflows that were unique to each content provider and often targeted a single output, such as traditional broadcasting. Much of the QC process was manual, relying on ‘golden ears & eyes.’ The move to file-based workflows combined with an increase in the number of outlets has led to a situation where the amount of content and versions is so large that automation is desired. It is also desired because a large part of the technical QC is abstract; human beings are good at verifying visual information, but not very good at sifting through gigabytes of an MXF file to spot a single byte error. Traditionally this was less of an issue, as the limited number of media supports (videocassettes) already guaranteed many of the technical parameters; putting a Betacam cassette in a DVCPro player was physically impossible.

QC Workflow

The amount of QC automation that can be achieved depends first of all on the types of checks a company wants to perform. Computers are good at (file) format checks, but humans are still better at spotting specific baseband video issues and, of course, at editorial QC. Practical workflows make use of this complementarity, typically starting with automated QC, followed by the manual checks. If a file can be failed fast automatically, it prevents wasting valuable human QC resources (Figure 5.5).

Figure 5.5 Example QC Workflow

Example workflow consisting of an automated QC step followed by a manual one.

QC Delivery Specifications

Media workflows often span multiple departments, companies, or even countries and continents. This complicates QC, as it risks checks not to be performed or performed multiple times, costing time and money. Also, media facilities do not necessarily possess the wide and deep skillsets to perform all QC in-house. This can be addressed by clearly specifying what QC is expected as part of a media organization’s delivery specifications. An example is the DPP,2 which has created a detailed QC specification for files delivered to the UK’s national broadcasters. The other element that is key is to use a common reporting format, so that results of QC processes can be compared and used to automatically trigger appropriate follow-up actions.

Standardizing QC

To make sure QC checks and their results can be compared between organizations, devices, and QC profiles3 (which may use different parameter settings for the same checks), the EBU has created a large open catalogue4 of hundreds of QC definitions for the industry to implement in tools and for content parties to create delivery specifications with. Each of the checks is illustrated by a card, which summarizes the key characteristics of what the check is about. A public API allows QC product creators, QC service providers, and users to download the latest definitions.

In fact, in BXF 6.0, a fully standardized XML structure that is completely compatible with the EBU set of QC definitions was added in XML form (Figure 5.6).

Figure 5.6 EBU QC Cards

Knowledge Base

The QC definitions not only provide detailed technical information, they also include descriptions and references, which help media staff better understand why a check is performed and what specifications it is based on. Examples and test material can be linked to the definitions, as well. The EBU QC catalogue is continuously being reviewed and updated to reflect new content formats such as IMF and platform requirements such as captioning.

QC Outlook

The biggest gain in the short term may lay in the widespread use of standardized QC profiles and QC reports, enabling more integrated workflows. This is reflected in the three main concepts in the EBU.IO/QC data model5: QC Catalogue, QC Profile, and QC Report.

We can expect the automation of QC to continue to make use of the latest innovations. Cloud-based QC is already a reality and Artificial Intelligence and Machine Learning promise more automation for audio/visual (baseband) checks. The dream of fully automated QC is unlikely to be realized any time soon, though, as especially the visual (baseband) validation of images still is a ‘tough cookie’ for machines to crack.

Annex A: Classification of QC Checks

A.1 QC Categories

QC checks can be classified in terms of how automatable or reproducible they are. This is especially important when using different QC devices. The following five definitions are used by the EBU.

Regulatory

A check that must be performed due to requirements set by a regulator or government. Has a published reference document or standard?

Absolute

A check defined in a standards document including a standard scale. May have a defined pass/fail threshold. As a user, I ought to be able to compare the results of different QC devices and see the same result.

Objective

A check that is measurable in a quantitative way against a defined scale, but no standard threshold is agreed. There may be no formal spec or standard, but an agreed method is implied by or exists in the check definition. As a user, I ought to be able to compare the results of different QC devices if I configure the same thresholds.

Subjective

The check may be measurable, but with only a medium degree of confidence. A failure threshold is not defined or is very vague. May require human interpretation of results. As a user, I cannot expect different QC devices to report identical results.

Human Review Only

A check that can only be carried out by human eyes or ears (Golden Eyes and Golden Ears) or where a human is required to fully interpret the results of an automated QC tool.

A.2 QC Layers

Another way to classify QC checks is the level at which the check is performed. The following definitions are used by the EBU.

Wrapper

Tests the structure/integrity of the file wrapper, or of the metadata within the wrapper.

Bit Stream

Tests the structure/integrity of the encoded bit stream, or of the metadata within the bit stream.

Baseband

A test applied to the decoded essence – the video frames or audio samples.

Cross-Check

A verification that the values in other layers agree. For example, if a baseband test of video frame size has been completed, and the video frame size metadata in the format and wrapper have been examined, the cross-check will verify that the values match.

Source: https://ebu.io/help/qc/categories

Repurposing Content

Monetization is a buzz word in the industry today, tossed about in meetings and forums as a new panacea for ailing media businesses or a grand, perhaps even hidden opportunity for content owners holding assets squirreled away in some dimly lit and cobwebbed library. The idea is that content, with rights owned or acquired, can be repurposed and sold to different markets, different demographics, and across different platforms, maximizing the income from the asset. This sounds like a clever idea, and it can be with forethought and planning, or it can be labor intensive, expensive, and frustrating.

Better than any group in the media and entertainment industry, post-production companies understand this monetization opportunity and its pitfalls and continue to adopt systems to minimize the labor requirements of repurposing media. Here are a few facts to consider:

  • Films and television programs are produced in a multiple of technical formats, including standard definition, high definition, 16×9 wide screen, Ultra HD (which is 4K or 8K screen line counts);

  • For every film, there are as many as 30 versions edited for content distribution in different parts of the globe where different restrictions apply;

  • There are 235+ Video-on-Demand formats used in the United States, and hundreds more used around the world;

  • Video codec formats and media file “wrappers” abound and have hundreds of combinations in use in the industry;

  • Audio formats are also complicated, with stereo pairs, secondary audio pairs (the Broadcast SAP channels), Dolby 5.1 and 7.1 surround sound, and Dolby Atmos multi-channel sound; and

  • Exciting new display innovations such as high dynamic range monitors have added new media packaging requirements to the existing processes.

Dealing with this proliferation of formats, version editing and various types of media is more than full-time work for many post houses. Think about the media on a typical BLU-RAY disk: It usually contains the program video, various audio channels, subtitle selections by language, and the program promotional trailer, and it may be extended with outtake compilation programs and descriptive audio channels, even electronic games and website links to enhance the viewing experience and fan interaction.

We are all familiar with re-purposed media: airline versions of popular, current films; short promotional clips posted to social media sites; video-on-demand; and OTT services like Netflix, Amazon Prime Video, and YouTube. We watch international films with English subtitles or dubbed language versions of Academy Award winning productions. But what does repurposing have to do with our media workflows? Let’s examine some of the more important repurposing chores that are executed every day on media assets.

In our commitment to the less fortunate among us, media is consistently re-purposed for accessibility, in particular adding closed captioning for the hearing impaired and deaf, and the relatively new descriptive video audio tracks that provide enhanced storytelling for the blind. In both of these cases, this media enhancement is often mandated by government laws or regulations.

Closed captioning, abbreviated CC, is the set of processes to display text on a television, video monitor, or other visual display to provide interpretive information. The term “closed” indicates that the captions are not displayed on the screen unless viewer selected by the remote-control command or a software menu option, while “open” or “hard-coded” captions are visible to all viewers and are permanently inserted into the video media. Usually, the captions consist of a transcription of the audio channel of the program as it is played or broadcast, sometimes including descriptions of unspoken picture elements. Most of the world does not distinguish captions from subtitles; however, in North America these terms do have different meanings. “Subtitles” assume the viewer can hear but cannot understand the language or accent, or the speech is not entirely clear, so they transcribe only dialogue and some on-screen text. “Captions” attempt to describe to the deaf all significant audio in the asset including spoken dialogue and non-speech information such as the identity of speakers and music or sound effects using words or symbols. Subtitles can be a similar displayed text, the difference between the two being in technical location of the data in the digital or transmitted media. Subtitle applications include providing a written alternate language translation of the main channel audio and this text is often “burned-in” (or “open”) on the video and cannot be removed during the viewing process. HTML5 defines subtitles as a “transcription or translation of the dialogue… when sound is available but not understood” by the viewer (for example, dialogue in a foreign language) and captions as a “transcription or translation of the dialogue, sound effects, relevant musical cues, and other relevant audio information… when sound is unavailable or not clearly audible,” for example, when audio is muted.

Closed Captioning has come to be used to refer to the North American EIA-608 encoding that is required by NTSC-compatible standard definition video broadcasts which requires that the text be placed in line 21 of the vertical blanking interval in the picture composition. For the US digital television specification ATSC programming, three text streams are encoded in the video: two are backward compatible “line 21” captions, and the third is a set of up to 63 additional caption streams encoded in EIA-708 format. International specifications are considered “subtitles” and their presence can be indicated by the teletext channel reference of “Subtitle 888” on the screen or simply as “888.” These notations refer to the PAL-Video standard definition specifications that employed a Ceefax teletext system. Captioning is modulated and stored differently in PAL and SECAM 625 “line 25” frame countries, where teletext is used rather than in EIA-608, but the methods of preparation and the line 21 field used are similar.

A series of congressional laws require Closed Captioning in the US: The Congressional Act entitled the Television Decoder Circuitry Act of 1990 gave the Federal Communications Commission (FCC) power to enact rules on the implementation of Closed Captioning and required all television receivers with screens of 13 inches or greater, either sold or manufactured, to have the ability to display closed captioning by July 1, 1993. In the same year, the Americans with Disabilities Act (ADA) in which the section Title III required that public facilities, including restaurants, bars, hospitals, and any public location displaying video screens (but not cinemas), provide access to written transcribed information on televisions, films, or slide shows. The Telecommunications Act of 1996 expanded the Decoder Circuity Act requirements to digital television receivers sold after July 1, 2002. Additionally, all Television stations and programming distributors in the U.S. were required to provide closed captions for Spanish-language video programming as of January 2010.

The H.R. 3101 bill, the Twenty-First Century Communications and Video Accessibility Act of 2010, was passed by the United States House of Representatives in July 2010, leading to a similar bill with the same name, S. 3304, to be passed by the United States Senate on August 5, 2010, by the House of Representatives on September 28, 2010, and was signed by President Obama on October 8, 2010. The Act requires, in part, for ATSC Digital decoding set-top box remote controls to have a button to manage the closed captioning display on the screen. This law also required broadcasters to provide captioning for television programs redistributed on the Internet.

Today captions for live programs are transcribed by phonetics-to-text software tools, and the process typically runs a few seconds behind the actual audio channel feed. Live captioning can be considered a form of real-time text, an IP specification. The NORPAK typography extension has overcome many of the initial shortcomings of closed captioning systems and the extended character set now supports all Western European characters as well as Korean and Japanese requirements. The full EIA-708 standard for digital television has worldwide character set support, but there has been little use of it due to EBU Teletext dominating DVB countries, which has its own extended character sets.

When the transcript is available beforehand, the captions are simply displayed during the program after edit work is completed. For prerecorded programs, commercials, and home videos, audio is transcribed and captions are prepared, positioned, and timed in advance. These are the media workflows that most concern our assets. Captions are edited for ease of reading, words substitutions, and reduction of the total amount of text to be presented per minute. Some words can be censored for audiences or to meet broadcast requirements. Video can be edited or “trimmed” to eliminate offensive words or better align with the caption timing.

In an effort to assist global organizations in setting an interoperable standard for captions and subtitles, the Society of Motion Picture and Television Engineers (SMPTE) defined the SMPTE profile of Timed Text Markup Language (TTML), designated SMPTE-TT, to be used for the representation of both “C” captions or “S” subtitles. This specification identifies the features required for interoperability between display systems and to preserve certain semantic features of their original formats, SMPTE-TT also defines some standard metadata terms and some extension types not found in in the basic TTML. The following excerpt from the SMPTE TTML publication provides an overview for our purposes of discussing workflows.

“Known officially as SMPTE ST 2052‐1:2010, or SMPTE 2052, the standard provides a common set of instructions for both authoring and distributing captions for broadband video content ‐‐ enabling broadcasters to reuse existing TV captions which, in turn, allows them to migrate programming with captions to the Web more easily and duplicate online the experience consumers enjoy on TV today. Labeled the Time Text Format (SMPTE‐TT), SMPTE 2052 defines the SMPTE profile of the W3C Timed Text Markup Language (TTML).…regulatory bodies and administrations around the world have, in both technical specification and quantitative requirement, made various forms of timed text (such as captions or subtitles) a necessary component of content. This Standard provides a framework for timed text to be supported for content delivered via broadband means, taking into consideration the economic and practical necessity of the ability to use source data delivered in pre - existing and regionally - specific formats (such as CEA -708, CEA -608, DVB Subtitles, and WST (World System Teletext)) as well as text that may be authored or provided in other formats. It is a goal of this Standard to make timed text essence more useful for broadband content, with equal status to the associated video and audio essence. Considering however that contemporary broadband content on many platforms (e.g. media players, media extenders, home servers, home networks, etc.) may be intended for display in a consumer electronic environment, and that there will be a transition period where such equipment can only utilize pre-existing formats, this Standard also defines functionality for the preservation of legacy formats. It is the intent, however, that new display devices will utilize the more capable TTML timed text structures rather than legacy data formats.”

Captioning and the necessary preparation workflows exist for other media including Blu-ray disks and DVDs, cinema presentations of films, and sporting venue live event enhancements.

Generic Descriptive Video Service (DVS) was proposed as a regulation by the FCC but a court ruled that requiring this service by broadcasters was overstepping their authority. Nevertheless, many network broadcasters in the top-25 markets complied with the initial FCC ruling and implemented the service in the secondary audio program, the SAP channel for broadcast programming. The process makes visual media, such as television programs, feature films, and home videos, more accessible to people who are blind or otherwise visually impaired. The term DVS is often used to describe the product itself.

The workflow for assets is relatively straight forward – DVS describers review a program and develop a script that describes all visual elements which are important to understanding the video action and the entire plot. The descriptions and their placement are limited by the natural pauses in the dialogue or are arranged in the time sequence of the accompanying audio to coincide with a particular event or sound effect. The describer then records the new DVS audio channel, and the producer mixes the new recording with a copy of the original soundtrack. This new DVS audio is synchronized with the original video master on a separate audio track for broadcast on the SAP or to its own “DVS master” for home video. For feature films, the descriptions are not mixed with the soundtrack, but kept separate as part of a DTS digital cinema soundtrack.

Multiple Language support: In the United States, it is common knowledge that we support English as a primary language and Spanish as a secondary language, with Federal government enforcement of SAP channel over the air broadcasts. Obviously in our cultural melting pot, this does not serve people of every national origin, and it may not surprise you to know that many specialty language networks exist in North America to address niche populations, including French, Portuguese, Italian, German, Russian, Japanese, Korean, Chinese, and many more. We even have a non-profit network called SCOLA dedicated to re-broadcast of international news programs in their original language for educational and information purposes, covering a wide variety of the world’s tongues. Other countries face daunting tasks when dealing with multi-language support. There are 700 individual languages still spoken in the Indian subcontinent and their broadcasters, for example the Times of India networks and Doordarshan (the Indian Government public broadcaster), translate and support broadcasts in 33 languages in an attempt to reach the maximum audience. Typically, Hollywood-based media producers will require their media to be translated into 65 foreign languages for international distribution.

With all these different language requirements, you wouldn’t be surprised to know there are post-production companies dedicated to multi-language support, and the workflows required can be quite complex. The primary workflow for assets is either the sharing of a transcript of program’s script, or the creation and sharing of a low-resolution media proxy file to enable audio translations of the main dialogue for voice actors to re-create the audio program. Professional voice actors and Foley operators, the people who create sound effects, can be employed for weeks on these specialized projects. New language recordings are sent back to the post-production facility for re-mixing into the original channels to create the new language version, or in some cases used as a “voice over” channel in a new language. Often, these audio workflows require video media editing, to match new audio timing, or to add post program credits (called a post-roll) to list the voice actors.

Similar translation workflows are used to create multi-language subtitle support, although some of the transcription work can be automated by using software to convert phonetics to text and translation software to put the text in different languages. Additional workflows in multi-language versioning are supported by the largest media producers and include translating the on-screen graphics to culturally appropriate referential material, especially when humor requires the audience to be able to read the text in the visual media. This workflow requires video and graphics editing steps, as well as approvals from region-specific experts who can adequately review and assess the cultural impact of the changes in the video media. If you expect your media to be funny in Montreal, you had best have a Quebecois review and approve your new version.

This highlights an important requirement of preparing assets for increased monetization and distribution, the importance of cultural and regulatory considerations. Not all media as originally supplied is welcome around the world. Even within the Unites States there are regional codes and accepted norms which a media executive, in maximizing their use of programs, must support. Most media are put through workflows that address these cultural and regulatory concerns before distribution and these workflows are usually manual operations that address edits and annotations. Media versions can include edits to audio channels to remove offensive language, and edits to video to remove nudity, sexual acts, depictions of violence, or culturally insensitive references or materials. Sometimes on-screen depictions where no rights are held for individuals require the media to mask or hide individuals inadvertently captured in the filming process or people who are in a protected state such as minor children. These workflows can also note and remove product placements that cannot be shared across markets or remove and replace music that is not licensed for a particular region. Primarily these editing workflows are reserved for sensitive content reduction.

Regulations around the world require the display of ratings. While these ratings and parental controls vary from region to region, it is illustrative to consider the methods adopted by the United States to gain an understanding of the requirements and application. In the Telecommunications Act of 1996, Congress asked the entertainment industry to voluntarily establish a television rating system to provide parents with information about material in programming that would work in conjunction with the new “V-Chip.” The V-Chip is a device built into television sets that enables parents to block programming they personally determine to be inappropriate for their families. By February 1996, segments of the entertainment industry joined together and pledged to create a system to support the Government’s regulation. They agreed that the guidelines would be applied by both over the air broadcast and cable networks to handle the large amount of programming that would be required to be reviewed, approximately 2,000 hours per day. The Industry advocacy groups, including the National Association of Broadcasters (NAB), the National Cable & Telecommunications Association (NCTA) and the Motion Picture Association of America (MPAA) submitted a revised ratings system to the FCC for review. Under this proposed rating system, television programming would continue to fall into one of the six ratings categories (TV-Y, TV-Y7, TV-G, TV-PG, TV-14, TV-MA). Additional content descriptors of D (suggestive dialogue), L (language), S (sexual content), V (violence), and FV (fantasy violence – exclusively for the TV-Y7 category) would be added to the ratings where appropriate. These ratings icons and associated content symbols would appear for 15 seconds at the beginning of all rated programming. On March 12, 1998, the FCC found that the Industry Video Programming Rating System was acceptable and subsequently adopted the technical requirements for the V-Chip. Similar ratings methodologies have been voluntarily adopted in Australia, Canada, and Europe.

The ratings are stored as metadata associated with the media, and inserted into the media in different locations depending on the dissemination format of the material, digital or standard definition broadcasts, Video-on-Demand files, etc. The workflows that address the ratings must place the data in its proper location embedded in the transmission program to trigger V-Chip activation according to the local regional requirements, or in XML packages that describe and annotate the media file to drive VOD devices or on-screen rating displays of modern digital monitors.

From these descriptions, it becomes apparent that there are lots of workflows that simply transform assets to prepare them for multiple platform distribution. A number of companies founded divisions dedicated to multiple platform support, specializing on creating distribution version of media. Initially this was a labor-intensive process, but it has become increasingly more automated through software innovations. Why are there so many versions required? It is a function of the way the industry grew and changed over time. While the world followed a similar path, it is easiest to use the United States to describe the history of how we got to where we are today, and the workflows that impact media assets because of this historical path.

Community access television began as a way to pool neighborhood resources, share costs, and create a multi-channel antenna system to give homeowners more viewing choices. In the 1950s these systems used microwave antennas to extend signals from distant locations and by the 1960s these systems added C-Band satellite antennas to capture and transmit network feeds. Along the way, entrepreneurs discovered that these community antenna systems were valued by more and more consumers and they began to expand their systems using the local telephone poles to carry the antenna leads – these became the amplified signals we call Cable-TV today. Local municipalities sought to protect their community and offered franchises to the entrepreneurs to maintain a commitment to a minimum level of service. According to the FCC’s introduction to Cable Television Regulation,

The Federal Communications Commission first established rules in 1965 for cable systems which received signals by microwave antennas. In 1966, the Commission established rules for all cable systems (whether or not served by microwave). The Supreme Court affirmed the Commission’s jurisdiction over cable in United States v. Southwestern Cable Co., 392 U.S. 157 (1968). The Court ruled that “the Commission has reasonably concluded that regulatory authority over CATV is imperative if it is to perform with appropriate effectiveness certain of its responsibilities”.

The Court found the Commission needed authority over cable systems to assure the preservation of local broadcast service and to affect an equitable distribution of broadcast services among the various regions of the country.

In 1972, new rules regarding cable television became effective. These rules required cable television operators to obtain a certificate of compliance from the Commission prior to operating a cable television system or adding a television broadcast signal. The rules applicable to cable operators fell into several broad subject areas – franchise standards, signal carriage, network program non-duplication and syndicated program exclusivity, non-broadcast or cablecasting services, cross-ownership, equal employment opportunity, and technical standards. Cable television operators who originated programming were subject to equal time, sponsorship identification and other provisions similar to rules applicable to broadcasters. Like broadcast Television operators, Cable operators were required to maintain certain records and to file annual reports with the Commission concerning general statistics, employment, and finances. In succeeding years, the Commission modified or eliminated many of the rules. Among the more significant actions, the Commission deleted most of the franchise standards in 1977, substituted a registration process for the certificate of compliance application process in 1978, and eliminated the distant signal carriage restrictions and syndicated program exclusivity rules in 1980.

The upshot of this regulatory process was that thousands of small franchised systems were built across the nation, and while major consolidation efforts brought hundreds of systems together under Multiple System Operators (MSAs) control, all of these small CATV systems built their own video distribution plants, called head-ends, each with different vendors and technology for managing clients, set-top boxes, and on-demand programming. Despite efforts by the industry standards organizations like the Society of Cable Television Engineers (SCTE), a plethora of video servers, control tools, satellite integrated receiver decoders, security and permission software systems, display systems, and network-specific products were introduced into these head-ends, each requiring different format support. To get the most value from distributing media, owners needed to create different versions for each platform they wanted to serve. In the US alone, there are over 235 platform-specific required versions for Video-on-Demand files.

Today this distribution versioning is further complicated by the OTT services like YOUTUBE, Netflix, by Ku-band direct to the home satellite services and their on-demand format requirements, streaming services, and mobile device services. For asset workflows, this versioning chore can be daunting, and many media owners are welcoming the innovations in software tools that support automating these processes. The steps required include formatting the video with transcode systems to support various video system playback requirements, and in some cases actual editing of the video to supply different versions (e.g., offering a PG version of a film edited from an R-rated original). By adding international versioning, a media distributor can manage up to a thousand different media platform formats to support maximizing their income from a program. Each platform format can have complicated workflows associated with its fulfillment.

This is not a new problem of our industry – our standards organizations have been watching the increasing complexity of version requirements with growing dismay for years.

The Interoperable Master Format

Acknowledging that all the numerous media formats have made it increasingly difficult to distribute and monetize assets, the international standards organizations have sought for decades to design a file format that would support all business models and quality expectations in our industry. Based on constraining the widely used SMPTE standard Master eXchange Format (MXF) and using the lessons learned through the development of the Digital Cinema Package (DCP) used as interchange between distributors and exhibitors for theatrical releases, the SMPTE Interoperable Master Format (IMF) is an international standard for file-based exchange of multi-version, finished audio-visual works. It supports multi-language, multi-reel, subtitles/closed captions, video inserts, and after-the-fact delivery of content with “supplemental packages.”

The key concepts underlying IMF include:

  • Facilitating an internal or business-to-business relationship: IMF media is not intended to be delivered to consumers;

  • While IMF is intended to be a specification for the Distribution Service Master, it can be used as an archival master;

  • Support for audio and video, and all media data essence in the form of subtitles, captions, etc.;

  • Support for descriptive and dynamic metadata synchronized to an essence;

  • Encapsulating (wrapping) of media essence, data essence as well as dynamic metadata into well-understood temporal units, called track files using the MXF (Material eXchange Format) file specification; and

  • Each content version is embodied in a “Composition,” which combines metadata and essences. An example of a composition might be a theatrical cut or a VOD edit.

IMF is built upon a core framework (SMPTE ST 2067-2) that includes essence containers, audio and timed text essence formats, basic descriptive metadata, complex playlists, delivery, etc. According to the SMPTE introduction to the specification, “This core framework is supplemented by incremental constraints (“applications”) specific to particular domains. A typical application will specify video codecs and image characteristics, and can specify additional descriptive metadata, for instance. By sharing the core framework across applications, IMF can adapt to evolving industry requirements while maintaining more than 95% commonality across implementations and practices.”

SMPTE has spearheaded the development of different specifications for different use, calling them applications. IMF Application 2 (SMPTE ST 2067-20) supports High Definition (1920×1080 frame size) Standard Dynamic Range (SDR) images and JPEG 2000 image coding. IMF Application 2e (SMPTE ST 2067-21) extends Application 2 with support for High Dynamic Range (HDR) images up to 4K frame size. Applications 2 and 2e have received support from studios, post-production facilities as well as the manufacturing community, and is being used as a delivery format to OTT services (Netflix). Application 2 and 2e are often referred to collectively as IMF Studio Profile, and the SMPTE sponsored “plug fests” where vendors test the interoperability of systems designed for compliancy are focused on the IMF Studio Profile. Other Applications exist, such as Application 3 (SMPTE ST 2067-30), while others still are being developed. In July of 2017, SMPTE and the UK-based Digital Production Partnership (DPP) agreed to partner to develop an IMF application specification for EU Broadcast and Online services with an eye to publish a specification in 2018. IMF is designed to support all possible international distribution versions (special editions, airline edits, alternative languages, etc.) of a high-quality finished work, including features, trailers, episodes, advertisements, etc. International media distributors such as Netflix and Sony Pictures Entertainment have standardized on the IMF for receipt and distribution of content. To quote Bruce Devlin, SMPTE Governor for the U.K. Region, “These features and the specification which is built upon the IMF standard overall are critical to the realization of file-based interoperability on a large scale, as they ensure that broadcasters can use IMF workflows with their existing content archives.”

Behind the technical specifications are solid methodologies that organize the media. When designing this specification, the SMPTE IMF committee went further than the original MXF group – it defined IMF as a new approach to media file organization and management, in other words a new way to think about the files that support better interoperability and version distribution. There are three key concepts that IMF introduces: Component Play Lists (CPLs), Output Play Lists (OPLs), and Identifiers.

A master version is defined in a Composition Playlist (CPL) which summarizes the playback timeline for the asset composition and includes metadata applicable to the composition in its entirety via XML. It is important to note that the IMF specification allows for the creation of many different distribution formats or versions from the same composition. This can be accomplished by specifying the processing/transcoding instructions through an Output Profile List (OPL). The OPL is a set of actions – the transformations that a CPL requires for a particular version, including format conversions, digital rights management, content delivery network preparations, etc. One CPL with an OPL defines an IMF complaint version of the asset.

The CPL is not designed to contain essence but rather to reference external Track Files that contain the actual essence. This construct allows multiple compositions to be managed and processed without duplicating common essence files – this feature of the methodology is important as it leads to efficiencies and cost savings which are illustrated later in this section. The IMF CPL is constrained to contain exactly one video track (image sequence). This flexible CPL mechanism decouples the playback timeline from the underlying track files, allowing for incremental updates to the timeline when necessary, to change an audio or subtitle track without impact to the entire package.

Each CPL is assigned a universally unique identifier (UUID) that tracks versioning of the playback timeline. Resources within the CPL reference essence data via each track file’s UUID.

A truly important concept introduced by the IMF committee is the idea of “Identifiers.” IMF media is comprised of many identifiers, on many different levels, and they are used to build a particular composition for a CPL, or mark segments of media or specific events on a timeline in the media. These identifiers can act like pointers, and an IMF compliant file can identify multiple sources of media to be “stitched” together to form a new segment for a distribution requirement. Simple content editing, splicing in advertisements or adding pre- or post-rolls is a straight-forward assembly chore in an IMF compliant media preparation system. These identifiers provide tracking mechanisms on many layers, and tracking tools enable workflow automation, making the process of building interoperable versions automatic with all the benefits computerization offers, making operations faster, easier, more efficient, and less labor intensive. This use of identifiers also enables new processes, such as sending identified linked components to change just a part of a previously delivered IMF file (a “supplement”).

By design, asset delivery and playback timeline aspects are decoupled in IMF. The delivery entity between two businesses is called an Interoperable Master Package (IMP): IMF media is wrapped for distribution, and the specification details the specific requirements of that package.

An IMP is described as including:

  • One Packing List (PKL  –  an XML file that describes a list of files), and all the files it references;

  • One or more complete or incomplete Compositions

    • A complete IMP contains the full set of assets comprising one or more Compositions

    • A partial IMP contains one or more incomplete Compositions. In other words, supplemental media not referenced by the PKL

Implications of IMF for Asset Management and Workflows

To maximize these advantages, Media Asset Management systems must be able to offer an end-to-end IMF workflow that leverages the IMF specifications without overloading processes with internal conversions or limitations in the content management structure. New version generation needs to be managed efficiently in order that no media is replicated until required: different versions should merely reference the media masters in unique compositions that includes the additional elements needed for new material; i.e., a localized version in French with text insertions and audio and subtitles in French; another commercial version with different edits; a different technical version; etc. And finally, content delivery must fit seamlessly into IMF specification compliancy so that the package that is delivered takes the advantages of all these “Logical Versions,” supporting a flexible mechanism to select the proper components to deliver (video, audio, subtitles), in the correct order and with the transformations to be accomplished for a particular distribution reception site.

According to Julian Fernandez-Campon, the Chief Technical Officer of MAM Solution vendor Tedial, “To summarize, new Media Asset Management Systems need to be able to support:

  • IMF Import, processing the IMF package and creating assets that can be physical or virtual, preserving and extending the IMF Composition playlist without media duplication

  • Content Enrichment, by creating new versions establishing logical references to the Master and taking advantage of the logical references to the media from the Master(s)

  • IMF Delivery not limited by the original IMF received packages, ready to generate new compliant packages with any compatible combination of the components for a specific Title: Video (SDR, HRD, different aspect ratios), Audio Languages, Subtitles.

  • IMF end-to-end workflows imply the full support of the IMF packages at all levels, not ‘IMF-ish’ MAM that stores and manages the media using IMF labels without actually processing the assets and maximizing the advantages of content management and exchange.

An end-to-end IMF workflow requires these stages:

  • IMF Import: IMF Content is received as a complete unit, or with additional supplemental packages, and is ingested into the MAM;

  • Content Enrichment: full support for the creation of new logical versions or new component aggregation, such as adding subtitles or audio tracks that were not received as IMF packages; and

  • IMF Delivery: Content can be delivered as the original IMF package received, or as new IMF packages based on logical combinations of the components, supplements and assemblies.”

For a successful IMF compliant MAM deployment, content enrichment must be considered a fundamental part of the workflows. Content enrichment includes all the processes and asset workflows that occur inside a media operational facility to enhance the received content and increase its value. In practice, there is limited content exchange based on an originally distributed IMF package, as every media company and broadcaster have their unique methods of enhancing the value of their incoming media. While there are many types of content-enrichment activities and an IMF compliant MAM supports them all, the most common scenarios include subtitles aggregation, audio tracks aggregation, and new versions of the image sequence.

Subtitles aggregation occurs when subtitles are generated externally to be attached to a particular title in the MAM. Subtitles are received as files in a specific format, converted if needed into IMSC1 to comply with the IMF-constrained specifications, and attached to the Asset Title within the MAM library hierarchy. If subtitles aggregation is required, it’s crucial that the MAM system can expand its asset management to display and manage all the multiple subtitle languages and types to be supported.

Audio tracks aggregation as individual files is similar to the subtitles case with the difference that there are more variants due to the sonic layout (2.0, 5.1, etc.) and the class (Full Mix, Music and Effects, etc.). In the IMF compliant MAM, audio tracks must be attached to the Asset Title and a proxy generated.

New Versions can be created within customer facilities using MAM tools to generate new localized versions (trimming, adding text insertions, etc.) that are related to the main title with a specific, annotated relationship. New version workflows present several challenges, depending on the customer business rules, such as slate insertion and any formatting of the video content including bars, audio tones, blacks, etc., that might need to be done with or without an off-line editor. A key feature for an IMF compliant MAM is to manage new version assembly or control the import of the EDL from an editor to identify new fragments and create a logical asset referencing the Master in the MAM. IMF identifiers can be crucial to streamlining and automating this operation.

IMF Publishing is the other key aspect of an end-to-end IMF workflow. To take a very simplistic view, the MAM system needs to be able to “export” IMF packages. One simple and straightforward approach is to “route” the imported IMF packages through the MAM and deliver them in the same compliant package as they were received, yet this simple workflow is not enough for a true end-to-end IMF workflow as the following features are in demand by media companies:

  • Export both Complete and Supplemental IMF Packages: To take the most advantage of the IMF capabilities and cost efficiencies, masters will be delivered as a complete IMF package and the localized or editorial versions as supplemental. This decision depends on customer business rules and their recipients’ requirements – in some cases a reception site requirement might force the export of a localized version as a complete package. Usually, a mix of complete and supplemental packaging is required;

  • Delivery of new combinations of audio/subtitles: The media company may need to deliver a combination of audio/subtitles that was never received in the original master IMF package, e.g., French version with English subtitles. Typically to keep their costs low, a translation company only delivers a French supplemental file that references the original master;

  • New IMF package generation, for those versions generated in-house that did not arrive as IMF Packages;

  • New components export in the IMF Package to allow the generation of new packages with a combination that was never received (e.g., English 2.0 + English 5.1 + French 2.0 + French 5.1); and

  • Transformations: Reception site–specific requirements for generation of site unique audio routing, video transformations, etc., based on the OPL delivery profiles stored in the IMF compliant MAM.

Customer Business Rules outside of IMF: IMF is standardized, well defined and is great for content distribution. The specification permits some level of freedom for customer-specific applications according to their needs, which implies some business rules must be designed to be compliant with the IMF requirements. When implementing workflows, the challenge for a MAM system is the solution must continue to allow business flexibility at the same time the core IMF support is maintained. Some examples that are in use in media workflows include custom metadata fields that become data model extensions; the co-relation of multiple masters (SDR, HDR, 16:9) that arrive as independent IMF Complete packages, yet need to be related in the MAM; restrictions in formats, for example where the customer does not allow HDR versions for specific media; specific essence conventions in the video/audio, to insert slates, tones, blacks, etc.; and associated media such as artwork, photography, trailers, and other related components that need to be linked with the title. All these business rules are outside the native IMF support in a compliant MAM. A successful solution can embed these variations within the workflows to comply with particular media companies’ business requirements and offer an efficient IMF end-to-end workflow tailored to specific needs.

The Business Case for IMF

Examples of the application of IMF principles can be drawn from multiple sources, such as this case study from the Netflix techblog entitled “House of Cards Season 3:”

Netflix started ingesting Interoperable Master Packages in 2014, when we started receiving Breaking Bad 4K masters. Initial support was limited to complete IMPs (as defined above), with constrained CPLs that only referenced one ImageSequence and up to two AudioSequences, each contained in its own track file. CPLs referencing multiple track files, with timeline offsets, were not supported, so these early IMPs are very similar to a traditional muxed audio / video file.

In February of 2015, shortly before the House of Cards Season 3 release date, the Netflix ident (the animated Netflix logo that precedes and follows a Netflix Original) was given the gift of sound.

Unfortunately, all episodes of House of Cards had already been mastered and ingested with the original video-only ident, as had all of the alternative language subtitles and dubbed audio tracks. To this date House of Cards has represented a number of critical milestones for Netflix, and it was important to us to launch season 3 with the new ident. While addressing this problem would have been an expensive, operationally demanding, and very manual process in the pre-IMF days, requiring re-QC of all of our masters and language assets (dubbed audio and subtitles) for all episodes, instead it was a relatively simple exercise in IMF versioning and component-ized delivery.

Rather than requiring an entirely new master package, the addition of ident audio to each episode required only new per-episode CPLs. These new CPLs were identical to the old but referenced a different set of audio track files for the first ~100 frames and the last ~100 frames. Because this did not change the overall duration of the timeline, and did not adjust the timing of any other audio or video resources, there was no danger of other, already encoded, synchronized assets (like dubbed audio or subtitles) falling out-of-sync as a result of the change.

The impact of IMF principles and methodologies can result in significant efficiencies and costs savings. Francois Abbe, CEO of Mesclado of France, commissioned multiple case studies to investigate the application of IMF to current workflows and the following summarizes the results of the case study #1 inquiry:

Case Study: Film Mastering and Versioning

Summary: Cost Savings 26% Using IMF

Background: IMF was born of the D-Cinema Package (DCP) standard success. IMF enables film distribution worldwide using a single package, each region receiving one or more versions of the content, also known as “Compositions.” Several compositions can share common content, thus reducing distribution complexity. In 2006, the Hollywood studios began to consider a similar format for their masters.

The Challenges encountered include reducing interoperability issues, supporting the latest video formats like UHD, HDR, and VR, and enhancing multi-versioning support whilst keeping infrastructure costs down. Outcomes of the study included automated versioning and delivery based on metadata, the ability to set archiving masters, and producing outputs in multiple delivery formats.

Our inquiry approach was to compare the operational expenditures from mastering to versioning, both with and without IMF in the following areas:

  1. Production

  2. Mastering

  3. Versioning

    • Dubbing & Subtitles

    • Trimming & Credits

  4. Storage

  5. Distribution

Production media:

  • One-hour documentary

  • Two versions: Original Version (OV) and International Version (IV)

Mastering:

Without IMF

  • Using ProRes 422 to produce one MXF or Quicktime file per version

  • File size: 60GB per version

With IMF

  • Using JPEG-2000 to produce the Interoperable Master Package (IMP) with higher quality images

  • File size: 100GB for the package

Versioning:

  • For versioning purposes, a proxy is created. The proxy is sent to a lab along with a subtitling or dubbing order.

  • Dubbing and subtitling: We order five subtitles and five extra audio languages (English, German, Italian, Spanish, and Arabic).

  • Without IMF and With IMF, in both cases we receive five .stl files and five audio files

    • Five new masters are produced for the dubbed versions (60GB each)

    • Supplemental packages are added to the original IMP with new audio and subtitles (~100GB total)

    • New CPLs are created for each version

Trimming & Credits:

  • We create two new versions for Arabic and German territories: first for censorship purposes (cutting scenes) and the second for co-production contract (adding credits)

Without IMF

  • Two new masters are produced.

  • The old versions are purged.

With IMF

The IMF package is updated:

  • German version: new CPL with extra credits (slate)

  • Arabic version: new CPL with a removed scene

Storage:

Without IMF

  • We store the OV, IV, German and English versions.

  • Other language versions are purged to save cost (60GB each)

With IMF

  • We store the IMF package. All the versions are included in one single IMP (~100GB total)

Distribution:

Without IMF

  • When ordered, each master is used to produce the version delivered to the client

  • Spanish and Italian versions are purged after use to make savings on storage

With IMF

  • When ordered, the IMP is used to produce the version delivered to the client based on the CPL

  • All versions are kept within the IMF package

Three years later an order is received for dubbed Spanish and Italian versions:

Without IMF

  • Two new masters are produced for the dubbed versions (60GB each)

With IMF

  • The IMP is used to produce the version delivered to the client based on the existing CPL

Analysis and conclusions of the study:

  • With all costs computed using a typical post-production rate card, total cost of ownership over four years is predicted to be:

    • TCO without IMF = $29,904

    • TCO with IMF = $22,017

  • Cost savings:

    • Storage cost is very small compared to the costs of editing and producing new versions (subtitling or dubbing), so it is wise saving all versions

    • Storage cost per version per year

    • The costs of saving extra versions is marginalized with IMF:

    • Cost without IMF = $14.00

    • Cost with IMF = $0.40

    • Cost savings 26% using IMF over non-IMF methods

Automating Workflows

As we have learned in the proceeding sections, media enrichment and versioning maximizes revenue, that “monetization” that each media and broadcasting company seeks to extend their bottom line profitability. Our development of workflows for our assets was predicated upon the business rules that each unique company has evolved to support their exclusive business models. In the beginning, all workflows started with humans doing their jobs manually, moving physical media before the advent of digital tapeless processes, then adapting to more software-based systems and digital media as it became viable in the industry. As the systems we use grew in complexity and our libraries increased in size and scope, automation of the operations became an important business driver for our organizations. Efficiency and cost effectiveness became the evaluation criteria for our workflow designs and today we focus our efforts on removing steps and enabling streamlined functions for human interaction while automating the workflows as much as possible. In a perfect system, everything would be automated. This section explores the underlying requirements for automation, the basics of automating workflows at the machine level and at the human level, monitoring the metrics, and the application of Artificial Intelligence and Machine Learning to enhance our workflow operations.

Like the Master Control automation, workflows automation requires deep integration into both the physical or virtualized IT network infrastructure and all the third-party tools in the proposed workflows or the entire media factory. Third-party tools can include both physical hardware like audio and video routers, media switchers, satellite integrated receiver- decoders, and computer servers, and software-based tools like transcoders, automatic quality control, content delivery networks such as Aspera, Signiant or File Catalyst, watermarking and digital rights management, etc. Integrations to workflows systems enable the workflow management software to control the steps that require external tools and orchestrate the overall stages into a complete process.

In order to organize our discussion of workflow to platform tool integrations, the topics can be separated into a few categories:

  • One-way vs. two-way interfaces

  • Simple command/control or reporting systems (SNMP traps)

  • Defined interface standards like BXF

  • APIs

  • Web services, SOAP, and REST

One or two-way integrations determine the complexity of the interface, the set of commands supported, and the scope of processes that can be automated. One-way interfaces are usually for devices, many of the older designs still in use, that support commands from an external system. These tend to be a set of commands structured with no answer protocol, a “fire and forget” process. These systems are phasing out as new technology becomes available, or their operation in a particular workflow does not depend on responses to complete. Examples include some content delivery network operations where an asset is simply posted to an FTP site or a watch folder, and the CDN transmits the files according to a pre-set profile for deliveries. Two-way communication between integrated devices is becoming more common, where response and reporting protocol is returned through the interface to update the workflow engine or orchestration tools.

Simple command/control and SNMP were some of the first integrations for workflow and system monitoring. Simple command and control is a one-way integration scheme, for example, commands to control a video tape recorder (VTR) is typically a single direction command set with an interface connect specification (i.e., RS422) that manages a simple set of control commands to play, stop, fast forward, rewind, find/set a tape location, etc. Another example of simple command and control is media switcher control which consists of an interface and commands to route the video and audio within a plant. These one-way systems may provide simple reporting back to the workflow orchestration software. Some systems rely on Simple Network Management Protocol (SNMP), a standardized IP protocol for collecting/organizing information about networked devices, and for modifying that data to change end device operations. Devices that typically support SNMP include cable modems, routers, switches, servers, workstations, printers, etc.

SNMP is widely used for network monitoring. SNMP provides management data in the form of variables on the managed systems organized in a Management Information Base (MIB, a type of database) which typically display the system configuration and current status in a graphical, topographic view. Variables can be remotely queried or “pinged” for a status update and managed by supervision applications. SNMP has been updated over time to add feature improvements in performance, flexibility, and security and version 3 is most current. As a standard for integration, SNMP is a component of the Internet Protocol Suite as defined by the Internet Engineering Task Force (IETF) and it consists of a set of standards for network management, including an application layer protocol, a database schema, and a set of standard data objects.

Integrating traffic systems, program schedulers, Master Control automation, and MAM-workflow tools, the current SMPTE BXF protocols include an XML schema definition (XSD) collection for schedules, as-run, content, content transfers, etc. Connecting the Master Control and traffic departments is the most common broadcast use. When properly implemented, BXF-based applications automate the flow of data between the systems increasing process efficiency, streamlining manual steps, maximizing the value of content inventory, and increasing flexibility for the sales department and client advertisers.

As an XML-based communication schema, the two-way BXF interface allows for near-real-time messaging and updating between disparate systems. The XML-based messages include instructions about program or interstitial changes, allowing an automated approach to as-run reporting and schedule changes in the connected systems. Other BXF capabilities include near-real-time dub orders, missing spots reports and content management.

In computer programming, an application programming interface (API) is a set of subroutine definitions, protocols, and tools for building integrations between specific software modules. In general terms, it is a set of clearly defined methods of communication between software components. A solid, well-defined API makes it easier to develop a computer program by providing building blocks, which are strung together by the programmer. There are many types of APIs, and they may be for a web-based system, operating system, database system, computer hardware, or software library. Because of this diversity, API specifications can be complicated and varied, and often include specifications for routines, data structures, object classes, variables, or remote calls. Documentation for the API is usually provided to facilitate usage.

A web service is a software service offered by one electronic device to another electronic device, communicating with each other via Internet Protocol (IP). It is a software function provided at a network address over the web, with the service always “on.” In a web service, the Web technology such as HTTP – originally designed for human-to-machine communication – is utilized for machine-to-machine communication, more specifically for transferring machine-readable file formats such as XML and JSON. In real-world practice, a web service typically provides an object-oriented web-based interface to a database server, to be employed by another web server or by a mobile application, that in turn provides a user interface to an end user. For the purposes of workflow integrations, a web service is a software system designed to support interoperable machine-to-machine interaction over a network.

Web services may use SOAP (Simple Object Access Protocol) over HTTP protocol, allowing less costly (more efficient) interactions over the Internet than via proprietary solutions. Besides SOAP over HTTP, web services can also be implemented on other reliable transport mechanisms like FTP. Developers identify two major classes of web services: REST- compliant web services, in which the primary purpose of the service is to manipulate XML representations of web resources using a uniform set of “stateless” operations; and arbitrary web services, in which the service may expose an arbitrary set of operations. The term “web service” describes a standardized way of integrating web-based applications using the XML, SOAP, WSDL (Web Services Description Language), and UDDI open standards over an Internet Protocol backbone. XML is the data format used to contain the data and provide metadata around it, SOAP is used to transfer the data, WSDL is used for describing the services available, and UDDI lists what services are available.

Many organizations, especially asset media workflows use multiple software systems for management, control, and orchestration. Different software systems often need to exchange data with each other, and a web service is a reliable method of communication that allows two software systems to exchange this data over the Internet. The software system that requests data is called a service requester, whereas the software system that would process the request and provide the data is called a service provider. A directory called UDDI (Universal Description, Discovery, and Integration) defines which software system should be contacted for which type of data. When one software system needs one particular report/data, it goes to the UDDI to discover which system it can contact to request that data. Once the software system confirms which system to contact, the process to contact that system uses a special protocol called SOAP. The service provider system would first validate the data request by referring to the WSDL file, and then process the request and send the data under the SOAP protocol.

Different software may use different programming languages, and hence there is a need for a method of data exchange that doesn’t depend upon a particular programming language. Most types of software can, however, interpret XML tags. Thus, web services can use XML files for data exchange. Rules for communication between different systems need to be defined in he published API documentation, such as:

  • Parameters for systems to request data from each other

  • Specific parameters required in a data request

  • The defined structure of the data produced (Normally, data is exchanged in XML files, and the structure of the XML file is validated against an .xsd file.)

  • Error messages to display when a certain rule for communication is not observed, to make troubleshooting easier

  • All of these rules for communication are defined in a file called WSDL, which has a .wsdl extension

Once integrations between devices can be established, we can map out the processes in our operation and look for points where the step is routine and repeatable as a location for automation. Workflow automations can take many different forms, beginning with the low tech but reliable method of employing Watch Folders and scripts. Pioneered decades ago, but still in use today, watch folders, sometimes called “Hot” folders, are receptacle/storage location constructs that wait for an object/ file or a pointer to an object/file to be placed in the storage location and trigger an action. “Scripts” are small coded action steps to be run against any asset or pointer to an asset that is placed in a particular “folder.” Scripts can be simple or complicated. The drawback to this method of triggering workflows is it is fairly rigid, offers little monitoring opportunity, and can be complicated to maintain. While many workflow automation systems use watch folders, primarily for ingesting newly arrived media, the development of more powerful workflow engines has eclipsed the watch folder method of automating operations.

Workflow engines operate at two levels: an abstracted, integration-media processing layer and the Business Process layer. Around the turn of the millennium, workflow software manufacturers began to separate their machine-to-machine management chores from their human processes. This led to innovations in integration and media movement tools. Building a technology stack with an abstracted integration and media processing layer removes the requirement for “hard coding” the integrations directly to the human processes. Previously, systems built without an integration layer were expensive to update or change workflows because the interfaces to third-party tools had to be re-written as part of the workflow orchestration. Separating the integration layer provides more flexibility and allows for automation of media movement behind the scenes, hidden from the average user.

People power: most workflow orchestration tools use Business Process Management (BPM) to map and manage their user, human-driven workflows. BPM employs a standardized design graphic set that makes it easy for all types of users to map their operational steps. The Internet standard Business Process Model and Notation (BPMN) provides businesses with the capability of understanding their internal business procedures in a graphical notation and gives organizations the ability to communicate these procedures in a standard manner. The current version of the specification is BPMN 2.0. The graphical notations facilitate the understanding of the performance collaborations and business transactions between departments and organizations to ensure that businesses have internal understanding between themselves and their participants in their business and enable organizations to adjust to new internal and B2B business circumstances quickly. Business Process Management Initiative (BPMI) developed BPMN, which has been maintained by the Object Management Group since the two organizations merged in 2005. Version 2.0 of BPMN was released in January 2011 at which point the name was adapted to Business Process Model and Notation as execution semantics were also introduced alongside the notational and diagramming elements.

BPM workflows can also be built to include unique business logic to further automate machine-measured decision processing. Third-party tool capacity monitoring and storage retention policies can be automated, and defined, measurable actions can be machine initiated based on criteria established by management. Examples include:

  • Sixty days after last usage, move media to tape storage;

  • Five days after the project was passed to an editing workstation, send email reminder about project due date to assigned user;

  • Two days before a transmission event, alert user that media has not arrived to fulfill placeholder

  • Etc.

What’s the difference between workflow and orchestration? It’s a matter of breadth and scope, as well as a little “marketing speak.” Workflows are a series of serial or branching parallel steps depicting an operation, while orchestration is a collection of nested workflows that organize a department’s chores, or a complete media factory end-to-end functionality. Every step, in a workflow and in an orchestration, can be managed and potentially automated.

Following the Deming management methods, task assignment to and management of users and user groups allows for closer management of human steps. Setting a task list for the human operators and controlling the steps of their BPM workflow ensures consistency across the staff. Activities can be monitored and adjusted to meet and exceed key performance indicators (KPIs) as well as administer appropriate leadership to personnel though accomplishment analysis.

Automation brings about measurements, reports, and metrics, allowing managers and executives to customize reports to their business goals. Dashboards can be run by external data-mining tools monitoring the database or display reports run every few seconds to update the data. Most orchestration workflow software has built in escalation processes which use business logic or workflow failures to trigger procedures designed to draw attention to the issue, failure, or delay. Email messages, SMS texts, and Dashboard alarms are examples of escalation procedures in workflow systems.

As media companies work to build full orchestrations, they are often faced with deciding which of their business systems will be the financial system of record. Many companies are finding that slaving workflow engines to external systems, such as work order managers, traffic and billing systems, Rights Management systems, or Content Management systems (CMS), lets their leading business financial system act as the dominant system to control the operations and personnel workflows. Tying their business systems to the workflow engines and MAM databases allows for deep data mining and analysis, providing unique insights to the business. Additionally, some companies use customer and client portals with search/retrieve/publish workflows dedicated to the behind the scenes operations, all tied together with a financial system for reporting and billing so they can maximize every opportunity to sell their content and assets.

In 2017 Artificial Intelligence and Machine learning began to be integrated with workflow engines to reduce labor and create more efficient, faster operations. These tools can be linked to social media and ratings systems to provide feedback and better tune their operations. Examples of AI in use in workflows today are:

  • Automated metadata annotation on ingested assets

  • Acquisition of external metadata to apply to the media (EIDR data, IMDb, etc.)

  • Program rating performance influencing live production; and

  • Live sports event EDL workflows to automatically capture, clip, and disseminate media to fan platforms and social media.

The key to successful asset media workflows today is automation and orchestration and a consistent plan for continual improvement based on statistic and measurement.

Cloud Impacts

The joke passing around a 2017 SMPTE conference was “the cloud is just someone else’s infrastructure.” While this was meant as a disparaging comment, it actually highlights a benefit of public and private cloud operations. In a recent IABM conference presentation, the VICELAND network described their real-world experience of implementing from their London headquarters an entirely new network linear play-out channel on the other side of the world, in New Zealand, in only six weeks’ time from start of project to live broadcasts. New technologies are disruptive, and the public and private cloud technologies are re- inventing business models worldwide. Our media industry is no different – the Internet and IP infrastructure has evolved to reliably support viable video quality of service requirements for broadcast and media distribution, as well as a host of production services. This is a boon for our media workflows.

While this technological revolution has increased the direct to consumer options and is quickly changing the economics of our industry, many of the benefits of cloud computing are variations of already tried and true methodologies used around the world. For example, European broadcasters have been outsourcing their transmission services to carriers and system providers for years, and this model has proven to be financially viable – cloud infrastructure is an extension of the framework that the service providers already manage with service level commitments to nearly 100% availability. As United States broadcasters and media companies look to better control costs and increase flexibility in their offerings, the cloud holds great promise.

According to Accenture, there are four potential benefits of cloud computing for broadcasters:

  1. Faster speed to market, which closes the gap on the faster service delivery cycle of the OTT entrants;

  2. Scalability to handle spikes in workload, including live events, and surges in the popularity of new services;

  3. The ability to collect, store, and conduct analytics on vast amounts of data, generating insights to drive personalization, service development, customer experience, and one-to-one relationships; and

  4. Driving on-going service innovation through agile development, constant iterative experimentation, and a culture of “fail fast and fail cheap – then move on.”

Media owners and broadcasters are faced with significant business model changes that drive cloud computing as a necessity: Consumers expect more choices and demand more customization, the modern broadcaster needs to react to these demands quickly and confidently, and the need for scalability of computing power that only a virtualized and extensible computing platform can provide. Fragmented viewing audiences are triggering changes in content preferences and our industry is adapting by embracing flexibility in managing complex business and revenue models to maintain and grow engagement across demographics.

Everybody wants everything yesterday – consumers “binge watch” every episode of a new series and quickly demand another season. The reality of these time pressures underlies the content delivery market today. Media workflows must reduce production and distribution cycles to reduce time to market for new assets and experiences, as well as consistently increase their existing libraries of content.

Cloud computing services are typically a combination of utility computing, storage, and network bandwidth with no regional limitations to implementation, hence the confusing and sometimes misused moniker, “cloud.” Because our industry is not new, we are adapting on-premises workflows to the new infrastructure, and the jargon most often used to describe the cloud operations is “Public,” “Private,” and “Hybrid.”

Public cloud infrastructure is typically utility services that are used by businesses to buy computing, storage, and bandwidth on a fixed monthly rate or on-demand. Some more mature cloud services companies have been offering services on-demand, including transcoding, versioning, editing, and distribution workflows. The designation of “Private” cloud refers to corporations that want or need to manage their own data and computing processes in their privately purchased environment, restricting access and maintaining tighter security access. The word “Hybrid” has come to have varying meanings in our industry, depending on a particular individual’s design, either a mix of private and public virtualized environments, or a mix of on-premises hardware and software systems, local virtualized infrastructure and public cloud services. One clear fact emerges from the market research on cloud computing in our industry, the most common cloud deployment in media workflows is the hybrid approach.

Content drives our workflows and the type of infrastructure does not change this core fact – the key to a successful move to a cloud-based operation is to ensure that your management team can rely on consistent and predictable financial savings and the security of your media, while taking advantage of the great flexibility and scalability the public virtualized infrastructure. Adoption of cloud has taken two paths: the first is the use of web scale suppliers such as Amazon Web Services, Google Cloud Services, and Microsoft Azure for hosting and delivering media. The second level of adoption is leveraging the cloud stored content for virtualized production, post-production versioning, and distribution workflows. Today cloud clients are adapting their workflows to manage the content flow from creators to distributors and broadcasters, move content to post-production services, post the materials and assets to specialized distribution platforms and, of course, support the growing number of direct to consumer options for distribution.

Start-up companies, like the VICELAND IABM case study referenced at the beginning of this section, are the greatest beneficiaries of a globally coordinated infrastructure leveraging Just-in-Time scale up and down operations. Other large broadcasters and media owners are employing cloud computing in response to market dynamics for “pop-up” channels, new or temporary workflows, and to address customer demand. These large corporations are finding innovative ways to compete with their nimble OTT competitors and use the super-fast cloud deployment speed to maximize their competitive advantages. There is no avoiding the fact that surviving in today’s media world requires a global mind-set, and has become an increasingly complex, expensive business. Teams distributed worldwide, thousands of content distribution outlets, diverse technology options, and the ever-increasing fragmentation of viewership are driving tremendous challenges in managing cost-effective global operations.

Broadcasting flexibility and cloud-based linear channel playout are now proven workflows. The public service providers have demonstrated their environments’ scalability to quickly adjust virtual infrastructure to the ever-changing requirements for digital content services, often permitting content rights holders the chance to cash in on overnight sensations where popularity for specific content products drives demand. While cost pressures in our market seem to intensify, driven by the entry of agile low price players, increasing rights costs and shortened technology life-cycles making expensive IT hardware obsolete, an interesting development is that “cost” of cloud services has become less important when compared to other benefits, especially the improvement to delivery mechanisms, and offering today’s media executives the opportunity to experiment with new ventures without incurring capital expenses and the opportunity to quickly shut down a trail that doesn’t meet expectations. Broadcasters and media distributors are using the public cloud to avoid upfront technology investments and align costs with revenues and cash flows. With the uncertainties of today’s industry, it’s a very appealing management decision to employ a “pay for what you need, when you need it” subscription model that cloud offers.

Big Data and the Internet of Things is now of increased concern for today’s modern broadcaster, and with the application of Artificial Intelligence and Machine Learning the storage and analysis of the tidal wave of data will drive cloud applications. More than just program ratings, content personalization, fan engagement, support for niche audiences, enhanced user experiences and the engagement that creates a one-to-one relationship with consumers will better connect our media companies to their revenue source. And most, if not all, of these services will be automated workflows based in cloud infrastructure to take advantage of the massive computing power needed for analysis.

Asset workflows in the cloud bring measurable benefits. The tapeless operations we have been building over the past decade can quickly adapt their workflows and libraries to the cloud infrastructure. Cloud services can lower the on-going cost of these functions. Live event coverage and the cost of remote operations, challenged by increased rights as well as production costs, will drive sports broadcasters to embrace IT resourcing in less investment- intensive cloud computing platforms. Video streaming and on-demand services are maturing, finding more ways to leverage richer content and using cloud workflows to better engage and attract new consumers. And a chief benefit of employing cloud computing is using the homogenizing efforts of the cloud suppliers to take advantage of the increasing compatibility of the platforms, allowing media companies to profit from rich content across all devices, smartphones, tablets, hybrids, and televisions.

Notes

  1. 1 Nielsen’s 2014 Advertising & Audiences Report.

  2. 2 https://www.digitalproductionpartnership.co.uk/publications/theme/quality-control/.

  3. 3 A QC profile is a collection of QC checks (and relevant configuration parameters). Users use such profiles in QC products a bit like a ‘macro.’ It saves them specifying the same settings over and over again, and it limits the room for errors. An example is using a specific ‘Ingest QC Profile’ to verify incoming content complies with the media company’s own delivery specification.

  4. 4 https://ebu.io/qc (CC-BY 4.0).

  5. 5 The original EBU.IO/QC data model was inspired by FIMS QA, and inspired BXF QC.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.191.228.88