Portable Document Format: Visual Effects in a PDF

The Adobe PDF (Portable Document Format) is utilized by Adobe Acrobat Readerand is used around the world for publishing rich media documents, which can include interactive 3D, digital videoand digital audiocontent, as well as digital imagery, digital illustration, or digital painting. I have new media fundamentals titles on all of these areas at www.Apress.com . Acrobat Reader is free, and its PDF data format has been open sourced! The Adobe Acrobat Professionalseries of publishing tools are still paid software packages, and are well worth the money if you need to publish via this widely accepted, rich media document publishing format. This PDF format supports many digital video formats, including MPEG-4 AVC H.264 and Windows Media Video (.WMV), as well as Shockwave Flash (.SWF), Flash Video (.FLV), Microsoft AudioVideo Interleaved (.AVI), RealMedia (.RAM), 3GPP Movies (.3GP), Advanced Streaming Format (.ASF), and QuickTime Movie(.MOV), which we’ve used during this book to output VFX content.

Digital audio is also supported in PDF publishing and includes similar formats, such as MPEG-4 Audio (.M4A), Windows Media Audio (.WMA), Windows Wave (.WAV), MPEG-3 Audio (.MP3) or Audio Interchange File Format (.AIFF), which is used in Apple products.

It used to be the PDF format was only used for creating text-based business documents. However, it has now been adopted as an e-Book format, so it needs to support new media elements. As a matter of fact, you may be reading this book using this PDF file format right now. Other popular e-Book formats include EPUB (EPUB3) and Kindle 8 (MOBI), which we will be covering later on during this chapter after we cover the more widely used open standards of PDF and HTML5.

Another significant advantage of PDF document publishing is that it offers Digital Rights Management(DRM) support. This is why I covered this format first. DRM allows you to copy protect (or lock) your document if you want to sell it for money. It is important to note here that HTML5, which we’re going to cover in the next section of this chapter, is going to feature a DRM model at some point in the future, since HTML5 is being widely utilized as an operating system and for HTML apps.

Adobe also has PDF server products that incorporate DRM features. These allow publishers to market their PDF content in a more secure fashion, allowing better monetization opportunities for digital video content projects and published IP assets.

Another thing to note about PDF, EPUB3, and HTML5 apps is that they are “encapsulated” and require no server to function. I call this a 100 percent “client-side” approach. This approach allows you to sell your digital products much like they were physical products. Purchase the product and it’s yours, just like a real product, and you can give it away, if you wish, but cannot copy it due to more and more advanced DRM.

Let’s look at HTML5 next.

HyperText Markup Language: HTML5 Digital Video

You’ve already taken a look at how to use MPEG-4 AVC H.264 and QuickTime MOV format during this book. HTML5 supports the MPEG-4 digital video data format as well as WebM. Using plugins, HTML5 could also support other digital video formats, like QuickTime Movie or Windows Media Video. HTML5 digital audio support is more extensive than its digital video data format support, and can also be extended (using plugins) to add additional digital audio data formats. If you’re also interested in digital video, you can read Digital Video Editing Fundamentals (Apress, 2016).

It used to be that HTML5 was only used for creating website designs. Recently, web browser manufacturers decided to utilize their web browser software (code) to create HTML5 OS versions for use on consumer electronics devices. They did this due to the success of Google’s Android, Samsung Bada, Apple iOS, and Research In Motion (RIM) Blackberry.

Putting this browser code, with app launch icon support, on top of the Linux OS Kernel, produced a Chrome OS(Motorola), Firefox OS(Panasonic iTV), and Opera OS(Sony Bravia iTV Sets).

There’s also Tizen OS(Samsung), which is managed by The Linux Foundation for the creator of the Linux OS, Linus Torvald. Tizen also utilizes HTML5 code, as does the Jolla Sailfish OS.

HTML5 is simple to implement under any platform, thanks to the open source WebKit API, which is also a part of Android Studio 2.2 (currently at Android OS 7.0, and soon Android 8.0).

HTML5 application and website publishing is therefore an excellent way to deliver digital video content using all of the embedded mobile OS, desktop OS, and web browser platforms. With HTML5, you can even make your digital video assets interactive!

This is why DRM is in the future of HTML5. HTML5 will be freely available to you forever for implementing visual effects assets with MPEG-4, WebM, PNG, JPEG, GIF, SVG, HTML5, CSS3, XML, JSON, AJAX, AJAJ, XSLT, Java 9, JavaFX, JavaScript, and similar open source languages, data formats, and publishing technology.

Many of the other formats (or platforms) we are going to be covering during this chapter are also supported under HTML5, so things are not as “clear cut” as you might think regarding a “where and how do I publish” decision. For instance, the Kindle reader runs “underneath” HTML5 browsers or OSes, and the JavaFX game engine found in Java 7, Java 8, and Java 9 is also supported under HTML5 browsers, and soon will be supported in HTML5 OSes.

Going another direction, the EPUB3 format supports HTML5 inside of its data format, so your HTML5 code could not only be parsed by HTML5 OSes and browsers but also by EPUB3! Kindle and Android also support “parsing,” or rendering, your HTML5 content.

Next, let’s take a closer look at the open source EPUB3 publishing standard used for e-Books—and soon, for much more.

Electronic Publishing: EPUB Digital Video Support

The EPUB specification is a distribution and interchange format standard for digital publications and documents. EPUB3, the third major release of the open EPUB standard, consists of four specifications, each defining an important component of an overall EPUB document. EPUB Publications3 defines publication-level semantics as well as conformance requirements for EPUB3 documents. EPUB Content Documents3 defines XHTML, SVG, and CSS3 profiles for use in the context of your EPUB3 publications. EPUB Open ContainerFormat 3, or OCF3, defines a file format as well as a processing model for encapsulating sets of related resource assets in one single ZIP file format (EPUB container). EPUB Media Overlays3 defines a format and a processing model for the data synchronization of text with digital media assets.

EPUB3 has been widely adopted as a format for digital books, also popularly known as e-Books. This version significantly increases the EPUB format’s capabilities so that it can support a range of new media publication requirements. These include complex layout, new media and interactivity, and international typography (fonts) support.

The hope is that EPUB3 will be utilized for a broader range of content, including e-Books, magazines, and educational, professional, artistic, as well as scientific publications.

EPUB3 supports digital video data embedded in documents using MPEG-4 AVC H.264 and ON2 VP8 WebM, which is now owned by Google and has been open sourced. It also supports your digital image and digital painting assets using PNG, JPEG, and GIF, and digital audio is supported using MP3 and MPEG-4 AAC data format as well. Digital illustration is also supported using the open source Scalable Vector Graphics, or SVG, vector data format.

EPUB3 should inherit the same function and feature sets that these data formats can provide your digital video in HTML5 as well as in Kindle, Java, JavaFX, Android 6.0, or other WebKit-capable content publishing and distribution platforms.

Another impressive new media feature in EPUB3 is called media overlay document. The media overlay document provides you with the capability of synchronizing your digital VFX creations with vector elements inside of the publishing content document.

Amazon Lumberyard: Royalty-Free AAA 3D Engine

Amazon recently acquired the Crytek AAA game engine code and created a Lumberyard AAA game engine , then made it free for VFX developers—and royalty free as well. You do have to use Amazon Web Services, which are best in class, and which you would most likely be using anyway, so this is not a big issue. Figure 20-1 shows the incredible quality level an Amazon Lumberyard project offers. This matches up perfectly with Fusion’s quality level, so the two products will work together perfectly in your 2D and 3D visual effects digital content production pipeline.

Figure 20-1. The Amazon Web Services Lumberyard web page

If you are interested in producing interactive 3D (i3D) content using Lumberyard and related 3D products, such as Daz Studio Pro and Allegorithmic Substance Painter.

Amazon Lumberyard supports the same .FBX i3D file format that we used in this book, and so using Fusion with these tools should be possible, if not probable, for i3D project pipelines and workflows if you are interested in this area of new media. You can find more information at http://aws.amazon.com/Lumberyard .

e-Book e-Readers: Kindle Fire, Android, Java, or PDF

The e-Book e-Readerhardware device is actually an Android tablet, which is why I added Android into the title for this section of the chapter. The world’s most popular e-Book e-Reader, Kindle Fire, runs Android OS, as do the Sony e-Book e-Reader and the Barnes and Noble NOOK. Even the Apple iPad runs Kindle, EPUB3, and PDF e-Book titles, as do Blackberry tablets and Microsoft Surface tablets. The reason I added Java in the title for this section is that Kindle has Java capabilities for interactive e-Books, and Android uses Java as well. Since e-Book e-Readers will also read PDF files, I added PDF into this title as well.

Since most e-Book e-Readers are actually Android tablets or iPads, there are a plethora of platforms for delivering visual effects using MPEG-4 H.264 and OpenGL 3.

This means you could deliver visual effects content with Android applications, HTML5 applications, Java applications, HTML5 websites, Kindle Apps, NOOK Apps, or 3D new media PDF documents. This will give you lots of flexibility in publishing digital video content for e-Book e-Readers using the MPEG-4 AVC H.264 format or OpenGL ES 3.1. E-Readers are one of the best-selling new consumer electronics devices, especially during the holidays, along with the brand-new HD and UHD iTV set products.

iTV Sets: Android TV, Java, JavaScript, and HTML5

The iTV set, or interactive television set, is the most recent consumer electronics device to hit the marketplace, and iTV set devices are expected to explode in sales during 2017, 2018, and 2019. This is the reason Google has developed a specialized version of the Android SDK (Software Development Kit) for iTV sets called the Android TV API (Application Programming Interface).

There are HTML5 OS iTV set products as well from Samsung (Tizen OS), Panasonic (Firefox OS), and Sony (Opera OS), so the iTV set consumer electronic device is much like an e-Reader device in that it will allow you to create and deliver digital VFX content with Java or JavaFX (Android, iOS, HTML5, Windows), HTML5 markup, CSS, or JavaScript (iOS, Android, HTML5, Windows).

It’s also important to realize that with iTV set devices viewers are going to be paying closer attention to content streams, including visual effects, digital audio, interactive 3D, digital illustration, digital imagery, and digital paintings.

This level of close attention to your content is not always the case with other devices, such as smartphones, tablets, smartwatches, or automobile dashboards (at least, let’s hope not).

If you wanted to deliver the digital video content asset across each of the iTV set device platforms, you should utilize HTML5. Android and iOS support HTML5, but HTML5 OS and websites do not currently support Android 5, 6, 7, or iOS applications.

The other side of that decision is that Apple and Google Play have more advanced app stores; therefore, if you’re going to monetize your digital video and digital audio content, you should consider developing apps using Java (Android) or JavaFX (iOS) rather than using JavaScript or ECMAscript under HTML5 OSes or HTML5 browsers. HTML5 OSes do fully support MPEG-4 and WebM for digital video as well as for digital audio. HTML5 also supports OpenGL ES 3 via WebGL2, so it is catching up with Windows 10, Android 7, Linux, iOS, and all the popular game consoles, since every device these days has a GPU.

Smartwatches: Android WEAR 2, Java, and HTML5

The smartwatch is the next most recent consumer electronics device genre to hit the market. Smartwatch devices are also expected to explode in sales during 2017 and 2018, primarily because there are hundreds of manufacturers making them. This is because the densely populated watch industry is now moving to release smartwatch products so that they do not lose market share to consumer electronics manufacturers, such as LGE, Sony, Motorola, and Samsung, who already have several smartwatch products each. One of the first custom Android APIs that Google ever developed was Android WEARalong with its Watch Faces API. Android 7 (Android Studio 2.2) will support the new WEAR 2 API.

Visual effects may be a popular feature for these smartwatch devices to support, because raster and vector formats can be optimized from a data-footprint standpoint, as I have often said in my writings.

Additionally, MPEG and OpenGL support is built into Android 7 Wear 2 hardware devices. What this means is that a smartwatch product is like a visual effects timepiece for the user’s wrist and can provide professional digital video, 3D, or digital audio asset playback results with overlaid smartwatch functionality.

This is significant for visual effects content as well as digital audio content production, and for digital video application development professionals.

Another important feature of smartwatches is that you’ll be able to combine visual effects assets with other interactive functional attributes, such as time, date, weather, fashion, and health features popular with smartwatches, like fitness and physical health monitoring (heart, pulse, etc.).

Once smartwatch screen resolutions go up from 480 pixels to 640, 800, or 960 pixels, even more functionality will become available to developers. The Huawei smartwatch already features a 400 x 400 pixel screen. LG’s Urbane 2 is going to leapfrog this with a 480 pixel screen! High-resolution smartwatches should be appearing during 2017 (or 2018), given smartphones already have 4K-pixel screens that only span five inches; an 800-pixel smartwatch screen is certainly possible, as the technology exists already.

VFX software applications such as Fusion 8 are perfectly suited to producing smartwatch faces and applications, as they have the advanced feature set needed to produce the radial, or round, artwork assets required to fit the largely round OLED displays used in some of the most popular smartwatches from Motorola and LGE. Pixel density for these screens is 350 dots per inch, which is a higher level of quality than most laser printers, so your VFX artwork should appear razor sharp, and would be photorealistic as well.

Smartphone and Tablet: Android, Java, and HTML5

Smartphones and tabletshave been around the longest, as has the hybrid between the two, commonly referred to as a phablet. The Android OS is used for all of these device types as well as personal computers that run the Android OS. There are currently billions of smartphones, as well as billions of tablets, and almost one hundred major consumer electronics manufacturers have made products for the open source Android operating system platform.

For this reason, these devices represent a significant opportunity for a visual effects artisan to create digital video and i3D content and applications based upon both of these i3D and VFX genres. This is because there are not as many digital video VFX applications as there are utilities, gaming, digital audio, and digital imaging or visual social media applications.

All popular smartphones and tablets include support for the MPEG-4 and WebM video formats, SVG command data, and SVG data file formats. Modern-day devices can render vector data in 2D or 3D into raster image data, which fits user device screens with pixel-for-pixel precision and can be overlaid on videos using algorithms similar to those utilized in Fusion.

Present-day i3D content publishing platform capabilities will therefore encourage very high-quality visual effects using HD digital audio, thus increasing i3D content consumption. This will allow digital videos, digital images, and digital illustrations to be combined, not only together but also with other new media genres like digital painting, digital audio, speech synthesis, and i3D. This means that you can create a “never before experienced” UI, or user interface, design and amazing user experiences with your digital video editing projects and special effects pipelines.

One indicator that consumer electronics devices have the GPU, CPU cores, and system memory to process the type of VFX we’ve been learning about in this book is that the Amazon Lumberyard AAA game engine actually delivers i3D applications for both iOS and Android mobile (phones, e-Readers, and tablets) platforms.

Game Console: Lumberyard, Android, Java, HTML

Since Android, Java, JavaFX, and HTML5 now support OpenGL ES 3.2, a plethora of advanced game-console products have appeared that are affordably priced between $60 and $180. This is yet another opportunity waiting to happen for the digital visual effects artisan, which you will be once you practice what you have learned in this book. The game consoles run Windows, Linux, or Android, and therefore support Lumberyard, Java, JavaFX, and HTML5 applications—and even e-Books, for that matter. There are over a dozen affordable game consoles currently on the market, and the type of VFX content created in Fusion is a perfect fit!

Some major industry brands are producing game controllers with Android 7 computers inside; for instance, an nVidia Shield or GameStick. Other major manufacturers, such as Amazon, manufacture a game console–iTV set hybrid product, such as the Amazon Fire TV. Others such as OUYA and GamePop make STB (set-top box) products that game controllers (and an iTV set) will plug into. Some, such as OUYA and Razer ForgeTV, come with both the STB and the game controller for a complete gaming package. Android 7 also features OpenGL ES 3.2 and the new Khronos Vulkan API as well.

Since all of these support Android, you can utilize digital video using MPEG-4 or WebM for digital video–based VFX projects, as well as vectors using SVG XML formats and SVG commands.

Visual effects project assets and special effects can be utilized inside the OpenGL ES 3.2 rendering engine, which renders many of today’s most popular i3D games. The visual effects asset can also be used for advanced animated 2D texture-mapping effects.

Texture maps are applied as “skins” to 3D mesh geometry, and thus 3D can take your career to all new levels as you can now work as a texture-mapping producer for 3D filmmakers, i3D game makers, and 3D television producers.

Texture mapping can be done by using open source i3D software packages, such as Blender 3D. You can download Blender 2.78 for free for Windows, Linux, or Mac at http://www.blender.org today and start to get up to speed on bridging digital video with 3D!

Future Devices: Robots, VR, and Home Appliances

The future of Android SDKs will surely bring more custom APIs. I expect to see Android VRfor virtual reality goggles, as well as Android HOMEfor home appliances or home control units, and maybe even an Android ROBOTSDK for Android-based robots. I have already seen many of these product types in the marketplace, so it’s up to Google to provide custom APIs in these product genres, all of which can be great visual effects as well as digital audio platforms. The platforms will provide all-new opportunities for visual effects designers, digital video editors, and i3D artisans, as well as for all multimedia producers and application developers who are digital video editors, 3D modelers and animators, or digital video effects artisans. The future is indeed bright for 2D and 3D visual effects artisans!

Digital video and digital effects will be important components in all of these emerging device genres. I’d expect that at least two of these genres, home appliances or VR, will showcase interactive visual effects as a way to increase user experience levels, because UHD home theaters will have the full attention of a viewer, comfortably seated upon their couch. Interactive 3D should be another area of increasing popularity.

Apple iPhone and iPad: Supported Media Formats

Apple Macintosh and iOS support MPEG-4 H.264 AVC , as do each of the other popular platforms and devices in the world. Apple also supports its own proprietary QuickTime Movie format, as you might have expected. This is also used in Fusion 8, but it does not have the highest level of quality, as you will see in many of the Fusion 8 discussion forums. Apple OSes also support i3D using OpenGL ES 3, and therefore will also render interactive 3D content. Apple also has a proprietary 3D engine called Metal, but content optimized for this only runs on Apple hardware, not on the other one hundred manufacturers’ products.

Apple applets support the MP4 and MOV digital video data formats. The MPEG-4 format is also currently being used in Java, JavaFX, Android 7.0, Kindle 8, HTML5, CSS3, EPUB3, JavaScript, PDF, XML, JSON, Windows Mobile, RIM Blackberry, and many others.

Windows Phone: Supported Digital Media Formats

As a proprietary format, Windows and WinPhone do not directly support WebM and SVG, as all of the other platforms and devices in the world do. WebM support was added in Internet Explorer 9, and there is an extension you can get for Microsoft Explorer to render SVG file thumbnails. As Microsoft and Apple represent an increasingly smaller operating system market share percentage as time goes on, SVG file format support issues will become less of a problem for digital painters, digital illustrators, audio or video editors, visual effect designers, and i3D artisans.

Additionally, open source platforms are far easier to obtain and to qualify, and to post applications and content to. Open source platforms continue to be adopted more rapidly by second-tier as well as third-tier consumer electronic device manufacturers due to lower costs and the freedom of intended usage. This is the reason open source has taken over software.

Open platforms are also far more widespread in third-world countries, because they are used on the affordable devices and hardware platforms owned there. A great example of this is the Mozilla Firefox OS platform, where the user’s hardware device plus OS will typically cost less than $25 US (or that country’s currency equivalent).

This gives i3D VFX developers a far more widespread user base to work with and ensures that more people have access to their work. The trend toward open platforms benefits everyone.