This book was written to serve as a guidance to assist the evolution of media streaming, eXtended Reality (XR), and metaverse services (www.coursera.org/learn/ar-technologies-video-streaming). Media streaming and metaverse services share many core technologies in which each can provide a synergistic effect to the others’ future evolution and growth.
Future technical advancements in XR devices will significantly help metaverse services and massively multiplayer online games (MMOGs) to improve. At the same time, advanced XR devices will also enable multimedia streaming services to reach new levels in 3D and holographic immersive presence media content.
It is expected that standalone XR devices will become lighter, sleek in design (fashion XR glasses), and more powerful in functionality, such that they will become much more popular in the market. At the same time, wireless networks (based on future 6G, Wi-Fi 7, etc.) and edge computing will become faster such that the wireless tethered (non-standalone) XR devices will also become very popular.
My personal desire is to help technology evolution toward a very light and cool XR smart device in eyeglasses form that is standalone, which switches to wireless tethered mode only when needed, and supports real-time metaverses, MMOGs, and multimedia streaming services with charge intervals exceeding 72 hours even under heavy usage.
The chapters of this book show that the core technologies applied to metaverse XR and multimedia systems are mostly the same and the networks and cloud computing requirements are very similar. In addition, considering that metaverse XR services are in search of new creative socializing themes, and multimedia streaming services are in need of new and unique content for their platforms, the merge of metaverse XR services and multimedia streaming services seems like a win-win make, which is one of the purposes of combining these two massive topics into one book.
Metaverse XR Technology Introduction
Metaverse XR Products and Business
Video Streaming Technology Introduction
Video Streaming Services and Business
All dollar “$” figures are based on the US dollar (USD).
Metaverse XR Technology Introduction
The term “metaverse” was coined from the Greek word meta (which means “beyond”) and “verse” (which comes from the word “universe”) by the American novelist Neal Stephenson in 1992 based on the science fiction novel titled Snow Crash (www.randomhousebooks.com/books/172832/). The term “metaverse” was used to describe what a virtual reality (VR) society could provide. The metaverse would be a digitally created virtual world that people could use avatars to work, play, and socialize. Metaverse was described as a future successor of the Internet, which could serve as an escape from the dystopian reality. The metaverse virtual society could be used to avoid suffering or injustice in the real-world society and enable a fresh new start in socializing through avatars. The realization of metaverses has already started, where it is used for new types of socialization, exploring, traveling, work, entertainment, games, and education. Although there may not be a dystopian reality that everyone wants to avoid, metaverses can serve as an escape from the dysfunctional or unsatisfying aspects of one’s society by entering a virtually created environment that could be customized to satisfy personal desires and be less stressful. One benefit of a metaverse is that it is a virtual world based on digital VR designs. So, a metaverse can be erased, modified, or rebuilt when needed, mostly requiring computer designing and programming efforts as well as one’s creativity and investment. Social value of a metaverse is added when multiple participants start to join and interact, forming a virtual society. When such a virtual society is formed, then the metaverse value increases, and from then, changes must be made more carefully such that the metaverse social ecosystem and values are maintained. When social values of a metaverse are not maintained, users will leave, thereby decreasing the size of the society and its total value. In some cases, inappropriate users may be forced to leave the metaverse to maintain a sufficient level of social values (https://youtu.be/ydWYJSpsCLE).
Metaverse and VR were first mentioned at a time when the Internet was still in its early stages. The origin of the Internet began in 1966 based on a project operated by the Advanced Research Projects Agency (ARPA) of the US Department of Defense (DoD). This project was focused on creating a military network called ARPANET, which grew extensively in functionality, size, and coverage. Extending from the military ARPANET, the modern Internet was formed in the early 1990s, which is when various enterprises and academia were connected through commercial networks, which triggered an exponential growth on the Internet in terms of content and services (https://en.wikipedia.org/wiki/Metaverse).
Recently, advancements in computing hardware, video display technology, content delivery technology, artificial intelligence (AI), VR and augmented reality (AR) software programming, head mounted display (HMD) devices (e.g., VR and AR goggles, headsets, or glasses), digital twin technology, edge computing, wireless and wired networks, development of Web3, smart devices, 5G and 6G mobile communication, and Wi-Fi and Bluetooth networking, as well as the COVID-19 pandemic, are all reasons why metaverse services and XR technology continuously grew.
As metaverse users are rapidly increasing and wider age groups start to use metaverse services in forms of massively multiplayer online games (MMOGs) or social network media, it is significantly important to properly deal with user safety and information privacy. In addition, user addiction issues related to mental and physical health as well as time waste need to be considered.
Currently, most metaverse services are based on VR user environments. This is mainly because many types of MMOG applications have been using VR technology to enable player avatars to compete in virtual game environments, which may include multiple domains, worlds, and universes. MMOGs are online video games that enable many players to simultaneously participate individually or collaborate in teams while competing in the same game environment. MMOG users rely on the Internet for worldwide connectivity, and they depend on the game server to provide a fun experience and fair competition and protect the user while providing privacy. All these features are expected from a metaverse as more personal activities are to be conducted in the metaverse while socializing. MMOGs require high-quality networks, game servers, high-performance player computer terminals, as well as virtually created multiple game domains, worlds, universes, and dimensions. Thus, MMOGs can be considered as one of the original purposes to create and use a metaverse.
However, it is predicted that XR will be used in creating future metaverse services, as XR technology can enable users to experience both real and virtual environments by interpolating mixed reality (MR) (which combines AR with VR) with various haptic user interfaces to make the user’s environment more interactive, realistic, broader, diverse, and immersive. New XR application programming interfaces (APIs) can enable various types of human-to-human, human-to-animal, human-to-machine, and machine-to-machine interaction.
Future metaverse services will need advanced XR equipment as well as high-performance wired and wireless networks to support the required data rate, time delay, and reliability requirements. Advanced metaverse services require high levels of Quality of Service (QoS) and Quality of Experience (QoE) to enable the XR system to provide a satisfying level of virtual immersive presence.
Based on this focus, new standards on XR equipment and networking services are being prepared by global organizations, like the 3rd Generation Partnership Project (3GPP) and the Institute of Electrical and Electronics Engineers (IEEE). Standards provide a way for new products and services to interoperate, safely be used, while satisfying the basic commonly required functionalities.
Just like MMOGs, metaverse systems can be supported by many types of networking platforms and be played using a personal computer (PC), video game console, smartphone, HMD, as well as various Internet of Things (IoT) wireless devices. Therefore, metaverse applications must be designed considering users in diverse environments using various interactive XR interfaces. The social connection aspect of the metaverse is what makes it different from most 3D virtual world-based video games like MMOGs. Some noticeable metaverse services are described in the following.
Sandbox Games
Sandbox games are video games that enable a player to freely explore options and the environment, select options, and create virtual features with or without any predetermined goal related to a game (www.merriam-webster.com/dictionary/sandbox). In addition, multiple player interaction and collaboration can be conducted in the sandbox. This unburdening personal exploration and socializing platform provided by sandbox games mimics a sandbox that children play in and socialize, which is where the name came from. From this aspect, the “sandbox” of sandbox games can be considered as an original form of the metaverse. Some of the early sandbox games are introduced in the following, which include Elite, The Sim, Habbo, Second Life, Minecraft, and more (https://en.wikipedia.org/wiki/Nonlinear_gameplay).
Elite
In September of 1984, Elite was released as one of the earliest sandbox-type games (www.frontierstore.net/games/elite-dangerous-cat/elite1984.html). It was focused on space trading and combat scenario competition. Based on a player’s performance, one’s combat rating could be increased, where reaching the exalted level of “Elite” could be achieved, which is where the name of the game originated from. Elite originated as a three-dimensional (3D) arcade game, so it originally did not have an online interface, nor did it have multiplayer trading features. Elite was originally programmed using assembly language, which is a machine coding software, which took 2 years of research and development. Elite was originally made for BBC Micro and Acorn Electron computers and was officially published by Acornsoft, which has its headquarters located in Cambridge, England. Elite contributed to initializing both the sandbox game era as well as advanced 3D graphics technology, as it was the first to enable wire-frame 3D graphics with hidden-line removal technology on home computer games. The Elite series of games are among the most successful and oldest ongoing video game franchises in history, which has significantly influenced many game models (https://en.wikipedia.org/wiki/Elite_(video_game)).
SimCity
In 1989, SimCity was released, which is a simulation-based city-building video game created by Maxis (www.ea.com/games/simcity). Maxis was founded in 1987 in Redwood Shores, California, USA. Due to the company’s great success in simulation games like SimCity, The Sims, and Spore, the company was acquired by Electronic Arts (EA) in 1997. SimCity started as a black-and-white two-dimensional (2D) graphics video game focused on building a new city as its mayor. The game’s objective is to build a city with the highest population based on a limited budget, which is based on taxes collected. A higher standard of living of the residents triggered the virtual city’s population to grow. As the city grew, it became more challenging to balance improvements considering each region’s environmental standards and issues while needing to continuingly develop higher-quality residential and industrial areas. Based on this high level of strategic planning involved in playing the game, SimCity contributed to breaking the barrier of thought that video games were only for the young and were not challenging enough for adults. SimCity was extremely popular and successful and won many accolades. However, due to the lack of multiple user avatar based social networking aspect of SimCity, some do not consider it as a true metaverse game. However, it is the origin of the urban simulation video game genre that is based on virtual environment strategic planning and designing, which is a metaverse feature. In addition, its spinoff “The Sims” has many aspects that significantly contributed to metaverse designs and technology, which is described in the following (https://en.wikipedia.org/wiki/SimCity).
The Sims
In February of 2000, “The Sims” was released which began the popular Sim game series, which are life simulation and social simulation-based sandbox-type video games (www.ea.com/games/the-sims). The Sims and its series were developed by Maxis and published by EA. Players would create their avatar (which are called “Sims”) and place them in existing homes, or players can create new homes. The original video games did not define any goals for the avatars to accomplish, thus socializing was the main purpose. However later expansion pack versions of the game included a gameplay-style factor to provoke more interaction among players. Worldwide, The Sims game series sold approximately 200 million copies and is one of the all-time best-selling video game series (https://en.wikipedia.org/wiki/The_Sims).
Habbo
In August of 2000, Habbo was launched by Sulake located in Finland, where it was formerly known as “Habbo Hotel” (www.habbo.com/). The service platform name and Internet domain name were changed from “Habbo Hotel” to “Habbo” in May of 2006. Habbo is a popular social world-building game that enables users to create their avatar and talk with other players. Habbo’s social network community was focused more on teens and young adults. On the social activity side, Habbo enabled users to provide caregiving for virtual pets and design and build rooms. On the gaming side, Habbo enables players to create new games and play games, which include completing team quests. By October of 2020, Habbo had accumulated 316 million avatars. In January 2021, it was reported that Habbo had over 800,000 monthly active users from more than 150 countries. Habbo has been classified as a metaverse platform, and due to the game activities included, some classify it as a massively multiplayer online role-playing game (MMORPG) (https://en.wikipedia.org/wiki/Habbo).
Grand Theft Auto III
In October of 2001, Grand Theft Auto III was released (www.rockstargames.com/games/grandtheftauto3). It was published by Rockstar Games (located in New York City, USA) and developed by DMA Design as a series following Grand Theft Auto II, which was released in 1999. Grand Theft Auto III is played by controlling the avatar Claude in Liberty City, which is a 3D virtual space that somewhat resembles New York City. The game received many praises and awards based on its 3D design, concept, and gameplay. In addition, it also received criticism due to its content related to violence, sex, and crime (e.g., grand theft auto). Grand Theft Auto III sold over 14.5 million copies and was the 2001 best-selling video game. Due to its 3D landmark designs and open world concept, it can be considered as an innovative sandbox game and contributor to metaverse technologies (https://en.wikipedia.org/wiki/Grand_Theft_Auto_III).
Second Life
In 2003, Second Life was launched by Linden Lab located in San Francisco, California, USA (https://secondlife.com/destination/metaverse-city). Second Life enables users to create their avatar that would live in an online virtual world, thereby enabling users to live a virtual “second life” by connecting to the application’s Internet platform. The popularity of Second Life grew, where it reached approximately one million users in 2013. The most recent upgraded stable release was provided in February of 2021. Second Life is different from a massively multiplayer online role-playing game (MMORPG) as it focuses on providing an online virtual social environment for user avatars but does not impose any required set of objectives for the users, nor does it create any manufactured conflict as games do. Due to these reasons, many consider Second Life as the first metaverse service realized (https://en.wikipedia.org/wiki/Second_Life).
Roblox
In September of 2006, the online game platform Roblox was released by the Roblox Corporation (located in San Mateo, California, USA) for use on computers based on Microsoft Windows (www.roblox.com/). Roblox enables its users to create new video games by programming. As new games are made, users can participate in games created by other users. This characteristic of Roblox served as a major contribution to the metaverse self-building features. The Roblox platform is based on the programming language Lua and hosts a library of multiple genres of online video games that are user-created. Although Roblox provides free games, in-game purchases can be made using the ROBUX cryptocurrency, which is a unique token used in Roblox. The popularity of Roblox significantly grew in the mid to late 2010s and especially during the COVID-19 pandemic. By August of 2020, the number of monthly active users of Roblox reached 164 million, which amazingly included more than half of all American children under 16. Roblox has received many positive reviews and accolades (https://en.wikipedia.org/wiki/Roblox).
Minecraft
In 2009, Minecraft was first made public in 2011 with its first full release. Minecraft was developed by Mojang Studios (in Stockholm, Sweden) as a sandbox video game (www.minecraft.net). Minecraft is considered as one of the best-selling video games of all time based on its 2021 status of having sold over 238 million copies and having nearly 140 million monthly active users through its publishers Mojang Studios, Xbox Game Studios, and Sony Interactive Entertainment. One characteristic of Minecraft is the virtual 3D world designed with blocky and procedurally generated terrain, which needs to be explored and in which new structures and earthworks need to be established. In Minecraft, players can modify their avatar, design new gameplay mechanics, as well as add new items and assets. Minecraft has several game modes. In the survival game mode, players use their avatars to obtain resources and build a virtual world, in which maintaining one’s avatar health while accomplishing multiple goals is the objective. In another game mode, players can use their avatar to join team-based or competition-based fighting games against computer-controlled mobs. Due to these aspects, Minecraft received many awards and is considered as one of the greatest video games of all time. The intellectual property rights of Minecraft and the company Mojang were acquired by Microsoft in 2014 for $2.5 billion, and since spin-off games of Minecraft have been released (https://en.wikipedia.org/wiki/Minecraft).
VRChat
In January of 2014, VRChat was released by VRChat, Inc. for the Oculus Rift DK1 prototype that was supported by Windows operating system (OS) computers (https://hello.vrchat.com). VRChat is based on a Unity engine and enables users to socialize using 3D avatars and worlds created by the users. The world and avatar creation capability of VRChat is similar to Second Life and Habbo Hotel. The VRChat platform has a versatile interface that allows various VR HMDs like the Oculus Quest and Rift series, Windows Mixed Reality (MR), and SteamVR headsets like the HTC Vive but also allows user access through a personal computer (PC) when using the “desktop mode” in which the mouse, keyboard, and/or gamepad are used for control. In VRChat, users are divided into “trust levels” and corresponding colors based on their experience of programming and creation on the platform. The beginner rank is the “Visitor” represented by gray. The next higher rank is the “New User” represented by blue. A “New User” can upload the content they have created using the VRChat software development kit (SDK). This Unity programming SDK was released to help creation and stable/secure importing of avatars as well as virtual spaces and infrastructure onto the VRChat platform. The higher ranks are “User” represented by green, “Known User” represented by orange, “Trusted User” represented by purple, and “Friends” represented by yellow. In addition, VRChat Plus was released in November of 2020, which enables up to 100 avatars to be saved in a user’s favorites (the earlier limit was 25), supports nameplates with custom images and invite requests with in-game photo attachment, and grants trust rating increases (https://en.wikipedia.org/wiki/VRChat).
Fortnite
In 2017, “Fortnite: Save the World” and “Fortnite Battle Royale” were released by Epic Games located in Cary, North Carolina, USA (www.epicgames.com/fortnite/en-US/home). Fortnite is an online video game that uses a common gameplay and game engine to support the three very popular game modes: “Fortnite Battle Royale,” “Fortnite: Save the World,” and “Fortnite Creative.” The “Fortnite Battle Royale” early access title released in 2017 became a great success by accumulating approximately 125 million players within a year. “Fortnite Battle Royale” is playable on various platforms such as Microsoft Windows, macOS, iOS, PlayStation 4, PlayStation 5, Xbox One, Xbox Series X/S, Nintendo Switch, and Android. In “Fortnite Battle Royale,” up to 100 players can participate in a last person standing free-to-play battle. “Fortnite: Save the World” is playable on various platforms such as Microsoft Windows, macOS, PlayStation 4, and Xbox One. In “Fortnite: Save the World,” up to four players can participate in a survival game using cooperative hybrid tower defense-shooters, traps, and fortifications to defend objects from zombie-like creatures. “Fortnite Creative” was released in December of 2018 as a sandbox game playable on various platforms such as Microsoft Windows, macOS, PlayStation 4, Xbox One, iOS, Nintendo Switch, and Android. In “Fortnite Creative,” players have full freedom to create new worlds and battle arenas. A significant part of the success of Fortnite games is based on the Unreal Engine developed by Epic Games. The Unreal Engine is a very powerful video game engine that is commercially available. The performance of the Unreal Engine earned official world-record recognition as the “most successful video game engine” from the Guinness World Records in 2014. The great success of Fortnite games resulted in a gross revenue of $9 billion by December of 2019. More details on the Unreal Engine will be provided in Chapter 2 (https://en.wikipedia.org/wiki/Fortnite).
Meta
Facebook rebranded its name to Meta in October of 2021, in which the company has invested a lot into metaverse technology and service development (https://about.facebook.com/). In December of 2021, Quest users in the United States and Canada were able to access Horizon Worlds. As of February of 2022, it was reported that 10,000 metaverse worlds have been created and there are 300,000 users. In addition, Meta’s creator groups consist of more than 20,000 members. This is a significant accomplishment within a very short period, which shows the interest in metaverse technology and services. Meta’s Horizon metaverses require a Quest headset for proper access. Considering that the Quest 2 VR HMD costs $299 or more, an investment from the user is required to actively participate. By the fourth quarter of 2021, Meta’s Quest 2 occupied 80% of the global VR, AR, MR, and XR headset shipment market share, which shows Meta’s dominant success. For virtual ware sales made on the Meta Quest store, a creator fee will be charged which includes Meta’s platform fee and a fee for using Horizon Worlds. This fee will apply to all metaverse products, apps, and games of Meta (https://en.wikipedia.org/wiki/Meta_Platforms).
Microsoft
Microsoft’s metaverse is focused more on enterprise-centric work, design, research, development, and education rather than social networking, entertainment, or games (www.microsoft.com/en-us/mesh). Microsoft’s Mesh MR collaboration platform will work with Microsoft’s Teams, in which both will be supported by Azure edge cloud technology (https://docs.microsoft.com/en-us/mesh/overview). Microsoft’s Mesh will provide virtual holographic MR systems that will be combined with Team’s virtual meetings, chats, and shared documents tools such that collaborative work and virtual presence sharing is possible (https://docs.microsoft.com/en-us/mesh/mesh-app/). Microsoft has accumulated experience from earlier projects that combined MR with Office tools, such as the “SharePoint Spaces” that was introduced in 2018. SharePoint enables users to create 3D virtual spaces to visualize and collaborate on product models and conduct data analysis (www.zdnet.com/article/microsoft-to-extend-its-enterprise-metaverse-strategy-with-mesh-for-teams/).
There are many more companies that have been providing metaverse services, where unfortunately not all can be described in this book. In the following, an overview of XR-, MR-, AR-, and VR-based metaverse technologies are provided, where more details will be provided in Chapter 2.
XR-, MR-, AR-, and VR-Based Metaverse Technologies
Virtual reality (VR) is the best technology to implement a metaverse as it can provide an immersive presence into totally new virtual domains where interaction in the metaverse is possible through an avatar (or parts of an avatar, like the user’s hands) and the metaverse design range is unlimited or limited to only the designer’s programming/design skills and creative capacity.
Compared to VR, AR requires additional projected augmented images and information that have to be accurately positioned to blend in with the user’s view of reality, which is a more difficult process. AR technology is based on the actual environment that the user is in, where augmented objects and/or information are added to provide information or conduct application-based functions. However, the actual environment view of the user is only partially augmented with objects and/or information. In order to be synchronized with the user’s actual view (which can quickly change), the augmented process has to be conducted very fast and has to comfortably match the environment of the user.
Mixed reality (MR) combines AR and VR services with seamless crossover capability such that the MR user’s actual AR service environment can transition into a virtual domain that is serviced by VR technology and seamlessly return from the VR domain to the AR environment as well.
XR adds various haptic interfaces to MR technology such that a more realistic experience can be obtained by the user. Various XR haptic interfaces are currently able to influence vision, hearing, and touch, which are three of the five human senses. In the future, the level of these three senses will be enhanced in many ways, and the missing two senses “taste” and “smell” may be added to XR haptic interfaces.
XR Implementation History
Head-up display (HUD) is commonly used on transparent displays/windows to allow the user to see the surrounding viewpoints/environment without requiring any additional head movement (https://en.wikipedia.org/wiki/Comparison_of_virtual_reality_headsets#Tethered_VR). On April 30th of 1958, the first HUD prototype was used in the Blackburn Buccaneer attack aircraft of the British Royal Navy. More HUD systems were developed for many other aircrafts in the 1950s and 1960s where flight instrument data and directional lines were projected on screens or cockpit windows so the pilot(s) could keep their “head up” while controlling the aircraft and not look down to read the flight control instruments. This is where the name “head-up display” came from. HUD is an original and practical form of AR technology.
In 1992, the Virtual Fixtures technology developed at the US Air Force’s Armstrong Laboratory is known to be the first MR system. In addition, projection mapping technology uses projectors to display graphical information or images onto physical objects. Projection mapping technology has been used for many purposes which include education, science, engineering, and military, and it is especially useful in architecture and art.
Ever since the first smartphone (the “iPhone”) was introduced by Apple in 2007, smartphones have been the most convenient platform to implement XR technology. This is because smartphones are a complete form of a portable computer that has numerous sensors, multiple wireless network interfaces, touch screen display, sufficient memory, high-performance computing processors (CPU, GPU, and SoC), high-resolution camera(s), and haptic devices. One of the earliest and most successful XR models implemented on a smartphone is “Pokémon Go” that was released by Niantic in collaboration with Nintendo and The Pokémon Company in 2016. Pokémon Go is an AR game for iOS- and Android-based smartphones. The GPS on smartphones were used to locate Pokémon virtual creatures which could be captured, trained, and put to battle the other Pokémon creatures. Pokémon Go was free to use as its business model was based on local advertisement and in-app purchases of various in-game items. XR technology combined with modern smartphones can implement simultaneous localization and mapping (SLAM)-based images and map creation. SLAM is a technology used to build a map using information or images collected from one agent roaming around or using a wireless network to collect information or images from multiple agents. SLAM uses the agent’s location and the sensor’s data or image of that location and combines all data to create an overview of the larger territory. The SLAM map can consist of sensor data and statistics and/or can be a combination of partial images to form a larger image map (picture connecting technology is needed). SLAM maps may have parts where data or images are missing. There are a variety of SLAM methods that are used in AR applications, such as parallel tracking and mapping (PTAM) technology (https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping).
XR headsets, HMDs, and eyeglasses are the most common types of XR devices, in which a list of various devices based on release date order is provided in the following section. XR devices (including all VR, AR, and MR systems) can be divided into “tethered” or “standalone.” Tethered XR devices require a computer (including laptops and tablets) or video game console to process the application where the user video information is sent to the XR headset wirelessly (commonly using Wi-Fi or Bluetooth) or through a wire connected to the computer or video game console. Wired XR headsets receive power through the wired line, so they do not need an internal battery. Standalone XR devices conduct all processing of the application in its system and therefore do not need a supporting external computer or video game console to support it. This is why tethered devices are commonly lighter than standalone devices and can be used longer. However, tethered devices that need a wired connection have a limited range of usage that is based on the length of the wire, and wireless tethered devices also require the user to be in the reliable range of the networked computer or the video game console. Due to this reason, standalone devices are expected to become more popular in the future. Some XR devices have been released as both tethered and standalone products. In the following is a list of XR devices ordered based on date of release.
Tethered XR Devices
In 2015, Razer OSVR HDK 1.4 was released.
In 2016, Oculus Rift, HTC Vive, Razer OSVR HDK 2, StarBreeze StarVR, and PlayStation VR were released.
In 2017, Fove 0, Pimax 4K, VRgineers VRHero 5K, Deepoon VR E3, Dell Visor, Acer AH101, HP WMR headset, Lenovo Explorer, Samsung Odyssey, and VRgineers VRHero 5K Plus were released.
In 2018, Asus HC102, HTC Vive Pro, VRgineers XTAL, StarVR One, Samsung Odyssey+, and Pimax 5K Plus were released.
In 2019, Varjo VR-1, Varjo Aero, Pimax 8K, Valve Index, HP Reverb, Oculus Rift S, HTC Vive Cosmos, Varjo VR-2, and Oculus Quest were released.
In 2020, Oculus Quest 2, HP Reverb G2, HTC Vive Pro 2, and Varjo VR-3 were released.
Standalone XR Devices
In 2015, Samsung Gear VR was released.
In 2016, Avegant Glyph, Pico VR Goblin, Qualcomm Snapdragon VR820, Alcatel Vision, Microsoft Hololens, and Helmet Vision were released.
In 2017, Pico Neo CV, Qualcomm Snapdragon 835 VRDK, and Woxter Neo VR100 were released.
In 2018, HTC Vive Focus, Oculus Go, Lenovo Mirage Solo, GFL Developer Kit, and Google Daydream View were released.
In 2019, Pico G2 4K and Oculus Quest were released.
In 2020, Oculus Quest 2 was released.
In 2021, HTC Vive Focus 3 and Pico Neo 3 Pro were released.
In 2022, HTC Vive Flow was released.
In the future, AR contact lenses and virtual retinal display devices could be created. As see-through electronics and miniature display systems have evolved, AR contact lenses are an attractive option that can have many useful applications. Among the earliest attempts, in 2013 at the Consumer Electronics Show (CES), Innovega presented a contact lenses prototype that worked with AR glasses. In 2020 at CES, Mojo Vision presented a contact lenses prototype that worked without needing AR glasses. AR contact lenses that function with AR spectacles are known to be under development for the US military. However, an AR contact lenses product has not been officially released yet. In addition, virtual retinal display AR devices have been under development for a while, but a product has not been released yet. A virtual retinal display device will display an image directly onto the retina inside the user’s eye. This technology could be used as a medical device that can help patients with visual disabilities (https://en.wikipedia.org/wiki/Comparison_of_virtual_reality_headsets#Tethered_VR).
Future metaverse systems are expected to evolve from the current VR-based interfaces into full XR-based systems, with hope that the content of this book will help to assist a successful and faster transition into XR-based metaverses. In the following section, details on metaverse and XR products and business are presented.
Metaverse XR Products and Business
The reason that metaverses are receiving much attention is not only due to the fun factor but also because there are many ways that a profit can be made, which include ownership, investments, events, device sales, creative design-based sales of virtual goods, platform usage fees, advertisement fees, and online store fees (www.youtube.com/watch?v=0HqV1cJ1Bdw). There are moneymaking methods suitable for large-scale and small-scale companies as well as methods that an individual can make a profit, which are explained in this section. Metaverses consist of multiple online virtual spaces that include virtual environments, buildings, infrastructure, transportation, and avatars, which each has its own individual value. These virtual objects can be designed, rented, sold, purchased, or invested through various forms of financial endeavors. For some virtual goods, if the original item has a unique value, then the original needs to be distinguishable to its duplicates/copies, and the buyer needs to be able to confirm that the purchasing object is an original, and its purchase needs to be verifiable and secure. This is where blockchain technology becomes very useful. Blockchain-based cryptocurrencies are commonly used to make purchases, with support of smart contracts and non-fungible tokens (NFTs).
Smart Contracts
A smart contract is a blockchain-based secure computer programmed contract that is capable of automatically managing and executing legal actions based on the contract terms (https://en.wikipedia.org/wiki/Smart_contract). Blockchains are used to secure the agreement of the contract terms which are unchangeable and confirmable through the blockchain records or a distributed ledger. Smart contracts are transparent and help to reduce the overhead cost of banks and legal fees needed to set up and execute contract terms. Without a smart contract, traditional financial contracts require trusted intermediators representing each company/firm, where fees for arbitrations and enforcement are significant and time-consuming. In addition, smart contracts rarely have any malicious or accidental exceptions occurring, and even if they occur, settlements are simpler because the contract execution process is easily traceable, and actions are transparent to the contracted users. The most popular smart contract platform is the Ethereum blockchain that was launched in 2015 (https://ethereum.org/en/). Most metaverse platforms use Ethereum for smart contract-based contracting and execution (https://en.wikipedia.org/wiki/Ethereum).
NFT
NFTs are blockchain-based financial securities of digital data recordings. The blockchain distributed ledger records all transactions of ownership of an NFT, allowing secure NFT selling and trading. Metaverse virtual objects are computer-based designs saved in a computer file format. Metaverse virtual objects are “fungible,” which means that they can be replaced by another identical item and are mutually interchangeable with a digital copy of the original. Blockchains and securing smart contracts need to be used to make the original version “non-fungible” and have a cryptocurrency-based token value. This is why the name non-fungible token (NFT) is used. But unlike cryptocurrency, each NFT is uniquely identifiable and secure through a blockchain platform, and smart contracts are used to support the process. NFTs are not only for metaverse virtual goods and property but are used for digital audio, video, art, and photo files. Due to the versatility of investment worthy merchandise and easy means of making a purchase/investment, the NFT market size grew significantly during the COVID-19 pandemic period, where the total NFT trading in 2020 were at the $82 million level, which grew to the $17 billion level in 2021 (https://en.wikipedia.org/wiki/Non-fungible_token).
Metaverses can be visited using a computer monitor, laptop computer, or tablet/pad computer. But the immersive presence of a metaverse is best experienced when using HMDs (goggle-type devices) or glasses that support XR, MR, VR, or AR services. In a metaverse, an infinite number of things can be done which include going to a concert, having a meeting, going on an exportation trip, exercising, education, playing games, etc. The metaverse virtual space provides a platform for the user avatars to work, study, explore, collaborate, and compete. Based on this definition, all aspects of the metaverse are investment worthy.
A metaverse is based on an environment that is newly created, where the entire virtual space can be sold for ownership. A virtual space can consist of the virtual environment, buildings, infrastructure, and transportation facilities, where each of these virtual entities is divided into purchasable or investment worthy parts and is being sold through companies using cryptocurrency tokens, blockchains, smart contracts, and NFTs. Some of the representative companies include Decentraland (https://decentraland.org/), Sandbox (https://www.sandbox.game/en/), Axie Infinity (https://axieinfinity.com/), Illuvium (https://www.illuvium.io/), and Roblox (https://www.roblox.com/). Commonly, metaverse virtual investment and trade companies use Ethereum smart contracts, and each has its own cryptocurrency tokens. For example, Decentraland uses MANA, Sandbox uses SAND, Axie Infinity uses AXS, Illuvium uses ILV, and Roblox uses ROBLOX as their cryptocurrency tokens when making purchases, investments, or exchanges of virtual property. Virtual real estate purchasing, exchanges, and investments can also be conducted through companies like Metaverse Properties (https://metaverse.properties). To support virtual real estate investments, this company provides Metaverse Real Estate and Investment Trust (REIT) services and a Metaverse Index (MVI). REIT and MVI provide detailed information to assist and make virtual real estate investments. In addition, virtual arts, avatars, wearables, rideable, property, and various goods of the metaverse can be sold using NFTs on metaverse service platforms such as Fiverr or Upwork. A profit can be made by selling property at a price higher than purchased, renting property (and receiving rent fees), and advertising on one’s property (and receiving advertisement fees), or various types of events could be held to make a profit. In addition, the value of virtual property can be increased by virtual land improvement and land amelioration, such as building useful facilities for avatars.
Alternatively, instead of virtual real estate, a metaverse investment could be made by investing into the cryptocurrency tokens like MANA (for Decentraland), SAND (for Sandbox), AXS (for Axie Infinity), ILV (for Illuvium), ROBLOX (for Roblox), etc. Obviously selling tokens for a price higher than the purchased price would be the objective. Instead of investing into a few selected tokens, you can also invest by purchasing an index of tokens using platforms such as Metaverse Index (MVI). An index includes an expert selected group of metaverse tokens, where the tokens to be included and their proportional amounts included in each index are different. So careful index selection is needed.
The next type of metaverse merchandise focuses on the avatar and its wearables and usable items, which can be made based on request or can be designed and directly sold on platforms such as Upwork and Fiverr. An avatar can resemble its user in detail (even can include live expression details) or can be made to be more abstract and only slightly resemble its user. Avatars can be beautified or made more handsome, cute, ugly, monstrous, etc. using filters with various fashion, culture, gender, or artistic preferences. The avatar’s wearable and usable items are very diverse and can easily be customized to one’s preference. Avatar wearables include clothes, shoes, sneakers, sandals, roller blades, ice skates, snowboards, skiis, watches, jewelry, accessories, glasses, etc. Avatar usable items include cars, trucks, buses, trains, tanks, motorcycles, bicycles, scooters, skates, roller blades, roller boards, boats, yachts, submarines, underground vehicles, blimps, combat jets, airplanes, rockets, etc. Avatar wearables and usable items designed by world top brands are very popular. Some of the top brands include Nike, Adidas, Gucci, Louis Vuitton, Toyota, etc. In real life, a product from Gucci or Louis Vuitton may be too expensive to buy for most customers, but in the metaverse world, these luxury brand products are much more affordable (as they are virtual goods) for one’s avatar.
These luxury brand virtual goods can easily provide a surrogate satisfaction by purchasing these for one’s avatar, which is one of the benefits of the metaverse. From the company business side, the metaverse provides a great way to advertise their products and brand name to a new generation of young metaverse users (www.youtube.com/watch?v=3U70-y-lNaM).
Another method is to invest into metaverse related company stocks, which include companies like Amazon, Microsoft, Meta/Facebook, Apple, Nvidia, etc. This is the most traditional way of making an investment into metaverse technology.
Wave
In 2016, the metaverse virtual entertainment platform Wave (https://wavexr.com) was founded in Los Angeles, California, USA. The virtual entertainment shows hosted are called “Waves” which support virtual collaboration of artists and audience based on real-time live virtual performances. An avatar of the artist is made with precision such that the artist’s voice, sound, motions, and emotions are accurately presented in the metaverse event space. Waves are conducted at the same time as the live concert in a location with or without a live audience. Wave enables a live music concert to include additional virtual factors (such as virtual games and shopping) for enhanced interaction with the entertainer and other fans. Wave’s real-time multicasting technology enables entertainment events to be streamed live globally on wave.watch (https://live.wavexr.com/home) and can be accessed using popular social media (e.g., YouTube, Twitter, Twitch, TikTok, and Facebook) and gaming platforms (e.g., Roblox). Some of the representative virtual events hosted by Wave include concerts by DJ David Guetta, Travis Scott, The Chainsmokers, Justin Bieber, Pentakill, Dillon Francis, Alison Wonderland, Lindsey Stirling, The Weeknd, and John Legend (https://metaverseinsider.tech/2022/05/01/metaverse-company-delivering-the-best-of-live-music-gaming-broadcast-technology-to-fans/).
ZEPETTO
In August of 2018, ZEPETTO was first released and now has over 200 million users, where 80% of these users are Generation Z users. Generation Z is defined as the generation after Millennials and before Generation Alpha that have birthdays approximately in the range from 1997 to 2012. One characteristic of Generation Z is that they were the first generation to grow up in a society where Internet access, portable computers, and smartphones were common and, therefore, have been called “digital natives” as well (https://m.k-odyssey.com/news/newsview.php?ncode=1065580437310285).
ZEPETTO enables avatars to be created based on a picture of its user, where the avatar aspects and features can be modified to fit personal desires. Various types of avatar clothing, footwear, accessories, and jewelry can be purchased with the ZEPETTO currency ZEMs and coins. Houses, buildings, gardens, and parks can be built and used to invite friends over for socializing activities. In addition, personal and group/team activities and games are commonly provided. ZEPETO opened a virtual art gallery which displayed some of the world’s most famous art masterpieces.
ZEPETTO is also used as an entertainment platform. ZEPETTO is a product of NAVER Z, which is a subsidiary of the parent NAVER Corporation in South Korea. Major K-pop companies in South Korea like YG Entertainment, JYP Entertainment, and Big Hit Music have made investments into ZEPETTO.
In November of 2021, YG Entertainment launched “the SamE” metaverse space in ZEPETTO, which is the same as an actual fan project-space in front of YG Entertainment building in South Korea. Due to the COVID-19 pandemic, YG Entertainment made a duplicated virtual space in the metaverse. The SamE project-space and the café “MD shop” were popular locations for live musical events and exhibitions. But as it became difficult for the fans to visit these spaces due to the COVID-19 pandemic, the agency reproduced these spaces into the ZEPETTO metaverse. In the ZEPETTO MD shop, BLACKPINK’s music videos are played, and YG artists’ products are sold. Popular avatar items include BLACKPINK’s 2019 Coachella Valley Music and Arts Festival costumes. In addition, when a new album was released, a special event that enabled fan avatars to take pictures with the BLACKPINK avatars was provided.
Likewise, JYP Entertainment is also active with K-pop entertainment events and fandom on ZEPETTO. JYP’s K-pop groups TWICE and ITZY had released metaverse music videos specifically for ZEPETTO.
In addition, Big Hit Music is also very active on ZEPETTO, as it is the agency of BTS. BTS has held metaverse fan events on ZEPETTO. The BTS avatars in ZEPETTO are very popular, in which the ZEPETTO code names of the BTS avatars are commonly memorized by the BTS Adorable Representative M.C. for Youth (ARMY) fan club members. In addition, BTS released their metaverse music video and debuted their “Dynamite” choreography video in Fortnite.
In September of 2020, at the BLACKPINK signing event on ZEPETTO, approximately 50 million fans visited and took pictures and received signatures from BLACKPINK avatars. This example shows the massive accessibility of metaverse services, as participation of 50 million fans at a physical entertainment event is impossible. Currently, the main metaverse contents are online games and entertainment, where approximately 70% (or more) of the users are teenagers and Generation Z (https://en.wikipedia.org/wiki/Generation_Z). For these users, online services are very comfortable, and use of avatars in the metaverse has no awkwardness and is preferred in many cases.
Metaverse Global Market
Based on a Fortune Business Insights report, the forecast of the metaverse global market is expected to grow significantly. For the period from 2022 to 2029, a compound annual growth rate (CAGR) of 47.6% is predicted (www.fortunebusinessinsights.com/metaverse-market-106574). This growth rate of the metaverse global market will take the 2022 prediction level of $100.27 billion to the $1,527.55 billion level by 2029. Based on the “Metaverse Market Revenue Worldwide from 2021 to 2030” report from Statista, the worldwide metaverse market size in 2021 was $38.85 billion which grew to $47.48 billion in 2022 and is expected to reach $678.8 billion by 2030 (www.statista.com/statistics/1295784/metaverse-market-size/). These reports differ in criteria of metaverse market size evaluations, so the values are different, but it is evident that the metaverse market size will grow significantly over the next decade. Metaverse services require Internet platforms as well as significant software and hardware support. XR devices will play a significant role as these XR HMDs (or glasses) will serve as the main interface to metaverse worlds. Therefore, in the following, an analysis of metaverse market size of XR devices is provided.
A pie chart of A R and V R Headset Market Share for 2020 in percent. V R standalone H M D is high at 43.76 and A R screenless viewer is low at 0.49.
AR/VR Headset Market Share for 2020
A pie chart of predicted A R and V R Headset Market Share for 2024 in percent. V R standalone H M D is high at 32.92 and A R screenless viewer is low at 0.03.
Predicted AR/VR Headset Market Share for 2024
Currently VR technology is well implemented in various types of devices. But due to many difficulties in AR technology implementation, an AR HMD is much more difficult to make and is therefore very expensive. Future improvements in AR technology are expected to highly improve the performance while reducing the power consumption such that AR standalone devices will become more usable and popular. The big increase in AR standalone devices in 2024 presented in Figure 1-2 reflects this expected change.
One of the focuses of this book is to help AR development, which is why Chapter 3 is focused on AR core technologies and Chapter 4 describes deep learning technologies. Once AR technology reaches a higher level, then MR devices (which combine AR and VR) can be developed at a higher level, and with additional haptic interfaces added to MR devices, then true XR devices can be made. So, there is still a long way to go to reach high-performance XR metaverses.
A bar graph of the market size in million U S dollars from 2017 to 2025. It is high in 2025 at 3688.3 and low in 2017 at 46.8.
MR Market Size Predictions
A bar graph of the market share forecast by Segment in percent in 2020. Video games are high at 34 and military is low at 4.
MR Market Share Forecast by Segment
A bar graph of the X R market size prediction from 2016 to 2025 in billion U S dollars. It is high in 2025 at 393 and low in 2016 at 5.2.
XR Market Size Predictions
A pie chart of X R Headset Global Brands Shares for 2020 in percent. Oculus is high at 53.5 and Pico is low at 4.8.
XR Headset Global Brands Share for 2020
This 2020 market trend radically changes, and by the fourth quarter of 2021, Meta’s Quest 2 occupied 80% of the global VR, AR, MR, and XR headset market share based on product shipments. The second highest was DPVR which occupied 8%, third was Pico which occupied 5%, and other company products occupied a combined 7%.
Among XR devices, the most popular are Oculus Quest and Meta Quest 2 by a far margin. In May of 2019, the Oculus Quest VR HMD was released by Facebook. The Quest’s introductory price was $399 for the 64 GB (gigabyte) storage memory device and $499 for the 128 GB storage memory device. In October of 2020, Oculus Quest 2 was released, and due to Facebook’s company name change to Meta, the device was rebranded as Meta Quest 2 in November of 2021. Quest 2’s initial price was $299 for the 64 GB storage memory model which was changed to 128 GB later and $399 for the 256 GB storage memory model.
DPVR’s E3 VR headset was released on September 1st of 2019, which uses the SteamVR platform and has a retail price of $399 for the HMD. Pico was acquired by ByteDance in 2021 and has been improving its Neo standalone VR HMDs, in which the Pico Neo 3 was recently released on May 24, 2022.
Among AR and MR devices, the Microsoft HoloLens and HoloLens 2 are very popular. In March 2016, the Development Edition of Microsoft HoloLens was released by Microsoft at a list price of $3,000. Microsoft HoloLens is a MR smart glasses HMD device that runs on the Windows Mixed Reality platform and uses the Windows 10 operating system (OS). In February 2019, HoloLens 2 was introduced at the Mobile World Congress (MWC) and later became purchasable at a preorder price of $3,500.
The presented product release timeline and large difference in product cost of the VR-based Oculus Quest and Meta Quest 2 devices compared to the AR-based HoloLens and HoloLens 2 products show that AR technology is much harder to implement than VR devices. This is because AR devices have to overlay information and/or images onto the actual view of the user in real time, which is very difficult. The overlay of information and/or images onto the actual view of the user has to be instantaneous and comfortably viewable to the user and the user’s surrounding environment in terms of color, brightness, and distance. Many additional XR field-of-view (FoV) aspects are described in Chapter 2.
Video Streaming Technology Introduction
Traditional multimedia was composed of text, audio, and video, which were delivered to users through broadcasting or unicasting. Except for the telephone, real-time interaction over traditional multimedia was not possible, and users could only passively receive the content and play it. Later as networks were built and networking technology evolved, multimedia content was able to be delivered to selected users through multicasting technology. Recently, the evolution of multimedia services is diversifying and speeding up due to the rapid development of the Internet and information technology and communications (ITC) systems and software. Multimedia now combines many content forms, and the delivery method has changed significantly. In addition, new ways to interact with multimedia content have been revolutionizing the way multimedia can be used.
Television (TV)
One reference point of multimedia technology is the black and white analog television (TV), which was used for approximately 30 years from 1936 to 1967. Black and white analog TV was replaced by color analog TV, which began broadcasting in 1954 in the United States. The three representative analog color TV standards are the National Television System Committee (NTSC), Phase Alternating Line (PAL), and the Electronic Color System with Memory (SECAM). NTSC was developed in North America and is the original analog color TV signal encoding system that was broadcasted since 1954. Western European countries wanted an analog color TV standard that was independent of NTSC and could perform better in Europe’s geographical and weather conditions. This led to the development of PAL and SECAM. The United Kingdom was the first to broadcast analog color TV signals in 1967 using PAL. The first analog color TV broadcast using SECAM was conducted in France in 1967. SECAM is originally written in French as SÉCAM, which is the acronym for système électronique couleur avec mémoire (https://en.wikipedia.org/wiki/Television).
Cable TV
In 1950, commercial services of cable television (TV) started in the United States. Cable TV has been abbreviated as “CATV” based on the acronym of the original 1948 technology “Community Access Television” or “Community Antenna Television” which is the origin of cable TV. CATV was needed because over-the-air broadcasted radio frequency (RF) TV reception has many limitations in areas far from TV transmitters or due to mountainous terrains. In such areas, a community antenna was constructed, and cables were connected to individual homes, which is how CATV was serviced. By 1968, only 6.4% of Americans used cable TV which expanded to 62.4% by 1994. Cable TV services became very popular in the United States, Canada, Europe, Australia, South Asia, East Asia, and South America. One of the common complaints consumers had with modern cable TV services was the bundling of channels that had to be purchased together, where in fact only a few desired channels were of interest. The cable TV bundling made cable TV services very expensive and gave users a feeling that they were wasting money. Based on cable TV technology, the first subscriber streaming service and the first “pay TV” service began with Home Box Office (HBO), which is discussed in the following (https://en.wikipedia.org/wiki/Cable_television).
HBO
HBO was the first to start “pay TV” services and subscriber video streaming services, which was brilliant and revolutionizing in both business and technology (www.hbo.com/). In November of 1972, HBO was launched by the parent company Home Box Office that is owned by Warner Bros. Discovery, which has headquarters in New York City, New York, USA. HBO is the original subscription-based television service provided through cable TV and multipoint distribution service (MDS) providers to direct subscriber TVs. HBO’s premium channels were provided to subscribers for an extra monthly fee (i.e., pay TV services). The media content on HBO’s premium channels did not have advertisements, and many programs included objectionable material without editing. In 1976, HBO was the first television channel that used worldwide satellite program transmission. HBO was also the first streaming media company to create and release original program content. In May of 1983, HBO’s first original television film “The Terry Fox Story” was released. Over the years, HBO has made numerous HBO Originals that cover the media genres of drama, comedy, miniseries, and docuseries. By September of 2018, HBO was available to approximately 35.656 million US households and had 34.939 million subscribers. As of 2018, HBO was globally available to at least 151 countries and had an estimated 140 million cumulative subscribers. As of 2017, an operating income of nearly $2 billion every year had been generated from HBO. In May of 2020, HBO Max was launched by Warner Bros. Discovery as a subscription-based VOD service that streams content from HBO, Warner Bros., and their related brands, which has been very successful (https://en.wikipedia.org/wiki/HBO).
Cinemax
In August of 1980, Cinemax was launched by HBO which is owned by Warner Bros. Discovery (www.cinemax.com/). Due to having the same parent company, HBO and Cinemax are sister companies. In August of 1991, HBO and Cinemax were the first two subscriber television services that offered complimentary multiplexed channels in the United States. The network service provided seven 24-hour, linear multiplex channels and the traditional HBO On Demand subscription video service. By September of 2018, Cinemax was available to approximately 21.736 million US households and had 21.284 million subscribers (https://en.wikipedia.org/wiki/Cinemax).
Blockbuster
In October of 1985, Blockbuster LLC was founded in Dallas, Texas, USA. Blockbuster provided home movies and video game rental services through its video rental shops, which expanded internationally throughout the 1990s due to its excellent business model. In 2004, Blockbuster had 9,094 stores and approximately 84,300 employees worldwide, where approximately 58,500 worked in the United States and 25,800 worked overseas. Blockbuster’s business model revolutionized multimedia rental services and influenced many media companies. Blockbuster made rental fees cheap to encourage frequent and economic means to media sharing to the public but charged heavier on late return fees to discourage users from returning rented items late, which seriously hindered circulation of media sharing and made customer services and reservations very difficult. The success of Blockbuster’s business model drew much attention, and many similar companies were established which also included some varied media rental business models.
In 1997, Netflix began postal mail-based DVD renting services that had no late return fees (further details on Netflix are described in the following section). In 2002, Tic Tok Easy Shop was created, which had Redbox automated retail kiosks (mostly installed in front of the shops for easy customer access) that enabled DVD rentals for $1 per day, which were very successful. Netflix’s DVD mail delivery services, Redbox automated DVD rental kiosks, HBO’s VoD services, and financial recessions in the economy resulted in many difficulties for Blockbuster. As a result, from 2005, Blockbuster’s revenue began to decrease. In 2007, Netflix began streaming media and VoD services in the United States. Following Netflix’s successful business model, many companies started OTT media streaming services. In 2010, Blockbuster filed for bankruptcy protection, and in 2014, the last remaining 300 Blockbuster owned stores all closed (https://en.wikipedia.org/wiki/Blockbuster_LLC).
IPTV
Internet Protocol television (IPTV) is the technology that enables television content to be transmitted over the Internet. Because IPTV uses the Internet Protocol (IP) and the Internet network, it is significantly more versatile in user control compared to wireless broadcast TV, satellite TV, and cable TV. IPTV is a core technology of streaming media. When using IPTV, the user can begin viewing while the media content is progressively being downloaded from the server. Media streaming provides many advantages over media technologies that require a full file download before viewing can start. Due to limitations in the supportable data rates of the Internet, IPTV services were not possible until the 1990s. In 1995, IPTV became possible with the company Precept Software releasing “IP/TV” to provide video services over the Internet. IP/TV used Mbone technology and was compatible with Windows and Unix-based systems and applications (https://en.wikipedia.org/wiki/Internet_Protocol_television).
Mbone is an abbreviated name for “multicast backbone” which is a protocol developed in the early 1990s to enable Internet backbone and virtual network services that can support Internet protocol version 4 (IPv4) multicast packet delivery. IP/TV could be used to deliver single and multisource video and audio IPv4 packets over the Internet by using Real-time Transport Protocol (RTP) and the RTP Control Protocol (RTCP). In 1998, Cisco acquired Precept and the IP/TV trademark.
As the Internet became faster and more reliable, IPTV services globally spread quickly. Global market expectations in 2009 were $12 billion in revenue based on 28 million subscribers, which reached $38 billion in revenue based on having 83 million subscribers by 2013. The dominance of IPTV technology and services has grown significantly ever since. Now IPTV services are supported by SmartTVs which have smartphone-like interfaces and control features as well as voice activation and control technology support.
IPTV services are commonly supported by one (or a contracted group of) content service providers which use separate IPTV networks or virtual private networks (VPNs) over public networks. In addition, IPTV services are displayed on IPTVs or SmartTVs. IPTV services (and cable TV) use dedicated and controlled networks to provide video on demand (VoD) (or related) services. Due to the network service provider having full control over the network, Cable TV and IPTV streaming speeds are fast and reliable, such that channel changes can be made instantly. These aspects are what distinguish IPTV to over-the-top (OTT) services, which are described in more detail in the following.
China’s Media Services
China has been a very successful early adopter in the media service domain, in which China’s rich resources of high-quality engineers have driven rapid development. China’s earliest video streaming and user created content (UCC) (i.e., also known as user generated content (UGC)) platforms were started by Youku in 2003 and Todou in 2005, and a few years later, companies like Tencent, Alibaba, and Baidu started to invest into this domain of business. In 2010, premium video streaming in China began with iQiyi being launched by Baidu, which was soon followed by Tencent Video (https://en.wikipedia.org/wiki/Mass_media_in_China).
YouTube
In February of 2005, former PayPal employees founded YouTube, which rapidly grew in popularity and content in North America (www.youtube.com/). In the following year in October of 2006, Google acquired YouTube for $1.65 billion, in which many considered it as an overpay at the time but it shows the remarkable business mind to realize valuable technology of the future by the Google leadership. YouTube has significantly contributed to expanding and improving the domain of UCC media content creation to include public users, which before it was mostly limited to the broadcast corporations and movie companies in the United States. In 2005, YouTube’s Internet-based service model influenced many companies. YouTube’s UCC revolution became even more influential due to Apple’s iPhone (first launched in 2007) and other smartphone companies that followed (most of them launched in 2009), because smartphones enabled easy video recording and editing as well as direct uploading to YouTube wirelessly. YouTube TV is a very strong contender because its platform has some features that other big media streaming companies like Netflix, Prime Video, and Disney+ do not have. YouTube TV enables users to view TV programs from various channels live as they are being broadcasted in real time, or programs can be stored to a library and viewed later, where the library has near unlimited capacity. YouTube TV was the fastest growing live streaming service in 2019 and already has the largest market share among live streaming services. Other live streaming service platforms include Vue, Sling, DirectTV NOW, etc. YouTube TV includes many TV programs, and users can select to subscribe or view free with advertisements included. Because YouTube TV is not different from YouTube, and because most smartphone users have YouTube installed, the accessibility of YouTube TV is very convenient, and already almost everyone knows how to search and find the desired content and view it due to the familiarity of using the YouTube app on smartphones for a long time (https://en.wikipedia.org/wiki/YouTube).
Streaming OTT
Streaming media services are commonly provided using OTT media delivery technology. The US Federal Communications Commission (FCC) considers OTT services to be a linear video service provided over the public Internet in which cable operators do not manage the video services or the cable systems. The term “linear” refers to a video service that is scheduled and virtually playable simultaneously with transmission. The Canadian Radio-Television and Telecommunications Commission (CRTC) considers OTT services to be a programming service provided over the Internet that is independent to the network (e.g., cable, satellite, etc.) and facilities used for its content delivery. The US FCC classifies OTT services into two types, multichannel video programming distributor (MVPD) or online video distributor (OVD). MVPD provides multiple channels of video programming through multichannel multipoint distribution services and direct broadcast satellite services, etc. Companies like YouTube TV, DirecTV Stream, Hulu + Live TV, FuboTV, and Sling TV can be considered as virtual MVPDs. An OVD provides video programming through the Internet in which the network does not include an MVPD nor is the network delivery path network (or a part of it) operated by that OVD. Companies like Netflix, Disney+, Amazon Prime Video, HBO Max, Hulu, Peacock, Discovery+, and Paramount+ can be considered as streaming OVDs. The term “streaming” commonly refers to media downloads that can start to be played before the full download of the media file is complete.
OTT streaming of media content is delivered directly to the viewers through the Internet in which users can receive OTT services on their computer, laptop, SmartTV, smartphone, and any other smart device (e.g., tablet, pad, etc.). In other words, unlike in IPTV services, OTT services can be accessed on a variety of receiving devices and are not limited to a specific Internet service provider. Users will commonly subscribe to multiple content providers (e.g., Netflix, Disney+, HBO Max, etc.) and stream media contents over the mobile/wireless Internet and view the content with the most convenient device based on their location and smart device availability and features. Therefore, OTT services can be broader in range and more diverse in content compared to IPTV services (https://en.wikipedia.org/wiki/Over-the-top_media_service).
Traditionally, video content was delivered by broadcast TV companies and later through cable and satellite TV companies. Therefore, the TV companies owned and operated most of the content creation, the film studios, and delivery platforms in the past. However, due to the revolution in Internet technology, computers, laptops, IPTV, SmartTVs, smartphones, tablet/pad smart devices, and the mobile Internet, new means of streaming media services that directly send to the users became more popular. Because streaming media services were delivered directly to the users bypassing (i.e., passing over) the traditionally mega TV companies and delivery platforms, the technology is called over-the-top (OTT).
Having to pay a subscription fee for streaming media services enabled the media content to be viewable at any time without any commercial interruptions. In addition, the media library included premium media content that could be viewed on demand based on a separate subscription and payment. Thus, newer, more valuable, or user exclusive content could be provided easily to the users, which helped to satisfy personal needs and desires. As the smartphone revolution began in 2007 with the Apple iPhone, and application (App) stores (e.g., Apple’s App Store, Google's Play Store, AWS Appstore, etc.) became common along with online payment methods, subscription fee based streaming media services became widely acceptable to the public easily.
In the evolution of video streaming, file downloading was the earlier option of use by media platforms, in which the entire media file was downloaded before viewing could be started. This was the preferred option because networks were not as stable and did not have sufficient bandwidth and throughput to provide a reliable continuous streaming session for a long period of time. In addition, the download before viewing process (which uses a “push” based protocol) had some security issues (with network firewalls) that are explained in Chapter 6 when introducing the features of MPEG-DASH technology, which is the most commonly used modern video streaming network protocol. Additionally, former media players were not as capable of multitasking (i.e., progressively downloading while playing the media) as modern media devices are; thus, having a completely downloaded file in the player’s memory before playing the media was the preferable method. The newer streaming media services commonly enable short previews of the content to assist the selection, and after content selection, the media begins to play even before the entire media file is downloaded to the user’s device. In addition, streaming media companies have application stores that support software installations and upgrades when needed for the user’s device. This is a necessity because smart devices have different hardware and software as well as frequent OS upgrades, and correspondingly the installed media streaming app will need to be upgraded and adjusted.
Brief History of Internet-Based Streaming Media
In May of 1993, the first video was streamed over the Internet at a one-twelfth lower frame rate (i.e., two frames per second) due to the Internet’s small transmission bandwidth. The transmitted video was titled “Wax or the Discovery of Television Among the Bees” which had been originally released in 1991.
In September of 1995, Real Networks enabled the first live streaming event over the Internet which was an ESPN SportsZone live radio broadcast of a Seattle Mariners and New York Yankees baseball game.
In 1995, the Shockwave Player was released by Macromedia to provide media streaming functionality for the Netscape Navigator.
In 1997, ShareYourWorld.com opened its video uploads and streaming services (closed in 2001).
In 1999, the Microsoft Windows Media Player 6.4 was released including the video streaming feature, Windows Media streaming protocols, and the ASF file format so multiple video and audio tracks could be stored in a single file.
In January of 2005, Google Video was launched.
In February of 2005, the Stickam website was launched to provide live video chatting.
In March of 2005, the Dailymotion website was founded (in French) to provide Internet video-sharing.
In April of 2005, YouTube opened its video uploads and streaming services.
In May of 2006, Crunchyroll was founded to provide online East Asian media video streaming. Its streaming content included drama, music, electronic entertainment, auto racing, anime, manga, etc.
In October of 2006, Justin.tv was founded to provide live video streaming services.
In September of 2006, Amazon Unbox was launched in the United States.
In October of 2006, YouTube merged with Google.
In October of 2006, LiveLeak website was founded in the United Kingdom to provide Internet-based video-sharing.
In December of 2006, Youku was founded in China to provide video streaming services.
In January of 2007, Netflix started video streaming services.
In September of 2007, Vevo was founded to provide music video streaming for the Universal Music Group and Sony Music Entertainment.
In September of 2007, the Microsoft Silverlight application was introduced to support rich Internet applications.
In March of 2008, Hulu was launched for public access in the United States providing online TV and movie streaming services. Its current owners are The Walt Disney Company (67%) and Comcast (33%) which is through NBC Universal.
In September of 2008, Amazon Unbox was renamed to Amazon Video on Demand.
In January of 2009, Google Video services were discontinued.
In November of 2009, HTTP Live Streaming (HLS) adaptive bitrate streaming technology was introduced by Apple.
In April of 2010, iQiyi was launched in China as a new online video platform.
In December of 2010, Viki was founded, which is an international video website. Viki provides TV shows, movies, and premium videos.
In April of 2011, Vudu launched online streaming services.
In June of 2012, Vine was founded, which supports short-form (up to 6 seconds) video-sharing services.
In December of 2012, Snapchat added on video send functionality in addition to photos.
In June of 2013, Instagram added its video-sharing function.
In 2014, Prime Video was added to Amazon’s Prime subscription in the United Kingdom, Germany, and Austria.
In March of 2015, the Periscope app was launched for live video streaming on iOS and Android smart devices.
In May of 2015, the Meerkat app was released for live video streaming on iOS and Android smart devices.
In 2015, Amazon Channels was launched which was Amazon’s Streaming Partners Program.
In January of 2016, Facebook Live was launched by Facebook.
In April of 2016, Amazon Prime and Prime Video were separated (for services in the United States).
In November of 2019, Disney+ was launched.
More details on the major OTT streaming services like Netflix, Hulu, Peacock, Disney+, Amazon Prime Video, and YouTube are provided in the following section.
Video Streaming Services and Business
Based on the “Video Streaming Market Share, 2022-2029” report from Fortune Business Insights, in 2021, the global video streaming market size was estimated to be worth $372.07 billion. The global video streaming market size in 2029 is projected to reach $1,690.35 billion based on a 19.9% compound annual growth rate (CAGR) for this forecast period. Alternatively, a report from Grand View Research estimates that the video streaming market size in 2022 has a value of $70.59 billion and the revenue forecast for 2030 reaches $330.51 billion based on a CAGR of 21.3% for the 2022 to 2030 period. CAGR represents the exponential growth rate over a multiple year period.
Forbes’ “The World’s Largest Media Companies 2022” ranking is based on company market value, where the top five on this list are (1) Comcast Corporation Class A, (2) Walt Disney Company, (3) Charter Communications, Inc. Class A, (4) Netflix, Inc., and (5) Paramount. There are additional factors that need to be considered beyond revenue and market share, which will be described in the following (www.forbes.com/sites/abigailfreeman/2022/05/12/the-worlds-largest-media-companies-2022-netflix-falls-in-the-ranks-after-subscriber-loss-disney-climbs-to-no-2/).
A bar graph of the U S Video Streaming Revenue and Projections from 2015 to 2025 in billion U S dollars. It is high in 2025 at 42 and low in 2015 at 7.1.
US Video Streaming App Revenue and Projections
In the graphs presented, the data related to Amazon Prime Video and Apple TV+ may be different compared to the number of subscribers because the data of Apple+ is influenced by free trial users and some Amazon Prime users are not Amazon Prime Video subscribers; therefore, parts of estimated data had to be used (based on Business of Apps report).
A bar graph of the U S Video Streaming Subscribers in millions. Netflix is high at 75 and Britbox is low at 1.5.
US Video Streaming Subscribers
Among Internet video streaming companies, the evolution of Netflix is of importance as several factors of its business model have highly influenced many other OTT video streaming companies. Therefore, more details on Netflix are provided in the following.
Netflix
In August of 1997, Netflix was founded in Scotts Valley, California, USA, with the business model of postal mail-based DVD renting services and selling DVDs. However, DVD sales were stopped the following year. The motive to create Netflix is related to Blockbuster. In 1997, Reed Hastings was charged a Blockbuster video late return fee of $40 which influenced him to create a new type of subscription-based rental company called Netflix later that year (on August 29) that enabled video rentals via mail orders which were delivered on DVDs and had no late fees. Because DVDs were easy to send and receive via postal mail, Netflix’s service model became very popular. For a $20 monthly fee, Netflix users could make a list of DVD rentals using the netflix.com website, where up to four DVDs at a time could be subscribed, and when a DVD was returned, the next video in queue would be sent to the subscriber. The business model of Netflix was challenging as a significant amount of labor was involved. Production of DVDs took time which resulted in delays, especially for new video releases. The growing popularity of Netflix did influence the video rental business significantly. However, in 2000, when Netflix was having a challenging time in revenue, its cofounders Reed Hastings and Marc Randolph made an offer to sell Netflix at $50 million to Blockbuster. Although the Netflix business model had an influence on Blockbuster, Blockbuster’s revenue was still significant, and especially the company was making a significant revenue (approximately $800 million) from late video returns, which was approximately 16% of Blockbuster’s total revenue. Eventually Blockbuster’s CEO John Antioco turned this offer down.
In 2007, Netflix began to provide streaming media and VoD services in the United States. Starting from 2010, Netflix expanded into Canada, Latin America, and the Caribbean. In February of 2013, Netflix debuted the company’s first original series “House of Cards,” which was followed by the original series “Orange Is the New Black” in the same year, which were great successes. Starting from 2016, Netflix expanded its services internationally. As of March 31, 2022, Netflix has 74.6 million subscribers in the United States and Canada and over 221.6 million subscribers worldwide.
Netflix’s contributions to the OTT streaming media platform were significant in terms of technology and content creation. In recognition of this, in 2012, Netflix received its first Emmy Award, which was the “Emmy Engineering Award” for Netflix’s profound influence on how the company changed the way television is watched. Additionally, Netflix original programs were receiving high accolades. In 2018, the number of Emmy nominations surpassed HBO. In 2019, Netflix received 14 Oscar nominations, which was the third most, following 20 total nominations for Fox (i.e., 15 for Fox Searchlight and 5 for 20th Century Fox) and 17 nominations for Walt Disney. In 2020, remarkably Netflix received 24 Oscar nominations, which was the most nominations among all film studios that year. This marked an evident point that the domain of high-quality original content creation had shifted, and platform exclusive content was equally acknowledged by the Academy of Motion Picture Arts and Sciences and the public.
A bar graph of Netflix Revenue in billion U S dollars from 2011 to 2020. It is high in 2020 at 24.9 and low in 2011 at 3.1.
Netflix Revenue
A bar graph of Netflix's profit in million U S dollars from 2012 to 2020. It is high in 2020 at 4500 and low in 2012 at 50.
Netflix Profit
A bar graph of Netflix Content Expenditure in billion U S dollars from 2016 to 2021. It is high in 2021 at 17 and low in 2016 at 6.88.
Netflix Content Expenditure
Netflix has a true global platform, as it is available and popular worldwide. As shown in Figure 1-9, in 2020, Netflix’s revenue was $24.9 billion with an operating profit of $4.5 billion. Among this massive revenue, the largest portion of $11.45 billion came from North America. Netflix’s profit was influenced by a $11.8 billion content expenditure in 2020. In 2021, Netflix content expenditure reached $17 billion, which was well enjoyed by its 209 million worldwide subscribers (www.businessofapps.com/data/netflix-statistics/).
Disney+
A bar graph of Disney plus Revenue in million U S dollars from 2020 to 2021. It is high in 2021 at 1440 and low in 2020 at 442.
Disney+ Revenue
Amazon Prime Video
A strong competitor with Netflix and Disney+ is Amazon Prime Video. In the case of Amazon, the company wisely used its online sales platform to expand its OTT streaming services, including its subscribers. In 2011, Amazon Video on Demand was changed into Amazon Instant Videos, and its services were provided to all Amazon Prime subscribers. Amazon Instant Videos expanded its content range through a collaboration deal with Epix. A special feature of Amazon Instant Videos was that it allowed subscribers to purchase the rights to place the content in one’s personal streaming library. In April of 2013, Amazon Prime Instant Video exclusively released its original programming comedy series “Betas,” “Alpha House,” and “Annedroids” which were very popular. Amazon Prime Video was launched worldwide in December of 2016, and as of April of 2021, the company had approximately 175 million users (https://en.wikipedia.org/wiki/Amazon_Prime_Video).
Hulu
In October of 2007, Hulu was co-launched by the News Corporation, NBC Universal, and Providence Equity Partners. In 2010, Hulu rebranded its company name to “Hulu Plus” and became the first media streaming company to use “Plus” in its name. Hulu’s content library contains Hulu original programming as well as a diverse program list of content from ABC, Freeform, FX Networks, etc. Hulu began investing into original content creation, where in January of 2011, Hulu’s first web series “The Morning After” was streamed. In February of 2012, Hulu premiered its first original scripted program “Battleground” which was a political documentary-style drama. In March of 2019, Disney obtained a 60% ownership of Hulu as 21st Century Fox was acquired by Disney. Currently, Hulu is owned 67% by The Walt Disney Company and 33% by Comcast through NBC Universal. As a result, Disney’s media streaming services include Hulu, ESPN+, and Hotstar. As of April 2, 2022, Hulu has 45.6 million subscribers (based on Wikipedia) and ranks fifth among the US video streaming subscribers company rankings presented in Figure 1-8 (https://en.wikipedia.org/wiki/Hulu).
Peacock
In July of 2020, Peacock was launched by NBC Universal which is owned by Comcast. The name “Peacock” came from the NBC logo. Considering that Peacock is owned by NBC and Universal, evidently Peacock has an extensive library of movies and TV programs. In May of 2020, Peacock became available for the Apple TV on-launch and iOS devices. In July of 2020, Peacock became accessible through app access on the PlayStation 4. In September of 2020, Peacock became accessible on Roku digital media players. In June of 2021, Peacock became accessible through app access on Samsung smartTVs, Amazon Fire TV, and Amazon Fire tablets (e.g., Fire HD). As of 2021 late July, Peacock had accumulated more than 54 million signups, and by April of 2022, Peacock had 13 million paid subscribers and a total of 28 million monthly active accounts. Among the US video streaming subscribers company rankings in Figure 1-8, NBC Universal Peacock ranks sixth (https://en.wikipedia.org/wiki/Peacock_(streaming_service)).
YouTube and YouTube Premium
A bar graph of U S Video Streaming Usage in millions. Youtube is high at 150 and Apple T V plus is low at 15.
US Video Streaming Usage
Household penetration rate is a common metric used to describe the market share of the streaming companies like Netflix, Prime Video, Disney+, etc. Comparing the recent US household penetration rates, Netflix is at 87%, Prime Video is at 53%, and Hulu is at 41.5%. Future projections of 2024 streaming subscribers show future expectations of Netflix at 235.6 million, Prime Video at 135 million, and Disney+ at 101.2 million. Household penetration rates are highly influenced by the quality and quantity of existing media content as well as expectations of future new content. This is why media streaming companies are investing into new exclusive content creation. In 2019 large investments into original program creation were made, where Netflix invested $15 billion, Prime Video invested $6 billion, Hulu invested $2.5 billion, and Disney+ invested $1 billion. Due to Netflix’s massive investment, the company was able to provide more than one new show on average for every day in 2019. Based on the acquisition of ESPN+, Fox (which operates Hulu), and India’s Hotstar, the streaming media content library of Disney+ has grown extensively.
In terms of watch time, YouTube dominates the competition. YouTube has 2 billion monthly users and has 250 million watch hours daily, which resulted in a $15 billion in advertisement revenue in 2019. YouTube TV had 2 million paying subscribers in 2020 (based on the Business of Apps report).
A bar graph of YouTube's annual revenue in billion U S dollars from 2010 to 2020. It is high in 2020 at 19.7 and low in 2010 at 0.8.
YouTube Annual Revenue
A bar graph of YouTube users in billions from 2012 to 2020. It is high in 2020 at 2.3 and low in 2012 at 0.8.
YouTube Users (Covers Monthly Active Users)
A bar graph of YouTube premium subscribers in millions from 2015 to 2020. It is high in 2020 at 30 and low in 2015 at 1.5.
YouTube Premium Subscribers
Summary
This chapter provided an overview of the characteristics of metaverse, XR, and video streaming services. Details of the core technologies as well as business aspects were covered, and selected parts of history were included focusing on companies, products, and services that made a significant influence. In addition, the market’s potential and future expectations were also introduced. In the following Chapter 2, details of metaverse XR technologies, which include metaverse platforms, XR devices, system components, workflow, hardware modules, software engines, as well as haptic devices, are introduced.