Chapter 6. YANG Data Modeling Developments in the Industry

This chapter covers

  • The industry “groups” that develop the different data model–driven management components

  • The “groups” that specify the YANG data models

  • How the industry collaborates... or overlaps

  • Where to find the YANG models: GitHub repositories and the YANG Catalog

  • Interoperability testing

Introduction

This chapter analyzes the current state of affairs of YANG in the industry, how the industry organizes itself, and who the key players involved are.

Because automation based on data model–driven management is a priority in the industry today, and because multiple “groups” are involved, the landscape keeps evolving. Those “groups” consist of standards-development organizations (SDOs), consortiums, forums, open source projects, and even networking equipment vendors with their native/proprietary models. We have attempted to keep a chronological order, whenever possible, to help you understand how the industry has evolved.

The Beginning: The IETF

As you know from the previous chapters, the Internet Engineering Task Force (IETF) specified the Network Configuration Protocol (NETCONF), then the RESTCONF protocol. Starting with the NETMOD IETF working group, the IETF started to specify some standard models. To provide you a look back in history, Figure 6-1 presents the key data model–driven RFCs.

A timeline depicting the history of key data model-drive RFCs.

Figure 6-1 Timeline for Key Data Model–Driven RFCs

Once YANG knowledge spread out throughout the IETF, each working group became responsible for the specifications of the YANG module for their specific technologies, which certainly makes sense. As an example, the segment routing experts in the IETF Source Packet Routing in Networking (SPRING) working group1 are the right persons to describe how to manage a segment routing network. With the help of the YANG Doctors2—a group of YANG experts at the IETF in charge of reviewing every YANG module before ratification for correctness, ease of use, and consistency—the IETF produced many YANG module specifications. At the time of writing, more than 85 YANG modules are standardized, as displayed in Figure 6-2 (source: http://www.claise.be/IETFYANGOutOfRFC.png).3 Note that many, if not all, reports, graphs, and figures initially created on http://www.claise.be4 are being migrated to the yangcatalog.org 5 website.

A graph presents the trend in the RFC YANG modules development since September 2015 until March 2019.

Figure 6-2 IETF YANG Modules and Submodules from RFCs

Table 6-1 (source: http://www.claise.be/IETFYANGOutOfRFC.html)6 contains all the YANG modules produced in RFCs. From the YANG module names themselves, you can already guess the content. Note also that some YANG modules have multiple versions, meaning that they were updated (for example, [email protected] and [email protected]). In this particular case, the latter version offers Network Management Datastore Architecture (NMDA) compliance.

Table 6-1 IETF YANG Modules and Submodules from RFCs

YANG Module (and Submodule)

RFC

YANG Module (and Submodule)

RFC

[email protected]7

RFC 7317

[email protected]

RFC 5717

[email protected]8

RFC 8348

[email protected]

RFC 7758

[email protected]

RFC 7224

[email protected]

RFC 6243

[email protected]

RFC 8294

[email protected]

RFC 6241

[email protected]

RFC 6095

[email protected]

RFC 8345

[email protected]

RFC 8342

[email protected]

RFC 8345

[email protected]

RFC 8313

[email protected]

RFC 8345

[email protected]

RFC 8407

[email protected]

RFC 8345

[email protected]

RFC 8348

[email protected]

RFC 8342

[email protected]

RFC 8348

[email protected]

RFC 8040

[email protected]

RFC 8431

[email protected]

RFC 8040

[email protected]

RFC 6021

[email protected]

RFC 8294

[email protected]

RFC 6991

[email protected]

RFC 8022

[email protected]

RFC 7223

[email protected]

RFC 8349

[email protected]

RFC 8343

[email protected]

RFC 7407

[email protected]

RFC 7277

[email protected]

RFC 7407

[email protected]

RFC 8344

[email protected]

RFC 7407

[email protected]

RFC 6728

[email protected]

RFC 7407

[email protected]

RFC 8022

[email protected]

RFC 7407

[email protected]

RFC 8349

[email protected]

RFC 7407

[email protected]

RFC 8022

[email protected]

RFC 7407

[email protected]

RFC 8349

[email protected]

RFC 7407

[email protected]

RFC 8022

[email protected]

RFC 7407

[email protected]

RFC 8349

[email protected]

RFC 7407

[email protected]

RFC 8177

[email protected]

RFC 7407

[email protected]

RFC 8466

[email protected]

RFC 7407

[email protected]

RFC 8346

[email protected]

RFC 7317

[email protected]

RFC 8346

[email protected]

RFC 6087

[email protected]

RFC 8049

[email protected]

RFC 8407

[email protected]

RFC 8299

[email protected]

RFC 8366

[email protected]

RFC 8194

[email protected]

RFC 8347

[email protected]

RFC 8194

[email protected]

RFC 7407

[email protected]

RFC 8194

[email protected]

RFC 7895

[email protected]

RFC 8512

[email protected]

RFC 7952

[email protected]

RFC 6536

[email protected]

RFC 8072

[email protected]

RFC 8341

[email protected]

RFC 6643

[email protected]

RFC 6022

[email protected]

RFC 6021

[email protected]

RFC 6470

[email protected]

RFC 6991

In order to help with the YANG module development, and in particular with the validation, all YANG modules are extracted from IETF drafts thanks to xym,9 (eXtract Yang Module), a tool initially developed by Jan Medved and then improved by Einar Nilsen-Nygaard. From there, the modules are run through four validators (pyang, yanglint, confdc, and yangdump-pro). All validation errors and warnings are analyzed and provided to the authors, via web reports on the same website.

The top line in Figure 6-3 (source: http://claise.be/IETFYANGPageCompilation.png)10 shows the constant increase of YANG modules in IETF drafts, even if some are now published in RFCs, at which point the related IETF drafts disappear. The middle line in Figure 6-3 shows the increase in YANG modules passing validation, implying that the YANG knowledge spread throughout the industry, while the bottom line shows the number of YANG modules facing a validator warning and no errors. xym and the YANG validators are discussed in more detail in Chapter 7, “Automation Is as Good as the Data Models, Their Related Metadata, and the Tools: For the Network Architect and Operator.”

A graph portrays number of YANG modules' compilation.

Figure 6-3 IETF YANG Modules and Submodules Extracted from IETF Drafts: Validation Results

The spikes in the graph are due to a combination of draft publication deadlines, draft expirations, validator improvements (as you learned about YANG, the validators improved along the way), or simply drafts becoming RFCs.

Probably the biggest issue in developing those technology-specific YANG modules is the interconnection between all these modules. Indeed, in order to create services based on those technologies, the YANG modules must work together. This explains the dependencies between them as well as the ripple effect of modifying one of the core modules.

During one of the early IETF hackathons, Jan Medved developed the symd.py11 tool, which generates a variety of YANG module dependency graphs and output suitable for visualization with D3.js12 tools. A typical example is shown in Figure 6-4 (source: http://www.claise.be/ietf-routing.png).13 This example illustrates the importance of the ietf-routing YANG module, as it is imported by many other YANG modules. This ietf-routing YANG module is represented as squares, as it is now published as RFC 8022, as opposed to circles, which indicate IETF drafts. The symd.py tool was later improved by Einar Nilsen-Nygaard and then by Hari Ananthakrishnan. Hari added an extra option to contact all authors of dependent YANG models, with the message, “new imported YANG model updated, please update yours.” This feature is especially important for the import-by-revision YANG feature because when the imported module is updated, all dependent YANG modules must be updated.

A screenshot shows a simulated dependency graph for ietf-routing.

Figure 6-4 ietf-routing Dependency Graph

During the IETF 97 Hackathon, Joe Clarke created an interesting visual dependency tool14 based on the output of symd. Whereas the symd output was either purely textual or static pictures, this tool has more functionalities:

  • The ability to display the IETF draft or RFC from which the YANG module is extracted by moving the mouse over the YANG module

  • The ability to move the YANG modules around (for example, to group the YANG modules stemming from the same document)

  • The ability to discover the next YANG module the community should focus on (the bottleneck is highlighted with a black circle)

For completeness, the same ietf-routing YANG module with the visual dependency tool is displayed in Figure 6-5 (source https://www.yangcatalog.org/yang-search/impact_analysis/?modtags=ietf-routing&orgtags=&recursion=0&show_rfcs=1&show_subm=1&show_dir=both).

The ietf-routing YANG modules are depicted using the visual dependency tool.

Figure 6-5 ietf-routing Dependency Graph in the Visual Dependency Tool

Years ago, by looking at the IETF dependency graph (which shows how all YANG modules extracted from RFCs and drafts depend on each other), the community realized the complexity of producing all those YANG modules. All modules are required at the same time, and they must all work with each other. The goal is certainly not to be able to read all entries in Figure 6-6, but to illustrate the problem of complexity from a high level.

Finally, for the IETF, there are blogs focusing on the current state of affairs of YANG data models in the industry (for example, “YANG Data Models in the Industry: Current State of Affairs,”15 on the claise.be website).

A screenshot shows a simulated dependency graph for all IETF YANG modules.

Figure 6-6 All IETF YANG Modules Dependency Graph

Embracing YANG Throughout the Industry

This section covers the history, in a more or less chronological order, behind an entire industry embracing YANG. Doing so, it explains how the different “groups” have been developing their YANG models for the respective technologies. The intention is to stress that the YANG data modeling language starts to cover many different areas in the industry, by listing the different technologies and use cases. However, there is no intention to start explaining all the references. The authors intentionally confined the scope of this chapter to provide pointers. We understand that reading all those references might be disconcerting, if this is not your field of interest.

The OpenDaylight16 controller project was one of the early YANG adopters. It’s an open source project formed under the Linux Foundation with the goal of furthering the adoption and innovation of software-defined networking (SDN) through the creation of a common vendor-supported framework. Its architecture is based on Model-Driven Service Adaptation Layer (MD-SAL), a set of infrastructure services aimed at providing common and generic support to application and plug-in developers. The MD-SAL approach, entirely based on YANG models, provides service abstraction to unify the northbound and southbound APIs of the SDN controller, but also provides the data structures used in various services and components of an SDN controller itself.

The OpenDaylight MD-SAL documentation, https://wiki.opendaylight.org/view/OpenDaylight_Controller:MD-SAL:Explained, mentions:

  • Decentralized extension mechanism(“augment”)

  • Data vs. document structure validation (“when”)

  • Extensible language (“extention”)

  • Pre-composed objects (“grouping”)

  • Extensible data type hierarchy

  • Inclusion of basic interactions (RPC and notification)

  • Existing tools and community (yang-central, NETCONF/NETMOD IETF WGs, etc.)

As proof of early YANG adoption, the OpenDaylight Lithium release in June 2015 already included more than 480 YANG modules—and that number kept growing with the most recent releases. The next two releases, Beryllium (Feb 2016) and Boron (Nov 2016), included 703 and 857 YANG modules, respectively.

Then the Broadband Forum (BBF),17 a nonprofit consortium focusing on the development of broadband specifications, adopted YANG as the data modeling language for the access network (not, for example, for the home, which is covered by TR-069 and its successor User Services Platform [USP]). At the time of writing, more than 200 BBF YANG (sub)modules cover technologies such as Ethernet, fiber, Digital Subscriber Line (DSL), Very-high-bit-rate DSL (VDSL), Fiber to the Distribution Point (FTTdP), PPPoE, Passive Optical Networking (PON), and more, as well as common YANG modules for interfaces, subinterfaces, subscribers, types, and quality of service (QoS).

BBF realized very early that it did not make sense to reinvent the wheel. Wherever possible and practicable, the BBF modules leveraged and extend modules from other SDOs; for example, some BBF modules import the IETF interface’s YANG module. Therefore, even though the “work in progress” in BBF is only available to BBF members, an exception was made for the YANG modules, which were posted in GitHub (https://github.com/YangModels/yang,18 under “draft”). This way, the toolchain in place for the IETF also benefits BBF.

MEF Forum19 also started to develop YANG modules. Formerly known as Metro Ethernet Forum, MEF is no longer focusing only on Carrier Ethernet; its mission is to enable the development and worldwide adoption of agile, assured, and orchestrated network services.

Its scope is all connectivity services, but primarily focusing on the product and service layers (as opposed to network devices). As such, MEF Forum focuses on the service’s YANG modules. Very early in the process, MEF Forum created MEF 38, “Service Operations, Administration, and Maintenance (OAM) Fault Monitoring,” and MEF 39 is “Service OAM Performance Measurement” (G.8013/Y.1731 PM). Then it focused on the following:

Note

More YANG models are expected for legato and presto APIs for other technologies (IP and optical) and other operations (service assurance, performance reporting, and so on). YANG for interprovider APIs is also expected.

Another SDO, the Institute of Electrical and Electronics Engineers (IEEE),22 which historically has specified MIB modules, even for configuration, started to develop YANG for its set of technologies. The IEEE 802.1 working group, dealing with local area networks (LANs) and metropolitan area networks (MANs), published its first YANG-based standard in June of 2018: A YANG model for 802.1Q bridging (802.1Qcp), i.e. Customer VLAN bridges, Provider bridges, and Two-Port MAC relays. There is also some work in progress in 802.1, related to 802.1Xck, Port-Based Network Access Control YANG model, close to being published as a standard:

  • 802.1Qcx. CFM (Continuity Fault Management OAM protocol).

  • 802.1Qcw. YANG data models related to TSN (Time sensitive networking).

  • 802.1CBcv FRER (Frame Replication and Elimination for Reliability). Project just started, related to TSN.

  • 802.1AX. Link aggregation. Some discussion about producing a related YANG model.

The 802.3 working group, which focuses on the physical layer and data link layer’s Media Access Control (MAC) of wired Ethernet, is busy working on an Ethernet YANG covering Ethernet interfaces, Link OAM, Ethernet PON and Power over Ethernet (in 802.3cf). For years now, there was a close collaboration between the IEEE and IETF, focusing on the transition from MIB to YANG in the IEEE. Stemming from this effort, the IEEE created the YANGsters, the IEEE 802 YANG editors’ coordination group. This group is responsible for discussing common practice for YANG models supporting IEEE 802 protocols. This common practice includes, but is not limited to, Universal Resource Name (URN) root, a common approach to style and layout for consistency in 802, tooling, and process. While the primary attendees are expected to be editors of existing IEEE 802 YANG projects, other experts interested in YANG are welcome.

Exactly like the IETF and BBF, both the MEF Forum and the IEEE posted their YANG modules in GitHub (https://github.com/YangModels/yang).18 This GitHub repository has evolved into a place where most YANG modules throughout the industry are posted: for specifications (ratified standards), for work in progress (drafts), and even for vendor-specific modules. As those modules depend on each other, having a single location with one set of toolchains makes a lot of sense. Reports containing the validation results and the statistics of all public industry efforts are reported at http://www.claise.be/2018/06/ietf-yang-modules-statistiques/.4

Some other “groups” that joined the YANG community include the following:

  • The Distributed Management (DMTF) task force considered YANG in their Redfish project,23 a RESTful interface utilizing JavaScript Object Notation (JSON) and OData, with the goal of addressing all the components in a data center with a consistent API. Basically, the Redfish project wants to use the networking YANG models developed by the IETF, convert them to Common Schema Definition Language (CSDL, a JSON encoding of OData, which is an OASIS standard) in order to have JSON schemas and provide access to the switches with a Redfish interface.

  • The Open Source Sysrepo project, a YANG-based24 configuration and operational datastore for Unix/Linux applications, typically home gateways. Applications can use sysrepo to store their YANG-modeled configuration instead of using flat configuration files. Sysrepo ensures data consistency of the data stored in the datastore and enforces data constraints defined by the YANG model. With the help of the Netopeer 2 NETCONF server, the applications also automatically become remotely manageable via NETCONF.

  • The ITU Telecommunication Standardization Sector (ITU-T)25 started developing their first YANG model for Ethernet ring protection.

  • The Open ROADM Multi-Source Agreement (MSA)26 defines interoperability specifications for Reconfigurable Optical Add/Drop Multiplexers (ROADM). Included are the ROADM switch as well as transponders and pluggable optics. Specifications consist of both optical interoperability as well as YANG data models. The GitHub location for the YANG module is https://github.com/OpenROADM/OpenROADM_MSA_Public.

  • FD.io, a vector packet processing technology. The “Honeycomb” agent exposes YANG models for VPP functionality via NETCONF and RESTCONF. A controller that supports NETCONF/YANG, such as OpenDaylight, can “mount” the Honeycomb management agent.

  • xRAN was formed to develop, standardize, and promote a multivendor, extensible “lower-layer split” Radio Access Network (xRAN) architecture. It has standardized the interfaces between critical elements of the xRAN architecture, specifically between the remote “radio unit” and the centralized “lower-layer-split central unit.” From a management plane perspective, xRAN has defined the use of IETF’s NETCONF/YANG standard for programmatically configuring and managing its lower-layer split RAN architecture.

  • The 3rd Generation Partnership Project (3GPP) produces the reports and specifications that define 3GPP technologies. Lately, it also focuses on producing YANG modules: 3GPP has active work items to define YANG models for the Network Resource Model (NRM), Network Slicing, and NR RAN (Radio Access Network). A “3GPP YANG solution set style guide” is also in the works.

  • European Telecommunication Standards Institute (ETSI) Network Functions Virtualization (NFV) is the home for the Industry Specification Group for NFV. Its publications provide detailed specifications (Release 2 and Release 3) and early proof of concepts (PoCs). As part of those specifications, ETSI has published a set of information models that define the Virtual Network Function Descriptor (VNFD) in ETSI NFV IFA 01127 and Network Services Descriptor (NSD) in ETSI NFV-IFA 014.28 The solutions Working Group within ETSI took up the task of defining data models for these information models. One of those tasks is defining a YANG model for the two information models. An early draft is available today, with the final standard being available by 2019.

Figure 6-7 shows some of the different open source projects and SDOs involved in YANG, mapping them based on the hierarchy of infrastructure, management and control, and services (svcs).

A figure depicts the various projects and SDOs involved in YANG.

Figure 6-7 Open Source and SDOs Landscape

The OpenConfig YANG Model

From the previous section, there is clearly one key “group” missing from the list: OpenConfig. In regard to its importance today, OpenConfig deserves its own section in this chapter, even though Chapter 2, “Data Model–Driven Management,” covers it briefly.

OpenConfig is an informal working group of network operators, let by Google, sharing the goal of moving networks toward a more dynamic, programmable infrastructure by adopting software-defined networking principles such as declarative configuration and model-driven management and operations.

Although the IETF started to specify some YANG modules earlier, the OpenConfig effort stemmed from a certain frustration that those YANG modules were not being produced fast enough. This was certainly reinforced by the strict IETF adherence to the YANG specifications, which dictate that a new YANG module version can only introduce backward-compatible changes. The OpenConfig consortium is hard at work defining standard YANG models for a variety of technologies. Since the OpenConfig consortium members are not equipment manufacturers, the models will contain the features the network operators are actually interested in, modeled in a way that is a little removed from the actual implementation. This is a great starting point. OpenConfig works with its own GitHub repository. Since there is a handful of committers for that repository, this guarantees some compatibility with a new set of YANG modules (called a bundle). OpenConfig allows non-backward-compatible changes, which are tagged with the semantic versioning (semver.org).

So far, OpenConfig has produced about 60,000 lines of code for YANG modules. This is on par with what IETF was able to produce, even though the IETF model saw many more participants and was developed over a much longer time period. Example 6-1 shows how many lines of YANG text there is in each directory of the OpenConfig repository. Each dot represents 100 lines.

Example 6-1 Amount of YANG Code in Different OpenConfig Subdirectories

Acl                 1.5k ...............
Aft                 1.2k ............
Bfd                  .7k .......
Bgp                 5.1k ...................................................
Catalog             1.0k .........
Interfaces          3.6k ....................................
Isis                7.2k .................................................
Lacp                 .5k .....
Lldp                 .9k .........
local-routing        .4k ....
mpls                5.5k ......................................................
multicast            .9k .........
network-instance    2.0k ....................
openflow             .4k ....
optical-transport   4.3k ...........................................
ospf                4.7k ...............................................
platform            2.1k .....................
policy              1.3k .............
policy-forwarding   1.0k ..........
probes               .6k ......
qos                 2.2k ......................
relay-agent          .8k ........
rib                 2.7k ...........................
segment-routing     1.2k ............
stp                 1.0k ..........
system              3.6k ....................................
telemetry            .9k .........
types               1.0k ..........
vlan                 .6k ......
wifi                2.6k ..........................

The OpenConfig YANG modules use a slightly different dialect of YANG than what is specified in YANG 1.0 (RFC 6020) or YANG 1.1 (RFC 7950). For example, the regular expressions used in pattern statements are so-called perl-regex rather than W3C-regex. For some reason, many XPath expressions have been and remain incorrect today, even in the published standard YANG files. The problems are certainly fixable by YANG-savvy engineers who can patch up the modules before compiling them, but this has led to a good amount of interoperability issues with other tools and vendors.

The OpenConfig model uses a different convention for structuring the content than the original IETF modules. It is also different from the new IETF NMDA style. Instead of having two separate lists for configuration and state data (original IETF), the OpenConfig model has a single list with a config branch and a state branch underneath, as shown in Example 6-2.

Example 6-2 A Tree Representation of openconfig-interfaces.yang (Abridged)

module: openconfig-interfaces
  +--rw interfaces
     +--rw interface* [name]
        +--rw name     -> ../config/name
        +--rw config
        |  +--rw name?          string
        |  +--rw type           identityref
        |  +--rw mtu?           uint16
        |  +--rw description?   string
        |  +--rw enabled?       boolean
        +--ro state
        |  +--ro name?           string
        |  +--ro type            identityref
        |  +--ro mtu?            uint16
        |  +--ro description?    string
        |  +--ro enabled?        boolean
        |  +--ro ifindex?        uint32
        |  +--ro admin-status    enumeration
        |  +--ro oper-status     enumeration
        |  +--ro last-change?    oc-types:timeticks64

This solves the problem of teaching computers how to navigate between the configuration and state data sides for a given interface (or other objects in other modules). A downside with this approach is that it does not lend itself well to situations with preconfiguration or operational objects that have no configuration. Only configured elements may exist here.

A minor quirk that anyone looking to be hands-on with OpenConfig modules needs to get acquainted with is that the name of every configuration item must be provided twice. Let’s say you’re configuring a new interface called “GigEth3/3.” You have to fill in this name under /interfaces/interface/name, then again under /interfaces/interface/config/name. The names have to match, or else the configuration is invalid.

As a reminder, Google also developed (under the OpenConfig umbrella) the gRPC Network Management Interface (gNMI) as a unified management protocol for streaming telemetry and configuration management that leverages the Open Source gRPC framework.

Industry Coordination Is Required

The rapid growth of YANG models has not come without its challenges. Primary among the challenges is the coordination of all these models. While all models are doing a great job of defining how a particular feature can be configured or monitored, service composition–based data model–driven management requires the modeling of all aspects of the configuration and monitoring. Therefore, all YANG should not only be published at the same time, but they need to interact with others. This interaction does not limit itself to the IETF, as other SDOs are also involved. A couple of years ago, as part of an investigation of the dependencies, the image shown in Figure 6-8 was created. This figure contains a few thousand YANG modules known to public GitHub repositories and displays the relationships among them.

A screenshot shows a simulated all YANG modules dependency graph, YANG catalog symbol.

Figure 6-8 All YANG Modules Dependency Graph, YANG Catalog Symbol

Clearly the development of all these YANG modules required cross-industry coordination, which the IETF tried to lead. In 2014, the IESG redistributed the area directors’ workload in order to allow for resources to be focused on YANG model coordination. Primary oversight responsibility and coordination of this work across areas (AD document ownership) become the responsibility of Benoit Claise. A YANG model coordination group was created to assist and complement the work of the YANG doctors and area director. The IETF’s goal was not to standardize all the YANG modules in the world, but to focus on the couple hundred YANG modules at the center of the picture (that is, the core set of models from which all others depend, such as the interfaces, the access lists, the routing, and so on).

The second reason why coordination across the industry is required concerns the versioning. Even if great care was taken when creating those initial YANG models, it is wholly unrealistic to believe that they are complete, perfect, and will remain so forever. This adds another dimension to the complexity of this picture: YANG module versioning. New YANG module versions incorporate two types of changes: backward-compatible changes for completeness and non-backward-compatible changes. How to document the latter and minimize its automation cost (for both the client and the server) is at the heart of many discussions these days.

To facilitate the coordination and versioning issues in the industry, the YANG Catalog (https://www.yangcatalog.org)5 was created, with the goal of becoming a one-stop site for YANG modules information and related tooling. The benefit, compared to a normal GitHub repository, resides in the following:

  • The ability to validate YANG modules (including IETF drafts) with multiple validators

  • The related metadata regarding implementation, maturity level, model type, and so on

  • The ability to visualize the dependencies between YANG modules, including the bottleneck in case of standardization at the IETF

  • The search capabilities on any field, YANG keyword, and metadata: a useful feature for YANG module consumers but also for YANG module designers (facilitating re-use)

  • The REST APIs to query and post any content

  • Demonstration of the connection to data model–driven management with open source tools such as YANG explorer and YANG Development Kit

The YANG Catalog adopted the image in Figure 6-8 as its primary logo on the website.

Interoperability Testing

Specifying standards is a compulsory first step, but testing that the different implementations interoperate is another important one. Interoperability tests help implementers identify any bugs in the code and any ambiguities in the IETF RFC specifications. In short, these tests help fix implementations and specifications. Also, note that the interoperability between genetically different implementations is one of the requirements in IETF in order to advance a protocol on the standards track (based on the conditions from RFC 6410).

A single-situation scenario does not impose software interoperability, which is the case of a single open source implementation that’s used by everybody. While this might be true for a ubiquitous tool, this is clearly not the case for the NETCONF/RESTCONF/gRPC/etc. protocols and YANG modules to be implemented by multiple vendors.

Tooling development and interoperability testing typically happen in hackathons, with the IETF hackathon29 being precursory in the world of data model–driven management. From there, different industry public (or even individual) efforts contribute to the toolbox, as you will see in Chapter 7.

The IETF also organized some protocols interoperability testing. For example, during the IETF 85, participants from CESNET, Jacobs University, Juniper, MG-Soft, SegueSoft, Tail-f, and YumaWorks tested the interoperability of their NETCONF client, NETCONF server, NETCONF browser, and test suite. As a consequence of those robust set of interoperable implementations (see the interop high-level report in the IETF 85 NETCONF proceedings,30 https://www.ietf.org/proceedings/85/slides/slides-85-netconf-3.pdf), NETCONF was deemed mature and the set of NETCONF-related RFCs ready for advancement on the standards track. For example, the NETCONF Configuration Protocol RFC 4741 was obsoleted by the Network Configuration Protocol (NETCONF) RFC 6241.

Independent of the IETF, the European Advanced Networking Test Center (EANTC) arranged a series of NETCONF, RESTCONF, and YANG interoperability test events. EANTC is an institute spun off from the Technical University in Berlin in 1991. Since then, the institute helped many organizations test their networking solutions, and frequently takes on the role of independent certification body.

One of the EANTC initiatives most appreciated in the industry is the yearly interop event held for two weeks on-site in Berlin in the early spring. This is neutral ground where vendors from all over the world meet and test out their implementations of new standards with each other. EANTC defines the test cases to build every year. When vendors manage to prove that a test case is working exactly the way prescribed in the EANTC test specification, the combination is recorded. All recorded interop combinations are then presented at the MPLS+SDN+NFV World Congress in Paris some weeks later and also described in a detailed, downloadable report.

All the interop work happens under a strict nondisclosure agreement (NDA), which means any failed combinations, of which there are many, are not spoken about publicly. This has the great advantage of allowing engineers to be bold and try out interesting combinations that are on the bleeding edge. This is truly a great place to be for learning. There aren’t many other occasions when interop testing happens, except at actual customer sites, where boldness is generally seen rather sparingly.

NETCONF/YANG interop testing has been part of the EANTC interop program since 2015. Both controllers/orchestrators and devices of diverse kinds show up there to participate in the prescribed test cases. The 2018 edition of the interop event had about 50 test cases described in the catalog, of which three were specifically targeting NETCONF/YANG. The test cases at the event involved an L3VPN, an L2VPN, and an NFV setup. Additional use cases involved NETCONF/YANG interfaces with a focus on other technologies. The official test report is found at http://www.eantc.de/fileadmin/eantc/downloads/events/2017-2020/MPLS2018/EANTC-MPLSSDNNFV2018-PostReport_online.pdf.31

Implementing More Than One YANG Model for a Specific Functionality

As you know from Chapter 2, there are different types of YANG modules: some stem from SDOs; some from consortiums, forums, and open source projects; and some from the native/proprietary models that directly map the networking vendor internal database or command-line interface (CLI) representations. The industry today still pursues all three tracks, sometimes with competing solutions.

Therefore, some devices implement more than one YANG model to manage the same functionality—for example, one proprietary (“native”) YANG module for interface management, plus the IETF standard module for the same functionality, and the OpenConfig interface module as well. Any one of these could be used to create an interface, change the interface description, or look at the number of lost packets. This is nice since it allows the network operator to choose the style of management interface they prefer and may have management applications adapted for already.

The complexity for the server-side implementer of supporting multiple models simultaneously can be very high, however. Often supporting one model or the other for a given functionality is straightforward, while supporting both (or more than two) at the same time conjures up dragons. It may not even be logically possible to do it right. This is not unlike the bad interactions sometimes seen when taking several medications together.

As a client, it is generally a good idea to choose one style of model and stick with that. At least, it is certainly advisable to not touch multiple YANG modules controlling the same underlying functionality within a single transaction. To explain why, let’s try breaking this recommendation in an example.

Say the user has an NMS application that creates an interface on a device using the OpenConfig YANG model. Then a need arises to update this application so that it also sets the interface frobozz flag to true. Unfortunately, the frobozz flag is proprietary and only exists in the native YANG model of the device. So the user updates the network management system (NMS) application so that after he creates the interface “vpn-${custno}” using the OpenConfig interface module, it uses the native module to set the frobozz flag on the same interface. However, in the native view, no such interface exists yet. Therefore, the NMS application has to create the interface in the native YANG module as well.

Will the device correctly handle the same interface being created twice in the same transaction? As it happens, the frobozz flag only makes sense for tunnel interfaces, so it has a YANG must statement to verify the type of interface. Will the device validate this correctly against the interface type set in the OpenConfig interface YANG model? On this device, when tunnel interfaces are created, they are usually mapped to the native L2TP YANG model in the device. When frobozz mode is enabled, however, many of the parameters should be mapped to functionality in the native Multiprotocol Label Switching (MPLS) YANG instead.

Will the device handle the frobozz mode correctly when the interface details come in through the OpenConfig model? Perhaps. The point is that the complexity of the model-to-model mapping starts to shine through when more than one model is being used. The convenient illusion of coherence the vendor built up with smoke and mirrors starts to come apart when you observe the system from several angles concurrently.

If “perhaps” is not the answer for you, then heed this advice: Choose one YANG module type for configuration and stick with it. There are no YANG rules or NETCONF principles on your side, should you ever get into trouble with a vendor that does not implement what you think is the logical combination of several different YANG modules. In fact, this place may be as far from “interoperable” as it is possible to get.

To top it off, remember that if you collect the configuration from a device using three different YANG models (that is, three different encodings of the same information), it’s bound to take proportionally longer and eat up more of your bandwidth.

On top of that, once you have selected a specific YANG module type for configuration (say, OpenConfig or IETF), stick to that YANG module type for monitoring as well. It is obviously important to be able to automatically map the YANG path used in configuration and in monitoring (for instance, in the case of telemetry), which is possible with the same YANG module type. However, the YANG module structures might be different among the various YANG module types: The IETF modules follow the NMDA structure, while the OpenConfig modules follow a different structure. If service configuration and monitoring use both types of YANG modules, the mapping of the information will require some extra tooling and intelligence, thus adding to the complexity of the service creation and monitoring. Note that the YANG Catalog now contains a new tool for automatic YANG modules mapping between IETF non-NMDA, IETF NMDA, and OpenConfig. This tool proposes a good starting point in mapping the structure of the different module types.

Interview with the Expert

Q&A with Carl Moberg

Carl Moberg is a technology director at Cisco in the Cloud Platform and Solutions Group (CPSG) and leads the engineering product management team for the Network Service Orchestrator (NSO) product. He has spent many years trying to solve network management issues, with special focus on automation and orchestration.

His current focus is on building solutions that make networks more programmable through better abstractions. This effort is largely based on work done in the IETF around the YANG language and the NETCONF and RESTCONF protocols. He is a contributor to the development of the YANG language and has authored or contributed several YANG modules across a number of standards organizations, including the first performance and fault management modules in the MEF, the OF-CONFIG modules for managing OpenFlow switches, and the first-generation YANG modules for CableLabs, and he is co-author of RFC 8199, “YANG Module Classification.” He is an active member of the YANG Doctors in the IETF, reviewing YANG modules as part of the IETF document lifecycle.

Question:

Carl, how do you see software mindsets, practices, and experiences influencing the way forward for networking in general? And why now?

Answer:

Clearly, the benefits of network transformation are quite real. My experience from working with many teams in charge of managing networks at scale is that getting there can be pretty challenging.

We all understand that we are simply not able rely on lengthy, error-prone manual processes and custom-coding efforts every time we need to set up, change, or tear down network services. But when we go after more advanced topics like network programmability and network functions virtualization—not to mention new operation models like DevOps—we quickly find that they are profoundly different from the way network service delivery has been handled in the past.

I believe that the concept of abstractions is fundamental to solving these challenges. In the networking domain, this means taking on the challenge of providing robust and effective abstractions for managing the configuration and state of physical and virtual networks for people with software tools. These abstractions are aimed at connecting the networking domain with the software development domain by mapping the concepts of networking through known and well-understood technologies into the tools used by software practitioners.

Delivery of network services obviously requires more than just automation and orchestration. In order to reliably deliver realistic services, full-stack solutions include at least the following other aspects:

  • Tracking physical resources still matters to a great degree in networking. Inventory systems track available resources and expose them to allow for appropriate placement decisions in the physical world. For example, which is the next available physical port for us to use?

  • Many fundamental networking constructs rely on configuring resources out of a pool. This ranges from picking appropriate (and non-overlapping) IP addresses or VLAN tags, to understanding which Border Gateway Protocol (BGP) community tags to use for which purposes. In these cases, automation systems need to be integrated with logical resource management systems (for example, IP Address Management [IPAM]) to request or hand back resources used by services in flight.

  • Assuring the services by reliably collecting, correlating, and reporting on appropriate measurable aspects of the services delivered in the network. The right approach here is to allow service and assurance automation systems to cooperate on what services are in flight, and how to measure for them.

The end goal is to tie together many sources of information and enrich the required data to be able to satisfy the needs of the end-state configuration and state of the network—and perhaps more importantly, being able to do this in a rapid and cheap way as adjacent systems shift and are being developed and employing tools and technologies that are comfortable to use.

All of this opens up for two disruptive processes. The first is to allow network engineers to venture into software to more efficiently manage their network without losing control over the network. The second is to lower the barrier of entry for software practitioners to become useful in the networking domain. This combined movement can also be said to be at the heart of applying DevOps-style culture and processes across the application and networking domain.

Question:

How did you witness the industry organizing, and how have you been contributing to the change? How did the way of working change?

Answer:

Introducing a new modeling language into an existing industry with its entrenched ways of working and tooling is no small undertaking. My observation is that the arrival of the YANG language was very timely. Mostly in the sense that the urgency in the problem space (programmatic networking) was rapidly growing and that the legacy approaches (for example, ASN.1 for MIBs in SNMP, OMG IDL for CORBA) were either running out of steam or did not meet the basic expectations of the market. YANG had, if you will, a first-mover advantage at the time as the problem space became better understood. It also nicely matched the influx of software engineers into the networking industry as the concept of software-defined networking took hold. This allowed us to build on some very well-understood concepts from computer science around schema languages, type systems, and some fundamental database concepts (for example, locks, rollbacks, and transactions).

This was also the time where Linux became the default operating system choice for new network equipment. This move quickly commoditized the user-land application layer in routers and switches and opened for much wider reuse of commercial and open source software in networking stacks including the management plane. This, combined with increasing pressure from the market (service providers and large enterprises) to focus on lowering the cost of managing networks, resulted in vendors adding more people to their management feature engineering teams, and we saw a sharp rise in people who designed and built products supporting YANG and NETCONF. Practically speaking, this resulted in an increased number of people authoring, reading, and writing YANG modules.

Another interesting effect of this transition was that as YANG became a requirement, we naturally saw many more vendors integrating YANG into their toolchain. This, in turn, contributed to a rise in the availability of open source and commercial tools for authoring and integrating YANG modules in products. We saw a formidable explosion of an ecosystem around this technology, including interoperability testing, standards work, and operator groups (for example, OpenConfig), and YANG became the subject matter of many talks and workshops across industry events.

As a result, we now have a well-understood and robust language, a growing set of standard modules, and vibrant ecosystem with both open source and commercial tooling and runtime software readily available. The cost has been lowered, and the time to market for manageable networking equipment has been significantly reduced. Vendors can focus on the value-providing aspects of their offerings and deliver software-defined manageability.

Question:

What is missing from the technology stack at this point?

Answer:

Going back to the three bullets mentioned in the first question, I think we still have some work to do around service assurance. Most of the fundamental building blocks are in place, but we need to come together for a fully integrated architecture around model-driven telemetry and correlation of the collected data with instances of services. This is, in fact, the next step of the industry. When we have that, we will have a fully understood and well-defined device-to-service software stack using robust standards based on the common experience of the industry.

The next step up in the stack is the OSS/BSS and ITSM systems, so watch this space.

Summary

As YANG became the defacto language for data model–driven management in the networking industry, the landscape of SDOs, consortiums, forums, and open source projects has been quickly evolving. Obviously, the situation still keeps evolving, with new segments of the industry adopting YANG on regular basis. Some of the latest examples, as discussed in this chapter, are the 3GPP and xRAN for the radio access networks (Note that xRAN merged to create the O-RAN alliance [https://www.o-ran.org/]). This chapter covered history, in a more or less chronological order, and explained how the different “groups” have been developing and how all these YANG models came together or actually diverged (with the OpenConfig example). Finally, it touched on the interoperability testing in the industry, with EANTC.

References in This Chapter

Instead of listing all possible links of all different projects, we’ll limit ourselves to just a few references—the primary starting points, if you will—as listed in Table 6-2.

Table 6-2 YANG-Related Documents for Further Reading

Topic

Content

yangcatalog.org

https://yangcatalog.org

Catalog of YANG modules, searchable on keywords and metadata

OpenConfig

http://www.openconfig.net

OpenConfig reference, including a link to the GitHub for YANG modules

YANG GitHub

https://github.com/YangModels/yang

Contains many of the YANG modules in the industry, including some native ones

Endnotes

1. https://datatracker.ietf.org/wg/spring/about/

2. https://datatracker.ietf.org/group/yangdoctors/about/

3. http://www.claise.be/IETFYANGOutOfRFC.png

4. http://www.claise.be/2018/06/ietf-yang-modules-statistiques/

5. https://www.yangcatalog.org

6. http://www.claise.be/IETFYANGOutOfRFC.html

7. https://www.yangcatalog.org/yang-search/module_details/?module=iana-crypt-hash@2014-08-06

8. https://www.yangcatalog.org/yang-search/module_details/?module=iana-hardware@2018-03-13

9. https://github.com/xym-tool/xym

10. http://claise.be/IETFYANGPageCompilation.png

11. https://github.com/xym-tool/symd/tree/7f757df8e901c040a4a74db9a4e7e4656d24ddee

12. https://d3js.org

13. http://www.claise.be/ietf-routing.png

14. https://www.yangcatalog.org/yang-search/impact_analysis/

15. http://www.claise.be/2018/03/yang-data-models-in-the-industry-current-state-of-affairs-march-2018/

16. http://www.opendaylight.org

17. https://www.broadband-forum.org

18. https://github.com/YangModels/yang

19. http://www.mef.net

20. https://www.mef.net/Assets/Technical_Specifications/PDF/MEF_10.3.pdf

21. https://www.mef.net/Assets/Technical_Specifications/PDF/MEF_26.2.pdf

22. https://www.ieee.org

23. http://redfish.dmtf.org

24. http://tools.ietf.org/html/rfc6020

25. https://www.itu.int/en/ITU-T/Pages/default.aspx

26. http://www.openroadm.org/home.html

27. https://www.etsi.org/deliver/etsi_gs/NFV-IFA/001_099/011/02.01.01_60/gs_NFV-IFA011v020101p.pdf

28. https://www.etsi.org/deliver/etsi_gs/NFV-IFA/001_099/014/02.01.01_60/gs_NFV-IFA014v020101p.pdf

29. https://ietf.org/how/runningcode/hackathons/

30. https://www.ietf.org/proceedings/85/slides/slides-85-netconf-3.pdf

31. http://www.eantc.de/fileadmin/eantc/downloads/events/2017-2020/MPLS2018/EANTCMPLSSDNNFV2018-PostReport_online.pdf

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.17.91