Chapter 12
YAML
YAML Ain’t Markup Language (YAML) is a user-friendly data serialization language that is useful for engineers working with data and building device configurations. YAML was first proposed in 2001 by Clark Evans. As per the initial draft, YAML was said to stand for “Yet Another Markup Language,” but the name was later revised to “YAML Ain’t Markup Language.” As the name suggests, the creators of YAML didn’t want it to become yet another random markup language but wanted to put more emphasis on data representation and human readability of data. YAML uses Unicode printable characters and has minimal structural characters, which means it is easy to use YAML to represent and understand data in a meaningful way. To date, three versions of YAML have been released, the latest of which is Version 1.2. The latest specification of YAML can be found at www.yaml.org. YAML is well supported by modern programming languages such as Python and Java.
Why do we really need another data representation language when there are already well-known data representation formats such as Extensible Markup Language (XML) and JavaScript Object Notation (JSON), which are heavily used in most network and web-based applications? After reading through this chapter and working with YAML you will realize that it is inherently easier to read and write data with YAML. Because of its exceptionally human-readable way of representing data, YAML has become an important language for IT operations and automation tools such as Ansible. Before digging into the fundamentals of YAML, let’s examine the key differences between XML, JSON, and YAML:
Data representation: XML is a markup language, whereas JSON and YAML are data formats. XML uses tags to define the elements and stores data in a tree structure, whereas data in JSON is stored like a map with key/value pairs. YAML, on the other hand, allows representation of data both in list or sequence format and in the form of a map with key/value pairs. JSON and YAML uses different indentation styles: JSON uses tabs, whereas YAML uses a hyphen (-) followed by whitespace.
Comments: Comments makes it easier to understand and interpret data. Whereas JSON has no concept of comments, XML allows you to add comments within a document. YAML was designed for readability and thus allows comments.
Data types: XML supports complex data types such as charts, images, and other non-primitive data types. JSON supports only strings, numbers, arrays, Booleans, and objects. YAML, on the other hand, supports complex data types such as dates and time stamps, sequences, nested and recursive values, and primitive data types.
Data readability: It is difficult to read and interpret data written in XML, but it is fairly easy to interpret data in JSON format, and it is much easier to read data in YAML than in JSON format.
Usability and purpose: XML is used for data interchange (that is, when a user wants to exchange data between two applications). JSON is better as a serialization format and is used for serving data to application programming interfaces (APIs). YAML is best suited for configuration.
Speed: XML is bulky and slow in parsing, leading to relatively slow data transmission. JSON files are considerably smaller than XML files, and JSON data is quickly parsed by the JavaScript engine, enabling faster data transmission. YAML, as a superset of JSON, also delivers faster data transmission, but it’s important to remember that JSON and YAML are used in different scenarios.
YAML data is stored in a file that has .yaml or .yml file extension. YAML includes important constructs that distinguish the language from the document markup, and thus there are two basic rules to remember when creating a YAML file:
YAML is case sensitive.
YAML does not allow the use of tabs. Use spaces instead.
As mentioned earlier, YAML provides support for various programming languages. In this chapter, we focus on using YAML-based files with Python. Later in this chapter, you will see how to use YAML files in Python code. Before we get there, the next section focuses on how different types of data are stored in YAML files.
YAML Structure
YAML allows you to easily and quickly format data in human-readable format. A YAML file begins with three hyphens (---), which separate the directives from the document. A single YAML file can contain multiple YAML documents, each beginning with three hyphens. Similarly, three dots (...) represent the end of a document (but do not start a new one). You can start a new YAML document without closing the previous one. Example 12-1 illustrates the beginning and closing of a document. In the first section of the example, notice that multiple documents are started within the same file, without the previous documents being closed. In the second section of the example, each document is closed before another document is started. Example 12-1 also illustrates the use of comments, which start with the pound sign (#). You can add comments anywhere in a document: at the beginning of the document, between the directives, or even on the same line as directives.
Example 12-1 Starting and Closing Documents in YAML
# Routing Protocols on a device --- - OSPF - EIGRP - BGP # Interfaces participating in OSPF --- - Ethernet1/1 # Connected to Gi0/1 on R11 - Ethernet1/2 # Connected to Gi0/2 on R12 - Ethernet1/3 # Connected to Gi0/3 on R13 # Interfaces participating in EIGRP --- - Ethernet1/4 - Ethernet1/5
# Network Events --- - datetime: 05/17/20202 20:03:00 - event: OSPF Flap - intf: Ethernet1/1 ... --- - time: 05/18/20202 13:45:07 - event: OSPF Flap - intf: Ethernet1/3 ...
YAML represents native data structures using a simple identifier called a tag; a single native data structure is called a node. Tagged nodes are rooted and connected using directed graphs, and all nodes can be reached from the root node. A node can have content in one of the three formats:
Sequence: An ordered series of zero or mode nodes that may contain the same node more than once or that may even contain itself.
Mapping: An ordered set of key/value node pairs.
Scalar: An opaque datum that can be presented as a series of zero or more Unicode characters.
YAML Sequences and mappings are both categorized as collections. Let’s now look at how data can be represented in YAML using these different formats.
Collections
YAML’s block collections include sequences and mappings. When you define a block collection, you begin each entry on its own line and use indentation to define the scope of the block. You define a block sequence by beginning an entry with a hyphen (-) followed by whitespace. A mapping is a simple representation of data in key/value pairs, where a key is followed by a colon (:), a space, and then the value. Example 12-2 shows sequences and mappings in YAML.
Example 12-2 Sequences and Mappings in YAML
# Sequence of Scalars # This sequence contains the names of nodes in the network - rtr-R1-ios - rtr-R2-ios-xr - rtr-R3-nx-os
# Mapping scalars to scalars # Below mapping represents node information such as name, OS and software version name: rtr-R1-ios OS: IOS-XE version: 17.1
Note
Maps don’t have to have string keys. A key in a map can be of type integer, float, or another type. The following is an example of a map with a float key:
0.11: a float key
Refer to yaml.org for more details.
Keys in a map can also be complex. You can use a question mark (?) followed by a space to represent a complex key; the key can span multiple lines. Example 12-3 illustrates how to create complex key for mapping.
Example 12-3 Complex Key for Mapping in YAML
# Complex Key ? | This is a Complex key and span across multiple lines : this is its value
YAML also provides the flexibility to represent one format of data in another format—that is, to perform cross-formatting. For instance, you can define a mapping of sequences or a sequence of mappings, as shown in Example 12-4.
Example 12-4 Mapping of Sequences and Sequence of Mappings in YAML
# Mapping of Sequences Nodes: - rtr-R1 - rtr-R2 Intf: - GigabitEthernet0/1 - GigabitEthernet0/2 - GigabitEthernet0/3
# Sequence of Mappings - name: rtr-R1 mgmt-ip: 192.168.1.1 user: admin - name: rtr-R2 mgmt-ip: 192.168.1.2 user: cisco
Along with cross-formatting, YAML supports the flow style of data representation. In the flow style, YAML uses explicit indicators instead of using indentation to denote scope. You write a flow sequence inside square brackets as comma-separated values, and you write a flow mapping inside curly braces. Example 12-5 illustrates the representation of data in a flow sequence and in a flow mapping. A flow sequence can also be understood as a sequence of sequences, and a flow mapping can be understood as a mapping of mappings.
Example 12-5 Flow Sequence and Flow Mapping in YAML
# Flow Sequence - [node-name, mgmt-ip, user ] - [rtr-R1, 192.168.1.1, admin] - [rtr-R2, 192.168.1.2, cisco]
# Flow Mapping
rtr-R1: {mgmt-ip: 192.168.1.1, user: admin}
rtr-R2: {
mgmt-ip: 192.168.1.2,
user: cisco
}
YAML also allows mapping between sequences using complex keys. However, some parsers may return syntax errors with such mappings. Example 12-6 demonstrates the creation of a mapping between sequences using complex keys. This example shows a sequence of node names that are mapped to management IP addresses.
Example 12-6 Mapping Between Sequences Using Complex Keys in YAML
# Mapping Between Sequences using Complex Keys ? - rtr-R1 - rtr-R2 : [192.168.1.1, 192.168.1.2]
Because YAML is a superset of JSON, you can create JSON-style maps and sequences in YAML. Example 12-7 shows a JSON-style representation of a sequence and maps in YAML.
Example 12-7 JSON-Style Sequence and Maps in YAML
# JSON Style Sequence and Maps
json_map: {"key": "value"}
json_seq: [3, 2, 1, "get set go"]
quotes optional: {key: [3, 2, 1, get set go]}
Scalars
YAML scalars are either written in literal block or in folded block styles. YAML scalars written in literal block format use the pipe character (|) literal, whereas scalars written in folded block style use the greater-than symbol (>). Literal block style preserves line breaks, and the literal continues until the end of the indentation and the leading indentation is stripped. If there are more indented lines, the following lines use the same indentation rule. Similarly, in a folded block, each line break is folded to a space unless it ends an empty or a more-indented line. For instance, a multiple-line string can be either written as a literal block using | or as a folded block using >, as shown in Example 12-8.
Example 12-8 Scalars with Literal Block Style and Folded Block Style in YAML
---
literal_block: |
This entire block of text is the value of 'literal_block' key,
with line breaks being preserved.
You can keep adding more lines of text below and maintain the same
Indentation as above.
The 'More indented' lines are represented in the following manner –
In this case, lines following the above line have same indentation.
--- folded_style: > This entire block of text is the value of 'folded_style' key. each linebreak is represented by a space.
Note
It is important to remember that indentation defines the scope of a block.
YAML also include flow scalars, which are represented in plain style or quoted style. The double-quoted style provides escape sequencing, whereas single-quoted style is applicable when escaping is not needed. All flow scalars can span multiple lines. Note that the line breaks are always folded. Example 12-9 illustrates flow scalars using different styles.
Example 12-9 Flow Scalars in YAML
# Plain style flow scalar plain: This is an unquoted scalar and it spans across multiple lines. # Single quoted scalar random: 'This is a single quoted text' str: 'This is not''a # comment''.' # Double quoted flow scalardouble-quoted: "this is a Quoted scalar. " unicode: "The code looks good.u263A"
Tags
YAML tags are used to represent native data structures as well as other data formats, such as sets and ordered maps. In addition, YAML has root nodes as well as nested nodes. The root object or root node, which continues for an entire document, is represented as a map that is equivalent to a dictionary or an object in a different language. When representing numeric data, you generally make use of integer or floating point data types, and such data is represented in the form of integer or floating point tags. Example 12-10 illustrates the representation of integer and floating point tags. Note that the integer tags include integer values, and floating point tags include decimal and exponential values.
Example 12-10 Integer and Floating Point Tags
# Integer Tags canonical: 12345 decimal: +12789 octal: 0o17 hexadecimal: 0xEF
# Floating Point Tags canonical: 1.23015e+3 exponential: 12.6415e+02 fixed: 9870.26 negative infinity: -.inf not a number: .NaN
You can represent null, string, and Boolean values in YAML. These fall under the category of miscellaneous tags. Example 12-11 illustrates the representation of such types of data. There are a few points to remember when dealing with Boolean and string data types. A Boolean value cannot be represented as 1 as 1 is interpreted as a number, not a Boolean value. Similarly, string data is not required to be quoted but can be. Note that the null key or value can be used to represent null data being returned by the data source.
Example 12-11 Miscellaneous Tags
# Miscellaneous Tags null: null_value: null true: y false: n booleans: [ true, false ] string: 'hello world'
YAML can also parse some extra types, such as ISO-formatted date and datetime types. Usually it is possible to parse date or datetime only in programming languages or databases, but YAML allows you to represent date- and time-related data by using date and datetime literals. Example 12-12 demonstrates how to use date and datetime literals in YAML. In Example 12-12, notice that date and datetime data are represented in different ways.
Example 12-12 Date and Datetime YAML Types
--- datetime: 2020-2-15T05:45:23.1Z datetime_with_spaces: 2019-10-17 00:02:43.10 -5 date: 2020-05-13
In YAML, there are also explicit tags and global tags. Explicit tags are denoted using the exclamation point (!) symbol. Explicit tags are used to explicitly represent particular data formats. For instance, the !binary explicit tag indicates that a string is actually a Base64-encoded representation of a binary blob. Global tags, on the other hand, are URIs and may be represented using a tag shorthand notation. Example 12-13 illustrates the representation of both explicit and global tags. Note that the primary tag handle is an exclamation point (!) symbol. Thus, the explicit tag for a string is represented in YAML as !!str.
Example 12-13 Explicit and Global Tags in YAML
# Explicit Tags date-string: !!str 2020-05-10 gif-file: !!binary | R0lGODlhDAAMAIQAAP//9/X17unp5WZmZgAAAOfn515eXvPz7Y6OjuDg4J+fn5 OTk6enp56enmlpaWNjY6Ojo4SEhP/++f/++f/++f/++f/++f/++f/++f/++f/+ +f/++f/++f/++f/++f/++SH+Dk1hZGUgd2l0aCBHSU1QACwAAAAADAAMAAAFLC AgjoEwnuNAFOhpEMTRiggcz4BNJHrv/zCFcLiwMWYNG84BwwEeECcgggoBADs=
# Global Tags %TAG ! tag:network.local,8080: --- !nodes # Use the ! handle for presenting # tag:network.local,8080:node1 - !node1 mgmt-ip: 192.168.1.1 username: admin password: cisco - !node1 mgmt-ip: 192.168.1.2 username: admin password: cisco!123
YAML also supports unordered sets and ordered mappings. A set is represented as a mapping where each key is associated with a null value. Ordered maps are represented as a sequence of mappings, with each mapping having one key. Example 12-14 demonstrates how to define unordered sets and ordered mappings in a YAML file.
Example 12-14 Unordered Sets and Ordered Mappings in YAML
# Sets
---
!!set
? item1
? item2
? item3
# Sets can also be written without any explicit tag
set1:
? item1
? item2
? item3
or: {item1, item2, item3}
# Sets are just maps with null values. set1 can also be represented as below
set2:
item1: null
item2: null
item3: null
# Ordered Maps # Below map lists the items and its prices --- !!omap - item1: 67 - item2: 64 - item3: 51
Anchors
Working with repetitive content can be tedious and tiresome, especially when you’re dealing with larger data sets and when certain section of data are being repeated multiple times. To overcome such challenges, YAML provides a very handy feature known as anchors. An anchor acts as a reference pointer to data already defined once in the document; that reference pointer can be used multiple times later in the YAML configuration files to duplicate the same data. For instance, you can create an anchor for a value of a key and reference that anchor at another place in the document for another key to ensure that the second key has the same values as the first key. An anchor is defined by using an ampersand (&) followed by the anchor name. After the anchor name is defined, it can be referenced at another place by using aliases, which are represented by an asterisk (*) followed by the specified anchor name. Example 2-15 illustrates the use of anchors to duplicate the content in a YAML file.
Example 12-15 Using Anchors in YAML
# Using Anchors key1: &val1 This is the value of both key1 and key2 key2: *val1
Anchors can not only be used for duplicating data but can also be used for duplicating or inheriting properties. This is achieved in YAML by using merge keys. A merge key, represented by a double less-than symbol (<<), is used to indicate that all the keys of one or more specified maps will be inserted into the current map. Example 12-16 demonstrates the use of merge keys in YAML.
Example 12-16 Using Merge Key in YAML
# Merge Key login: &login user: admin pass: cisco node1: &node1 <<: *login mgmt-ip: 192.168.1.1 node: &node2 <<: *login mgmt-ip: 192.168.1.2
YAML Example
So far, this chapter has described the fundamentals of YAML and how various YAML data representation techniques are used to represent data in a human-readable format. Now it is time to put your new knowledge into practice and create a YAML file that makes it easier to understand the data. Example 12-17 shows a configuration file for setting up and enabling networking for network devices and virtual cloud servers in a small data center. You could use this configuration with automation tools for one-click deployment of servers in the network. In this example, default credentials are defined for servers as well as network devices such as routers. Then, under the networking node, domain name and domain name server information that is going to be same across the network is specified. Under the networking node are subnodes for routers and host networks, which are then further expanded to various host profiles, such as tenant, management, storage, API, and so on. Each profile represents a specific configuration for that segment of the network. This example also shows a very efficient use of merge keys to duplicate the credentials at different places in the configuration file.
Example 12-17 YAML Network Configuration Example
SERVER_COMMON: &srvrcred {server_username: root, password: cisco!123}
DEFAULT_CRED: &login
- user: admin
password: Cisco@123
- user: network-admin
password: Cisco!123
NETWORKING:
domain_name: cisco.com
domain_name_servers: [172.16.101.1]
routers:
- pool: [192.168.1.2 to 192.168.1.253]
gateway: 192.168.1.1
<<: *login
host_networks:
- gateway: 192.168.50.1
pool: [192.168.50.10 to 192.168.50.250]
segments: [management, provision]
subnet: 192.168.50.0/24
vlan_id: 50
<<: *srvrcred
- gateway: 10.117.0.1
pool: [10.117.0.5 to 10.117.0.254]
segments: [tenant]
subnet: 10.117.0.0/24
vlan_id: 117
<<: *srvrcred
- gateway: 10.118.0.1
pool: [10.118.0.5 to 10.118.0.254]
segments: [storage]
subnet: 10.118.0.0/24
vlan_id: 118
- gateway: 10.35.22.1
segments: [api]
subnet: 10.35.22.0/24
vlan_id: 3522
Handling YAML Data Using Python
All data formats are supported by and can be integrated with various programming languages. YAML is a data serialization format designed for better human readability and easy interaction with scripting languages, and it is supported by Python. When dealing with specific format of data, such as XML or JSON, a basic requirement for a library in any programming language is a parser. Python has a YAML parser known as PyYAML. PyYAML is a complete YAML parser for YAML Version 1.1 that supports YAML tags and provides Python-specific tags that allow programmers to represent arbitrary Python objects.
Before you can work with the PyYAML package, you need to install PyYAML. Example 12-18 demonstrates the installation of the PyYAML package inside a virtual environment. When the package is installed, you can go to the Python terminal and import the package named yaml. You can use the dir() method to find all the properties and methods available for the yaml object. Some of the common methods are highlighted in Example 12-18.
Example 12-18 Installing PyYAML and Exploring the PyYAML Package
[root@node1 ~]# virtualenv yaml
Using base prefix '/usr'
No LICENSE.txt / LICENSE found in source
New python executable in /root/yaml/bin/python3
Also creating executable in /root/yaml/bin/python
Installing setuptools, pip, wheel...
done.
[root@node1 ~]# source yaml/bin/activate
(yaml) [root@node1 ~]# pip3 install pyyaml
Collecting pyyaml
Downloading PyYAML-5.3.1.tar.gz (269 kB)
|,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,
ñà,ñà,ñà,ñà,ñà,ñà,ñã| 266 kB 10.7 MB/s eta 0:00:
|,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,ñà,
ñà,ñà,ñà,ñà,ñà,ñà,ñà| 269 kB 10.7 MB/s
Building wheels for collected packages: pyyaml
Building wheel for pyyaml (setup.py) ... done
Created wheel for pyyaml: filename=PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl
size=44621 sha256=10808863673407f7436cc15e471caf30cd06fd1462ac9f7f512d3c431ef37509
Stored in directory: /root/.cache/pip/wheels/e5/9d/ad/2ee53cf262cba1ffd8afe1487eef
788ea3f260b7e6232a80fc
Successfully built pyyaml
Installing collected packages: pyyaml
Successfully installed pyyaml-5.3.1
(yaml) [root@node1 ~]#
[root@node1 demo]# python3
Python 3.7.6 (default, Dec 30 2019, 19:38:28)
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import yaml
>>> dir(yaml)
['AliasEvent', 'AliasToken', 'AnchorToken', 'BaseDumper', 'BaseLoader',
'BlockEndToken', 'BlockEntryToken', 'BlockMappingStartToken',
'BlockSequenceStartToken', 'CollectionEndEvent', 'CollectionNode',
'CollectionStartEvent', 'DirectiveToken', 'DocumentEndEvent',
'DocumentEndToken', 'DocumentStartEvent', 'DocumentStartToken', 'Dumper',
'Event', 'FlowEntryToken', 'FlowMappingEndToken', 'FlowMappingStartToken',
'FlowSequenceEndToken', 'FlowSequenceStartToken', 'FullLoader', 'KeyToken',
'Loader', 'MappingEndEvent', 'MappingNode', 'MappingStartEvent', 'Mark',
'MarkedYAMLError', 'Node', 'NodeEvent', 'SafeDumper', 'SafeLoader', 'ScalarEvent',
'ScalarNode', 'ScalarToken', 'SequenceEndEvent', 'SequenceNode',
'SequenceStartEvent', 'StreamEndEvent', 'StreamEndToken', 'StreamStartEvent',
'StreamStartToken', 'TagToken', 'Token', 'UnsafeLoader', 'ValueToken',
'YAMLError', 'YAMLLoadWarning', 'YAMLObject', 'YAMLObjectMetaclass',
'__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__',
'__package__', '__path__', '__spec__', '__version__', '__with_libyaml__',
'_warnings_enabled', 'add_constructor', 'add_implicit_resolver',
'add_multi_constructor', 'add_multi_representer', 'add_path_resolver',
'add_representer', 'compose', 'compose_all', 'composer', 'constructor',
'dump', 'dump_all', 'dumper', 'emit', 'emitter', 'error', 'events', 'full_load',
'full_load_all', 'io', 'load', 'load_all', 'load_warning', 'loader', 'nodes',
'parse', 'parser', 'reader', 'representer', 'resolver', 'safe_dump',
'safe_dump_all', 'safe_load', 'safe_load_all', 'scan', 'scanner', 'serialize',
'serialize_all', 'serializer', 'tokens', 'unsafe_load', 'unsafe_load_all',
'warnings']
This section focuses on some of the most common methods from the yaml package that can be used when dealing with YAML data. The first function we look at is the yaml.dump() function, which you use to serialize a Python object into a YAML stream. In other words, this function allows you to convert data into a more human-readable format. It produces a YAML document as a UTF-8-encoded str object. You can change the encoding by specifying the encoding attribute as a parameter in the yaml.dump() function. Example 12-19 demonstrates how to use the yaml.dump() method to represent some data about a user in a more readable format. The first section of the example shows some information saved inside a data variable that is in an understandable format; the second section of the example shows that serializing the data into a YAML stream makes the data even easier to understand.
Example 12-19 Serializing a Python Object into a YAML Stream by Using the yaml.dump() Method
! demo.py
import yaml
data = {
"Name": "John Doe",
"Age": 40,
"Company": "XYZ",
"Social-media": {"Facebook": "johndoe",
"twitter": "johndoe"},
"Job Title": "Network Automation Engineer",
"Skills": ["Python", "Nexus", "IOS-XE", "YAML", "JSON"]
}
print(yaml.dump(data))
[root@node1 demo]# python demo.py Age: 40 Company: XYZ Job Title: Network Automation Engineer Name: John Doe Skills: - Python - Nexus - IOS-XE - YAML - JSON Social-media: Facebook: johndoe twitter: johndoe
The yaml.dump() method is used to serialize a Python object and also to write the serialized YAML stream into a file. Two parameters are passed in the yaml.dump() function: the Python object to be converted in the YAML stream and the file to which the stream will be written (see Example 12-20).
Example 12-20 Saving a YAML Stream to a File
! demo.py
import yaml
data = {
"Name": "John Doe",
"Age": 40,
"Company": "XYZ",
"Social-media": {"Facebook": "johndoe",
"twitter": "johndoe"},
"Job Title": "Network Automation Engineer",
"Skills": ["Python", "Nexus", "IOS-XE", "YAML", "JSON"]
}
with open('test.yaml', 'w') as f:
yaml.dump(data, f)
[root@node1 demo]# python3 demo.py [root@node1 demo]# cat test.yaml Age: 40 Company: XYZ Job Title: Network Automation Engineer Name: John Doe Skills: - Python - Nexus - IOS-XE - YAML - JSON Social-media: Facebook: johndoe twitter: johndoe
The yaml.dump() method also allows you to sort a data stream based on keys. The sort_keys attribute, when set to True, sorts the keys in the YAML stream; when set to False, it presents the data as it was inserted in the YAML stream. Example 12-21 demonstrates the difference between sorted and unsorted YAML data streams.
Example 12-21 Sorting YAML Data Streams
import yaml
data = {
"Name": "John Doe",
"Age": 40,
"Company": "XYZ",
"Social-media": {"Facebook": "johndoe",
"twitter": "johndoe"},
"Job Title": "Network Automation Engineer",
"Skills": ["Python", "Nexus", "IOS-XE", "YAML", "JSON"]
}
print("---------Sorted----------")
print(yaml.dump(data, sort_keys=True))
print("--------Unsorted---------")
print(yaml.dump(data, sort_keys=False))
[root@node1 demo]# python3 demo.py ---------Sorted---------- Age: 40 Company: XYZ Job Title: Network Automation Engineer Name: John Doe Skills: - Python - Nexus - IOS-XE - YAML - JSON Social-media: Facebook: johndoe twitter: johndoe --------Unsorted--------- Name: John Doe Age: 40 Company: XYZ Social-media: Facebook: johndoe twitter: johndoe Job Title: Network Automation Engineer Skills: - Python - Nexus - IOS-XE - YAML - JSON
Another important function in the yaml package is the yaml.load() method, which accepts a byte string, a Unicode string, an open binary file object, or an open text file object as the parameter and returns a Python object. In other words, it converts a YAML document into a Python object. Example 12-22 demonstrates how the yaml.load() function returns a Python object. In this example, the data.yaml file contains a mapping of sequences of routers and switches. The demo.py file opens the file data.yaml, and the object is passed as a parameter to the yaml.load() function. Notice that this example also has the Loader attribute set to yaml.FullLoader. The FullLoader parameter handles the conversion from YAML scalar values to Python dictionary format.
Example 12-22 Using the yaml.load() Function
! data.yaml
routers:
- rtr-ios-R1
- rtr-xr-R2
- rtr-nx-R3
switches:
- sw-ios-SW1
- sw-xe-SW2
- sw-nx-SW3
! demo.py
import yaml
with open('data.yaml') as f:
data = yaml.load(f, Loader=yaml.FullLoader)
print(data)
print("
----------YAML Serialized Format------------
")
print(yaml.dump(data))
[root@node1 demo]# python3 demo.py
{'routers': ['rtr-ios-R1', 'rtr-xr-R2', 'rtr-nx-R3'], 'switches': ['sw-ios-SW1',
'sw-xe-SW2', 'sw-nx-SW3']}
----------YAML Serialized Format------------
routers:
- rtr-ios-R1
- rtr-xr-R2
- rtr-nx-R3
switches:
- sw-ios-SW1
- sw-xe-SW2
- sw-nx-SW3
Note
For demonstration purposes, some examples in this section use the yaml.load() function. However, it is not recommended to use the yaml.load() function when data is received from an external and untrusted source. It is thus recommended to use the yaml.safe_load() function. Alternatively, you can use the yaml.load() function but with the Loader value set to yaml.SafeLoader.
The yaml.load() function is useful when you need to parse through a single YAML document. However, if a string or a YAML file contains multiple documents, you should use the yaml.load_all() function to load all the documents. Example 12-23 illustrates the use of the yaml.load_all() function to load all the documents in a YAML file.
Example 12-23 Loading Multiple YAML Documents by Using the yaml.load_all() Function
! data.yaml
# First YAML Document
network-devices:
routers:
- rtr-ios-R1
- rtr-xr-R2
- rtr-nx-R3
switches:
- sw-ios-SW1
- sw-xe-SW2
- sw-nx-SW3
---
# Second YAML Document
hosts:
- srvr1: "linux"
patched: True
OS: CentOS
- srvr2: "Microsoft Windows"
patched: False
OS: “Windows 10”
! demo.py
import yaml
with open('data.yaml') as f:
# yaml.load_all() loads all the YAML documents present in a YAML file
# and returns an object, which can be further iterated
docs = yaml.load_all(f, Loader=yaml.FullLoader)
for doc in docs:
for i,j in doc.items():
print(i, "->", j)
[root@node1 demo]# python3 demo.py network-devices -> {'routers': ['rtr-ios-R1', 'rtr-xr-R2', 'rtr-nx-R3'], 'switches': ['sw-ios-SW1', 'sw-xe-SW2', 'sw-nx-SW3']} hosts -> [{'srvr1': 'linux', 'patched': True, 'OS': 'CentOS'}, {'srvr2': 'Microsoft Windows', 'patched': False, 'OS': 'Windows 10'}]
The YAML package also provides access to lower-level APIs when parsing YAML files. The yaml.scan() method scans through a YAML file and generates tokens that can be used to understand the kind of YAML data being worked on. As described earlier in this chapter, data is presented in YAML in three forms:
Scalar
Sequence
Mapping
The tokens generated by the scan() method list the start and end tokens when parsing any of these three forms of data, as shown in Example 12-24. The scan() method is useful for debugging purposes as it allows you to take relevant actions in the code when a certain type of data is presented in YAML.
Example 12-24 Using the yaml.scan() Method
network-devices:
routers:
- rtr-ios-R1
- rtr-xr-R2
- rtr-nx-R3
switches:
- sw-ios-SW1
- sw-xe-SW2
- sw-nx-SW3
servers:
- srvr1: "linux"
patched: True
OS: CentOS
- srvr2: "Microsoft Windows"
patched: False
OS: "Windows 10"
import yaml
with open('data.yaml') as f:
data = yaml.scan(f, Loader=yaml.FullLoader)
for tkn in data:
print(tkn)
[root@node1 demo]# python3 demo.py StreamStartToken(encoding=None) BlockMappingStartToken() KeyToken() ScalarToken(plain=True, style=None, value='network-devices') ValueToken() BlockMappingStartToken() KeyToken() ScalarToken(plain=True, style=None, value='routers') ValueToken() BlockEntryToken() ScalarToken(plain=True, style=None, value='rtr-ios-R1') BlockEntryToken() ScalarToken(plain=True, style=None, value='rtr-xr-R2') BlockEntryToken() ScalarToken(plain=True, style=None, value='rtr-nx-R3') KeyToken() ScalarToken(plain=True, style=None, value='switches') ValueToken() BlockEntryToken() ScalarToken(plain=True, style=None, value='sw-ios-SW1') BlockEntryToken() ScalarToken(plain=True, style=None, value='sw-xe-SW2') BlockEntryToken() ScalarToken(plain=True, style=None, value='sw-nx-SW3') KeyToken() ScalarToken(plain=True, style=None, value='servers') ValueToken() BlockEntryToken() BlockMappingStartToken() KeyToken() ScalarToken(plain=True, style=None, value='srvr1') ValueToken() ScalarToken(plain=False, style='"', value='linux') KeyToken() ScalarToken(plain=True, style=None, value='patched') ValueToken() ScalarToken(plain=True, style=None, value='True') KeyToken() ScalarToken(plain=True, style=None, value='OS') ValueToken() ScalarToken(plain=True, style=None, value='CentOS') BlockEndToken() BlockEntryToken() BlockMappingStartToken() KeyToken() ScalarToken(plain=True, style=None, value='srvr2') ValueToken() ScalarToken(plain=False, style='"', value='Microsoft Windows') KeyToken() ScalarToken(plain=True, style=None, value='patched') ValueToken() ScalarToken(plain=True, style=None, value='False') KeyToken() ScalarToken(plain=True, style=None, value='OS') ValueToken() ScalarToken(plain=False, style='"', value='Windows 10') BlockEndToken() BlockEndToken() BlockEndToken() StreamEndToken()
As you have learned in this chapter, YAML is commonly used to build configuration files. These configuration files can be used along with Jinja templates to render configurations for various networking devices. Example 12-25 shows a BGP configuration. In this example, the data.yaml file has a number of BGP-related configurations, such as router ID, network advertisements, and neighbors. This data is rendered into a proper BGP configuration using Jinja templates. Building any configuration using YAML and Jinja involves three basic steps:
Step 1. Load the YAML data.
Step 2. Set the directory path and select the Jinja template.
Step 3. Render the data into the Jinja template to generate the configuration.
Example 12-25 demonstrates this process of generating a configuration.
Example 12-25 Building a Configuration Using YAML and Jinja Templates
! data.yaml
---
bgp:
id: 100
router_id: 1.1.1.1
networks:
- 192.168.1.1/32
- 192.168.12.0/24
- 192.168.13.0/24
neighbors:
- id: 2.2.2.2
remote_as: 200
afi: "ipv4 unicast"
- id: 3.3.3.3
remote_as: 300
afi: "ipv4 unicast"
! templates/bgp_template.j2
router bgp {{ bgp.id }}
router-id {{ bgp.router_id }}
address-family ipv4 unicast
{% for network in bgp.networks %}
network {{ network }}
{% endfor %}
exit address-family
{% for neighbor in bgp.neighbors %}
neighbor {{ neighbor.id }}
remote-as {{ neighbor.remote_as }}
address-family {{ neighbor.afi }}
exit
exit
{% endfor %}
import yaml
from jinja2 import Environment, FileSystemLoader
config_data = yaml.load(open('data.yaml'), Loader=yaml.FullLoader)
env = Environment(loader = FileSystemLoader('./templates'), trim_blocks=True,
lstrip_blocks=True)
template = env.get_template('bgp_template.j2')
print(template.render(config_data))
[root@node1 demo]# python3 demo.py router bgp 100 router-id 1.1.1.1 address ipv4 unicast network 192.168.1.1/32 network 192.168.12.0/24 network 192.168.13.0/24 exit address-family neighbor 2.2.2.2 remote-as 200 address-family ipv4 unicast exit exit neighbor 3.3.3.3 remote-as 300 address-family ipv4 unicast exit exit
Note
Chapter 6, “Python Applications,” further discusses working with YAML data and Jinja templates as well as other Python libraries, such as NAPALM and Nornir.
Summary
YAML is a human-readable data serialization language used to build configurations. YAML is also a superset of JSON, and so there are many similarities between JSON and YAML. YAML allows you to represent data in three different formats:
Scalars
Sequence
Mapping
In this chapter, you have learned that YAML represents native data structures using nodes and tags. Unlike other formats, such as JSON, YAML supports comments. YAML allows you to represent common data structures, such as strings, integers, floating points, and Booleans, as well as complex data structures. YAML supports anchors and merge keys, which help with duplication of data as well as inheritance.
The Python PyYAML package allows programmers to serialize Python objects into a YAML stream by using the yaml.dump() method and a YAML stream to a Python object by using the yaml.load() method. Programmers and network automation engineers can also use Jinja templates along with the PyYAML package to build configurations for network devices.