Practical Network Automation, Part 2: Nornir
In the first post we explored the power of templating (jinja2), structured data (yaml), and python. In this post we’ll use the Nornir python package to auto-generate device configs and push them to every device in inventory.
Problem Statement
Let’s say you’ve been tasked with configuring VLANs and interfaces on a few switches amongst multiple sites. There are some common VLANs amongst them but there are unique VLANs per site as well. Each device will have different interface configs.
We could choose to hand-configure every switch (insert manual process here) or we could approach the problem programmatically in a way that’s repeatable and scales to hundreds of switches. Let’s tackle this programmatically.
The programmatic approach will need these components:
- Network Device Inventory
- This allows us to manage and group large numbers of devices
- Structured data sets with configuration data
- Typically these would be variables used in configuration files
- One or more template files per device platform (junos, eos, ios, etc)
- You’ll only have to write these once. They are intended to be re-used for every network configuration add/change/delete
- A method for handling communication to devices
- Login, run commands, retrieve operational data, etc.
Nornir is a great tool for the job!
Introduction to Nornir
Nornir is a robust automation framework. It’s raw python, so debugging is simple, and the possibilities are only limited by one’s own python skills. There are many plugins developed for nornir that make network automation seamless.
The highlights of nornir and why we’ll use it:
- It has many inventory options including plugins to DCIM software (netbox, nautobot)
- It supports threading, so operations performed on large inventories happen quickly compared to non-threaded methods
- There’s a rich plugin ecosystem (we’ll use plugins for napalm and jinja2)
- Jinja2 plugin will allow us to generate configurations
- Napalm plugin will be used to push rendered configs to devices in inventory
- Debugging is straightforward (we won’t have to dig through abstraction layers)
Note that we’re using nornir with napalm. Napalm is a common abstraction layer for interacting with network devices and should be investigated by the reader.
Setting Up The Environment
For this example, I’ve deployed 3 vEOS switches in GNS3. To get a copy of vEOS, you only need to create an account with Arista. No active service contracts required.
The code provided can easily be expanded to support any platform with a valid napalm driver.
I’m using Python3.9, but other versions in that ballpark should work fine. Let’s get started!
Check python version:
ahouse@MacBook-Pro ~ % python3.9 -V
Python 3.9.15
Create working directory:
ahouse@MacBook-Pro ~ % mkdir nornir_dev_env
ahouse@MacBook-Pro ~ % cd nornir_dev_env
Create python virtual environment (a self contained development environment):
ahouse@MacBook-Pro nornir_dev_env % python3.9 -m venv nornir_venv
ahouse@MacBook-Pro nornir_dev_env % source nornir_venv/bin/activate
Upgrade pip:
(nornir_venv) ahouse@MacBook-Pro nornir_dev_env % pip install --upgrade pip
Clone code from git repo:
(nornir_venv) ahouse@MacBook-Pro nornir_dev_env % git clone https://github.com/housepbass9664/nornir_examples.git
Navigate to working directory and download required packages:
(nornir_venv) ahouse@MacBook-Pro nornir_dev_env % cd nornir_examples/nornir_env/nornir
(nornir_venv) ahouse@MacBook-Pro nornir % pip install -r ../requirements.txt
Directory Structure
Let’s have a look at the contents in this directory:
(nornir_venv) ahouse@MacBook-Pro nornir % tree
.
├── config.yaml <-- Nornir configuration file
├── get_facts.py <-- Simple nornir play to get device info
├── inventory <-- Inventory directory for hosts and groups
│ ├── groups.yaml <-- Group file
│ └── hosts.yaml <-- Hosts inventory file
├── push_configs.py <-- Nornir play to render and push configs
├── render_configs.py <-- Nornir play to render configs
└── templates <-- Jinja templates directory. Can be expanded
└── eos <-- Arista configuration template
Refer to Nornir’s Documentation for an in depth explanation of the nornir files.
For now, if you’re following the examples, edit your hosts.yaml file to reflect the switches in your lab. Everything in the data and groups values are optional. Just know that the groups map to sites in this example.
inventory/hosts.yaml
switch-1:
hostname: 172.16.77.11
username: admin
password: admin
platform: eos
groups:
- site_1
data:
interfaces:
- name: Et1
mode: access
access_vlan: 50
- name: Et2
mode: access
access_vlan: 51
- name: Et3
mode: access
access_vlan: 100
switch-2:
hostname: 172.16.77.12
username: admin
password: admin
platform: eos
groups:
- site_2
data:
interfaces:
- name: Et1
mode: access
access_vlan: 60
- name: Et2
mode: access
access_vlan: 61
- name: Et3
mode: access
access_vlan: 100
switch-3:
hostname: 172.16.77.13
username: admin
password: admin
platform: eos
groups:
- site_3
data:
interfaces:
- name: Et1
mode: access
access_vlan: 70
- name: Et2
mode: access
access_vlan: 71
- name: Et3
mode: access
access_vlan: 100
As you can see, I’m assigning each host to a group (one site per group). In groups.yaml, each group has unique settings. Any hosts assigned to a group will inherit the group’s settings – ie all hosts assigned to group site_1 inherit all of site_1’s VLANs.
inventory/groups.yaml
global:
data:
standard_vlans:
- id: 100
name: wired_users
- id: 200
name: wireless_users
- id: 300
name: printers
site_1:
groups:
- global
data:
site_vlans:
- id: 50
name: ISP-1
- id: 51
name: ISP-2
site_2:
groups:
- global
data:
site_vlans:
- id: 60
name: ISP-1
- id: 61
name: ISP-2
site_3:
groups:
- global
data:
site_vlans:
- id: 70
name: ISP-1
- id: 71
name: ISP-2
If you look closely, you’ll notice there’s an inheritance model. Hosts can have host-specific data, but can also inherit group data. Groups can inherit data from other groups.
In this configuration, switch-1 inherits site_vlans from group site_1, which inherits standard_vlans from group global. This model is true for all hosts in this configuration.
I highly encourage you to visit this nornir deep dive to aqcuire a deep understanding of nornir’s mechanics.
Let’s look at a nornir play that logs into all devices and retrieves information like software version and number of interfaces.
get_facts.py
from nornir import InitNornir
from nornir_utils.plugins.functions import print_result
from nornir_napalm.plugins.tasks import napalm_get
nr = InitNornir(config_file="config.yaml") #Initialize nornir
results = nr.run(task=napalm_get, getters=["facts"]) #Run nornir task 'napalm_get' against inventory
print_result(results) #Print all results in an ansible-like format
And now let’s run the get_facts play:
python get_facts.py
(nornir_venv) ahouse@MacBook-Pro nornir % python get_facts.py
napalm_get**********************************************************************
* switch-1 ** changed : False **************************************************
vvvv napalm_get ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
{ 'facts': { 'fqdn': 'switch-1',
'hostname': 'switch-1',
'interface_list': [ 'Ethernet1',
'Ethernet2',
'Ethernet3',
'Ethernet4',
'Ethernet5',
'Ethernet6',
'Ethernet7',
'Ethernet8',
'Ethernet9',
'Ethernet10',
'Ethernet11',
'Ethernet12',
'Management1'],
'model': 'vEOS',
'os_version': '4.19.10M-9408997.41910M',
'serial_number': '',
'uptime': 763.7573001384735,
'vendor': 'Arista'}}
^^^^ END napalm_get ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* switch-2 ** changed : False **************************************************
vvvv napalm_get ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
{ 'facts': { 'fqdn': 'switch-2',
'hostname': 'switch-2',
'interface_list': [ 'Ethernet1',
'Ethernet2',
'Ethernet3',
'Ethernet4',
'Ethernet5',
'Ethernet6',
'Ethernet7',
'Ethernet8',
'Ethernet9',
'Ethernet10',
'Ethernet11',
'Ethernet12',
'Management1'],
'model': 'vEOS',
'os_version': '4.19.10M-9408997.41910M',
'serial_number': '',
'uptime': 73923.86437511444,
'vendor': 'Arista'}}
^^^^ END napalm_get ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* switch-3 ** changed : False **************************************************
vvvv napalm_get ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
{ 'facts': { 'fqdn': 'switch-3',
'hostname': 'switch-3',
'interface_list': [ 'Ethernet1',
'Ethernet2',
'Ethernet3',
'Ethernet4',
'Ethernet5',
'Ethernet6',
'Ethernet7',
'Ethernet8',
'Ethernet9',
'Ethernet10',
'Ethernet11',
'Ethernet12',
'Management1'],
'model': 'vEOS',
'os_version': '4.19.10M-9408997.41910M',
'serial_number': '',
'uptime': 73926.61123299599,
'vendor': 'Arista'}}
^^^^ END napalm_get ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Excellent! Our devices are up and have returned information about themselves, as intended.
The naplm EOS driver leverages the EOS rest api for interacting with devices. Make sure this is enabled on the switches by configuring
management api http-commands, thenno shutdown.
The configurations on these switches are mostly out of the box aside from credentials and api settings. The next play we want to run is push_configs.py, which will render all the configurations for each switch according to the inventory file data. The host in site_1 will get site_1’s VLANs, the host in site_2 will get site_2’s VLANs, etc. All hosts will get the VLANs from the global group’s standard_vlans list.
Here’s what the play looks like:
push_configs.py
from nornir import InitNornir
from nornir_utils.plugins.functions import print_result
from nornir_jinja2.plugins.tasks import template_file
from nornir_napalm.plugins.tasks import napalm_configure
nr = InitNornir(config_file="config.yaml")
def generate_config_and_push(task):
"""Render unique device configuration and push to device"""
rendered_config = task.run(task=template_file, template=f"{task.host.platform}", path="templates/").result
configure_devices = task.run(task=napalm_configure, dry_run=False, configuration=rendered_config)
results = nr.run(task=generate_config_and_push)
print_result(results)
When run, this play will generate configs based on inventory data, then push the rendered configs to every host. Here’s the final result:
python push_configs.py
(nornir_venv) ahouse@MacBook-Pro nornir % python push_configs.py
generate_config_and_push********************************************************
* switch-1 ** changed : True ***************************************************
vvvv generate_config_and_push ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- template_file ** changed : False ------------------------------------------ INFO
vlan 50
name ISP-1
vlan 51
name ISP-2
vlan 100
name wired_users
vlan 200
name wireless_users
vlan 300
name printers
interface Et1
no shutdown
switchport mode access
switchport access vlan 50
interface Et2
no shutdown
switchport mode access
switchport access vlan 51
interface Et3
no shutdown
switchport mode access
switchport access vlan 100
---- napalm_configure ** changed : True ---------------------------------------- INFO
@@ -12,11 +12,29 @@
!
username admin privilege 15 role network-admin secret sha512 $6$EwKLU.u9uOVgqAkC$Ab8sPFJK.GmZcvL2SdoFPYYr6kYG17q39DClMAv3C6zaoEOlWy0IwJ68cdzkd1CKlU0f18G2DShox3Bzwoz630
!
+vlan 50
+ name ISP-1
+!
+vlan 51
+ name ISP-2
+!
+vlan 100
+ name wired_users
+!
+vlan 200
+ name wireless_users
+!
+vlan 300
+ name printers
+!
interface Ethernet1
+ switchport access vlan 50
!
interface Ethernet2
+ switchport access vlan 51
!
interface Ethernet3
+ switchport access vlan 100
!
interface Ethernet4
!
^^^^ END generate_config_and_push ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* switch-2 ** changed : True ***************************************************
vvvv generate_config_and_push ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- template_file ** changed : False ------------------------------------------ INFO
vlan 60
name ISP-1
vlan 61
name ISP-2
vlan 100
name wired_users
vlan 200
name wireless_users
vlan 300
name printers
interface Et1
no shutdown
switchport mode access
switchport access vlan 60
interface Et2
no shutdown
switchport mode access
switchport access vlan 61
interface Et3
no shutdown
switchport mode access
switchport access vlan 100
---- napalm_configure ** changed : True ---------------------------------------- INFO
@@ -12,11 +12,29 @@
!
username admin privilege 15 role network-admin secret sha512 $6$EwKLU.u9uOVgqAkC$Ab8sPFJK.GmZcvL2SdoFPYYr6kYG17q39DClMAv3C6zaoEOlWy0IwJ68cdzkd1CKlU0f18G2DShox3Bzwoz630
!
+vlan 60
+ name ISP-1
+!
+vlan 61
+ name ISP-2
+!
+vlan 100
+ name wired_users
+!
+vlan 200
+ name wireless_users
+!
+vlan 300
+ name printers
+!
interface Ethernet1
+ switchport access vlan 60
!
interface Ethernet2
+ switchport access vlan 61
!
interface Ethernet3
+ switchport access vlan 100
!
interface Ethernet4
!
^^^^ END generate_config_and_push ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* switch-3 ** changed : True ***************************************************
vvvv generate_config_and_push ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- template_file ** changed : False ------------------------------------------ INFO
vlan 70
name ISP-1
vlan 71
name ISP-2
vlan 100
name wired_users
vlan 200
name wireless_users
vlan 300
name printers
interface Et1
no shutdown
switchport mode access
switchport access vlan 70
interface Et2
no shutdown
switchport mode access
switchport access vlan 71
interface Et3
no shutdown
switchport mode access
switchport access vlan 100
---- napalm_configure ** changed : True ---------------------------------------- INFO
@@ -12,11 +12,29 @@
!
username admin privilege 15 role network-admin secret sha512 $6$EwKLU.u9uOVgqAkC$Ab8sPFJK.GmZcvL2SdoFPYYr6kYG17q39DClMAv3C6zaoEOlWy0IwJ68cdzkd1CKlU0f18G2DShox3Bzwoz630
!
+vlan 70
+ name ISP-1
+!
+vlan 71
+ name ISP-2
+!
+vlan 100
+ name wired_users
+!
+vlan 200
+ name wireless_users
+!
+vlan 300
+ name printers
+!
interface Ethernet1
+ switchport access vlan 70
!
interface Ethernet2
+ switchport access vlan 71
!
interface Ethernet3
+ switchport access vlan 100
!
interface Ethernet4
!
^^^^ END generate_config_and_push ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In the output is the result of the template_file and napalm_configure tasks. When run successfully, they show us the rendered templates and the configuration changes made on the devices.
As you can see, we’re quickly ramping up potential for configuration management at large scale. If we wanted to include some juniper and cisco switches, all that’s needed is a jinja2 template file in the templates directory named after the platform defined for the hosts. Note the platform should match napalm’s expected platform name.
It is extremely important to note that we’ve just performed a configuration merge, which is not ideal for a good long term solution. I will cover the proper solution and give more examples later in the series.
The next post will outline important things to consider when automating your network.