Posts on Welcome to the World of Tomorrow

NATS and Configuration Minions

Mon, 02 Dec 2024 14:53:36 -0800

Introduction

Two blogs in one year. Who do I think I am? In all seriousness, thank you for all the feedback on the previous blog. As always, it really means a lot. This one might go a bit sideways, but building something like this has always been on my mind, and I got another spark to get it done while attending AutoCon2.

I caught the first half of a talk by Mircea Ulinic from Digitial Ocean. In that talk, Mircea went over how the company leverages automation at an incredible scale. They eventually covered a bit on Salt and how they leverage proxy minions to effectively have a one-to-one mapping to a networking node. Imagine one proxy being deployed for each device; the proxy would handle the job when a task needs to be performed on said device.

I wanted to try making something similar, but with the huge caveat I would only tackle one portion—configuration management. I know everyone talks about configuration management, but stick with me and let me have a little fun in my small world. This is a breakdown of how I got a minimal viable product (MVP) running.

What is NATS

I’m not going to bore anyone with a drawn-out definition, but at its core, NATS is a messaging system that ensures your message will arrive at most once at a subscribed destination. NATS does this in the way of subjects. We might publish a message to say.hello and someone else will be subscribed to listen to messages on say.hello. The subscriber would then get the message and whatever content is included in the message. The NATS server handles the passing of messages. There you go, NATS in a nutshell or paragraph. Side note, the NATS documentation is excellent. Please go check it out.

The Workflow

That sounds cool, but again, this blog is focused mainly on Network Engineering, so why do we care? For this example, we will leverage NATS as our messaging system to communicate with our minions or runners or whatever other name you’d like to give them. We’ll do this through subjects like you saw earlier, but our subjects will be named after nodes and the action we would like performed.

The Topology

The topology below looks a bit wild at first glance, but honestly, it’s not as wild as some network diagrams I’ve seen in the past 😅. I’ll go over each portion piece by piece.

The Builder and the Watch Service

Everything in this lab is running on my 2019 Lenovo X1 Carbon laptop. You should be able to run this as well 😅. I think phones are more powerful at this point. The builder (my computer) is running AVD; this isn’t required as we’re only using AVD to generate our final device configurations. You can use whatever methodology or framework you prefer. AVD will generate device configurations in the deployment/intended/configs directory. Each file will be called something like leaf1.cfg.

The watch service is fairly short-winded for this example. We leverage the watchfiles Python package and point it to a directory to look for file changes. Once we see a change, we massage some data to get our device name and read the contents. When this is done, we leverage the NATS Python package to publish a message to the specific subject.

import asyncio
import os

import aiofiles
import nats
from watchfiles import awatch


async def watch_directory(nc, path):
    async for changes in awatch(path):
        for _, file_path in changes:
            async with aiofiles.open(file_path, mode="r") as f:
                contents = await f.read()
                device_name = os.path.splitext(os.path.basename(file_path))
                await nc.publish(f"dc1.{device_name[0]}.configure", contents.encode())


async def main():
    nc = await nats.connect("nats://localhost:4222")
    await watch_directory(nc, "deployment/intended/configs")
    await asyncio.Event().wait()


if __name__ == "__main__":
    asyncio.run(main())

The NATS Server

The NATS Server will handle all the communication between the publishers and subscribers. There’s a lot we could do with the configuration, but again, I kept it as simple as possible for MVP. The code block below is all that is required to get a NATS server running with Containerlab. If you have NATS Server installed, all you need to get a server running is the following command: nats-server 😊.

topology:
  nodes:
    nats:
      kind: linux
      image: nats:latest
      ports:
        - 4222:4222

The NATS Runners or Minions

The NATS runners are where we get more complex. The runners are more containers. Just like the Server is running as a container, so will the runners. The runners themselves contain a simple Python application that is subscribed to specific subjects. The block below is our Dockerfile definition for building the runners.

FROM python:3.12-alpine

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY config-replace.py .

CMD ["python", "config-replace.py"]

Again, trying to keep things as contained as possible. We start with a reasonably small Python base image. We then copy the requirements for our application. Once copied, we immediately install the requirements.

aiofiles
napalm
nats-py
watchfiles

We then copy over our config-replace.py file. This file holds the subscriber code and configuration replacement task for our runners. We also specify the command to run once the container is started.

import asyncio
import os

import nats
from napalm import get_network_driver

# Load DEVICE environment variable to build device construct
DEVICE = os.environ["DEVICE"]

device = {
    "hostname": DEVICE,
    "username": "admin",
    "password": "admin",
}


async def send_config_replace(data):
    driver = get_network_driver("eos")
    conn = driver(**device)
    try:
        print(f"Connecting to {device['hostname']}...")
        conn.open()  # Open the connection to the device
        print("Connection established.")

        # Load and preview the replacement configuration
        print("Loading configuration...")
        conn.load_replace_candidate(config=data)
        print("Configuration changes preview:")
        print(conn.compare_config())

        # Commit the changes
        if conn.compare_config():
            print("Applying configuration...")
            conn.commit_config()
            print("Configuration replaced successfully.")
        else:
            print("No changes detected. Skipping commit.")

    except Exception as e:
        print(f"An error occurred: {e}")
        # Rollback in case of any failure
        print("Rolling back configuration...")
        conn.rollback()
    finally:
        # Close the connection
        conn.close()
        print(f"Disconnected from {device['hostname']}.")


async def main():
    nc = await nats.connect("nats://nats:4222")

    async def message_handler(msg):
        subject = msg.subject
        data = msg.data.decode()
        print(f"Received message on subject '{subject}':\n{data}")
        await send_config_replace(data)

    await nc.subscribe(f"dc1.{DEVICE}.configure", cb=message_handler)
    # Wait indefinitely until the program is interrupted
    await asyncio.Event().wait()


if __name__ == "__main__":
    asyncio.run(main())

That’s neat, but a lot of it is just some NAPALM code to perform a configuration replacement on our devices. There are two interesting points to mention. When we start these containers, we will pass along an environment variable to tell the runner which device it’s effectively assigned to. We use that to build the device dictionary.

DEVICE = os.environ["DEVICE"]

device = {
    "hostname": DEVICE,
    "username": "admin",
    "password": "admin",
}

Then, in our main function, we initially connect to our NATS Server on nats:4222. We also build a message handler to instruct the application to do something once a message is received on a particular subject. In our case, we will call the send_config_replace function while also passing along the configuration data.

async def main():
    nc = await nats.connect("nats://nats:4222")

    async def message_handler(msg):
        subject = msg.subject
        data = msg.data.decode()
        print(f"Received message on subject '{subject}':\n{data}")
        await send_config_replace(data)

    await nc.subscribe(f"dc1.{DEVICE}.configure", cb=message_handler)
    # Wait indefinitely until the program is interrupted
    await asyncio.Event().wait()

Once we have all of these pieces defined, we can build the local container image by running the following:

docker build -t nats-config:latest .

The code block below is how we define the runners in our topology file and pass along their respective node assignments.

topology:
  nodes:
...
    runner1:
      kind: linux
      image: nats-config:latest
      env:
        DEVICE: spine1
    runner2:
      kind: linux
      image: nats-config:latest
      env:
        DEVICE: leaf1
    runner3:
      kind: linux
      image: nats-config:latest
      env:
        DEVICE: leaf2

Watch it Live

One thing I did not capture in the clips, you need to be running the watch application while testing this out. You can do so by running python watch.py on the builder. The clips below are woefully small, and I apologize in advance. The first clip shows a few things from left to right:

AVD data models have been updated, and we are rebuilding the configuration file artifacts
Localhost is running nats sub "dc1.*.configure" so we can view the messages (giant wall of configuration)
Viewing watch show ip name-server from leaf1
Viewing watch show ip name-server from leaf2
Viewing watch show ip name-server from spine1

In the second clip, we get a view from within one of the runners—in this case, the runner assigned to leaf1. Once we run a new build, we can see the print statements from the config-replace script.

Outro and Links

With the correct control mechanisms, I could see something like this leveraged in areas where a lot of change occurs frequently. Another area where I could see this is if you are starting to run into the scale limitations of running entire jobs from one server or control node. NATS deployments can include multiple servers in a dynamic mesh. I could see an army of minions assigned to the cluster, all listening for configuration jobs.

Another area where this could be interesting is sending messages to minions to perform things like health checks or validations—something for down the road. Another point I want to emphasize is that the code you see here is nowhere near production level. The config replace portion could use a lot of love and someone who really knows how to write async stuff. Thank you all for reading this far. I hope you enjoyed it and got something out of it.

Codespaces for Network Engineers and Educators

Mon, 25 Nov 2024 15:08:58 -0700

Introduction

Can you believe it’s been almost a year since my last blog? It’s incredible how fast time flies. Thank you all for continuing to read the blog and sharing it with engineers in the community. I can’t thank you enough for all the support and encouragement I’ve received on this journey.

This blog is going to take a bit of a turn. I usually write in the form of a tool I have used or been presented with and think, “This is incredible; more people should know about it.” My other perspective is, “Can I explain this thing in a somewhat more consumable way.” This stems from the idea that to really challenge your understanding, teach.

Teaching leads us to the idea behind this blog. I was recently at AutoCon 2 by Network Automation Forum (great event) and I met an interesting individual from New Mexico. I won’t say their name publicly, but let’s call them Bob for this blog. Bob is an instructor at a college and teaches future engineers about all things networking. Their students go through Introductory courses (think CCNA level). However, Bob has an interesting challenge: they want their students to continue learning advanced networking concepts. Bob has a tight budget (if any) and would like to avoid managing a fleet of resources to manage students' lab environments.

You all know Containerlab was at the forefront of my mind. Containerlab and a concept a colleague of mine has been working on to build reproducible labs with GitHub Codespaces got my brain spinning on a possible solution for Bob and their students.

The Idea Behind Codespaces

Some of you may be wondering, “What a Codespace?” Think of a Codespace as an environment running on GitHub’s infrastructure. Another way to look at it is like a portable development environment that can be accessed by a web browser. In its most basic form, all you need to access a Codespace is a GitHub account (free), computer, and internet access. Please see the official documentation for more information. The image below was inspired by the official documentation. In this example, our Codespace would be the box highlighted in red.

Codespaces for Network Engineers and Educators

I started pondering what a Network Engineer and Educator would need in a Codespace. We would definitely need network nodes in the environment. Reading about concepts is excellent, but seeing them come to life is even more powerful. We should also make the environment easy to use for anyone at any stage of their journey. Because styling is life, it must be pretty. Lastly, we need to be able to present the information to students in something other than a README file (even though I love a good README).

Building a Codespace

Below is an example of our Codespaces (link at the bottom of the blog) project. I’ll go over most of the files to give you an idea of how to build an environment like this.

> tree
.
├── .devcontainer
│   ├── devcontainer.json
│   └── Dockerfile
│   └── postCreate.sh
├── ne101
│   └── 01-routing-ospf
│       └── lesson.md
├── startup
│   ├── r1.cfg
│   └── r2.cfg
└── topo.yml

.devcontainer/devcontainer.json

The .devcontainer directory will host most of our workflow for this environment. For starters, we have a devcontainer.json file. If your project didn’t require a lot of extra tooling (maybe a Python project), the file could look something like the following.

// For format details, see https://aka.ms/devcontainer.json. For config options, see the
{
  "name": "Python 3",
  "image": "mcr.microsoft.com/devcontainers/python:0-3.11-bullseye",
}

The example above is using a pre-built container image hosted by Microsoft. In this case, the project is leveraging Python 3.11. The example below provides a more in-depth and realistic view of a .devcontainer.json file.

{
    "build": { "dockerfile": "Dockerfile" },
    "hostRequirements": {
        "cpus": 4,
        "memory": "16gb",
        "storage": "32gb"
    },
    "containerEnv": {
        "ARISTA_TOKEN": "${localEnv:ARTOKEN}",
        "CEOS_LAB_VERSION": "4.32.2.1F"
    },
    "features": {
        "ghcr.io/devcontainers/features/github-cli:1": {},
        "ghcr.io/devcontainers/features/docker-in-docker:2": {}
    },
    "secrets": {
        "ARTOKEN": {
            "description": "token to auto-download EOS images from arista.com."
        }
    },
    "customizations": {
        "vscode": {
            "extensions": [
                "catppuccin.catppuccin-vsc",
                "catppuccin.catppuccin-vsc-icons",
                "fabiospampinato.vscode-terminals",
                "marp-team.marp-vscode"
            ]
        }
    },
    "postCreateCommand": "postCreate.sh",
    "postStartCommand": "sudo clab deploy -t topo.yml --reconfigure"
}

build and a Dockerfile

The first portion of the JSON file calls out a build step. We are not using a pre-built container image to load our environment. We will load a Dockerfile locally for our project to spin up the environment and install the required tooling. We could, in theory, build our container image outside of this workflow and call it with the image key you saw earlier. Below is a quick view of our Dockerfile.

FROM mcr.microsoft.com/devcontainers/base:ubuntu

ARG CEOS_LAB_VERSION_ARG

ENV CEOS_LAB_VERSION=${CEOS_LAB_VERSION_ARG}

# install some basic tools inside the container
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
    sshpass \
    iputils-ping \
    htop \
    python3 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/* \
    && rm -Rf /usr/share/doc && rm -Rf /usr/share/man \
    && apt-get clean

# copy postCreate script
COPY ./postCreate.sh /bin/postCreate.sh
RUN chmod +x /bin/postCreate.sh

USER vscode

# install the latest containerlab and eos-downloader
RUN bash -c "$(curl -sL https://get.containerlab.dev)" \
    && pip3 install --user "eos-downloader>=0.10.1"

In summary, this file is performing the following:

Start FROM the base Ubuntu image hosted by Microsoft
We use the ARG key to pass data into the build environment (in our case this comes from the .devcontainer.json file)
We then set an ENV variable that will be available within the container (notice that we pass along the ARG we set earlier)
We then RUN a set of commands to install basic tooling like ping and Python3.
We then COPY a file called postCreate.sh (more on this later) and make it executable
Towards the end, we have one more RUN command; this ensures we have Containerlab and eos-downloader installed in our environment

hostRequirements

When starting a Codespace, you can size it to one of the options given to us by GitHub. The hostRequirements key allows us to set the minimum values required to run this lab. Suppose you don’t have the option to use a larger Codespace. In that case, you may need to contact GitHub to give you access to larger images (still free but uses more free credits per month).

    "hostRequirements": {
        "cpus": 4,
        "memory": "16gb",
        "storage": "32gb"
    },

containerEnv and secrets

I’ll combine these two sections. contanierEnv sets environment variables to which our container will have access. One is used as a token to download our Arista cEOS images. If you would like to leverage the Codespace in this example, please create an account with a business email on arista.com (free). The other variable will be used in a future workflow to specify which image we would like to download.

It’s been brought to my attention that Arista customers who register will get a token to download images. Apologies for the confusion.

    "containerEnv": {
        "ARISTA_TOKEN": "${localEnv:ARTOKEN}",
        "CEOS_LAB_VERSION": "4.32.2.1F"
    },
    "secrets": {
        "ARTOKEN": {
            "description": "token to auto-download EOS images from arista.com."
        }
    },

The ARTOKEN secret will be available to our container in future workflows. The screenshot below shows an example before setting any value for ARTOKEN.

features

features is another incredible feature of Codespaces and devcontainers. We can tell our environment what extra features we want to install. Think of these as pre-made scripts that handle the installation for us. We add the GitHub CLI feature to simplify our future workflows. Maybe the students aren’t Git wizards and need a more straightforward interface to interact with Git. We also install the docker-in-docker feature to allow us to work with Containerlab. Features are incredibly powerful; if you want to see a list of features, please see the official documentation.

    "features": {
        "ghcr.io/devcontainers/features/github-cli:1": {},
        "ghcr.io/devcontainers/features/docker-in-docker:2": {}
    },

customizations.vscode.extensions

These extensions allow us to extend the functionality or look of VS Code. I am adding my current favorite theme extension: catppuccin. If you were a University or Company, you could create a custom extension with your own styling. I also load a terminals extension that will give our students the ability to quickly open terminals to the different nodes in the environment. I also install the Marp extension, but more on that later.

    "customizations": {
        "vscode": {
            "extensions": [
                "catppuccin.catppuccin-vsc",
                "catppuccin.catppuccin-vsc-icons",
                "fabiospampinato.vscode-terminals",
                "marp-team.marp-vscode"
            ]
        }
    },

.vscode/settings.json

We’re taking a small break to look at our settings.json file. Again, this can be very deep and complicated, but for this example, we kept it as simple as possible.

{
    "workbench.colorTheme": "Catppuccin Frappé",
    "workbench.iconTheme": "catppuccin-frappe",
    "markdown.marp.themes": [
        "https://raw.githubusercontent.com/mastern2k3/marpit-nord-theme/master/build/nord-theme.css"
    ]
}

Here, we set the theme of our VS Code instance and icon theme. We also set our Marp theme (again, explained in a moment). The final version has a few more Marp themes, but that was more for exploration and testing.

.vscode/terminals.json

Another small break, the code block below shows the configuration of our Terminals extension. We set the extension to not autorun and then defined the terminals as a list of dictionaries. We can then provide basic information like the name, icon, color, and command to run on the new terminal.

{
    "autorun": false,
    "autokill": true,
    "terminals": [
      {
        "name": "r1",
        "description": "r1 view",
        "icon": "code",
        "color": "terminal.ansiBlue",
        "command": "ssh student@r1",
        "recycle": false,
        "open": true,
        "focus": true
      },
      {
        "name": "r2",
        "description": "r2 view",
        "icon": "code",
        "color": "terminal.ansiRed",
        "command": "ssh student@r2",
        "recycle": false,
        "open": true,
        "focus": false
      }
    ]
}

The GIF below shows an example of running 'cmd/ctrl+alt+t' from VS Code to connect to any pre-defined terminal or running Terminals:run to connect to all defined terminals.

postCreateCommand

The following portion will run after the container has finished building. Nothing too fancy; we are using eos-downloader to get a specific version of cEOS and have it run the Docker import step. We then immediately tag the image with ceos:latest.

    "postCreateCommand": "postCreate.sh",

#!/usr/bin/env bash

set +e

ardl get eos --image-type cEOS --version ${CEOS_LAB_VERSION} --import-docker
docker tag arista/ceos:${CEOS_LAB_VERSION} ceos:latest

postStartCommand

We made it to the final portion of the Codespace build, how we deployed the lab. This is no surprise, as we are running the Containerlab deploy command to ensure our cEOS nodes are running. We run it with the --reconfigure flag to ensure we have no issues when the Codespace shuts down and turns back on.

The only thing of note here is that we created startup configurations to set management network stuff in a separate VRF and a student user that did not require passwords to connect. Those can be seen in the startup directory.

    "postStartCommand": "sudo clab deploy -t topo.yml --reconfigure"

You can see some of the Containerlab output in the GIF above.

Slides with Marp

We are almost there. We now have an environment to learn about networking concepts or even network automation. An instructor needs a tool to present this information. Again, I love a good README, but giving that to a student and telling them “good luck” is going to have a wide range of results.

In comes Marp, Which tags itself as a Markdown Presentation Ecosystem. That sounds fancy, but at its core, it allows us to write presentations in Markdown format. I wrote about a distant relative, slidev, in an earlier blog. For this example, I felt slidev was a bit heavy, and all that I needed to get Marp going was a VS Code extension.

Below is an example of the slides when rendered within the Codespace. To view this presentation, we just hit “preview” on the markdown file like we would any file in VS Code. You can find this presentation in the ne101/01-routing-ospf/lesson.md file. If you started the Codespace, feel free to follow along with the lesson.

Outro and Links

Thank you all for reading this far; it really means a lot. I hope you learned a little bit about Codespaces and that it inspires you to create something beautiful in the future. If you have any feedback or improvements, please reach out. All the best!

Network CI and Open Source

Sat, 25 Nov 2023 19:33:45 -0800

Introduction

Wow, it’s been a long time since my last blog post. I recently attended the first AutoCon event focusing on network automation. Long story short, it was terrific. Check it out and be on the lookout for the next one. Shortly after AutoCon, two community members sprung up with sweet blog posts. My colleague and friend Dan Hertzberg are resurrecting the blog. The first post went over leveraging ygot to create Go structs. My now in-person friend Danny Wade also made a new post that recapped AutCon. Both are excellent reads and good people; check them out.

Shortly after the event, I wanted to run a simple continuous integration pipeline with Containerlab and some Arista cEOS nodes. Long story short, it worked! However, I had to add a twist to the pipeline by leveraging a GitHub self-hosted runner.

If you’ve read my blogs in the past, you may have run into my six-part blog series on building a CICD pipeline. Think of this post as a modification or add-on to that previous work. I’ll review more tools and provide examples of how they can help you on your network automation journey. I hope you enjoy it.

The why

If you got this far, then I’ve either hooked you with the introduction, or you’re just a fan of my work. Either way, thank you. The main reason for this post is to look at different open-source tools and see how they can be used in our workflows to improve or automate some of the tasks.

One important note! I’ll mention a lot of different tools in this blog, don’t feel like you have to use all of them. Feel free to explore and choose what works for you and your environment.

Requirements

If you want to follow along, you should have a local development machine and a Linux VM or laptop to run the self-hosted runner. In my case, I have it running on a small Linux VM on my Intel NUC (not sponsored). It would be helpful if your development machine has Docker installed, but the Linux VM will need it.

GitHub self-hosted runner

GitHub has robust CI tooling within its platform. We can leverage the tooling to create workflows to automate project tasks. These tasks could be auto-assigning team members to issues, testing code, building artifacts, or validating changes on the network.

These “actions” can run within GitHub’s infrastructure to spin up ephemeral environments. This is fantastic, but it could be better when we need this environment to access our containerized network images. The only vendor I am aware of that allows public pulling of their container image is Nokia with their SRLinux platform.

For my setup, I will be using Arista’s cEOS images. Instead of hosting the image in a public repository (which I don’t think is on the up-and-up), I leveraged the self-hosted runner. Since we manage the self-hosted runner, we can have whatever we want, including container images. More on this in a moment.

GitHub has incredible documentation on installing a self-hosted runner. I won’t duplicate that effort, and I’ll link you to their excellent docs. One important thing to note is that their documentation walks us through installing the application and running it within the terminal. We need it to run in the background as a service. To do this, make sure you follow GitHub’s docs on doing just that. Below is an example output from my VM running the application.

❯ sudo ./svc.sh status

/etc/systemd/system/actions.runner.JulioPDX-ci-avd-clab.dev.service
● actions.runner.JulioPDX-ci-avd-clab.dev.service - GitHub Actions Runner (JulioPDX-ci-avd-clab.dev)
     Loaded: loaded (/etc/systemd/system/actions.runner.JulioPDX-ci-avd-clab.dev.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-11-22 12:28:11 PST; 3 days ago
   Main PID: 1713 (runsvc.sh)
      Tasks: 23 (limit: 4915)
     Memory: 1.6G
        CPU: 44min 55.008s
     CGroup: /system.slice/actions.runner.JulioPDX-ci-avd-clab.dev.service
             ├─1713 /bin/bash /home/juliopdx/actions-runner/runsvc.sh
             ├─1716 ./externals/node16/bin/node ./bin/RunnerService.js
             └─1724 /home/juliopdx/actions-runner/bin/Runner.Listener run --sta…

Hint: Some lines were ellipsized, use -l to show in full.

Containerlab

Containerlab is my favorite platform for running labs. Whether I’m personally learning, testing, or incorporating it into a blog post like this one, the speed and simplicity Containerlab provides are unmatched. If you are following along, install it with this one-liner.

bash -c "$(curl -sL https://get.containerlab.dev)"

Arista cEOS images

In one of my earlier blog posts, I talked about how to obtain Arista cEOS images. Make a free account on the site, download whatever version you want, and import the image with Docker. Way before I was an Arista employee, I appreciated that the EOS images were free to obtain; more vendors should do this. A new tool is also available called eos-downloader; it is a command line interface to download Arista EOS images. You should check it out.

That is all you need on the Linux VM running the GitHub self-hosted runner application. At the end of this blog, I’ll post a few alternate ways to improve this workflow; feel free to use them as self-learning opportunities.

The basic workflow

In the most high-level view of our build, we have a local development machine where we can make proposed changes to our test network. The changes in “code” are then pushed up to GitHub, where automated workflows will run to lint our code, build the configurations, and test it against a short-lived network.

But it’s so much more

If only the world was that simple. A decent amount of tools are available to make this all happen. Feel free to fork or clone the repository and mess with whatever you want. Below is a view of the folder structure of the repository. Stick with me; I’ll do my best to break down every piece.

git clone https://github.com/JulioPDX/ci-avd-clab.git

> tree -I venv
.
├── ansible.cfg
├── anta
│   ├── anta-inv.yml
│   └── anta-test.yml
├── artifacts
│   ├── config_backup
│   ├── documentation
│   ├── intended
│   │   ├── configs
│   │   ├── structured_configs
│   │   └── test_catalogs
│   └── reports
│       └── anta_reports
├── group_vars
│   ├── all.yml
│   ├── ATD_FABRIC.yml
│   ├── ATD_LAB.yml
│   ├── ATD_SERVERS.yml
│   └── ATD_TENANTS_NETWORKS.yml
├── inventory.yml
├── lab.yml
├── LICENSE
├── playbooks
│   ├── build.yml
│   ├── deploy-net.yml
│   ├── validate-net-anta.yml
│   └── validate-net.yml
├── README.md
├── requirements-dev.txt
├── requirements.txt
├── requirements.yml
└── tasks.py

Preparing the local environment

The repository also includes a .devcontainer file as well. It is handy if you want to develop within a container environment. I’ll walk through setting up a local Python development environment. This assumes you have Python installed. To test more in-depth locally, ensure you have Docker, Containerlab, and a cEOS image available.

cd ci-avd-clab
python3 -m venv venv
source venv/bin/activate
pip install -r requirements-dev.txt
ansible-galaxy collection install -r requirements.yml

GitHub Actions

GitHub Actions is the CI/CD platform within GitHub. We can leverage GitHub Actions to create automated workflows within our repository. These workflows can be as simple as linting our codebase for any errors or deploying changes to a test network and notifying us of a pass or fail. Below is the entire look at the GitHub Actions file for this example. The file is located in the .github/workflows folder.

# .github/workflows/dev.yml
---
name: Network CI

on:
  pull_request:
    branches:
      - main
  push:
    branches:
      - main

jobs:
  run-linters:
    timeout-minutes: 15
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v4

      - name: setup python
        uses: actions/setup-python@v4

      - name: run pre-commit
        uses: pre-commit/action@v3.0.0

  dev-build-deploy-test:
    needs: run-linters
    env:
      ANTA_PASSWORD: ${{ secrets.ANTA_PASSWORD }}
      SUDO_PASS: ${{ secrets.SUDO_PASS }}
      CEOS_VERSION: 4.30.2F

    timeout-minutes: 15
    runs-on: avd-ci
    steps:
      - name: checkout
        uses: actions/checkout@v4

      - name: setup Python
        uses: actions/setup-python@v4
        with:
          python-version: "3.9"

      - name: install requirements
        run: pip install invoke && invoke setup-base

      - name: create topology with containerlab
        run: echo $SUDO_PASS | sudo -S containerlab deploy -t lab.yml --reconfigure

      - name: build with AVD
        run: ansible-playbook playbooks/build.yml

      - name: deploy configuration to nodes
        run: |
          ansible-playbook playbooks/deploy-net.yml
          sleep 10          

      - name: run ANTA cli tests
        run: |
          anta \
              --username admin \
              --password $ANTA_PASSWORD \
              --inventory anta/anta-inv.yml \
              --log-file artifacts/anta.log \
              nrfu --catalog anta/anta-test.yml text          

      - name: validate configuration on nodes
        run: ansible-playbook playbooks/validate-net.yml

      - name: validate configuration on nodes w/anta
        run: ansible-playbook playbooks/validate-net-anta.yml

      - name: destroy lab
        if: always()
        run: echo $SUDO_PASS | sudo -S containerlab destroy -t lab.yml -c

      - name: cleanup self hosted runner
        if: always()
        uses: TooMuch4U/actions-clean@v2.1

I’ll break this down piece by piece and show you most of the tools from my local development machine along the way.

# .github/workflows/dev.yml
---
name: Network CI

on:
  pull_request:
    branches:
      - main
  push:
    branches:
      - main
...

We define a name for the workflow at the root of the GitHub actions file. This doesn’t have any impact on the workflow itself, but it’s always helpful to pick a name with meaning. The on key is where things start to get interesting. Here, we can specify what events and branches we would like this workflow to run on.

Currently, we have it running on a pull request event or a push event, both heading towards the main branch. This definition is not something to use in a production environment. You would likely have a workflow file for your feature/fix branches and another for the main branch (the production network).

Jobs, Linters, and pre-commit

# .github/workflows/dev.yml
...
jobs:
  run-linters:
    timeout-minutes: 15
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v4

      - name: setup python
        uses: actions/setup-python@v4

      - name: run pre-commit
        uses: pre-commit/action@v3.0.0
...

jobs in GitHub actions define the individual steps we would like performed for a particular… job. In our example, we have one job that handles the linting of the code base and another that deals with building and testing a virtual network. Right below the job key is the unique name of run-linters, which is user-defined, but again, it should make sense. We then have a runs-on key. The runs-on key defines what platform we would like under the hood for this workflow. You could have a macOS, Windows, or Linux environment. This could also be on the GitHub infrastructure or self-hosted.

The run-linters job doesn’t need access to our self-hosted runner or the container images. It only needs access to the repository. To save on my personal resources, this job uses the ubuntu-latest variant of a GitHub runner (the node performing the steps), which runs on GitHub’s infrastructure. Another thing to note: the timeout-minutes will end the workflow if we exceed that time. This workflow should be far from that, so hitting that limit should signal that we have an issue.

The steps in actions are the units of work we would like performed. We jokingly called them a collection of bash scripts that do a thing. The first two steps are standard in Python projects; we use checkout to make the repository available to the runner and setup-python to ensure Python is in our PATH.

pre-commit

The last step in this job is to run pre-commit. Pre-commit allows us to automate processes before or during the commit process in our development workflow. For example, we could run linters to check for extra new lines in files, lint YAML, or maybe even Python code.

Pre-commit works by defining a .pre-commit.yaml file at the root of our project. Below is a view of the local one from the repository. Automating these processes enables not only yourself but contributors to your project. There is no question on what rules to follow or how Python code should look, “just run pre-commit.”

# .pre-commit-config.yaml
---
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: end-of-file-fixer
      - id: trailing-whitespace
  - repo: https://github.com/ansible/ansible-lint
    rev: v6.22.0
    hooks:
      - id: ansible-lint
  - repo: https://github.com/adrienverge/yamllint.git
    rev: v1.33.0
    hooks:
      - id: yamllint
  - repo: https://github.com/DavidAnson/markdownlint-cli2
    rev: v0.11.0
    hooks:
      - id: markdownlint-cli2
  - repo: https://github.com/astral-sh/ruff-pre-commit
    # Ruff version.
    rev: v0.1.6
    hooks:
      # Run the linter.
      - id: ruff
      # Run the formatter.
      - id: ruff-format

The pre-commit configuration for each tool is mostly the same; some have additional available arguments. In the configuration file, we set the repo location, the rev or version we want, and what hook from the repository. Many hooks can exist in a single repository. For more examples, check out the list of available pre-commit hooks (not all-encompassing).

There are two options when running pre-commit. You can add it to your git hooks(pre-commit install), and it will run anytime you try and commit, or you can run it manually (pre-commit run -a). Below is a quick example of pre-commit in action.

The following shows some user errors in files and pre-commit attempts to correct them.

> pre-commit run -a
check yaml...............................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing lab.yml
Fixing group_vars/ATD_TENANTS_NETWORKS.yml

trim trailing whitespace.................................................Passed
Ansible-lint.............................................................Passed
yamllint.................................................................Passed
markdownlint-cli2........................................................Passed
ruff.....................................................................Passed
ruff-format..............................................................Failed
- hook id: ruff-format
- files were modified by this hook

1 file reformatted

The following is an example of pre-commit passing all checks.

> pre-commit run -a
check yaml...............................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
Ansible-lint.............................................................Passed
yamllint.................................................................Passed
markdownlint-cli2........................................................Passed
ruff.....................................................................Passed
ruff-format..............................................................Passed

Feel free to explore the documentation of the various linters used and see if any others will benefit you in your workflows.

The second job

# .github/workflows/dev.yml
...
  dev-build-deploy-test:
    needs: run-linters
    env:
      ANTA_PASSWORD: ${{ secrets.ANTA_PASSWORD }}
      SUDO_PASS: ${{ secrets.SUDO_PASS }}
      CEOS_VERSION: 4.30.2F

    timeout-minutes: 15
    runs-on: avd-ci
...

Moving down the actions file, we now have our second job. Again, we have a name, but now you can see a new key called needs. This key tells this job to wait for the run-linters job to complete before it runs. We also specify three environment variables we would like available to the runner. The ANTA_PASSWORD reflects the password used to connect to EOS nodes in future steps. SUDO_PASS is used to pass along the credential when running Containerlab. These are within the GitHub actions' secrets; set them under the secrets view.

The last variable specifies the version of cEOS I’m using, which is then referenced in the Containerlab topology file. A preview of that is below. In my case, the image is tagged as ceos:4.30.2F.

...
topology:
  kinds:
    ceos:
      image: ceos:${CEOS_VERSION:=4.30.2F}
...

The end of the workflow file above mentions a new runs-on value. Installing the self-hosted application instructs us to set a tag or use the default. Below is a view of my runner with the avd-ci tag available. Your installation will be different.

Invoke and Containerlab

# .github/workflows/dev.yml
...
    steps:
      - name: checkout
        uses: actions/checkout@v4

      - name: setup Python
        uses: actions/setup-python@v4
        with:
          python-version: "3.9"

      - name: install requirements
        run: pip install invoke && invoke setup-base

      - name: create topology with containerlab
        run: echo $SUDO_PASS | sudo -S containerlab deploy -t lab.yml --reconfigure
...

Top-level portions are similar to our first job; the main difference is that we are specifying a specific Python version that should be in our PATH this time. Similar to my earlier comment that pre-commit hooks have additional arguments, action steps also have them.

The last two steps use the run variant. This simply means “run the following commands on the runner.” Invoke was new for me on this journey; I like to think of Invoke as a more Pythonic replacement for Makefiles. Invoke allows us to automate tasks as well by defining them in Python. Below is an example of the setup_base function, which leverages two other functions; install_python_reqs and install_ansible_collections. We can use this to ensure our self-hosted runner has all our requirements installed.

The last step listed shows the use of our SUDO_PASS environment variable; we need the variable to run Containerlab as sudo. Please let me know if you know of a better or easier way to make this work. The Containerlab command always runs with the --reconfigure flag. We basically always want the environment to be a fresh deployment. You could also build this to never shut down the test environment; that would remove the build times in the workflow.

# tasks.py
...
@task()
def setup_base(ctx):
    """
    Set up the development container environment.
    This task runs other tasks to install various requirements.
    """

    # Install Python project and development requirements
    install_python_reqs(ctx)

    # Install Ansible collection requirements
    install_ansible_collections(ctx)


@task()
def install_python_reqs(ctx):
    """
    Install project Python requirements.
    """
    print("Installing Project Python Requirements")
    ctx.run("pip install --upgrade pip")
    ctx.run(f"pip install -r {REQUIREMENTS_FILE_PYTHON_DEV}")


@task()
def install_ansible_collections(ctx):
    """
    Install Ansible collection requirements, including Arista AVD collection.
    """
    print("Installing Project Ansible Collection Requirements")
    ctx.run(
        f"ansible-galaxy collection install -r {REQUIREMENTS_FILE_ANSIBLE} --force-with-deps"
    )
...

A view of our lab file is below. It’s a single spine and two leaf deployment.

# lab.yml
---
name: ci-avd
prefix: ""

mgmt:
  network: statics
  ipv4-subnet: 172.100.100.0/24

topology:
  kinds:
    ceos:
      image: ceos:${CEOS_VERSION:=4.30.2F}
  nodes:
    spine1:
      kind: ceos
      mgmt-ipv4: 172.100.100.11
    leaf1:
      kind: ceos
      mgmt-ipv4: 172.100.100.13
    leaf2:
      kind: ceos
      mgmt-ipv4: 172.100.100.14
  links:
    - endpoints: ["spine1:eth1", "leaf1:eth1"]
    - endpoints: ["spine1:eth2", "leaf2:eth1"]

If you are following along or would like to test locally, feel free to spin up the lab

sudo clab deploy -t lab.yml

Arista Validated Designs

# .github/workflows/dev.yml
...
      - name: build with AVD
        run: ansible-playbook playbooks/build.yml

      - name: deploy configuration to nodes
        run: |
          ansible-playbook playbooks/deploy-net.yml
          sleep 10          
...

AVD is an interesting framework. It allows us to define our network using abstracted data models. For example, if I want to add a new VLAN or VRF, I would define that in a common YAML file related to the nodes in my topology. I may want to define common services like NTP, DNS, etc. I would do that from a common or specific (depending on the use case) YAML file. For example, run the following Ansible playbook to see what AVD accomplishes.

ansible-playbook playbooks/build.yml

The definitions for this deployment are defined under the group_vars directory.

# playbooks/build.yml
---
- name: Build EOS config
  hosts: ATD_FABRIC
  connection: local
  gather_facts: false

  tasks:
    - name: Generate intended variables
      ansible.builtin.import_role:
        name: arista.avd.eos_designs

    - name: Generate device intended config and documentation
      ansible.builtin.import_role:
        name: arista.avd.eos_cli_config_gen

Please note every CLI output in this blog is in real-time.

AVD leverages roles within Ansible to abstract away a lot of the complexity. The neat thing is that we have complete running and structured configurations (in YAML) per device and site documentation.

├── artifacts
│   ├── config_backup
│   ├── documentation
│   │   ├── ATD_FABRIC
│   │   │   ├── ATD_FABRIC-documentation.md
│   │   │   ├── ATD_FABRIC-p2p-links.csv
│   │   │   └── ATD_FABRIC-topology.csv
│   │   └── devices
│   │       ├── leaf1.md
│   │       ├── leaf2.md
│   │       └── spine1.md
│   ├── intended
│   │   ├── configs
│   │   │   ├── leaf1.cfg
│   │   │   ├── leaf2.cfg
│   │   │   └── spine1.cfg
│   │   ├── structured_configs
│   │   │   ├── leaf1.yml
│   │   │   ├── leaf2.yml
│   │   │   └── spine1.yml

Feel free to review the generated artifacts or modify the ATD_TENANTS_NETOWRKS.yml file with an additional SVI and rerun the build playbook.

The bonus example

When AVD was first created, it was available as an Ansible collection; now we can leverage the Python implementation of AVD using pyavd. A reasonably rough Python script is in the repository if you want to test it. Please note pyavd is in beta, so your mileage may vary.

You may need to install ansible instead of only installing ansible-core.

pip install ansible rich pyavd

# avd-py.py
import copy
import difflib
import io
import sys
from ansible.inventory.manager import InventoryManager
from ansible.parsing.dataloader import DataLoader
from ansible.vars.manager import VariableManager
from rich.progress import Progress
from pyavd import (
    get_avd_facts,
    get_device_structured_config,
    get_device_config,
    validate_inputs,
)
from rich import print


# Recreating the inventory objects to be like  Ansible
dl = DataLoader()
im = InventoryManager(loader=dl, sources=["inventory.yml"])
vm = VariableManager(loader=dl, inventory=im)

hosts = {}

# grab all the hosts and add the hostname with relevant
# variables to the hosts dictionary
# deep copy required to make sure information is not overwritten
for host in im.get_hosts("all"):
    host_vars = vm.get_vars(host=host)
    hosts[str(host)] = copy.deepcopy(host_vars)

with Progress() as progress:
    # progress bars for pretty output
    total = len(hosts)
    task1 = progress.add_task("[red]facts...", total=1)
    task2 = progress.add_task("[blue]validate inputs...", total=total)
    task3 = progress.add_task("[yellow]structured config...", total=total)
    task4 = progress.add_task("[green]build config...", total=total)

    facts = get_avd_facts(hosts)
    progress.update(task1, advance=1)

    # loop to check the state of validate status and print
    # any errors
    for k, v in hosts.items():
        validate = validate_inputs(v)

        if validate.failed:
            print(f"The following validation errors were seen for {k}")
            for issue in validate.validation_errors:
                print(issue)
            sys.exit(1)
        progress.update(task2, advance=1)

        struct_conf = get_device_structured_config(k, v, facts)
        progress.update(task3, advance=1)

        config = get_device_config(struct_conf)
        buf = io.StringIO(config)
        print(k)
        try:
            with open(f"artifacts/intended/configs/{k}.cfg") as file:
                old = file.readlines()

            for line in difflib.unified_diff(old, buf.readlines(), lineterm=""):
                print(line)

            with open(f"artifacts/intended/configs/{k}.cfg", "w") as f:
                f.writelines(config)

        except FileNotFoundError:
            with open(f"artifacts/intended/configs/{k}.cfg", "w") as f:
                f.writelines(config)

        progress.update(task4, advance=1)

The last step deals with running a configuration replacement on our nodes from the generated configurations.

# playbooks/deploy-net.yml
---
- name: Build EOS config
  hosts: ATD_FABRIC
  connection: local
  gather_facts: false

  tasks:
    - name: Generate intended variables
      ansible.builtin.import_role:
        name: arista.avd.eos_config_deploy_eapi

I don’t have a Python configuration replacement example but feel free to experiment and create one. You could always use something like NAPALM, Netmiko, or Scrapli.

AVD validate state & Arista Network Test Automation (ANTA)

...
      - name: run ANTA cli tests
        run: |
          anta \
              --username admin \
              --password $ANTA_PASSWORD \
              --inventory anta/anta-inv.yml \
              --log-file artifacts/anta.log \
              nrfu --catalog anta/anta-test.yml text          

      - name: validate configuration on nodes
        run: ansible-playbook playbooks/validate-net.yml

      - name: validate configuration on nodes w/anta
        run: ansible-playbook playbooks/validate-net-anta.yml
...

AVD has a role to validate the state of our deployed network. Below is an example of the playbook and the output. The validate state role will also produce reports on passing or failing tests.

ansible-playbook playbooks/validate-net.yml

artifacts/reports
├── anta_reports
├── ATD_FABRIC-state.csv
└── ATD_FABRIC-state.md

ANTA is the new kid on the block for network testing. There are a few ways to implement ANTA. We can use it directly in Python, leverage the CLI tool, or in a preview release within AVD Ansible. The ANTA AVD variant makes things fairly automated as far as test definitions. We basically set the use_anta variable to true. Below is the view of the playbook and the output.

---
- name: Validate deployed config
  hosts: ATD_FABRIC
  connection: httpapi
  gather_facts: false

  vars:
    use_anta: true

  tasks:
    - name: Validate EOS state with ANTA
      ansible.builtin.import_role:
        name: arista.avd.eos_validate_state

Below is the output from the ANTA validate playbook.

Below is a quick example of one of the generated artifacts.

ANTA also has a CLI, which requires two main parts: an inventory definition and a test catalog.

Here’s a view of our inventory.

# anta/anta-inv.yml
---
anta_inventory:
  hosts:
    - host: 172.100.100.11
      name: spine1
      tags: ["fabric", "spine"]
    - host: 172.100.100.13
      name: leaf1
      tags: ["fabric", "leaf", "pod1"]
    - host: 172.100.100.14
      name: leaf2
      tags: ["fabric", "leaf", "pod2"]

Here’s a look at the test catalog. These are YAML representations of the Python classes in the code. Head to the documentation to see what’s available. I’ll show an example of a pass and a fail.

---
anta.tests.software:
  # Verifies the device is running one of the allowed EOS version.
  - VerifyEOSVersion:
      # List of allowed EOS versions.
      versions:
        - 4.30.2F-33092737.4302F (engineering build)

anta.tests.routing.generic:
  - VerifyRoutingProtocolModel:
      model: "multi-agent"

anta.tests.routing.bgp:
  - VerifyBGPPeersHealth:
      address_families:
        - afi: ipv4
          safi: unicast

anta.tests.vxlan:
  - VerifyVxlan1Interface:
  - VerifyVxlanConfigSanity:

export ANTA_PASSWORD=somesecretpassword

anta \
--username admin \
--password $ANTA_PASSWORD \
--inventory anta/anta-inv.yml \
--log-file artifacts/anta.log \
nrfu --catalog anta/anta-test.yml table

I’ll slightly edit by modifying the Routing Protocol Model.

Cleaning the self-hosted runner

# .github/workflows/dev.yml
...
      - name: destroy lab
        if: always()
        run: echo $SUDO_PASS | sudo -S containerlab destroy -t lab.yml -c

      - name: cleanup self hosted runner
        if: always()
        uses: TooMuch4U/actions-clean@v2.1

The GitHub runners are usually ephemeral; we don’t have to worry about cleanup. The self-hosted runner is different since the environment can last indefinitely. We use these last steps to clean up files we don’t need and destroy our topology. Please note the new key of if; we want these steps to run no matter what happens in the pipeline run.

Run the Pipeline

Wow, that was a lot of information. Please let me know if you have any questions or feel something needs clarification. I’ll add a new service to the network by editing the group_vars/ATD_TENANTS_NETWORKS.yml file and then pushing it up to the repository, which should kick off a pipeline run.

# ATD_TENANTS_NETWORKS.yml
---
tenants:
  # Tenant A Specific Information - VRFs / VLANs
  - name: Tenant_A
    mac_vrf_vni_base: 10000
    vrfs:
      - name: Tenant_A_OP_Zone
        vrf_vni: 10
        vtep_diagnostic:
          loopback: 100
          loopback_ip_range: 10.255.1.0/24
        svis:
          - id: 110
            name: Tenant_A_OP_Zone_1
            tags: [opzone]
            enabled: true
            ip_address_virtual: 10.1.10.1/24
          - id: 111
            name: Tenant_A_OP_Zone_2
            tags: [opzone]
            enabled: true
            ip_address_virtual: 10.1.11.1/24
    l2vlans:
      - id: 160
        vni_override: 55160
        name: Tenant_A_VMOTION
        tags: [vmotion]

git add .
git commit -m "something"
git push

Outro and links

Thank you all so much for reading this far. I hope something in here is helpful in your automation journey. Please explore and plug tools into different areas to see if you can make it work. Massive shout-out to Ryan Merolle; he deserves much credit for the work on this repository. Thank you so much for contributing.

Building my first mechanical keyboard

Sat, 29 Apr 2023 11:14:03 -0700

Introduction

I recently went down the rabbit hole that is mechanical keyboards and, even worse, attempted to build my own. This blog will show you some random dude’s journey to making his first mechanical keyboard and avoiding some of my mistakes. I hope you enjoy the blog, and thank you for visiting the site.

Why build vs. buy

Why would you build a keyboard? I mean, you could buy one, right? That is correct, and there’s nothing wrong with buying a keyboard or mechanical keyboard pre-assembled. In fact, two other keyboards I own came pre-assembled. One is the Keyboardio Atreus (still use this for travel) and the Iris keyboard I purchased from Keebio. Both are excellent, and I still use them today.

Even though I purchased two previous sets, I wondered if I could build one myself. I compare it to the same journey of putting a puzzle together, a Lego set, or a Gundam kit. Making something and seeing it come together brings a vast amount of joy.

Why Ortholinear

If you clicked on the previous links for the two keyboards mentioned above, you might wonder, “Why do those keyboards look so weird?” Fair question. These keyboards use an ortholinear layout where all the keys are the same size and have a uniform grid vs. a staggered layout found in traditional keyboards. The thought being this replicates the natural movement of our fingers. However, from my reading, there isn’t any evidence that this improves the typing experience or leads to fewer wrist injuries.

Why split

If you click on the Iris keyboard link, you may notice that it’s split into two pieces. The primary benefit here, since we have two unique pieces, we can rotate them to align with the natural resting position of our wrists vs. angling our wrists inward to align with a traditional staggered layout. The video below goes over all of this and more in excellent detail.

Picking the right keyboard

I was now set on a split keyboard with an ortholinear layout. This is where it gets fun or tricky. There are many options for mechanical keyboards, and any portion of the build can be customized to fit your needs. The layout, materials, key switches, keycaps, and colors for everything. It can get overwhelming very fast.

I had some general points to follow to make the selection process manageable. This is in no particular order.

Wood case or at least wood-filled filament option
Tactile switches
Split layout
Compact

I don’t remember how I found my final choice, but I’m happy I did. Have you ever found a thing that makes you say, “This fits me.” When I saw the Minidox for the first time, I knew that was the keyboard I wanted to build. The Minidox is a more compact version of the popular ErgoDox. The PCB for the Minidox was created by That-Canadian, and I can’t thank them enough for making this free and open source for the community.

Assembling the required tools

I’ll provide links to the items I used but feel free to source what works for you. You may have some items that I didn’t have when I started. I’ll include some optional pieces as well.

Soldering iron kit (Highly recommend)
PCB and electronics from Profet Keyboards
Any switch you like, I went with Glorious Pandas pre-lubed
Any MX style switches you prefer
Case STL files, printed with treatstock
M3 screws for the case
TRRS cable

The parts above are what I would consider required. Since I was going down this route, I wanted to see if I could improve the sound of the keyboard as much as possible. The items below are optional.

Replacement USB-C Pro Micro for the main side (left). The right side continues to use the micro USB that comes from Profet Keyboards.
Keyboard PE switch pads for sound dampening.
Neoprene roll for more sound dampening and to fill in the case.
Something to practice soldering.

Practice, practice, and more practice

The soldering practice item above is optional, but it helped me tremendously. I last soldered eight years ago, and for many, this may be your first time looking at or handling a soldering iron. If I was going to do this, I didn’t want the keyboard PCB to be what I practice on. If you like to live dangerously, then go right ahead.

Getting started

I leveraged the build guide by That-Canadian to complete this board. With everything in hand, I started soldering all the main components; diodes, TRRS jacks, and controllers. The image below is one of the few I captured in this build. Take note of how small those diodes are. If you have trouble focusing on small items, I recommend getting a desk magnifying glass or headset unit. The build guide mentions that the diodes can go on the top or the bottom of the PCB. I recommend you install them on the bottom. In case something doesn’t work right or you need to replace a non-working diode.

Flashing QMK firmware

Now that all of the diodes are in. This is a great time to flash the controllers before mounting them on the PCB. This build uses QMK to build and flash the firmware on the controllers. There’s a GUI interface for folks on Windows or Mac. I use a Linux machine as my daily driver and went with the command-line route.

Set a Python virtual environment to install dependencies.

python3 -m venv venv
source venv/bin/activate
pip3 install qmk
qmk setup

I leveraged all the default options during the setup stage. Although you may need to install additional dependencies on Linux, QMK should warn you if there’s a missing dependency.

You can test whether QMK is ready by running the qmk doctor command. You can also attempt to compile a keymap.

qmk compile -kb maple_computing/minidox/rev1 -km default

This is all great, but here’s something that took me a little while to figure out. By default, QMK will try to flash a controller compatible with caterina, at least for the Minidox. Please read the documentation of your controller to see what bootloader is available. The USB C Pro Micro I linked earlier runs the DFU bootloader. This required one change in the keyboard’s rules.mk file.

That sounds great but I had no idea where that file lived. Again, I’m just a new user of all this tooling and all the documentation seems to assume you know where this lives. If you followed the default install steps for QMK, this will clone down the QMK repository to your main directory.

cat qmk_firmware/keyboards/maple_computing/minidox/rules.mk
# Build Options
#   change yes to no to disable
#
BOOTMAGIC_ENABLE = no       # Enable Bootmagic Lite
MOUSEKEY_ENABLE = yes       # Mouse keys
EXTRAKEY_ENABLE = yes       # Audio control and System control
CONSOLE_ENABLE = no         # Console for debug
COMMAND_ENABLE = yes        # Commands for debug and configuration
NKRO_ENABLE = no            # Enable N-Key Rollover
BACKLIGHT_ENABLE = no       # Enable keyboard backlight functionality
RGBLIGHT_ENABLE = no        # Enable keyboard RGB underglow
AUDIO_ENABLE = no           # Audio output

SPLIT_KEYBOARD = yes

DEFAULT_FOLDER = maple_computing/minidox/rev1

The required setting for this controller is to add BOOTLOADER = atmel-dfu to the rules.mk file.

Now if we follow the required steps, we should see a new option for the bootloader to try and flash.

Compile your keymap of choice
run qmk flash -kb maple_computing/minidox/rev1 -km default
Plug in the controller USB to your PC
Either hit the reset button if installed or use tweezers on the reset and ground pins.

qmk flash -kb maple_computing/minidox/rev1 -km default
Ψ Compiling keymap with gmake --jobs=1 maple_computing/minidox/rev1:default:flash


QMK Firmware 0.20.5
Making maple_computing/minidox/rev1 with keymap default and target flash

avr-gcc (GCC) 5.4.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Size before:
   text    data     bss     dec     hex filename
      0   19794       0   19794    4d52 maple_computing_minidox_rev1_default.hex

Compiling: quantum/command.c                                                                        [OK]
Linking: .build/maple_computing_minidox_rev1_default.elf                                            [OK]
Creating load file for flashing: .build/maple_computing_minidox_rev1_default.hex                    [OK]
Copying maple_computing_minidox_rev1_default.hex to qmk_firmware folder                             [OK]
Checking file size of maple_computing_minidox_rev1_default.hex                                      [OK]
 * The firmware size is fine - 19794/28672 (69%, 8878 bytes free)
Flashing for bootloader: atmel-dfu
Bootloader not found. Make sure the board is in bootloader mode. See https://docs.qmk.fm/#/newbs_flashing
 Trying again every 0.5s (Ctrl+C to cancel)...

Ultimately, I settled on a keymap variation of the one posted by khitsule.

The color coding references what the keys will do depending on whether we hold the lower or raise key down.

Why isn’t this working

At this point, everything is flashed and ready to go. I even have some switches installed on the case and the controller. I have enough connected to connect the two boards and see if all the keys work. The dream is that everything works on the first try, and I won’t have to desolder anything. I was so wrong. Whenever I connected the two boards with the TRRS jacks, both boards would shut off. Long story short, after a lot of troubleshooting and desoldering the right side PCB not once but TWICE!!! The error could have been avoided if I just read the manual.

Suppose you decide to build your own mechanical keyboard. In that case, I am sending you all the good vibes that you never have to remove the Pro Micro, at least not for troubleshooting reasons. Isn’t this the ugliest thing you’ve ever seen?

Quick note on cases

My father-in-law was kind enough to spare his time and material to print the case for my first build. That is very awesome of him. The case was linked earlier, but shout out to designers willing to share your awesome designs with the community.

Minor spoiler, but in case you don’t have access to a 3D printer, you can use Treat Stock to order 3D prints. I don’t have any affiliate links or sponsors. I used the service and think it’s a neat option. Treat Stock gives you access to a vast list of printers. Think of it like Etsy for 3D prints. This allows you to select whatever color and materials are available.

Read the FU***** manual

Back to my reading the manual thought. After reading the build guide and soldering 30 diodes, I got too confident. So much so that I looked at the build guide but didn’t read the not-so-fine print but rather the large print. The right-side controller has to be installed upside down because these PCBs are reversible. I can only blame myself. Just look at this screengrab from the actual build guide.

When I was desoldering the Pro Micro not once but TWICE, I seriously thought about quitting on this project. At this point it was getting spendy and time consuming. A veteran of building keyboards could probably put this together in under an hour. I was easily over two weekends on this. I pushed through, desoldered the Pro Micro, and soldered everything back into place.

The sweet feeling of success

Once everything was back in place, it was time to test again. Although the step back at the time seemed insurmountable, it was now just a funny memory of the past. Seeing it all work is a beautiful thing.

Would I do it again

Oddly enough, I had so much fun with the build that I completed the second one in a day or so, this time with no Pro Micro issues!

I used the Keyboard PE switch pads only on the second build. I cut out three neoprene foam layers to fill the case and improve the sound. If anyone is curious about a sound test, please let me know. I can always update the blog with a small audio clip.

Pro Micro is installed correctly on the right side :). I used Azure dragon V2 switches on the second build.

The case for the second build was ordered through Treat Stock in bronze PLA.

Outro

Thank you all for reading this far. It really means a lot. Stepping into the world of mechanical keyboards has been fun. I’ll make at least one more Minidox in the future, and after that, who knows. So let me know what you think, and stay safe out there.

Featured image by Kirk Lai
List of split keyboards

RPKI in the Lab

Mon, 02 Jan 2023 14:18:17 -0800

Introduction

I recently finished the last chapters of Optimal Routing Design (great read). One of the chapters covered routing protocol security with protocols such as BGP, EIGRP, and OSPF. This echoed something in my head regarding Resource Public Key Infrastructure (RPKI). I knew the book’s printing was around 2003 and figured RPKI wouldn’t be in the book. This led me to wonder if I could get RPKI running in the lab, and the following blog post details some of that journey.

I knew very little about RPKI when I started down this road (arguably still know very little). However, I knew it was essential to set up to protect networks from the fate of route hijacks (someone advertising your network space without permission by accident or on purpose).

By default, peering over BGP isn’t secure. The workflow is usually Autonomous systems (AS) peering together and start advertising networks. There’s a default trust put in place that each peer will advertise the correct routes. Over time, we’ve developed filtering or route policies to ensure we receive and advertise the proper routes. However, time has shown that these options alone aren’t enough to prevent route hijacking or leaks.

RPKI overview

RPKI provides a way to connect an AS number to IP addresses in its simplest form. The beauty is that this pairing is associated with something we trust. For example, this could be one of the regional internet registries (RIR).

graph TD subgraph RIR AFRINIC ARIN APNIC LACNIC RIPENCC end IANA --> RIR AFRINIC --> LIR ARIN --> LIR APNIC --> LIR LACNIC --> LIR RIPENCC --> LIR LIR --> Customers

The local internet registries (LIR) can create Route Origination Authorizations (ROA). The ROAs is the piece that states what AS can advertise a specific prefix. Once we have valid ROAs, we can determine if a route is valid, invalid, or unknown. You may wonder, “okay, but where do these ROAs get stored?” The ROAs are actually stored in multiple repositories. For example, the RIRs are the trusted source for RPKI information, including ROAs.

If I want to validate IP space given out by APNIC, then I’ll subscribe to their repository and trust that the information is accurate and up-to-date. There is so much more involved in RPKI, but I’d like to keep a blog short for once. I’ll include some helpful links at the end if you want to learn more.

Getting RPKI in the lab with Routinator

In this lab, we’ll leverage a local Validator of routes. This Validator can pull the latest ROA information from the defined repositories. The Validator we will use is NLnetLabs Routinator. The Validator will use RPKI to Router (RTR) protocol to signal if a route is valid, invalid, or unknown. This has the benefit of reducing control plane impact on our routers. All the validation and checks are done by the Validator.

graph TD R1(R1) ---|172.100.100.11|MGMT R2(R2) --- |172.100.100.12|MGMT RT(Routinator) --- |172.100.100.13|MGMT

Containerlab to simplify the deployment

NLnetLabs is very kind to include a container image of their Routinator software. Adding this to Containerlab is relatively easy. The example below is our lab file.

name: rpki
prefix: ""

mgmt:
  network: statics
  ipv4_subnet: 172.100.100.0/24

topology:
  kinds:
    ceos:
      image: ceos:4.29.0.2F
    linux:
      image: nlnetlabs/routinator
  nodes:
    R1:
      kind: ceos
      mgmt_ipv4: 172.100.100.11
      startup-config: startup/R1.conf
    R2:
      kind: ceos
      mgmt_ipv4: 172.100.100.12
      startup-config: startup/R2.conf
    RPKI:
      kind: linux
      mgmt_ipv4: 172.100.100.13
      ports:
      - 80:8323
  links:
    - endpoints: ["R1:eth1", "R2:eth1"]

I kept the topology file as simple and small as possible. We will use two Arista EOS containers to form a BGP relationship and advertise routes. We will then implement RPKI to check the validity of those advertisements. The lab’s startup configurations and GitHub repository will be linked at the end of this post.

Simple BGP config and advertising routes

We will simulate Google and Cloudflare as our two peering AS in our lab. We will advertise a network from each AS and another random network used for testing.

flowchart LR subgraph AS 15169 R1(R1) end subgraph AS 13335 R2(R2) end R1 -->|192.168.100.1/32| R2 R1 -->|8.8.8.0/24| R2 R2 -->|192.168.100.2/32| R1 R2 -->|1.1.1.0/24| R1 linkStyle 0 stroke:green linkStyle 1 stroke:green linkStyle 2 stroke:green linkStyle 3 stroke:green

Below is a simple BGP configuration from R1’s perspective.

router bgp 15169
   router-id 192.168.100.1
   neighbor 172.16.0.2 remote-as 13335
   network 8.8.8.0/24
   network 192.168.100.1/32

R1#show ip bgp | b Net
          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      1.1.1.0/24             172.16.0.2            0       -          100     0       13335 i
 * >      8.8.8.0/24             -                     -       -          -       0       i
 * >      192.168.100.1/32       -                     -       -          -       0       i
 * >      192.168.100.2/32       172.16.0.2            0       -          100     0       13335 i
R1#

Next, we’ll add Routinator to our BGP configuration.

Connecting to Routinator

Setting up our environment with Routinator is relatively simple. Since all nodes in this topology share a management network, we’ll use that in our RPKI configuration. Below is the configuration from R1’s perspective.

router bgp 15169
   router-id 192.168.100.1
   neighbor 172.16.0.2 remote-as 13335
   network 8.8.8.0/24
   network 192.168.100.1/32
   !
   rpki cache routinator
      host 172.100.100.13 port 3323
      !
      transport tcp
   !
   rpki origin-validation
      ebgp local

Once this is implemented, we can check if our router is communicating with our Validator.

R1#show bgp rpki cache
routinator:
Host: 172.100.100.13 port 3323
VRF: default
Refresh interval: 326 seconds
Retry interval: 600 seconds
Expire interval: 7200 seconds
Preference: 5
Protocol version: 1
State: synced
Session ID: 23300
Serial number: 18
Last update sync: never
Last full sync: 0:00:10 ago
Last serial query: never
Last reset query: 0:00:41 ago
Entries: 390484
Connection: Active (0:00:41)
Transport information:
  Protocol: TCP
ROA tables: default


R1#

If we check our routes now, we will see two new codes.

V: Valid RPKI routes
U: Unknown RPKI routes

R1#show ip bgp | b Net
          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >    V 1.1.1.0/24             172.16.0.2            0       -          100     0       13335 i
 * >      8.8.8.0/24             -                     -       -          -       0       i
 * >      192.168.100.1/32       -                     -       -          -       0       i
 * >    U 192.168.100.2/32       172.16.0.2            0       -          100     0       13335 i
R1#

Putting policy and filtering in place

At this point, we can see that our router is correctly communicating with Routinator and labeling routes as valid or unknown. However, you may notice that the routes are still in our routing table, even if they are unknown or invalid. In this case, we are simulating the 192.168.100.0 routes as bad. We can correct this by implementing some simple filtering.

flowchart LR subgraph AS 135533 R1(R1) end subgraph AS 135534 R2(R2) end R1 --x|192.168.100.1/32| R2 R1 -->|8.8.8.0/24| R2 R2 --x|192.168.100.2/32| R1 R2 -->|1.1.1.0/24| R1 linkStyle 0 stroke:red linkStyle 1 stroke:green linkStyle 2 stroke:red linkStyle 3 stroke:green

!
ip as-path access-list 1 permit ^$ any
!
route-map local-only permit 10
   description Only export local routes
   match as-path 1
!
route-map rpki-reject-non-valid deny 10
   description Reject non valid routes
   match origin-as validity invalid
!
route-map rpki-reject-non-valid deny 20
   description Reject not found routes
   match origin-as validity not-found
!
route-map rpki-reject-non-valid permit 100
!
router bgp 15169
   router-id 192.168.100.1
   neighbor 172.16.0.2 remote-as 13335
   neighbor 172.16.0.2 route-map rpki-reject-non-valid in
   neighbor 172.16.0.2 route-map local-only out

Below we can see the updates after filtering and compare that with all routes received from our neighbor.

R1#show ip bgp | b Net
          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >    V 1.1.1.0/24             172.16.0.2            0       -          100     0       13335 i
 * >      8.8.8.0/24             -                     -       -          -       0       i
 * >      192.168.100.1/32       -                     -       -          -       0       i
R1#

R1#show bgp neighbors 172.16.0.2 received-routes | b AS Path
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >    V 1.1.1.0/24             172.16.0.2            -       -          -       -       13335 i
        U 192.168.100.2/32       172.16.0.2            -       -          -       -       13335 i
R1#

If you’d like, you can also check out the Routinator GUI at http://. Feel free to check out the number of RTR connections or the source for all the repositories.

Outro and links

That’s about it. I hope you learned a little about RPKI and got to play with it in the lab. Please check out the GitHub repository and modify it for other AS and network combos you may want to learn with. Happy new year, and I wish you all the best. Thank you for reading this far. It really means a lot!

Static website with Pulumi, YAML, MkDocs, GCP, and CI/CD

Sun, 06 Nov 2022 14:20:59 -0800

Introduction

A few weeks ago, a co-worker asked the team if there was a way to convert a set of Markdown files produced from the AVD Ansible Collection to nice HTML. I then created a simple GitHub Repo that explored various ways to build HTML from Markdown. That repository is combined with the example we’ll walk through in this blog.

I then went down the rabbit hole of getting a static site deployed in the various Clouds. If you look through the old commits, you can see the site is also deployed in Azure using a static web app. By the time you read this, that link may or may not be active. Although, it’s using the free tier, I’ll probably leave it up.

I’ve used GCP to take advantage of the private container registry to push up some images. I wanted to challenge myself with deploying the static site in GCP (and learn a bit about GCP along the way). So I decided to make it harder (or easier?) on myself and leverage Pulumi to deploy the infrastructure..

What is Pulumi

I’ve written about Pulumi in the past on Packet Pushers (read it!), so I won’t go into deep detail about it. But, in summary, it allows us to deploy infrastructure to the cloud using various programming languages (TypeScript, Python, Go, YAML). So, for example, if the team is very comfortable working with Go or Python, we can maintain those skills and use them to provision our cloud infrastructure. If you want to follow along, install Pulumi locally and create an account for their backend service.

curl -fsSL https://get.pulumi.com | sh
pulumi login

Pulumi & YAML

Did you see YAML in that list earlier? What the heck!? Pulumi recently (wow, I just checked, and it was back in May) announced the support for YAML as another option to deploy infrastructure. Once again, hating myself, I decided to use YAML as the language of choice for this example (I’ve used Python in the past).

Static sites

Wow, that was a long intro, and it’s still going. The original question was, “we have a bunch of Markdown, and we want this converted to HTML.” I’ve been using MkDocs heavily in the last few months, so that’s what I’ll use in this example. Please note there are a lot of static site generators (SSG) out there. For example, this blog is created with Hugo, but others such as Jekyll, Gatsby, etc. will work just fine.

MkDocs

MkDocs is so easy to use that it’s really grown on me these last few months. If you want to follow along, either clone the repository linked at the end or start fresh. We can begin by creating a Python virtual environment and installing a few items.

pre-commit
mkdocs
mkdocs-material
mkdocs-include-dir-to-nav

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Pre-commit will be explained later on. The other installs will leverage MkDocs, the fantastic Material theme, and a simple MkDocs plugin for navigation (optional). Great, now that we have that installed, we can review our mkdocs.yaml file. This is used to configure the various settings for our static site.

site_name: Fabric Docs

# Configuration
use_directory_urls: false
site_url: ""
edit_uri: ""
docs_dir: avd-project
theme:
  name: material
  palette:
    - media: "(prefers-color-scheme: light)"
      scheme: default
      toggle:
        icon: material/weather-sunny
        name: Switch to dark mode
    - media: "(prefers-color-scheme: dark)"
      scheme: slate
      toggle:
        icon: material/weather-night
        name: Switch to light mode
plugins:
  - include_dir_to_nav:
  - search
extra_css:
  - stylesheets/extra.css
extra_javascript:
  - https://unpkg.com/tablesort@5.3.0/dist/tablesort.min.js
  - stylesheets/tables.js
nav:
  - Home: README.md
  - documentation
  - reports

docs_dir: Points to a directory where our documents are stored and will be referenced from (default is docs).
theme.name: This tells MkDocs that we will be leveraging the material theme.
plugins: Any additional plugins we would like enabled. If you did not install mkdocs-include-dir-to-nav that can be disabled. I am leveraging to point mkdocs to folders within docs_dir to build the navigation tree. Instead of specifying them one by one.
extra_css: Not required, adds different styling from the default.
extra_javascript: Again, not required, used here to sort generated tables.
nav: This is used to build all the navigation for our site.

Excellent, clear as mud. We can “serve” the documents to get a live view as we modify them.

juliopdx in static-hosting-example on main [✘!?] via dev via 🐍 took 6s
❯ mkdocs serve
INFO     -  Building documentation...
INFO     -  Cleaning site directory
INFO     -  Documentation built in 0.84 seconds
INFO     -  [00:00:50] Watching paths for changes: 'avd-project', 'mkdocs.yml'
INFO     -  [00:00:50] Serving on http://127.0.0.1:8000/

If we open that in the browser, we will see something like the following.

Great, we have a simple static site ready for deployment in the cloud. If you run mkdocs build, mkdocs will package your static site in a new folder called site.

juliopdx in static-hosting-example on main [✘!?] via dev via 🐍 took 12m32s
❯ mkdocs build
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /home/juliopdx/repos/static-hosting-example/site
INFO     -  Documentation built in 0.67 seconds

juliopdx in static-hosting-example on main [✘!?] via dev via 🐍
❯

Deploying the static site to GCP

On the first build of this site, I used the official documentation. That was a great exercise and showed how all the various resources are connected. But this example will show the steps to deploy this with Pulumi and YAML. If you are following along, you will need to install the Google Cloud SDK and authorize with a user account.

Pulumi

If you are using this repository, you can edit a few values and run pulumi up. Create a directory and start a new Pulumi project if you are creating your own project.

mkdir quickstart && cd quickstart && pulumi new gcp-yaml

Please note, you will need to create a GCP project and enable the Google Compute API in your project to create resources.

Bucket and rule for the public

This example will leverage a storage bucket to store our site files.

name: static-hosting-example
description: Static hosting on GCP
runtime: yaml
resources:
  # Create storage bucket
  web-bucket:
    type: gcp:storage:Bucket
    properties:
      website:
        mainPageSuffix: index.html
        notFoundPage: 404.html
      location: US
      uniformBucketLevelAccess: true
      forceDestroy: true

  # Enable public access to view site
  web-bucket-binding:
    type: gcp:storage:BucketIAMBinding
    properties:
      bucket: ${web-bucket.name}
      role: "roles/storage.objectViewer"
      members: ["allUsers"]

At the root, we have basic information like name, description, and runtime (YAML). We then list out the individual resources as dictionaries. For example, the web-bucket is of type gcp:storage:Bucket. I thought this was interesting syntax. In a programming language, you may see this as gcp.storage.Bucket. It essentially maps to what function will be executed to create the particular resource.

We then fill in the properties key with our unique configuration. We could also add a name key to hardcode what name we want to give a resource. I left this to Pulumi to name from the top-level key. For example, web-bucket will actually be named something crazy like web-bucket12345445.

We specify two essential files for our storage bucket, one for our main page and another when something is not found. I added the forceDestroy property so that I could destroy the bucket while testing. I believe if something is in the bucket and forceDestroy is not set, it will not delete the bucket.

Apologies that this section is so long, but it’s important to understand how Pulumi works. The second resource is just setting an access rule to make our files viewable to the public. But notice, to pass in what bucket this rule should be associated with, we use ${web-bucket.name}. This is an available parameter from our storage bucket resource. You can run pulumi up -y to create the resources.

Load balancer and SSL certs

The goal of building this was to make it secure. Users get weird when they browse a site with the famous “Your connection is not private, invalid SSL cert” error. The note below is directly from the guide.

Cloud Storage doesn’t support custom domains with HTTPS on its own, so you also need to set up an SSL certificate attached to an HTTPS load balancer to serve your website through HTTPS.

Okay, that sounds super easy…

Load balancer and SSL certificate

We’ll need a static IP for future assignment to our global forwarding rules (also used for our DNS records).

name: static-hosting-example
description: Static hosting on GCP
runtime: yaml
resources:
# ...
  # Global address
  webglobaladdress:
    type: gcp:compute:GlobalAddress

I don’t want to manage SSL certs, so I’ll just let GCP handle that. I then list any domains I plan to leverage with this certificate.

# ...
  # GCP managed cert with list of domains
  managedcert:
    type: gcp:compute:ManagedSslCertificate
    properties:
      managed:
        domains:
        - juliopdx.org
        - www.juliopdx.org

The backend service is interesting. First, we enable CDN (Content Delivery Network) since we want users to get quick and close delivery of our content. Although I’m guessing GCP gets you with the pricing here, I would plan ahead. I’m currently leveraging the free account on GCP for this deployment, which will be short-lived. We then associate our web-bucket where the site will be stored.

  # backend service leveraging our storage bucket
  sitebackend:
    type: gcp:compute:BackendBucket
    properties:
      description: Contains a sweet site
      bucketName: ${web-bucket.name}
      enableCdn: true
      cdnPolicy:
        cacheMode: CACHE_ALL_STATIC
        clientTtl: 300
        defaultTtl: 300
        maxTtl: 1800

From my research, it seems like URLmaps can get really complex. URLMaps route requests to backend services (sitebackend seen above). We don’t have any defined, so it will just use the default. The default will route all the things to our sitebackend.

  # Basic URL map pointing to backend
  webservicepaths:
    type: gcp:compute:URLMap
    properties:
      defaultService: ${sitebackend.id}

The HTTPS proxy ties everything together. At this point, we associate our URLMap and newly managed SSL certificate. Our global forwarding rule can use the proxy to route incoming HTTPS requests to our webservicepaths URLMap.

  # HTTP proxy leveraging managed cert
  httpsproxy:
    type: gcp:compute:TargetHttpsProxy
    properties:
      urlMap: ${webservicepaths.selfLink}
      sslCertificates:
      - ${managedcert.id}

We tie the httpsproxy and webglobaladdres with our global forwarding rule. The loadBalancingScheme: EXTERNAL_MANAGED makes this a fancy load balancer vs. the classic one. You will probably get more features with the new one.

  # Forwarding rule pointing to HTTPS proxy
  defaultglobalforwardingrule:
    type: gcp:compute:GlobalForwardingRule
    properties:
      target: ${httpsproxy.id}
      portRange: 443
      loadBalancingScheme: EXTERNAL_MANAGED
      ipAddress: ${webglobaladdress.id}

You can run pulumi up -y to create all the resources. Please make sure to update the domain names for the certificate with your information. Below is an example from the GCP console and the Load Balancer we just created.

HTTP to HTTPS redirect

We’re almost done with the deployment. One other piece is directing users to HTTPS when they try to hit our site on port 80 or HTTP. The build-out is similar to our initial load balancer without the certificate or SSL policy.

  # HTTP to HTTPS redirect
  redirecturlmap:
    type: gcp:compute:URLMap
    properties:
      defaultUrlRedirect:
        httpsRedirect: true
        stripQuery: false

  redirecttargethttpproxy:
    type: gcp:compute:TargetHttpProxy
    properties:
      urlMap: ${redirecturlmap.id}

  redirectglobalforwardingrule:
    type: gcp:compute:GlobalForwardingRule
    properties:
      loadBalancingScheme: EXTERNAL_MANAGED
      target: ${redirecttargethttpproxy.id}
      portRange: 80
      ipAddress: ${webglobaladdress.id}

This diagram does a great job of summing up this workflow and our initial load balancer.

Note: Pulumi also has this neat feature called outputs. We can save the output and use it later on in other programs or within the command line. Example in a moment.

Connecting domain to load balancer

Now that everything is created, we can update our DNS records to point to our public address. We can either use the console or the command line. The load balancer image shown earlier also reveals the public address.

juliopdx in static-hosting-example on main [!⇡] via dev via 🐍 took 2s
❯ gcloud config set project pulumi-web
Updated property [core/project].

juliopdx in static-hosting-example on main [!⇡] via dev via 🐍
❯ gcloud compute addresses list
NAME                      ADDRESS/RANGE  TYPE      PURPOSE  NETWORK  REGION  SUBNET  STATUS
webglobaladdress-e33548f  34.98.78.165   EXTERNAL                                    IN_USE

I wanted to use www.juliopdx.org and juliopdx.org. I created two “A” records that matched my setup.

Uploading files using gsutil

I couldn’t figure out how to send a folder using the Pulumi YAML provider. Please send me a message if you know how to do that. After some research, I found this great post by Mickael Vieira, where they use Hugo and GitHub actions to deploy a static site. This led me to gsutil. It’s a Python application that lets you interact with GCP storage from the command line. If you already ran mkdocs build, you should have a site folder in your directory. The following command can be used to send the files from a particular directory.

gsutil -m rsync -d -c -r site gs://

From Mickael’s blog:

m to perform parallel synchronization;
r to synchronize the directories and subdirectories recursively;
c to use files’ checksum instead of their modification time to avoid sending all files during each deployment;
d to delete the files that are no longer necessary.

We can use the console to find out what the name of our bucket is, or we can use a Pulumi Output. For example, the end of our Pulumi.yaml file has this line.

outputs:
    # Export the DNS name of the bucket
    bucketName: ${web-bucket.url}

We can leverage this in our gsutil command.

gsutil -m rsync -d -c -r site $(pulumi stack output bucketName)

juliopdx in static-hosting-example on main [!⇡] via dev via 🐍
❯ gsutil -m rsync -d -c -r site $(pulumi stack output bucketName)
Building synchronization state...
Starting synchronization...
Copying file://site/sitemap.xml [Content-Type=application/xml]...
Copying file://site/sitemap.xml.gz [Content-Type=application/xml]...
- [2/2 files][  1.0 KiB/  1.0 KiB] 100% Done
Operation completed over 2 objects/1.0 KiB.

After some time for the certificate to be created and DNS to propagate, you should see something like this. It’s beautiful.

Thats great and all, but lets talk CI/CD with documentation

So far, we have built up the infrastructure and the site content with local development, but we can incorporate CI/CD into our documentation. Contributors would still create content locally, but that content would then get pushed up to our Git repository to be checked before going out to production. In the following sections, I’ll explain some fantastic tools and examples of moving some of these steps to GitHub actions.

pre-commit

Pre-commit is pretty awesome. You can supply a series of hooks that can then be used to lint your various programming languages, fix common file issues (trailing whitespace), or in our case, fix our documentation written in Markdown. If you followed along, you should have pre-commit installed.

pip install pre-commit

Pre-commit relies on a .pre-commit-config.yaml file at the root of your project to know what linters or hooks to run.

# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.4.0
    hooks:
      - id: trailing-whitespace
        exclude: ansible_collections/arista/avd/molecule
      - id: end-of-file-fixer
        exclude_types: [svg, json]

You can extend this as much as you want or add more hooks related to your project. To get started, run the following command.

juliopdx in static-hosting-example on main [!⇡] via  dev via 🐍
❯ pre-commit install --install-hooks
pre-commit installed at .git/hooks/pre-commit
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...

We can then check our code base for any issues.

❯ pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed

Great, everything passed. What about with an error? I removed the extra new line in Pulumi.yaml, which we want.

❯ pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing Pulumi.yaml

Pre-commit is much more powerful than I’m showing here, but it’s good for demonstration.

Markdownlint

We can also add Markdownlint to our pre-commit workflow.

# ...
  - repo: https://github.com/igorshubovych/markdownlint-cli
    rev: v0.32.1
    hooks:
      - id: markdownlint
        name: Check for Linting errors on MarkDown files
        args:
          - --config=.github/.markdownlint.yaml
          - --ignore-path=.github/.markdownlintignore
          - --fix

I will only go into some detail since this is already so long. First, I created two files in the .github directory. One that points to the rules we want to enable or disable. Check out the repo if you would like to leverage them. I then ignore specific directories for files that may be created by automation. This is similar to a .gitignore file but for Markdown files. We can then run pre-commit again.

I’ll add an error by adding another H1 heading in the README file.

# Static site example

Simple workflows to view and build static site stuff.

# Test heading

## Getting started

❯ pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check for Linting errors on MarkDown files...............................Failed
- hook id: markdownlint
- exit code: 1

README.md:5 MD025/single-title/single-h1 Multiple top-level headings in the same document [Context: "# Test heading"]
Usage: markdownlint [options]

In the case of Markdownlint, it will try to fix some errors, but it cant assume everything. The heading is not automatically removed for us, but some mistakes will be corrected. I’ll remove the duplicate H1 heading and run pre-commit again.

❯ pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check for Linting errors on MarkDown files...............................Passed

For more inforamtion, check out markdownlint and markdown-cli.

Vale on the command line

Vale is a command line tool that can be used to check prose (your writing style). It does this based on a series of rules. The user can create these rules, or you can leverage pre-made styles. For example, Googles developer style guide (errata-ai version of the style guide). To use Vale, you must install the binary.

I’ve found the easiest way to get Vale running is through Homebrew.

brew install vale

Like most things, we need to provide Vale a configuration. This is done by Vale reading the .vale.ini file at the root of our project.

StylesPath = .github/styles
MinAlertLevel = error # suggestion, warning or error

# Only Markdown; change to whatever you're using.
[*.{md}]
# List of styles to load.
BasedOnStyles = Google

I have the Google style in my repository in that StylesPath. I then chose the alert level of error. Feel free to play with the alert levels and the Google style rules that you agree or don’t agree with. My good friend, Daniel Teycheney, has an excellent blog post on using Vale. Check it out for more details.

I’ll run Vale against one file with no errors.

❯ vale README.md
✔ 0 errors, 0 warnings and 0 suggestions in 1 file.

Now I’ll introduce one error: Google hates exclamation marks in headings. Maybe hate is a strong word.

# Static site example!

Simple workflows to view and build static site stuff.

❯ vale README.md

 README.md
 1:21  error  Don't use exclamation points    Google.Exclamation
              in text.

✖ 1 error, 0 warnings and 0 suggestions in 1 file.

I think that’s slick. With tools like these, you can ensure a consistent style and quality of your blog, documentation, or whatever else you’re building. But the fun doesn’t stop there. We can take all the tools mentioned and move them to GitHub actions.

GitHub actions

I have the following files in my .github/workflows/ directory.

❯ tree .github/workflows
.github/workflows
├── pr-manage.yml
└── static-web-app.yml

0 directories, 2 files

pr-manage.yml will be used for any pull requests. This workflow can run pre-commit, Vale, and Pulumi preview.

---

name: "Doc testing"

on:
  # For a pull_request event, only branches and tags on the base are evaluated.
  push:
    branches-ignore:
      - main   # excludes main

jobs:
  pre_commit:
    name: Run pre-commit validation hooks
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-python@v3
    - uses: pre-commit/action@v3.0.0

  vale:
    name: runner / vale
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: errata-ai/vale-action@reviewdog
        env:
          # Required, set by GitHub actions automatically:
          # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret
          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
        with:
          files: avd-project/
          reporter: github-check

  pulumi-gcp-preview:
    name: runner / pulumi
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      ###### GCP connect ######
      - name: gcp auth
        uses: 'google-github-actions/auth@v0'
        with:
          credentials_json: '${{ secrets.GCP_CREDENTIALS }}'
      - name: Set up Cloud SDK
        uses: 'google-github-actions/setup-gcloud@v0'
      - name: pulumi preview
        uses: pulumi/actions@v3
        with:
          command: preview
          stack-name: dev
        env:
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

A lot is going on, but two secrets must be set to run this workflow. First, you need to create a service account user in GCP and add a new key in the form of JSON. This can then be added to repository secrets. You will also need to create a token in the Pulumi backend. This is a great guide.

I’ll create a branch and push this up with some description added to one of our resources.

Looking closely, you can see we have an update for the description on the ManagedSslCertificate resource.

Push on main

Now that it is working, I’ll run a pull request and merge these changes to our main branch. Then, instead of a preview, Pulumi will perform the changes.

name: GCP & Pulumi static web app

on:
  push:
    branches:
      - main


jobs:
  build_and_deploy_job:
    runs-on: ubuntu-latest
    name: Build and Deploy Job
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true

      - uses: actions/setup-python@v2
        with:
          python-version: 3.x

      - run: pip install -r requirements.txt
      - run: mkdocs build

      ###### GCP push portion ######
      - name: auth
        uses: 'google-github-actions/auth@v0'
        with:
          credentials_json: '${{ secrets.GCP_CREDENTIALS }}'

      - name: Set up Cloud SDK
        uses: 'google-github-actions/setup-gcloud@v0'

      - name: Use gcloud CLI
        run: 'gcloud info'

      - name: deploy infrastructure pulumi
        uses: pulumi/actions@v3
        with:
          command: up
          stack-name: dev
        env:
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

      - name: Push updated files to bucket
        run: gsutil -m rsync -d -c -r site $(pulumi stack output bucketName)

We’ll run through the MkDocs steps to build our site. We then login to GCP using our secrets and run pulumi up. Once the infrastructure is set, we run gsutil to update the files if required. It seems that at the time of this writing, the set-ouput is deprecated, so I can’t show the difference in the GitHub actions workflow. Check out the job run and the issue. We can see the updates on the console.

  # GCP managed cert with list of domains
  managedcert:
    type: gcp:compute:ManagedSslCertificate
    properties:
      description: Some sweet cert
      managed:
        domains:
        - juliopdx.org
        - www.juliopdx.org

Final thoughts and links

Wow, that was a lot of information. I hope you enjoyed the read and hopefully learned something along the way. Thank you for reading this far. As always, it really means a lot. I provided links where I may not have been as clear. Please check them out; they are great resources. Like anything I write, what I show here is just a subset of what these tools can do. Best of luck and happy labbing and learning.

Featured image by Kristopher Roller
Google guide for hosting a static site
Pulumi
MkDocs
MkDocs Material theme
pre-commit
Vale
Markdownlint
Blogs for inspiration
- Joel Rozen - Create a Cloud Run service and https load balancer with Pulumi
- Michael Vieira - Deploying a Static Website to Google Cloud Storage With Github Actions
GitHub repository

Slidev

Tue, 01 Nov 2022 21:38:29 -0700

Introduction

A while back, I was building a technical presentation. Like most presentations, this would include diagrams, code blocks, images, etc. So I did what any of us usually did and looked at something like Google Slides or Microsoft PowerPoint.

For the longest time, I wasn’t a fan of how code blocks looked in these presentations or the gymnastics performed when taking a screenshot of some highlighted code or opening another app to create an image that would then be added to the presentation. After some searches on the internet, I was blessed with something incredible, Slidev.

What is Slidev

Slidev is maintained by Anthony Fu and aims to be “Presentation Slides for Developers.” I’m guessing Anthony was tired of running through certain hoops and figured, “Why not just make some presentation software that mimics what I do daily?”

I’ll walk through some examples and my favorite features in a sec. Slidev makes the presentation-building workflow essentially like presentations as code. When building slides, developers and engineers can take advantage of using Markdown, CSS, and HTML.

Getting started

Getting started with Slidev is as simple as one command (requires Node.js >= 14.0).

npm init slidev@latest

You will then be prompted to select a project name, if you would like to install it now, and what agent to use for the install. Example below.

❯ npm init slidev@latest
npx: installed 22 in 1.688s

  ●■▲
  Slidev Creator  v0.36.10

✔ Project name: … example
  Scaffolding project in example ...
  Done.

✔ Install and start it now? … yes
✔ Choose the agent › npm

The slides should open in your web browser automatically. You can run it manually using npm run dev if they don’t. The slides will then be viewable at localhost:3030. By default, the slides can be modified at slides.md. In this case, I’ll delete the built-in ones made by default and run through some examples.

Favorite features

Slidev has so many features that breaking them all down would take a while. So instead, I’ll run through some simple examples, but I highly encourage you to look at the official documentation (linked at the end).

Creating in Markdown

The ability to create slides in Markdown format was huge for me. Learning the syntax for creating documentation (and slides) is nice and straightforward. Below is an example of the initial slide.

---

# Welcome to the world of tomorrow

These slides are sweet!

---

Themes

The default themes in Slidev are pretty sweet, but you can also use custom themes. I’m a fan of the Purplin theme, mainly for its support of a bottom bar. I can change the front matter at the top of the slides.md file and run npm run dev.

---
# try also 'default' to start simple
theme: purplin
# random image from a curated Unsplash collection by Anthony
# like them? see https://unsplash.com/collections/94734566/slidev
background: https://source.unsplash.com/collection/94734566/1920x1080
# apply any windi css classes to the current slide
class: 'text-center'
# https://sli.dev/custom/highlighters.html
highlighter: prism
# show line numbers in code blocks
lineNumbers: false
# some information about the slides, markdown enabled
info: |
  ## Slidev Starter Template
  Presentation slides for developers.

  Learn more at [Sli.dev](https://sli.dev)  
# persist drawings in exports and build
drawings:
  persist: false
# use UnoCSS
css: unocss
---

❯ npm run dev

> example@ dev /home/juliopdx/Documents/slidev-example/example
> slidev --open

? The theme "slidev-theme-purplin" was not found in your project, do you want to install it now? › (Y/n)

You’ll lose the sweet image in your initial slide at this point, but that’s okay for now. Next, I’ll add another slide and include the bottom bar component with some additional Slidev features.

---
layout: iframe-right

# the web page source
url: https://juliopdx.com
---

# Hello, world

- Some sweet blog from a random dude on the internet <mdi-terminal />
- Thank you for stopping by <mdi-check-box />

<BarBottom  title="Slidev Example">
  <Item text="JulioPDX">
    <carbon:logo-github />
  Item>
  <Item text="@Julio_PDX">
    <carbon:logo-twitter />
  Item>
  <Item text="juliopdx.com">
    <carbon:globe />
  Item>
BarBottom>
---

There is a lot going on here but stick with me.

At the top, we have front matter for each slide representing a particular layout we may want.
In this case, I chose an iframe-right to display a live web page as we go through a presentation. Imagine presenting project documentation on the fly.
I added some random header of Hello, world and a list of information.
A little subtle, but we can also add emojis all over the place.
The bar bottom piece is almost a copy of the Purplin documentation. It allows you to present data at the bottom of each slide.

Diagrams

Slidev supports mermaid diagrams which I thought was awesome. Below you will see a rough example I was working on showing some roles in the Arista AVD Collection.

---
layout: center
---

# Ansible AVD Collection

```mermaid {scale: .58}
flowchart LR
    1(YAML);2(eos_designs_role);3(eos_cli_config_gen_role);4(eos_config_deploy_cvp_role);5(eos_validate_state_role);6(CSV/MD);7(YAML);8(EOS.CFG);9(YAML);10(CSV/MD);11(CloudVision)
    subgraph vars
      1
    end
    subgraph eos_designs
    2
    end
    subgraph eos_cli_config_gen
    3
    end
    subgraph eos_config_deploy_cvp
    4
    end
    subgraph eos_validate_state
    5
    end
    subgraph Documentation
      6
    end
    subgraph Structured-EOS-Config
      7
    end
    subgraph EOS-CLI-Config
      8
    end
    subgraph Ansible-Inventory
      9
    end
    subgraph Reports
      10
    end
    subgraph CloudVision
      11
    end
    subgraph Arista-EOS-Fabric
      S1(S1);S2(S2);L1(L1);L2(L2);L3(L3);L4(L4)
      S1 & S2 --- L1 & L2 & L3 & L4
    end
    vars --> eos_designs
    eos_designs --> Documentation
    eos_cli_config_gen --> Documentation
    eos_designs <--> Structured-EOS-Config
    Structured-EOS-Config --> eos_cli_config_gen
    Structured-EOS-Config --> eos_validate_state
    eos_cli_config_gen --> EOS-CLI-Config
    EOS-CLI-Config --> eos_config_deploy_cvp
    Ansible-Inventory --> eos_config_deploy_cvp
    eos_validate_state <--> Reports
    eos_validate_state <--> Arista-EOS-Fabric
    CloudVision <--> eos_config_deploy_cvp
    CloudVision <--> Arista-EOS-Fabric



    classDef arista_blue fill:#27569B,stroke:#333,stroke-width:2px;
    class 1,2,3,4,5,6,7,8,9,10,11 arista_blue
```

<BarBottom  title="Slidev Example">
  <Item text="JulioPDX">
    <carbon:logo-github />
  Item>
  <Item text="@Julio_PDX">
    <carbon:logo-twitter />
  Item>
  <Item text="juliopdx.com">
    <carbon:globe />
  Item>
BarBottom>
---

Syntax highlighting

I honestly cant pick a favorite feature, but syntax highlighting might be it. You can create code blocks like any markdown document, and the code will be highlighted for you. Slidev can also highlight specific lines as you go through a presentation.

---

# Colors 

<br>
<br>

<div grid="~ cols-3 gap-2" m="-t-3">

```yaml {all|3-5}
api: 1

group: com.company
artifact: LoremIpsum
version: 1.0-SNAPSHOT
type: txt
```

```json {all|8-13}
{
  "water": [
    true,
    343168281.98631763,
    1364186132
  ],
  "sky": true,
  "strong": [
    {
      "stopped": "quickly",
      "kitchen": "knife",
      "after": -506746570.6858001
    },
    "began",
    true
  ]
}
```

```python {all|4}
my_numbers = [7, 7, 7]
first = "Julio"
last = "PDX"
print("Hello, world")
```

div>

<BarBottom  title="Slidev Example">
  <Item text="JulioPDX">
    <carbon:logo-github />
  Item>
  <Item text="@Julio_PDX">
    <carbon:logo-twitter />
  Item>
  <Item text="juliopdx.com">
    <carbon:globe />
  Item>
BarBottom>
---

Again, a lot going on, but please stick with me.

We added a small amount of HTML within the slide to lower the code blocks. We then split the slide into three columns. We could also use the layout called two-cols in Slidev. But that only gets us two.
We then added three code blocks, one for YAML, JSON, and Python.
We can then ask Slidev to highlight portions of the code block as we go through the presentation. So, for example, {all|3-5} means, initially highlight everything, then on the next slide press, highlight lines three through five.

Start

A few click later

Final thoughts and links

Thank you for reading this far. It means a lot. I hope you learned a bit about Slidev and give it a shot. I’ve had a blast working with it so far. I took a long blogging break but hope to ramp back up. Maybe slower than I was publishing before. I wish you all the best, and thanks again.

Node Connections With Netreplica and Containerlab

Mon, 28 Mar 2022 12:08:20 -0700

Introduction

Thank you all for checking out another blog post. All the support for the blog has been tremendous and I can’t thank you all enough. Packet Pushers invited me to write on their platform and it’s been a pleasure working with Drew and team. If you would like to check those out, I will link them at the end of this post! Today we will focus on an interesting tool in the community, Netreplica, but specifically Graphite.

Some time ago Alex(@bortok) shared an example of using NextUI to build a visual from a Containerlab file. I asked if there was a way to click on the icon and just open an SSH session. This could then open a terminal like SecureCRT, web page, or some other means of interacting with a session.

Netreplica Graphite

I’ll walk through an example to get going and current pros and cons. Please note, Graphite is on initial phases of development. Please reach out to the folks at Netreplica(GitHub link below) for any feedback. There’s a couple options to get going with Graphite but I found standing up a container within a Containerlab file was the most simple. In the example below we are running 4 containers. One is a standalone container(ssh) with no connections to the rest of the topology. The Graphite container is using a special tag with the webssh implementation.

simple.yaml

name: simple
prefix: ""

mgmt:
  network: statics
  ipv4_subnet: 172.100.100.0/24

topology:
  kinds:
    srl:
      image: ghcr.io/nokia/srlinux
  nodes:
    pdx-rtr-01:
      kind: srl
      mgmt_ipv4: 172.100.100.11
    pdx-rtr-02:
      kind: srl
      mgmt_ipv4: 172.100.100.12
    pdx-rtr-03:
      kind: srl
      mgmt_ipv4: 172.100.100.13
    ssh:
      kind: linux
      image: netreplica/graphite:webssh2
      mgmt_ipv4: 172.100.100.100
      env:
        GRAPHITE_DEFAULT_TYPE: clab
        GRAPHITE_DEFAULT_TOPO: simple
        CLAB_SSH_CONNECTION: ${SSH_CONNECTION}
      binds:
        - .:/htdocs/clab
      ports:
        - 8080:80
      exec:
        - sh -c 'generate_offline_graph.sh'
        - sh -c 'graphite_motd.sh 8080'
      labels:
        graph-hide: yes
  links:
    - endpoints: ["pdx-rtr-01:e1-1", "pdx-rtr-02:e1-1"]
    - endpoints: ["pdx-rtr-01:e1-2", "pdx-rtr-03:e1-1"]
    - endpoints: ["pdx-rtr-02:e1-2", "pdx-rtr-03:e1-2"]

There’s a couple things to note:

Static IP addresses for management required.
Topology file should be shorthand and use longer yaml syntax. For example simple.clab.yml will not work, but simple.yaml will.

❯ sudo -E containerlab deploy -t simple.yaml
INFO[0000] Containerlab v0.25.1 started
INFO[0000] Parsing & checking topology file: simple.yaml
INFO[0000] Creating lab directory: /home/juliopdx/git/gcl/clab-simple
INFO[0000] Creating docker network: Name="statics", IPv4Subnet="172.100.100.0/24", IPv6Subnet="", MTU="1500"
INFO[0000] Creating container: "ssh"
INFO[0000] Creating container: "pdx-rtr-03"
INFO[0000] Creating container: "pdx-rtr-02"
INFO[0000] Creating container: "pdx-rtr-01"
INFO[0000] Creating virtual wire: pdx-rtr-01:e1-1 <--> pdx-rtr-02:e1-1
INFO[0000] Creating virtual wire: pdx-rtr-01:e1-2 <--> pdx-rtr-03:e1-1
INFO[0000] Creating virtual wire: pdx-rtr-02:e1-2 <--> pdx-rtr-03:e1-2
INFO[0001] Running postdeploy actions for Nokia SR Linux 'pdx-rtr-01' node
INFO[0001] Running postdeploy actions for Nokia SR Linux 'pdx-rtr-02' node
INFO[0001] Running postdeploy actions for Nokia SR Linux 'pdx-rtr-03' node
INFO[0015] Adding containerlab host entries to /etc/hosts file
INFO[0015] Executed command 'sh -c 'generate_offline_graph.sh'' on ssh. stdout:
Visualizing topology: simple.yaml
--truncated--
+---+------------+--------------+-----------------------------+-------+---------+--------------------+--------------+
| # |    Name    | Container ID |            Image            | Kind  |  State  |    IPv4 Address    | IPv6 Address |
+---+------------+--------------+-----------------------------+-------+---------+--------------------+--------------+
| 1 | pdx-rtr-01 | 81cd5aa9e62e | ghcr.io/nokia/srlinux       | srl   | running | 172.100.100.11/24  | N/A          |
| 2 | pdx-rtr-02 | 29a158c32b37 | ghcr.io/nokia/srlinux       | srl   | running | 172.100.100.12/24  | N/A          |
| 3 | pdx-rtr-03 | 4fe7e0beac6f | ghcr.io/nokia/srlinux       | srl   | running | 172.100.100.13/24  | N/A          |
| 4 | ssh        | 3eb1e14ca34c | netreplica/graphite:webssh2 | linux | running | 172.100.100.100/24 | N/A          |
+---+------------+--------------+-----------------------------+-------+---------+--------------------+--------------+

Now that we’ve deployed the topology, we can go to localhost:8080/graphite/. In my case I’m running Containerlab on my local laptop. Depending on your Containerlab file, you should see something like below.

Here’s where it gets fun. Check out what happens when we click on one of the nodes.

Now if we click on that link within the node, you will get a new connection window. In this case, I get a new Chrome window to the selected device.

Final Thoughts and Links

I’m a big fan of these tools that are making the lives easier for end users. Whether getting started with containers, learning network technologies, or just lowering the barrier of entry. Check out Graphite if you get a chance and reach out to the team on GitHub if you have any feedback or issues. Thank you for reading this far and I wish you all the best!

ExaBGP in the Lab

Fri, 25 Feb 2022 15:06:14 -0800

Intro

I was recently working my way through Optimal Routing design by Cisco Press… Can you tell that I get through books very slowly? I’ve probably used that sentence in my blog posts three or four times now! Either way, this chapter focused on scaling BGP and dealing with external connections. The section that really got me was external connections. The books goes over a lot, for example, handling inbound traffic, outbound traffic, full routes vs partial vs default. I wanted to see if there was a way to get a sample of the internet routing table in the lab. If you’re reading this then the goal was accomplished or I failed miserably and I feel like venting to the world. Either way, if you skipped over this paragraph then you missed my bad humor. Shame on you!

The Topology

graph LR 1((cmpa1)) 2((cmpa2)) 3((ispa)) 4((ispb)) 5((cloud)) 6((internet)) 1---3 2---4 3---5 4---5 6---5

Company A(cmpa) has two connections to different ISPs. ISPA is in AS 65001 and ISPB is in AS 65002. Company A also has an iBGP peering between its edge nodes in cmpa1 and cmpa2. In this example I used Arista cEOS nodes as the company routers and Nokia SR Linux at the ISP side.

Working with ExaBGP

When going down a particular route, I usually always check if something similar has been done before. I really don’t like to reinvent the wheel and if someone has created it freely for the world, why not give it a shot! Thankfully I found two really great resources for ExaBGP. One from Jason Murray, who made this happen before in GNS3 and a few great posts by Mat Wood. These posts were incredibly insightful but the step to convert the BGP data from RIPE to an ExaBGP format was time consuming.

Converting Manual Steps to Dockerfile

I eventually decided on wrapping these steps in a Dockerfile, which could then be used to build a container image. My eventual goal was to use this in a lab with something like Containerlab or Net-sim Tools.

FROM ubuntu:20.04

# Install dependencies
COPY bgp.cfg .
RUN apt update
RUN apt install python3-pip net-tools wget mrtparse vim nano -y && \
    rm -rf /var/lib/apt/lists/* && apt clean
RUN pip install exabgp==4.2.17

# Download latest table and remove when complete
RUN wget https://data.ris.ripe.net/rrc16/latest-bview.gz
RUN mrt2exabgp -G -P -4 172.16.2.1 latest-bview.gz > fullbgptable.py
RUN rm -rf latest-bview.gz

The default configuration file I created for ExaBGP supports two neighbors on AS 65001 and 65002 respectively. Vim and nano are included to allow users to modify this for their deployment. For example you could have ISP nodes on the same AS to simulate multiple external connections to the same ISP.

process announce-routes {
    run python3 fullbgptable.py;
    encoder json;
}

template {
    neighbor AS_65000 {
        router-id 172.16.2.1;
        local-as 65000;
        local-address 172.16.2.1;
    }
}

neighbor 172.16.2.2 {
    inherit AS_65000;
    peer-as 65001;
}

neighbor 172.16.2.3 {
    inherit AS_65000;
    peer-as 65002;
}

This container is now on Docker Hub, feel free to pull it down and check it out. After it was on Docker Hub, I had this random idea to automate the updates to the docker image. For example, the BGP view from RIPE is updated a decent amount throughout the day. I think it would be neat to grab the latest update at the end of the day and rebuild the docker image. This could then be tagged with latest and the date it was created.

Migrating Docker Build to GitHub Actions

GitHub Actions was fairly new to me, besides some hello world example. I found this neat example from Chamod Shehanka that I used as a starting point. I still had to add scheduled updates and syncing the readme/description on Docker Hub with the repository on GitHub. The latter is only available for some pro version of Docker Hub automated builds but luckily there’s a GitHub Action to get around this. Example starting workflow file below.

name: Docker Image CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:

  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: docker login
      env:
        DOCKER_USER: ${{secrets.DOCKER_USER }}
        DOCKER_PASSWORD: ${{secrets.DOCKER_PASSWORD }}
      run: |
                docker login -u $DOCKER_USER -p $DOCKER_PASSWORD
    - name: Build the Docker image
      run: docker build . --file Dockerfile --tag juliopdx/exabgp-irt:$(date +%s)

    - name: Docker Push
      run: docker push juliopdx/exabgp-irt

Adding GitHub Actions

Shorty after I discovered two really slick GitHub Actions from some awesome folks to streamline the workflow and arguably make it look a bit cleaner. One being Docker Build & Push by Sean Smith and the other sync-readme by Damian Mee. Both offer a simplicity to the workflow file that I really enjoy. This led me to fairly finalizing this docker workflow.

name: Docker Image CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    - cron: "0 0 * * *"

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: check out code
        uses: actions/checkout@v2

      - name: build and push docker image
        uses: mr-smithers-excellent/docker-build-push@v5
        with:
          image: juliopdx/exabgp-irt
          registry: docker.io
          username: ${{ secrets.DOCKER_USER }}
          password: ${{ secrets.DOCKER_PASSWORD }}
          tags: latest, $(date +%F)

      - name: sync desc and readme
        uses: meeDamian/sync-readme@v1.0.6
        with:
          user: ${{secrets.DOCKER_USER }}
          pass: ${{ secrets.DOCKER_PASSWORD }}

Incorporating ExaBGP in the Lab

Now that we have the docker image created, we can incorporate it in our lab. For this example I leveraged Containerlab.

name: irt
prefix: ""

topology:

  kinds:
    ceos:
      image: ceos:4.27.2F
    srl:
      image: ghcr.io/nokia/srlinux

  nodes:
    ispa:
      kind: srl
      startup-config: conf/ispa.json
      mgmt_ipv4: 172.20.20.11
    ispb:
      kind: srl
      startup-config: conf/ispb.json
      mgmt_ipv4: 172.20.20.12
    internet: # ifconfig eth1 172.16.2.1 netmask 255.255.255.0 up
      kind: linux
      image: juliopdx/exabgp-irt
      mgmt_ipv4: 172.20.20.13
    cmpa1:
      kind: ceos
      startup-config: conf/cmpa1.conf
      mgmt_ipv4: 172.20.20.14
    cmpa2:
      kind: ceos
      startup-config: conf/cmpa2.conf
      mgmt_ipv4: 172.20.20.15
    cloud:
      kind: ceos
      mgmt_ipv4: 172.20.20.16

  links:
    - endpoints: ["cloud:eth1", "internet:eth1"]
    - endpoints: ["cloud:eth2", "ispa:e1-1"]
    - endpoints: ["cloud:eth3", "ispb:e1-1"]
    - endpoints: ["ispa:e1-2", "cmpa1:eth1"]
    - endpoints: ["ispb:e1-2", "cmpa2:eth1"]
    - endpoints: ["cmpa1:eth2", "cmpa2:eth2"]

For starters I’ll configure the internet node(ExaBGP) with an IP address and activate ExaBGP. In this case not advertising routes, just validating peering.

root@internet:/# cat bgp.cfg
#process announce-routes {
#    run python3 fullbgptable.py;
#    encoder json;
#}

template {
    neighbor AS_65000 {
        router-id 172.16.2.1;
        local-as 65000;
        local-address 172.16.2.1;
    }
}

neighbor 172.16.2.2 {
    inherit AS_65000;
    peer-as 65001;
}

neighbor 172.16.2.3 {
    inherit AS_65000;
    peer-as 65002;
}
root@internet:/#

juliopdx@containerlab:~$ docker exec -it internet bash
root@internet:/# ifconfig eth1 172.16.2.1 netmask 255.255.255.0 up
root@internet:/# exabgp bgp.cfg
04:47:01 | 22     | welcome       | Thank you for using ExaBGP
04:47:01 | 22     | version       | 4.2.17
04:47:01 | 22     | interpreter   | 3.8.10 (default, Nov 26 2021, 20:14:08)  [GCC 9.3.0]
04:47:01 | 22     | os            | Linux internet 5.4.0-99-generic #112-Ubuntu SMP Thu Feb 3 13:50:55 UTC 2022 x86_64
04:47:01 | 22     | installation  | /usr/local
04:47:01 | 22     | advice        | environment file missing
04:47:01 | 22     | advice        | generate it using "exabgp --fi > /usr/local/etc/exabgp/exabgp.env"
04:47:01 | 22     | cli           | could not find the named pipes (exabgp.in and exabgp.out) required for the cli
04:47:01 | 22     | cli           | we scanned the following folders (the number is your PID):
04:47:01 | 22     | cli control   |  - /run/exabgp/
04:47:01 | 22     | cli control   |  - /run/0/
04:47:01 | 22     | cli control   |  - /run/
04:47:01 | 22     | cli control   |  - /var/run/exabgp/
04:47:01 | 22     | cli control   |  - /var/run/0/
04:47:01 | 22     | cli control   |  - /var/run/
04:47:01 | 22     | cli control   |  - /usr/local/run/exabgp/
04:47:01 | 22     | cli control   |  - /usr/local/run/0/
04:47:01 | 22     | cli control   |  - /usr/local/run/
04:47:01 | 22     | cli control   |  - /usr/local/var/run/exabgp/
04:47:01 | 22     | cli control   |  - /usr/local/var/run/0/
04:47:01 | 22     | cli control   |  - /usr/local/var/run/
04:47:01 | 22     | cli control   | please make them in one of the folder with the following commands:
04:47:01 | 22     | cli control   | > mkfifo //run/exabgp.{in,out}
04:47:01 | 22     | cli control   | > chmod 600 //run/exabgp.{in,out}
04:47:01 | 22     | configuration | performing reload of exabgp 4.2.17
04:47:01 | 22     | reactor       | loaded new configuration successfully
04:47:01 | 22     | reactor       | connected to peer-1 with outgoing-1 172.16.2.1-172.16.2.2
04:47:01 | 22     | reactor       | connected to peer-2 with outgoing-1 172.16.2.1-172.16.2.3

juliopdx@containerlab:~$ docker exec -it ispa sr_cli
Using configuration file(s): []
Welcome to the srlinux CLI.
Type 'help' (and press ) if you need any help using this.
--{ running }--[  ]--
A:ispa# show network-instance default protocols bgp neighbor
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BGP neighbor summary for network-instance "default"
Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+-------------------+---------------------------+-------------------+-------+----------+---------------+---------------+--------------+---------------------------+
|     Net-Inst      |           Peer            |       Group       | Flags | Peer-AS  |     State     |    Uptime     |   AFI/SAFI   |      [Rx/Active/Tx]       |
+===================+===========================+===================+=======+==========+===============+===============+==============+===========================+
| default           | 10.0.0.1                  | cmpa              | S     | 65003    | established   | 0d:0h:4m:51s  | ipv4-unicast | [0/0/2]                   |
| default           | 172.16.2.1                | internet          | S     | 65000    | established   | 0d:0h:3m:21s  | ipv4-unicast | [0/0/2]                   |
+-------------------+---------------------------+-------------------+-------+----------+---------------+---------------+--------------+---------------------------+
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Summary:
2 configured neighbors, 2 configured sessions are established,0 disabled peers
0 dynamic peers
--{ running }--[  ]--
A:ispa#

juliopdx@containerlab:~$ docker exec -it ispb sr_cli
Using configuration file(s): []
Welcome to the srlinux CLI.
Type 'help' (and press ) if you need any help using this.
--{ running }--[  ]--
A:ispb# show network-instance default protocols bgp neighbor
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BGP neighbor summary for network-instance "default"
Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+-------------------+---------------------------+-------------------+-------+----------+---------------+---------------+--------------+---------------------------+
|     Net-Inst      |           Peer            |       Group       | Flags | Peer-AS  |     State     |    Uptime     |   AFI/SAFI   |      [Rx/Active/Tx]       |
+===================+===========================+===================+=======+==========+===============+===============+==============+===========================+
| default           | 10.0.0.2                  | cmpa              | S     | 65003    | established   | 0d:0h:6m:18s  | ipv4-unicast | [0/0/2]                   |
| default           | 172.16.2.1                | internet          | S     | 65000    | established   | 0d:0h:4m:49s  | ipv4-unicast | [0/0/2]                   |
+-------------------+---------------------------+-------------------+-------+----------+---------------+---------------+--------------+---------------------------+
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Summary:
2 configured neighbors, 2 configured sessions are established,0 disabled peers
0 dynamic peers
--{ running }--[  ]--
A:ispb#

juliopdx@containerlab:~$ docker exec -it cmpa1 Cli
cmpa1>en
cmpa1#show ip bgp summary
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 65003
Neighbor Status Codes: m - Under maintenance
  Neighbor V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.0.0.0 4 65001             24        29    0    0 00:10:22 Estab   2      2
  10.0.0.7 4 65003             17        16    0    0 00:10:22 Estab   2      2
cmpa1#

Advertising Many NLRI

At this point all peering has been verified. I will reenable ExaBGP but advertise all the routes this time. This process does take some time to complete and I will explain a few caveats at the end of this post.

root@internet:/# cat bgp.cfg
process announce-routes {
    run python3 fullbgptable.py;
    encoder json;
}

template {
    neighbor AS_65000 {
        router-id 172.16.2.1;
        local-as 65000;
        local-address 172.16.2.1;
    }
}

neighbor 172.16.2.2 {
    inherit AS_65000;
    peer-as 65001;
}

neighbor 172.16.2.3 {
    inherit AS_65000;
    peer-as 65002;
}
root@internet:/# exabgp bgp.cfg

I’ll save your eyes, the output is a lot! For fun, lets check out some crazy prefix numbers on one ISP node and one Company A node.

A:ispa# show network-instance default protocols bgp summary
------------------------------------------------------------------------------------------------------------------------------------------------
BGP is enabled and up in network-instance "default"
Global AS number  : 65001
BGP identifier    : 172.16.2.2
------------------------------------------------------------------------------------------------------------------------------------------------
  Total paths               : 990
  Received routes           : 74867
  Received and active routes: 74467
  Total UP peers            : 2
  Configured peers          : 2, 0 are disabled
  Dynamic peers             : None
------------------------------------------------------------------------------------------------------------------------------------------------
--{ running }--[  ]--
A:ispa#

cmpa1#show ip bgp summary
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 65003
Neighbor Status Codes: m - Under maintenance
  Neighbor V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.0.0.0 4 65001           1672       721    0    0 00:21:26 Estab   87670  87670
  10.0.0.7 4 65003           1684      1668    0    0 00:21:26 Estab   86701  86701
cmpa1#

That is beautiful, 74K routes on ISPA and 87K routes on Company A node.

Use Cases

Reducing Excess Information Client Side

Let me walk through a few examples on how this could be useful in a lab scenario or for general learning. You may have noticed that the Company A node is receiving routes from both ISPA and the iBGP neighbor. Seems like a bit of a waste in resources and memory. We can configure the iBGP neighbors to only send the default route from the ISP. The below configuration is done on both node 1 and 2 of Company A.

ip prefix-list default seq 10 permit 0.0.0.0/0
!
route-map default permit 10
   match ip address prefix-list default
router bgp 65003
   neighbor 10.0.0.7 route-map default out

cmpa1#show ip bgp summary
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 65003
Neighbor Status Codes: m - Under maintenance
  Neighbor V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.0.0.0 4 65001           8341      7566    0    0 00:31:11 Estab   18898  18898
  10.0.0.7 4 65003           8540      8454    0    0 00:31:11 Estab   1      1
cmpa1#

Now we are only receiving the default route from our iBGP neighbor! This iBGP route map also has the benefit of preventing our AS from being a transit AS. Routes learned from one node will not be advertised to the other ISP and possibly severely hindering our edge devices. Maybe we want to filter the routes inbound from the ISPs.

router bgp 65003
   neighbor 10.0.0.0 route-map default in

cmpa1#show ip bgp sum
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 65003
Neighbor Status Codes: m - Under maintenance
  Neighbor V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.0.0.0 4 65001          30589      7657    0    0 01:09:18 Estab   257372 1
  10.0.0.7 4 65003           8585      8499    0    0 01:09:17 Estab   1      1
cmpa1#show ip bgp | b Network
          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      0.0.0.0/0              10.0.0.0              0       -          100     0       65001 199524 3257 i
 *        0.0.0.0/0              10.0.0.7              0       -          100     0       65002 199524 3257 i
cmpa1#

Reducing Excess Information Provider Side

Maybe we have a really simple site and we no longer need full tables. Our days of being fancy are over. I’ll remove the previous configuration besides the iBGP route-map. Then we will update the ISP to only send the default.

juliopdx@containerlab:~$ docker exec -it ispa sr_cli
Using configuration file(s): []
Welcome to the srlinux CLI.
Type 'help' (and press ) if you need any help using this.
--{ running }--[  ]--
A:ispa# enter candidate
--{ candidate shared default }--[  ]--
A:ispa# routing-policy
--{ candidate shared default }--[ routing-policy ]--
A:ispa# set policy default-route default-action reject
A:ispa# set policy default-route statement 10 match prefix-set default
A:ispa# set policy default-route statement 10 action accept
A:ispa# set prefix-set default prefix 0.0.0.0/0 mask-length-range exact
A:ispa# /network-instance default protocols bgp
--{ * candidate shared default }--[ network-instance default protocols bgp ]--
A:ispa# set group cmpa export-policy default-route
A:ispa# commit save
/system:
    Saved current running configuration as initial (startup) configuration '/etc/opt/srlinux/config.json'

All changes have been committed. Leaving candidate mode.
--{ running }--[ network-instance default protocols bgp ]--
A:ispa# show neighbor 10.0.0.1 advertised-routes ipv4
------------------------------------------------------------------------------------------------------------------------------------------------
Peer        : 10.0.0.1, remote AS: 65003, local AS: 65001
Type        : static
Description : None
Group       : cmpa
------------------------------------------------------------------------------------------------------------------------------------------------
Origin codes: i=IGP, e=EGP, ?=incomplete
+---------------------------------------------------------------------------------------------------------------------------------------------+
|          Network                  Next Hop              MED           LocPref                       AsPath                       Origin     |
+=============================================================================================================================================+
| 0.0.0.0/0                    10.0.0.0                    -              100        [65001, 199524, 3257]                            i       |
+---------------------------------------------------------------------------------------------------------------------------------------------+
------------------------------------------------------------------------------------------------------------------------------------------------
1 advertised BGP routes
------------------------------------------------------------------------------------------------------------------------------------------------
--{ running }--[ network-instance default protocols bgp ]--
A:ispa#

cmpa1#  show ip bgp sum
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 65003
Neighbor Status Codes: m - Under maintenance
  Neighbor V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.0.0.0 4 65001          35789      7711    0    0 01:30:56 Estab   1      1
  10.0.0.7 4 65003           8615      8526    0    0 01:30:56 Estab   1      1
cmpa1#

Outbound Traffic Steering

Partial Routes on Each Node by AS Path Length

Maybe we want to control outbound paths on each Company A node. One node gets routes with AS path length of three or less with a default route. The other will get routes with an AS length of four or more and a default. The iBGP route map will be removed.

# cmpa1
ip prefix-list default seq 10 permit 0.0.0.0/0
!
route-map default permit 5
   match as-path length <= 3
!
route-map default permit 10
   match ip address prefix-list default
!
router bgp 65003
   neighbor 10.0.0.0 route-map default in

# cmpa2
ip prefix-list default seq 10 permit 0.0.0.0/0
!
route-map default permit 5
   match as-path length >= 4 and <= 4000
!
route-map default permit 10
   match ip address prefix-list default
!
router bgp 65003
   neighbor 10.0.0.3 route-map default in

I’ll share a small sample from active BGP NLRI on each node. Note the AS path lengh on that information.

cmpa1#show ip bgp neighbors 10.0.0.0 routes | b AS Path
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      0.0.0.0/0              10.0.0.0              0       -          100     0       65001 199524 3257 i
 * >      1.0.0.0/24             10.0.0.0              0       -          100     0       65001 52320 13335 i
 * >      1.1.1.0/24             10.0.0.0              0       -          100     0       65001 52320 13335 i
 * >      1.9.0.0/16             10.0.0.0              0       -          100     0       65001 6939 4788 i
 * >      1.12.0.0/20            10.0.0.0              0       -          100     0       65001 52320 132203 i
 * >      1.12.34.0/23           10.0.0.0              0       -          100     0       65001 52320 132203 i
 * >      1.37.0.0/16            10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.0.0/17            10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.27.0/24           10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.32.0/24           10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.35.0/24           10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.36.0/22           10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.47.0/24           10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.86.0/24           10.0.0.0              0       -          100     0       65001 6939 4775 i
 * >      1.37.128.0/17          10.0.0.0              0       -          100     0       65001 6939 4775 i

cmpa2#show ip bgp neighbors 10.0.0.3 routes | b AS Path
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      0.0.0.0/0              10.0.0.3              0       -          100     0       65002 199524 3257 i
 * >      1.0.4.0/22             10.0.0.3              0       -          100     0       65002 6939 4826 38803 i
 * >      1.0.4.0/24             10.0.0.3              0       -          100     0       65002 6939 4826 38803 i
 * >      1.0.5.0/24             10.0.0.3              0       -          100     0       65002 6939 4826 38803 i
 * >      1.0.6.0/24             10.0.0.3              0       -          100     0       65002 6939 4826 38803 i
 * >      1.0.7.0/24             10.0.0.3              0       -          100     0       65002 6939 4826 38803 i
 * >      1.0.64.0/18            10.0.0.3              0       -          100     0       65002 52320 1299 2497 7670 18144 i
 * >      1.0.128.0/17           10.0.0.3              0       -          100     0       65002 6939 38040 23969 i
 * >      1.0.128.0/18           10.0.0.3              0       -          100     0       65002 6939 38040 23969 i
 * >      1.0.128.0/19           10.0.0.3              0       -          100     0       65002 6939 38040 23969 i
 * >      1.0.128.0/24           10.0.0.3              0       -          100     0       65002 6939 4651 23969 i
 * >      1.0.129.0/24           10.0.0.3              0       -          100     0       65002 6939 4651 23969 i
 * >      1.0.130.0/23           10.0.0.3              0       -          100     0       65002 6939 38040 23969 i
 * >      1.0.132.0/24           10.0.0.3              0       -          100     0       65002 6939 38040 23969 i
 * >      1.0.133.0/24           10.0.0.3              0       -          100     0       65002 6939 38040 23969 i

Partial Routes on Each Node by Prefix

Please see the updated configuration on each node and their respective NLRI.

# cmpa1
ip prefix-list default seq 10 permit 0.0.0.0/0
ip prefix-list half seq 10 permit 0.0.0.0/1 le 32
!
route-map default permit 5
   match ip address prefix-list half
!
route-map default permit 10
   match ip address prefix-list default
!
router bgp 65003
   neighbor 10.0.0.0 route-map default in

# cmpa2
ip prefix-list default seq 10 permit 0.0.0.0/0
ip prefix-list half seq 10 permit 128.0.0.0/2 le 32
!
route-map default permit 5
   match ip address prefix-list half
!
route-map default permit 10
   match ip address prefix-list default
!
router bgp 65003
   neighbor 10.0.0.3 route-map default in

cmpa1# show ip bgp neighbors 10.0.0.0 routes | b AS Path
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      0.0.0.0/0              10.0.0.0              0       -          100     0       65001 199524 3257 i
 * >      1.0.0.0/24             10.0.0.0              0       -          100     0       65001 52320 13335 i
 * >      1.0.4.0/22             10.0.0.0              0       -          100     0       65001 6939 4826 38803 i
 * >      1.0.4.0/24             10.0.0.0              0       -          100     0       65001 6939 4826 38803 i
 * >      1.0.5.0/24             10.0.0.0              0       -          100     0       65001 6939 4826 38803 i
 * >      1.0.6.0/24             10.0.0.0              0       -          100     0       65001 6939 4826 38803 i
 * >      1.0.7.0/24             10.0.0.0              0       -          100     0       65001 6939 4826 38803 i
 * >      1.0.64.0/18            10.0.0.0              0       -          100     0       65001 52320 1299 2497 7670 18144 i
 * >      1.0.128.0/17           10.0.0.0              0       -          100     0       65001 6939 38040 23969 i
 * >      1.0.128.0/18           10.0.0.0              0       -          100     0       65001 6939 38040 23969 i

cmpa2#show ip bgp neighbors 10.0.0.3 routes | b AS Path
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      0.0.0.0/0              10.0.0.3              0       -          100     0       65002 199524 3257 i
 * >      128.0.1.0/24           10.0.0.3              0       -          100     0       65002 52320 9009 i
 * >      128.0.116.0/24         10.0.0.3              0       -          100     0       65002 52320 1299 57878 i
 * >      128.0.118.0/24         10.0.0.3              0       -          100     0       65002 52320 1299 57878 i
 * >      128.1.36.0/24          10.0.0.3              0       -          100     0       65002 6939 4766 i
 * >      128.1.76.0/24          10.0.0.3              0       -          100     0       65002 6939 4766 i
 * >      128.28.0.0/16          10.0.0.3              0       -          100     0       65002 52320 2914 2514 2514 i
 * >      128.45.0.0/16          10.0.0.3              0       -          100     0       65002 52320 1267 24608 i
 * >      128.53.0.0/16          10.0.0.3              0       -          100     0       65002 52320 2914 2514 2514 i
 * >      128.65.96.0/21         10.0.0.3              0       -          100     0       65002 6939 42010 i

Caveats

Now for the issues I had with ExaBGP. I really shouldn’t say issues, the things I ran into were mostly user error. I’ve spent about two days with it and I really like it so far. Like anything I write, this was just a tiny fraction of what ExaBGP can do. If you see some error in the Dockerfile or process in general, please reach out to me. I’d love to update this and get it nicely packaged into something useful. One error I ran into, at some point in the NLRI announcements, ExaBGP will just stop sending or keep a peer session established. It’s something I haven’t figured out and if someone more familiar with the tool can give me some insight, I’m all ears!

Outro and Links

Thank you all for reading this far. Really means the world to me and writing this one was a joy. Anytime I get to play with BGP in the lab is a good time. So much to learn with BGP… I tried to include links of posts or documents that helped me throughout this small journey. Please check them out if you have time. Below are some additional links that helped me out a lot!

Network Simulation Tools and Containerlab

Sun, 13 Feb 2022 21:07:43 -0800

Introduction

I’ve been progressing through a series of technical books, some of which I’ve shared on other blogs. A few of them focus on BGP. BGP being so broad, I decided to create a challenge lab. Creating the challenge/troubleshooting labs has really made more concepts stick. I’m trying to use the principle of teaching someone to make the learning last. These have been incredibly fun to create and the community interaction has been amazing. One brave soul(Jeroen van Bemmel) shared his solution. I was fascinated on his solution and how he created his topology with Containerlab and net-sim tools.

net-sim tools

Network Simulation Tools was created by Ivan Pepelnjak of ipspace.net. I think Ivan was tired of the point and click required when making a lab topology with other tools. The aim of the tool is to take infrastructure as code principles and leverage them for the creation of network topologies. Topologies can be defined in a simple YAML file and net-sim will take care of link connections, addressing(IPv4/6), routing protocols, etc…

Usually when creating a lab, there is a slow process of starting up nodes, connecting links, selecting address space, configuring hostname Rx, configuring ip add 10.x.x.x 255.255.255.0, no shut, and so on and so forth. At this point, I don’t think I need to configure an IP address on an interface to know I’ve mastered that skill :).

Getting Started

In this post we will leverage Containerlab with net-sim tools and below are the requirements to get started.

There are a few options to get started with net-sim tools, but I think using a python virtualenv was fairly simple.

python3 -m venv venv
source venv/bin/activate
pip install netsim-tools
pip install ansible #decided to use whatever was latest and it works just fine for me

I know this seems like a lot but the benefits are definitely worth it.

Defining a Topology

Let me share a basic topology and walk you through what will be created. Please note, some of the options I have defined are already defaults, I just decided to add them to the topology file for reader clarity.

name: basic
provider: clab
defaults:
  device: eos
  devices:
    eos.clab.image: ceos:4.27.2F


addressing:
  loopback:
    ipv4: 10.0.0.0/24
    ipv6: 200::/7
  router_id:
    ipv4: 10.0.0.0/24
    prefix: 32
  p2p:
    ipv4: 10.1.0.0/16
    prefix: 30
    ipv6: true

nodes: [R1, R2, R3]

links:
  - R1-R2
  - R2-R3

This Topology will perform the following:

Use Containerlab to deploy our nodes
Sets a default image version for EOS devices(saves a lot of typing)
Set prefix information for different connections, not required since net-sim will do this for you.
Define how many nodes we want
Define how we want the nodes connected.

A few things to note. I defined the IP space but that is not required when creating lab topologies. No need to worry about interface names, the logic within net-sim tools will just increment over the next available interface.

Providers

At the time of this writing, net-sim tools supports three popular providers; vagrant-libvirt, Vagrant VirtualBox, and Containerlab. As long as the user is familiar with their provider of choice, the interaction with net-sim tools is similar across the board. In the example topology file above, we defined Containerlab as our provider. This will cause net-sim tools to create a clab.yml file in the structure Containerlab is used to. This clab.yml file will then be used to deploy our network devices.

Images

Net-sim does not come pre packaged with any network operating systems(NOS) images. A few vendors have made their images publicly available(see Nokia/FRR) as a simple docker pull. For any other images, please see their documentation to obtain an image. For this example, I will be using some Arista cEOS nodes. They can be obtained by signing up to their support site(free).

Networks

Net-sim gives you pretty much any option you could want for network addressing. You can either use built-in address pools, assign statics, or define your own pools. Net-sim will then handle the IP addressing for each network device and interface. Very simple example below, please check out their in-depth addressing tutorial!

  p2p:
    ipv4: 10.1.0.0/16
    prefix: 30
    ipv6: true

nodes: [R1, R2, R3]

links:
  - R1-R2
  - R2-R3

graph LR R1((R1)) --- R2((R2)) R2 --- R3((R3))

Here we have three nodes with two point to point links. R1 will have node ID 1, R2 will be node ID 2, etc… The first /30 prefix in the 10.1.0.0/16 network will be 10.1.0.0-3. This will be assigned to the link between R1-R2. The next available /30 will be 10.1.0.4-7. Same as before, this network will be assigned to the R2-R3 link.

Deployments and How It All Works

The CLI has a few neat options, let me show you some I think will be used the most.

netlab create

This command will create any files necessary to deploy your containers and configure devices with Ansible. Great if you would just like to view the data without actually performing any deployments.

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ netlab create basic.yml
Created provider configuration file: clab.yml
Created group_vars for all
Created group_vars for eos
Created host_vars for R1
Created host_vars for R2
Created host_vars for R3
Created minimized Ansible inventory hosts.yml
Created Ansible configuration file: ansible.cfg
(venv) juliopdx@containerlab:~/repos/cl-nst-demo$

Lets take a look at the clab.yml file.

name: basic

topology:
  nodes:
    R1:
      kind: ceos
      image: ceos:4.27.2F

    R2:
      kind: ceos
      image: ceos:4.27.2F

    R3:
      kind: ceos
      image: ceos:4.27.2F



  links:
  - endpoints:
    - "R1:eth1"
    - "R2:eth1"
  - endpoints:
    - "R2:eth2"
    - "R3:eth1"

This can then be used to run either netlab up basic.yml or sudo containerlab deploy -t clab.yml. One of the more interesting files created is the host_vars//topology.yml. This gives you a great look at everything that will be deployed on the device with Ansible

# Ansible inventory created from ['basic.yml', 'package:topology-defaults.yml']
#
---
af:
  ipv4: true
  ipv6: true
box: ceos:4.27.2F
clab:
  kind: ceos
hostname: clab-basic-R1
interfaces:
- clab:
    name: eth1
  ifindex: 1
  ifname: Ethernet1
  ipv4: 10.1.0.1/30
  ipv6: true
  linkindex: 1
  name: R1 -> R2
  neighbors:
  - ifname: Ethernet1
    ipv4: 10.1.0.2/30
    ipv6: true
    node: R2
  remote_id: 2
  remote_ifindex: 1
  type: p2p
loopback:
  ipv4: 10.0.0.1/32
  ipv6: 200:0:0:1::1/64
mgmt:
  ifname: Management0
  ipv4: 192.168.121.101
  mac: 08-4F-A9-00-00-01

netlab up

This command will actually deploy our nodes with Containerlab and configure the initial configuration with Ansible. You can use this command instead of using individual commands to deploy the topology.

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ netlab up basic.yml
Created provider configuration file: clab.yml
Created group_vars for all
Created group_vars for eos
Created host_vars for R1
Created host_vars for R2
Created host_vars for R3
Created minimized Ansible inventory hosts.yml
Created Ansible configuration file: ansible.cfg

Step 2: Checking virtualization provider installation
============================================================
.. all tests succeeded, moving on

Step 3: starting the lab
============================================================
INFO[0000] Containerlab v0.23.0 started
INFO[0000] Parsing & checking topology file: clab.yml
INFO[0000] Creating lab directory: /home/juliopdx/repos/cl-nst-demo/clab-basic
INFO[0000] Creating docker network: Name='clab', IPv4Subnet='172.20.20.0/24', IPv6Subnet='2001:172:20:20::/64', MTU='1500'
INFO[0000] Creating container: R2
INFO[0000] Creating container: R1
INFO[0000] Creating container: R3
INFO[0001] Creating virtual wire: R1:eth1 <--> R2:eth1
INFO[0001] Creating virtual wire: R2:eth2 <--> R3:eth1
INFO[0001] Running postdeploy actions for Arista cEOS 'R1' node
INFO[0001] Running postdeploy actions for Arista cEOS 'R2' node
INFO[0001] Running postdeploy actions for Arista cEOS 'R3' node
INFO[0053] Adding containerlab host entries to /etc/hosts file
+---+---------------+--------------+--------------+------+---------+----------------+----------------------+
| # |     Name      | Container ID |    Image     | Kind |  State  |  IPv4 Address  |     IPv6 Address     |
+---+---------------+--------------+--------------+------+---------+----------------+----------------------+
| 1 | clab-basic-R1 | ffb8857ca087 | ceos:4.27.2F | ceos | running | 172.20.20.2/24 | 2001:172:20:20::2/64 |
| 2 | clab-basic-R2 | c533fc2411e0 | ceos:4.27.2F | ceos | running | 172.20.20.3/24 | 2001:172:20:20::3/64 |
| 3 | clab-basic-R3 | da9fedb7ce7f | ceos:4.27.2F | ceos | running | 172.20.20.4/24 | 2001:172:20:20::4/64 |
+---+---------------+--------------+--------------+------+---------+----------------+----------------------+

Step 4: deploying initial device configurations
============================================================

PLAY [Deploy device configuration] *************************************************************

TASK [Find initial configuration template] *****************************************************
skipping: [R1] => (item=/home/juliopdx/repos/cl-nst-demo/venv/lib/python3.8/site-packages/netsim/ansible/templates/initial/eos.j2)
skipping: [R1]
skipping: [R3] => (item=/home/juliopdx/repos/cl-nst-demo/venv/lib/python3.8/site-packages/netsim/ansible/templates/initial/eos.j2)
skipping: [R3]
skipping: [R2] => (item=/home/juliopdx/repos/cl-nst-demo/venv/lib/python3.8/site-packages/netsim/ansible/templates/initial/eos.j2)
skipping: [R2]

TASK [Deploy initial device configuration] *****************************************************
included: /home/juliopdx/repos/cl-nst-demo/venv/lib/python3.8/site-packages/netsim/ansible/tasks/deploy-config/eos.yml for R2, R1, R3 => (item=/home/juliopdx/repos/cl-nst-demo/venv/lib/python3.8/site-packages/netsim/ansible/tasks/deploy-config/eos.yml)

TASK [wait_for_connection] *********************************************************************
ok: [R2]
ok: [R1]
ok: [R3]

TASK [arista.eos.eos_config] *******************************************************************
[WARNING]: To ensure idempotency and correct diff the input configuration lines should be
similar to how they appear if present in the running configuration on device including the
indentation
changed: [R3]
changed: [R1]
changed: [R2]

TASK [Deploy module-specific configurations] ***************************************************

TASK [Deploy custom deployment templates] ******************************************************

PLAY RECAP *************************************************************************************
R1                         : ok=3    changed=1    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0
R2                         : ok=3    changed=1    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0
R3                         : ok=3    changed=1    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$

Checking the configuration on R1.

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ docker exec -it clab-basic-R1 Cli
R1>en
R1#show run
! Command: show running-config
! device: R1 (cEOSLab, EOS-4.27.2F-26069621.4272F (engineering build))
!
no aaa root
!
username admin privilege 15 role network-admin secret sha512 $6$KKU2DJRmdlpXm3wG$dZKDhT4eC5ZcNcwyPtklQfNrOnUw28P4dNYTSDB5LiBp/gV0r24ET/D.4L9AeBLmCwcwyMsgtopRkaHigNkHC.
!
transceiver qsfp default-mode 4x10G
!
service routing protocols model multi-agent
!
hostname R1
ip host R2 10.0.0.2 10.1.0.2 10.1.0.5
ip host R3 10.0.0.3 10.1.0.6
!
spanning-tree mode mstp
!
management api http-commands
   no shutdown
!
management api gnmi
   transport grpc default
!
management api netconf
   transport ssh default
!
interface Ethernet1
   description R1 -> R2
   mac-address 52:dc:ca:fe:01:01
   no switchport
   ip address 10.1.0.1/30
   ipv6 enable
!
interface Loopback0
   ip address 10.0.0.1/32
   ipv6 address 200:0:0:1::1/64
!
interface Management0
   ip address 172.20.20.2/24
   ipv6 address 2001:172:20:20::2/64
   no lldp transmit
   no lldp receive
!
ip routing
!
ipv6 unicast-routing
!
end
R1#

I think that is pretty awesome. With one command we have deployed nodes, set base parameters, and interface configurations. At this point someone could proceed to learn whatever the objective is for the day or test some design.

netlab down

This command will remove all containers and remove any directory created by Containerlab(startup configs). If you would like to keep startup configurations, I would recommend you shutdown the lab with Containerlab commands.

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ netlab down basic.yml

Step 1: Checking virtualization provider installation
============================================================
.. all tests succeeded, moving on


Step 2: stopping the lab
============================================================
INFO[0000] Parsing & checking topology file: clab.yml
INFO[0000] Destroying lab: basic
INFO[0001] Removed container: clab-basic-R1
INFO[0001] Removed container: clab-basic-R3
INFO[0001] Removed container: clab-basic-R2
INFO[0001] Removing containerlab host entries from /etc/hosts file
(venv) juliopdx@containerlab:~/repos/cl-nst-demo$

Python, Ansible, and Jinja

Behind the scenes Python is wrapping everything that has been defined into usable data. This can then be used with Ansible and Jinja to create vendor specific configurations. This could be interface settings, BGP, OSPF, and others. For a list of supported configurations by NOS, check it out here. I have probably atrociously oversimplified this process, thankfully the creators made this in-depth guide on the process.

Sample Use Cases

So far I think this is pretty neat. Similar to Containerlab, I think net-sim lends itself incredibly well in learning environments or testing solutions/designs. Let me walk you through some simple workflows someone might go through when learning a technology.

Basic Lab for OSPF Learning

We have shown this topology already but bear with me. In this example we have three nodes that we would like to deploy to start learning some OSPF. The topology file can deploy nodes and configure all interface settings.

name: basic
provider: clab
defaults:
  device: eos
  devices:
    eos.clab.image: ceos:4.27.2F


addressing:
  loopback:
    ipv4: 10.0.0.0/24
    ipv6: 200::/7
  router_id:
    ipv4: 10.0.0.0/24
    prefix: 32
  p2p:
    ipv4: 10.1.0.0/16
    prefix: 30
    ipv6: true

nodes: [R1, R2, R3]

links:
  - R1-R2
  - R2-R3

Multi Area OSPF

Maybe we want to deploy an OSPF topology with multiple areas. This could be used to get the concepts of IA vs O vs E vs N routes down. This file will deploy R2 in area 0. R1 will be in area 1, but the link between R1 and R2 will be in area 0. R3 will be in area 3, but the link between R2 and R3 will be in area 0.

name: ospf
provider: clab
defaults:
  device: eos
  devices:
    eos.clab.image: ceos:4.27.2F


addressing:
  loopback:
    ipv4: 10.0.0.0/24
    ipv6: 200::/7
  router_id:
    ipv4: 10.0.0.0/24
    prefix: 32
  p2p:
    ipv4: 10.1.0.0/16
    prefix: 30
    ipv6: true

module: [ospf]
ospf:
  area: 0

nodes:
  - name: R1
    ospf:
      area: 1
  - name: R2
  - name: R3
    ospf:
      area: 3

links:
  - R1:
      ospf.area: 0
    R2:
  - R2:
    R3:
      ospf.area: 0

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ netlab up ospf.yml
Created provider configuration file: clab.yml
Created group_vars for all
Created group_vars for eos
Created host_vars for R1
Created host_vars for R2
Created host_vars for R3
Created minimized Ansible inventory hosts.yml
Created Ansible configuration file: ansible.cfg

Step 2: Checking virtualization provider installation
============================================================
.. all tests succeeded, moving on

Step 3: starting the lab
============================================================
INFO[0000] Containerlab v0.23.0 started
INFO[0000] Parsing & checking topology file: clab.yml
INFO[0000] Creating lab directory: /home/juliopdx/repos/cl-nst-demo/clab-ospf
INFO[0000] Creating docker network: Name='clab', IPv4Subnet='172.20.20.0/24', IPv6Subnet='2001:172:20:20::/64', MTU='1500'
INFO[0000] Creating container: R1
INFO[0000] Creating container: R2
INFO[0000] Creating container: R3
INFO[0001] Creating virtual wire: R2:eth2 <--> R3:eth1
INFO[0001] Creating virtual wire: R1:eth1 <--> R2:eth1
INFO[0001] Running postdeploy actions for Arista cEOS 'R3' node
INFO[0001] Running postdeploy actions for Arista cEOS 'R2' node
INFO[0001] Running postdeploy actions for Arista cEOS 'R1' node
INFO[0054] Adding containerlab host entries to /etc/hosts file
+---+--------------+--------------+--------------+------+---------+----------------+----------------------+
| # |     Name     | Container ID |    Image     | Kind |  State  |  IPv4 Address  |     IPv6 Address     |
+---+--------------+--------------+--------------+------+---------+----------------+----------------------+
| 1 | clab-ospf-R1 | ba1c4a47b6cb | ceos:4.27.2F | ceos | running | 172.20.20.3/24 | 2001:172:20:20::3/64 |
| 2 | clab-ospf-R2 | dbda30424bc2 | ceos:4.27.2F | ceos | running | 172.20.20.4/24 | 2001:172:20:20::4/64 |
| 3 | clab-ospf-R3 | 3234e3265e9c | ceos:4.27.2F | ceos | running | 172.20.20.2/24 | 2001:172:20:20::2/64 |
+---+--------------+--------------+--------------+------+---------+----------------+----------------------+

etc...
(venv) juliopdx@containerlab:~/repos/cl-nst-demo$

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ docker exec -it clab-ospf-R1 Cli
R1>en
R1#show ip ospf neighbor
Neighbor ID     Instance VRF      Pri State                  Dead Time   Address         Interface
10.0.0.2        1        default  0   FULL                   00:00:38    10.1.0.2        Ethernet1
R1#show ip route ospf

VRF: default

 O        10.0.0.2/32 [110/20] via 10.1.0.2, Ethernet1
 O IA     10.0.0.3/32 [110/30] via 10.1.0.2, Ethernet1
 O        10.1.0.4/30 [110/20] via 10.1.0.2, Ethernet1

R1#

OSPF Lab for BGP Learning

In this topology, the lab is predeployed with OSPF(area 0) and the students could then focus on the BGP learning.

name: basico
provider: clab
defaults:
  device: eos
  devices:
    eos.clab.image: ceos:4.27.2F


addressing:
  loopback:
    ipv4: 10.0.0.0/24
    ipv6: 200::/7
  router_id:
    ipv4: 10.0.0.0/24
    prefix: 32
  p2p:
    ipv4: 10.1.0.0/16
    prefix: 30
    ipv6: true

module: [ospf]

nodes: [R1, R2, R3]

links:
  - R1-R2
  - R2-R3

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ netlab up basic_ospf.yml
Created provider configuration file: clab.yml
Created group_vars for all
Created group_vars for eos
Created host_vars for R1
Created host_vars for R2
Created host_vars for R3
Created minimized Ansible inventory hosts.yml
Created Ansible configuration file: ansible.cfg

Step 2: Checking virtualization provider installation
============================================================
.. all tests succeeded, moving on

Step 3: starting the lab
============================================================
INFO[0000] Containerlab v0.23.0 started
INFO[0000] Parsing & checking topology file: clab.yml
INFO[0000] Creating lab directory: /home/juliopdx/repos/cl-nst-demo/clab-basico
INFO[0000] Creating docker network: Name='clab', IPv4Subnet='172.20.20.0/24', IPv6Subnet='2001:172:20:20::/64', MTU='1500'
INFO[0000] Creating container: R2
INFO[0000] Creating container: R1
INFO[0000] Creating container: R3
INFO[0001] Creating virtual wire: R1:eth1 <--> R2:eth1
INFO[0001] Creating virtual wire: R2:eth2 <--> R3:eth1
INFO[0001] Running postdeploy actions for Arista cEOS 'R3' node
INFO[0001] Running postdeploy actions for Arista cEOS 'R2' node
INFO[0001] Running postdeploy actions for Arista cEOS 'R1' node
INFO[0055] Adding containerlab host entries to /etc/hosts file
+---+----------------+--------------+--------------+------+---------+----------------+----------------------+
| # |      Name      | Container ID |    Image     | Kind |  State  |  IPv4 Address  |     IPv6 Address     |
+---+----------------+--------------+--------------+------+---------+----------------+----------------------+
| 1 | clab-basico-R1 | f079a253b1d1 | ceos:4.27.2F | ceos | running | 172.20.20.2/24 | 2001:172:20:20::2/64 |
| 2 | clab-basico-R2 | ed28240d8b0f | ceos:4.27.2F | ceos | running | 172.20.20.4/24 | 2001:172:20:20::4/64 |
| 3 | clab-basico-R3 | 0f643cc95fc7 | ceos:4.27.2F | ceos | running | 172.20.20.3/24 | 2001:172:20:20::3/64 |
+---+----------------+--------------+--------------+------+---------+----------------+----------------------+
etc...

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ docker exec -it clab-basico-R1 Cli
R1>en
R1#show ip route ospf

VRF: default

 O        10.0.0.2/32 [110/20] via 10.1.0.2, Ethernet1
 O        10.0.0.3/32 [110/30] via 10.1.0.2, Ethernet1
 O        10.1.0.4/30 [110/20] via 10.1.0.2, Ethernet1

R1#

OSPF and BGP Lab Complete

In this topology, we have a complete OSPF deployment. R1 and R3 will be iBGP peers.

%%{init: {'securityLevel': 'loose', 'theme':'dark'}}%% flowchart TD subgraph OSPF subgraph AS 65000 1((R1)) -.iBGP.- 3((R3)) end 1 --- 2((R2)) 3 --- 2((R2)) end

name: bgp
provider: clab
defaults:
  device: eos
  devices:
    eos.clab.image: ceos:4.27.2F


addressing:
  loopback:
    ipv4: 10.0.0.0/24
    ipv6: 200::/7
  router_id:
    ipv4: 10.0.0.0/24
    prefix: 32
  p2p:
    ipv4: 10.1.0.0/16
    prefix: 30
    ipv6: true

module: [ospf]

nodes:
  - name: R1
    module: [ospf,bgp]
    bgp.as: 65000
  - R2
  - name: R3
    module: [ospf,bgp]
    bgp.as: 65000

links:
  - R1-R2
  - R2-R3

(venv) juliopdx@containerlab:~/repos/cl-nst-demo$ docker exec -it clab-bgp-R1 Cli
R1>en
R1#show ip bgp summary
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 65000
Neighbor Status Codes: m - Under maintenance
  Description              Neighbor V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  R3                       10.0.0.3 4 65000             10        10    0    0 00:05:32 Estab   1      1
R1#show ip bgp | b Network
          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >      10.0.0.1/32            -                     -       -          -       0       i
 * >      10.0.0.3/32            10.0.0.3              0       -          100     0       i
R1#

Outro and Links

Thank you all for reading this far. I really appreciate it! I am really looking forward to using net-sim tools to create future labs. I can already see the speed to create troubleshooting labs will be increased tremendously. Thank you, Ivan and team for all the work on this tool!

Learning Labs With Containerlab

Fri, 28 Jan 2022 15:45:51 -0800

Hello everyone and thank you for checking in. I’m back with another post. To be honest this one is not the norm as it will just redirect you to a project I’ve been working on with Containerlab. I’m still progressing through the Optimal Routing Design book by Cisco Press. I still haven’t finished the OSPF chapter but little by little.

A while back I mentioned that Containerlab could be an amazing resource in learning environments for individuals getting into IT or instructors trying to show students some routing design or technology involved. This could also lead the class to discuss design concepts or why they were chosen, possibly even collaborate with each other. For example, sharing a Containerlab file is incredibly easy. Going through this process has shown me how difficult it really is to create training content. Thank you to everyone who writes blogs, creates training videos, and helps everyone learn no matter where they are in their career.

I wanted to challenge myself and see if I could create something like this myself. I took a bit of inspiration from CS50X and Nick Russo’s amazing OSPF content. This is definitely not on that level or even difficulty. Currently I am on about week 7 of the CS50X course and it is seriously awesome. In that course, students build a script or a project and then automated testing is performed to validate if a project is acceptable. I created an OSPF troubleshooting lab that has a few problems and then automated testing at the end. Check it out on the GitHub link below as well as the other links shared.

Thank you and happy labbing/troubleshooting.

EIGRP Network Design in the Lab

Tue, 25 Jan 2022 15:50:47 -0800

Introduction

I just finished the chapter on EIGRP network design in Optimal Routing Design (Cisco Press), I wanted to provide a high level overview of what I learned and incorporated that in the lab. This book really has been great as it provides advanced routing techniques but keeps it in a maintainable format. I hope you enjoy the post and learn something along the way. I would advise you to research on your own and compare the pros and cons. Like anything, it depends.

The Topology

Lets break down the topology above. Everything is in EIGRP AS 12345. I just chose a random number but for the most part, using multiple autonomous systems in EIGRP does not limit the query scope, so we will use just one. Every link is configured as a /31 point to point to save on address space. We will be using a fairly traditional 3 tier design. The DMZ node is advertising a default route throughout the network and has a few loopbacks representing public IP addresses. Example of that below as well as one core nodes routing table before any optimization is performed.

!dmz node redistributing a default route
router eigrp 12345
 network 10.0.0.11 0.0.0.0
 network 10.0.0.13 0.0.0.0
 redistribute static
 eigrp router-id 0.0.0.12
!
ip route 0.0.0.0 0.0.0.0 Null0

core3#show ip route eigrp | b Gate
Gateway of last resort is 10.0.0.13 to network 0.0.0.0

D*EX  0.0.0.0/0 [170/281600] via 10.0.0.13, 00:01:21, Ethernet0/3
      10.0.0.0/8 is variably subnetted, 54 subnets, 3 masks
D        10.0.0.0/31 [90/307200] via 10.0.0.4, 00:01:21, Ethernet0/1
                     [90/307200] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.0.0.6/31 [90/307200] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.0.0.10/31 [90/307200] via 10.0.0.13, 00:01:21, Ethernet0/3
                      [90/307200] via 10.0.0.4, 00:01:21, Ethernet0/1
D        10.0.0.14/31 [90/307200] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.0.0.16/31 [90/307200] via 10.0.0.4, 00:01:21, Ethernet0/1
D        10.1.2.0/24 [90/332800] via 10.0.0.4, 00:01:21, Ethernet0/1
                     [90/332800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.3.0/24 [90/332800] via 10.0.0.2, 00:30:01, Ethernet0/0
D        10.1.4.0/24 [90/332800] via 10.0.0.2, 00:30:01, Ethernet0/0
D        10.1.5.0/24 [90/332800] via 10.0.0.2, 00:30:01, Ethernet0/0
D        10.1.30.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.31.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.32.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.33.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.34.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.40.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.41.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.42.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.43.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.44.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.50.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.51.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.52.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.53.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.54.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.1.255.253/32 [90/435200] via 10.0.0.2, 00:30:01, Ethernet0/0
D        10.1.255.254/32 [90/435200] via 10.0.0.4, 00:01:21, Ethernet0/1
D        10.2.3.0/24 [90/332800] via 10.0.0.4, 00:01:21, Ethernet0/1
D        10.2.4.0/24 [90/332800] via 10.0.0.4, 00:01:21, Ethernet0/1
D        10.2.5.0/24 [90/332800] via 10.0.0.4, 00:01:21, Ethernet0/1
D        10.2.30.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.31.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.32.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.33.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.34.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.40.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.41.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.42.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.43.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.44.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.50.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.51.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.52.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.53.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.54.0/24 [90/460800] via 10.0.0.4, 00:01:21, Ethernet0/1
                      [90/460800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        10.2.255.253/32 [90/435200] via 10.0.0.2, 00:30:01, Ethernet0/0
D        10.2.255.254/32 [90/435200] via 10.0.0.4, 00:01:21, Ethernet0/1
      172.16.0.0/16 is variably subnetted, 15 subnets, 3 masks
D        172.16.17.0/31 [90/307200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.17.2/31 [90/307200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.17.8/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.17.16/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.17.24/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.17.32/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.17.253/32 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
                          [90/435200] via 10.0.0.2, 00:01:21, Ethernet0/0
D        172.16.17.254/32 [90/409600] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.18.0/31 [90/332800] via 10.0.0.9, 00:01:21, Ethernet0/2
                        [90/332800] via 10.0.0.2, 00:01:21, Ethernet0/0
D        172.16.18.8/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.18.16/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.18.24/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.18.32/29 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
D        172.16.18.253/32 [90/435200] via 10.0.0.9, 00:01:21, Ethernet0/2
                          [90/435200] via 10.0.0.2, 00:01:21, Ethernet0/0
D        172.16.18.254/32 [90/409600] via 10.0.0.9, 00:01:21, Ethernet0/2
core3#

At the moment since no optimizations have been added, if we were to lose any of these links, the network would have to ask every node if they are aware of an alternate path to reach a particular destination. Not very efficient at all. A few places where we can improve are the 10.1.0.0 and 10.2.0.0 networks. Those are all coming from nodes R3, R4, and R5. Think of these as our fake access layer in a campus. The networks in the 172.16.17.0 and 172.16.18.0 networks are coming from a services edge of the network. This could be servers or whatever services you can think of. In this case it is just one node called servers advertising some routes.

Optimizations in the Core Layer

The main idea behind the core is to push the traffic as fast as possible. Try to limit the extra stuffs that can consume resources within the core. Please note that I said within, do summarize from the core to the aggregation layer. How does this look like in practice? From the aggregation standpoint, all we really need from the core is a default route. One way to do this is to configure a summary on the interfaces towards the aggregates.

!core1
interface Ethernet0/2
 ip summary-address eigrp 12345 0.0.0.0 0.0.0.0
!
interface Ethernet0/3
 ip summary-address eigrp 12345 0.0.0.0 0.0.0.0

There’s a catch though. When you create a summary route, EIGRP will add the sweet null0 route… At this point core1 believes it has the better default route over the one the dmz node is sending. For example, lets try and ping the 8.8.8.8 network from svcs1.

svcs1#ping 8.8.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
U.U.U
Success rate is 0 percent (0/5)
svcs1#

We have effectively black holed traffic. So how do we get around this? We make that summary route look terrible so it will not be added into the routing table.

!core1-3
router eigrp 12345
 summary-metric 0.0.0.0/0 distance 255
 network 0.0.0.0

Please note, you will see the advertise all the things command (network 0.0.0.0) in the configurations. In reality, you should be explicit (network 10.1.2.1 0.0.0.0). Lets try that ping again!

svcs1#ping 8.8.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
svcs1#

I’ll add that config and summary metric to all core nodes pointing to their different aggregation layers!

Optimizations in the Aggregation Layer

A bit similar as before, we can advertise summary routes from the aggregation layer to the core. The catch here is that if one of the aggregation nodes loses access to a route, it will still advertise that summary if a route within the summary is present. The solution here is to connect the aggregation nodes either physically or using a tunnel. I decided to run a direct connection for simplicity. Here is what it looks like to summarize the 10.1.0.0/16 and 10.2.0.0/16 networks from dis1 and dis2. It should be noted, the creation of the null0 route for these summaries is not a big deal, the local network will see more specific routes to reach each destination.

!dis1 and dis2
interface Ethernet1/0
 ip summary-address eigrp 12345 10.1.0.0 255.255.0.0
 ip summary-address eigrp 12345 10.2.0.0 255.255.0.0

Similar summaries can be performed from svcs1 and svcs2.

!svcs1 and svcs2
interface Ethernet0/0
 ip summary-address eigrp 12345 172.16.17.0 255.255.255.0
 ip summary-address eigrp 12345 172.16.18.0 255.255.255.0

Lets check the access layer and see how a route table is looking.

R3#  show ip route eigrp | b Gate
Gateway of last resort is 10.2.3.2 to network 0.0.0.0

D*EX  0.0.0.0/0 [170/332800] via 10.2.3.2, 00:17:27, Ethernet0/1
      10.0.0.0/8 is variably subnetted, 55 subnets, 3 masks
D        10.0.0.14/31 [90/307200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.0.0.16/31 [90/307200] via 10.2.3.2, 00:57:32, Ethernet0/1
D        10.1.2.0/24 [90/307200] via 10.2.3.2, 00:57:32, Ethernet0/1
                     [90/307200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.4.0/24 [90/307200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.5.0/24 [90/307200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.40.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.41.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.42.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.43.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.44.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.50.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.51.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.52.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.53.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.54.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.255.253/32 [90/409600] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.1.255.254/32 [90/409600] via 10.2.3.2, 00:57:32, Ethernet0/1
D        10.2.4.0/24 [90/307200] via 10.2.3.2, 00:57:32, Ethernet0/1
D        10.2.5.0/24 [90/307200] via 10.2.3.2, 00:57:32, Ethernet0/1
D        10.2.40.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.41.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.42.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.43.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.44.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.50.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.51.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.52.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.53.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.54.0/24 [90/435200] via 10.2.3.2, 00:57:32, Ethernet0/1
                      [90/435200] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.255.253/32 [90/409600] via 10.1.3.1, 00:57:32, Ethernet0/0
D        10.2.255.254/32 [90/409600] via 10.2.3.2, 00:57:32, Ethernet0/1

This is a bit overkill for the access layer and we can make improvements here as well. We could add another summary command and metric but lets look at another method. We will limit what is advertised to the access layer using the distribute command coupled with a prefix list. Please see example below.

!dis1 and dis2 pointing towards access
!Similar on svcs1 and svcs2
router eigrp 12345
 distribute-list prefix DEFAULT out Ethernet0/1
 distribute-list prefix DEFAULT out Ethernet0/2
 distribute-list prefix DEFAULT out Ethernet0/3
!
ip prefix-list DEFAULT seq 20 permit 0.0.0.0/0

Lets check that routing table again on R3.

R3#show ip route eigrp | b Gate
Gateway of last resort is 10.2.3.2 to network 0.0.0.0

D*EX  0.0.0.0/0 [170/332800] via 10.2.3.2, 00:34:32, Ethernet0/1
R3#

Optimizations in the Access Layer

The amount of filtering and summarizing performed already, we have limited the amount of queries EIGRP will send out to search for alternate paths of a route. Another addition that can be made at the access layer is setting the stub flag. Why use the stub feature? Lets say R4 had a loopback1 network of “10.1.48.1/24”. If we were to shut this interface, then we would see a query from dis1 and dis2 towards R3

R3#debug eigrp packets query
    (QUERY)
EIGRP Packet debugging is on
R3#
*Jan 25 20:48:49.461: EIGRP: Received QUERY on Et0/1 - paklen 44 nbr 10.2.3.2
*Jan 25 20:48:49.461:   AS 12345, Flags 0x0:(NULL), Seq 2628/0 interfaceQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0
*Jan 25 20:48:49.462: EIGRP: Received QUERY on Et0/0 - paklen 44 nbr 10.1.3.1
*Jan 25 20:48:49.462:   AS 12345, Flags 0x0:(NULL), Seq 976/0 interfaceQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0
R3#u all
All possible debugging has been turned off

In this example R3 would not have an alternate path to this route and overall we may not want to use these access layer nodes as a transit network to reach other destinations. These devices may have lower bandwidth or overall resources. The stub feature is a neat way to say “do not query me”. Lets test this out!

!Stub setting added to all access layer nodes
R3#show run | s router
router eigrp 12345
 network 0.0.0.0
 eigrp router-id 0.0.0.6
 eigrp stub connected
R3#
R3#debug eigrp packets query
    (QUERY)
EIGRP Packet debugging is on
R3#

Okay that’s a bit hard to visualize in writing but test it out and you’ll see that no query is sent to R3.

dis1# show ip eigrp neighbors detail
EIGRP-IPv4 Neighbors for AS(12345)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
0   10.1.3.3                Et0/1                    14 00:06:35   11   100  0  712
   Version 18.0/2.0, Retrans: 0, Retries: 0, Prefixes: 11
   Topology-ids from peer - 0
   Stub Peer Advertising (CONNECTED ) Routes
   Suppressing queries

!dmz gets to be special because it is redistributing a static
router eigrp 12345
 network 10.0.0.11 0.0.0.0
 network 10.0.0.13 0.0.0.0
 redistribute static
 eigrp router-id 0.0.0.12
 eigrp stub static

!Almost forgot that core3 routing table!
core3#show ip route eigrp | b Gate
Gateway of last resort is 10.0.0.13 to network 0.0.0.0

D*EX  0.0.0.0/0 [170/281600] via 10.0.0.13, 00:28:48, Ethernet0/3
      10.0.0.0/8 is variably subnetted, 15 subnets, 3 masks
D        10.0.0.0/31 [90/307200] via 10.0.0.4, 02:10:02, Ethernet0/1
                     [90/307200] via 10.0.0.2, 02:10:02, Ethernet0/0
D        10.0.0.6/31 [90/307200] via 10.0.0.2, 02:10:02, Ethernet0/0
D        10.0.0.10/31 [90/307200] via 10.0.0.4, 00:28:52, Ethernet0/1
D        10.0.0.14/31 [90/307200] via 10.0.0.2, 02:10:02, Ethernet0/0
D        10.0.0.16/31 [90/307200] via 10.0.0.4, 02:10:02, Ethernet0/1
D        10.1.0.0/16 [90/435200] via 10.0.0.4, 01:58:15, Ethernet0/1
                     [90/435200] via 10.0.0.2, 01:58:15, Ethernet0/0
D        10.2.0.0/16 [90/435200] via 10.0.0.4, 01:58:15, Ethernet0/1
                     [90/435200] via 10.0.0.2, 01:58:15, Ethernet0/0
      172.16.0.0/24 is subnetted, 2 subnets
D        172.16.17.0 [90/409600] via 10.0.0.9, 01:50:23, Ethernet0/2
D        172.16.18.0 [90/409600] via 10.0.0.9, 01:50:23, Ethernet0/2
core3#

Outro and Links

I originally planned to build this using FRR but it looks like EIGRP is in Alpha stage. I will play with FRR more in the future. Due to this, the topology wont be neatly packaged in a Containerlab file. I will include the configurations below. I hope you all learned something today and thank you for stopping by. We didn’t even go into the new style of configuring EIGRP but I hope you get the point. Named EIGRP just moves some options from the interface level to the af-interface level :). Stay tuned for the next episode of dragon bal…. wait this isn’t that show! In all seriousness, I have something pretty cool planned when we get to OSPF.

Multipoint Redistribution and SR Linux

Sat, 22 Jan 2022 15:25:24 -0800

Introduction

I’ve been working my way through Optimal Routing Design by Russ White, Don Slice, and Alvaro Retana. It has been a great read so far and I highly recommend it. When I work through a book I usually try and lab up any concept I can, this helps me make it stick as much as possible. Early on in the book the authors mention redistribution and possible issues that can come with doing it at more than one point in the network. So here we are reading this post. I hope you enjoy and possibly learn something along the way.

Doubling Up the Learning With SR Linux

I’ve been itching to try out SR Linux for some time now. Roman from Nokia kept giving me the friendly nudge to try it out. well, a few days ago it was finally that day! I wont go deep into SR Linux on this post but most all of the examples will use 8 nodes running on my laptop using Containerlab. Nokia was kind enough to make their NOS container publicly available. You can pull down the image with no registrations or sacrificing an email address to spam. Thank you <3.

Why Redistribute?

Just in case folks reading at home are not familiar with redistribution, I’ll give a high level overview. Lets say you operate a domain that only runs OSPF, some time passes and you need to merge with another domain. Now this domain only runs EIGRP, and for some reason they don’t want to use OSPF. You could point static routes to each other if there is a clean break in IP addresses (unlikely). To make this scale or get around adding a lot of static routes, we can redistribute between OSPF and EIGRP. Essentially letting each domain advertise routes between routing protocols.

Single Point Redistribution

A while back I wrote a piece on redistributing OSPF to EIGRP, check it out here. In single point redistribution, there is little chance of major issues, since there is only one entry and exit point to each domain. Where things get really interesting is when we have multipoint redistribution and that is the focus of this post.

Multipoint Redistribution

If we were to lose the link at the single point redistribution image above, network connectivity to the other domain would be lost. In this case we can redistribute at multiple points in the network.

Multipoint Redistribution Topology

Above we have 8 SR Linux nodes with the left portion of the network running OSPF as well as the right portion of the network. In the middle everything is running eBGP, each router has an autonomous system of 6500X, where X is the router number. R1 is advertising the network of “10.1.1.0/24” and R8 is advertising network “8.8.8.8/32”. Routers R3, R4, R6, and R7 are redistributing routes between the protocols. I added a simple filter to only advertise the network mentioned in each domain. For example, R3 and R4 will only advertise the network “10.1.1.0/24” into BGP. Example configurations of R3 are below. Please note; R3, R4, R6, and R7 have very similar policies.

A:R3# info routing-policy
    routing-policy {
        prefix-set FILTER {
            prefix 10.1.1.0/24 mask-length-range exact {
            }
        }
        policy OSPFtoBGP {
            statement 10 {
                match {
                    prefix-set FILTER
                }
                action {
                    accept {
                    }
                }
            }
        }
        policy pass-all {
            statement 10 {
                match {
                    protocol bgp
                }
                action {
                    accept {
                    }
                }
            }
        }
    }

--{ running }--[ network-instance default protocols ]--
A:R3# info
    bgp {
        autonomous-system 65003
        router-id 0.0.0.3
        group EXTERNAL {
            admin-state enable
            export-policy OSPFtoBGP
            import-policy pass-all
            local-as 65003 {
            }
        }
        neighbor 10.3.5.1 {
            peer-as 65005
            peer-group EXTERNAL
        }
    }
    ospf {
        instance v2 {
            admin-state enable
            version ospf-v2
            router-id 0.0.0.3
            export-policy pass-all
            asbr {
            }
            area 0.0.0.0 {
                interface ethernet-1/1.0 {
                    interface-type point-to-point
                }
            }
        }
    }

Checking Routes at R5

You probably noticed R5 is at the center of this mess, so lets check the routing table from that perspective. Remember R5 is only running BGP.

A:R5# show network-instance default protocols bgp routes ipv4 summary
-------------------------------------------------------------------------------------------------------------------------------
Show report for the BGP route table of network-instance "default"
-------------------------------------------------------------------------------------------------------------------------------
Status codes: u=used, *=valid, >=best, x=stale
Origin codes: i=IGP, e=EGP, ?=incomplete
-------------------------------------------------------------------------------------------------------------------------------
+-----+--------------+---------------------+-----+-----+----------------------------------------+
| Sta |   Network    |      Next Hop       | MED | Loc |                Path Val                |
| tus |              |                     |     | Pre |                                        |
|     |              |                     |     |  f  |                                        |
+=====+==============+=====================+=====+=====+========================================+
| u*> | 8.8.8.8/32   | 10.5.6.1            | 16  | 100 | [65006] i                              |
| *   | 8.8.8.8/32   | 10.5.7.1            | 16  | 100 | [65007] i                              |
| u*> | 10.1.1.0/24  | 10.3.5.0            | 32  | 100 | [65003] i                              |
| *   | 10.1.1.0/24  | 10.4.5.0            | 32  | 100 | [65004] i                              |
| u*> | 10.3.5.0/31  | 0.0.0.0             | -   | 100 |  i                                     |
| u*> | 10.4.5.0/31  | 0.0.0.0             | -   | 100 |  i                                     |
| u*> | 10.5.6.0/31  | 0.0.0.0             | -   | 100 |  i                                     |
| u*> | 10.5.7.0/31  | 0.0.0.0             | -   | 100 |  i                                     |
+-----+--------------+---------------------+-----+-----+----------------------------------------+
-------------------------------------------------------------------------------------------------------------------------------
8 received BGP routes: 6 used, 8 valid, 0 stale
6 available destinations: 2 with ECMP multipaths
-------------------------------------------------------------------------------------------------------------------------------
--{ running }--[  ]--
A:R5#

So far this looks great, R5 learned about the “8.8.8.8/32” network from R6(10.5.6.1) and R7(10.5.7.1). We can see that R5 also learned about the “10.1.1.0/24” network from R3(10.3.5.0) and R4(10.4.5.0). Lets see if R1 can reach the remote network!

R1 Reachability to 8.8.8.8/32

A:R1# ping network-instance default -c 4 -I 10.1.1.1 8.8.8.8
Using network instance default
PING 8.8.8.8 (8.8.8.8) from 10.1.1.1 : 56(84) bytes of data.
From 10.2.3.1 icmp_seq=1 Time to live exceeded
From 10.2.3.1 icmp_seq=2 Time to live exceeded
From 10.2.3.1 icmp_seq=3 Time to live exceeded

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3004ms

--{ running }--[  ]--
A:R1#

Hmm, that doesn’t look good. Lets try a trace and maybe we can see whats going on!

A:R1# traceroute network-instance default 8.8.8.8
Using network instance default
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  10.1.2.1 (10.1.2.1)  15.831 ms * *
 2  10.2.3.1 (10.2.3.1)  12.367 ms * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * 10.2.3.1 (10.2.3.1)  15.266 ms *
 9  10.2.3.0 (10.2.3.0)  14.912 ms * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * 10.2.3.0 (10.2.3.0)  12.849 ms *
16  10.2.3.1 (10.2.3.1)  18.093 ms * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * 10.2.3.1 (10.2.3.1)  7.931 ms *
23  10.2.3.0 (10.2.3.0)  5.755 ms * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * 10.2.3.0 (10.2.3.0)  24.179 ms *
30  10.2.3.1 (10.2.3.1)  15.856 ms * *
--{ running }--[  ]--
A:R1#

It seems like we reached the max hops, by default 30. At this point, we can most likely come to the conclusion that we have a routing loop. Lets check a few routes and see what they think the best path to reach the “8.8.8.8/32” network is.

# Nothing crazy here, R1 will send traffic towards R2(10.1.2.1)
A:R1# show network-instance default route-table ipv4-unicast prefix 8.8.8.8/32 detail
-------------------------------------------------------------------------------------------------------------------------------
IPv4 unicast route table of network instance default
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : ospfv2
Route Owner   : ospf_mgr
Metric        : 1
Preference    : 150
Best          : true
Last change   : 2022-01-22T18:50:05.915Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.1.2.1 (direct) via [ethernet-1/1.0]
-------------------------------------------------------------------------------------------------------------------------------
--{ running }--[  ]--
A:R1#

# Again nothing weird, R2 will go to R3(10.2.3.1) to reach this route
A:R2# show network-instance default route-table ipv4-unicast prefix 8.8.8.8/32 detail
-------------------------------------------------------------------------------------------------------------------------------
IPv4 unicast route table of network instance default
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : ospfv2
Route Owner   : ospf_mgr
Metric        : 1
Preference    : 150
Best          : true
Last change   : 2022-01-22T18:50:05.930Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.2.3.1 (direct) via [ethernet-1/2.0]
-------------------------------------------------------------------------------------------------------------------------------
--{ running }--[  ]--
A:R2#

# Thats odd, R3 is sending the route back to R2(10.2.3.0)
A:R3# show network-instance default route-table ipv4-unicast prefix 8.8.8.8/32 detail
-------------------------------------------------------------------------------------------------------------------------------
IPv4 unicast route table of network instance default
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : ospfv2
Route Owner   : ospf_mgr
Metric        : 1
Preference    : 150
Best          : true
Last change   : 2022-01-22T18:50:05.910Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.2.3.0 (direct) via [ethernet-1/1.0]
-------------------------------------------------------------------------------------------------------------------------------
--{ running }--[  ]--
A:R3#

This manual validation has shown us that the path R1 will take to reach “8.8.8.8/32” is from R1 -> R2 -> R3 -> R2, a routing loop!

The problem

When we have multiple points of redistribution there is a chance that loops can be introduced into the network. The reason is that routes advertised into one side of the network can be redistributed right back to our domain, possibly with a better metric. This will then have the effect of a router preferring a local path to a remote network vs using the valid path towards R5.

The Solutions

There are a few ways to solve this. Some of the more popular ones being filtering, route tagging, and changing administrative distances/preferences. I’ll give an example of tagging later on in this post. Lets see if we can solve this with filtering and changing the preferences first. The goal being, eBGP routes to be preferred over external OSPF routes.

Solve With Filtering and Preference

We mentioned before that we are only sending “10.1.1.0/24” into BGP and “8.8.8.8/32” for the other end of the network. We can add a filter to only allow the remote network from BGP into OSPF. Example of configuration below!

A:R3# info routing-policy
    routing-policy {
        prefix-set ALLOW {
            prefix 8.8.8.8/32 mask-length-range exact {
            }
        }
        policy BGPtoOSPF {
            default-action {
                reject {
                }
            }
            statement 10 {
                match {
                    prefix-set ALLOW
                }
                action {
                    accept {
                    }
                }
            }
        }

We added a default action reject statement on the policy used to redistribute routes from BGP into OSPF. This will basically deny any routes not matching this policy. Similar statement was added to each router running redistribution. Lets check the connectivity from R3 and R4 standpoint.

--{ + running }--[ network-instance default ]--
A:R3# show route-table ipv4-unicast prefix 8.8.8.8/32 detail
-------------------------------------------------------------------------------------------------------------------------------
IPv4 unicast route table of network instance default
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : ospfv2
Route Owner   : ospf_mgr
Metric        : 1
Preference    : 150
Best          : true
Last change   : 2022-01-22T21:37:23.962Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.2.3.0 (direct) via [ethernet-1/1.0]
-------------------------------------------------------------------------------------------------------------------------------
--{ + running }--[ network-instance default ]--
A:R3#

Interesting, R3 is preferring a route to “8.8.8.8/32” with a preference of 150 towards R2. What about R4?

--{ + running }--[ network-instance default ]--
A:R4# show route-table ipv4-unicast prefix 8.8.8.8/32 detail
-------------------------------------------------------------------------------------------------------------------------------
IPv4 unicast route table of network instance default
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : ospfv2
Route Owner   : ospf_mgr
Metric        : 1
Preference    : 150
Best          : true
Last change   : 2022-01-22T21:37:42.270Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.2.4.0 (direct) via [ethernet-1/1.0]
-------------------------------------------------------------------------------------------------------------------------------
--{ + running }--[ network-instance default ]--
A:R4#

R4 is also preferring a route towards R2. At this point it seems like a race condition on who wins the route, BGP will see an internal route to reach that network and pick that over an eBGP route. Lets update the preference for external OSPF routes to be worse than 170. Why 170? Lets shutdown OSPF on R3 and see what route R4 uses to reach “8.8.8.8/32”.

--{ + running }--[ network-instance default ]--
A:R4# show route-table ipv4-unicast prefix 8.8.8.8/32 detail
-------------------------------------------------------------------------------------------------------------------------------
IPv4 unicast route table of network instance default
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : bgp
Route Owner   : bgp_mgr
Metric        : 0
Preference    : 170
Best          : true
Last change   : 2022-01-22T21:45:53.701Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.4.5.1 (indirect) resolved by route to 10.4.5.0/31 (local)
  via 10.4.5.0 (direct) via [ethernet-1/2.0]
-------------------------------------------------------------------------------------------------------------------------------
--{ + running }--[ network-instance default ]--
A:R4#

We will change the external OSPF preference to be 175 and test if this allows R3 and R4 to prefer the BGP route.

--{ +* candidate shared default }--[ network-instance default protocols ospf ]--
A:R3# info
    instance v2 {
        admin-state disable
        version ospf-v2
        router-id 0.0.0.3
        export-policy BGPtoOSPF
        external-preference 175
        asbr {
        }
        area 0.0.0.0 {
            interface ethernet-1/1.0 {
                interface-type point-to-point
            }
        }
    }

We can see below that R3 now prefers the BGP route to reach “8.8.8.8/32”!

--{ + running }--[ network-instance default ]--
A:R3# show route-table ipv4-unicast prefix 8.8.8.8/32 detail
-------------------------------------------------------------------------------------------------------------------------------
IPv4 unicast route table of network instance default
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : bgp
Route Owner   : bgp_mgr
Metric        : 0
Preference    : 170
Best          : true
Last change   : 2022-01-22T21:45:52.736Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.3.5.1 (indirect) resolved by route to 10.3.5.0/31 (local)
  via 10.3.5.0 (direct) via [ethernet-1/2.0]
-------------------------------------------------------------------------------------------------------------------------------
Destination   : 8.8.8.8/32
ID            : 0
Route Type    : ospfv2
Route Owner   : ospf_mgr
Metric        : 1
Preference    : 175
Best          : false
Last change   : 2022-01-22T21:57:57.628Z
Resilient hash: false
-------------------------------------------------------------------------------------------------------------------------------
Next hops: 1 entries
10.2.3.0 (direct) via [ethernet-1/1.0]
-------------------------------------------------------------------------------------------------------------------------------
--{ + running }--[ network-instance default ]--
A:R3#

The rest of the redistribution nodes have been configured to match these preferences. Lets test connectivity from R1 again!

Validating from R1

A:R1# ping network-instance default -c 4 -I 10.1.1.1 8.8.8.8
Using network instance default
PING 8.8.8.8 (8.8.8.8) from 10.1.1.1 : 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=60 time=6.39 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=60 time=16.7 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=60 time=15.4 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=60 time=17.4 ms

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 6.392/13.973/17.422/4.435 ms
--{ running }--[  ]--
A:R1# traceroute network-instance default  8.8.8.8
Using network instance default
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  10.1.2.1 (10.1.2.1)  4.861 ms * *
 2  10.2.3.1 (10.2.3.1)  4.752 ms * *
 3  10.3.5.1 (10.3.5.1)  8.857 ms * *
 4  10.5.6.1 (10.5.6.1)  8.739 ms * *
 5  8.8.8.8 (8.8.8.8)  12.996 ms  12.990 ms  12.971 ms
--{ running }--[  ]--
A:R1#

Solve with Route Tags

The solution above works but route tags would definitely scale more. A route tag is just a numerical value that can be assigned to routes. The strength comes in when you can query those tags to perform redistribution or policy enforcement. I could not find documentation for tagging routes using SR Linux at the time of this writing, not a knock on them I’m just very new to the NOS. In this case I’ll show some example from Cisco IOS and a high level diagram.

Bear with me, I know SR Linux does not run EIGRP :). Here we have the same topology but Cisco nodes are in place. From R3’s perspective, when redistributing routes from OSPF to EIGRP, R3 will deny any routes tagged with 444 and set any routes not matching that to 333. In the opposite direction, routes from EIGRP to OSPF with a tag of 444 will be denied, and anything left will be tagged with 333. R4 essentially does the reverse to make sure anything from 333 is not sent to EIGRP or OSPF. Example configurations below.

!R3 Config
router eigrp 34567
 default-metric 1 1 1 1 1
 network 10.3.5.0 0.0.0.0
 redistribute ospf 1 route-map filter-tags
!
router ospf 1
 router-id 0.0.0.3
 redistribute eigrp 34567 subnets route-map filter-tags
 network 10.2.3.1 0.0.0.0 area 0
!
route-map filter-tags deny 10
 match tag 444
!
route-map filter-tags permit 20
 set tag 333

!R4 Config
router eigrp 34567
 default-metric 1 1 1 1 1
 network 10.4.5.0 0.0.0.0
 redistribute ospf 1 route-map filter-tags
!
router ospf 1
 router-id 0.0.0.4
 redistribute eigrp 34567 subnets route-map filter-tags
 network 10.2.4.1 0.0.0.0 area 0
!
route-map filter-tags deny 10
 match tag 333
!
route-map filter-tags permit 20
 set tag 444

R5#show ip route 10.1.1.0
Routing entry for 10.1.1.0/24
  Known via "eigrp 34567", distance 170, metric 2560025856
  Tag 444, type external
  Redistributing via eigrp 34567
  Last update from 10.4.5.0 on Ethernet0/1, 00:11:08 ago
  Routing Descriptor Blocks:
  * 10.4.5.0, from 10.4.5.0, 00:11:08 ago, via Ethernet0/1
      Route metric is 2560025856, traffic share count is 1
      Total delay is 1010 microseconds, minimum bandwidth is 1 Kbit
      Reliability 1/255, minimum MTU 1 bytes
      Loading 1/255, Hops 1
      Route tag 444
    10.3.5.0, from 10.3.5.0, 00:11:08 ago, via Ethernet0/0
      Route metric is 2560025856, traffic share count is 1
      Total delay is 1010 microseconds, minimum bandwidth is 1 Kbit
      Reliability 1/255, minimum MTU 1 bytes
      Loading 1/255, Hops 1
      Route tag 333
R5#

Here we can see the “10.1.1.0” network from each node with the appropriate tag.

Validating from R1 Again

R1#ping 8.8.8.8 source 10.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of 10.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/3 ms
R1#traceroute 8.8.8.8 source 10.1.1.1
Type escape sequence to abort.
Tracing the route to 8.8.8.8
VRF info: (vrf in name/id, vrf out name/id)
  1 10.1.2.1 1 msec 1 msec 1 msec
  2 10.2.3.1 1 msec 0 msec 1 msec
  3 10.3.5.1 2 msec 1 msec 1 msec
  4 10.5.6.1 1 msec 2 msec 1 msec
  5 10.6.8.1 2 msec *  3 msec
R1#

Outro and Links

Thank you all for reading this far, seriously means a lot to me. I hope you found something useful or learned a bit along the way. If I have made any errors in the writing, please let me know! The goal is definitely to be technically accurate. Another option, grab the Github repository where this lab is stored (linked below) and run a pull request! I created a Containerlab repository for this lab and probably future labs I make with Containerlab. Happy labbing and thank you for stopping by!

Learning a Little About Search Algorithms With Python

Wed, 12 Jan 2022 15:12:30 -0800

Introduction

Hello and thank you for tuning in to another episode. First off, I was recently notified that this blog made it to the Cisco IT Blog Awards final! Pretty wild to think about, but thank you to all those involved. This one is definitely going to be different and out of my comfort zone and I appreciate you sticking with me.

I have been working through the Introduction to Computer Science course by CS50x (Harvard). The course does an incredible job of walking learners through the very basics of programming with Scratch and the C language. The course eventually shifts over to Python for the remainder of the course. While taking the course, you learn about arrays, algorithms, memory, and more. David Malan and team really do an incredible job and try to make the barrier of entry as low as possible (free and includes a web based VScode editor).

In week three we learn a bit about algorithms. This was easily one of my favorite weeks. The class walks through different search and sorting algorithms. These are then compared performance wise. This post will go over two search algorithms; linear and binary. I will use some subnets from the 10.0.0.0/8 network for our examples.

Creating the Data

I decided to create a big list of subnets with some Python. The code below will produce three files that can be used for testing.

ip_create.py

"""Small script to create test files"""
import ipaddress
import random

net = ipaddress.ip_network("10.0.0.0/8").subnets(new_prefix=24)

networks = [str(n.network_address) for n in net]

# Ordered list of subnets
with open("ordered_ip.txt", "w", encoding="utf-8") as file:
    for net in networks:
        file.write(f"{net}\n")

# Reversed list of subnets
networks.reverse()
with open("reverse_ip.txt", "w", encoding="utf-8") as file:
    for net in networks:
        file.write(f"{net}\n")

# Random list of subnets
random.shuffle(networks)
with open("random_ip.txt", "w", encoding="utf-8") as file:
    for net in networks:
        file.write(f"{net}\n")

ordered_ip.txt

10.0.0.0
10.0.1.0
10.0.2.0
10.0.3.0
10.0.4.0
10.0.5.0
10.0.6.0
10.0.7.0
10.0.8.0
10.0.9.0
...

random_ip.txt

10.44.148.0
10.47.64.0
10.166.151.0
10.139.81.0
10.237.230.0
10.139.3.0
10.179.165.0
10.96.37.0
10.26.247.0
10.12.255.0
...

reverse_ip.txt

10.255.255.0
10.255.254.0
10.255.253.0
10.255.252.0
10.255.251.0
10.255.250.0
10.255.249.0
10.255.248.0
10.255.247.0
...

Linear Search

Linear search is fairly simple in nature. We take a list or array and check one by one until we find our desired value.

Value: 10 20 30 40 50 60 70
Index:  0  1  2  3  4  5  6

For example, using the data above. If we wanted to check for the value of 70, we would first have to check 10 then 20 then 30 and so on. This means it would take us “n” steps. Think of “n” as the size of the array. In our example the array is of size 7. Worst case scenario would be 7 checks. This can then be referenced as “O(n)” or O of n. Now what about the best case scenario? Well, if we are looking for the 10 in this array, then it only requires one step. This is known as “Ω(1)” or Omega of one.

Binary Search

Binary search works a bit differently than linear search. Binary search does have one additional requirement, the array must be sorted. If not, the algorithm breaks. Binary search works by continuously making the size of the array smaller. Essentially making the search size smaller on each iteration.

# First Check
Value: 10 20 30 40 50 60 70
Index:  0  1  2  3  4  5  6
Start: 0
End: 6
Midpoint: 3

Same array as above but with additional information. We have the start of the array at 0 and the end of the array at 6. The midpoint is found by (start + end) / 2. In this case we are looking for 70. The midpoint(40) is lower than our target(70) so the start will shift to midpoint + 1.

# Second Check
Value: 50 60 70
Index:  4  5  6
Start: 4
End: 6
Midpoint: 5

On the second check we are searching through a smaller piece of the array. Midpoint is found by (6 + 4) / 2 = 5. The target is higher than our midpoint of 70. So again we will shift the start to midpoint (5) + 1.

# Third Check
Value: 70
Index:  6
Start: 6
End: 6
Midpoint: 6

# Value is Found

On this third check the midpoint is equal to our target, the value is found! This will also work in the opposite, but we will decrease the end to (midpoint – 1). Whats the big deal? In binary search, when the size of the array increases, the amount of steps will not grow as large as linear search. Making the algorithm much more efficient. Best case scenario, on the initial midpoint we find our target. This is similar to linear search, Ω(1). What about worst case? That is known as O(log n). Long story short, much faster.

Comparing With Subnets

We will be working with subnets as our data. To make things more simple, I will convert these to their decimal equivalent. What do I mean by this? Please see example below!

>>> from ipaddress import ip_address
>>> int(ip_address("0.0.0.1"))
1
>>> int(ip_address("0.0.1.0"))
256
>>>

When I run the comparisons, in reality they will be comparing integer values vs strings of subnets.

Linear Seach in Code

with open(sys.argv[1], "r", encoding="utf-8") as file:
    data = [int(ip_address(line.strip())) for line in file.readlines()]

my_target = int(
    (ip_address(input("What /24 prefix in 10.0.0.0/8 are you looking for? ")))
)

for prefix in data:
    time.sleep(DELAY)
    if prefix == my_target:
        print(f"Found: {ip_address(my_target)} with linear search")
        break

Initially we open one of the text files and use some list comprehension to create our list of integers that represent different subnets. Then we ask the user to enter a subnet, for example, “10.80.40.0”. We will then compare each prefix or subnet against our target. If we find it, we will convert the integer back to a cleaner format. You may have caught the sleep setting in the code. Well it turns out computers are really fast, so we will add some artificial delay. This is used in both search algorithms.

Binary Search in Code

def bin_search(some_array: list, target: int):
    """Simple binary search function"""
    array = sorted(some_array)
    start = 0
    end = len(array) - 1
    found = False
    while not found and start <= end:
        time.sleep(DELAY)
        middle = int((start + end) / 2)
        if int(array[middle]) == target:
            found = True
            print(f"Found: {ip_address(target)} with binary search")
        elif int(array[middle]) > target:
            end = middle - 1
        else:
            start = middle + 1

I wrapped this in a function and set it to accept two parameters, some array and a target. We will first sort the array since that is required in binary search. We then set our start and end values. Once that is done, we will run a while loop and keep making the search smaller and smaller until the target is found. Lets check out some examples!

Examples

# Best case scenario for Linear
❯ python algo.py ordered_ip.txt
What /24 prefix in 10.0.0.0/8 are you looking for? 10.0.0.0
Found: 10.0.0.0 with linear search
Elapsed time: 0.0801 seconds
Found: 10.0.0.0 with binary search
Elapsed time: 0.0045 seconds

# Worst case scenario for Linear
# Binary did not increase much at all!
❯ python algo.py ordered_ip.txt
What /24 prefix in 10.0.0.0/8 are you looking for? 10.255.255.0
Found: 10.255.255.0 with linear search
Elapsed time: 11.0045 seconds
Found: 10.255.255.0 with binary search
Elapsed time: 0.0050 seconds

# Random list of subnets
❯ python algo.py random_ip.txt
What /24 prefix in 10.0.0.0/8 are you looking for? 10.80.40.0
Found: 10.80.40.0 with linear search
Elapsed time: 10.6104 seconds
Found: 10.80.40.0 with binary search
Elapsed time: 0.0159 seconds

Outro and Links

Thank you all for reading this far, it really means a lot and please go vote for your favorite IT blogs.

My Journey and Experience With Containerlab

Fri, 10 Dec 2021 14:32:22 -0800

Introduction

Thank you all for checking out this post. This is going to be a special one. Due to the fact that I had such a joy using this new tool (new to me). I don’t like to over hype tools or new things just to hype it up. There are times where a tool or technology does deserve the credit and time in the limelight. I think Containerlabs is one of those tools.

Where are we?

Before I get started on defining Containerlab, I think its important to mention where we have come from in the networking industry. In the past when a learner or fellow engineer wanted to test a technology, design, or topology, they would have to acquire physical gear. This would take up a lot of space, power, and can you imagine the cost back in the day? I say back in the day like it was 30 years ago. More like less than 10.

The next phase added virtual versions of network operating systems, these could then be added to some hypervisor and stand up a virtual lab. Much more efficient but still required a lot of steps to get going and now required a server or some machine to host all the virtual machines. In the last few years, tools have sprung up to make this process even more simple (EVE-NG, GNS3, CML2).

I don’t want to knock these tools as I still think they are pretty incredible and provide a lot of useful features. Like anything in tech, there are trade offs. With any of these tools, there is still a requirement to have a VM or workstation available that can run said tool and then run your NOS images. For the most part these can be CPU and memory intensive, depending on the image. The process of adding the individual images can be a pain as well depending on the tool itself.

Why Containerlab?

So this is where Containerlab comes in. If you read my previous CI/CD blog series, I mentioned how quick and easy it was to get applications going by just running a packaged Docker container. This could be Drone, Batfish, or Suzeiq. Imagine that simplicity but for networking images. Containerlab was created to simplify the process of creating networking topologies within a Docker environment. I’ll do my best to walk you through some of the features of Containerlab and hopefully you walk away as impressed as I was.

Getting Started With Containerlab

One of the main requirements of Containerlab is to have an environment with Linux and Docker available. In my case, I am running a Linux laptop running PopOS. For instructions on getting Docker installed, please see this example by the team at Digital Ocean.

Installing Containerlab

# download and install the latest release (may require sudo)
bash -c "$(curl -sL https://get-clab.srlinux.dev)"

How to Add Networking Images to Containerlab

At this time there are a few network vendors that make acquiring container images for their NOS simple. Nokia (SR Linux) is a great example of that. In their case, you can just pull the image from a public repository to make it available to Containerlab. Others may require you to sign up for some guest account in their portal to get access to images. There are some that make you pick up the old telephone and ask for access, and others that don’t make any of these an option. To all vendors “please make container images of your NOS and make it publicly available”. I will walk through the process of adding an Arista cEOS image to be used with Containerlab.

Create Arista account, image access should be available right after verifying
Head to software downloads

Now that the image is downloaded, we need to make it available to Docker and Containerlab.

❯ docker import cEOS-lab-4.27.0F.tar ceos:4.27.0F
❯ docker images
REPOSITORY                        TAG            IMAGE ID       CREATED          SIZE
ceos                              4.27.0F        b7119ccccbcd   27 seconds ago   1.68GB

The Topology Definition File

Now that we have a NOS container image available, we can start the initial process of standing up a lab topology. In Containerlab, this is known as the clab file. In the root on my current directory, I will create a file called net.clab.yaml. Example of my file is below. I will break down each portion in a bit.

name: cicd
prefix: ""

mgmt:
  network: statics
  ipv4_subnet: 172.100.100.0/24

topology:
  kinds:
    ceos:
      image: ceos:4.27.0F
    linux:
      image: ghcr.io/hellt/network-multitool
  nodes:
    pdx-rtr-eos-01:
      kind: ceos
      mgmt_ipv4: 172.100.100.11
    pdx-rtr-eos-02:
      kind: ceos
      mgmt_ipv4: 172.100.100.12
    pdx-rtr-eos-03:
      kind: ceos
      mgmt_ipv4: 172.100.100.13
    pdx-rtr-eos-04:
      kind: ceos
      image: ceos:4.27.0F
      mgmt_ipv4: 172.100.100.14
    client1:
      kind: linux
      mgmt_ipv4: 172.100.100.21
    client2:
      kind: linux
      mgmt_ipv4: 172.100.100.22
  links:
    - endpoints: ["pdx-rtr-eos-01:eth1", "pdx-rtr-eos-02:eth1"]
    - endpoints: ["pdx-rtr-eos-02:eth2", "pdx-rtr-eos-03:eth1"]
    - endpoints: ["pdx-rtr-eos-03:eth2", "pdx-rtr-eos-04:eth1"]
    # Client connections
    - endpoints: ["client1:eth1", "pdx-rtr-eos-01:eth2"]
    - endpoints: ["client2:eth1", "pdx-rtr-eos-04:eth2"]

name: cicd
prefix: ""

All labs must have a name at the top of the tree. This is great if you are deploying multiple labs and will prevent any conflict between them. By default lab nodes will be named using the following structure: clab-{{lab-name}}-{{node-name}}. For my example, rtr-01 would have the name of clab-cicd-pdx-rtr-eos-01. I am setting the prefix to “”, this will remove the prefix of clab-cicd on each node. That is more of a preference and not required.

mgmt:
  network: statics
  ipv4_subnet: 172.100.100.0/24

Here we are telling Containerlab to stand up a Docker network named statics that can then be assigned to our nodes.

topology:
  kinds:
    ceos:
      image: ceos:4.27.0F
    linux:
      image: ghcr.io/hellt/network-multitool

This is something I thought was really useful. Containerlab works by defining nodes with what kind of node it is and what image will be used for that node. The kind can be something like ceos, linux, crpd, and others. The documentation does an incredible job of breaking it all down. I will link the docs at the end of this post. In this example I am setting the kind of “ceos.image = ceos:4.27.0F”. This helps remove a lot of duplication when building out topology files with similar nodes.

  nodes:
    pdx-rtr-eos-01:
      kind: ceos
      mgmt_ipv4: 172.100.100.11
    pdx-rtr-eos-02:
      kind: ceos
      mgmt_ipv4: 172.100.100.12
    pdx-rtr-eos-03:
      kind: ceos
      mgmt_ipv4: 172.100.100.13
    pdx-rtr-eos-04:
      kind: ceos
      image: ceos:4.27.0F
      mgmt_ipv4: 172.100.100.14
    client1:
      kind: linux
      mgmt_ipv4: 172.100.100.21
    client2:
      kind: linux
      mgmt_ipv4: 172.100.100.22

This is where a lot of what has been defined is tied together. I will break down the first eos node and client node. Here, pdx-rtr-eos-01, is assigned the kind of ceos and will inherit the image of “ceos:4.27.0F”. This node will then be accessible either by IP address at 172.100.100.11 or by hostname or by the docker exec commands. I will show these examples in a bit. The client image is similar, it is a kind of linux and will inherit the image of “ghcr.io/hellt/network-multitool”, a publicly available image. If you notice I never had to pull down the image myself. It was handled by Containerlab.

  links:
    - endpoints: ["pdx-rtr-eos-01:eth1", "pdx-rtr-eos-02:eth1"]
    - endpoints: ["pdx-rtr-eos-02:eth2", "pdx-rtr-eos-03:eth1"]
    - endpoints: ["pdx-rtr-eos-03:eth2", "pdx-rtr-eos-04:eth1"]
    # Client connections
    - endpoints: ["client1:eth1", "pdx-rtr-eos-01:eth2"]
    - endpoints: ["client2:eth1", "pdx-rtr-eos-04:eth2"]

The links portion is fairly simple but it does require some planning on the front end. The link numbering is defined in the Containerlab documentation and even the examples helped me create this links format. It looks fairly simple but the magic going on behind the scenes is awesome.

Deploying a Topology

I may have lost you at this point but please bear with me. Lets get to the fun stuff.

❯ time sudo containerlab deploy -t net.clab.yaml
INFO[0000] Parsing & checking topology file: net.clab.yaml
INFO[0000] Creating lab directory: /home/juliopdx/git/gcl/cicd
INFO[0000] Creating docker network: Name='statics', IPv4Subnet='172.100.100.0/24', IPv6Subnet='', MTU='1500'
INFO[0000] Creating container: client1
INFO[0000] Creating container: client2
INFO[0000] Creating container: pdx-rtr-eos-04
INFO[0000] Creating container: pdx-rtr-eos-03
INFO[0000] Creating container: pdx-rtr-eos-01
INFO[0000] Creating container: pdx-rtr-eos-02
INFO[0001] Creating virtual wire: client2:eth1 <--> pdx-rtr-eos-04:eth2
INFO[0001] Creating virtual wire: pdx-rtr-eos-03:eth2 <--> pdx-rtr-eos-04:eth1
INFO[0001] Creating virtual wire: client1:eth1 <--> pdx-rtr-eos-01:eth2
INFO[0001] Creating virtual wire: pdx-rtr-eos-01:eth1 <--> pdx-rtr-eos-02:eth1
INFO[0001] Creating virtual wire: pdx-rtr-eos-02:eth2 <--> pdx-rtr-eos-03:eth1
INFO[0001] Running postdeploy actions for Arista cEOS 'pdx-rtr-eos-03' node
INFO[0001] Running postdeploy actions for Arista cEOS 'pdx-rtr-eos-04' node
INFO[0001] Running postdeploy actions for Arista cEOS 'pdx-rtr-eos-01' node
INFO[0001] Running postdeploy actions for Arista cEOS 'pdx-rtr-eos-02' node
INFO[0056] Adding containerlab host entries to /etc/hosts file

Lets walk through this a bit. Containerlab will spin up a directory named after our lab name, cicd. It will then create any management networks we have defined, in our case this will be the 172.100.100.0/24 network. After that each container is created and virtual links added. At the tail end you may have noticed that the Containerlab hosts were added to the /etc/hosts file. Containerlab is nice enough to provide a summary of everything that was deployed. Example below.

At the bottom you can see that it took 57 seconds to deploy the 4 cEOS containers and two Linux containers that will act as hosts. In theory we should be able to connect to the nodes?

Connecting to Nodes

Great, now we have everything deployed but we need to do something with it. If folks are testing or learning a NOS, naturally we want to connect to the nodes to start laying down some sweet configuration. Luckily, we have a few options to connect to each node.

❯ ssh admin@cicd-pdx-rtr-eos-01
Password:
pdx-rtr-eos-01>en
pdx-rtr-eos-01#show version | inc image version
Software image version: 4.27.0F-24305004.4270F (engineering build)
pdx-rtr-eos-01#

❯ ssh admin@172.100.100.11
Password:
pdx-rtr-eos-01>en
pdx-rtr-eos-01#show version | inc image version
Software image version: 4.27.0F-24305004.4270F (engineering build)
pdx-rtr-eos-01#

❯ docker exec -it cicd-pdx-rtr-eos-01 Cli
pdx-rtr-eos-01>en
pdx-rtr-eos-01#show version | inc image version
Software image version: 4.27.0F-24305004.4270F (engineering build)
pdx-rtr-eos-01#

Viewing the Topology

I’m so used to using tools like GNS3 and EVE-NG. Both of these tools provide some type of GUI to view the current topology. Maybe its because I’m a visual person, but I really like to view the topology. Can you believe with a bit of digging… I found out that Containerlab can render a topology image. Example below. FYI, I don’t think the graph enjoyed all the dashes in my hostnames. I’ll post the original when I found this feature!

❯ sudo containerlab graph -t net.clab.yaml
INFO[0000] Parsing & checking topology file: net.clab.yaml
INFO[0000] Listening on :50080...

Above is the original image I generated after I found this out. Seriously awesome! I would like to see the interfaces displayed though… I know I know, we had to define this in the links so who cares? I care! I want to see them in the generated image! 🙂

Automating Some Configuration

A while back I wrote a series on building a CI/CD pipeline. In that series, I used a generic topology with four Arista nodes running OSPF and BGP. I wanted to see if I could recreate this topology in Containerlab and utilize some of the automation skills I have been practicing. I wont go all in on this post but I will drop some configuration with Nornir and NAPALM using the Ansible inventory Containerlab created. Oh yeah, Containerlab creates an Ansible inventory when deploying a topology. Example of the inventory that was created after I deployed this topology is below.

all:
  children:
    ceos:
      hosts:
        cicd-pdx-rtr-eos-01:
          ansible_host: 172.100.100.11
        cicd-pdx-rtr-eos-02:
          ansible_host: 172.100.100.12
        cicd-pdx-rtr-eos-03:
          ansible_host: 172.100.100.13
        cicd-pdx-rtr-eos-04:
          ansible_host: 172.100.100.14
    linux:
      hosts:
        cicd-client1:
          ansible_host: 172.100.100.21
        cicd-client2:
          ansible_host: 172.100.100.22

Nornir With Ansible Inventory

Nornir has a plugin to interact with an Ansible inventory, after getting Nornir installed, its just a matter of installing the plugin and setting the inventory to use that plugin.

pip install nornir nornir_ansible nornir_napalm

Nornir config.yaml

---
inventory:
  plugin: AnsibleInventory
  options:
    hostsfile: "cicd/ansible-inventory.yml"

runner:
  plugin: threaded
  options:
    num_workers: 10

Below is a function used to set credentials and platform for the nodes in this deployment.

Deployment Scripts

tools.py

"""Tools script that holds a variety of functions"""

import os


def nornir_set_creds(norn, username="admin", password="admin", platform="eos"):
    """
    Handler for credentials and platform
    """
    if not username:
        username = os.environ.get("NORNIR_USER")
    if not password:
        password = os.environ.get("MY_SECRET")

    for host_obj in norn.inventory.hosts.values():
        host_obj.username = username
        host_obj.password = password
        host_obj.platform = platform

deploy.py

#!/usr/bin/env python

"""Script used to configure the network"""

from nornir import InitNornir
from nornir_utils.plugins.functions import print_result
from nornir_napalm.plugins.tasks import napalm_configure
from tools import nornir_set_creds


def deploy_network(task):
    """Configures network with Scrapli"""
    if "client" in task.host.name:
        pass
    else:
        task1_result = task.run(
            name=f"Configuring {task.host.name}!",
            task=napalm_configure,
            filename=f"configs/post/{task.host.name}.cfg",
            replace=True,
        )


def main():
    """Used to run all the things"""
    norn = InitNornir(config_file="nornir_settings/config.yaml")
    nornir_set_creds(norn)
    result = norn.run(task=deploy_network)
    print_result(result)


if __name__ == "__main__":
    main()

Sample Configuration

cicd-pdx-rtr-eos-01.cfg

! Command: show running-config
! device: pdx-rtr-eos-01 (cEOSLab, EOS-4.27.0F-24305004.4270F (engineering build))
!
no aaa root
!
username admin privilege 15 role network-admin secret sha512 $6$RxQ5ae0GOW6SAiCU$7qzQNGX2pSIqWIYBIYGF8Xh30lo/s418/diYEEZj9rPrTJiAkYv0s6AvjpTfUHMGz.a58Hg29Yy/nV0Zvplux0
!
transceiver qsfp default-mode 4x10G
!
service routing protocols model multi-agent
!
hostname pdx-rtr-eos-01
!
spanning-tree mode mstp
!
management api http-commands
   no shutdown
!
management api gnmi
   transport grpc default
!
management api netconf
   transport ssh default
!
interface Ethernet1
   description connection to pdx-rtr-eos-02
   no switchport
   ip address 10.0.12.1/24
   ip ospf network point-to-point
   ip ospf area 0.0.0.0
!
!
interface Ethernet2
   no switchport
   ip address 192.168.1.1/24
   ip ospf area 0.0.0.0
!
interface Loopback1
   ip address 10.0.0.1/32
   ip ospf area 0.0.0.0
!
interface Management0
   ip address 172.100.100.11/24
!
ip routing
!
router bgp 65001
   router-id 10.0.0.1
   timers bgp 10 30
   neighbor 10.0.0.4 remote-as 65004
   neighbor 10.0.0.4 update-source Loopback1
   neighbor 10.0.0.4 ebgp-multihop 3
!
router ospf 1
   router-id 10.0.0.1
   passive-interface Ethernet2
   passive-interface Loopback1
   max-lsa 12000
!
endtext

Script output below! I’ll set the client1 and client2 IP addresses in the meantime!

❯ docker exec -it cicd-client1 bash
bash-5.0# ifconfig eth1 192.168.1.2 netmask 255.255.255.0 up
bash-5.0# route add default gw 192.168.1.1
bash-5.0# ping -c 4 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.064 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.074 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.080 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=64 time=0.073 ms

--- 192.168.1.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3072ms
rtt min/avg/max/mdev = 0.064/0.072/0.080/0.005 ms
bash-5.0# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr AA:C1:AB:94:0D:B0
          inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a8c1:abff:fe94:db0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9500  Metric:1
          RX packets:1754 errors:0 dropped:0 overruns:0 frame:0
          TX packets:26 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:218275 (213.1 KiB)  TX bytes:2112 (2.0 KiB)

bash-5.0# exit

❯ python deploy.py
deploy_network******************************************************************
* cicd-client1 ** changed : False **********************************************
vvvv deploy_network ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
^^^^ END deploy_network ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* cicd-client2 ** changed : False **********************************************
vvvv deploy_network ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
^^^^ END deploy_network ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* cicd-pdx-rtr-eos-01 ** changed : True ****************************************
vvvv deploy_network ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Configuring cicd-pdx-rtr-eos-01! ** changed : True ------------------------ INFO
@@ -22,12 +22,37 @@
    transport ssh default
 !
 interface Ethernet1
+   description connection to pdx-rtr-eos-02
+   no switchport
+   ip address 10.0.12.1/24
+   ip ospf network point-to-point
+   ip ospf area 0.0.0.0
 !
 interface Ethernet2
+   no switchport
+   ip address 192.168.1.1/24
+   ip ospf area 0.0.0.0
+!
+interface Loopback1
+   ip address 10.0.0.1/32
+   ip ospf area 0.0.0.0
 !
 interface Management0
    ip address 172.100.100.11/24
 !
-no ip routing
+ip routing
+!
+router bgp 65001
+   router-id 10.0.0.1
+   timers bgp 10 30
+   neighbor 10.0.0.4 remote-as 65004
+   neighbor 10.0.0.4 update-source Loopback1
+   neighbor 10.0.0.4 ebgp-multihop 3
+!
+router ospf 1
+   router-id 10.0.0.1
+   passive-interface Ethernet2
+   passive-interface Loopback1
+   max-lsa 12000
 !
 end
^^^^ END deploy_network ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* cicd-pdx-rtr-eos-02 ** changed : True ****************************************
vvvv deploy_network ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Configuring cicd-pdx-rtr-eos-02! ** changed : True ------------------------ INFO
@@ -22,12 +22,26 @@
    transport ssh default
 !
 interface Ethernet1
+   description connection to pdx-rtr-eos-01
+   no switchport
+   ip address 10.0.12.2/24
+   ip ospf network point-to-point
+   ip ospf area 0.0.0.0
 !
 interface Ethernet2
+   description connection to pdx-rtr-eos-03
+   no switchport
+   ip address 10.0.23.2/24
+   ip ospf network point-to-point
+   ip ospf area 0.0.0.0
 !
 interface Management0
    ip address 172.100.100.12/24
 !
-no ip routing
+ip routing
+!
+router ospf 1
+   router-id 10.0.0.2
+   max-lsa 12000
 !
 end
^^^^ END deploy_network ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* cicd-pdx-rtr-eos-03 ** changed : True ****************************************
vvvv deploy_network ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Configuring cicd-pdx-rtr-eos-03! ** changed : True ------------------------ INFO
@@ -12,6 +12,8 @@
 !
 spanning-tree mode mstp
 !
+vrf instance MGMT
+!
 management api http-commands
    no shutdown
 !
@@ -22,12 +24,27 @@
    transport ssh default
 !
 interface Ethernet1
+   description connection to pdx-rtr-eos-02
+   no switchport
+   ip address 10.0.23.3/24
+   ip ospf network point-to-point
+   ip ospf area 0.0.0.0
 !
 interface Ethernet2
+   description connection to pdx-rtr-eos-04
+   no switchport
+   ip address 10.0.34.3/24
+   ip ospf network point-to-point
+   ip ospf area 0.0.0.0
 !
 interface Management0
    ip address 172.100.100.13/24
 !
-no ip routing
+ip routing
+no ip routing vrf MGMT
+!
+router ospf 1
+   router-id 10.0.0.3
+   max-lsa 12000
 !
 end
^^^^ END deploy_network ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* cicd-pdx-rtr-eos-04 ** changed : True ****************************************
vvvv deploy_network ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Configuring cicd-pdx-rtr-eos-04! ** changed : True ------------------------ INFO
@@ -2,7 +2,7 @@
 !
 no aaa root
 !
-username admin privilege 15 role network-admin secret sha512 $6$80vAK4C7egYGmMCr$aQs6Oe1HPzToV9KkGBQozUNUzaelo8cM6EXUzDfsjPF4q/LJDb3WOtP01uqrCzKJWk3KpOMno40Df1nsilfwI/
+username admin privilege 15 role network-admin secret sha512 $6$RxQ5ae0GOW6SAiCU$7qzQNGX2pSIqWIYBIYGF8Xh30lo/s418/diYEEZj9rPrTJiAkYv0s6AvjpTfUHMGz.a58Hg29Yy/nV0Zvplux0
 !
 transceiver qsfp default-mode 4x10G
 !
@@ -22,12 +22,37 @@
    transport ssh default
 !
 interface Ethernet1
+   description connection to pdx-rtr-eos-03
+   no switchport
+   ip address 10.0.34.4/24
+   ip ospf network point-to-point
+   ip ospf area 0.0.0.0
 !
 interface Ethernet2
+   no switchport
+   ip address 192.168.4.1/24
+   ip ospf area 0.0.0.0
+!
+interface Loopback1
+   ip address 10.0.0.4/32
+   ip ospf area 0.0.0.0
 !
 interface Management0
    ip address 172.100.100.14/24
 !
-no ip routing
+ip routing
+!
+router bgp 65004
+   router-id 10.0.0.4
+   timers bgp 10 30
+   neighbor 10.0.0.1 remote-as 65001
+   neighbor 10.0.0.1 update-source Loopback1
+   neighbor 10.0.0.1 ebgp-multihop 3
+!
+router ospf 1
+   router-id 10.0.0.4
+   passive-interface Ethernet2
+   passive-interface Loopback1
+   max-lsa 12000
 !
 end
^^^^ END deploy_network ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Simple Verifications

pdx-rtr-eos-01#show ip ospf neighbor
Neighbor ID     Instance VRF      Pri State                  Dead Time   Address         Interface
10.0.0.2        1        default  0   FULL                   00:00:38    10.0.12.2       Ethernet1
pdx-rtr-eos-01#show ip route ospf

 O        10.0.0.4/32 [110/40] via 10.0.12.2, Ethernet1
 O        10.0.23.0/24 [110/20] via 10.0.12.2, Ethernet1
 O        10.0.34.0/24 [110/30] via 10.0.12.2, Ethernet1
 O        192.168.4.0/24 [110/40] via 10.0.12.2, Ethernet1

pdx-rtr-eos-01#show ip bgp summary
BGP summary information for VRF default
Router identifier 10.0.0.1, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor         V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.0.0.4         4 65004             86        86    0    0 00:11:34 Estab   0      0
pdx-rtr-eos-01#ping 192.168.4.1
PING 192.168.4.1 (192.168.4.1) 72(100) bytes of data.
80 bytes from 192.168.4.1: icmp_seq=1 ttl=62 time=0.164 ms
80 bytes from 192.168.4.1: icmp_seq=2 ttl=62 time=0.041 ms
80 bytes from 192.168.4.1: icmp_seq=3 ttl=62 time=0.033 ms
80 bytes from 192.168.4.1: icmp_seq=4 ttl=62 time=0.034 ms
80 bytes from 192.168.4.1: icmp_seq=5 ttl=62 time=0.033 ms

--- 192.168.4.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.033/0.061/0.164/0.051 ms, ipg/ewma 0.094/0.110 ms
pdx-rtr-eos-01#

At this point client1 (192.168.1.2) should be able to reach client2 (192.168.4.2)

❯ docker exec -it cicd-client1 bash
bash-5.0# ping -c 4 192.168.4.2
PING 192.168.4.2 (192.168.4.2) 56(84) bytes of data.
64 bytes from 192.168.4.2: icmp_seq=1 ttl=60 time=0.202 ms
64 bytes from 192.168.4.2: icmp_seq=2 ttl=60 time=0.186 ms
64 bytes from 192.168.4.2: icmp_seq=3 ttl=60 time=0.215 ms
64 bytes from 192.168.4.2: icmp_seq=4 ttl=60 time=0.254 ms

--- 192.168.4.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3066ms
rtt min/avg/max/mdev = 0.186/0.214/0.254/0.025 ms
bash-5.0#

Destroying a Topology

When destroying the lab there are two main options. Well really there is more but the other sounds dangerous! You can either destroy the lab which is more like shutting it down. Or you can destroy a lab and cleanup all directories/configurations that have been configured. The former will maintain configurations when you deploy again and the latter will revert to a factory reset if you will.

# Maintain configurations after shutdown
sudo containerlab destroy -t net.clab.yaml
# Destroy all configurations
sudo containerlab destroy -t net.clab.yaml --cleanup

Closing Thoughts

I think Containerlabs is pretty freaking awesome. The fact that I can deploy this topology in under a minute is crazy and just from my laptop. I can see Containerlab in so many different avenues. Architects testing out a design, Engineers doing it all, or students on their initial journey down the networking path or IT in general. One of the coolest things that comes to mind is Containerlab in a classroom settings. Just imagine the Instructor or teacher creates a topology file. At this point were assuming the class has a Linux system and Docker installed. The instructor then says “Okay students today we’ll be going over topology .clab.yaml. Students then learn about why a design was chosen or the technologies involved.

Outro and Links

Thank you all so much for reading this far. I hope this wasn’t too long. GitHub repository is linked below! I was kind of building this on the fly and the documentation for Containerlab is pretty excellent. If you have any questions seriously check it out. Most all of your questions will be answered there!

My Journey Learning About Juniper Junos

Mon, 06 Dec 2021 13:57:09 -0800

Introduction

Hello and thank you for checking out another blog post. I’ve been under the weather for a bit but I am back on the keys. This post will be all about some neat things I discovered while learning a bit about the Juniper Junos NOS (network operating system). Previously I had never used Juniper, it was a vendor that always eluded me. My previous employers did not utilize this NOS and I never got around to adding it in the lab. Sit back and enjoy, I’ll include a neat automation example at the end.

What is Junos?

Junos is the main operating system used in Juniper network devices. This could be the MX (routing), QFX (switching), and SRX (firewall). Since the same NOS is used across the product line, moving between them all in a configuration feels like being in the same device. This has a lot of benefits like eliminating that mode in your head of “which CLI am I in or which version of OS am I running?”. For the most part, it doesn’t matter, one configuration setting can transfer across all devices. At this point I’m probably overselling it, I’m sure there are some differences but you get the point.

Reading the Junos CLI

When I first saw Junos CLI output… I’ll be honest, the first thought in my head was “WTF is that?”. Is it JSON? Is it a weird version of JSON? Most of my past experience was on the Cisco CLI and going from Cisco to this was a bit of a surprise. Below is a snippet from a Junos configuration (modified for brevity).

Junos Config Snippet

juliopdx@R1> show configuration
## Last commit: 2021-12-06 11:19:02 PST by juliopdx
version 18.2R1.9;
system {
    login {
        message "Welcome to the world of tomorrow!";
    }
    root-authentication {
        encrypted-password "$6$xUmeLj6B$NBFrTm."; ## SECRET-DATA
    }
    host-name R1;
    time-zone America/Los_Angeles;
    services {
        ssh;
    }
}
interfaces {
    ge-0/0/0 {
        unit 0 {
            family inet {
                address 10.0.12.1/24;
            }
        }
    }
}
routing-options {
    static {
        route 0.0.0.0/0 {
            next-hop 192.168.10.1;
            no-readvertise;
        }
    }
    router-id 1.1.1.1;
}
protocols {
    ospf {
        reference-bandwidth 1g;
        area 0.0.0.0 {
            interface ge-0/0/0.0;
        }
    }
}

Looks a bit odd right? In reality, everything is in a nice orderly place. For example, any system configurations are neatly nested under that block. Any interface based settings would be under the interfaces block. Anything for OSPF is nested under protocols ospf.

CLI Modes

I think I’m getting a bit ahead of myself. Lets talk about the CLI modes. Junos mainly uses two CLI modes; operational and configuration. The mode you see displayed above is operational mode. Think of this as the mode you would use to view items or perform some… wait for it… operational tasks (monitor, troubleshoot, connectivity… ping). The second mode, configuration, is used to perform any and all configuration tasks on a Junos device.

Configuration Mode

juliopdx@R1> configure
Entering configuration mode

[edit]
juliopdx@R1# "Hi this is configuration mode"

Earlier I ran show configuration to view the configuration in operational mode. These commands can be executed in configuration mode as well by prepending run to the command. Please note, show works a bit differently when in configuration mode. Typing show will simply display the current configuration tree we are in (hold that thought for just a moment).

Using ping with run

juliopdx@R1# run ping 10.0.23.3
PING 10.0.23.3 (10.0.23.3): 56 data bytes
64 bytes from 10.0.23.3: icmp_seq=0 ttl=64 time=331.137 ms
64 bytes from 10.0.23.3: icmp_seq=1 ttl=64 time=28.864 ms
64 bytes from 10.0.23.3: icmp_seq=2 ttl=64 time=697.261 ms
64 bytes from 10.0.23.3: icmp_seq=3 ttl=64 time=1142.495 ms
64 bytes from 10.0.23.3: icmp_seq=4 ttl=64 time=148.471 ms
^C
--- 10.0.23.3 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 28.864/469.646/1142.495/405.256 ms

[edit]
juliopdx@R1#

Configuring Junos

At some point now we need to actually configure something. Let me just show you a small snippet of an interface configuration and we can walk through how the configuration goes.

interfaces {
    ge-0/0/0 {
        unit 0 {
            family inet {
                address 10.0.12.1/24;
            }
        }
    }

When you want to configure or remove something in Junos, the commands will start with either set or delete. If we wanted to configure interface ge-0/0/0 with IP address 10.0.12.1/24, the command below would do just that.

set interfaces ge-0/0/0 unit 0 family inet address 10.0.12.1/24

If you compare the configuration output to the command, its so similar, its basically the command in a tree format. You probably saw this and thought “What the heck is unit 0 and inet?”. Well unit 0 will be used to create a sub-interface on the interface. Think of this as the default sub-interface that is created or all interfaces. Kind of how there is a secret default VRF in Cisco land but its not really displayed. The inet is essentially the IPv4 address family. Can you guess what inet6 is?

That command got a bit long and there is an option to shorten our commands. Lets say for example we were about to drop a long configuration stanza on interface ge-0/0/0 and we wanted to shorten the amount of typing. We can actually tell Junos what section of the configuration we would like to edit. Example below.

juliopdx@R1# edit interfaces ge-0/0/0 unit 0 family inet

[edit interfaces ge-0/0/0 unit 0 family inet]
juliopdx@R1#

At this point we are in configuration mode and in the ge-0/0/0 interface hierarchy. Remember when I said hold that though on the show command? If we type show now, it will only display what section we are currently on. Another example below.

juliopdx@R1# show
address 10.0.12.1/24;

[edit interfaces ge-0/0/0 unit 0 family inet]
juliopdx@R1

Now if we wanted to configure that address, the command would be much shorter.

juliopdx@R1# set address 10.0.12.1/24

[edit interfaces ge-0/0/0 unit 0 family inet]
juliopdx@R1#

Replace, Compare, and Commit

I really like how Junos does this. If we were to make a change in Junos, it doesn’t immediately activate the change. It stores this in something called a candidate configuration. Lets test this. Currently interface ge-0/0/0 has an IP of 10.0.12.1, the remote side has an address of 10.0.12.2. Lets change the config and see if we can ping the remote side.

[edit interfaces ge-0/0/0 unit 0 family inet]
juliopdx@R1# top

[edit]
juliopdx@R1# replace pattern 10.0.12.1/24 with 10.0.50.1/24

[edit]
juliopdx@R1#

I’ll walk through what is going on here. We entered top to return us to the highest level of the config hierarchy. We then execute another way of implementing a change, by using replace! This allows us to change a text patter with another, without stepping through the entire configuration. Lets check out the current state of our changes.

juliopdx@R1# show | compare
[edit interfaces ge-0/0/0 unit 0 family inet]
+       address 10.0.50.1/24;
-       address 10.0.12.1/24;

[edit]
juliopdx@R1# run ping 10.0.12.2
PING 10.0.12.2 (10.0.12.2): 56 data bytes
64 bytes from 10.0.12.2: icmp_seq=0 ttl=64 time=269.266 ms
64 bytes from 10.0.12.2: icmp_seq=1 ttl=64 time=640.029 ms
64 bytes from 10.0.12.2: icmp_seq=2 ttl=64 time=116.563 ms
64 bytes from 10.0.12.2: icmp_seq=3 ttl=64 time=3.603 ms
^C
--- 10.0.12.2 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 3.603/257.365/640.029/240.205 ms

[edit]
juliopdx@R1#

At this point we can see that 10.0.12.1 has been replaced with 10.0.50.1 under the ge-0/0/0 interface. But we can still ping the remote side? This is because the change is only on the candidate configuration and not the running configuration. I’ll commit these changes and we should lose access to the remote side.

juliopdx@R1# commit comment "This is a bad config"
commit complete

[edit]
juliopdx@R1# run ping 10.0.12.2
PING 10.0.12.2 (10.0.12.2): 56 data bytes
36 bytes from 4.68.38.157: Destination Net Unreachable
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 45fd   0 0000  3c  01 5773 192.168.10.143  10.0.12.2

36 bytes from 4.68.38.157: Destination Net Unreachable
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 4637   0 0000  3c  01 5739 192.168.10.143  10.0.12.2

36 bytes from 4.68.38.157: Destination Net Unreachable
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 464e   0 0000  3c  01 5722 192.168.10.143  10.0.12.2

^C
--- 10.0.12.2 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

[edit]
juliopdx@R1#

Rollback and Commit Confirmed

Clearly our change was not successful. Imagine this but at a grander scale than just my simple address change. Junos has a very simple option to go back to a previously working configuration. You may have noticed that we added a comment to our bad configuration change. Lets check out our previous commits.

Previous Commits

[edit]
juliopdx@R1# run show system commit
0   2021-12-06 14:32:57 PST by juliopdx via cli
    This is a bad config
1   2021-12-06 13:41:27 PST by juliopdx via netconf

I’m staying in configuration mode so some of these commands are prepended with run. It looks like commit 0, the latest commit is our bad commit. Commit 1 is our last known good state. Lets roll this back and see if that gets us operational.

juliopdx@R1# rollback 1
load complete

[edit]
juliopdx@R1# show interfaces ge-0/0/0
unit 0 {
    family inet {
        address 10.0.12.1/24;
    }
}

[edit]
juliopdx@R1# commit
commit complete

[edit]
juliopdx@R1# run ping 10.0.12.2
PING 10.0.12.2 (10.0.12.2): 56 data bytes
64 bytes from 10.0.12.2: icmp_seq=0 ttl=63 time=338.004 ms
64 bytes from 10.0.12.2: icmp_seq=1 ttl=63 time=8.134 ms
64 bytes from 10.0.12.2: icmp_seq=2 ttl=63 time=6.880 ms
64 bytes from 10.0.12.2: icmp_seq=3 ttl=63 time=13.623 ms
^C
--- 10.0.12.2 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 6.880/91.660/338.004/142.249 ms

[edit]
juliopdx@R1#

Well that was easy! Another option for safety is commit confirmed. Basically, if you run commit confirmed, you will need to run commit again to make sure you want to implement the change. This is great in case you cut yourself off from the remote device. Example below.

juliopdx@R1# replace pattern 10.0.12.1/24 with 10.0.50.1/24

[edit]
juliopdx@R1# commit confirmed 1
commit confirmed will be automatically rolled back in 1 minutes unless confirmed
commit complete

# commit confirmed will be rolled back in 1 minute
[edit]
juliopdx@R1#
Broadcast Message from root@R1
        (no tty) at 14:44 PST...

Commit was not confirmed; automatic rollback complete.


[edit]
juliopdx@R1#

Pretty neat right? Since we did not type commit again, the change was automatically discarded.

Viewing Configuration as Set Commands

There is another really neat way of viewing what has been configured on the device, display set. Imagine every set command that has been executed on the device being stored in a nice format that can be reused again. Please see snippet below.

[edit]
juliopdx@R1# run show configuration | display set
set interfaces ge-0/0/0 unit 0 family inet address 10.0.12.1/24
set interfaces ge-0/0/1 unit 0 family inet address 10.0.13.1/24
set interfaces ge-0/0/2 unit 0 family inet address 10.0.0.1/24
set interfaces fxp0 unit 0 family inet address 192.168.10.143/24
set interfaces lo0 unit 0 family inet address 1.1.1.1/32
set snmp description somedevice
set snmp location "123 fake street"
set snmp contact "juliopdx@example.com"
set snmp community juliopdx authorization read-only
set snmp trap-group test categories chassis
set snmp trap-group test categories link
set snmp trap-group test categories routing
set snmp trap-group test categories startup
set snmp trap-group test categories services
set snmp trap-group test targets 192.168.10.198
set routing-options static route 0.0.0.0/0 next-hop 192.168.10.1
set routing-options static route 0.0.0.0/0 no-readvertise
set routing-options router-id 1.1.1.1
set protocols ospf reference-bandwidth 1g
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 passive
set protocols ospf area 0.0.0.0 interface lo0.0 passive
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0

[edit]
juliopdx@R1#

Bonus: Automating the Configuration

I couldn’t help myself, I wanted to take a shot at automating some of the configuration during my Junos learning. I’ll keep this short and sweet but I stayed with the common tools of Nornir, NAPALM, and some neat Jinja2 templates. I have covered everything in the repository (linked below) before besides the Jinja2 templates. I’ll go over those and the script to close this one out.

Junos Configuration Hierarchy and jinja2

The Junos text output I showed you initially in this post really lends itself to being automated. Even from a text standpoint. Everything has really natural breaks in separation. I mentioned this before but if you wanted to modify interfaces, this would all be done under the interfaces level. I took that same approach and broke the Jinja templates out. We have a main base.j2 template that will then call on the rest. This will then build the entire running configuration of a device.

(venv) juliopdx@juliopdx-pop:~/git/junos-auto$ tree templates/
templates/
├── base.j2
├── interfaces.j2
├── protocols.j2
├── routing-options.j2
├── snmp.j2
└── system.j2

base.j2

version 18.2R1.9;
{% include 'system.j2' %}

{% include 'interfaces.j2' %}

{% include 'snmp.j2' %}

{% include 'routing-options.j2' %}

{% include 'protocols.j2' %}

interfaces.j2

interfaces {
{% for interface in host.info.interfaces %}
    {{ interface.name }} {
        unit {{ interface.unit }} {
            family {{ interface.family }} {
                address {{ interface.address }};
            }
        }
    }
{% endfor %}
}

Defining Host Data

I kept it simple and used YAML files to store data about each node. Please note this is a very simple example. There are definitely more ways to improve this and remove some of the duplicate information.

(venv) juliopdx@juliopdx-pop:~/git/junos-auto$ tree host_vars/
host_vars/
├── R1.yaml
├── R2.yaml
└── R3.yaml

R1.yaml

---
interfaces:
  - name: ge-0/0/0
    unit: 0
    family: inet
    address: 10.0.12.1/24
  - name: ge-0/0/1
    unit: 0
    family: inet
    address: 10.0.13.1/24
  - name: ge-0/0/2
    unit: 0
    family: inet
    address: 10.0.0.1/24
  - name: fxp0
    unit: 0
    family: inet
    address: 192.168.10.143/24
  - name: lo0
    unit: 0
    family: inet
    address: 1.1.1.1/32

routing_options:
  statics:
    - route: 0.0.0.0/0
      next_hop: 192.168.10.1
      no_readvertise: True
  router_id: 1.1.1.1

protocols:
  ospf:
    ref_band: 1g
    areas:
      0.0.0.0:
        interfaces:
          - name: ge-0/0/2.0
            passive: True
          - name: lo0
            passive: True
          - name: ge-0/0/0.0
          - name: ge-0/0/1.0

Nornir and NAPALM

The script below was a mix of work I’ve done in the past and a great video by IPvZero (linked below). In the video John, demonstrates the use of the load_yaml and template_file methods in Nornir. The former will load YAML data that will then be assigned to the current host. This data can then be used by the template_file method to create configurations files for each host.

junos_norn.py

"""Script used to interfact with some Junos gear"""
from nornir import InitNornir
from nornir_napalm.plugins.tasks import napalm_configure
from nornir_utils.plugins.functions import print_result
from nornir_utils.plugins.tasks.data import load_yaml
from nornir_jinja2.plugins.tasks import template_file


def load_vars(task):
    """Loads data to be added to each host from vars files"""
    my_vars = task.run(task=load_yaml, file=f"./host_vars/{task.host}.yaml")
    task.host["info"] = my_vars.result
    template_configurations(task)


def template_configurations(task):
    """Builds network templates"""
    temp = task.run(
        name="Building device configuration",
        task=template_file,
        path="templates",
        template="base.j2",
    )
    config = temp.result
    deploy_configurations(task, config)


def deploy_configurations(task, config):
    """Loads device configurations"""
    task.run(
        name=f"Configuring {task.host}!",
        task=napalm_configure,
        configuration=config,
        replace=True,
    )


def main():
    """Used to run all the things"""
    norn = InitNornir(config_file="configs/config.yaml", core={"raise_on_error": True})
    result = norn.run(task=load_vars)
    print_result(result)


if __name__ == "__main__":
    main()

Outro and Links

Thank you all for reading this far. I hope you learned a bit about Junos and some automation. What I showed in this post barely scratches the surface on what Junos can do, definitely more to learn! This wasn’t created in a bubble and I will link some awesome resources I had when learning about Junos and automation. Please check it out if you get a chance. I will definitely be using Junos more in my day to day labbing and learning.

My Journey Learning About the Palo Alto Networks Python SDK

Mon, 22 Nov 2021 13:18:29 -0800

Introduction

Hello and thank you for checking out another blog post. This one will be all about Python and Palo Alto Networks (PAN). I was recently going through a PAN Firewall course on Pluralsight by Craig Stansbury. Craig does an excellent job of walking learners through the process of administering and securing a PAN firewall. I wanted to take this opportunity to double dip and try to wrap my head around the PAN pan-os-python library. This blog wont go into details of the course but it will go through the process of configuring the firewall using the PAN Python SDK.

Object-Oriented Programming (OOP)

I will do my best to break down OOP but in case my explanations are not clear enough, I will link a great post by the pros at Real Python. The Palo SDK uses objects very heavily in their code base. For example, if you create an instance of a firewall object, this not only creates a firewall but will also inherit attributes and methods from a parent class. Please see my router class example below.

class Router:
    pizza_box = True

    def __init__(self, vendor, os):
        self.vendor = vendor
        self.os = os

class CiscoRouter(Router):
    pass

>>> my_router = CiscoRouter("Cisco", "IOS")
>>> my_router.vendor
'Cisco'
>>> my_router.os
'IOS'
>>> my_router.pizza_box
True
>>>

In the example above, we have a parent class called Router and a child of that class called CiscoRouter. We set “Cisco” and “IOS” as the vendor and os respectively. Even though we did not specify the “pizza_box” attribute in the CiscoRouter class, it was inherited from the Router parent class. Please view this as an oversimplification. There is so much more to learn in this arena and I have a long way to go.

OOP in the PAN SDK

The PAN SDK works much in the same way. There are main parent classes and many child classes below them that will inherit attributes and available methods. Let me show you an example straight from the pan-os-python code base. Lets look at a firewall object.

class Firewall(PanDevice):
    """A Palo Alto Networks Firewall
    This object can represent a firewall physical chassis,virtual firewall, or individual vsys.
    """

The Firewall class is actually a child class of the PanDevice class. Lets instantiate a firewall object to get us going.

>>> fw = firewall.Firewall("192.168.10.192", "admin", "PaloAlto123!")
>>> type(fw)
<class 'panos.firewall.Firewall'>
>>> from rich import inspect
>>> inspect(fw)
╭──────────────────── <class 'panos.firewall.Firewall'> ─────────────────────╮
│ A Palo Alto Networks Firewall                                              │
│                                                                            │
│ ╭────────────────────────────────────────────────────────────────────────╮ │
│ │ <Firewall '192.168.10.192' None at 0x7ffa0c5e5ac0>                     │ │
│ ╰────────────────────────────────────────────────────────────────────────╯ │
│                                                                            │
│                   api_key = 'LUFRPT1hUEZ2K0w3UGFzMHdxQVcxcVJLZ3VzS1NKNmc9… │
│              CHILDMETHODS = ()                                             │
│                  children = []                                             │
│                CHILDTYPES = (                                              │
│                                 'device.AuthenticationProfile',            │
│                                 'device.AuthenticationSequence',           │
│                                 'device.Vsys',                             │
│                                 'device.VsysResources',                    │
│                                 'device.SystemSettings',                   │
│                                 'device.LogSettingsSystem',                │
│                                 'device.LogSettingsConfig',                │
│                                 'device.PasswordProfile',                  │
│                                 'device.Administrator',                    │
│                                 'device.Telemetry',                        │
│                                 'device.SnmpServerProfile',                │
│                                 'device.EmailServerProfile',               │
│                                 'device.LdapServerProfile',                │
│                                 'device.SyslogServerProfile',              │
│                                 'device.HttpServerProfile',                │
│                                 'ha.HighAvailability',                     │
│                                 'objects.AddressObject',                   │
│                                 'objects.AddressGroup',                    │
│                                 'objects.ServiceObject',                   │
│                                 'objects.ServiceGroup',                    │
│                                 'objects.Tag',                             │
│                                 'objects.ApplicationObject',               │
│                                 'objects.ApplicationGroup',                │
│                                 'objects.ApplicationFilter',               │
│                                 'objects.ApplicationContainer',            │
│                                 'objects.ScheduleObject',                  │
│                                 'objects.SecurityProfileGroup',            │
│                                 'objects.CustomUrlCategory',               │
│                                 'objects.LogForwardingProfile',            │
│                                 'objects.DynamicUserGroup',                │
│                                 'objects.Region',                          │
│                                 'objects.Edl',                             │
│                                 'policies.Rulebase',                       │
│                                 'network.EthernetInterface',               │
│                                 'network.AggregateInterface',              │
│                                 'network.LoopbackInterface',               │
│                                 'network.TunnelInterface',                 │
│                                 'network.VlanInterface',                   │
│                                 'network.Vlan',                            │
│                                 'network.VirtualRouter',                   │
│                                 'network.ManagementProfile',               │
│                                 'network.VirtualWire',                     │
│                                 'network.IkeGateway',                      │
│                                 'network.IpsecTunnel',                     │
│                                 'network.IpsecCryptoProfile',              │
│                                 'network.IkeCryptoProfile',                │
│                                 'network.GreTunnel',                       │
│                                 'network.Dhcp',                            │
│                                 'network.Zone'                             │
│                             )                                              │
╰────────────────────────────────────────────────────────────────────────────╯
>>>

Think of those CHILDTYPES as things you can associate with this firewall object. For now lets take a quick detour and look at the PanDevice class.

class PanDevice(PanObject):
    """A Palo Alto Networks device
    The device can be of any type (currently supported devices are firewall, or panorama). The class handles common device functions that apply to all device types.
    Usually this class is not instantiated directly. It is the base class for a firewall.Firewall object or a panorama.Panorama object.
    """

    NAME = "hostname"

    def __init__(
        self,
        hostname,
        api_username=None,
        api_password=None,
        api_key=None,
        port=443,
        is_virtual=None,
        timeout=1200,
        interval=0.5,
        *args,
        **kwargs
    )

At this point even PanDevice is a child of another class, PanObject. When we created our firewall object we did not specify a timeout. Lets check the firewall and see if this was inherited.

>>> fw.timeout
1200

Lets take one more detour and look at the PanObject class.

class PanObject(object):
    """Base class for all package objects
    This class defines an object that can be placed in a tree to generate configuration."""

It looks like PanObject is the base for all objects. Now you might be asking… why do I care? When configuring a PAN firewall, almost everything has some type of reference or dependency for something else to exist. For example, I cant assign an interface to a virtual router named “DMZ” if that does not exist first. Another example, I cant assign address objects to an address group if the address objects don’t exist. Once I wrapped my head around the concept of inheritance and configuration trees, the SDK made much more sense.

From pan-os-python readthedocs

Configuring Zone Example

I love providing examples and its one of the best ways I learn. I’ll walk you through configuring a firewall zone and the configuration tree involved to get the correct sequences down. For reference below is an example configuration tree from the official pan-os-python documentation.

>>> from panos import firewall
>>> from panos import network
>>> fw = firewall.Firewall("192.168.10.192", "admin", "PaloAlto123!")
>>> lan_zone = network.Zone(
...     name="User LAN",
...     mode="layer3",
...     zone_profile="DefaultZoneProtectionProfile",
...     log_setting="Zone-Forwarding-Profile",
...     enable_packet_buffer_protection=True,
... )
>>> fw.add(lan_zone)
<Zone User LAN 0x7ffa0b8aa7f0>
>>> lan_zone.create()
>>>

Lets break that all down. We are importing a few required libraries from panos. One will be used to instantiate a firewall object and the other will be used to build a zone object. We then define any required parameters to the zone object. This is then added to the fw object using the add method. There are many methods available, for this proof of concept I heavily used the create method.

Repeating Myself

Once I had the process down to configure zones, interfaces, or anything else available in the SDK, I started to get everything on paper. Eventually the zone portion of my script looked something like below…

# Creating Zones on Firewall
lan_zone = network.Zone(
    name="User LAN",
    mode="layer3",
    zone_profile="DefaultZoneProtectionProfile",
    log_setting="Zone-Forwarding-Profile",
    enable_packet_buffer_protection=True,
)
fw.add(lan_zone)
lan_zone.create()


dc_zone = network.Zone(
    name="Datacenter",
    mode="layer3",
    zone_profile="DefaultZoneProtectionProfile",
    log_setting="Zone-Forwarding-Profile",
    enable_packet_buffer_protection=True,
)
fw.add(dc_zone)
dc_zone.create()


dmz_zone = network.Zone(
    name="DMZ",
    mode="layer3",
    zone_profile="DefaultZoneProtectionProfile",
    log_setting="Zone-Forwarding-Profile",
    enable_packet_buffer_protection=True,
)
fw.add(dmz_zone)
dmz_zone.create()


outside_zone = network.Zone(
    name="Outside",
    mode="layer3",
    zone_profile="OutsideZoneProtectionProfile",
    log_setting="Zone-Forwarding-Profile",
    enable_packet_buffer_protection=True,
)
fw.add(outside_zone)
outside_zone.create()

For every zone, the steps required to create them is all the same. This is true for other pieces of the firewall configuration; interfaces, tags, etc. Eventually my configuration script grew rather large and had repetition all over the place. I left that file (connect.py) in the original repository so individuals could see how something can be transformed to be a bit more manageable.

Making Zones and Not Repeating Myself

I eventually turned all of those steps into a simple function. The function would take two to three parameters. Usually something like the firewall object and the data required to create a zone.

from configs.zones import zones

def create_zone(fire, curent_zone):
    """Creates firewall zone and returns object"""
    set_zone = network.Zone(**curent_zone)
    fire.add(set_zone)
    set_zone.create()
    return set_zone


for zone in zones:
    create_zone(fw, zone)

We now have a new import, zones! I figured it was more manageable to break out the parameters into individual Python files. An example of the directory and zones.py file is below.

(venv) juliopdx@juliopdx-pop:~/git/pan-auto$ tree configs/
configs/
├── address_groups.py
├── address_objects.py
├── __init__.py
├── interfaces.py
├── nats.py
├── routing.py
├── security_policies.py
├── tags.py
└── zones.py

zones.py

zones = [
    {
        "name": "User LAN",
        "mode": "layer3",
        "enable_packet_buffer_protection": True,
    },
    {
        "name": "Datacenter",
        "mode": "layer3",
        "enable_packet_buffer_protection": True,
    },
    {
        "name": "DMZ",
        "mode": "layer3",
        "enable_packet_buffer_protection": True,
    },
    {
        "name": "Outside",
        "mode": "layer3",
        "enable_packet_buffer_protection": True,
    },
    {
        "name": "Test",
        "mode": "layer3",
        "enable_packet_buffer_protection": True,
    },
]

In the previous example you may have seen the line below and been a bit confused (I was).

set_zone = network.Zone(**curent_zone)

What does “**” mean??? In Python this is a known as Kwargs or keyword arguments. This allows us to pass in the dictionaries we created to our object call and reference the correct parameters. This has the benefit of decoupling the object from the data completely. Every zone can have as many or as little keyword arguments as necessary to create the object.

Making Prints Pretty

I’ve been sharing terminal output like this on twitter during this learning journey. This isn’t required by any means but if I’m on the terminal all the time, why not make it look good? I’ll break down one example of these tasks, the rest follow a very similar structure.

from rich.progress import Progress, SpinnerColumn, BarColumn, TextColumn

with Progress(
    SpinnerColumn("bouncingBall", speed=0.6),
    BarColumn(),
    TextColumn("[progress.percentage]{task.description} {task.percentage:>3.0f}%"),
) as progress:

    job1 = progress.add_task("[bright_green]Configuring Zones", total=len(zones))

    while not progress.finished:
        for zone in zones:
            create_zone(fw, zone)
            progress.update(job1, advance=1)

This portion of the script will use the Rich Python Library from Will McGugan, linked below! I’m importing anything required to build the progress bar output. This example was based off of the included example under Rich. I select a type of spinner to display and what text will be shown. I then add a task to progress. In this case job1 has a total equal to length of the zones variable. We have five zones so this will equate to five. Once the for loop starts, it will advance the task up by one every time. Eventually this will hit 5 and progress will be finished.

Outro and Links

Thank you all for reading this post. Really means a lot and I hope you found something in here useful. Initially working with this SDK was a pain in the rear, but after digging into the configuration trees and inheritance it all clicked. Looking through some source code never hurt either! Please note, what’s included here and in the repository linked below is just a subset of what is possible using these tools. I wish you all the best and happy automating!

Building a Network CI/CD Pipeline Part 6

Fri, 12 Nov 2021 12:26:21 -0800

Introduction

Hello all and thank you for tuning in to the last part of the network CI/CD series. It really means a lot and I hope you have learned a bit along the way. In this post we will focus on testing after a change is made, more specifically, we will be looking at Suzieq. If you stick around until the end, we will look at maintaining a “golden state” of configuration using cron and a message from Rocky.

What is Suzieq?

Suzieq allows operators and network engineers the ability to gather data about their network using an agentless model. This data is then stored in a data format known as Parquet. Once this multi-vendor data is normalized, we can begin to ask the network a numerous amount of questions. This could be things from MTU mismatches, OSPF neighbor relationship state, and many more.

Batfish and Suzieq

At this point you may be wondering if this is similar to Batfish. Batfish uses a “snapshot” of the network or a view of a change we are proposing to the network. Batfish will then build a model around this snapshot to infer if any issues are present with the configuration. There are no actual connections performed to network devices using Batfish. Suzieq on the other hand uses a Poller to continuously connect to our devices and gather network state. This data can then be used to run checks or see how our network is configured and performing. I think Batfish lends itself really well to performing precheck validation. Suzieq can perform precheck validation but for this example we will focus on post change testing. I will link a great write up by the maintainers of these respective tools at the end of this blog.

Getting Suzieq Up and Running

We wont stray far from actions that have been performed on previous posts… that’s right, more docker! Using the docker instance of Suzieq really provides the fastest way to get going. I’ll start at the root of our Ubuntu server that is running our docker containers.

Docker Container

juliopdx@drone:~$ docker ps
CONTAINER ID   IMAGE                         COMMAND                  NAMES
e5d807183f32   drone/drone-runner-docker:1   "/bin/drone-runner-d…"   runner
89b442422f25   drone/drone:2                 "/bin/drone-server"      drone
2df0b3ee02fb   batfish/batfish               "java -XX:-UseCompre…"   batfish
juliopdx@drone:~$

Building the Inventory File

At the moment Suzieq supports a built in yaml format as well as using a standard yaml Ansible inventory. For this scenario I kept things simple and used the built in format. Please note, there is word from the developers to add more dynamic inventory plugins, using Netbox for example. Below is the inventory I created for this deployment, more examples can be found in the Suzieq documentation. Please note, there will be examples of old school(0.15.0) commands and new formats introduced in version 0.16.0. If you are using a different version, please check out their documentation.

Version 0.15.0 inv.yaml

- namespace: eos
  hosts:
    - url: ssh://192.168.10.121 username=suzie password=suzie
    - url: ssh://192.168.10.122 username=suzie password=suzie
    - url: ssh://192.168.10.123 username=suzie password=suzie
    - url: ssh://192.168.10.124 username=suzie password=suzie

Version 0.16.0 inv.yaml

sources:
- name: eos-source
  type: native
  hosts:
    - url: ssh://192.168.10.121
    - url: ssh://192.168.10.122
    - url: ssh://192.168.10.123
    - url: ssh://192.168.10.124

devices:
- name: eos-devices
  devtype: eos
  ignore-known-hosts: true

auths:
- name: suzie-user
  username: suzie
  password: plain:suzie

namespaces:
- name: eos
  source: eos-source
  device: eos-devices
  auth: suzie-user

Installing the Docker Container/Poller

docker run -d --restart always -itd \
-v /home/juliopdx/suz/parquet-out:/suzieq/parquet \
-v /home/juliopdx/inv.yaml:/suzieq/inv.yaml \
--name sq-poller netenglabs/suzieq:latest

The command above is doing a few things. It will create a link to a local “/suz/parquet-out” directory and copy the “inv.yaml” file we created over to the docker container.

juliopdx@drone:~$ docker ps
CONTAINER ID   IMAGE                         COMMAND                  NAMES
a091d6fe3b07   netenglabs/suzieq:latest      "/bin/bash"              sq-poller
e5d807183f32   drone/drone-runner-docker:1   "/bin/drone-runner-d…"   runner
89b442422f25   drone/drone:2                 "/bin/drone-server"      drone
2df0b3ee02fb   batfish/batfish               "java -XX:-UseCompre…"   batfish
juliopdx@drone:~$

Now we will attach to the container and get the service running. To disconnect from the Poller and leave the service running, type “CTRL + p, CTRL + q”

juliopdx@drone:~$ docker attach sq-poller
root@a091d6fe3b07:/suzieq# sq-poller -D inv.yaml -k # version 0.15.0
root@eceb70607520:/suzieq# sq-poller -I inv.yaml # version 0.16.0+

Side Note, The Suzieq CLI

In my case I will be interacting with Suzieq using their Python library and the Poller, but you can interact with Suzieq using a CLI or GUI as well. For the moment, the Poller has most likely gathered some data about our network. Lets stop the Poller(CTRL + c) and connect to the CLI!

From the CLI we can quickly explore a ton of information about our network and run some asserts to see if our routing protocols are working as expected. As an example below are some working routing neighbor relationships and an example when one is down(I added a duplicate router ID).

Assert Pass

Assert Fail

I think you get the idea. The Suzieq CLI is a quick way to explore not only the capabilities of Suzieq but also your network. At this point lets exit the CLI, restart the Poller, and exit from the container(CTRL + p, CTRL + q).

root> exit
root@a091d6fe3b07:/suzieq# sq-poller -D inv.yaml -k # version 0.15.0
root@eceb70607520:/suzieq# sq-poller -I inv.yaml # version 0.16.0+
read escape sequence
juliopdx@drone:~$

Directory Permissions

You may not run into this but in my case I ran into an issue where I had to give more permissions to the directory on my host machine running the containers.

sudo chmod -R a+rwx suz/

The Challenge: Getting the Data to the Pipeline Runner

The Poller collects a wide range of data from our network devices, to get an idea of what is being collected, check out the tree breakdown below.

juliopdx@drone:~$ tree -d -L 3 suz
suz
└── parquet-out
    ├── arpnd
    │   └── sqvers=1.0
    ├── bgp
    │   └── sqvers=2.0
    ├── coalesced
    │   ├── arpnd
    │   ├── bgp
    │   ├── devconfig
    │   ├── device
    │   ├── ifCounters
    │   ├── interfaces
    │   ├── lldp
    │   ├── ospfIf
    │   ├── ospfNbr
    │   ├── routes
    │   ├── sqCoalescer
    │   ├── sqPoller
    │   ├── time
    │   └── vlan
    ├── devconfig
    │   └── sqvers=1.0
    ├── device
    │   └── sqvers=2.0
    ├── ifCounters
    │   └── sqvers=1.0
    ├── interfaces
    │   └── sqvers=3.0
    ├── lldp
    │   └── sqvers=1.0
    ├── ospfIf
    │   └── sqvers=1.0
    ├── ospfNbr
    │   └── sqvers=2.0
    ├── routes
    │   └── sqvers=2.0
    ├── sqPoller
    │   └── sqvers=2.0
    ├── time
    │   └── sqvers=2.0
    └── vlan
        └── sqvers=2.0

42 directories
juliopdx@drone:~$

Here is where things got interesting. In our pipeline, every run stands up a fresh docker container to execute our steps. How can we expose this data to the container to run our Python test scripts against? With a bit of research I came to find this golden nugget. So it is possible! But there’s another caveat, it must be from trusted repositories. This led me to recreating the drone server and runners, but this time giving my GitHub user account admin privileges. I have updated the previous blog to reflect this change. Below is the same example in the previous blog post.

docker run \
  --volume=/var/lib/drone:/data \
  --env=DRONE_GITHUB_CLIENT_ID=your-id \
  --env=DRONE_GITHUB_CLIENT_SECRET=super-duper-secret \
  --env=DRONE_RPC_SECRET=super-duper-secret \
  --env=DRONE_SERVER_HOST=drone.company.com \
  --env=DRONE_SERVER_PROTO=http \
  --env=DRONE_USER_CREATE=username:{{GitHub-Username}},admin:true \
  --publish=80:80 \
  --publish=443:443 \
  --restart=always \
  --detach=true \
  --name=drone \
  drone/drone:2

After that was done it was just a matter of flipping a flag under the repository settings.

.drone.yml Updates

Now that everything is set correctly, we have to update our pipeline file to include the Suzieq steps and mount the host directory to the runner. Below is an example. Lines 14-17 are naming a volume and then giving it a path that is on our Ubuntu host. Lines 10-12 attach that volume to the runner at a specified path.

.drone.yml

- name: Suzieq Check
  image: python:3.8
  commands:
  - pip install suzieq rich
  - python test_suzieq.py
  when:
    branch:
    - master
    - main
  volumes:
  - name: suzieq
    path: /tmp/suz

volumes:
- name: suzieq
  host:
    path: /home/juliopdx/suz

Suzieq and Python

The final piece was actually getting a Python script to interact with the mounted data. If you are using Python, Suzieq will look at either ./suzieq-cfg.yml or ~/.suzieq/suzieq-cfg.yml for a configuration file. There is a lot that can be modified here but the most important I think is where the parquet data is stored. Below is my configuration file for reference.

suzieq-cfg.yml

data-directory: /tmp/suz/parquet-out
temp-directory: /tmp/suzieq
logging-level: WARNING

analyzer:
  # By default, the timezone is set to the local timezone. Uncomment
  # this line if you want the analyzer (CLI/GUI/REST) to display the time
  # in a different timezone than the local time. The timestamp stored in
  # the database is always in UTC.
  timezone: America/Los_Angeles

Suzieq Python Examples

Lets start with a few examples in the Python interpreter.

Import required libraries

(suzie-venv) juliopdx@drone:~$ python
Python 3.8.10 (default, Sep 28 2021, 16:10:42)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> from suzieq.sqobjects import get_sqobject

Next we will get a sqobject from the data that has been gathered from the Poller. In this case we are grabbing the latest interfaces data.

>>> int_tbl = get_sqobject("interfaces")
>>> int_tbl().get() #data omitted for brevity
   namespace        hostname       ifname state adminState      type    mtu
0        eos  pdx-rtr-eos-02         MGMT    up         up       vrf   1500
1        eos  pdx-rtr-eos-02    Ethernet5    up         up  ethernet   9214
2        eos  pdx-rtr-eos-02    Ethernet4    up         up  ethernet   9214
3        eos  pdx-rtr-eos-02    Ethernet7    up         up  ethernet   9214
4        eos  pdx-rtr-eos-02    Ethernet6    up         up  ethernet   9214
5        eos  pdx-rtr-eos-02    Ethernet3    up         up  ethernet   9214
6        eos  pdx-rtr-eos-02    Ethernet8    up         up  ethernet   9214
7        eos  pdx-rtr-eos-02  Management1    up         up  ethernet   1500
8        eos  pdx-rtr-eos-02    Ethernet1    up         up  ethernet   1500
9        eos  pdx-rtr-eos-02    Ethernet2    up         up  ethernet   1500
10       eos  pdx-rtr-eos-04    Ethernet6    up         up  ethernet   9214
11       eos  pdx-rtr-eos-04         MGMT    up         up       vrf   1500
12       eos  pdx-rtr-eos-04    Ethernet5    up         up  ethernet   9214
13       eos  pdx-rtr-eos-04    Ethernet4    up         up  ethernet   9214
14       eos  pdx-rtr-eos-04    Ethernet7    up         up  ethernet   9214
15       eos  pdx-rtr-eos-04    Ethernet1    up         up  ethernet   1500
16       eos  pdx-rtr-eos-04    Ethernet2    up         up  ethernet   1500
17       eos  pdx-rtr-eos-04    Loopback1    up         up  loopback  65535
18       eos  pdx-rtr-eos-04    Ethernet8    up         up  ethernet   9214
19       eos  pdx-rtr-eos-04  Management1    up         up  ethernet   1500
20       eos  pdx-rtr-eos-04    Ethernet3    up         up  ethernet   9214
21       eos  pdx-rtr-eos-03         MGMT    up         up       vrf   1500
22       eos  pdx-rtr-eos-03    Ethernet5    up         up  ethernet   9214
23       eos  pdx-rtr-eos-03    Ethernet7    up         up  ethernet   9214
24       eos  pdx-rtr-eos-03    Ethernet6    up         up  ethernet   9214
25       eos  pdx-rtr-eos-03    Ethernet4    up         up  ethernet   9214
26       eos  pdx-rtr-eos-03    Ethernet3    up         up  ethernet   9214
27       eos  pdx-rtr-eos-03    Ethernet2    up         up  ethernet   1500
28       eos  pdx-rtr-eos-03    Ethernet8    up         up  ethernet   9214
29       eos  pdx-rtr-eos-03  Management1    up         up  ethernet   1500
30       eos  pdx-rtr-eos-03    Ethernet1    up         up  ethernet   1500
31       eos  pdx-rtr-eos-01    Ethernet5    up         up  ethernet   9214
32       eos  pdx-rtr-eos-01  Management1    up         up  ethernet   1500
33       eos  pdx-rtr-eos-01    Ethernet8    up         up  ethernet   9214
34       eos  pdx-rtr-eos-01    Loopback1    up         up  loopback  65535
35       eos  pdx-rtr-eos-01    Ethernet2    up         up  ethernet   1500
36       eos  pdx-rtr-eos-01    Ethernet3    up         up  ethernet   9214
37       eos  pdx-rtr-eos-01    Ethernet1    up         up  ethernet   1500
38       eos  pdx-rtr-eos-01    Ethernet6    up         up  ethernet   9214
39       eos  pdx-rtr-eos-01    Ethernet7    up         up  ethernet   9214
40       eos  pdx-rtr-eos-01    Ethernet4    up         up  ethernet   9214
41       eos  pdx-rtr-eos-01         MGMT    up         up       vrf   1500
>>>

You can also filter on data when making these calls. Maybe we only care about certain interfaces from a particular host?

>>> int_tbl().get(hostname=["pdx-rtr-eos-01"], ifname=["Ethernet1", "Ethernet2"])
  namespace        hostname     ifname state adminState      type   mtu  ipAddressList
0       eos  pdx-rtr-eos-01  Ethernet2    up         up  ethernet  1500  [192.168.1.1/24]
1       eos  pdx-rtr-eos-01  Ethernet1    up         up  ethernet  1500  [10.0.12.1/24]
>>>

Maybe we want to see what device owns a particular route?

>>> routes_tbl = get_sqobject("routes")
>>> routes_tbl().get(prefix=["10.0.0.1"], protocol="connected")
  namespace        hostname      vrf       prefix nexthopIps         oifs   protocol
0       eos  pdx-rtr-eos-01  default  10.0.0.1/32         []  [Loopback1]  connected
>>>

What I’ve shown you above and what is included in the pipeline post check is just a small subset of what Suzieq can do. Below is a very basic script that will pull data from the parquet directory, depending on a pass or fail, different messages will be printed. If any failures are seen, a counter will run and exit the program with a code of 1. This will signal the pipeline to register the run as a failure. I’ll include a few snapshots of running this in the terminal, the rich library makes even the failures look pretty.

test_suzie.py

import sys
import pandas as pd
from suzieq.sqobjects import get_sqobject
from rich import print


# OSPF Testing
ospf_tbl = get_sqobject("ospf")
ospf_df = pd.DataFrame(ospf_tbl().aver())
ospf_fail = 0
for index, row in ospf_df.iterrows():
    if row["assert"] != "pass":
        print(
            f":triangular_flag_on_post: OSPF, {row['hostname']} {row['ifname']} {row['assertReason']} :triangular_flag_on_post:"
        )
        ospf_fail += 1

# BGP testing
bgp_tbl = get_sqobject("bgp")
bgp_df = pd.DataFrame(bgp_tbl().get())
bgp_fail = 0
for index, row in bgp_df.iterrows():
    if row["state"] != "Established":
        print(
            f":triangular_flag_on_post: {row['hostname']}(AS {row['asn']}) to {row['peer']}(AS {row['peerAsn']}) is in {row['state']} state :triangular_flag_on_post:"
        )
        bgp_fail += 1

if bgp_fail == 0:
    print(":white_heavy_check_mark: All BGP checks passed :white_heavy_check_mark:")
if ospf_fail == 0:
    print(":white_heavy_check_mark: All OSPF checks passed :white_heavy_check_mark:")
if bgp_fail or ospf_fail != 0:
    sys.exit(1)

Below is an example of me adding a duplicate router ID on pdx-rtr-eos-01.

Now lets fix that issue and see if we can see the transitions on the network. OSPF neighbors should come back and BGP soon after.

Suzieq Pipeline Run

Below is an example pipeline run after a pull request was submitted and approved. I’ll show you the last step which is new to us in this series. To my knowledge Suzieq is only tested against Python 3.7 and 3.8. Th docker container I created is based on 3.9.7. Drone makes this fairly trivial, as I can just reference another docker image and install the two dependencies we have; Suzieq and Rich.

.drone.yml

- name: Suzieq Check
  image: python:3.8
  commands:
  - pip install suzieq rich
  - python test_suzieq.py

BONUS! Maintain State With Cron

Rapid fire here. Lets just say we had a rogue engineer making all kinds of changes outside of the pipeline. How could we remediate this? Drone and I’m sure other CI tools have a cron feature. Essentially, we can execute pipeline runs at set intervals. Since the main branch is our “production” state. We can set up a cron job to run every day, week, or hour??? I’m going to make some random changes to all four routers and see how the pipeline corrects them. I will set the cron job to run every hour so we don’t have to wait a day.. or a week.

As you can see above, all the bogus VLANs and loopbacks have been removed from the configuration.

Be Like Rocky

You, me, or nobody is gonna hit as hard as life. But it ain’t how hard you hit; it’s about how hard you can get hit, and keep moving forward.

I love that Rocky quote. I’m highlighting my failures here to show you that you can do it. When learning something new you may be scratching your head or lost in the sauce. Keep going and when you hit that magical moment, its really really fun.

Outro and Links

Thank you all for reading this far. This blog series was an absolute treat to write. I hope something along the way has sparked your interest! Please note, what I’ve shown in these posts is just a subset of what these tools can do. Feel free to explore and make it better!

Building a Network CI/CD Pipeline Part 5

Mon, 08 Nov 2021 11:58:46 -0800

Introduction

Hello all, thank you for reading the previous posts in this series. It has been fun interacting with all of you and see a few folks taking the challenge of standing this up on their own. In this post we will go over the framework that is performing our configuration deployments, Nornir and NAPALM. I will also provide an example worklflow on how this is all tied together and executed from a network engineers perspective. Please note, the steps that are performed by Nornir can also be performed by something like Ansible.

Nornir

My previous automation experience revolved heavily around Ansible for interacting with APIs or network nodes. I really like both tools and I feel each has their space in the network automation world. Where Ansible uses yaml for writing playbooks or steps to be executed on nodes, interacting with Nornir stays within the python ecosystem. This lends itself well to using an IDE like Visual Studio Code and all the other tools available within python (linting, debugging, tab completion).

Init Nornir

Lets get started with initializing Nornir and setting some of the configuration. A link to all the code used in this post will be at the end. Below we are importing the “InitNornir” function as well as a local function that is used to handle device credentials.

from nornir import InitNornir
from tools import nornir_set_creds

Below is the main function that is used to initialize Nornir and kick off any tasks that are defined within our deployment script.

def main():
    """Used to run all the things"""
    norn = InitNornir(config_file="configs/config.yaml", core={"raise_on_error": True})
    nornir_set_creds(norn)
    result = norn.run(task=deploy_network)
    print_result(result)


if __name__ == "__main__":
    main()

At this point we are initializing Nornir with a file located at “configs/config.yaml”, lets take a look at that directory.

(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ tree configs/
configs/
├── config.yaml
├── groups.yaml
└── hosts.yaml

This makes me chuckle a bit, I just mentioned that Nornir is pretty much all python and Ansible uses yaml for developing playbooks. Yet here we are making yaml files. Jokes aside, yaml is really good at being human readable and building configuration files. Lets take a look at the “config.yaml” file.

---
inventory:
  plugin: SimpleInventory
  options:
    host_file: "./configs/hosts.yaml"
    group_file: "./configs/groups.yaml"
runner:
  plugin: threaded
  options:
    num_workers: 10

This file can be used to set all kinds of settings. I’ve tried to keep it fairly simple, we are using the “SimpleInventory” plugin. This can read in our hosts and groups file that I will go over in a bit. Other inventroy plugins (Ansible, Netbox, table_inventory) can also be used, check more out here. The second half of the file is utilizing the built-in threaded runner. Nornir will spin up threads to allow task executions to multiple devices vs executed one by one. Next up lets check out the “groups.yaml” file.

---
eos:
  platform: eos

As you can see I have barely used this file. If we were running multiple vendors or maybe devices located at different sites, they could be assigned different configuration options that would be added within this groups file. Below is an example of that concept in case my explanation is a bit terrible.

---
eos:
  platform: eos
cisco_ios:
  platform: ios
branch:
  dns:
  - 8.8.8.8
  - 1.1.1.1
dc:
  dns:
  - 10.0.0.125
  - 10.0.0.126

Last but not least the “hosts.yaml” file. For small deployments I can see how maintaining this file wouldn’t be the worst thing in the world. For scale, I would use something like Netbox to build your inventory file. All nodes in this case are assigned to the “eos” group and will inherit “eos” as the platform.

---
pdx-rtr-eos-01:
  hostname: 192.168.10.121
  groups:
    - eos

pdx-rtr-eos-02:
  hostname: 192.168.10.122
  groups:
    - eos

pdx-rtr-eos-03:
  hostname: 192.168.10.123
  groups:
    - eos

pdx-rtr-eos-04:
  hostname: 192.168.10.124
  groups:
    - eos

Deploy Configurations

Earlier I showed you the main function which is calling on a certain task to be executed. Below is the main function again.

def main():
    """Used to run all the things"""
    norn = InitNornir(config_file="configs/config.yaml", core={"raise_on_error": True})
    nornir_set_creds(norn)
    result = norn.run(task=deploy_network)
    print_result(result)

The main function will call the “deploy_network” function, which will be executed against our network devices in the inventory. Below is the “deploy_network” function.

from nornir_napalm.plugins.tasks import napalm_configure

def deploy_network(task):
    """Configures network with NAPALM"""
    task1_result = task.run(
        name=f"Configuring {task.host.name}!",
        task=napalm_configure,
        filename=f"./snapshots/configs/{task.host.name}.txt",
        dry_run=args.dry,
        replace=True,
    )

We are adding one more import to our script. The “napalm_configure” function is part of the nornir_napalm library. Under the hood it will use NAPALM to connect and configure network devices. The plugin options are pretty extensive depending on what your use case is (deploying entire config or just a subset). This function will look at the file name that matches our device hostname. It will then attempt to push the entire configuration to the device.

Side Note on Infrastructure as Code

For this example we have the device configurations located under the snapshots directory. The entire configuration can be viewed and modified. For a IaC approach, you might develop some yaml or json data that can then be passed into a jinja template to define a configuration file. This file can then be used to configure your entire device. The data files could contain interface settings, routing information, services, pretty much anything that is part of the configuration. Check out this great blog series by NWMichl on Network Infrastructure as Code.

Workflow

Lets take a look at our reference architecture below to get an idea of the workflow we will run through to make a change to our network.

At the bottom of our diagram we have a simple OSPF and BGP network with four Arista devices. We have a task for our junior network engineer to add interface descriptions to all interfaces connected to networking nodes. Since this is a really safe change with no impact to the network, besides maybe confusing some operators if they are incorrect. This seems like a great learning opportunity for the junior engineer.

The engineer starts by cloning the teams repository from GitHub.

git clone https://github.com/JulioPDX/ci_cd_dev.git

Great, now the repository is local to our development environment. We would most likely have some standards to not allow direct commits to the main branch. All changes must be made to a separate branch and then merged by pull request to main. Main in this case will signal our production configuration state. Next up the junior engineer will create a dev branch named “add_interface_desc”.

(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ git checkout -b add_interface_desc
Switched to a new branch 'add_interface_desc'
(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ git status
On branch add_interface_desc
nothing to commit, working tree clean

Now the engineer is on the new branch and can modify the configuration files in the snapshots directory. Lets look at what files have been modified.

(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ git status
On branch add_interface_desc
Changes not staged for commit:
  (use "git add ..." to update what will be committed)
  (use "git restore ..." to discard changes in working directory)
        modified:   snapshots/configs/pdx-rtr-eos-01.txt
        modified:   snapshots/configs/pdx-rtr-eos-02.txt
        modified:   snapshots/configs/pdx-rtr-eos-03.txt
        modified:   snapshots/configs/pdx-rtr-eos-04.txt

no changes added to commit (use "git add" and/or "git commit -a")

Great, now lets push those changes up to the remote repository.

(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ git add .
(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ git commit -m "Adding interface descriptions"
[add_interface_desc 9b8d899] Adding interface descriptions
 4 files changed, 6 insertions(+)
(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ git push -u origin add_interface_desc
Enumerating objects: 15, done.
Counting objects: 100% (15/15), done.
Delta compression using up to 8 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (8/8), 781 bytes | 390.00 KiB/s, done.
Total 8 (delta 5), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (5/5), completed with 5 local objects.
remote:
remote: Create a pull request for 'add_interface_desc' on GitHub by visiting:
remote:      https://github.com/JulioPDX/ci_cd_dev/pull/new/add_interface_desc
remote:
To https://github.com/JulioPDX/ci_cd_dev.git
 * [new branch]      add_interface_desc -> add_interface_desc
Branch 'add_interface_desc' set up to track remote branch 'add_interface_desc' from 'origin'.
(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$

From our previous work this should have kicked off a pipeline run without making any actual changes to the nodes.

At this point every precheck stage has passed in the pipeline and we can see what the changes would be. We can now create a pull request and merge these changes into our main branch.

Above we can see the branch has a passing build. The reviewer or senior engineer can now merge the pull request. After that is done another pipeline run should execute but this time the changes will be applied to our network.

Now there is a new stage in the pipeline…deploy configurations! Lets take a look at one of the devices and see if the change was indeed applied.

Outro and Links

The change we made was really trivial but its a good way to show the new process in the world of CI/CD. Thanks again for checking out this post. I think we have one more to go in the series. The last one will go over using Suzieq for network observability and post change testing.

Building a Network CI/CD Pipeline Part 4

Sun, 31 Oct 2021 11:26:48 -0800

Introduction

Hello everyone and thank you for checking out part four in this series. I went on vacation for a bit, but I’m glad to be back on the keys. In this post I will break down all of the steps performed before a change hits our network devices. This is important because we have the opportunity to stop incorrect or invalid configurations from ever hitting our network devices. A few steps that we will cover are: Black for code formatting, Batfish to validate configuration updates, and NAPALM dry run for testing the legitimacy of the configuration files.

Black Code Style

When adding steps to this pipeline I have concentrated on easy wins and easy tools to add to the chain. One of the slickest of them being Black. Black is a code formatter based on the Python PEP 8 standard. You can see the steps in the pipeline file below.

.drone.yml

steps:
- name: Black Code Format Check
  image: juliopdx/netauto
  commands:
  - black . --check

Black will traverse all Python files in our directory and fail if any changes are required. The “–check” option will trigger the failure in our pipeline. Below is an example of what a failure would look like. I added a new file to our repository with a long list variable.

my_list.py

my_list = ["something1", "something2", "something3", "something4", "something5", "something6"]

I will commit and push this up to our dev branch.

The Black format check has failed and listed what files would need to be updated. This is very useful to maintain some set standards for yourself or the team on the code style you choose to follow. Below is an example of a success, in this case I have updated that new Python file. Another option to add in this step is something like Pylint. At this time I have not added this to the pipeline but I encourage the reader to check it out. Pylint will go as far as giving the code a score and provide improvement options.

my_list.py updated

my_list = [
    "something1",
    "something2",
    "something3",
    "something4",
    "something5",
    "something6",
]

Validating Changes With Batfish

Batfish dubs itself as “an open source network configuration analysis tool”. Batfish can validate pre-deployment checks by modeling the network from a source of configurations. This could vary from mismatched BGP neighbor settings, OSPF mismatches, and many more. I’ll walk through installing the Batfish service and run through some examples of what it can find!

Lets get Batfish up and running. Thankfully this can be done by running the latest docker container. This is building on what we have already created from previous articles!

Batfish Docker Run

docker run --name batfish \
    -v batfish-data:/data  \
    -p 9997:9997 -p 9996:9996 \
    -d batfish/batfish

Great, now that the service is running we need to create a configuration snapshot. Think of this as our production network or the changes we would like to push to our production network. This could be one site or the whole enterprise. Be mindful, you may need more horsepower if you are testing against a large network. In our case we have a very simple four node topology running OSPF and BGP between R1 and R4. To get the configuration snapshot up and running I created a simple backup script to store the files in the correct location.

Directory Structure

(venv) juliopdx@juliopdx-pop:~/git/ci_cd_dev$ tree snapshots/
snapshots/
└── configs
    ├── pdx-rtr-eos-01.txt
    ├── pdx-rtr-eos-02.txt
    ├── pdx-rtr-eos-03.txt
    └── pdx-rtr-eos-04.txt

Batfish Assertion Helpers

While exploring Batfish, I ran into their assertion helpers page. There is some gold in that page and I highly recommend you check it out. I used these heavily in this test deployment to see what errors Batfish would find in the configuration. Pretty much all of these require the same bit of information (a snapshot is required). I’ll walk through the script and one of the assert functions, the rest follow the same path.

Below are some of the standard pybatfish imports followed by a decent amount of the assert helpers. Importing Rich because it’s awesome and I like colors.

Imports

from pybatfish.client.commands import *
from pybatfish.question import load_questions
from pybatfish.client.asserts import (
    assert_no_duplicate_router_ids,
    assert_no_incompatible_bgp_sessions,
    assert_no_incompatible_ospf_sessions,
    assert_no_unestablished_bgp_sessions,
    assert_no_undefined_references,
)
from rich import print as rprint

Below is the function used to test if we have duplicate router IDs. It requires one argument of snap or snapshot.

def test_duplicate_rtr_ids(snap):
    """Testing for duplicate router IDs"""
    assert_no_duplicate_router_ids(
        snapshot=snap,
        protocols={"ospf", "bgp"},
    )

This function will look at BGP and OSPF within our snapshot to see if duplicate router IDs are set. I omitted a few Rich prints because they are not pertinent to the functionality of the function. Below if the main function that calls all of our assert helpers. A lot of this is created from samples in the Batfish documentation. For example, we name our snapshot and point it to the correct directory. We also point the script to the correct IP that is running the service.

Main Function

def main():
    """init all the things"""
    NETWORK_NAME = "PDX_NET"
    SNAPSHOT_NAME = "snapshot00"
    SNAPSHOT_DIR = "./snapshots"
    bf_session.host = "192.168.10.184"
    bf_set_network(NETWORK_NAME)
    init_snap = bf_init_snapshot(SNAPSHOT_DIR, name=SNAPSHOT_NAME, overwrite=True)
    load_questions()
    test_duplicate_rtr_ids(init_snap)
    test_bgp_compatibility(init_snap)
    test_ospf_compatibility(init_snap)
    test_bgp_unestablished(init_snap)
    test_undefined_references(init_snap)


if __name__ == "__main__":
    main()

Test for Duplicate Router IDs

All of our routers have RIDs set as 10.0.0.x, x being the router number. Lets change the router ID under OSPF for R1 to be 10.0.0.2, this is done under the snapshot directory configuration file for R1. For an infrastructure as code build these would be created from YAML files or some source of truth and maybe some jinja sprinkled in there. This change would be a duplicate of R2 and we should see the pipeline fail.

R1 OSPF Configuration

router ospf 1
   router-id 10.0.0.2
   passive-interface Ethernet2
   passive-interface Loopback1
   max-lsa 12000

We will commit and push this to our dev branch, the pipeline output is below. In the bottom right you can barely see the duplicate router IDs error and towards the bottom of the output it will list the devices that are in error. In this case both R1 and R2 have router IDs set to “10.0.0.2”.

Test for Incompatible BGP Sessions

Lets add another error. In this case R1 and R4 are running multihop EBGP. R1 has an autonomous system (AS) of 65001 and R4 has an AS of 65004. Lets just say someone misconfigured BGP and set R1 to have a neighbor of 65003 vs 65004. As we can see R1 is set to have a neighbor with R4 using AS 65003 but R4 is configured with AS 65004. You may have noticed towards the top of the pipeline that no duplicate router IDs were found in this run.

R1 Bad BGP

router bgp 65001
   router-id 10.0.0.1
   neighbor 10.0.0.4 remote-as 65003
   neighbor 10.0.0.4 update-source Loopback1
   neighbor 10.0.0.4 ebgp-multihop 3

Test for OSPF Network Mismatches

Okay this is the last example we will use but I think you all get the idea. All OSPF neighbors have interfaces configured with point to point interfaces. Lets remove the R1 to R2 point to point interface configuration in our snapshot. This should result in a network type mismatch error.

interface Ethernet1
   no switchport
   ip address 10.0.12.1/24
 - ip ospf network point-to-point
   ip ospf area 0.0.0.0

At this point we have prevented a few configuration errors from entering the network. Please note, this is only scratching the surface on what can be done with Batfish. Please check out their documentation and code examples on more ideas to test your network. Links at the end of this post.

Validating Configuration With NAPALM Dry Run

Batfish is great and it can catch a lot of errors or misconfigurations. What if there is something that can’t be caught by Batfish as a configuration error. How would we validate what we are sending is even a valid configuration in general? In this case I leveraged Nornir with NAPALM. NAPALM has a “napalm_configure” task that will attempt to send a complete configuration file to a network device. If it is not valid, it should report an error. Remember that we are in the precheck stage and don’t want any actual changes to hit our network devices. Below is a snippet on what I came up with to keep the code small but also functional between precheck deployments and actual deployments.

build.py

import argparse
from nornir import InitNornir
from nornir_napalm.plugins.tasks import napalm_configure
from nornir_utils.plugins.functions import print_result
from tools import nornir_set_creds


parser = argparse.ArgumentParser()
parser.add_argument(
    "--dry_run", dest="dry", action="store_true", help="Will not run on devices"
)
parser.add_argument(
    "--no_dry_run", dest="dry", action="store_false", help="Will run on devices"
)
parser.set_defaults(dry=True)
args = parser.parse_args()


def deploy_network(task):
    """Configures network with NAPALM"""
    task1_result = task.run(
        name=f"Configuring {task.host.name}!",
        task=napalm_configure,
        filename=f"./snapshots/configs/{task.host.name}.txt",
        dry_run=args.dry,
        replace=True,
    )


def main():
    """Used to run all the things"""
    norn = InitNornir(config_file="configs/config.yaml", core={"raise_on_error": True})
    nornir_set_creds(norn)
    result = norn.run(task=deploy_network)
    print_result(result)


if __name__ == "__main__":
    main()

The top portions of the script import anything required to interact with Nornir and NAPALM. We then use “argparse” to create an argument with the script that will set a variable to True or False. This can then be used during the “napalm_configure” task to either run in dry mode or actually implement changes. Below is how the precheck looks like in the “.drone.yml” file.

Configuration Precheck

- name: Precheck Configuration Diff
  image: juliopdx/netauto
  environment:
    MY_SECRET:
      from_secret: MY_SECRET
  commands:
  - python build.py --dry_run

NAPALM Dry Run With Error

Below is an example of adding something to the configuration that will definitely not work and how it looks like in the pipeline.

Error Configuration

router bgp 65001
   router-id 10.0.0.1
   neighbor 10.0.0.4 remote-as 65004
   neighbor 10.0.0.4 update-source Loopback1
   neighbor 10.0.0.4 ebgp-multihop 3
!
something super fake
   welcome to the world of tomorrow
   its over 9000
   router-id infinity
!
router ospf 1
   router-id 10.0.0.1
   passive-interface Ethernet2
   passive-interface Loopback1
   max-lsa 12000
!

NAPALM Dry Run With Valid Configuration

Below is a standard change, adding a simple description to an interface. Included pipeline output as well.

Valid Configuration

!
interface Ethernet1
   description Welcome to the world of tomorrow!
   no switchport
   ip address 10.0.12.1/24
   ip ospf network point-to-point
   ip ospf area 0.0.0.0
!

Outro and Links

Whats included in this post is just a subset of options that are available in prechecks. We could add ACL, input validation, Pylint, routing tests, and more. I hope what’s included here sparks some ideas or gives you something to add to your CI/CD workflow. I think in the next post we will go over the tool used to actually deploy configurations once they have passed our prechecks. In our case Nornir, but this could be Ansible, Scrapli, or some other solution.

Building a Network CI/CD Pipeline Part 3

Wed, 20 Oct 2021 11:05:49 -0800

Introduction

Thank you for checking out part three in this series, it really means a lot! So far we have installed Docker on an Ubuntu host machine and the Drone server/runners. In this post we will go over the .drone.yml file and why I decided to create my first Docker container image. Stick around after the break…. who am I kidding there is no break, what is this an infomercial?

.drone.yml or .drone.yaml or .. something?

Pretty much all CI systems use some kind of configuration file, essentially the steps you want performed on your pipeline run. They all have their own flavor, but essentially walk the same line. My only previous experience was with Azure Devops pipelines. The .drone.yml file should be placed at the root of your directory structure of the project. Below is one of the first versions of the configuration file in my repository.

Initial .drone.yml file

kind: pipeline
type: docker
name: Testing Python CI/CD


steps:
- name: Install and deploy
  image: python
  environment:
    MY_PASS:
      from_secret: MY_PASS
  commands:
  - pip3 install -r requirements.txt
  - python3 bat.py

trigger:
  branch:
    exclude:
    - main
    - master

Lines 1-3 go over the highest level settings of the pipeline. We are using a docker type, since we installed the docker runners. If you are using something like exec, you would have to place that there and install exec runners as well. Lines 6-14 describe the step or steps performed. My simple example is running one step and executing two commands. The image being used is a python Docker image from Docker hub. Once that is pulled down, the commands will execute on the container. The first will install some requirements from my code base and the second will execute the script bat.py. The script itself is just performing a simple connection to devices in my lab. We are utilizing the secret we set earlier in the repository settings here in the script.

The Problem With Requirements

If your environment has very little or no external dependencies, you may not run into this problem. As I got the pipeline working and starting adding more to the code base, this also added more requirements that needed to be installed during the pipeline run. The server created that is hosting these docker containers is not very powerful. That could be the cause of some of these delays. Check out a pipeline run below(a bit exaggerated but you get the point).

17 minutes!!! Yikes! I think software development has a concept of feedback loops. Where we would like to know if something passed or failed relatively quickly. Over time this would be crazy, imagine waiting that long to find out you had a small code syntax error, I guess it would make for a good time to get a coffee. This then led me down the path of creating a Docker container image.

Creating a Docker Container Image (Optional)

So whats the big deal with this container image? For one, I’ve never even come close to thinking of making one. I’m trying to show you, the reader ,that if there is some technology you don’t know or aren’t comfortable with, with a little time and persistence you can make really cool things happen. I wanted to see if there was some way to make a lightweight python container and then just add on my requirements. I did what any sane person would do… I hit up my friend Google.

Google then led me to this incredible write up by Samuel James, check it out here. We already had Docker installed on our Ubuntu server so that wasn’t required. I did have to create a Docker Hub account and run “docker login”. Once logged in I proceeded with creating the Dockerfile.

Dockerfile

The Dockerfile essentially describes how you want to build your image. Check out my example below.

FROM python:3.9.7-slim

# Install dependencies:
COPY requirements.txt .
RUN pip install -r requirements.txt

That’s about it… no, I’m kidding. The FROM statement is telling docker if we want to base our image from any other image. For this example we are using the python 3.9.7-slim variant. After that I have a requirements.txt file locally that will be copied to the container. Then RUN will execute that command on the container, which will install our dependencies.

requirements.txt

netmiko
rich
black
pybatfish
napalm
pytest
nornir
nornir-utils
nornir-napalm

Build Container and Publish

docker build -t juliopdx/netauto .
docker push juliopdx/netauto:latest

Now that we have a new container image we can use this in our .drone.yml file. If you are really curious check out the docker hub page here. Isn’t it pretty? Check out the updated configuration file below.

.drone.yml

kind: pipeline
type: docker
name: Testing Python CI/CD


steps:
- name: Black Code Format Check
  image: juliopdx/netauto
  commands:
  - black . --check

- name: Precheck OSPF and BGP
  image: juliopdx/netauto
  commands:
  - pytest test.py --disable-pytest-warnings -s

- name: Deploy Configurations
  image: juliopdx/netauto
  environment:
    MY_PASS:
      from_secret: MY_PASS
  commands:
  - python build.py
  when:
    branch:
    - master
    - main

trigger:
  event:
    exclude:
    - pull_request

Lines 1-3 are essentially the same. I updated the steps as well. The first step just makes sure we are adhering to black formatting standards. If there is a file to change the pipeline will fail. The second step runs a few Batfish checks on OSPF and BGP, more on that in a later post. The last step will actually deploy configurations to our test network, but only if it is a main or master branch. The trigger at the bottom is just used to exclude pull requests. Every step pulls down the exact same image (I believe only pulling it down once), to my knowledge each step is actually executed individually. So requirements installed in one step would not exist in another. Okay okay, but what about the pipeline deployment time? Please see below.

Outro and Links

Isn’t that the most beautiful thing ever? A whole 19 seconds <3. Seriously thank you all for reading this far. Next up I think we will go over adding Batfish to the equation. Stayed tuned for the next episode…

Building a Network CI/CD Pipeline Part 2

Wed, 20 Oct 2021 10:41:26 -0800

Introduction

Thank you for checking out part two of this series. The last post went through installing docker on an Ubuntu server to begin our journey to building a CI/CD pipeline. In this post it will be all about connecting Drone to our code repository, in this case GitHub. From our reference diagram below, we will be focusing on the connections from GitHub to the Drone runners.

The Pipeline Server and Runners

Now that we have docker installed, we need something to test our code or execute it. This is one of the main pieces of CI/CD. There are many tools out there that can perform this function. Some of them are Jenkins, Drone, Travis CI, and many more. In this case I went with Drone. Simply because it was new to me and I wanted to play with it. The Drone server interacts with your source control of choice, in my case GitHub. The Drone server and runners can be installed as containers. The runners are actually executing your pipelines. This could be code linting, testing, and deployment of configurations.

One of the neat things about drone is that each runner is an isolated container environment. For example if we wanted to run a pipeline using a python based app, we could pull down a python image and run that as our execution container. Or maybe we are testing some powershell and needed a windows execution environment, we could pull down that image. Once the runners are done with their job, they are destroyed and leave nothing to cleanup.

Why NGROK?

From the diagram above you may have seen NGROK and thought, “what the heck is that?”. Since we will be integrating GitHub with Drone, GitHub needs a public URL or address to hit when making webhook calls to the Drone server. This is more than suitable for our testing purposes. NGROK will form a secure tunnel between your node and their service. All requests that hit this URL will then be forwarded to your application or web server for example. Please note NGROK has free and paid versions. The free version is more than what we need for testing.

Installing NGROK

mkdir temp && cd temp
wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip
sudo apt install unzip
unzip ngrok-stable-linux-amd64.zip
./ngrok http 80

At this point you will see some output similar to below. I didn’t see an option to run NGROK detached, so in the meantime leave this running and connect to the server on another session. Save these URLs as they will be used to build an OAuth application on GitHub.

Creating GitHub OAuth Application

Login to GitHub account and go to Settings > Developer Settings > OAuth apps. From there click on the “”New OAuth App” icon.

Remember that URL we had to note down a while ago? Go ahead and fill in the info, example below as reference.

After clicking on “Register application”, click on “Generate a new client secret”. Write down or save those values somewhere. You will need them when setting up the drone server. Please note, if you restart the NGROK service, a new URL will be generated and you will have to update your OAuth application with that URL.

Installing Drone Server

One more credential needs to be created, the shared secret (RPC_SECRET) between the drone server and runners. Execute the command below from the Ubuntu server.

Example from drone.io

$ openssl rand -hex 16
bea26a2221fd8090ea38720fc445eca6

docker pull drone/drone:2

docker run \
  --volume=/var/lib/drone:/data \
  --env=DRONE_GITHUB_CLIENT_ID={{DRONE_GITHUB_CLIENT_ID}} \
  --env=DRONE_GITHUB_CLIENT_SECRET={{DRONE_GITHUB_CLIENT_SECRET}} \
  --env=DRONE_RPC_SECRET={{DRONE_RPC_SECRET}} \
  --env=DRONE_SERVER_HOST={{123456.ngrok.io}} \
  --env=DRONE_SERVER_PROTO=https \
  --env=DRONE_USER_CREATE=username:{{GitHub Username}},admin:true \
  --publish=80:80 \
  --publish=443:443 \
  --restart=always \
  --detach=true \
  --name=drone \
  drone/drone:2

Feel free to set environment variables or just replace the {{}} settings with the values you have gathered so far. If you want a more in depth breakdown of all the variables, please check out the Drone docs!

Installing Drone Runners

The runners are interesting, there’s a few versions of the runners depending on what type of environment you are working with. For example, the exec runner will execute pipelines directly on the host, no isolation provided. Useful if you are in an environment that cant utilize containers. This really didn’t seem like a good idea to me especially as I was testing and destroying things all the time. Drone recommends starting with the docker runner if you are new to Drone and I would echo that. For more info on the Drone runner install, check out their docs!

docker pull drone/drone-runner-docker:1

docker run --detach \
  --volume=/var/run/docker.sock:/var/run/docker.sock \
  --env=DRONE_RPC_PROTO=https \
  --env=DRONE_RPC_HOST={{123456.ngrok.io}} \
  --env=DRONE_RPC_SECRET={{super-duper-secret}} \
  --env=DRONE_RUNNER_CAPACITY=3 \
  --env=DRONE_RUNNER_NAME=my-first-runner \
  --publish=3000:3000 \
  --restart=always \
  --name=runner \
  drone/drone-runner-docker:1

Please note, I had issues when running the dynamic URL and Chrome, if you run into this issue please test with Firefox.

Welcome and Activating a Repository

At this point you should be able to hit your public URL, something like “http://1234567.ngrok.io”. You will be greeted with the Drone welcome page. Feel free to login and authenticate with your GitHub account. Now that you are logged in, you may see something like below.

No repositories are active and being synced with Drone. If you already have a repository created, feel free to click on it and then click on “ACTIVATE REPOSITORY”. Something like this will then be displayed.

I left my build fairly vanilla but something that would be interesting is cron jobs from the server. Imagine the main or golden config repository is executed everyday at midnight to prevent configuration drift. We will get into the configuration file in a moment. In the meantime head to “Secrets” under organization or under general.

Lets say our script needs to connect to our devices using a key or password. We can set secrets here that can then be utilized by our build environments. In my lab I will be using Arista devices with simple username and password for authentication. The scripts will then look for an environment variable of “MY_PASS”. Go ahead and save changes once you are happy with them. You probably noticed there is a setting for “Configuration” and it is currently set to .drone.yml. Stayed tuned on the next episode of Dragon Ball… wait a minute that’s not this program. Seriously, on the next part I will go over creating a .drone.yml file and creating our very own docker container image!

Building a Network CI/CD Pipeline Part 1

Wed, 20 Oct 2021 10:26:36 -0800

Introduction

Hello all and thank you for joining me on another blog post! In this post or series of posts I hope to walk you through my journey on building a network CI/CD pipeline. This pipeline is not perfect and I’m sure there’s much more efficient ways to produce the same outcomes. I did make the choice to leave out a few things as the technologies involved and complexity kept growing. For example, I did not go deep into infrastructure as code with this example but I will link a great series from a fellow network engineer at the bottom of this post.

I tried to make this deployment as simple as possible and a few of the technologies are very new to me. I say this because I feel a lot of us get lost in the sauce when so many new technologies are mentioned or even displayed in one architecture diagram. I hope to minimize some of that restraint and break the pieces down as much as possible.

What is CI/CD?

Some of you might be wondering what CI/CD is or even why we should care? CI/CD is a software development concept where software is created, tested and eventually released in a continuous manner. The pieces involved for different applications may differ but I hope to show you some of the main concepts.

How does this relate to network engineering? Lets say you didn’t have a CI/CD pipeline and you wanted to make a change on the production network. The steps would be something like the following:

User requests change to the network
Change control ticket is created detailing change and test plan
Login to a test environment if there is one and test the change
Login to the devices one by one and implement change
Validate expected outcomes after change on the devices

This process is fairly slow and has a lot of touch points. For example, the testing plan by different engineers may differ or a missed validation could lead to a network outage or an unexpected behavior from the change. Using a CI/CD pipeline we hope to automate a lot of these steps to ensure the change is correct and safe to implement on our network. This isn’t to say that a pipeline will eliminate all errors, far from that. This will allow us to continually improve our testing and the network over time to make it easier to manage and implement changes quickly and safely.

Pieces to the Puzzle

Below are some of the technologies we will be breaking down over this series of posts:

GitHub
Docker
Drone(pipeline server and runners)
NGROK
Batfish
Nornir/NAPALM
Suzieq

Getting Started

Please note, the local code development can be done on any laptop or PC of choice. The docker pieces can be done on a local machine as well but I chose to stand up a simple Ubuntu 20.04 virtual machine with 4 CPUs and 8G of RAM.

GitHub

If you are following along or just getting started then I would highly recommend creating a GitHub account. This is a great place to share and collaborate with other creators in a wide variety of spaces. If you are studying a programming language or going down the path of NetDevOps, you will definitely want some version control.

Docker

I’ll be honest, I’m fairly new to docker and containers as a whole. Imagine a docker container being one of those Lunchables you ate as a kid. Everything you need is inside of that container. No external dependencies are involved. As mentioned before I created an Ubuntu VM for this deployment to run docker and any containers. The first step was getting docker installed. Credit to the team at Digital Ocean for the great write up on installing docker, check it out here.

sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
apt-cache policy docker-ce
sudo apt install docker-ce
sudo systemctl status docker

Using Docker without Sudo

sudo usermod -aG docker ${USER}
su - ${USER}

Useful Docker Commands

View a list of running containers

juliopdx@drone:~$ docker ps
CONTAINER ID   IMAGE                         COMMAND                  STATUS         NAMES
ed76314f72f7   drone/drone-runner-docker:1   "/bin/drone-runner-d…"   Up 2 minutes   runner
500785d2e6d4   drone/drone:2                 "/bin/drone-server"      Up 3 minutes   drone
2df0b3ee02fb   batfish/batfish               "java -XX:-UseCompre…"   Up 8 days      batfish

Stop or start containers “docker stop/start

juliopdx@drone:~$ docker stop runner
runner
juliopdx@drone:~$ docker start runner
runner
juliopdx@drone:~$

If you want to explore more Docker commands, check out the official documentation!

Outro and Links

I’ll end this one here. Stay tuned for the next post, setting up the Drone server and runners!

How to Generate Cisco Lifecycle Documentation With Python and APIs

Thu, 23 Sep 2021 10:06:16 -0800

Introduction

I was recently in a Twitter conversation with a few network engineers, in that feed was great dialogue on how often folks upgrade their network environments. Whether its when a massive vulnerability hits or just keeping up with vendor recommended releases. I mentioned that the Cisco Support APIs are an incredible resource. These APIs allow users with the appropriate access to get information about their devices. This could be recommended software releases, EoX dates, last day of support, vulnerabilities, and others.

I wanted to see if there was a way to incorporate a few of these APIs together to create meaningful documentation for the operator in the field. This documentation could then be used to plan out device upgrades, replacements, or just a way to track software versions of the entire device fleet. I hope you find something useful or maybe take bits and pieces so it may help you in your environment.

Cisco Support APIs

As mentioned earlier, these APIs really are an incredible resource. If you are a Cisco Smart Net Total Care customer or a Cisco Partner Support Services partner, you most likely have access to these APIs. If you would like to follow along you will have to register an application to get started. It sounds more daunting than it really is, check out this guide to get that process started.

Handling Support API Authentication

When interacting with APIs there are usually a few options to choose from. You may use a username/password combination, token, or a combination of client IDs and secrets. I chose the latter. The default timer for tokens used for the support APIs is good for about one hour. This will require the user to reauthenticate every hour. I tried to keep this simple by having a local credentials.yaml file in the repo, which is then added to the .gitignore file.

credentials.yaml example

support_client_id: SomeClientId
support_client_secret: SomeClientSecret

Once you have the client ID and client secret from the app that is registered. You can execute the auth.py script in the repo. Repository is linked at the bottom of this post!

auth.py

import requests
import yaml


URL = "https://cloudsso.cisco.com/as/token.oauth2"

with open("./credentials.yaml", encoding="utf-8") as file:
    myvars = yaml.safe_load(file)

payload = f"grant_type=client_credentials&client_id={myvars['support_client_id']}&client_secret={myvars['support_client_secret']}"

headers = {
    "Content-Type": "application/x-www-form-urlencoded",
}

response = requests.request("POST", URL, headers=headers, data=payload)

if response.status_code == 200:
    with open("credentials.yaml", encoding="utf-8") as f:
        creds = yaml.load(f, Loader=yaml.SafeLoader)

    creds["support_auth_token"] = response.json()["access_token"]

    # Rewriting credentials.yaml with new token variable
    with open("credentials.yaml", "w", encoding="utf-8") as f:
        yaml.dump(creds, f)
        print("Auth token updated successfully!!!")
else:
    print("Failed to update token, please check client ID or client secret")

At the top of the script we are importing requests to interact with APIs and yaml for working with yaml files. This will then load the current client secret and client ID from our credentials.yaml file. On line 16 we run our API POST call to retrieve a token. This token can then be used to interact with the support APIs. The second half of the script will essentially recreate the credentials.yaml file, but with the token now in place. Example below!

credentials.yaml

support_auth_token: SomeToken
support_client_id: SomeClientId
support_client_secret: SomeClientSecret

Chicken and Egg Problem

Now that we have a working token, we could use this to run some kind of API GET to gather some data about a device or platform based on the serial number or part ID. Here is the main issue I ran into. The support APIs require either a serial number or part ID to gather relevant information. At some point you will need an inventory or a list of serial numbers to feed into the APIs to gather information for you. I am cheating a bit and started with an inventory file of three devices. This inventory only included the device IP addresses and their platform for handling connections. Example inventory below.

hosts.yaml

---
# Please note, the cisco_ios group is only used to set the device platform type
some-router-01:
  hostname: "192.168.10.10"
  groups:
    - cisco_ios

some-router-02:
  hostname: "192.168.10.11"
  groups:
    - cisco_ios

some-other-node:
  hostname: "192.168.10.12"
  groups:
    - cisco_ios

Imports and CSVs

We will be using Nornir and NAPALM for the initial connection to our devices. The csv import is used for creating our file with all of our information. The yaml import is used to interact with our credentials.yaml file and load the token. You may notice a lot of imports from “tools”, these are a few neat functions I made for this small project. More on those in a bit.

info.py imports and auth

import csv
import yaml
from nornir import InitNornir
from nornir_napalm.plugins.tasks import napalm_get
from tools import manu_year_cisco, nornir_set_creds, product_info, software_release, eox


with open("./credentials.yaml", encoding="utf-8") as file:
    myvars = yaml.safe_load(file)

my_token = myvars["support_auth_token"]

Moving a bit down on the script is the initial interaction with a csv file. We will use the CSV library to create our file and write a few headers that will soon be populated with relevant information.

info.py snippet

with open("device_info.csv", mode="w", encoding="utf-8") as csv_file:
    fieldnames = [
        "Name",
        "Platform",
        "Model",
        "Base_PID",
        "Replacement",
        "Serial",
        "EoS",
        "EoSM",
        "LDoS",
        "EoCR",
        "Manufacture Year",
        "Current SW",
        "Recommended SW",
    ]
    writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
    writer.writeheader()

Nornir and NAPALM

This script utilizes Nornir and NAPALM for the initial device connection. This is only used for a very small run to use a NAPALM getter. In this case we are using the “get_facts” getter. This will retrieve a ton of useful information. For my case I will be capturing the hostname, model, version, serial, and platform. Snippet below of this process.

info.py snipper

def all_the_things(task):
    """It does all the things"""
    task1_result = task.run(
        name=f"Get facts for {task.host.name}!", task=napalm_get, getters=["get_facts"]
    )

    name = task1_result[0].result["get_facts"]["hostname"]
    model = task1_result[0].result["get_facts"]["model"]
    ver = task1_result[0].result["get_facts"]["os_version"]
    version = ver.split(",")[1]
    serial = task1_result[0].result["get_facts"]["serial_number"]
    platform = task1_result[0].result["get_facts"]["vendor"]

Support APIs and Functions

Now that we have the device serial number and model, we can utilize the support APIs to get any other relevant information. When I initially started getting my thoughts into code, the script started getting rather long. Too long for the actual process I was implementing. I eventually split out the API calls to functions within the “tools.py” file. Remember all those imports? I’ll go over one that sets the credentials (thanks Kirk Byers) and one that interacts with the support APIs. The support APIs all follow the same general flow.

I was having a bit of trouble elegantly setting the device username and password without being in any file. I eventually saw a simple function by Kirk, one of the creators of Nornir.

info.py main

def main():
    """Used to run all the things"""
    norn = InitNornir(config_file="config/config.yaml")
    nornir_set_creds(norn)
    norn.run(task=all_the_things)
# Nornir is initialized as "norn"
# The "norn" object is then passed into the nornir_set_creds function

The version I went with will look for environment variables to be set, you could set this to prompt as well but I got tired of typing.

Setting Credentials

def nornir_set_creds(norn, username=None, password=None):
    """
    Handler so credentials are not stored in cleartext.
    Thank you Kirk!
    """
    if not username:
        username = os.environ.get("NORNIR_USER")
    if not password:
        password = os.environ.get("NORNIR_PASS")

    for host_obj in norn.inventory.hosts.values():
        host_obj.username = username
        host_obj.password = password

I’ll do my best to breakdown the most involved function for the support APIs, EoX! This API returns so much useful information that I made the function just return a dictionary of useful things that can be passed to our CSV. The function takes a serial number as the first parameter and a token as the second. These values will then be passed to the headers of the API call as well as the URL. I run a simple check to see if there’s even been an announcement for that serial number. If not, then set all the values to “N/A”. It gets really interesting after the else statement. You probably notice “Dq” all over the place. “Dq”, short for dictionary query, allows you to query a dictionary in really useful ways. This tool comes from the popular Genie library. In all honesty, I just really wanted to play with it! It really does make finding relevant information in a dictionary a lot shorter. I’ll give you an example after the script.

eox function

def eox(serial: str, token: str):
    """Returns all eox information and replacement option from serial number"""
    device = {}
    headers = {"Authorization": f"Bearer {token}"}
    url = f"https://api.cisco.com/supporttools/eox/rest/5/EOXBySerialNumber/1/{serial}"
    req = requests.request("GET", url, headers=headers)
    content = req.json()
    if content["EOXRecord"][0]["EOXExternalAnnouncementDate"]["value"] == "":
        device["eos"] = "N/A"
        device["eosm"] = "N/A"
        device["ldos"] = "N/A"
        device["eocr"] = "N/A"
        device["replacement"] = "N/A"
    else:
        device["eos"] = Dq(content).contains("EndOfSaleDate").get_values("value", 0)
        device["eosm"] = (
            Dq(content).contains("EndOfSWMaintenanceReleases").get_values("value", 0)
        )
        device["ldos"] = (
            Dq(content).contains("LastDateOfSupport").get_values("value", 0)
        )
        device["eocr"] = (
            Dq(content).contains("EndOfServiceContractRenewal").get_values("value", 0)
        )
        device["replacement"] = (
            Dq(content)
            .contains("MigrationProductId")
            .get_values("MigrationProductId", 0)
        )
    return device

Dq Example

Lets say we had the following dictionary to parse through. Our goal is to get the state value of the “10.10.10.10” OSPF neighbor.

Example dictionary

>>> device = {
...     "router": {
...         "ospf": {
...             "neighbors": [
...                 {
...                     "10.10.10.10": {
...                         "state": "up"
...                     }
...                 },
...                 {
...                     "20.20.20.20": {
...                         "state": "down"
...                     }
...                 }
...             ]
...         }
...     }
... }
>>> device["router"]["ospf"]["neighbors"][0]["10.10.10.10"]["state"]
'up'

What you see above is one way of doing it. Mind you, in my example this dictionary is only going so many levels deep. Imagine if you were going down a deeply nested dictionary. Below is how you would do the same using Dq.

Dq Example

>>> Dq(device).contains("10.10.10.10").get_values("state")
['up']
>>> # Notice that Dq returns a list, below is how to get only the string
>>> Dq(device).contains("10.10.10.10").get_values("state", 0)
'up'

Send Data to CSV

Now that we execute all of our functions to hit the support APIs, we have all the information we need to build our CSV. At the top we are setting variables to the result of what is returned from our functions. Some functions only require a serial, like calculating the manufacture year, and others require a serial number and token. After that we use the CSV library to write all the data to the file we created earlier, in this call we are opening the file in append mode with the mode=’a' set.

info.py snippet

    manu_year = manu_year_cisco(serial)
    base_pid = product_info(serial, my_token)
    upgrade = software_release(base_pid, my_token)
    my_eox = eox(serial, my_token)

    with open("device_info.csv", mode="a", encoding="utf-8") as myfile:
        write = csv.DictWriter(myfile, fieldnames=fieldnames)
        write.writerow(
            {
                "Name": name,
                "Model": model,
                "Current SW": version,
                "Serial": serial,
                "Platform": platform,
                "Manufacture Year": manu_year,
                "Base_PID": base_pid,
                "Recommended SW": upgrade,
                "EoS": my_eox["eos"],
                "EoSM": my_eox["eosm"],
                "LDoS": my_eox["ldos"],
                "EoCR": my_eox["eocr"],
                "Replacement": my_eox["replacement"],
            }
        )

Example CSV

Below is a snip from the CSV example in the repo. If you want to test this out, feel free to clone down the repository and modify your hosts file.

Outro and Links

Thank you all for reading this far! This was a really fun tool to create and allowed me to learn a bit about the support APIs, working with CSVs (with Python), and using Dq! I hope you found this even a bit useful. Something that could be a possibility with a script like this is populating a tool like Netbox!

Converting Network Icons for Labs With Python

Fri, 17 Sep 2021 09:31:09 -0800

Introduction

First of all, the diagrams used in this post and repository were originally created by ecceman. Please head over to the original repository and show some love by giving it a star! When going through studies or testing a solution, I tend to lab a lot. On top of that I usually try and make my labs and topologies look pretty. I like to tell myself if the topology looks good, then I will learn more effectively. Example of the icons are above in the featured image.

I found eccemans repository after watching this video from The Network Berg. Check out the video as they demonstrate the proper folder location where images can be uploaded on an EVE-NG server. I’ll also post a small example in this post. The original files are in SVG format, whereas we need PNG files for EVE-NG.

Manual Conversion

At first the workflow would be something like the following:

Clone the repository
Convert image from SVG to PNG
Right size image to 52×52

Imagine repeating this process 10 times or even a couple hundred times. You can see how this would be pretty annoying after a while. I recall someone mentioning to me once, the best way to learn a programming language or any tool is to actually put it to use. Force the mind into a real world problem and use what has been learned to develop a solution.

Automated Conversion

I’ll be completely honest, some of the libraries used were a first for me. But that points to the power of programming languages. Once you learn the basics, with a bit of reading and examples, one can develop very strong solutions. I’ll try and walkthrough the process of developing a simple python script that became really helpful for me.

Research

I started where most of us start. Just heading to a web browser and typing “how to convert SVG to PNG using python”. While there were a lot of decent examples, folks new to programming may find some of them overwhelming. Luckily for you and I, stack overflow is a thing and the community usually post awesome answers to questions like the one I had. If you would like to check out the post on stack, check it out here!

libvips and pyvips

I wont go too much into detail on these libraries. The short of it is, libvips is an image processing library that can do a heck of a lot more than what I’m using it for. If you would like to check out their docs, click here. I like to think of pyvips as an entry point that leverages libvips under the hood. Write program in python and then it will call functions from libvips to execute.

sudo apt-get install libvips
pip3 install pyvips

Interating Over Folders and Files

Now that I had tools that looked promising for our end goal, I needed a way to iterate over all folders and SVG files within them. Initially I figured the script could just create a PNG folder under the shape and color. This way users could go to svg > circle > blue > PNG, and all the PNG files would be available for circle icons that were blue. Below is an example of what I had to work with. Please note this is just a snippet.

SVG Folder Structure

├── svg
│   ├── circle
│   │   ├── blue
│   │   │   ├── c_bug.svg
│   │   │   ├── c_camera_blue.svg
│   │   │   ├── c_camera_dome_blue.svg
│   │   ├── gray
│   │   │   ├── c_bug.svg
│   │   │   ├── c_camera.svg
│   │   │   ├── c_camera_dome.svg
│   │   ├── green
│   │   │   ├── c_bug_green.svg
│   │   │   ├── c_camera_dome_green.svg
│   │   │   ├── c_camera_green.svg
│   │   └── red
│   │       ├── c_bug_red.svg
│   │       ├── c_camera_dome_red.svg
│   │       ├── c_camera_red.svg
│   ├── naked
│   │   ├── bug.svg
│   │   ├── camera.svg
│   │   ├── camera_dome.svg
│   └── square
│       ├── blue
│       │   ├── sq_bug_blue.svg
│       │   ├── sq_camera_blue.svg
│       │   ├── sq_camera_dome_blue.svg
│       ├── gray
│       │   ├── sq_bug.svg
│       │   ├── sq_camera.svg
│       │   ├── sq_camera_dome.svg
│       ├── green
│       │   ├── sq_bug_green.svg
│       │   ├── sq_camera_dome_green.svg
│       │   ├── sq_camera_green.svg
│
│       └── red
│           ├── sq_bug_red.svg
│           ├── sq_camera_dome_red.svg
│           ├── sq_camera_red.svg

Iterating over this structure initially seemed very complex but python makes this fairly simple. I leveraged “walk” from “os”. This was a bit of trial and error but when I was able to print the correct files, I could proceed with converting them. Small example below of printing root and files. I pass in a PATH variable to walk as a starting point.

>>> from os import walk
>>> PATH = "svg/"
>>> for (root, dirs, files) in walk(PATH):
...     for f in files:
...         if ".svg" in f:
...             print(f"{root}/{f}")
...
svg/circle/blue/c_bug.svg
svg/circle/blue/c_camera_blue.svg
svg/circle/blue/c_camera_dome_blue.svg
svg/circle/blue/c_client_blue.svg
svg/circle/blue/c_client_vm_blue.svg

If you notice, root is the same for all the files shown above, “svg/circle/blue/”. All that I needed now from my initial thinking was to add a PNG folder at the end of that root. Then all the shapes and colors would have a PNG folder for the images to be converted. Please see one solution below.

Creating PNG Folder

try:
    mkdir(path=f"{root}/png/")
except OSError as error:
    pass

This isn’t the best solution at all or even the most efficient. This is basically using “mkdir” to create a directory under the root and adding png. The negative here is that it will throw an error after the first time it is created. I just handle it by setting the script to continue after running into this error.

Converting All the Things

The conversion wasnt too bad at all. I’ll explain the code upfront this time. pyvips has a function to create a thumbnail based off of a file. In this case we are looping over every single file that ends with “.svg”. We size the thumbnail as a height and width of 52. I believe these are recommended from the EVE-NG import icons guide. After that we write the file, when doing this we will place each new file under the root we are iterating over, for example “svg/circle/blue/”, appending “/png/” and ending it by changing the extension name from “svg” to “png”. Clear as mud right?

Itsy Bitys Function

def main():

    """
    All the action
    """

    for (root, dirs, files) in walk(PATH):
        for f in files:
            if ".svg" in f:
                try:
                    mkdir(path=f"{root}/png/")
                except OSError as error:
                    pass
                convert_file = pyvips.Image.thumbnail(f"{root}/{f}", 52, height=52)
                convert_file.write_to_file(f"{root}/png/{f.replace('svg', 'png')}")

Importing to EVE-NG

At this point all the SVG files have been converted to PNG format. If you would like to add these icons to EVE-NG, you can use something like winscp or build some automation to send them to your EVE-NG server. The path you will need to store them in is “/opt/unetlab/html/images/icons/”.

Bonus, Automating the Reference File

Something that I thought was interesting, at the time there was no overall view of every icon. Or some type of directory to check out what each icon looked like, unless you opened each file individually. Since that time ecceman has added a nice PNG that displays all the icons.

Before this existed a fellow networker also had a similar idea. Many thanks to @pstavirs for the push!

I’ve worked with markdown before and adding image URLs. At this point we had every PNG file sized and ready to go. Why not just write a script that could build this markdown file for us. I ended up using “walk” again since I now needed to loop over every single PNG image. We decided on three icons per row, with name on the right side of the icon. I played with a bit of simple math in the loop to tell the program when to drop to the next line. If you have any questions on the process feel free to ping me. If you want to see the file in the raw click here, crazy right? Rendered version below!

ref_build.py

def main():

    """
    All the action
    """

    with open("reference.md", "w") as my_file:
        # Headers and settings alignments for file
        my_file.write("# Reference for Icons\n")
        my_file.write("\n")
        my_file.write("|Icon|Name|Icon|Name|Icon|Name|\n")
        my_file.write("|:---:|:---:|:---:|:---:|:---:|:---:|\n")
        for (root, dirs, files) in walk(PATH):
            count = 0
            # Iterating over all files with .png extension
            for f in files:
                if ".png" in f:
                    # Simple math checks on matches and reset
                    # counter on third entry
                    count += 1
                    if count == 1:
                        my_file.write(f"|![{f}]({root}/{f})|{f.replace('.png', '')}|")
                    if count == 2:
                        my_file.write(f"![{f}]({root}/{f})|{f.replace('.png', '')}|")
                    if count == 3:
                        my_file.write(f"![{f}]({root}/{f})|{f.replace('.png', '')}|\n")
                        count = 0
        my_file.write("\n")

Outro and Links

Thank you all for reading this far, it really means a lot. This was probably one of the most asked questions I get, “How do you get those icons in your topology?”. If you would like to add these images to your EVE instance, feel free to clone down the repository and add whatever you like. Putting what you learn to use has many advantages and this one was a lot of fun. Shout out to Jordi, Wasif, and Srivats. Thank you for being so positive in the community! I wish everyone all the best on your future studies and happy labbing!

Original repository from ecceman, amazing!
Fork of original repo to add conversion script and reference builder
Featured Image credit to Wasif Bhatti, thank you!

Integrating Nornir With FastAPI

Wed, 01 Sep 2021 18:52:28 -0800

Introduction

I recently saw a fellow engineer share an amazing article on Real Python. It was a post by Sebastián Ramírez, the creator of FastAPI. I’m fairly comfortable working with APIs but I’ve never even thought about making one. The prospect seems so daunting, especially when I’m still trying to master the foundational skills of python. FastAPI does a ton of heavy lifting to make this process incredibly easy to get started and build web APIs.

When learning something new I try and incorporate it to something that interest me. A decent amount of the time it relates to network engineering. Another example could be learning how to interact with a Pokemon API. I wanted to see if I could possibly integrate Nornir with FastAPI.

Objective

My initial thinking for this experiment was to use Nornir and NAPALM to connect to devices and retrieve information. If this could then be linked to separate URLs, that would make for a pretty slick way of interacting with networking devices. I really thought I would be making some script that was much much longer than what I actually implemented.

I wont go too deep into Nornir or FastAPI. The Real Python article linked is an amazing resource as well as the FastAPI documentation. I’ve written about Nornir before, and if you are curious about the structure, feel free to check out one of my previous posts.

Topology

The topology consists of three networking nodes from three popular vendors. I tried to include a variety to show how automation can remove some thought when trying to return data from devices. Whether its vendor A or vendor B, at the end of the day we want the information required for the task at hand.

Script Breakdown

All the Imports

from fastapi import FastAPI
from nornir import InitNornir
from nornir_napalm.plugins.tasks import napalm_get
from yaml import safe_load
import urllib3

FastAPI – Used to create a class instance of FastAPI.
InitNornir – Similar to FastAPI above, in this case Nornir will be initialized with a config file.
napalm_get – Heavily used to retrieve all the information in our API calls.
safe_load – Will be used to load hosts.yaml file (specific API call).
urllib3 – At the moment this will simply be disabling insecure warnings in the terminal output.

Inits

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

nr = InitNornir(config_file="config/config.yaml")

app = FastAPI()


@app.get("/")
async def root():
    """Says hi!"""
    return {"message": "Hello JulioPDX"}

As mentioned previously, we will be disabling the insecure warnings for our lab. We are initializing Nornir with the nr variable and FastAPI to the app variable. Something new is the ‘@app.get(“/”)’ on line eight. From the article this is known as a decorator in python and it relates to the function definition immediately below it. You will see this pattern further in this writing. For this example, if we run the app pointing to “/” it should return “Hello JulioPDX”. Lets start the app!

Starting the App

(venv) [juliopdx@pbpro net_auto_fastapi]$ uvicorn play:app --reload
INFO:     Will watch for changes in these directories: ['/home/juliopdx/git/net_auto_fastapi']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [99179] using watchgod
INFO:     Started server process [99184]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

Sticking close to the learning so far, uvicorn is used as our server to run the app. The reload option will restart the service anytime a change is made to our play.py file and saved. Lets check out the path mentioned in earlier code.

Now that I have my local environment to the point of the original article, I wanted to start with something simple to test my knowledge. Could there be a way to let users know what all devices will be loaded and available in Nornir? Check out the code below.

Get All the Things

@app.get("/devices")
async def get_devices():
    """Returns a list of devices loaded from our hosts file"""
    with open("./config/hosts.yaml", encoding="utf-8") as file:
        devices = safe_load(file)
    return {"devices": devices}

This is pretty neat, its a simple python operation to open a file and set the contents to a variable. Incorporating the output into FastAPI was incredibly easy. This will also show a user what devices they can work with at a simple URL or endpoint. Something else to note is that the credentials are not exposed to the user, only the portions within the hosts.yaml file.

This next portion was the most interesting to me. NAPALM has a concept called getters. Basically these functions will go and grab different pieces of information from a device. I initially thought I would have to write a function for every getter. Luckily for me and you viewers watching at home, we can use path parameters when defining these endpoints! For example, if I wanted to interact with the ArubaCX device and the facts getter, I would need to provide two parameters to the endpoint. This allows the operator to just enter the URL with the correct device name and getter, all in one function definition!

All the Getters

@app.get("/devices/{hostname}/napalm_get/{getter}")
async def get_config(hostname: str, getter: str):
    """Function used to interact with NAPALM and devices"""
    rtr = nr.filter(name=f"{hostname}")
    return rtr.run(name=f"Get {hostname} {getter}", task=napalm_get, getters=[f"{getter}"])

Outro and Links

Thank you for reading this far, really means a lot. So much of this can be improved and I’m excited to keep learning about python and FastAPI. Please check out the links below for more information.

My Journey Learning About Internet Exchange Points (IXPs)

Sun, 29 Aug 2021 18:04:42 -0800

Introduction

Hello all and thank you for checking out one of my blog posts. Really means a lot! Have you ever heard IXP mentioned in network engineering speak and thought, what the heck is that? I definitely have, and in my day to day I don’t usually interact with an IXP at all. So here I am a curious individual, wanting to learn the things. I figured, what the heck, now is as good a time as any to deep dive some IXP knowledge. Please note, what I breakdown in this post is just a sliver of what is done at an IXP or even at the internet service provider side.

What is an IXP?

In the absolute simplest terms… a managed switch. Seriously, that’s pretty much it. I’m only kidding a bit. IXPs allow internet service providers or content providers to have a general space to come together and peer with each other. Each participant will have a router at the IXP site and peer with every participant, route server, or a subset of the participants. Peering will be done using external BGP (eBGP). This can also be a combination of peering with a route server and a few participants. The cost to run an IXP is usually divided between the participants.

Why do we need them?

Imagine two internet providers with customers of their own in the west coast. The ISPs only peer with each other on the east coast. If customers on the west coast want to talk to each other, that traffic would have to travel from coast to coast just to end up back on the west coast. Not very efficient or economical. The ISPs could peer with each other at the west coast, but what if there’s another ISP or five? This would not scale very well and would become very costly. Below is a simple list of other IXP benefits:

Save money
Traffic stays local
Performance
Better customer experience

Our Topology

In the topology above we have a dual site IXP. Why a dual site? In cases where IXPs become very large, the requirement for a second site may arise. I also just wanted to play with a dual site topology for more learnings. Please note the IXP sites are not directly connected. Good for decreasing failure domain.

We have four ISPs, each ISP is advertising IPv4 and IPv6 networks. Every ISP has a three router setup. Routers ISP-X-1 and ISP-X-2 are edge nodes for the ISP and ISP-X-3 is internal to the ISP network (advertising the routes). I’ll mainly focus on ISP-A since all others ISPs have a very similar configurations, sub some IP settings. The IXP site also has a node acting as a route server/router collector. More on that in a bit!

IXP Network Addressing

Site	IXP1	IXP2
IPv4 Peering LAN	10.0.0.0/23	10.0.2.0/23
IPv6 Peering LAN	2001:db8::/64	2001:db8:0:1::/64
ASN	777(should be private ASN)	777 (should be private ASN)

I saw some examples mentioning the IXP LAN can be public addressing or private addressing. In my case I chose private addressing for IPv4 and the general documentation prefix for IPv6. The ASN at the IXP should be a private ASN. When I started this build I chose 777 because luck.

ISP Network Addressing

ISP	A	B	C	D
IPv4 Block	1.1.1.0/24	2.2.2.0/24	3.3.3.0/24	4.4.4.0/24
IPv6 Block	2001:db8:111::/48	2001:db8:222::/48	2001:db8:333::/48	2001:db8:444::/48
ASN	111	222	333	444

Lets just pretend in our fun world that these fictional ISPs own these public addresses. The ISP-X-3 router at each ISP site will be advertising the networks shown above.

ISP Topology

Each ISP has two edge routers and one router internal to the ISP autonomous system. Each ISP is running OSPFv3 to redistribute internal routes and addresses on loopback interfaces for BGP peering. The internal router, in this case ISP-A-3, acts as a route reflector. A-1 and A-2 do not directly peer with each other using BGP. They are neighbors using OSPFv3 IPv4 and IPv6 address families.

OSPFv3 Configuration

ISP-A-3 OSPFv3 Configuration

interface Loopback0
 ip address 1.1.1.1 255.255.255.0
 ipv6 address 2001:DB8:111::1/48
!
interface Loopback1
 ip address 192.168.1.3 255.255.255.255
 ipv6 address FE80::3 link-local
 ipv6 address 2001:DB8:1::3/128
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv4 area 0
!
interface Ethernet0/0
 description To ISP-A-1
 ip address 192.168.0.3 255.255.255.254
 ipv6 address FE80::3 link-local
 ospfv3 1 ipv4 area 0
 ospfv3 1 ipv6 area 0
 bfd interval 300 min_rx 300 multiplier 3
!
interface Ethernet0/1
 description To ISP-A-2
 ip address 192.168.0.5 255.255.255.254
 ipv6 address FE80::3 link-local
 ospfv3 1 ipv4 area 0
 ospfv3 1 ipv6 area 0
 bfd interval 300 min_rx 300 multiplier 3
!

router ospfv3 1
 router-id 0.0.0.3
 bfd all-interfaces
 !
 address-family ipv4 unicast
  passive-interface Loopback0
  passive-interface Loopback1
 exit-address-family
 !
 address-family ipv6 unicast
  passive-interface Loopback0
  passive-interface Loopback1
 exit-address-family
!

IPv4 addressing is the same across all ISPs in the topology as well as the link local addressing. We enable OSPFv3 under both address families and set loopbacks to passive in OSPFv3. I enabled BFD on pretty much any link I could to speed up failure detection, especially with BGP peers.

iBGP Configuration

ISP-A-3 iBGP Configuration

router bgp 111
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor iBGP peer-group
 neighbor iBGP remote-as 111
 neighbor iBGP update-source Loopback1
 neighbor iBGP fall-over bfd
 neighbor iBGPv6 peer-group
 neighbor iBGPv6 remote-as 111
 neighbor iBGPv6 update-source Loopback1
 neighbor iBGPv6 fall-over bfd
 neighbor 2001:DB8:1::1 peer-group iBGPv6
 neighbor 2001:DB8:1::2 peer-group iBGPv6
 neighbor 192.168.1.1 peer-group iBGP
 neighbor 192.168.1.2 peer-group iBGP
 !
 address-family ipv4
  network 1.1.1.0 mask 255.255.255.0
  neighbor iBGP route-reflector-client
  neighbor 192.168.1.1 activate
  neighbor 192.168.1.2 activate
 exit-address-family
 !
 address-family ipv6
  network 2001:DB8:111::/48
  neighbor iBGPv6 route-reflector-client
  neighbor 2001:DB8:1::1 activate
  neighbor 2001:DB8:1::2 activate
 exit-address-family

This configuration looks a bit long but lets break it down. We disable the default BGP for IPv4 setup. We will be using both address families for neighbor relationships. There is a peer group for IPv4 neighbors and IPv6 neighbors to cut down on commands. Especially if more neighbors were involved. After that we will advertise the routes we need and set the peers in the peer groups to be route reflector clients.

ISP-A-1 iBGP Configuration

router bgp 111
 neighbor 2001:DB8:1::3 remote-as 111
 neighbor 2001:DB8:1::3 update-source Loopback0
 neighbor 2001:DB8:1::3 fall-over bfd
 neighbor 192.168.1.3 remote-as 111
 neighbor 192.168.1.3 update-source Loopback0
 neighbor 192.168.1.3 fall-over bfd
 !
 address-family ipv4
  neighbor 192.168.1.3 activate
  neighbor 192.168.1.3 next-hop-self
  neighbor 192.168.1.3 route-map iBGP_IPv4_IN in
 exit-address-family
 !
 address-family ipv6
  neighbor 2001:DB8:1::3 activate
  neighbor 2001:DB8:1::3 next-hop-self
  neighbor 2001:DB8:1::3 route-map iBGP_IPv6_IN in
 exit-address-family

I’ll spare you from the configuration on ISP-A-2, it is so close to being a copy of ISP-A-1. Not much new here, just enabling the neighbor relationship with ISP-A-3. Something new is the route-map being applied inbound. One of the resources I used mentioned the ISP border routers at the IXP should, for the most part, not carry the full internet routing table. Lets say for example, ISPA wanted to advertise only networks 1.1.1.0/24 and 2001:db8:111::/48 at this IXP. I accomplished this by filtering all incoming updates from our iBGP neighbor to only accept those prefixes. See prefix lists and route maps below.

Inbound Prefix Lists and Route Maps

!
ip prefix-list INBOUND seq 5 permit 1.1.1.0/24
!
ipv6 prefix-list INBOUND_v6 seq 5 permit 2001:DB8:111::/48
!
route-map iBGP_IPv4_IN permit 10
 match ip address prefix-list INBOUND
!
route-map iBGP_IPv6_IN permit 10
 match ipv6 address prefix-list INBOUND_v6
!

eBGP Configuration

ISP-A-1 eBGP Configuration

router bgp 111
 bgp router-id 10.0.0.21
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor IXP peer-group
 neighbor IXP fall-over bfd
 neighbor IXPv6 peer-group
 neighbor IXPv6 fall-over bfd
 neighbor 10.0.0.1 remote-as 777
 neighbor 10.0.0.1 peer-group IXP
 neighbor 10.0.0.22 remote-as 222
 neighbor 10.0.0.22 peer-group IXP
 neighbor 10.0.0.23 remote-as 333
 neighbor 10.0.0.23 peer-group IXP
 neighbor 10.0.0.24 remote-as 444
 neighbor 10.0.0.24 peer-group IXP
 neighbor 2001:DB8::1 remote-as 777
 neighbor 2001:DB8::1 peer-group IXPv6
 neighbor 2001:DB8::222:1 remote-as 222
 neighbor 2001:DB8::222:1 peer-group IXPv6
 neighbor 2001:DB8::333:1 remote-as 333
 neighbor 2001:DB8::333:1 peer-group IXPv6
 neighbor 2001:DB8::444:1 remote-as 444
 neighbor 2001:DB8::444:1 peer-group IXPv6
 !
 address-family ipv4
  network 1.1.1.0 mask 255.255.255.0
  neighbor IXP send-community both
  neighbor IXP soft-reconfiguration inbound
  neighbor IXP route-map IXP_IPv4_OUT out
  neighbor 10.0.0.1 activate
  neighbor 10.0.0.22 activate
  neighbor 10.0.0.23 activate
  neighbor 10.0.0.24 activate
 exit-address-family
 !
 address-family ipv6
  network 2001:DB8:111::/48
  neighbor IXPv6 send-community both
  neighbor IXPv6 soft-reconfiguration inbound
  neighbor IXPv6 route-map IXP_IPv6_OUT out
  neighbor 2001:DB8::1 activate
  neighbor 2001:DB8::222:1 activate
  neighbor 2001:DB8::333:1 activate
  neighbor 2001:DB8::444:1 activate
 exit-address-family
!

I’ll admit the eBGP configuration is a bit more busy. Here we have peer groups for both IPv4 (IXP) and IPv6 (IXPv6). Sending community attributes for potential traffic engineering or policies. Something that I should mention for more official implementation, authentication between BGP peers would be great as well as filtering per neighbor. For example, if we were to peer with 10.0.0.22 and expect to receive the 2.2.2.0/24 network. We could add a filter to only accept that network inbound from 10.0.0.22. I created a simple filter to only filter the routes we are sending to our eBGP neighbors. Example is below.

Outbound Prefix Lists and Route Maps

!
ip prefix-list OUTBOUND seq 5 permit 1.1.1.0/24
!
ipv6 prefix-list OUTBOUND_v6 seq 5 permit 2001:DB8:111::/48
route-map IXP_IPv4_OUT permit 10
 match ip address prefix-list OUTBOUND
!
route-map IXP_IPv6_OUT permit 10
 match ipv6 address prefix-list OUTBOUND_v6
!

IXP Switching

I did not go deep in the reading for IXP hardware. My best thought would be a pretty decent switch with speeds up to 400G at this point. I’m sure there are a lot of factors that go into these decisions. Check out the IXP wish list from euro-ix linked at the end.

I’m going to turn our attention to the actual switch configuration. This is again only a sliver of what should be configured on the switch. Below is what I came up with in this lab scenario.

IXP Switchport Configuration

!
interface Ethernet0/0
 description To ISP-A-1
 switchport access vlan 1234
 switchport mode access
 switchport port-security
 switchport port-security violation restrict
 switchport port-security mac-address sticky
 switchport port-security mac-address sticky aabb.cc00.0310
 duplex auto
 no cdp enable
 spanning-tree portfast
 spanning-tree bpdufilter enable
!
interface Ethernet1/0
 description To RS1
 switchport access vlan 1234
 switchport mode access
 switchport port-security
 switchport port-security violation restrict
 switchport port-security mac-address sticky
 switchport port-security mac-address sticky 5000.000b.0001
 duplex auto
 spanning-tree portfast
 ip dhcp snooping trust
!

I included just the ports to ISP-A-1 and RS1 for brevity and save your eyes a bit. VLAN 1234 will be used for peering. Portfast is enabled to bring up the interfaces quicker. Port security and MAC address sticky is used to prevent a rogue device from being connected and trying to join our peering network. In our case the single MAC will be remembered even on a switch reboot. I set the mode to restrict in this lab, this should allow the port to stay up and packets from rogue MAC addresses will be dropped. Alerts can then be sent to a network management system. I disabled CDP just in case some rogue device gets access, they at least wont know what network the neighbor device is on. BPDU filter is on just in case some rogue switch is connected and prevent things from going horribly wrong. All other interfaces not in use are shut down.

IXP Services

Apart from giving ISPs or content providers the ability to peer with each other. IXPs can offer several additional services:

Country Code Top Level Domain – Ability to host country’s top level DNS
Root Servers – Used to reduce latency of DNS resolutions
Route Collector – Ability to view the routing information at the IXP
Looking Glass – Public or members only view of the Route Collector routing information
NTP – Possibility to host a stratum 1 time source

Route Collector

The services mentioned above are only a handful of what an IXP can provide. I’m going to focus on one in particular, the Route Collector. An IXP can have a policy for each participant that they must peer with a local route collector. The collector has a simple function, receive all the routes, and distribute none. You may have noticed two nodes in the topology named RS1 and RS2. The name may be misleading as it can stand for route server, but for our purposes just think of them as a route collector. If you are wondering, route servers can make BGP configuration a bit more simple for ISPs at an IXP since they only have to peer with the route server vs each individual participant. This has a possible negative of the ISP losing a bit of policy control with BGP.

Our route collectors are Cumulus Linux VMs running a neat BGP setup. I’m not entirely sure on the correct way to setup the route collector but this made sense to me. Since the route collector cannot forward any routes that are learned, we need to create a policy. The most simple thing that came to mind was to create a general deny all route map and apply it to our peer group. Check it out below. I’ll focus on RS1 as RS2 has a very very similar config. So much so that I basically ran “net show configuration commands”, copied the output, updated about two addresses, and then pasted them over to RS2.

RS1 BGP Configuration

router bgp 777
  no bgp default ipv4-unicast
  neighbor IXP peer-group
  neighbor IXP remote-as external
  neighbor IXP bfd 3 300 300
  neighbor IXPv6 peer-group
  neighbor IXPv6 remote-as external
  neighbor IXPv6 bfd 3 300 300
  bgp listen limit 90
  bgp listen range 10.0.0.0/23 peer-group IXP
  bgp listen range 2001:db8::/64 peer-group IXPv6

  address-family ipv4 unicast
    neighbor IXP activate
    neighbor IXP soft-reconfiguration inbound
    neighbor IXP route-map RS_DENY out
    neighbor IXP send-community both

  address-family ipv6 unicast
    neighbor IXPv6 activate
    neighbor IXPv6 soft-reconfiguration inbound
    neighbor IXPv6 route-map RS_DENY out
    neighbor IXPv6 send-community both

route-map RS_DENY deny 1

Okay so the config was small enough I just included the whole thing. At the bottom you can see my generic deny route map. Check out those bgp listen commands, I think its the coolest thing. In my little IXP world, I didn’t want to keep manually adding all the BGP peers to the route collector. So the route collector is listening for any peer requests that come from our peering LAN. Probably not the most secure, I would recommend reading up on the best way to set that up.

Validate

Lets check some neighbors and routes!

ISP-A-3 OSPFv3 Neighbors

ISP-A-3#show ospfv3 neighbor

          OSPFv3 1 address-family ipv4 (router-id 0.0.0.3)

Neighbor ID     Pri   State           Dead Time   Interface ID    Interface
0.0.0.2           1   FULL/BDR        00:00:35    5               Ethernet0/1
0.0.0.1           1   FULL/BDR        00:00:37    5               Ethernet0/0

          OSPFv3 1 address-family ipv6 (router-id 0.0.0.3)

Neighbor ID     Pri   State           Dead Time   Interface ID    Interface
0.0.0.2           1   FULL/BDR        00:00:34    5               Ethernet0/1
0.0.0.1           1   FULL/BDR        00:00:33    5               Ethernet0/0

ISP-A-3 BGP Neighbors

ISP-A-3#show bgp ipv4 unicast summary | b Nei
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.1.1     4          111    1439    1429       14    0    0 21:30:04        3
192.168.1.2     4          111    1370    1373       14    0    0 20:36:57        3
ISP-A-3#show bgp ipv6 unicast summary | b Nei
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
2001:DB8:1::1   4          111    1474    1424       43    0    0 21:11:37        3
2001:DB8:1::2   4          111    1486    1440       43    0    0 21:11:36        3
ISP-A-3#

ISP-A-3 BGP Networks

ISP-A-3#show bgp ipv4 unicast | b Net
     Network          Next Hop            Metric LocPrf Weight Path
 *>  1.1.1.0/24       0.0.0.0                  0         32768 i
 * i 2.2.2.0/24       192.168.1.2              0    100      0 222 i
 *>i                  192.168.1.1              0    100      0 222 i
 * i 3.3.3.0/24       192.168.1.2              0    100      0 333 i
 *>i                  192.168.1.1              0    100      0 333 i
 * i 4.4.4.0/24       192.168.1.2              0    100      0 444 i
 *>i                  192.168.1.1              0    100      0 444 i
ISP-A-3#show bgp ipv6 unicast | b Net
     Network          Next Hop            Metric LocPrf Weight Path
 *>  2001:DB8:111::/48
                       ::                       0         32768 i
 * i 2001:DB8:222::/48
                       2001:DB8:1::2            0    100      0 222 i
 *>i                  2001:DB8:1::1            0    100      0 222 i
 * i 2001:DB8:333::/48
                       2001:DB8:1::2            0    100      0 333 i
 *>i                  2001:DB8:1::1            0    100      0 333 i
 * i 2001:DB8:444::/48
                       2001:DB8:1::2            0    100      0 444 i
 *>i                  2001:DB8:1::1            0    100      0 444 i
ISP-A-3#

RS1 BGP Neighbors

cumulus@RS1:mgmt:~$ net show bgp summary
show bgp ipv4 unicast summary
=============================
BGP router identifier 10.0.0.1, local AS number 777 vrf-id 0
BGP table version 72
RIB entries 7, using 1344 bytes of memory
Peers 4, using 85 KiB of memory
Peer groups 2, using 128 bytes of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
*10.0.0.21      4        111     27760     28419        0    0    0 23:41:01            1        0
*10.0.0.22      4        222     63678     65191        0    0    0 2d06h19m            1        0
*10.0.0.23      4        333     63678     65191        0    0    0 2d06h19m            1        0
*10.0.0.24      4        444     28302     28971        0    0    0 1d00h08m            1        0

Total number of neighbors 4
* - dynamic neighbor
4 dynamic neighbor(s), limit 90


show bgp ipv6 unicast summary
=============================
BGP router identifier 10.0.0.1, local AS number 777 vrf-id 0
BGP table version 49
RIB entries 7, using 1344 bytes of memory
Peers 4, using 85 KiB of memory
Peer groups 2, using 128 bytes of memory

Neighbor         V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
*2001:db8::111:1 4        111     27760     28417        0    0    0 23:40:55            1        0
*2001:db8::222:1 4        222     63716     65191        0    0    0 2d06h19m            1        0
*2001:db8::333:1 4        333     63715     65191        0    0    0 2d06h19m            1        0
*2001:db8::444:1 4        444     63714     65191        0    0    0 2d06h19m            1        0

Total number of neighbors 4
* - dynamic neighbor
4 dynamic neighbor(s), limit 90


show bgp l2vpn evpn summary
===========================
% No BGP neighbors found
cumulus@cumulus:mgmt:~$

RS1 BGP Routes

cumulus@cumulus:mgmt:~$  net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 72, local router ID is 10.0.0.1, vrf id 0
Default local pref 100, local AS 777
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       10.0.0.21                              0 111 i
*> 2.2.2.0/24       10.0.0.22                              0 222 i
*> 3.3.3.0/24       10.0.0.23                              0 333 i
*> 4.4.4.0/24       10.0.0.24                              0 444 i

Displayed  4 routes and 4 total paths


show bgp ipv6 unicast
=====================
BGP table version is 49, local router ID is 10.0.0.1, vrf id 0
Default local pref 100, local AS 777
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2001:db8:111::/48
                    fe80::a8bb:ccff:fe00:310
                                                           0 111 i
*> 2001:db8:222::/48
                    fe80::a8bb:ccff:fe00:510
                                                           0 222 i
*> 2001:db8:333::/48
                    fe80::a8bb:ccff:fe00:710
                                                           0 333 i
*> 2001:db8:444::/48
                    fe80::a8bb:ccff:fe00:810
                                                           0 444 i

Displayed  4 routes and 4 total paths
cumulus@cumulus:mgmt:~$

Outro and Links

Thank you again for checking out this post, I really appreciate it. Please send over any tips or just ping me if something I mentioned is way out into left field. If you are curious about IXPs, please check out the links below! I will include all configurations for the devices in the topology on my GitHub (linked below). Thank you to all the IXP operators around the world!

Basic Network Testing With Nornir and NTC Templates

Sun, 15 Aug 2021 11:12:03 -0800

Introduction

Hello everyone and thank you for checking out another blog post. This time looking at testing the network. In my previous post, I mentioned testing could be a really neat and much needed addition to the code base. I had some spare time this weekend and couldn’t wait. Ill be honest, I consider myself a novice in python and programming in general. There is definitely many ways to accomplish a task and I’m sure there are plenty of more efficient ways to perform the actions I’ll share with you.

The Road to Network Testing

Initially I wanted to incorporate pytest for this weekend project. It generally made sense and experienced folks working in python use this to test anything from simple functions to extremely large code bases. I ran into a few issues when trying to incorporate pytest. Again, I’m fairly inexperienced in this realm and I’m sure a more experienced person would not run into these issues. The main issue I had with pytest was jobs failing after the first failed assert. Even when explicitly setting the –maxfail option. Overall I thought working with pytest was really fun and I cant wait to dig into it more.

Phase two of network testing had me working with Nornir and Netmiko to send a command, set use_textfsm=True, and then run a few if statements to check matching configuration. I’ll include a portion of what that looked like below.

Nornir and Netmiko

result = nornir.run(
    name="GET VRF",
    task=netmiko_send_command,
    command_string="show vrf",
    use_textfsm=True,
)

for host in nornir.inventory.hosts.keys():
    vrf_parsed = result[host].result
    list_of_vrf_values = [value for elem in vrf_parsed for value in elem.values()]

    print()

    for value in main_vrfs:
        if value in list_of_vrf_values:
            pretty(f"[bold blue]{value} VRF is configured on {host}[/bold blue]")
        else:
            pretty(f"[bold red]{value} VRF is not configured on {host}[/bold red]")

This approach generally worked but I had an issue in my lab. I’m not sure if it was a Netmiko issue or just my lab being weird and getting connection timeouts. In the end I decided to use Nornir with Scrapli to send the commands. Connections when using Scrapli seemed to just work, great job Carl! Once this was retrieved, I could then use the parse_output function from the ntc_templates library. Feel free to check out the repository with the test script (linked below).

VRF Validation

Imports

import os
import yaml
from rich import print as pretty
from nornir import InitNornir
from nornir_scrapli.tasks import send_command
from ntc_templates.parse import parse_output

The imports are nothing fancy. Just importing everything we will need for the script.

os – Set environemt variables during script run
yaml – Handle interaction with yaml stuffs
rich – Because rich, thanks Will
InitNornir- Run all the things for Nornir
send_command – To send commands 🙂
parse_output – Great library for turning output into structured data

load yaml

main_vrfs = []

# Create list of VRFs defined in groups.yaml
with open("groups.yaml") as f:
    data = yaml.load(f, Loader=yaml.FullLoader)["pe"]["data"]["vrfs"]
    for vrf in data:
        main_vrfs.append(vrf["name"])

I thought this was pretty slick. We are opening the groups.yaml file seen from the previous post. This file lists all of our VRFs to be configured on the PE devices. Why not reuse it and save some typing? Once that is done we set the VRF portion of the yaml file to a variable called data. After this we loop over all the VRFs and add just the name to our empty list above. Hold that thought as this list will be used later in the script.

test_vrf function intro

def test_vrf():

    """
    Main VRF test
    """
    os.environ[
        "NET_TEXTFSM"
    ] = "./venv/lib/python3.9/site-packages/ntc_templates/templates"

    nornir = InitNornir(config_file="config.yaml")

    result = nornir.run(
        name="GET VRF",
        task=send_command,
        command="show vrf",
    )

If any of you have worked with pytest, you can tell by the function naming that I was serious about incorporating it. I will in the future! Most of this has been shown in previous posts but one little difference is that os.environ section. When I was building this, the script couldn’t locate the ntc_templates. Essentially, we are just telling the script where to find the templates. You will probably have to update this for your environment.

For all the hosts

    for host in nornir.inventory.hosts.keys():
        remote_vrfs = []
        vrf_parsed = parse_output(
            platform="cisco_ios", command="show vrf", data=result[host].result
        )

        # Looping over all parsed VRFs to build new list
        for remote_vrf in vrf_parsed:
            remote_vrfs.append(remote_vrf["name"])

I really love this about python and programming in general, what you see above is fairly basic and learned pretty early on in programming studies. But the power it can demonstrate is really neat to me. At this point in the script, all hosts have been connected to and information captured. We are using this initial for loop to loop over the data that was gathered from the show vrf command. We create an empty list called remote_vrfs, then run the parse_output function from the ntc_templates library to get data in a structured format. Once that is done we run another for loop to add VRF names gathered from PE routers to our previously empty list. At this point we now have two lists created. One for VRFs we want configured on devices and one from the actual VRFs already configured on the PE routers.

Set Theory

        set_diff = set(main_vrfs) - set(remote_vrfs)

        # If a set difference is found, print the nice red output!
        if set_diff:
            pretty(
                f"[bold red]The following VRFs are not configured on {host}: {set_diff [/bold red]"
            )
        else:
            pretty(f"[bold blue]All VRF tests passed on router {host}[/bold blue]")

Good old set theory to the rescue. I learned this a while back from a course by… who other than Nick Russo. Discrete mathematics in high school may have helped but that was way too long ago. We take the two lists we have already created, then convert them to sets. From there we just look for the difference between main_vrfs set and the remote_vrfs set. Below is an example on the interpreter of what I hope to accomplish.

>>> main_vrfs = {"CUSTOMER_777","CUSTOMER_789"}
>>> remote_vrfs = {"CUSTOMER_777","CUSTOMER_789","MGMT"}
>>> main_vrfs - remote_vrfs
set()
>>>

Script Output

Outro and Links

Thank you for staying this long. I hope you enjoyed the post and found something useful. As you can see I have a long way to go with network testing but it really is an important skill. I’ll include some links below!

Automating MPLS L3VPNs With Nornir

Sat, 14 Aug 2021 10:38:25 -0800

Introduction

Hello all and thank you for checking out another one of my blog posts. It really means a lot. I recently completed the Cisco ENARSI exam after a few months of studying. It was honestly pretty difficult for me but the pass was worth it! I may have to write a blog post on studying for and passing that exam. In this post I will break down how I automated some parts of an MPLS L3VPN deployment. Configuring the provider edge devices to allow connectivity between customer edge routers.

In serious news, that was the intro you see in those recipe websites where you just want to see the ingredients and the cooking directions but there’s a whole life story on the dish or how it came about…. well this blog post is similar! Usually when studying for an exam I tend to get tunnel vision and focus on the task at hand. That also had me dropping the love for automation a bit. One of the main points in ENARSI is learning about MPLS and more specifically MPLS L3VPNs. I had such a joy learning more about this topic and running through multiple labs to get used to the technology. I wont go too deep into MPLS in this post since I’m assuming the reader is trying to automate that exact task or is familiar with it already.

Topology Brief

As you can see by the featured image in this post, we have four provider edge (PE) routers and four customer edge (CE) routes. Each PE device connects to one CE device, and all PE routers connect to the provider (P) router, also acting as our BGP route reflector. You might be thinking, why the route reflector? You could run iBGP sessions between all PE devices but down the road that can lead to a lot of overhead. In this case all PE devices only have one internal BGP relationship between the P router and one external BGP peering with the CE routers. This also makes the iBGP configuration on the PE routers very cookie cutter. In this case we are pretending not to have control of the CE devices, so those will already be preconfigured. I will include the configurations on the nodes so you can see what is all preconfigured.

Ce Routers Configuration Example

interface Loopback0
 ip address 172.16.1.1 255.255.255.0
!
interface Ethernet0/0
 ip address 10.0.11.2 255.255.255.0
!

router bgp 789
 bgp log-neighbor-changes
 network 172.16.1.0 mask 255.255.255.0
 neighbor 10.0.11.1 remote-as 12345
 neighbor 10.0.11.1 allowas-in

Every CE router looks very similar, one lookback interface to be advertised to the PE router and a small bit of BGP configuration. You could use different autonomous system (AS) numbers between customer routers, in this case I used the same AS for the same customer. In BGP this would introduce a loop since the router will see its own AS in the path. This design option adds the command of allowas-in to our BGP configuration. This feature allows a route to be received even if the routers own AS is in path.

PE Routers Configuration Example

interface Ethernet0/1
 ip address 10.0.15.1 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!

router ospf 1
 router-id 1.1.1.1
 network 1.1.1.1 0.0.0.0 area 0
 network 10.0.15.1 0.0.0.0 area 0
!
router bgp 12345
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 5.5.5.5 remote-as 12345
 neighbor 5.5.5.5 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 5.5.5.5 activate
  neighbor 5.5.5.5 send-community extended
 exit-address-family
 !
!

Internally we are running OSPF to give MPLS all the paths to work with. Every PE device has essentially the same iBGP configuration towards the P router. If you look closely we are not even bringing up the peering relationship with the IPv4 address family. Only the VPNv4 address family will be used. This is due to the BGP peering only being used to distribute the VPNv4 routes across the PE routers.

P/RR Router Configuration Example

interface Loopback0
 ip address 5.5.5.5 255.255.255.255
!
interface Ethernet0/0
 ip address 10.0.15.5 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface Ethernet0/1
 ip address 10.0.25.5 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface Ethernet0/2
 ip address 10.0.35.5 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
interface Ethernet0/3
 ip address 10.0.45.5 255.255.255.0
 ip ospf network point-to-point
 mpls ip
!
router ospf 1
 router-id 5.5.5.5
 network 0.0.0.0 255.255.255.255 area 0
!
router bgp 12345
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor iBGP peer-group
 neighbor iBGP remote-as 12345
 neighbor iBGP update-source Loopback0
 neighbor 1.1.1.1 peer-group iBGP
 neighbor 2.2.2.2 peer-group iBGP
 neighbor 3.3.3.3 peer-group iBGP
 neighbor 4.4.4.4 peer-group iBGP
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor iBGP send-community extended
  neighbor iBGP route-reflector-client
  neighbor 1.1.1.1 activate
  neighbor 2.2.2.2 activate
  neighbor 3.3.3.3 activate
  neighbor 4.4.4.4 activate
 exit-address-family
!

The P router configuration does look a bit more involved but in reality its not too bad. In the case of the P router, I just advertised all the things in OSPF for simplicity. As I mentioned earlier with the PE routers, only the VPNv4 peering will be used. I created a peer group to bring down the number of commands required to make this all work.

Introduction to Nornir

I will break down the structure of the repository I made but I wont go too deep into details. Mainly because there’s a lot of great resources out there to check out (will link at the bottom of the post), and I want to try and keep this post some what short. Nornir is a network automation framework that is written in Python. This allows developers and engineers to build automation and tools with a very powerful programming language. This also allows individuals to easily extend the functionality of the framework to meet their needs.

config.yaml

---
inventory:
  plugin: SimpleInventory
  options:
    host_file: "hosts.yaml"
    group_file: "groups.yaml"
runner:
  plugin: threaded
  options:
    num_workers: 10

The config.yaml file is pretty important in Nornir. In the sample above we are using the SimpleInventory plugin and specifying where the host and group files are located. I am using 10 workers but my lab is fairly small so I could have used less here.

hosts.yaml

---
PE1:
  hostname: "192.168.10.176"
  groups:
    - pe
  data:
    interfaces:
      - name: Ethernet0/0
        vrf: CUSTOMER_789
        ip: 10.0.11.1/24
    bgp:
      neighbors:
        - vrf_name: CUSTOMER_789
          remote_ip: 10.0.11.2
          remote_as: 789

I included the setup for our PE1 router, the rest follow a similar structure. Anything host specific I set at the host.yaml file and anything group specific is set at the groups.yaml file you will see shortly. I assign all PE routers to the pe group and set certain interface parameters as well as BGP information that will be used in the future.

groups.yaml

---
pe:
  platform: ios
  username: cisco
  password: cisco
  data:
    vrfs:
      - name: CUSTOMER_789
        rd: "789:1"
        rt_import: "789:1"
        rt_export: "789:1"

      - name: CUSTOMER_777
        rd: "777:1"
        rt_import: "777:1"
        rt_export: "777:1"

The groups file is pretty neat, since all of my routers are part of the pe group and have a similar platform as well as authentication parameters, I decided to include all of that information under the group. In this automation example I wanted all VRFs to be included on all PE routers. Therefore I also added the VRF information under the group file vs under the host data.

Breaking Down Script

l3vpn_deploy.py snippet

from nornir import InitNornir
from nornir_scrapli.tasks import send_configs
from nornir_jinja2.plugins.tasks import template_file
from nornir_napalm.plugins.tasks import napalm_get
from nornir_utils.plugins.tasks.files import write_file
from nornir_utils.plugins.functions import print_result
from net_utils import address, mask

The portion above is essentially importing any tasks or functions we will need to run Nornir. Whether its importing InitNornir or plugins used to connect to devices and send commands. To learn more about plugins that can be used with Nornir please check out the nornir.tech site.

l3vpn_deploy.py snippet

def main():

    """
    Main that calls l3vpn function
    """

    nornir = InitNornir(config_file="config.yaml")
    print("Nornir initialized with the following hosts:\n")
    for host in nornir.inventory.hosts.keys():
        print(f"{host}\n")

    result = nornir.run(task=l3vpn)

    print_result(result)

This portion of the script is our main function and essentially kicks off all the other portions of our script to execute. Nornir is initialized with our config.yaml file that includes information about our hosts, groups, and connection parameters. There is a simple print statement at the bottom to let the operator know what inventory was just initialized for this job run. We will call on the l3vpn function that is defined towards the top of the script that I will walk through in a bit.

Creating the L3VPNs

We will need the following items on our PE routers to complete our L3VPN build:

Customer VRFs created with RD and RTs
Assign VRF to interface facing customer router
Configure BGP relationship between PE and CE device under new VRF

Now that we have the main portions required for our build, it was just a matter of converting the manual configurations into simple Jinja templates. I will spare you from reading about every task since they all follow the same pattern; create commands from Jinja template, split commands, send commands to PE routers. Here is a snippet below of what I mean.

l3vpn_deploy.py snippet

def l3vpn(task):

    """
    Main function built for L3VPN deployment tasks
    """
    task1_result = task.run(
        name=f"{task.host.name}: Creating VRFs Configuration",
        task=template_file,
        template="vrf.j2",
        path="templates/",
        data=task.host["vrfs"],
    )
    vrf_config = task1_result[0].result

    task2_result = task.run(
        name=f"{task.host.name}: Configuring VRFs on PE Nodes",
        task=send_configs,
        configs=vrf_config.split("\n"),
    )

I name the tasks for clarity when seeing the job output. Initially we run a template_file task that will create commands from a template. Then its a matter of pointing it to our specific template and feeding it data. Once that is done I set the output of that task to a variable called vrf_config, this can then be fed into the next task to push the commands to our PE routers.

{% for vrf in data %}
vrf definition {{ vrf.name }}
 rd {{ vrf.rd }}
 route-target export {{ vrf.rt_export }}
 route-target import {{ vrf.rt_import }}
 !
 address-family ipv4
 exit-address-family
{% endfor %}

Example of the VRF Jinja template above. The rest of the items for L3VPNs use the same structure. Feel free to check out the git repository linked below to see the rest. Ill show the output from a few PE and CE routers before and after the changes.

PE1#show vrf brief
  Name                             Default RD            Protocols   Interfaces
  MGMT                                          ipv4        Et0/3
PE1#
PE2#show vrf brief
  Name                             Default RD            Protocols   Interfaces
  MGMT                                          ipv4
PE2#
PE3#show vrf brief
  Name                             Default RD            Protocols   Interfaces
  MGMT                                          ipv4
PE3#
PE4#show vrf brief
  Name                             Default RD            Protocols   Interfaces
  MGMT                                          ipv4        Et0/3
PE4#
##### CE Devices #####
CE1#show ip bgp  summary | b Neigh
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.11.1       4        12345       0       0        1    0    0 00:10:51 Active
CE1#show ip bgp | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.16.1.0/24    0.0.0.0                  0         32768 i
CE1#
CE2#show ip bgp summary | b Neigh
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.22.1       4        12345       0       0        1    0    0 00:13:22 Idle
CE2#show ip bgp | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.16.2.0/24    0.0.0.0                  0         32768 i
CE2#
CE3#show ip bgp summary | b Neigh
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.33.1       4        12345       0       0        1    0    0 never    Active
CE3#show ip bgp | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.16.1.0/24    0.0.0.0                  0         32768 i
CE3#
CE4#show ip bgp summary | b Neigh
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.44.1       4        12345       0       0        1    0    0 never    Active
CE4#show ip bgp | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.16.2.0/24    0.0.0.0                  0         32768 i
CE4#

Script Output

(venv) [juliopdx@pbpro auto_mpls_l3vpn]$ time python l3vpn_deploy.py
Nornir initialized with the following hosts:

PE1

PE2

PE3

PE4

l3vpn***************************************************************************
* PE1 ** changed : True ********************************************************
vvvv l3vpn ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- PE1: Creating VRFs Configuration ** changed : False ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family

---- PE1: Configuring VRFs on PE Nodes ** changed : True ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family


---- PE1: Create VRF to Interfaces Configuration ** changed : False ------------ INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_789
 ip address 10.0.11.1 255.255.255.0
!

---- PE1: Configuring VRFs on Interfaces ** changed : True --------------------- INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_789
 ip address 10.0.11.1 255.255.255.0
!


---- PE1: Create BGP Neighbor Configuration ** changed : False ----------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_789
  neighbor 10.0.11.2 remote-as 789
  neighbor 10.0.11.2 activate
 exit-address-family

---- PE1: Configuring BGP Neighbors under VRFs ** changed : True --------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_789
  neighbor 10.0.11.2 remote-as 789
  neighbor 10.0.11.2 activate
 exit-address-family


^^^^ END l3vpn ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* PE2 ** changed : True ********************************************************
vvvv l3vpn ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- PE2: Creating VRFs Configuration ** changed : False ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family

---- PE2: Configuring VRFs on PE Nodes ** changed : True ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family


---- PE2: Create VRF to Interfaces Configuration ** changed : False ------------ INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_777
 ip address 10.0.22.1 255.255.255.0
!

---- PE2: Configuring VRFs on Interfaces ** changed : True --------------------- INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_777
 ip address 10.0.22.1 255.255.255.0
!


---- PE2: Create BGP Neighbor Configuration ** changed : False ----------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_777
  neighbor 10.0.22.2 remote-as 777
  neighbor 10.0.22.2 activate
 exit-address-family

---- PE2: Configuring BGP Neighbors under VRFs ** changed : True --------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_777
  neighbor 10.0.22.2 remote-as 777
  neighbor 10.0.22.2 activate
 exit-address-family


^^^^ END l3vpn ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* PE3 ** changed : True ********************************************************
vvvv l3vpn ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- PE3: Creating VRFs Configuration ** changed : False ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family

---- PE3: Configuring VRFs on PE Nodes ** changed : True ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family


---- PE3: Create VRF to Interfaces Configuration ** changed : False ------------ INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_777
 ip address 10.0.33.1 255.255.255.0
!

---- PE3: Configuring VRFs on Interfaces ** changed : True --------------------- INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_777
 ip address 10.0.33.1 255.255.255.0
!


---- PE3: Create BGP Neighbor Configuration ** changed : False ----------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_777
  neighbor 10.0.33.2 remote-as 777
  neighbor 10.0.33.2 activate
 exit-address-family

---- PE3: Configuring BGP Neighbors under VRFs ** changed : True --------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_777
  neighbor 10.0.33.2 remote-as 777
  neighbor 10.0.33.2 activate
 exit-address-family


^^^^ END l3vpn ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* PE4 ** changed : True ********************************************************
vvvv l3vpn ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- PE4: Creating VRFs Configuration ** changed : False ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family

---- PE4: Configuring VRFs on PE Nodes ** changed : True ----------------------- INFO
vrf definition CUSTOMER_789
 rd 789:1
 route-target export 789:1
 route-target import 789:1
 !
 address-family ipv4
 exit-address-family
vrf definition CUSTOMER_777
 rd 777:1
 route-target export 777:1
 route-target import 777:1
 !
 address-family ipv4
 exit-address-family


---- PE4: Create VRF to Interfaces Configuration ** changed : False ------------ INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_789
 ip address 10.0.44.1 255.255.255.0
!

---- PE4: Configuring VRFs on Interfaces ** changed : True --------------------- INFO
interface Ethernet0/0
 vrf forwarding CUSTOMER_789
 ip address 10.0.44.1 255.255.255.0
!


---- PE4: Create BGP Neighbor Configuration ** changed : False ----------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_789
  neighbor 10.0.44.2 remote-as 789
  neighbor 10.0.44.2 activate
 exit-address-family

---- PE4: Configuring BGP Neighbors under VRFs ** changed : True --------------- INFO
router bgp 12345
  address-family ipv4 vrf CUSTOMER_789
  neighbor 10.0.44.2 remote-as 789
  neighbor 10.0.44.2 activate
 exit-address-family


^^^^ END l3vpn ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

real    0m9.216s
user    0m6.683s
sys     0m1.079s
(venv) [juliopdx@pbpro auto_mpls_l3vpn]$

VRF Creation Validation

PE1#show ip vrf
  Name                             Default RD            Interfaces
  CUSTOMER_777                     777:1
  CUSTOMER_789                     789:1                 Et0/0
  MGMT                                          Et0/3
PE1#
PE2#show ip vrf
  Name                             Default RD            Interfaces
  CUSTOMER_777                     777:1                 Et0/0
  CUSTOMER_789                     789:1
  MGMT                             
PE2#
PE3#show ip vrf
  Name                             Default RD            Interfaces
  CUSTOMER_777                     777:1                 Et0/0
  CUSTOMER_789                     789:1
  MGMT                             
PE3#
PE4#show ip vrf
  Name                             Default RD            Interfaces
  CUSTOMER_777                     777:1
  CUSTOMER_789                     789:1                 Et0/0
  MGMT                                          Et0/3
PE4#

BGP Peers and Routes on CE Routers

CE1#show ip bgp  summary | b Neigh
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.11.1       4        12345      19      20        3    0    0 00:13:17        1
CE1#show ip bgp | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.16.1.0/24    0.0.0.0                  0         32768 i
 *>  172.16.2.0/24    10.0.11.1                              0 12345 789 i
CE1#traceroute 172.16.2.1 source 172.16.1.1 probe 1 numeric
Type escape sequence to abort.
Tracing the route to 172.16.2.1
VRF info: (vrf in name/id, vrf out name/id)
  1 10.0.11.1 0 msec
  2 10.0.15.5 [MPLS: Labels 300/24 Exp 0] 2 msec
  3 10.0.44.1 [MPLS: Label 24 Exp 0] 2 msec
  4 10.0.44.2 1 msec
CE1#
CE3#show ip bgp summary | b Neigh
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.33.1       4        12345      25      23        3    0    0 00:17:34        1
CE3#show ip bgp | b Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.16.1.0/24    0.0.0.0                  0         32768 i
 *>  172.16.2.0/24    10.0.33.1                              0 12345 777 i
CE3#traceroute 172.16.2.1 source 172.16.1.1 probe 1 numeric
Type escape sequence to abort.
Tracing the route to 172.16.2.1
VRF info: (vrf in name/id, vrf out name/id)
  1 10.0.33.1 1 msec
  2 10.0.35.5 [MPLS: Labels 302/407 Exp 0] 2 msec
  3 10.0.22.1 [MPLS: Label 407 Exp 0] 2 msec
  4 10.0.22.2 2 msec
CE3#

Outro and Links

I think there’s always room for improvement. Even off the top of my head a few additions could be the following:

Automated testing
Validation with something like pyATS to validate VRF creation on PE routers

I hope you found this a bit useful and maybe inspires you to build something you can use in everyday workload. Feel free to check out the links below to a few resources I found very helpful. Thank you again for reading this far, really means a lot.

Python Slack Bot for Network Engineers

Mon, 24 May 2021 10:19:32 -0800

Introduction

Hello and thank you for joining me in another blog post. I’ve wanted to mess with getting a bot running on Slack for a while now. After coming across a fantastic post by Mason Egger at Digital Ocean (linked at the end), I figured now is as good a time as any. Think of that post as a prerequisite to get you started before following along with this one.

Caveats

A few caveats I should include. I’m a fairly novice python user, expect to see areas where code can be refactored and even total rewrites that would make the organization better. What I will demonstrate in this post is what made sense to me at this point in time. I hope you enjoy and maybe get inspired to build something you can use to assist you in your daily workflow.

Network Engineering Bot

Below are a few of the goals I had when creating this bot.

Must be multi-vendor
Ability to call different (non static) devices
Bot should react to user messages

Lets walk through some code and I hope to answer how those goals were met. I’ll try not to repeat information that was already shared in Masons post. The initial code from Masons example has you create a class called CoinBot. It only made sense to me to create a separate class to define our network engineering bot. In this case we will call it NetBot, but you can name your bot whatever you like.

get_network.py

import json
from napalm import get_network_driver
from yaml import safe_load
import urllib3

# Disable warnings
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

class NetBot:
    """
    Defining a class called NetBot
    """

    USERNAME = "admin"
    PASSWORD = "admin"
    NET_BLOCK = {
        "type": "section",
        "text": {
            "type": "mrkdwn",
            "text": (
                "Getting interface information for device :slightly_smiling_face:"
            ),
        },
    }

    def __init__(self, channel, device_name):
        """
        constructor for class
        """
        self.channel = channel
        self.device_name = device_name

A few things to note. The variables you see in all caps are called class attributes/constants. Essentially things in the program that wont change. I kept it simple and created some credentials for authentication as class attributes. In reality this would be an environment variable or pulled from a secure source. The NET_BLOCK class attribute is just there so the bot can respond with some generic message to the user.

One of the first things that came to mind when building the bot is how will users know how to run the bot? I created the following method to solve that little problem. We are basically reading the readme file in our root directory and then returning all the data required to compose a message in slack. The following is added under our NetBot class from above.

get_network.py

    def send_help(self):
        """
        Sending all the help
        """
        with open("README.md", "r") as reader:
            readme = reader.read()
        return {
            "channels": self.channel,
            "filename": "README.md",
            "filetype": "markdown",
            "content": readme,
        }

Interacting With Network Devices

I mentioned before I wanted this to be multi-vendor. In this case we are going to use the popular NAPALM library!

pip install napalm
pip install pyaoscx
pip install napalm-aruba-cx

In theory you can use whatever you are comfortable with; Netmiko, NAPALM, Nornir, or Ansible. As long as the end result returns data in a format we can use to craft a Slack message. For our networking example, we will use NAPALM to retrieve the network interfaces from two devices. One being a Cisco IOL router and the other an Aruba CX switch. Below you will see the two methods used to put this all together. The first is pretty standard syntax to connect to device, run the get_interfaces function, and return as pretty JSON.

Quick deviation, I almost forgot to mention the inventory. In a grander scale, you would most likely have some dynamic inventory that is cached to the app running or the code actually interacts with your inventory source to pull the correct information. Think Netbox API call to get device information for a device named XYZ. This would then have management IP, platform, and whatever else would be needed to connect using NAPALM. In our case we are using a very simple YAML file with the two hosts mentioned earlier.

hosts.yaml


---
R1:
  name: R1
  platform: ios
  mgmt: "192.168.10.168"
ArubaCX:
  name: ArubaCX
  platform: aoscx
  mgmt: "192.168.10.169"

As promised, here is the code that interacts with our network devices.

get_network.py

    def _get_facts(self):
        """
        Method that will retrieve host facts
        """
        with open("hosts.yaml", "r") as handle:
            host_root = safe_load(handle)

        driver = get_network_driver(host_root[f"{self.device_name}"]["platform"])
        conn = driver(
            hostname=host_root[f"{self.device_name}"]["mgmt"],
            username=self.USERNAME,
            password=self.PASSWORD,
        )
        conn.open()
        facts = conn.get_interfaces()
        my_file = json.dumps(facts, indent=2)
        return my_file

The second portion of interacting with the networking device is constructing the message. The following method will return a working format to post a file type message in Slack. Notice towards the bottom the “content” key is actually calling the “_get_facts” method above? The_get_facts method is just returning some pretty formatted JSON string that we can then pass into our method to build a slack message.

get_network.py

    def get_file_payload(self):
        """
        Method used to post files from data gathered on device
        """
        return {
            "channels": self.channel,
            "filetype": "javascript",
            "content": self._get_facts(),
            "filename": f"{self.device_name}-interfaces.json",
        }

Running the Application

As mentioned previously, this code is heavily borrowed from Masons post! So much so that I left in the CoinBot learnings because it is an amazing reference and the code comments are great for learners. Ill cut out some of the extra information to just focus on the action, full code is available on my GitHub (linked below).

app.py

import os
import logging
from flask import Flask
from slack import WebClient
from slackeventsapi import SlackEventAdapter
from coinbot import CoinBot
from get_network import NetBot

# Initialize a Flask app to host the events adapter
app = Flask(__name__)

# Create an events adapter and register it to an endpoint in the slack app for event ingestion.
slack_events_adapter = SlackEventAdapter(
    os.environ.get("SLACK_EVENTS_TOKEN"), "/slack/events", app
)

# Initialize a Web API client
slack_web_client = WebClient(token=os.environ.get("SLACK_TOKEN"))

# Define bot ID so it will not respond to itself
BOT_ID = slack_web_client.api_call("auth.test")["user_id"]

Lets break that code down a bit. We will be utilizing Flask to do most of the heavy lifting. The imports mentioned above will import the required Flask, Slack, and NetBot packages. We utilize two environment variables for authentication. The BOT_ID constant is used in code later on to make sure the bot does not respond to itself. The next portions actually perform the execution of all of our code. I will break down one portion to keeps things short and sweet, but most all of them follow the same workflow.

app.py

def get_network_info(channel, device_name):
    """
    run the get_message_payload and get_file_upload method
    """
    net_bot = NetBot(channel, device_name)
    my_message = net_bot.get_message_payload()
    file_output = net_bot.get_file_payload()
    slack_web_client.chat_postMessage(**my_message)
    slack_web_client.files_upload(**file_output)

We are creating a function called get_network_info that takes in two parameters. When this function is executed, it will instantiate an instance of NetBot, send a generic message, and upload the file created from the get_file_payload method.

The next portion of the code is used to handle how the bot interacts with messages seen or what is executed when a certain message is seen. From part one of Masons post, we subscribed the bot to events (and we filter on messages). Once we get a message we strip out certain portions that can then be reused to respond in the proper channel, react to a specific message (timestamp), and make sure the bot does not respond to itself.

app.py

# When a 'message' event is detected by the events adapter, forward that payload
# to this function.
@slack_events_adapter.on("message")
def message(payload):
    """
    Parse the message event, and if the activation string is in the text,
    simulate something and send result
    """

    # Get various portions of message
    event = payload.get("event", {})
    text = event.get("text")
    user_id = event.get("user")
    timestamp = event.get("ts")
    channel_id = event.get("channel")

    # Making sure the bot doesnt respond to itself
    if BOT_ID != user_id:

The next piece of code is used to parse the messages that are seen by the bot. If a string matches one of the if clauses, some code will be executed. In this case we are looking for “netbot get network interfaces”. If you noticed in the readme, we are asking users to enter “netbot get network interfaces device=something”. The code will parse this output and strip “device=” to get the final device name. This will then be passed into the get_network_info function and execute the program.

app.py

        if "netbot get network interfaces" in text.lower():
            full_text = text.split()
            device = full_text[-1]
            my_device = device.replace("device=", "")
            slack_web_client.reactions_add(
                channel=channel_id, name="robot_face", timestamp=timestamp
            )
            slack_web_client.reactions_add(
                channel=channel_id, name="rocket", timestamp=timestamp
            )
            return get_network_info(channel_id, device_name=my_device)

You may have noticed a few commands the are prepend with “slack_web_client.reactions_add”. I wanted a way for the bot to respond to a user inputting a message that triggers the bot. Almost like a confirmation of message received. This can be message specific as I have made it for “netbot help” or “netbot get network interfaces”. Remember just a bit ago how we strip certain information from the message received like channel and timestamp ID? This is used throughout the code but here we are statically setting the types of reactions executed for this match. In the case of “get_network_info” the bot will react with a robot face and rocket.

The rest of the code is boilerplate to execute the application and enable logging. No changes from the original post at Digital Ocean. I want to thank Mason for the amazing blog post to kick start this spark for me. Thank you for reading this far, really means a lot! I hope what I’ve written makes a bit of sense and maybe inspires you to create something awesome.

Bugs/Issues

Bot reruns get network interface code even when new message is not sent (not seen when reactions are used).
When reaction is used multiple runs are not seen but app reports error 500 (already reacted).

Route Redistribution OSPFv3 and EIGRPv6

Mon, 03 May 2021 00:00:00 +0000

Introduction

Hello and thank you for checking out this post. I was recently working through OSPF and EIGRP for ENARSI studies and something came up on a lab I was building. Redistributing routes between OSPFv3 and EIGRPv6. I am specifically speaking of the IPv6 variant of these protocols.

In the topology you see below. We have three routing devices. One running only EIGRPv6, one running OSPFv3 (Aruba CX), and the redistribution node running both protocols. Both protocols can form neighbors using only link local addressing, so we wont see any global IPv6 addresses on those links.

OSPFv3

The OSPFv3 node is an Aruba CX switch running …OSPFv3! I have a few loopbacks configured so we can add some networks for testing. All of OSPF will be in area 0 for simplicity. We set all interfaces to passive besides the uplink to REDIS. Configuration snippet is below.

OSPFv3 Node Configuration

hostname OSPFv3
!
interface 1/1/1
    no shutdown
    description to REDIS
    ipv6 address link-local fe80::3/64
    ipv6 ospfv3 1 area 0.0.0.0
    no ipv6 ospfv3 passive
    ipv6 ospfv3 network point-to-point
interface loopback 0
    ipv6 address link-local fe80::3/64
    ipv6 address 2001:db8:456:4::1/64
    ipv6 ospfv3 1 area 0.0.0.0
interface loopback 1
    ipv6 address link-local fe80::3/64
    ipv6 address 2001:db8:456:5::1/64
    ipv6 ospfv3 1 area 0.0.0.0
interface loopback 2
    ipv6 address link-local fe80::3/64
    ipv6 address 2001:db8:456:6::1/64
    ipv6 ospfv3 1 area 0.0.0.0
!
router ospfv3 1
    router-id 3.3.3.3
    passive-interface default
    area 0.0.0.0
!

EIGRPv6

In the case of EIGRPv6, we will be using named EIGRP with an address family of IPv6. We add a few loopbacks for testing as well. One difference you’ll see in the configurations, EIGRPv6 is enabled on all interfaces with IPv6 addresses. There is no specific enablement of an interface to run EIGRPv6 or network statements. Configuration snippet is below.

EIGRPv6 Node Configuration

hostname EIGRPv6
!
ipv6 unicast-routing
!
interface Loopback0
 no ip address
 ipv6 address FE80::1 link-local
 ipv6 address 2001:DB8:123:1::1/64
!
interface Loopback1
 no ip address
 ipv6 address FE80::1 link-local
 ipv6 address 2001:DB8:123:2::1/64
!
interface Loopback2
 no ip address
 ipv6 address FE80::1 link-local
 ipv6 address 2001:DB8:123:3::1/64
!
interface GigabitEthernet0/0
 description to REDIS
 no ip address
 ipv6 address FE80::1 link-local
!
router eigrp JULIOPDX
 !
 address-family ipv6 unicast autonomous-system 123
  !
  topology base
  exit-af-topology
  eigrp router-id 1.1.1.1
 exit-address-family

REDIS

This node is special, it gets to run two routing protocols and perform redistribution between them! The configuration is very basic indeed. Only two interfaces set for link local addresses. Oh remember when I mentioned EIGRPv6 is enabled for all IPv6 interfaces? I will shutdown the interface towards OSPFv3. Good to reduce all the chatter. Configuration snippet is below.

REDIS Node Initial Configuration

hostname REDIS
!
ipv6 unicast-routing
!
interface GigabitEthernet0/0
 description to EIGRPv6
 no ip address
 ipv6 address FE80::2 link-local
!
interface GigabitEthernet0/1
 description to OSPFv3
 no ip address
 ipv6 address FE80::2 link-local
 ospfv3 network point-to-point
 ospfv3 1 ipv6 area 0
!
router eigrp JULIOPDX
 !
 address-family ipv6 unicast autonomous-system 123
  !
  af-interface GigabitEthernet0/1
   shutdown
  exit-af-interface
  !
  topology base
  exit-af-topology
  eigrp router-id 2.2.2.2
 exit-address-family
!
router ospfv3 1
 router-id 2.2.2.2
 !
 address-family ipv6 unicast
  passive-interface default
  no passive-interface GigabitEthernet0/1
 exit-address-family

Neighbors and Routes

EIGRPv6 Neighbor

REDIS#show ipv6 eigrp neighbors
EIGRP-IPv6 VR(JULIOPDX) Address-Family Neighbors for AS(123)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                    (sec)         (ms)       Cnt Num
0   Link-local address:     Gi0/0                    11 00:23:40  275  1650  0  11
     FE80::1
REDIS#

EIGRPv6 Routes

REDIS#show ipv6 route eigrp
D   2001:DB8:123:1::/64 [90/10880]
     via FE80::1, GigabitEthernet0/0
D   2001:DB8:123:2::/64 [90/10880]
     via FE80::1, GigabitEthernet0/0
D   2001:DB8:123:3::/64 [90/10880]
     via FE80::1, GigabitEthernet0/0
REDIS#

OSPFv3 Neighbor

REDIS#show ipv6 ospf neighbor
        OSPFv3 Router with ID (2.2.2.2) (Process ID 1)
Neighbor ID     Pri   State           Dead Time   Interface ID    Interface
3.3.3.3           0   FULL/  -        00:00:35    1360007168      GigabitEthernet0/1
REDIS#
#### Notice, no DR or BDR due to the point to point network

OSPFv3 Routes

REDIS#show ipv6 route ospf
O   2001:DB8:456:4::1/128 [110/1]
     via FE80::3, GigabitEthernet0/1
O   2001:DB8:456:5::1/128 [110/1]
     via FE80::3, GigabitEthernet0/1
O   2001:DB8:456:6::1/128 [110/1]
     via FE80::3, GigabitEthernet0/1
REDIS#

I wont bother sharing the neighbor states of EIGRPv6 or OSPFv3, but below you will see their routes before redistribution.

Routes on Nodes

#### EIGRPv6 node first
EIGRPv6#show ipv6 route eigrp
EIGRPv6#
##### Now for OSPFv3 node
OSPFv3# show ipv6 route ospf
No ipv6 routes configured
OSPFv3#

The Redistribution

REDIS#show run | s router
router eigrp JULIOPDX
 !
 address-family ipv6 unicast autonomous-system 123
  !
  topology base
   redistribute ospf 1 metric 1000000 1 255 1 1500
  exit-af-topology
 exit-address-family
router ospfv3 1
 address-family ipv6 unicast
  redistribute eigrp 123
 exit-address-family

New Routes on OSPFv3 and EIGRPv6

EIGRPv6#show ipv6 route eigrp
EX  2001:DB8:456:4::1/128 [170/15360]
     via FE80::2, GigabitEthernet0/0
EX  2001:DB8:456:5::1/128 [170/15360]
     via FE80::2, GigabitEthernet0/0
EX  2001:DB8:456:6::1/128 [170/15360]
     via FE80::2, GigabitEthernet0/0
EIGRPv6#

OSPFv3# show ipv6 route ospf
2001:db8:123:1::/64, vrf default
        via  fe80::2%1/1/1,  [110/20],  ospf
2001:db8:123:2::/64, vrf default
        via  fe80::2%1/1/1,  [110/20],  ospf
2001:db8:123:3::/64, vrf default
        via  fe80::2%1/1/1,  [110/20],  ospf
OSPFv3#

Simple Ping and Trace

EIGRPv6#ping 2001:db8:456:4::1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2001:DB8:456:4::1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 13/18/27 ms
EIGRPv6#traceroute 2001:db8:456:4::1
Type escape sequence to abort.
Tracing the route to 2001:DB8:456:4::1
  1 FE80::2 39 msec 15 msec 16 msec
  2 2001:DB8:456:4::1 13 msec 19 msec 19 msec
EIGRPv6#

OSPFv3# ping6 2001:db8:123:1::1 repetitions 1
PING 2001:db8:123:1::1(2001:db8:123:1::1) 100 data bytes
108 bytes from 2001:db8:123:1::1: icmp_seq=1 ttl=63 time=21.0 ms

--- 2001:db8:123:1::1 ping statistics ---
 1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 21.016/21.016/21.016/0.000 ms
OSPFv3# traceroute6 2001:db8:123:1::1
traceroute to 2001:db8:123:1::1 (2001:db8:123:1::1) from 2001:db8:456:4::1, 30 hops max, 3 sec. timeout, 3 probes, 24 byte packets
 1  fe80::2 (fe80::2)  18.681 ms  13.902 ms  15.768 ms
 2  2001:db8:123:1::1 (2001:db8:123:1::1)  20.081 ms  15.943 ms  19.944 ms
OSPFv3#

Thank you for reading this far, I really do appreciate it. I wish you the best and I hope you find something useful or interesting in this post. Cheers!

Simple Radius Server in EVE-NG

Fri, 16 Apr 2021 00:00:00 +0000

Introduction

I was recently going through an advanced routing course on Pluralsight by Nick Russo, great course by the way. Check it out HERE. During the course we inevitably get to the services portion. The portion most of us dread as network engineers. This is all the extra stuff. Where the routing and switching might be the sexy topic, services is arguably not that exciting… but very much necessary.

If we are in the process of testing AAA, we will need some form of TACACS or RADIUS server. Deploying a full on RADIUS VM can definitely eat up a lot of resources. GNS3, for example, has an appliance that can be added to provide this functionality. To my knowledge, EVE-NG does not. We will be leveraging FreeRADIUS on a lightweight Ubuntu VM. If you want to learn more about FreeRADIUS, check out the link mentioned below.

I hope to keep this as simple as possible to help the next engineer that has to stand up a simple RADIUS server in their topologies. I will have a few assumptions in this writing. For example, I assume you have some knowledge of using EVE-NG, adding Linux nodes, and most importantly AAA. I will include configurations of the Cisco nodes and useful links at the end of the post!

Topology

The topology you see above is pretty bare bones. Two nodes running OSPF and all the networks in area 0. One Linux machine will be used for SSH testing and the other is the FreeRADIUS server (Ubuntu 18.04). I linked the image used above but you can use any standard Ubuntu 18.04 image to run FreeRADIUS.

FreeRADIUS

If your topology already has internet access, you may be able to skip a few of these steps. In my case, this topology is isolated. Start by adding a network connection, this will connect the server to my internal network and give the server access to the internet.

Within the VM console session, run the following commands:

FreeRADIUS Install

sudo apt-get update
sudo apt-get install freeradius -y

Static IP for Ubuntu Server

At this point, the server has FreeRADIUS installed. Now we connect the server as shown in the topology image above and assign a static IP address. In Ubuntu 18.04 this can be done by editing the /etc/netplan/01-netcfg.yaml file. You can edit this file using vi, vim, or nano. Doesn’t really matter. I believe vi and nano come pre installed on this image. Edit that file and save. Example below.

/etc/netplan/01-netcfg.yaml

network:
   version: 2
   renderer: networkd
   ethernets:
     ens3:
       dhcp4: no
       addresses:
         - 192.168.2.2/24
       gateway4: 192.168.2.1

Once that file is saved, run the command below to apply the configuration.

sudo netplan apply

Add Clients and Users

The next step isn’t too bad. You essentially have to modify two files. One for user logins and the other for clients (network devices). The files have a lot of examples and working options. Most all of it is commented out. I’ll include just the portions that are active. Again, feel free to modify this with whatever editor you are comfortable with.

/etc/freeradius/3.0/clients.conf

client vIOS1 {
    ipaddr = 10.0.0.1
    secret = freeradius
}
client vIOS2 {
    ipaddr = 10.0.0.2
    secret = freeradius
}

/etc/freeradius/3.0/mods-config/files/authorize

Julio    Cleartext-Password := "PDX"
         Reply-Message = "Welcome to the world of tomorrow",
         cisco-avpair := "shell:priv-lvl=15"

Once that is completed, run the following commands to activate FreeRADIUS:

sudo service freeradius stop
sudo freeradius -X

SSH Validation

Outro and Links

Overall I don’t think it was too much work to get this going. If you build in some automation on the radius server, you could populate all the client entries with Ansible or Python. I think someone with a bit more brain power could probably create a pre built image that has FreeRADIUS pre packaged as an EVE-NG appliance. Thank you for reading this far and I hope this helps you down the road. Best of luck!

Automating Multi Vendor Environments With Netmiko

Fri, 02 Apr 2021 00:00:00 +0000

Introduction

Hello again! I was working through Nick Russos’ awesome Automating Networks with Python course on Pluralsight, check it out HERE. In the course, Nick does a great job of breaking down the code and the functionality of each bit. I’m still doing a lot of leaning in Python so please bear with me.

The course uses an MPLS environment as an example of network automation with Python. I decided to use the principles taught in the Netmiko portion of the course to try and automate some OSPF between multiple vendors. As you can see above, we are featuring Arista, Aruba, and Cisco. The topology is nothing crazy. All router facing ports are in area 0 and sharing routes. Every router will have a loopback interface to host a network. The only other caveat is that interfaces are passive by default and we enable each interface separately.

Host Variables

Each host has its own variable file. I tried to use something that made sense to me. There’s probably a better way to organize this file, it works, so I can dig it. Below is a sample of the Arista vEOS node. Each one is essentially the same besides two or three variables.

vEOS.yaml

---
hostname: vEOS
interfaces:
  - name: Ethernet1
    ip: 10.0.0.2/24
    ospf:
      process: 1
      area: 0
      passive: False
  - name: Loopback0
    ip: 192.168.2.1/24
    ospf:
      process: 1
      area: 1
ospf:
  process: 1
  router_id: 2.2.2.2
  area:
    - 0
    - 1

OSPF and Jinja

The jinja file process isn’t too bad. If you know loops, if, and else statements, you’ll be just fine. I’m pretty familiar with Aruba and Cisco but not so much with Arista. Lucky for me Arista is eerily similar to Cisco Syntax. I generally work my way through configuring the devices manually once and then convert the configurations to a template. Below is the Aruba version of the jinja template, each vendor gets its own jinja file.

hp_procurve.j2

{% if data.ospf %}
router ospf {{ data.ospf.process }}
    router-id {{ data.ospf.router_id }}
    passive-interface default
{% for area in data.ospf.areas %}
    area {{ area }}
{% endfor %}
{% for int in data.interfaces %}
{% if int.ospf %}
interface {{ int.name }}
{% if "back" not in int.name %}
    no shutdown
{% endif%}
    ip address {{ int.ip }}
    ip ospf {{ int.ospf.process }} area {{ int.ospf.area }}
{% if int.ospf.passive == False %}
    no ip ospf passive
{% endif %}
{% endif %}
{% endfor %}
{% endif %}
copy running-config startup-config

The Script

I wont go too deep in breaking down the file as I think it would take a while and I would really recommend you check out Nicks’ course I linked above. If I were to break down the script at a high level, below are the steps that are being performed.

Import the required libraries.
Define functions for address management (more on this later).
Define the main function to load in host variables, jinja environment, and loop over each host.
Bonus, network backup task (why not?).

ospf_netmiko.py

#! /usr/bin/env python

# Import requirements
from yaml import safe_load
from netmiko import Netmiko
from jinja2 import Environment, FileSystemLoader
from netaddr import IPNetwork
from rich import print as pr

# functions to be used in jinja templates for IP management
def address(a):
    a = str(IPNetwork(a).ip)
    return a


def mask(b):
    b = str(IPNetwork(b).netmask)
    return b


def main():

    # Open hosts file as variable for future use
    with open("hosts.yaml", "r") as handle:
        host_root = safe_load(handle)
    pr(host_root)

    # Set platform map to match netmiko
    platform_map = {"ios": "cisco_ios", "arista": "arista_eos", "aruba": "hp_procurve"}

    # Assigning platform variable to each host
    for host in host_root["host_list"]:
        platform = platform_map[host["platform"]]

        # Load in the host specific vars
        with open(f"vars/{host['name']}.yaml", "r") as handle:
            ospf = safe_load(handle)

        # This portion is essentially configuring our jinja environment
        j2_env = Environment(
            loader=FileSystemLoader("."), trim_blocks=True, autoescape=True
        )
        # https://www.kite.com/python/answers/how-to-call-a-function-in-a-jinja2-template-in-python
        j2_env.globals["address"] = address
        j2_env.globals["mask"] = mask

        template = j2_env.get_template(f"templates/netmiko/{platform}.j2")
        new_ospf_config = template.render(data=ospf)
        pr(f"\n{new_ospf_config}")

        conn = Netmiko(
            host=host["mgmt"],
            username=host["username"],
            password=host["password"],
            device_type=platform,
        )

        pr(f"\n#### Logged into {conn.find_prompt()}, woohoo! ####")

        result = conn.send_config_set(new_ospf_config.split("\n"))

        print(result)

        with open(f"backups/{host['name']}.conf", "w") as writer:
            result = conn.send_command("show run")
            writer.writelines(result)

        conn.disconnect()


if __name__ == "__main__":
    main()

Bonus, The Ansibilism

Something you will run into when diving into network automation is address management. Hmm or should I say address format conversion. Some network operating systems accept the format of “192.168.1.1/24” for address assignment and others accept “192.168.1.1 255.255.255.255.0”. My background is more in Ansible and I would usually do something like the following.

address = "192.168.1.1/24"
# Jinja
interface gig0/0
   ip address {{ address | ipaddr('address') }} {{ address | ipaddr('netmask') }}
   no shutdown

I was curious if something similar was available when using Jinja in Python. Honestly, I’m not sure and I bet there’s an easier way. If there is please let me know, and tell me “this is the way”. Long story short, this led me to the link included in the script and a bit of time on the python interpreter for testing. My testing went something like the following.

testing

In [1]: from netaddr import *
In [2]: a = "192.168.1.1/24"
In [3]: str(IPNetwork(a).ip)
Out[3]: '192.168.1.1'
In [4]: str(IPNetwork(a).netmask)
Out[4]: '255.255.255.0'
In [5]:

Now that I had the format, I created two functions, one for the address portion and one for the netmask. Ill show the relevant snippets from all the pieces involved below!

Odds and Ends

from netaddr import IPNetwork

def address(a):
    a = str(IPNetwork(a).ip)
    return a


def mask(b):
    b = str(IPNetwork(b).netmask)
    return b

# https://www.kite.com/python/answers/how-to-call-a-function-in-a-jinja2-template-in-python
        j2_env.globals["address"] = address
        j2_env.globals["mask"] = mask

# From Template
ip address {{ address(int.ip) }} {{ mask(int.ip) }}

Script Output

{
     'host_list': [
         {
             'name': 'vIOS',
             'platform': 'ios',
             'mgmt': '192.168.10.122',
             'username': 'cisco',
             'password': 'cisco'
         },
         {
             'name': 'ArubaCX',
             'platform': 'aoscx',
             'mgmt': '192.168.10.142',
             'username': 'admin',
             'password': 'aruba'
         },
         {
             'name': 'vEOS',
             'platform': 'eos',
             'mgmt': '192.168.10.151',
             'username': 'admin',
             'password': 'arista'
         }
     ]
 }
 Configuration to be loaded on vIOS:
 router ospf 1
     router-id 3.3.3.3
     passive-interface default
     no passive-interface GigabitEthernet0/0
 interface GigabitEthernet0/0
     no shutdown
     ip address 10.0.0.3 255.255.255.0
     ip ospf 1 area 0
 interface Loopback0
     ip address 192.168.3.1 255.255.255.0
     ip ospf 1 area 1
 do wr
 Logged into vIOS#, woohoo!
 configure terminal
 Enter configuration commands, one per line.  End with CNTL/Z.
 vIOS(config)#router ospf 1
 vIOS(config-router)#    router-id 3.3.3.3
 vIOS(config-router)#    passive-interface default
 vIOS(config-router)#    no passive-interface GigabitEthernet0/0
 vIOS(config-router)#interface GigabitEthernet0/0
 vIOS(config-if)#    no shutdown
 vIOS(config-if)#    ip address 10.0.0.3 255.255.255.0
 vIOS(config-if)#    ip ospf 1 area 0
 vIOS(config-if)#interface Loopback0
 vIOS(config-if)#    ip address 192.168.3.1 255.255.255.0
 vIOS(config-if)#    ip ospf 1 area 1
 vIOS(config-if)#do wr
 Building configuration…
 [OK]
 vIOS(config-if)#end
 vIOS#
 Configuration to be loaded on ArubaCX:
 router ospf 1
     router-id 1.1.1.1
     passive-interface default
     area 0.0.0.0
     area 0.0.0.1
 interface 1/1/1
     no shutdown
     ip address 10.0.0.1/24
     ip ospf 1 area 0.0.0.0
     no ip ospf passive
 interface loopback 0
     ip address 192.168.1.1/24
     ip ospf 1 area 0.0.0.1
 copy running-config startup-config
 Logged into ArubaCX#, woohoo!
 configure terminal
 ArubaCX(config)# router ospf 1
 ArubaCX(config-ospf-1)#     router-id 1.1.1.1
 ArubaCX(config-ospf-1)#     passive-interface default
 ArubaCX(config-ospf-1)#     area 0.0.0.0
 ArubaCX(config-ospf-1)#     area 0.0.0.1
 ArubaCX(config-ospf-1)# interface 1/1/1
 ArubaCX(config-if)#     no shutdown
 ArubaCX(config-if)#     ip address 10.0.0.1/24
 ArubaCX(config-if)#     ip ospf 1 area 0.0.0.0
 ArubaCX(config-if)#     no ip ospf passive
 ArubaCX(config-if)# interface loopback 0
 ArubaCX(config-loopback-if)#     ip address 192.168.1.1/24
 ArubaCX(config-loopback-if)#     ip ospf 1 area 0.0.0.1
 ArubaCX(config-loopback-if)# copy running-config startup-config
 Copying configuration: []
 Copying configuration: [|]
 Copying configuration: [/]
 Copying configuration: [-]
 Copying configuration: []
 Copying configuration: [|]
 Copying configuration: [/]
 Copying configuration: [-]
 Copying configuration: [Success]
 ArubaCX(config-loopback-if)# end
 ArubaCX#
 Configuration to be loaded on vEOS:
 ip routing
 router ospf 1
     router-id 2.2.2.2
     passive-interface default
     no passive-interface Ethernet1
 interface Ethernet1
     no switchport
     no shutdown
     ip address 10.0.0.2/24
     ip ospf area 0.0.0.0
 interface Loopback0
     ip address 192.168.2.1/24
     ip ospf area 0.0.0.1
 do wr
 Logged into vEOS#, woohoo!
 configure terminal
 vEOS(config)#ip routing
 vEOS(config)#router ospf 1
 vEOS(config-router-ospf)#    router-id 2.2.2.2
 vEOS(config-router-ospf)#    passive-interface default
 vEOS(config-router-ospf)#    no passive-interface Ethernet1
 vEOS(config-router-ospf)#interface Ethernet1
 vEOS(config-if-Et1)#    no switchport
 vEOS(config-if-Et1)#    no shutdown
 vEOS(config-if-Et1)#    ip address 10.0.0.2/24
 vEOS(config-if-Et1)#    ip ospf area 0.0.0.0
 vEOS(config-if-Et1)#interface Loopback0
 vEOS(config-if-Lo0)#    ip address 192.168.2.1/24
 vEOS(config-if-Lo0)#    ip ospf area 0.0.0.1
 vEOS(config-if-Lo0)#do wr
 Copy completed successfully.
 vEOS(config-if-Lo0)#end
 vEOS#

Outro and Links

Thank you for reading this far. I really do appreciate it. Stay safe and cheers!

Aruba Spine Leaf Deployment With OSPFv3 and Link Local Addresses

Mon, 29 Mar 2021 00:00:00 +0000

Introduction

I was recently on my way to finishing IPv6 Fundamentals by Rick Graziani. I will admit I’m not the fastest reader! In the book Rick mentions the following:

You could configure router R2’s interfaces with only link-local addresses, no global unicast addresses. This is because R2 has no end user interfaces. RFC 7404, Using Only Link-Local Addressing inside an IPv6 Network, discusses implementing routing protocols using only link-local addresses on infrastructure links.

In my previous post about deploying an Aruba spine leaf, I mentioned the possibility of wasting IP space. In IPv6, we do have an abundance of IP space, but this is still something the operator would have to maintain and work into their workflow or automation. Once I read this, I immediately looked up RFC 7404 “Using Only Link-Local Addressing inside an IPv6 Network”. The RFC does a really great job of breaking down the pros and cons. In this post we’ll be going over the pros. If want to learn more, feel free to check out the RFC.

Here is a list of a few of the advantages this provides. We’ll be going over most of these in this post.

Simple address management (automation)
Lower configuration complexity (automation and reduced errors)
Smaller routing tables
Reduced attack surface (less routed links)

Simple Address Management

If you recall from the previous post, we had to assign the IPv6 equivalent of a point to point address on each spine to leaf connections. This would increase the amount of IP addresses used as well as burden the operator to maintain these configurations. When assigning a link local address on a router, we can essentially assign the same address to each interface. One note as well, I mentioned that Aruba did not support IP unnumbered in the last post. If you are assigning the same address to each interface, we are essentially arriving at the same objective. Below is a sample configuration of a leaf and spine router. Please be aware that we are still configuring a global unicast address (GUA) on a loopback for these devices. This can also be used for device connections (SSH) or handling traffic to NMS.

Spine01 Configuration

interface 1/1/1
     no shutdown
     description link to leaf01
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
 interface 1/1/2
     no shutdown
     description link to leaf02
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
 interface 1/1/3
     no shutdown
     description link to leaf03
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
 interface 1/1/4
     no shutdown
     description link to leaf04
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
 interface loopback 0
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 address 2001:db8:cafe:ffff::1/128
     ipv6 ospfv3 1 area 0.0.0.0
 !
 router ospfv3 1
     router-id 1.1.1.1
     passive-interface default
     area 0.0.0.0

Only the loopback 0 interface on spine01 has a GUA. Essentially, every interface connected to the leaf nodes has the exact same configuration. Easy win for our automation folks!

Leaf01 Configuration

lan 1,10
 interface mgmt
     no shutdown
     ip dhcp
 interface 1/1/1
     no shutdown
     description link to spine01
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
 interface 1/1/2
     no shutdown
     description link to spine02
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
 interface 1/1/6
     no shutdown
     description link to Linux1
     no routing
     vlan access 10
 interface loopback 0
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 address 2001:db8:cafe:fd00::1/128
     ipv6 ospfv3 1 area 0.0.0.0
 interface vlan 10
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 address 2001:db8:cafe:a::1/64
     no ipv6 nd suppress-ra
     ipv6 ospfv3 1 area 0.0.0.0
 !
 router ospfv3 1
     router-id 10.0.0.1
     passive-interface default
     area 0.0.0.0

Difference in leaf nodes are the port VLAN assignments and VLAN interfaces.

Lower Configuration Complexity

My main background on automation is around Ansible. I will provide some basic YAML variable file as an idea on how the configuration has been simplified. I’ll use spine01 for this example.

Spine01 variables with GUA addresses

---
hostname: spine01
link_local: fe80:face:cafe::1/64
interfaces:
  1/1/1:
    ipv6_address: 2001:db8:cafe:fe01::a/127
    description: link to leaf01
  1/1/2:
    ipv6_address: 2001:db8:cafe:fe02::a/127
    description: link to leaf02
  1/1/3:
    ipv6_address: 2001:db8:cafe:fe03::a/127
    description: link to leaf03
  1/1/4:
    ipv6_address: 2001:db8:cafe:fe04::a/127
    description: link to leaf04
  loopback0:
    ipv6_address: 2001:db8:cafe:ffff::1/128

Spine01 variables with only link-local

---
hostname: spine01
link_local: fe80:face:cafe::1/64
interfaces:
  1/1/1:
    description: link to leaf01
  1/1/2:
    description: link to leaf02
  1/1/3:
    description: link to leaf03
  1/1/4:
    description: link to leaf04
  loopback0:
    ipv6_address: 2001:db8:cafe:ffff::1/128

The example is basic in nature but you can see how mistakes can be reduced and configuration can be simplified.

Smaller Routing Tables

Since we are using link local addresses, they have a scope that is… local to the link. The routing protocols will not propagate this as reachable networks, this in turn will reduce the total size of our routing tables. This has a great benefit of saving memory and speeding up convergence times. For comparison, check out the routing table on leaf01 when using GUA addresses and only using link local.

Using GUA /127 addresses

leaf01# show ipv6 ospfv3 routes | b 2001
  2001:db8:cafe:a::/64 (i) area:0.0.0.0
       directly attached to interface vlan10, cost 100 distance 110
  2001:db8:cafe:14::/64 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
  2001:db8:cafe:14::/64 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
  2001:db8:cafe:1e::/64 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
  2001:db8:cafe:1e::/64 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
  2001:db8:cafe:28::/64 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
  2001:db8:cafe:28::/64 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
  2001:db8:cafe:fd00::2/128 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
  2001:db8:cafe:fd00::2/128 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
  2001:db8:cafe:fd00::3/128 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
  2001:db8:cafe:fd00::3/128 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
  2001:db8:cafe:fd00::4/128 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
  2001:db8:cafe:fd00::4/128 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
  2001:db8:cafe:fe01::a/127 (i) area:0.0.0.0
       directly attached to interface 1/1/1, cost 100 distance 110
  2001:db8:cafe:fe02::a/127 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
  2001:db8:cafe:fe03::a/127 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
  2001:db8:cafe:fe04::a/127 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
  2001:db8:cafe:ff01::a/127 (i) area:0.0.0.0
       directly attached to interface 1/1/2, cost 100 distance 110
  2001:db8:cafe:ff02::a/127 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
  2001:db8:cafe:ff03::a/127 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
  2001:db8:cafe:ff04::a/127 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
  2001:db8:cafe:ffff::1/128 (i) area:0.0.0.0
       via fe80:face:cafe::1 interface 1/1/1, cost 100 distance 110
  2001:db8:cafe:ffff::2/128 (i) area:0.0.0.0
       via fe80:face:cafe::2 interface 1/1/2, cost 100 distance 110

Only link local addresses

leaf01#   show ipv6 ospfv3 routes | b 2001
 2001:db8:cafe:a::/64 (i) area:0.0.0.0
      directly attached to interface vlan10, cost 100 distance 110
 2001:db8:cafe:14::/64 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
 2001:db8:cafe:14::/64 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
 2001:db8:cafe:1e::/64 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
 2001:db8:cafe:1e::/64 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
 2001:db8:cafe:28::/64 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
 2001:db8:cafe:28::/64 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
 2001:db8:cafe:fd00::2/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fd00::2/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:fd00::3/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fd00::3/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:fd00::4/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fd00::4/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:ffff::1/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 100 distance 110
 2001:db8:cafe:ffff::2/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 100 distance 110

Trace from leaf01 and linux04

leaf01# traceroute6 2001:db8:cafe:28:5200:ff:fe0a:0
 traceroute to 2001:db8:cafe:28:5200:ff:fe0a:0 (2001:db8:cafe:28:5200:ff:fe0a:0) from 2001:db8:cafe:a::1, 30 hops max, 3 sec. timeout, 3 probes, 24 byte packets
  1  2001:db8:cafe:ffff::1 (2001:db8:cafe:ffff::1)  4.712 ms  70.151 ms  24.438 ms
  2  2001:db8:cafe:28::1 (2001:db8:cafe:28::1)  18.437 ms  58.055 ms  17.812 ms
  3  2001:db8:cafe:28:5200:ff:fe0a:0 (2001:db8:cafe:28:5200:ff:fe0a:0)  12.547 ms  53.539 ms  17.951 ms
 leaf01#

Last little bit, if you notice the output of the trace, the first hop is the GUA of loopback0 on spine01

Wrap Up

Thank you for reading this far, I really do appreciate it. If you want to learn more on using link-local addresses between router links, check out RFC 7404! Take care and stay safe! Cheers!

Previous post: Aruba Spine Leaf with OSPFv3 and IPv6

Aruba Spine Leaf With OSPFv3 and IPv6

Thu, 18 Mar 2021 00:00:00 +0000

Introduction

Hello and thank you for checking out this post! In this post I hope to breakdown the topology you see above. I’ll walk through the design, IPv6 IP allocations, and OSPFv3. I have a very small background in IPv6, basically enough to get past a few Cisco exams, which inevitably gets forgotten about after some time of little to no use. I purchased IPv6 Fundamentals by Rick Graziani, I want to say almost a year ago. Its been sitting there haunting me for months.

I finally cracked it open and I can say it is incredible in getting you used to IPv6 and understanding the technologies involved. I’m about a third of the way done with the book and I hope to put some of what I’ve learned to practice in the topology above and in this post. Also, maybe helping folks learn a bit about spine leaf deployments and IPv6 in general.

Since its just you and me here. Lets pretend our site was assigned the 2001:db8:cafe::/48 global unicast prefix. Let me tell you, this gives us a whole bunch of IP space. I mean unimaginable amounts of IP space. This prefix will be more important as we go through the post. Please note, this design is fairly basic in nature and doesn’t use any fancy stuff like EVPN/VXLAN/LAG.

Design

The design you see above is using Aruba CX nodes with version 10.06.0001 running on EVE-NG. The base of the topology is a spine leaf design. In a spine leaf design, every spine connects to every leaf. Every leaf connects to every spine. Spines do not connect with each other. In some situations, leaf switches can connect together. For example, when dual connecting a host. In the case of this topology we have two spines and four leaf nodes. The Linux nodes you see below are just running Linux Slax. They will be used for some lightweight testing and verification of connectivity.

Lets knock out a few of the simple things. I’ll be using OSPFv3 in this example. The nodes were required to have a router ID set. In this case I stuck with spine01 having 1.1.1.1, spine02 having 1.1.1.2, and so on. For the leaf nodes I used the 10.0.0.x pattern. X being the number of the leaf.

The leaf to spine connections just follow a pattern, port 1 on each leaf connects to spine01 and port 2 on each leaf connects to spine02. Speaking of the leaf to spine connections, you will notice that these are point to point connections. In the IPv4 world this would usually mean we have a /30 or /31 network on the link. IPv6 has something similar, the /127 network.

We will use a portion of the highest network in 2001:db8:cafe::/48. Connections to spine01 will be on 2001:db8:cafe:fe::/56 and connections to spine02 will be on the 2001:db8:cafe:ff::/56 network. You will probably notice the pattern I followed in the diagram. Leaf nodes will always get the ::b address and spines will be assigned the ::a address.

At this point you might be thinking, wow this is wasting a lot of IP space. Remember, with IPv6 you have an incredible amount of IP space. This doesn’t mean you shouldn’t plan ahead but also don’t spend months on this. One small caveat, I’m not saying this is a great or even good IP plan for this deployment. Just going by what I’ve learned so far and putting it to use. Below you’ll see the link configurations on spine01. The output for spine02 is essentially the same.

OSPFv3 and Spine Nodes

Spine01 Configuration

!
interface mgmt
     no shutdown
     ip dhcp
interface 1/1/1
     no shutdown
     description link to leaf01
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 address 2001:db8:cafe:fe01::a/127
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
interface 1/1/2
     no shutdown
     description link to leaf02
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 address 2001:db8:cafe:fe02::a/127
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
interface 1/1/3
     no shutdown
     description link to leaf03
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 address 2001:db8:cafe:fe03::a/127
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
interface 1/1/4
     no shutdown
     description link to leaf04
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 address 2001:db8:cafe:fe04::a/127
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
interface loopback 0
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 address 2001:db8:cafe:ffff::1/128
     ipv6 ospfv3 1 area 0.0.0.0
!
router ospfv3 1
     router-id 1.1.1.1
     passive-interface default
     area 0.0.0.0
!

A lot to unpack there right? Ill do my best to break it down below. Lets just focus on the small snippet below.

Interface Snippet

interface 1/1/4
     no shutdown
     description link to leaf04
     ipv6 address link-local fe80:face:cafe::1/64
     ipv6 address 2001:db8:cafe:fe04::a/127
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
!
router ospfv3 1
     router-id 1.1.1.1
     passive-interface default
     area 0.0.0.0

The link local address comes with every IPv6 enabled interface. Whether its Windows, Mac, or whatever. Doesn’t matter. You can leave this alone and the device will generate a link local address on its own. In our world, its better to just configure the link local address ourselves. This helps when working with routing protocols and figuring out what the source of traffic may be. You can configure the same link local address on each interface of a device. Link local addresses start with fe80, and as you can see, I’ve assigned the same throughout each interface.

Not much to add on the global unicast address (2001:). This is following the plan mentioned above. Note that all spine01 GUA end in ::a. “ipv6 ospfv3 1 area 0.0.0.0”, just enabling OSPF under the interface. One good thing to note, if you noticed there is “no ipv6 ospfv3 passive” under each point to point interface. By default we are setting each OSPF enabled interface to passive.

This is a good way of limiting the amount of chatter in a OSPF network. This will also skip some OSPF states since no DR/BDR elections occur or even need to occur on point to point links. Under the main OSPFv3 configuration; we are setting every interface to passive by default, setting router ID, and activating area 0. Bidirectional forwarding detection (BFD) is enabled under each point to point interface to help with failure detection on links.

I want to take a quick second and mention IP unnumbered. This does seem like a lot of IP addresses to manage, even if we are following a pattern. To my knowledge and some google searches, I don’t believe Aruba supports this feature. IP unnumbered essentially lets you borrow an IP address from an interface. For example, borrowing the loopback 0 address in our deployment and using it on all point to point links. If your network operating system supports this feature, use it! I think that’s enough for the spines, lets take a look at one of the leaf nodes. Below is leaf01, again, all other leaf nodes are essentially the same.

OSPFv3 and Leaf Nodes

Leaf01 Configuration

vlan 1,10
interface 1/1/1
     no shutdown
     description link to spine01
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 address 2001:db8:cafe:fe01::b/127
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
interface 1/1/2
     no shutdown
     description link to spine02
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 address 2001:db8:cafe:ff01::b/127
     ipv6 ospfv3 1 area 0.0.0.0
     no ipv6 ospfv3 passive
     ipv6 ospfv3 network point-to-point
     ipv6 ospfv3 bfd
interface 1/1/6
     no shutdown
     description link to Linux1
     no routing
     vlan access 10
interface loopback 0
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 address 2001:db8:cafe:fd00::1/128
     ipv6 ospfv3 1 area 0.0.0.0
interface vlan 10
     ipv6 address link-local fe80:beef:cafe::1/64
     ipv6 address 2001:db8:cafe:a::1/64
     no ipv6 nd suppress-ra
     ipv6 ospfv3 1 area 0.0.0.0
!
router ospfv3 1
     router-id 10.0.0.1
     passive-interface default
     area 0.0.0.0

Looks fairly similar right? Lets just focus on the new stuff. I added VLAN 10 on leaf01, VLAN 20 on leaf02, and so on. After that I just assigned the host interface to the appropriate VLAN. Here is where things get fun. In hex you get the range of 1-15. A-F make up what we know as 10-15. Remember our whole network is 2001:db8:cafe::/48. I decided to use the next available prefix that matched the VLAN number.

For example if the VLAN is 10, that would give us 2001:db8:cafe:a::/64. If the VLAN was 20, that would give us 2001:db8:cafe:14::/64. Looking at that last IP, the 1 in 14 is in the 16’s place, so that adds up to 16, and the 4 is in the 1’s place. 16 + 4 = 20, or VLAN 20. Now you might be wondering, why are you using a /64. Well, the powers that be recommend all LAN networks be a /64. This helps nodes acquire addresses automatically without a DHCP server. More on that in a sec. The /64 would give you over 18,000,000,000,000,000,000 possible Interface IDs…. on one LAN… wow.

Hosts

You may have wondered, what does the “no ipv6 nd suppress-ra” command do? This is a great time to talk about hosts or end devices. Usually these systems will have static addresses for management and tracking. I left the Linux hosts as default so they could use the built in processes with IPv6 to acquire an address. Back to the /64 prefix size. We have just split the 128 bit address in half. 64 are for the prefix/subnet and what’s left are for hosts, called an interface ID in IPv6.

In IPv6 hosts can use a process called SLAAC (stateless address autoconfiguration). I wont go deep into details but essentially the host sends router solicitation (RS) messages and the router responds with router advertisement (RA) messages. The host uses the information in the RA message, with the prefix to determine how to go about getting an interface ID.

In our case the host will use EUI-64. Another new thing! This ones not so bad. Essentially the host uses its 48 bit MAC address, splits it in half to insert FFFE in the middle, and then flips the seventh bit to make it a zero or a one. Don’t worry if its over your head. Just know the host uses its MAC to build the interface ID and inserts FFFE in the middle!

Back to our “no ipv6 nd suppress-ra” command. From my testing it seems that Aruba CX devices wont send the RA message needed by the hosts by default. In this case, just add that command and away you go. I’ll finish the technical stuff with a snap of Wireshark between Linux1 and leaf01

Packet capture between leaf01 and linux01

Packet Information

Packet #	Type	Description
8	Router Solicitation (RS)	Linux1 using its link local address (fe80:5200:ff:fe07:0) to reach the All-routers address (ff02::2)
9	Router Advertisement (RA)	leaf01 using its link local address (fe80:beef:cafe::1) to reach the All-nodes address (ff02::1)

Check out the ICMPv6 options sent in the RA message. The big one to note here is the prefix information of 2001:db8:cafe:a::/64, the VLAN 10 prefix information on leaf01. Just a note, the RA message is also the only way a node can get a default gateway in IPv6. Whether its SLAAC or DHCPv6, the default gateway comes from the RA message.

What’s a good network engineering article without some routes and pings!

OSPFv3 Routes at leaf01

leaf01# show ipv6 ospfv3 routes | b 2001
 2001:db8:cafe:a::/64 (i) area:0.0.0.0
      directly attached to interface vlan10, cost 100 distance 110
 2001:db8:cafe:14::/64 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
 2001:db8:cafe:14::/64 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
 2001:db8:cafe:1e::/64 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
 2001:db8:cafe:1e::/64 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
 2001:db8:cafe:28::/64 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 300 distance 110
 2001:db8:cafe:28::/64 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 300 distance 110
 2001:db8:cafe:fd00::2/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fd00::2/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:fd00::3/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fd00::3/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:fd00::4/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fd00::4/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:fe01::a/127 (i) area:0.0.0.0
      directly attached to interface 1/1/1, cost 100 distance 110
 2001:db8:cafe:fe02::a/127 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fe03::a/127 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:fe04::a/127 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 200 distance 110
 2001:db8:cafe:ff01::a/127 (i) area:0.0.0.0
      directly attached to interface 1/1/2, cost 100 distance 110
 2001:db8:cafe:ff02::a/127 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:ff03::a/127 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:ff04::a/127 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 200 distance 110
 2001:db8:cafe:ffff::1/128 (i) area:0.0.0.0
      via fe80:face:cafe::1 interface 1/1/1, cost 100 distance 110
 2001:db8:cafe:ffff::2/128 (i) area:0.0.0.0
      via fe80:face:cafe::2 interface 1/1/2, cost 100 distance 110

Trace from leaf01 to VLAN 40 at leaf04

Outro and Links

Thank you so much for reading this far! I am just getting my bearings with IPv6 but it really is great. Maybe as I work through the book I’ll add some border leaf nodes and get some NAT64 going! Check out the links below if curious about learning more on IPv6 or data center networking. Too many RFCs to list but you can easily google them. For example RFC 6164, “Using 127-Bit IPv6 Prefixes on Inter-Router Links”.

Cloud Native Data Center Networking: Architecture, Protocols, and Tools - I cant find the free link!
IPv6 Fundamentals: A Straightforward Approach to Understanding IPv6 Second Edition
FYI, these are not sponsored links at all. I earn nothing if you buy or don’t buy a product.
Follow up post: Using only link-local addresses between routers

My Journey to Completing the David Goggins 4x4x48 Challenge

Mon, 08 Mar 2021 00:00:00 +0000

Introduction

Hello and thank you for tuning in. Just for clarity, the 4x4x48 challenge was created by David Goggins. The challenge requires participants to run 4 miles every 4 hours for 48 hours. Participants are allowed to substitute running with walking or general exercise. The main goal is to get people moving as well as donate to charity!

The next thought may be why the hell would someone want to run 48 miles on a perfectly nice weekend? Everyone will have their own reasons, heck for some this may just be a training run, shout-out to all the ultra runners out there (I am not one). I had a few reasons for attempting this challenge. Most of them are pretty selfish of me to be honest. This was a physical test for a distance I’ve never even come close to reaching. Even greater than the physical challenge was the mental challenge I would eventually be faced with. This impacted me mentally way more than physically when it was all done with.

I think this is a good spot to give you all a bit of a background on my running history. I have an okay athletic background. Never been the fastest, strongest, or lowest weight. Heck before this challenge I was sitting right at 235 lbs. My time in the military only required running 1.5 miles under 13 minutes or so to get a decent physical score. I’ve competed with teams on endurance races, but even then I was responsible for maybe one or two legs of the eventual 70 plus miles. Where each leg I may be responsible for 3-5 miles. For this event I wasn’t just the leg, I was the team… yikes!

Since my time in the military I still run 3-4 times a week for about 3-5 miles each run. Last year I had the idea of training for a marathon. It would be my first and I thought it would be a sweet goal. I was up to 16 miles in one session, which I thought was great. Inevitably, the current pandemic we are all going through put a stop to all races. I figured I’d keep training and eventually I could get one marathon under my belt. Well we are all human and around August/September of last year, I dealt with a nagging upper calf injury that wouldn’t go away. I did some PT and took a lot of time off (months). Injuries aren’t new to me, hell I’ve torn the ACL in my left knee 3 times! At least that injury puts you all the way down for some time.

Run Preparation

That was a really long winded background but its my blog so its all good! Now for how I prepared for this challenge. Full disclosure, I am no running expert and this is definitely not the go to plan or even a good plan. Its just what worked for me. I continued running 3-4 times a week for about 3-5 miles after I recovered from the calf injury. I also added weight training using those fancy adjustable dumbbells, yay for home gyms! I eventually started running right after doing a leg day workout. Trying to trick my mind into dealing with the soreness I would face during this challenge. Oh boy I couldn’t be more wrong.

Running Technique

The running technique I use is pretty damn simple. I run for a few minutes… then walk for 30 seconds. This technique was developed or popularized by Jeff Galloway. I heard about Jeff from a fellow co worker and picked up his book to train for the marathon, “Marathon, You Can Do It!”. The thought behind this technique is that you run for a certain amount of time and then walk. You basically do this in intervals and repeat. The small walk breaks in between running lowers the muscle breakdown and soreness experienced from constant running. Runners have even accomplished personal bests using this technique. Some folks may put you down because you aren’t constantly running, but take my advice and tell them to FUCK right off.

Nutrition

Sorry for getting so spicy there at the end. While running the challenge, when I hit mile 30+ and other weekend runners are doing their thing and passing you, don’t get down. Just tell yourself “IM ON MILE FUCKING 30!”. Just a bit on nutrition, again I’m no expert. I basically searched the internets, what are the best fruits to eat after runs? I saw bananas (cramps) / oranges (muscle recovery). Besides the usual protein source to fuel muscles and a whole lot of carbs. I had some chicken and rice as my main meals. Pictured below is my race set up (thanks to my lovely wife).

Pre Race

So now its the night before the race. I tried for a while to place it in the back of my mind. Thinking about that many miles in general makes me nervous and boy was I nervous the night before. My advice would be to lay out all your clothes, gear, food, and chargers in one room or general area. It makes transitioning from relaxing to running so much easier. In my case I left all chargers connected so I just dropped off running watch or earbuds on their charger when a leg was done. Don’t forget this! My running light ran out of juice during one run and damn it was dark!

I did some simple math and figured we would all be running 12 total legs to complete the challenge. Initially I wanted to break down this write up leg by leg and honestly I think its better if I group them into thirds. Since the pain and tiredness you experience is similar between them.

Legs 1-4

Some of the fastest legs here for obvious reason haha. I mentioned before I reached 16 miles in one running session. In my mind I knew I could at the very least make it to mile 16, which would take me to the end of leg 4. These runs were overall pretty fun and the muscle soreness and fatigue was low.

Midnight Runs

I have to mention this. Being a two day event, you will be running in the dark. I would recommend getting a reflective vest and a light for the front and back. Stay safe out there folks and protect yourselves. The night runs for me were truly special. The world is so silent but loud at the same time. You hear everything; the wind moving grass, stop signs shaking, small critters, the sound of your breath, or the college kid puking his guts out over the porch. I’m not hating, enjoy yourself young man!

Legs 5-8

These legs were honestly a total surprise to me. It would be mileage I have never reached before and that might be what fueled me during these runs. My overall times were pretty good. I maintained a good pace and muscle soreness around the quad, hip, and calf areas just started. Remember when I said “IM ON MILE FUCKING 30!”

Legs 9-12 “Go get it”

I wont lie to anyone. Unless you are some experienced distance runner, these legs will hurt! I say that for a good reason. No one is making us do this. You can stop at anytime. Whatever the reason is that you chose to do this, I’m so happy and proud of you. It’ll hurt to even think of leaving that warm comfy bed at 4 AM. Just know that you made it this far and you should be so proud. “One more to go!! Go get it!!” – John Spiegel

Some of you may have wondered, if you have to run 4 miles every four hours, why the hell is this dude running 4.4? Again, for selfish reasons. I’ve never gone for this many miles. This got me thinking, “if I can make it that far, how much will I need to run to hit 52.4”. The equivalent of TWO marathons. From never running one to running two in 48 hours. I thought that would be incredible and pretty damn special.

In Between Time

What do you do in between each leg? I developed a pretty normal routine. I ate half a banana after each leg and some orange slices. I would then hydrate with plain water and get a meal in at normal times of the day. I stretched and foam rolled after each and every run, no exceptions. Even if you have to stretch in bed at 1 AM… do it! Besides that I just played some video games and read messages from awesome supporters.

Support

When embarking on great challenges, having a support team or person is critical. Thank you to everyone who like, comment, and subscribed… wait a minute this isn’t YouTube. Seriously, thank you to everyone that supported me and left a positive message. Thank you to all the support teams out there, y’all rock! Last but definitely not least. Big thanks to my wife Amanda, without her support I don’t know what I would be doing. Definitely not running 52 miles…

You Can Do It

This experience has showed me if you put your mind to it, you really can do anything. Cliché I know. Take me as a small example. After the third ACL tear, I could’ve given up on all athletic hopes and dreams. Maybe now I can claim the title of being the greatest to ever run 52 miles with a twitter handle of Julio_PDX…

Network Validation With Nornir & Napalm

Sat, 27 Feb 2021 00:00:00 +0000

Intro

I wanted to get my feet wet with Nornir/NAPALM and Python network automation in general. One simple goal to start with in network automation is validating configurations or deploying “show commands” and returning some kind of useful information. I wanted to start with something very simple, as easy wins make me want to keep progressing. In this post I will break down how I validate SNMP information on network devices using Nornir and NAPALM.

I think its worth knowing a bit of background to set the stage for future network engineers that are curious about diving into python for network automation. For starters, you can do it! I started my career with zero knowledge about python or programming in general. My initial and current automation efforts revolve mostly around using Ansible. I think both tools are great and they each have their place in this space. In the bottom of this post I’ll include some links on resources I’ve used.

I wont go too deep into explaining Nornir and NAPALM but I’ll do my best. Nornir is a framework in which, it can provide a foundation for plugins or tools to be constructed around it. Think of Ansible but pure Python. NAPALM acts almost like a translator for multivendor environments. For example, if our goal is to get interface information or SNMP information from a device, we don’t care what commands are executed. We just need the data returned to use in a constructed manner.

NAPALM does this in a concept it calls “getters”. Getters are basically functions in NAPALM that interact with networking devices and return structured information. For a list of all supported vendors and getters, please see here.

Current Structure

(venv) juliopdx@juliopdx-virtual-2004:~/git/nornir_snmp_validation$ tree -I venv
 .
 ├── config.yaml
 ├── inventory
 │   ├── defaults.yaml
 │   ├── groups.yaml
 │   └── hosts.yaml
 ├── nornir.log
 ├── README.md
 ├── requirements.txt
 ├── snmp_validate.py
 └── validate
     └── cisco.yaml
 2 directories, 9 files

# Creating python environment
python3 -m venv venv

# Activating virtual environment
source venv/bin/activate

# Clone git repo
git clone https://github.com/JulioPDX/nornir_snmp_validation.git
cd nornir_snmp_validation/

# Install requirements
pip install -r requirements.txt

Nornir was recently updated to split some of the functionality from Nornir core and the additional plugins. I just went ahead and installed pretty much all of the third party plugins at once. Here is a link to those plugins.

 nornir==3.0.0
 nornir-ansible==2020.9.26
 nornir-jinja2==0.1.1
 nornir-napalm==0.1.1
 nornir-netbox==0.2.0
 nornir-netmiko==0.1.1
 nornir-pyez==0.0.10
 nornir-scrapli==2021.1.30
 nornir-utils==0.1.1

lets take a look at the config.yaml file. If you are coming from an Ansible background, just think about this as the ansible.cfg file. This allows you to set the inventory plugin, runner (# of ssh sessions and serial vs threaded), and locations for certain files that we’ll talk about in a bit.

config.yaml

---
 inventory:
   plugin: SimpleInventory
   options:
     host_file: "inventory/hosts.yaml"
     group_file: "inventory/groups.yaml"
     defaults_file: "inventory/defaults.yaml"
 runner:
   plugin: threaded
   options:
     num_workers: 10

Now lets go into our inventory folder. You will notice that we have three files; hosts.yaml (for host data), group.yaml (group data), and defaults.yaml (defaults/global data). I found that Nornir is really flexible on where you can store information. I’ll show you what made sense to me.

---
 port: 22
 username: cisco
 password: cisco
 platform: ios
# Since I am using only Cisco devices in this lab, I just set the
# login information at the defaults level.
# In multivendor deployments, you would probably break this out
# into the groups.yaml file or under each hosts...

At the moment I didn’t do much of anything with the goups.yaml file. In my testing it seems that if a group is called out in the hosts.yaml file, it must exist in the groups.yaml file.

groups.yaml

---
cisco:
   data:
 switches:
   data:
 routers:
   data:

I tried to keep the host.yaml file as simple as possible. I created some fake data under each host to signify what site they belong to. This was mainly as an example from the official documentation. I assigned different hosts to different groups depending on if they were switches or routers, and what vendor. Oh, going back to the group variables. If different systems had different connection parameters, you could put that under the groups.yaml file and just assign the host to that group.

hosts.yaml

---
NY-SW-01:
  hostname: 192.168.10.37
  groups:
    - cisco
    - switches
  data:
    site: NY
NE-RTR-01:
  hostname: 192.168.10.38
  groups:
    - cisco
    - routers
  data:
    site: NE
OR-SW-01:
  hostname: 192.168.10.36
  groups:
    - cisco
    - switches
  data:
    site: OR

I’m going to side step a bit here and get into NAPALM. NAPALM has a lot of great plugin tasks that work with Nornir but the two that I used in this test were napalm_get and napalm_validate. The reason I used the get task? I needed to see how the information was returned by NAPALM so that I could then build the source file to validate against. A seasoned professional could probably just go straight to the docs, but I needed a bit more. The following NAPALM docs helped in creating this. Napalm Validate.

Below is a view of the /validate/cisco.yaml file. This was created from the link mentioned above and running the napalm_get task. Notice that the cisco.yaml file lists the name of the getter at the top or root of the file. You could lists multiple getters to validate multiple pieces of information, as long as its supported with your device.

/validate/cisco.yaml

---
- get_snmp_information:
    community:
      myreadonly:
        acl: N/A
        mode: ro
      mysecurestring:
        acl: N/A
        mode: rw
    contact: JulioPDX
    location: mylocation

Here is a snippet from each devices SNMP configuration. Looking at the validate file above and the snippets, can you guess which device will fail?

SNMP Configurations

NY-SW-01(config)#do show run | inc snmp
 snmp-server community mysecurestring RW
 snmp-server community myreadonly RO
 snmp-server location mylocation
 snmp-server contact JulioPDX

NE-RTR-01(config)#do show run | inc snmp
 snmp-server community public RO
 snmp-server community private RW
 snmp-server location mylocation
 snmp-server contact JulioPDX

OR-SW-01(config)#do show run | inc snmp
 snmp-server community myreadonly RO
 snmp-server community mysecurestring RW
 snmp-server location mylocation
 snmp-server contact JulioPDX

snmp_validate.py

# Import required modules
from nornir import InitNornir
from nornir_utils.plugins.functions import print_result
from nornir_napalm.plugins.tasks import napalm_get, napalm_validate

# Initializing Nornir
nr = InitNornir(config_file="config.yaml")

get_snmp = nr.run(name="GATHERING SNMP", task=napalm_get, getters=["get_snmp_information"])

print_result(get_snmp)

validate = nr.run(name="VALIDATE SNMP", task=napalm_validate, src="./validate/cisco.yaml")

print_result(validate)

Here is the output of the script, main parts to look at are towards the bottom when the snmp validations are printed to the screen.

(venv) juliopdx@juliopdx-virtual-2004:~/git/nornir_snmp_validation$ python snmp_validate.py
 GATHERING SNMP**
 NE-RTR-01 ** changed : False *
 vvvv GATHERING SNMP ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
 { 'get_snmp_information': { 'chassis_id': '',
                         'community': { 'private': { 'acl': 'N/A',
                                                     'mode': 'rw'},
                                        'public': { 'acl': 'N/A',
                                                    'mode': 'ro'}},
                         'contact': 'JulioPDX',
                         'location': 'mylocation'}}
 ^^^^ END GATHERING SNMP ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 NY-SW-01 ** changed : False
 vvvv GATHERING SNMP ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
 { 'get_snmp_information': { 'chassis_id': '9NA76XNH9Z8',
                         'community': { 'myreadonly': { 'acl': 'N/A',
                                                        'mode': 'ro'},
                                        'mysecurestring': { 'acl': 'N/A',
                                                            'mode': 'rw'}},
                         'contact': 'JulioPDX',
                         'location': 'mylocation'}}
 ^^^^ END GATHERING SNMP ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 OR-SW-01 ** changed : False
 vvvv GATHERING SNMP ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
 { 'get_snmp_information': { 'chassis_id': '9QA913OVFJS',
                         'community': { 'myreadonly': { 'acl': 'N/A',
                                                        'mode': 'ro'},
                                        'mysecurestring': { 'acl': 'N/A',
                                                            'mode': 'rw'}},
                         'contact': 'JulioPDX',
                         'location': 'mylocation'}}
 ^^^^ END GATHERING SNMP ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 VALIDATE SNMP***
 NE-RTR-01 ** changed : False *
 vvvv VALIDATE SNMP ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
 { 'complies': False,
 'get_snmp_information': { 'complies': False,
                         'extra': [],
                         'missing': [],
                         'present': { 'community': { 'complies': False,
                                                     'diff': { 'complies': False,
                                                               'extra': [],
                                                               'missing': [ 'myreadonly',
                                                                            'mysecurestring'],
                                                               'present': { }},
                                                     'nested': True},
                                      'contact': { 'complies': True,
                                                   'nested': False},
                                      'location': { 'complies': True,
                                                    'nested': False}}},
 'skipped': []}
 ^^^^ END VALIDATE SNMP ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 NY-SW-01 ** changed : False
 vvvv VALIDATE SNMP ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
 { 'complies': True,
 'get_snmp_information': { 'complies': True,
                         'extra': [],
                         'missing': [],
                         'present': { 'community': { 'complies': True,
                                                     'nested': True},
                                      'contact': { 'complies': True,
                                                   'nested': False},
                                      'location': { 'complies': True,
                                                    'nested': False}}},
 'skipped': []}
 ^^^^ END VALIDATE SNMP ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 OR-SW-01 ** changed : False
 vvvv VALIDATE SNMP ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
 { 'complies': True,
 'get_snmp_information': { 'complies': True,
                         'extra': [],
                         'missing': [],
                         'present': { 'community': { 'complies': True,
                                                     'nested': True},
                                      'contact': { 'complies': True,
                                                   'nested': False},
                                      'location': { 'complies': True,
                                                    'nested': False}}},
 'skipped': []}
 ^^^^ END VALIDATE SNMP ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 (venv) juliopdx@juliopdx-virtual-2004:~/git/nornir_snmp_validation$

Did you catch which one failed? If you guessed NE-RTR-01 earlier you would be correct. If you look at the output for the NE router, there is a key for “complies” and it is set to False. So now we know one device in our three device environment did not pass our SNMP validation check. This could then be tied to kick off another task to fix SNMP, report to monitoring, or some other workflow. All up to the imagination.

Outro and Links

I hope this post has shown you how NAPALM and Nornir can be leveraged to validate device configurations and dive into a bit of network automation with Python. Not too bad right? Below you’ll find some links to resources I have used and a link to the git repo. There really are so many great resources in this space for network engineers. I cant possibly list them all. Thank you for reading this far, it is much appreciated! Stay safe out there.

Creating List of IP Addresses With Ansible Filter Plugin

Tue, 02 Feb 2021 00:00:00 +0000

Intro

I recently had a use case where I needed a list of IP addresses from a prefix. My first step was to go through Ansible documentation on current filters to see if something would match my need. Long story short, there wasn’t. At least, nothing I could find which honestly is probably a huge possibility, and in that case this post is pointless. Ansible does have IP address filters but I noticed most all of the examples assume you already have a list of IP addresses to pass to the filters.

I should probably explain what a filter does just in case the reader isn’t aware. Filters allow you to transform data from something to something else. The simplest example being, turn this lower case string to upper case. See example below.

---
- name: Filter Test
  connection: local
  hosts: localhost
  gather_facts: no
  vars:
    my_variable: mylowercasestring

  tasks:
    - name: Turning lower case string to UPPER CASE
      debug:
        msg: "{{ my_variable | upper }}"

...

PLAY [Filter Test]
 TASK [Turning lower case string to UPPER CASE]
 ok: [localhost] => {
     "msg": "MYLOWERCASESTRING"
 }

As you can see, filters are pretty neat ways to transform data. So there I was trying to create a list of IP addresses from a prefix, with no end in sight, and Google-fu failing me. I’m sure there is some whacky ways to make it happen, like a jinja template and reading a file or using some kind of range/with sequence option. In the end I came up with the following python script. This can be placed in the /filter_plugins directory where your playbook is stored.

ip_list.py

#!/usr/bin/python

from netaddr import IPNetwork

def ip_stuff(prefix):
    ips = []

    for ip in IPNetwork(prefix):
        ips.append(str(ip))
    return ips

class FilterModule(object):
    def filters(self):
        return {'ip_list': ip_stuff}

Breaking down the code, we first import one library, I know some folks aren’t a fan of this on filters but oh well. Then we create an empty list called ips, we then trigger a for loop to each individual entry under the prefix that will be passed in. Each entry will be added to the ips list. After that is all done we return the final list. Shout out to the oddbit blog for giving me a place to start on creating this filter.

ip_list_test.yaml

---
- name: IP List Test
  connection: local
  hosts: localhost
  gather_facts: no
  vars:
    my_prefix: 10.10.10.0/29

  tasks:
    - name: Setting a fact for my list of IP addresses
      set_fact:
        my_ip_addresses: "{{ my_prefix | ip_list }}"

    - name: Print list of IP addresses
      debug:
        msg: "{{ my_ip_addresses }}"

...

PLAY [IP List Test] *
 TASK [Setting a fact for my list of IP addresses] ***
 ok: [localhost]
 TASK [Print list of IP addresses] *
 ok: [localhost] => {
     "msg": [
         "10.10.10.0",
         "10.10.10.1",
         "10.10.10.2",
         "10.10.10.3",
         "10.10.10.4",
         "10.10.10.5",
         "10.10.10.6",
         "10.10.10.7"
     ]
 }

That’s about it. I hope you all enjoyed this write up and now you can make a list of IP addresses from a prefix… woohoo! Thank you for reading this far.