fbpx

Network-Engineering | 6 min read

How to save time while building a data center fabric

ngworx Team
May 2021
written by Remi Locherer
Senior Network & Security Engineer

Find out how tools provided by Arista helped me in saving a lot of time while I was preparing configs for a data center network.

In May 2021, I was able to participate in a proof of concept (PoC) with Arista. For an insurance company, we built an EVPN/VXLAN data center fabric and explained the concepts and configs.

Preparing full configs for an EVPN data center fabrics can be a tedious task. Luckily, my preparation work was simplified a lot by two things: vEOS-lab and Arista’s Ansible collection AVD.

The first is a virtual implementation of an Arista switch. Unlike similar virtualized switches, it consumes relatively little resources. (I used 1vCPU and 2GB or RAM for the virtual EOS switches.) This allowed me to run it even on my Notebook. It is available as a virtual disk image for VMware, Linux KVM and (with a bit of work) for Hyper-V. This was helpful since I do not have a stack of noisy data center switches in my home office. 😉 vEOS-lab can be downloaded from Arista’s website with a free guest account.

The much bigger time-saver was the availability of Ansible collections. Arista maintains three of them. One is for managing EOS switches and another one for Cloud Vision Portal. The third one is the real gem which in my opinion differentiates Arista from other networking vendors when looking for Ansible support: AVD. The acronym stands for “Arista Validated Designs”.

The goal of this Ansible collection is not to configure an access VLAN on a single switch. Instead, the goal is to configure an entire data center network while still providing the flexibility to choose for example the preferred underlay routing protocol.

Since AVD is used by Arista engineers, you can be sure that it incorporates Arista’s best practices for the various configs it generates.

The first step for using AVD is installing Ansible. It should be at least version 2.10 since Ansible introduced major changes on how it deals with the networking modules. Then, the arista.avd collection can be installed with the ansible-galaxy command.

pip3 install ansible=2.10.7
ansible-galaxy collection install arista.avd -p /usr/share/ansible/collections

After that step, it is time to create an inventory for Ansible. An example of a simplified inventory in YAML format shows the structure.

# inventory.yml
---
POC:
  children:
    POC_FABRIC:
      children:
        POC_SPINES:
          hosts:
            POC-SPINE1:
            POC-SPINE2:
        POC_L3LEAFS:
          children:
            POC_LEAF1:
              hosts:
                POC-LEAF1A:
                POC-LEAF1B:
            POC_LEAF2:
              hosts:
                POC-LEAF2:
            POC_LEAF3:
              hosts:
                POC-LEAF3:
     
    POC_TENANTS_NETWORKS:
      children:
        POC_L3LEAFS:
 
    POC_SERVERS:
      children:
        POC_L3LEAFS:

The full structure is not absolutely required. Using this set of groups and sub-groups has the advantage that Ansible group variables can be defined and reused on each level (organization, fabric, spine layer, etc.). The group_vars file for the fabric could look like this:

# group_vars/POC_FABRIC.yml
---
fabric_name: POC_FABRIC
underlay_p2p_network_summary: 10.80.250.0/24
overlay_loopback_network_summary: 10.80.255.0/24
vtep_loopback_network_summary: 10.80.254.0/24
 
mlag_ips:
  leaf_peer_l3: 10.80.251.0/24
  mlag_peer: 10.80.252.0/24
 
vxlan_vlan_aware_bundles: true
 
spine:
  platform: default
  bgp_as: 65001
  leaf_as_range: 65101-65103
  nodes:
    POC-SPINE1:
      id: 1
      mgmt_ip: 10.80.253.101/24
    POC-SPINE2:
      id: 2
      mgmt_ip: 10.80.253.102/24

This is not the full file (leaves are missing) but it demonstrates the idea. There are many more options than can be set. For example, it can be specified that IS-IS should be used as the underlay routing protocol instead of the default eBGP.

To then generate the configs for all switches in the fabric, this playbook can be used:

# build_conf.yml
---
- name: Build Switch configuration
  hosts: all
  tasks:
 
    - name: build local folders
      tags: [build]
      import_role:
        name: arista.avd.build_output_folders
 
    - name: generate intented variables
      tags: [build]
      import_role:
        name: arista.avd.eos_designs
 
    - name: generate device intended config and documentation
      tags: [build]
      import_role:
        name: arista.avd.eos_cli_config_gen

Once this playbook has been run, the generated configs can be found in the folder intended/configs/. In addition, it also generates artifacts like documentation in mark down format or a fabric cabling list in csv format. The following snippet shows the vxlan, mlag and bgp config part generated for a leaf device:

interface Vxlan1
   vxlan source-interface Loopback1
   vxlan virtual-router encapsulation mac-address mlag-system-id
   vxlan udp-port 4789
   vxlan vlan 110 vni 10110
   vxlan vlan 111 vni 50111
   vxlan vlan 112 vni 50112
   vxlan vlan 120 vni 10120
[...]
   vxlan vrf Tenant_A_APP_Zone vni 12
   vxlan vrf Tenant_A_DB_Zone vni 13
   vxlan vrf Tenant_A_OP_Zone vni 10
[...]
!
ip virtual-router mac-address 00:1c:73:00:dc:01
!
ip address virtual source-nat vrf Tenant_A_OP_Zone address 10.255.1.3
!
ip routing
no ip routing vrf MGMT
ip routing vrf Tenant_A_APP_Zone
ip routing vrf Tenant_A_DB_Zone
ip routing vrf Tenant_A_OP_Zone
[...]
!
ip prefix-list PL-LOOPBACKS-EVPN-OVERLAY
   seq 10 permit 10.80.255.0/24 eq 32
   seq 20 permit 10.80.254.0/24 eq 32
!
mlag configuration
   domain-id POC_LEAF1
   local-interface Vlan4094
   peer-address 10.80.252.1
   peer-link Port-Channel55
   reload-delay mlag 300
   reload-delay non-mlag 330
!
ip route vrf MGMT 0.0.0.0/0 10.80.253.1
!
route-map RM-CONN-2-BGP permit 10
   match ip address prefix-list PL-LOOPBACKS-EVPN-OVERLAY
!
route-map RM-MLAG-PEER-IN permit 10
   description Make routes learned over MLAG Peer-link less preferred on spines to ensure optimal routing
   set origin incomplete
!
router bfd
   multihop interval 300 min-rx 300 multiplier 3
!
router bgp 65101
   router-id 10.80.255.3
   update wait-install
   no bgp default ipv4-unicast
   distance bgp 20 200 200
   graceful-restart restart-time 300
   graceful-restart
   maximum-paths 4 ecmp 4
   neighbor EVPN-OVERLAY-PEERS peer group
   neighbor EVPN-OVERLAY-PEERS update-source Loopback0
   neighbor EVPN-OVERLAY-PEERS bfd
   neighbor EVPN-OVERLAY-PEERS ebgp-multihop 3
   neighbor EVPN-OVERLAY-PEERS password 7 q+VNViP5i4rVjW1cxFv2wA==
   neighbor EVPN-OVERLAY-PEERS send-community
   neighbor EVPN-OVERLAY-PEERS maximum-routes 0
   neighbor IPv4-UNDERLAY-PEERS peer group
   neighbor IPv4-UNDERLAY-PEERS remote-as 65001
   neighbor IPv4-UNDERLAY-PEERS password 7 AQQvKeimxJu+uGQ/yYvv9w==
   neighbor IPv4-UNDERLAY-PEERS send-community
   neighbor IPv4-UNDERLAY-PEERS maximum-routes 12000
   neighbor MLAG-IPv4-UNDERLAY-PEER peer group
   neighbor MLAG-IPv4-UNDERLAY-PEER remote-as 65101
   neighbor MLAG-IPv4-UNDERLAY-PEER next-hop-self
   neighbor MLAG-IPv4-UNDERLAY-PEER password 7 vnEaG8gMeQf3d3cN6PktXQ==
   neighbor MLAG-IPv4-UNDERLAY-PEER send-community
   neighbor MLAG-IPv4-UNDERLAY-PEER maximum-routes 12000
   neighbor MLAG-IPv4-UNDERLAY-PEER route-map RM-MLAG-PEER-IN in
   neighbor 10.80.250.0 peer group IPv4-UNDERLAY-PEERS
   neighbor 10.80.250.2 peer group IPv4-UNDERLAY-PEERS
   neighbor 10.80.251.1 peer group MLAG-IPv4-UNDERLAY-PEER
   neighbor 10.80.255.1 peer group EVPN-OVERLAY-PEERS
   neighbor 10.80.255.1 remote-as 65001
   neighbor 10.80.255.1 description POC-SPINE1
   neighbor 10.80.255.2 peer group EVPN-OVERLAY-PEERS
   neighbor 10.80.255.2 remote-as 65001
   neighbor 10.80.255.2 description POC-SPINE2
   redistribute connected route-map RM-CONN-2-BGP
   !
   vlan-aware-bundle Tenant_A_APP_Zone
      rd 10.80.255.3:12
      route-target both 12:12
      redistribute learned
      vlan 130-131
   !
   vlan-aware-bundle Tenant_A_DB_Zone
      rd 10.80.255.3:13
      route-target both 13:13
      redistribute learned
      vlan 140-141
   !
   vlan-aware-bundle Tenant_A_NFS
[...]
   !
   vrf Tenant_A_OP_Zone
      rd 10.80.255.3:10
      route-target import evpn 10:10
      route-target export evpn 10:10
      router-id 10.80.255.3
      neighbor 10.80.251.1 peer group MLAG-IPv4-UNDERLAY-PEER
      redistribute connected
   !
   vrf Tenant_A_WAN_Zone
      rd 10.80.255.3:14
      route-target import evpn 14:14
      route-target export evpn 14:14
      router-id 10.80.255.3
      neighbor 10.80.251.1 peer group MLAG-IPv4-UNDERLAY-PEER
      redistribute connected
   !
   vrf Tenant_A_WEB_Zone
      rd 10.80.255.3:11
      route-target import evpn 11:11
      route-target export evpn 11:11
      router-id 10.80.255.3
      neighbor 10.80.251.1 peer group MLAG-IPv4-UNDERLAY-PEER
      redistribute connected
   !
   vrf Tenant_B_OP_Zone
[...]

With additional Ansible roles that are included in AVD, the configs can be pushed to the devices, either directly to the switches SSH (or eAPI) or via Cloud Vision. Once everything is configured, the setup can be verified with another role that is provided.

The use of the AVD Ansible collection is documented in great detail on its dedicated website https://avd.sh. The code behind this collection is hosted on GitHub.

The fact that I could use such a great framework like AVD not only saved me a lot of time but also gave me great confidence in the setup we showed during the PoC.

Should you wish to get some assistance in setting up Ansible and make use of AVD, feel free to contact ngworx.

ngworx Team
May 2021
written by Remi Locherer
Senior Network & Security Engineer

Most Popular

Network-Engineering | 8 min read

Junos upgrade – filesystem is full

Not enough storage during Junos upgrade (EX2300 and EX3400). An extension of Juniper's article…

Read more