Lab: Multisite Scenario

From Iwan
Jump to: navigation, search

In this article I am going to focus on the Multisite Scenario #4.

The other Multisite Scenario overview can be found here: NSX Multisite routing (diagram options)

A big thank you to Shank Mohan that pointed me in the right direction on turning off URPF mode on the Tier-0 Gateway (uplink) Interfaces. (Configuration Details → Site A (Paris) / Site B (London) → Tier-0 Gateway Configuration → Interfaces)

The diagram (from a vSphere Cluster and Edge Cluster perspective)

Untitled.png

The diagram (from a BGP routing perspective)

Untitled%201.png

Routing Tables

Untitled%202.png

Site A & Site B
WAN-router
cumulus@cumulus:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B   0.0.0.0/0 [20/0] via 192.168.110.4, swp1, weight 1, 2d03h01m
S   0.0.0.0/0 [1/0] via 192.168.0.1, swp2, weight 1, 01w0d03h
K>* 0.0.0.0/0 [0/0] via 192.168.0.1, swp2, 01w0d03h
C>* 10.10.20.0/24 is directly connected, swp4, 01w0d03h
C>* 10.10.30.0/24 is directly connected, swp4, 01w0d03h
C>* 10.20.20.0/24 is directly connected, swp4, 01w0d03h
C>* 10.20.30.0/24 is directly connected, swp4, 01w0d03h
B>* 172.16.10.0/24 [20/0] via 192.168.110.4, swp1, weight 1, 2d03h01m
B>* 172.16.11.0/24 [20/0] via 192.168.210.4, swp4, weight 1, 2d02h57m
C>* 192.168.0.0/24 is directly connected, swp2, 01w0d03h
C>* 192.168.100.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.110.0/24 is directly connected, swp1, 01w0d03h
C>* 192.168.120.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.130.0/24 is directly connected, vlan130, 3d11h49m
C>* 192.168.135.0/24 is directly connected, vlan135, 3d11h49m
C>* 192.168.140.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.150.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.200.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.210.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.220.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.230.0/24 is directly connected, vlan230, 3d11h49m
C>* 192.168.235.0/24 is directly connected, vlan235, 3d11h49m
C>* 192.168.240.0/24 is directly connected, swp4, 01w0d03h
B>* 192.168.240.10/32 [20/0] via 192.168.110.4, swp1, weight 1, 2d03h03m
B>* 192.168.240.15/32 [20/0] via 192.168.110.2, swp1, weight 1, 2d03h03m
B>* 192.168.240.20/32 [20/0] via 192.168.110.5, swp1, weight 1, 2d03h03m
B>* 192.168.240.25/32 [20/0] via 192.168.110.3, swp1, weight 1, 2d03h03m
B>* 192.168.240.30/32 [20/0] via 192.168.210.4, swp4, weight 1, 2d03h03m
B>* 192.168.240.35/32 [20/0] via 192.168.210.2, swp4, weight 1, 2d03h03m
B>* 192.168.240.40/32 [20/0] via 192.168.210.5, swp4, weight 1, 2d03h03m
B>* 192.168.240.45/32 [20/0] via 192.168.210.3, swp4, weight 1, 2d03h03m
C>* 192.168.250.0/24 is directly connected, swp4, 01w0d03h
C>* 192.168.252.254/32 is directly connected, lo, 01w0d03h
cumulus@cumulus:~$
Site A (Paris)
isp-router-01a
cumulus@isp-router-01a:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B>* 0.0.0.0/0 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h04m
B>* 172.16.10.0/24 [20/0] via 192.168.251.2, vlan150, weight 1, 2d03h01m
  *                       via 192.168.251.3, vlan150, weight 1, 2d03h01m
B>* 172.16.11.0/24 [20/0] via 192.168.110.1, swp1, weight 1, 2d02h58m
C>* 192.168.0.0/24 is directly connected, swp2, 3d12h15m
C>* 192.168.110.0/24 is directly connected, swp1, 3d12h15m
B>* 192.168.240.10/32 [20/0] via 192.168.110.4, swp1, weight 1, 2d03h04m
C>* 192.168.240.15/32 is directly connected, lo, 3d12h15m
B>* 192.168.240.20/32 [20/0] via 192.168.110.5, swp1, weight 1, 2d03h04m
B>* 192.168.240.25/32 [20/0] via 192.168.110.3, swp1, weight 1, 2d03h04m
B>* 192.168.240.30/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h04m
B>* 192.168.240.35/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h04m
B>* 192.168.240.40/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h04m
B>* 192.168.240.45/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h04m
C>* 192.168.251.0/25 is directly connected, vlan150, 2d03h04m
cumulus@isp-router-01a:~$
isp-router-02a
cumulus@isp-router-02a:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B>* 0.0.0.0/0 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h05m
B>* 172.16.10.0/24 [20/0] via 192.168.252.2, vlan250, weight 1, 2d03h02m
  *                       via 192.168.252.3, vlan250, weight 1, 2d03h02m
B>* 172.16.11.0/24 [20/0] via 192.168.110.1, swp1, weight 1, 2d02h59m
C>* 192.168.0.0/24 is directly connected, swp2, 3d12h04m
C>* 192.168.110.0/24 is directly connected, swp1, 3d12h04m
B>* 192.168.240.10/32 [20/0] via 192.168.110.4, swp1, weight 1, 2d03h05m
B>* 192.168.240.15/32 [20/0] via 192.168.110.2, swp1, weight 1, 2d03h05m
B>* 192.168.240.20/32 [20/0] via 192.168.110.5, swp1, weight 1, 2d03h05m
C>* 192.168.240.25/32 is directly connected, lo, 3d12h04m
B>* 192.168.240.30/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h05m
B>* 192.168.240.35/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h05m
B>* 192.168.240.40/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h05m
B>* 192.168.240.45/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h05m
C>* 192.168.252.0/25 is directly connected, vlan250, 2d03h05m
cumulus@isp-router-02a:~$
wan-router-01a
cumulus@wan-router-01a:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B   0.0.0.0/0 [20/0] via 192.168.253.2, vlan100, weight 1, 2d03h03m
                     via 192.168.253.3, vlan100, weight 1, 2d03h03m
S   0.0.0.0/0 [1/0] via 192.168.0.1, swp2, weight 1, 01w1d07h
K>* 0.0.0.0/0 [0/0] via 192.168.0.1, swp2, 01w1d07h
B>* 172.16.10.0/24 [20/0] via 192.168.253.2, vlan100, weight 1, 2d03h03m
  *                       via 192.168.253.3, vlan100, weight 1, 2d03h03m
B>* 172.16.11.0/24 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h00m
C>* 192.168.0.0/24 is directly connected, swp2, 01w1d07h
C>* 192.168.110.0/24 is directly connected, swp1, 01w1d07h
C>* 192.168.240.10/32 is directly connected, lo, 01w1d07h
B>* 192.168.240.15/32 [20/0] via 192.168.110.2, swp1, weight 1, 2d03h06m
B>* 192.168.240.20/32 [20/0] via 192.168.110.5, swp1, weight 1, 2d03h06m
B>* 192.168.240.25/32 [20/0] via 192.168.110.3, swp1, weight 1, 2d03h06m
B>* 192.168.240.30/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h06m
B>* 192.168.240.35/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h06m
B>* 192.168.240.40/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h06m
B>* 192.168.240.45/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h06m
C>* 192.168.253.0/25 is directly connected, vlan100, 2d03h06m
cumulus@wan-router-01a:~$
wan-router-02a
cumulus@wan-router-02a:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B   0.0.0.0/0 [20/0] via 192.168.254.2, vlan200, weight 1, 2d03h03m
                     via 192.168.254.3, vlan200, weight 1, 2d03h03m
S   0.0.0.0/0 [1/0] via 192.168.0.1, swp2, weight 1, 01w1d07h
K>* 0.0.0.0/0 [0/0] via 192.168.0.1, swp2, 01w1d07h
B>* 172.16.10.0/24 [20/0] via 192.168.254.2, vlan200, weight 1, 2d03h04m
  *                       via 192.168.254.3, vlan200, weight 1, 2d03h04m
B>* 172.16.11.0/24 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h01m
C>* 192.168.0.0/24 is directly connected, swp2, 01w1d07h
C>* 192.168.110.0/24 is directly connected, swp1, 01w1d07h
B>* 192.168.240.10/32 [20/0] via 192.168.110.4, swp1, weight 1, 2d03h07m
B>* 192.168.240.15/32 [20/0] via 192.168.110.2, swp1, weight 1, 2d03h07m
C>* 192.168.240.20/32 is directly connected, lo, 01w1d07h
B>* 192.168.240.25/32 [20/0] via 192.168.110.3, swp1, weight 1, 2d03h07m
B>* 192.168.240.30/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h07m
B>* 192.168.240.35/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h07m
B>* 192.168.240.40/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h07m
B>* 192.168.240.45/32 [20/0] via 192.168.110.1, swp1, weight 1, 2d03h07m
C>* 192.168.254.0/25 is directly connected, vlan200, 2d03h07m
cumulus@wan-router-02a:~$
Tier-0 SR edge-01a
edge-01a(tier0_sr[2])> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP, o - OSPF
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 19

b  > * 0.0.0.0/0 [20/0] via 192.168.251.1, uplink-297, 2d03h15m
b  > * 0.0.0.0/0 [20/0] via 192.168.252.1, uplink-291, 2d03h15m
t0c> * 100.64.200.0/31 is directly connected, linked-283, 4d14h03m
t0c> * 169.254.0.0/25 is directly connected, downlink-293, 2d03h15m
isr> * 169.254.0.128/25 is directly connected, inter-sr-287, 5d02h38m
t1c> * 172.16.10.0/24 [3/0] via 100.64.200.1, linked-283, 4d14h03m
b  > * 172.16.11.0/24 [20/0] via 192.168.253.1, uplink-299, 2d03h11m
b  > * 172.16.11.0/24 [20/0] via 192.168.254.1, uplink-295, 2d03h11m
b  > * 172.16.11.0/24 [20/0] via 192.168.251.1, uplink-297, 2d03h11m
b  > * 172.16.11.0/24 [20/0] via 192.168.252.1, uplink-291, 2d03h11m
t0s> * 192.168.110.0/24 [1/0] via 192.168.254.1, uplink-295, 5d02h38m
b  > * 192.168.240.10/32 [20/0] via 192.168.253.1, uplink-299, 2d03h15m
b  > * 192.168.240.15/32 [20/0] via 192.168.251.1, uplink-297, 2d03h15m
b  > * 192.168.240.20/32 [20/0] via 192.168.254.1, uplink-295, 2d03h15m
b  > * 192.168.240.25/32 [20/0] via 192.168.252.1, uplink-291, 2d03h15m
b  > * 192.168.240.30/32 [20/0] via 192.168.253.1, uplink-299, 2d03h15m
b  > * 192.168.240.30/32 [20/0] via 192.168.254.1, uplink-295, 2d03h15m
b  > * 192.168.240.30/32 [20/0] via 192.168.251.1, uplink-297, 2d03h15m
b  > * 192.168.240.30/32 [20/0] via 192.168.252.1, uplink-291, 2d03h15m
b  > * 192.168.240.35/32 [20/0] via 192.168.253.1, uplink-299, 2d03h15m
b  > * 192.168.240.35/32 [20/0] via 192.168.254.1, uplink-295, 2d03h15m
b  > * 192.168.240.35/32 [20/0] via 192.168.251.1, uplink-297, 2d03h15m
b  > * 192.168.240.35/32 [20/0] via 192.168.252.1, uplink-291, 2d03h15m
b  > * 192.168.240.40/32 [20/0] via 192.168.253.1, uplink-299, 2d03h15m
b  > * 192.168.240.40/32 [20/0] via 192.168.254.1, uplink-295, 2d03h15m
b  > * 192.168.240.40/32 [20/0] via 192.168.251.1, uplink-297, 2d03h15m
b  > * 192.168.240.40/32 [20/0] via 192.168.252.1, uplink-291, 2d03h15m
b  > * 192.168.240.45/32 [20/0] via 192.168.253.1, uplink-299, 2d03h15m
b  > * 192.168.240.45/32 [20/0] via 192.168.254.1, uplink-295, 2d03h15m
b  > * 192.168.240.45/32 [20/0] via 192.168.251.1, uplink-297, 2d03h15m
b  > * 192.168.240.45/32 [20/0] via 192.168.252.1, uplink-291, 2d03h15m
t0c> * 192.168.251.0/25 is directly connected, uplink-297, 5d02h38m
t0c> * 192.168.252.0/25 is directly connected, uplink-291, 5d02h38m
t0c> * 192.168.253.0/25 is directly connected, uplink-299, 5d02h38m
t0c> * 192.168.254.0/25 is directly connected, uplink-295, 5d02h38m
Mon Sep 26 2022 UTC 19:21:15.250
edge-01a(tier0_sr[2])>
Tier-0 SR edge-02a
edge-02a(tier0_sr[2])> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP, o - OSPF
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 19

b  > * 0.0.0.0/0 [20/0] via 192.168.251.1, uplink-292, 2d03h16m
b  > * 0.0.0.0/0 [20/0] via 192.168.252.1, uplink-296, 2d03h16m
t0c> * 100.64.200.0/31 is directly connected, linked-284, 4d14h04m
t0c> * 169.254.0.0/25 is directly connected, downlink-294, 2d03h13m
isr> * 169.254.0.128/25 is directly connected, inter-sr-298, 5d02h38m
t1c> * 172.16.10.0/24 [3/0] via 100.64.200.1, linked-284, 4d14h04m
b  > * 172.16.11.0/24 [20/0] via 192.168.253.1, uplink-282, 2d03h13m
b  > * 172.16.11.0/24 [20/0] via 192.168.251.1, uplink-292, 2d03h13m
b  > * 172.16.11.0/24 [20/0] via 192.168.254.1, uplink-290, 2d03h13m
b  > * 172.16.11.0/24 [20/0] via 192.168.252.1, uplink-296, 2d03h13m
t0s> * 192.168.110.0/24 [1/0] via 192.168.254.1, uplink-290, 5d02h38m
b  > * 192.168.240.10/32 [20/0] via 192.168.253.1, uplink-282, 2d03h16m
b  > * 192.168.240.15/32 [20/0] via 192.168.251.1, uplink-292, 2d03h16m
b  > * 192.168.240.20/32 [20/0] via 192.168.254.1, uplink-290, 2d03h16m
b  > * 192.168.240.25/32 [20/0] via 192.168.252.1, uplink-296, 2d03h16m
b  > * 192.168.240.30/32 [20/0] via 192.168.253.1, uplink-282, 2d03h16m
b  > * 192.168.240.30/32 [20/0] via 192.168.252.1, uplink-296, 2d03h16m
b  > * 192.168.240.30/32 [20/0] via 192.168.251.1, uplink-292, 2d03h16m
b  > * 192.168.240.30/32 [20/0] via 192.168.254.1, uplink-290, 2d03h16m
b  > * 192.168.240.35/32 [20/0] via 192.168.253.1, uplink-282, 2d03h16m
b  > * 192.168.240.35/32 [20/0] via 192.168.252.1, uplink-296, 2d03h16m
b  > * 192.168.240.35/32 [20/0] via 192.168.251.1, uplink-292, 2d03h16m
b  > * 192.168.240.35/32 [20/0] via 192.168.254.1, uplink-290, 2d03h16m
b  > * 192.168.240.40/32 [20/0] via 192.168.253.1, uplink-282, 2d03h16m
b  > * 192.168.240.40/32 [20/0] via 192.168.252.1, uplink-296, 2d03h16m
b  > * 192.168.240.40/32 [20/0] via 192.168.251.1, uplink-292, 2d03h16m
b  > * 192.168.240.40/32 [20/0] via 192.168.254.1, uplink-290, 2d03h16m
b  > * 192.168.240.45/32 [20/0] via 192.168.253.1, uplink-282, 2d03h16m
b  > * 192.168.240.45/32 [20/0] via 192.168.252.1, uplink-296, 2d03h16m
b  > * 192.168.240.45/32 [20/0] via 192.168.251.1, uplink-292, 2d03h16m
b  > * 192.168.240.45/32 [20/0] via 192.168.254.1, uplink-290, 2d03h16m
t0c> * 192.168.251.0/25 is directly connected, uplink-292, 5d02h38m
t0c> * 192.168.252.0/25 is directly connected, uplink-296, 5d02h38m
t0c> * 192.168.253.0/25 is directly connected, uplink-282, 5d02h38m
t0c> * 192.168.254.0/25 is directly connected, uplink-290, 5d02h38m
Mon Sep 26 2022 UTC 19:22:46.239
edge-02a(tier0_sr[2])>
Tier-1 DR edge-01a
edge-01a(vrf[4])> get forwarding
Mon Sep 26 2022 UTC 19:28:01.837
Logical Router
UUID                                   VRF    LR-ID  Name                              Type
a236ecd1-6da9-4a0c-b6d2-a973ed101521   4      4098   DR-t1-gateway-a                   DISTRIBUTED_ROUTER_TIER1
IPv4 Forwarding Table
IP Prefix          Gateway IP                                Type        UUID                                   Gateway MAC
0.0.0.0/0          100.64.200.0                              route       00fe72fd-3666-4ed3-994b-6031aa6e51ee
100.64.200.0/31                                              route       00fe72fd-3666-4ed3-994b-6031aa6e51ee
100.64.200.1/32                                              route       d2ca1c4a-197b-53c9-8dd7-2f559a8b7eb1
169.254.0.0/28                                               route       7adf8516-7532-4e17-a587-c690f6b92b49
169.254.0.1/32                                               route       d2ca1c4a-197b-53c9-8dd7-2f559a8b7eb1
172.16.10.0/24                                               route       7b520e9e-9a1b-4f29-95e1-2cc883011142
172.16.10.1/32                                               route       d2ca1c4a-197b-53c9-8dd7-2f559a8b7eb1
edge-01a(vrf[4])>
Tier-1 DR edge-02a
edge-02a(vrf[4])> get forwarding
Mon Sep 26 2022 UTC 19:29:14.546
Logical Router
UUID                                   VRF    LR-ID  Name                              Type
a236ecd1-6da9-4a0c-b6d2-a973ed101521   4      4098   DR-t1-gateway-a                   DISTRIBUTED_ROUTER_TIER1
IPv4 Forwarding Table
IP Prefix          Gateway IP                                Type        UUID                                   Gateway MAC
0.0.0.0/0          100.64.200.0                              route       00fe72fd-3666-4ed3-994b-6031aa6e51ee
100.64.200.0/31                                              route       00fe72fd-3666-4ed3-994b-6031aa6e51ee
100.64.200.1/32                                              route       d2ca1c4a-197b-53c9-8dd7-2f559a8b7eb1
169.254.0.0/28                                               route       7adf8516-7532-4e17-a587-c690f6b92b49
169.254.0.1/32                                               route       d2ca1c4a-197b-53c9-8dd7-2f559a8b7eb1
172.16.10.0/24                                               route       7b520e9e-9a1b-4f29-95e1-2cc883011142
172.16.10.1/32                                               route       d2ca1c4a-197b-53c9-8dd7-2f559a8b7eb1
edge-02a(vrf[4])>
Site B (London)
isp-router-01b
cumulus@isp-router-01b:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B>* 0.0.0.0/0 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h08m
B>* 172.16.10.0/24 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h06m
B>* 172.16.11.0/24 [20/0] via 192.168.251.130, vlan350, weight 1, 2d03h02m
  *                       via 192.168.251.131, vlan350, weight 1, 2d03h02m
C>* 192.168.0.0/24 is directly connected, swp2, 3d12h07m
S>* 192.168.110.0/24 [1/0] via 192.168.210.1, swp1, weight 1, 3d12h07m
C>* 192.168.210.0/24 is directly connected, swp1, 3d12h07m
B>* 192.168.240.10/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h08m
B>* 192.168.240.15/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h08m
B>* 192.168.240.20/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h08m
B>* 192.168.240.25/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h08m
B>* 192.168.240.30/32 [20/0] via 192.168.210.4, swp1, weight 1, 2d03h08m
C>* 192.168.240.35/32 is directly connected, lo, 3d12h07m
B>* 192.168.240.40/32 [20/0] via 192.168.210.5, swp1, weight 1, 2d03h08m
B>* 192.168.240.45/32 [20/0] via 192.168.210.3, swp1, weight 1, 2d03h08m
C>* 192.168.251.128/25 is directly connected, vlan350, 2d03h09m
cumulus@isp-router-01b:~$
isp-router-02b
cumulus@isp-router-02b:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B>* 0.0.0.0/0 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h09m
B>* 172.16.10.0/24 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h07m
B>* 172.16.11.0/24 [20/0] via 192.168.252.130, vlan450, weight 1, 2d03h03m
  *                       via 192.168.252.131, vlan450, weight 1, 2d03h03m
C>* 192.168.0.0/24 is directly connected, swp2, 3d12h08m
S>* 192.168.110.0/24 [1/0] via 192.168.210.1, swp1, weight 1, 3d12h08m
C>* 192.168.210.0/24 is directly connected, swp1, 3d12h08m
B>* 192.168.240.10/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h09m
B>* 192.168.240.15/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h09m
B>* 192.168.240.20/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h09m
B>* 192.168.240.25/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h09m
B>* 192.168.240.30/32 [20/0] via 192.168.210.4, swp1, weight 1, 2d03h09m
B>* 192.168.240.35/32 [20/0] via 192.168.210.2, swp1, weight 1, 2d03h09m
B>* 192.168.240.40/32 [20/0] via 192.168.210.5, swp1, weight 1, 2d03h09m
C>* 192.168.240.45/32 is directly connected, lo, 3d12h08m
C>* 192.168.252.128/25 is directly connected, vlan450, 3d11h56m
cumulus@isp-router-02b:~$
wan-router-01b
cumulus@wan-router-01b:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B   0.0.0.0/0 [20/0] via 192.168.253.130, vlan300, weight 1, 2d03h04m
                     via 192.168.253.131, vlan300, weight 1, 2d03h04m
S   0.0.0.0/0 [1/0] via 192.168.0.1, swp2, weight 1, 01w1d05h
K>* 0.0.0.0/0 [0/0] via 192.168.0.1, swp2, 01w1d05h
B>* 172.16.10.0/24 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h08m
B>* 172.16.11.0/24 [20/0] via 192.168.253.130, vlan300, weight 1, 2d03h04m
  *                       via 192.168.253.131, vlan300, weight 1, 2d03h04m
C>* 192.168.0.0/24 is directly connected, swp2, 01w1d05h
S>* 192.168.110.0/24 [1/0] via 192.168.210.1, swp1, weight 1, 01w0d03h
C>* 192.168.210.0/24 is directly connected, swp1, 01w1d05h
B>* 192.168.240.10/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h10m
B>* 192.168.240.15/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h10m
B>* 192.168.240.20/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h10m
B>* 192.168.240.25/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h10m
C>* 192.168.240.30/32 is directly connected, lo, 01w1d05h
B>* 192.168.240.35/32 [20/0] via 192.168.210.2, swp1, weight 1, 2d03h10m
B>* 192.168.240.40/32 [20/0] via 192.168.210.5, swp1, weight 1, 2d03h10m
B>* 192.168.240.45/32 [20/0] via 192.168.210.3, swp1, weight 1, 2d03h10m
C>* 192.168.253.128/25 is directly connected, vlan300, 2d03h10m
cumulus@wan-router-01b:~$
wan-router-02b
cumulus@wan-router-02b:~$ net show route ipv4
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B   0.0.0.0/0 [20/0] via 192.168.254.130, vlan400, weight 1, 2d03h05m
                     via 192.168.254.131, vlan400, weight 1, 2d03h05m
S   0.0.0.0/0 [1/0] via 192.168.0.1, swp2, weight 1, 01w1d05h
K>* 0.0.0.0/0 [0/0] via 192.168.0.1, swp2, 01w1d05h
B>* 172.16.10.0/24 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h09m
B>* 172.16.11.0/24 [20/0] via 192.168.254.130, vlan400, weight 1, 2d03h05m
  *                       via 192.168.254.131, vlan400, weight 1, 2d03h05m
C>* 192.168.0.0/24 is directly connected, swp2, 01w1d05h
S>* 192.168.110.0/24 [1/0] via 192.168.210.1, swp1, weight 1, 01w0d03h
C>* 192.168.210.0/24 is directly connected, swp1, 01w1d05h
B>* 192.168.240.10/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h11m
B>* 192.168.240.15/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h11m
B>* 192.168.240.20/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h11m
B>* 192.168.240.25/32 [20/0] via 192.168.210.1, swp1, weight 1, 2d03h11m
B>* 192.168.240.30/32 [20/0] via 192.168.210.4, swp1, weight 1, 2d03h11m
B>* 192.168.240.35/32 [20/0] via 192.168.210.2, swp1, weight 1, 2d03h11m
C>* 192.168.240.40/32 is directly connected, lo, 01w1d05h
B>* 192.168.240.45/32 [20/0] via 192.168.210.3, swp1, weight 1, 2d03h11m
C>* 192.168.254.128/25 is directly connected, vlan400, 2d03h11m
cumulus@wan-router-02b:~$
Tier-0 SR edge-01b
edge-01b(tier0_sr[1])> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP, o - OSPF
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 19

b  > * 0.0.0.0/0 [20/0] via 192.168.251.129, uplink-295, 18:28:36
b  > * 0.0.0.0/0 [20/0] via 192.168.252.129, uplink-282, 18:28:36
t0c> * 100.64.0.0/31 is directly connected, linked-278, 3d03h18m
t0c> * 169.254.0.0/25 is directly connected, downlink-280, 18:28:38
isr> * 169.254.0.128/25 is directly connected, inter-sr-287, 3d03h18m
b  > * 172.16.10.0/24 [20/0] via 192.168.253.129, uplink-276, 18:28:35
b  > * 172.16.10.0/24 [20/0] via 192.168.254.129, uplink-289, 18:28:35
b  > * 172.16.10.0/24 [20/0] via 192.168.252.129, uplink-282, 18:28:35
b  > * 172.16.10.0/24 [20/0] via 192.168.251.129, uplink-295, 18:28:35
t1c> * 172.16.11.0/24 [3/0] via 100.64.0.1, linked-278, 2d03h14m
b  > * 192.168.110.0/24 [20/0] via 192.168.253.129, uplink-276, 17:07:24
b  > * 192.168.110.0/24 [20/0] via 192.168.254.129, uplink-289, 17:07:24
b  > * 192.168.240.10/32 [20/0] via 192.168.254.129, uplink-289, 18:28:36
b  > * 192.168.240.10/32 [20/0] via 192.168.252.129, uplink-282, 18:28:36
b  > * 192.168.240.10/32 [20/0] via 192.168.253.129, uplink-276, 18:28:36
b  > * 192.168.240.10/32 [20/0] via 192.168.251.129, uplink-295, 18:28:36
b  > * 192.168.240.15/32 [20/0] via 192.168.254.129, uplink-289, 18:28:36
b  > * 192.168.240.15/32 [20/0] via 192.168.252.129, uplink-282, 18:28:36
b  > * 192.168.240.15/32 [20/0] via 192.168.253.129, uplink-276, 18:28:36
b  > * 192.168.240.15/32 [20/0] via 192.168.251.129, uplink-295, 18:28:36
b  > * 192.168.240.20/32 [20/0] via 192.168.254.129, uplink-289, 18:28:36
b  > * 192.168.240.20/32 [20/0] via 192.168.252.129, uplink-282, 18:28:36
b  > * 192.168.240.20/32 [20/0] via 192.168.253.129, uplink-276, 18:28:36
b  > * 192.168.240.20/32 [20/0] via 192.168.251.129, uplink-295, 18:28:36
b  > * 192.168.240.25/32 [20/0] via 192.168.254.129, uplink-289, 18:28:36
b  > * 192.168.240.25/32 [20/0] via 192.168.252.129, uplink-282, 18:28:36
b  > * 192.168.240.25/32 [20/0] via 192.168.253.129, uplink-276, 18:28:36
b  > * 192.168.240.25/32 [20/0] via 192.168.251.129, uplink-295, 18:28:36
b  > * 192.168.240.30/32 [20/0] via 192.168.253.129, uplink-276, 18:28:36
b  > * 192.168.240.35/32 [20/0] via 192.168.251.129, uplink-295, 18:28:36
b  > * 192.168.240.40/32 [20/0] via 192.168.254.129, uplink-289, 18:28:37
b  > * 192.168.240.45/32 [20/0] via 192.168.252.129, uplink-282, 18:28:36
t0c> * 192.168.251.128/25 is directly connected, uplink-295, 3d03h18m
t0c> * 192.168.252.128/25 is directly connected, uplink-282, 3d03h18m
t0c> * 192.168.253.128/25 is directly connected, uplink-276, 3d03h18m
t0c> * 192.168.254.128/25 is directly connected, uplink-289, 3d03h18m
Sun Sep 25 2022 UTC 10:32:29.158
edge-01b(tier0_sr[1])>
Tier-0 SR edge-02b
edge-02b(tier0_sr[1])> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP, o - OSPF
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 19

b  > * 0.0.0.0/0 [20/0] via 192.168.251.129, uplink-286, 18:27:40
b  > * 0.0.0.0/0 [20/0] via 192.168.252.129, uplink-284, 18:27:40
t0c> * 100.64.0.0/31 is directly connected, linked-282, 3d03h12m
t0c> * 169.254.0.0/25 is directly connected, downlink-280, 18:27:37
isr> * 169.254.0.128/25 is directly connected, inter-sr-279, 3d03h12m
b  > * 172.16.10.0/24 [20/0] via 192.168.253.129, uplink-283, 18:27:34
b  > * 172.16.10.0/24 [20/0] via 192.168.254.129, uplink-273, 18:27:34
b  > * 172.16.10.0/24 [20/0] via 192.168.252.129, uplink-284, 18:27:34
b  > * 172.16.10.0/24 [20/0] via 192.168.251.129, uplink-286, 18:27:34
t1c> * 172.16.11.0/24 [3/0] via 100.64.0.1, linked-282, 3d03h12m
b  > * 192.168.110.0/24 [20/0] via 192.168.253.129, uplink-283, 17:06:23
b  > * 192.168.110.0/24 [20/0] via 192.168.254.129, uplink-273, 17:06:23
b  > * 192.168.240.10/32 [20/0] via 192.168.251.129, uplink-286, 18:27:40
b  > * 192.168.240.10/32 [20/0] via 192.168.252.129, uplink-284, 18:27:40
b  > * 192.168.240.10/32 [20/0] via 192.168.254.129, uplink-273, 18:27:40
b  > * 192.168.240.10/32 [20/0] via 192.168.253.129, uplink-283, 18:27:40
b  > * 192.168.240.15/32 [20/0] via 192.168.251.129, uplink-286, 18:27:40
b  > * 192.168.240.15/32 [20/0] via 192.168.252.129, uplink-284, 18:27:40
b  > * 192.168.240.15/32 [20/0] via 192.168.254.129, uplink-273, 18:27:40
b  > * 192.168.240.15/32 [20/0] via 192.168.253.129, uplink-283, 18:27:40
b  > * 192.168.240.20/32 [20/0] via 192.168.251.129, uplink-286, 18:27:40
b  > * 192.168.240.20/32 [20/0] via 192.168.252.129, uplink-284, 18:27:40
b  > * 192.168.240.20/32 [20/0] via 192.168.254.129, uplink-273, 18:27:40
b  > * 192.168.240.20/32 [20/0] via 192.168.253.129, uplink-283, 18:27:40
b  > * 192.168.240.25/32 [20/0] via 192.168.251.129, uplink-286, 18:27:40
b  > * 192.168.240.25/32 [20/0] via 192.168.252.129, uplink-284, 18:27:40
b  > * 192.168.240.25/32 [20/0] via 192.168.254.129, uplink-273, 18:27:40
b  > * 192.168.240.25/32 [20/0] via 192.168.253.129, uplink-283, 18:27:40
b  > * 192.168.240.30/32 [20/0] via 192.168.253.129, uplink-283, 18:27:40
b  > * 192.168.240.35/32 [20/0] via 192.168.251.129, uplink-286, 18:27:40
b  > * 192.168.240.40/32 [20/0] via 192.168.254.129, uplink-273, 18:27:40
b  > * 192.168.240.45/32 [20/0] via 192.168.252.129, uplink-284, 18:27:40
t0c> * 192.168.251.128/25 is directly connected, uplink-286, 3d03h12m
t0c> * 192.168.252.128/25 is directly connected, uplink-284, 3d03h12m
t0c> * 192.168.253.128/25 is directly connected, uplink-283, 3d03h12m
t0c> * 192.168.254.128/25 is directly connected, uplink-273, 3d03h12m
Sun Sep 25 2022 UTC 10:31:28.076
edge-02b(tier0_sr[1])>
Tier-1 DR edge-01b
edge-01b(vrf[4])> get forwarding
Mon Sep 26 2022 UTC 19:31:00.285
Logical Router
UUID                                   VRF    LR-ID  Name                              Type
90359865-a641-420b-8d93-3b903b8f6dbd   4      4100   DR-t1-gateway-b                   DISTRIBUTED_ROUTER_TIER1
IPv4 Forwarding Table
IP Prefix          Gateway IP                                Type        UUID                                   Gateway MAC
0.0.0.0/0          100.64.0.0                                route       47408ff9-eaac-4971-834a-dc216dccfa27
100.64.0.0/31                                                route       47408ff9-eaac-4971-834a-dc216dccfa27
100.64.0.1/32                                                route       9da6ab96-5342-58e6-b34d-19cec352c44d
169.254.0.0/28                                               route       49a360a9-58c4-4ece-8fcf-80fefc033ed9
169.254.0.1/32                                               route       9da6ab96-5342-58e6-b34d-19cec352c44d
172.16.11.0/24                                               route       1b1dddbd-b669-4c35-8b99-c762982fe31e
172.16.11.1/32                                               route       9da6ab96-5342-58e6-b34d-19cec352c44d
edge-01b(vrf[4])>
Tier-1 DR edge-02b
edge-02b(vrf[4])> get forwarding
Mon Sep 26 2022 UTC 19:32:26.708
Logical Router
UUID                                   VRF    LR-ID  Name                              Type
90359865-a641-420b-8d93-3b903b8f6dbd   4      4100   DR-t1-gateway-b                   DISTRIBUTED_ROUTER_TIER1
IPv4 Forwarding Table
IP Prefix          Gateway IP                                Type        UUID                                   Gateway MAC
0.0.0.0/0          100.64.0.0                                route       47408ff9-eaac-4971-834a-dc216dccfa27
100.64.0.0/31                                                route       47408ff9-eaac-4971-834a-dc216dccfa27
100.64.0.1/32                                                route       9da6ab96-5342-58e6-b34d-19cec352c44d
169.254.0.0/28                                               route       49a360a9-58c4-4ece-8fcf-80fefc033ed9
169.254.0.1/32                                               route       9da6ab96-5342-58e6-b34d-19cec352c44d
172.16.11.0/24                                               route       1b1dddbd-b669-4c35-8b99-c762982fe31e
172.16.11.1/32                                               route       9da6ab96-5342-58e6-b34d-19cec352c44d
edge-02b(vrf[4])>

Pings, Trace-routes, VM Accessibility when both sites are up

Ping

Web-01a → Control Center
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:8C:12:73
          inet addr:172.16.10.2  Bcast:172.16.10.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe8c:1273/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10554 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5909 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:799180 (780.4 KiB)  TX bytes:578358 (564.8 KiB)

tc@TinyVM:~$ ping 192.168.110.10 -c 4
PING 192.168.110.10 (192.168.110.10): 56 data bytes
64 bytes from 192.168.110.10: seq=0 ttl=125 time=2.569 ms
64 bytes from 192.168.110.10: seq=1 ttl=125 time=93.069 ms
64 bytes from 192.168.110.10: seq=2 ttl=125 time=2.002 ms
64 bytes from 192.168.110.10: seq=3 ttl=125 time=1.527 ms

--- 192.168.110.10 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 1.527/24.791/93.069 ms
Web-01a → Internet
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:8C:12:73
          inet addr:172.16.10.2  Bcast:172.16.10.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe8c:1273/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10554 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5909 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:799180 (780.4 KiB)  TX bytes:578358 (564.8 KiB)

tc@TinyVM:~$ ping 8.8.8.8 -c 4
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=108 time=47.391 ms
64 bytes from 8.8.8.8: seq=1 ttl=108 time=47.808 ms
64 bytes from 8.8.8.8: seq=2 ttl=108 time=47.517 ms
64 bytes from 8.8.8.8: seq=3 ttl=108 time=48.103 ms

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 47.391/47.704/48.103 ms
tc@TinyVM:~$
DB-01a → Control Center
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:8C:2D:19
          inet addr:172.16.11.2  Bcast:172.16.11.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe8c:2d19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9855 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5272 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:744880 (727.4 KiB)  TX bytes:538006 (525.3 KiB)

tc@TinyVM:~$ ping 192.168.110.10 -c 4
PING 192.168.110.10 (192.168.110.10): 56 data bytes
64 bytes from 192.168.110.10: seq=0 ttl=124 time=6.543 ms
64 bytes from 192.168.110.10: seq=1 ttl=124 time=2.752 ms
64 bytes from 192.168.110.10: seq=2 ttl=124 time=2.651 ms
64 bytes from 192.168.110.10: seq=3 ttl=124 time=2.256 ms

--- 192.168.110.10 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 2.256/3.550/6.543 ms
DB-01a → Internet
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:8C:2D:19
          inet addr:172.16.11.2  Bcast:172.16.11.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe8c:2d19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9855 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5272 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:744880 (727.4 KiB)  TX bytes:538006 (525.3 KiB)

tc@TinyVM:~$ ping 8.8.8.8 -c 4
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=108 time=47.748 ms
64 bytes from 8.8.8.8: seq=1 ttl=108 time=47.742 ms
64 bytes from 8.8.8.8: seq=2 ttl=108 time=47.844 ms
64 bytes from 8.8.8.8: seq=3 ttl=108 time=47.770 ms

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 47.742/47.776/47.844 ms
tc@TinyVM:~$
Web-01b → Control Center
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:9C:DB:6D
          inet addr:172.16.10.3  Bcast:172.16.10.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe9c:db6d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9803 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5184 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:738929 (721.6 KiB)  TX bytes:500032 (488.3 KiB)

tc@TinyVM:~$ ping 192.168.110.10 -c 4
PING 192.168.110.10 (192.168.110.10): 56 data bytes
64 bytes from 192.168.110.10: seq=0 ttl=125 time=5.184 ms
64 bytes from 192.168.110.10: seq=1 ttl=125 time=1.715 ms
64 bytes from 192.168.110.10: seq=2 ttl=125 time=1.989 ms
64 bytes from 192.168.110.10: seq=3 ttl=125 time=2.531 ms

--- 192.168.110.10 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 1.715/2.854/5.184 ms
Web-01b → Internet
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:9C:DB:6D
          inet addr:172.16.10.3  Bcast:172.16.10.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe9c:db6d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9803 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5184 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:738929 (721.6 KiB)  TX bytes:500032 (488.3 KiB)

tc@TinyVM:~$ ping 8.8.8.8 -c 4
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=108 time=48.446 ms
64 bytes from 8.8.8.8: seq=1 ttl=108 time=47.487 ms
64 bytes from 8.8.8.8: seq=2 ttl=108 time=47.487 ms
64 bytes from 8.8.8.8: seq=3 ttl=108 time=47.838 ms

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 47.487/47.814/48.446 ms
tc@TinyVM:~$
DB-01b → Control Center
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:9C:73:16
          inet addr:172.16.11.3  Bcast:172.16.11.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe9c:7316/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10043 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5387 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:756976 (739.2 KiB)  TX bytes:585492 (571.7 KiB)

tc@TinyVM:~$ ping 192.168.110.10 -c 4
PING 192.168.110.10 (192.168.110.10): 56 data bytes
64 bytes from 192.168.110.10: seq=0 ttl=124 time=3.954 ms
64 bytes from 192.168.110.10: seq=1 ttl=124 time=2.021 ms
64 bytes from 192.168.110.10: seq=2 ttl=124 time=2.241 ms
64 bytes from 192.168.110.10: seq=3 ttl=124 time=2.189 ms

--- 192.168.110.10 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 2.021/2.601/3.954 ms
DB-01b → Internet
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:9C:73:16
          inet addr:172.16.11.3  Bcast:172.16.11.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe9c:7316/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10043 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5387 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:756976 (739.2 KiB)  TX bytes:585492 (571.7 KiB)

tc@TinyVM:~$ ping 8.8.8.8 -c 4
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=108 time=50.027 ms
64 bytes from 8.8.8.8: seq=1 ttl=108 time=47.533 ms
64 bytes from 8.8.8.8: seq=2 ttl=108 time=47.626 ms
64 bytes from 8.8.8.8: seq=3 ttl=108 time=47.285 ms

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 47.285/48.117/50.027 ms
tc@TinyVM:~$
Control Center → Web-01a
C:\Users\Administrator>ping 172.16.10.2

Pinging 172.16.10.2 with 32 bytes of data:
Reply from 172.16.10.2: bytes=32 time=2ms TTL=61
Reply from 172.16.10.2: bytes=32 time=2ms TTL=61
Reply from 172.16.10.2: bytes=32 time=1ms TTL=61
Reply from 172.16.10.2: bytes=32 time=1ms TTL=61

Ping statistics for 172.16.10.2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 1ms, Maximum = 2ms, Average = 1ms
Control Center → DB-01a
C:\Users\Administrator>ping 172.16.10.3

Pinging 172.16.10.3 with 32 bytes of data:
Reply from 172.16.10.3: bytes=32 time=2ms TTL=61
Reply from 172.16.10.3: bytes=32 time=2ms TTL=61
Reply from 172.16.10.3: bytes=32 time=2ms TTL=61
Reply from 172.16.10.3: bytes=32 time=1ms TTL=61

Ping statistics for 172.16.10.3:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 1ms, Maximum = 2ms, Average = 1ms
Control Center → Web-01b
C:\Users\Administrator>ping 172.16.11.2

Pinging 172.16.11.2 with 32 bytes of data:
Reply from 172.16.11.2: bytes=32 time=3ms TTL=60
Reply from 172.16.11.2: bytes=32 time=2ms TTL=60
Reply from 172.16.11.2: bytes=32 time=2ms TTL=60
Reply from 172.16.11.2: bytes=32 time=2ms TTL=60

Ping statistics for 172.16.11.2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 2ms, Maximum = 3ms, Average = 2ms
Control Center → DB-01b
C:\Users\Administrator>ping 172.16.11.3

Pinging 172.16.11.3 with 32 bytes of data:
Reply from 172.16.11.3: bytes=32 time=2ms TTL=60
Reply from 172.16.11.3: bytes=32 time=2ms TTL=60
Reply from 172.16.11.3: bytes=32 time=2ms TTL=60
Reply from 172.16.11.3: bytes=32 time=2ms TTL=60

Ping statistics for 172.16.11.3:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 2ms, Maximum = 2ms, Average = 2ms

Trace-routes

Site-A

Web-01a → Control Center (via 192.168.254.1)
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:8C:12:73
          inet addr:172.16.10.2  Bcast:172.16.10.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe8c:1273/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10632 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5985 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:805144 (786.2 KiB)  TX bytes:586494 (572.7 KiB)

tc@TinyVM:~$ traceroute 192.168.110.10
traceroute to 192.168.110.10 (192.168.110.10), 30 hops max, 38 byte packets
 1  dlr-01a-app-lif.corp.local (172.16.10.1)  0.223 ms  0.393 ms  0.058 ms
 2  100.64.200.0 (100.64.200.0)  0.111 ms  0.128 ms  0.126 ms
 3  192.168.254.1 (192.168.254.1)  0.775 ms  1.453 ms  0.880 ms
 4  controlcenter.corp.local (192.168.110.10)  1.456 ms  *  2.096 ms

Untitled%203.png

Web-01a → Internet (via 192.168.252.1)
tc@TinyVM:~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 38 byte packets
 1  dlr-01a-app-lif.corp.local (172.16.10.1)  0.351 ms  0.147 ms  0.132 ms
 2  100.64.200.0 (100.64.200.0)  0.142 ms  0.104 ms  0.071 ms
 3  192.168.252.1 (192.168.252.1)  0.999 ms  192.168.251.1 (192.168.251.1)  1.102 ms  192.168.252.1 (192.168.252.1)  1.180 ms
 4  192.168.110.1 (192.168.110.1)  1.080 ms  1.124 ms  0.572 ms
 5  192.168.0.1 (192.168.0.1)  1.093 ms  1.610 ms  1.100 ms
 6  172.25.4.1 (172.25.4.1)  1.194 ms  1.046 ms  0.932 ms
 7  172.16.3.125 (172.16.3.125)  1.356 ms  1.738 ms  1.481 ms
 8  172.31.240.30 (172.31.240.30)  2.184 ms  172.31.240.28 (172.31.240.28)  1.775 ms  1.743 ms
 9  172.31.240.10 (172.31.240.10)  1.615 ms  1.737 ms  172.31.240.0 (172.31.240.0)  1.488 ms
10  10.147.22.222 (10.147.22.222)  2.229 ms  2.166 ms  1.612 ms
11  10.147.22.225 (10.147.22.225)  1.703 ms  2.306 ms  1.992 ms
12  10.255.80.148 (10.255.80.148)  34.864 ms  35.108 ms  34.815 ms
13  svl-edge-19.inet.qwest.net (67.131.255.209)  35.343 ms  35.583 ms  35.098 ms
14  sjp-brdr-04.inet.qwest.net (67.14.34.38)  46.800 ms  46.958 ms  46.689 ms
15  72.14.218.2 (72.14.218.2)  38.975 ms  36.301 ms  36.666 ms
16  *  *  *
17  dns.google (8.8.8.8)  48.170 ms  48.185 ms  47.739 ms
tc@TinyVM:~$

Untitled%204.png

DB-01a → Control Center (via 192.168.253.129)
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:8C:2D:19
          inet addr:172.16.11.2  Bcast:172.16.11.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe8c:2d19/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10029 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5393 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:758106 (740.3 KiB)  TX bytes:550480 (537.5 KiB)

tc@TinyVM:~$ traceroute 192.168.110.10
traceroute to 192.168.110.10 (192.168.110.10), 30 hops max, 38 byte packets
 1  172.16.11.1 (172.16.11.1)  2.384 ms  0.201 ms  2.113 ms
 2  100.64.0.0 (100.64.0.0)  0.260 ms  0.240 ms  0.321 ms
 3  192.168.253.129 (192.168.253.129)  1.293 ms  1.381 ms  2.244 ms
 4  router.corp.local (192.168.100.1)  1.426 ms  1.158 ms  3.850 ms
 5  controlcenter.corp.local (192.168.110.10)  2.399 ms  *  3.140 ms

Untitled%205.png

DB-01a → Internet (via 192.168.252.129)
tc@TinyVM:~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 38 byte packets
 1  172.16.11.1 (172.16.11.1)  0.108 ms  0.126 ms  0.142 ms
 2  100.64.0.0 (100.64.0.0)  0.074 ms  0.055 ms  0.069 ms
 3  192.168.252.129 (192.168.252.129)  1.207 ms  0.866 ms  192.168.251.129 (192.168.251.129)  0.690 ms
 4  router.corp.local (192.168.100.1)  1.242 ms  0.825 ms  1.042 ms
 5  192.168.0.1 (192.168.0.1)  1.538 ms  2.380 ms  1.412 ms
 6  172.25.4.1 (172.25.4.1)  1.378 ms  1.450 ms  1.178 ms
 7  172.16.3.125 (172.16.3.125)  1.622 ms  2.007 ms  1.915 ms
 8  172.31.240.28 (172.31.240.28)  1.679 ms  172.31.240.30 (172.31.240.30)  3.238 ms  2.294 ms
 9  172.31.240.0 (172.31.240.0)  2.193 ms  2.739 ms  172.31.240.10 (172.31.240.10)  2.196 ms
10  10.147.22.222 (10.147.22.222)  2.423 ms  2.551 ms  2.228 ms
11  10.147.22.225 (10.147.22.225)  2.120 ms  2.545 ms  2.173 ms
12  10.255.80.148 (10.255.80.148)  34.997 ms  35.333 ms  35.296 ms
13  svl-edge-26.inet.qwest.net (67.131.255.209)  36.376 ms  35.561 ms  35.926 ms
14  snj-edge-07.inet.qwest.net (67.14.34.38)  47.495 ms  47.101 ms  47.343 ms
15  72.14.218.2 (72.14.218.2)  37.388 ms  72.14.217.242 (72.14.217.242)  37.052 ms  37.130 ms
16  *  *  *
17  142.251.70.104 (142.251.70.104)  37.966 ms  dns.google (8.8.8.8)  48.185 ms  48.064 ms
tc@TinyVM:~$

Untitled%206.png

Site-B

Web-01b → Control Center (via 192.168.254.1)
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:9C:DB:6D
          inet addr:172.16.10.3  Bcast:172.16.10.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe9c:db6d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9956 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5293 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:750533 (732.9 KiB)  TX bytes:511754 (499.7 KiB)

tc@TinyVM:~$ traceroute 192.168.110.10
traceroute to 192.168.110.10 (192.168.110.10), 30 hops max, 38 byte packets
 1  dlr-01a-app-lif.corp.local (172.16.10.1)  1.385 ms  0.186 ms  0.005 ms
 2  100.64.200.0 (100.64.200.0)  0.099 ms  0.098 ms  0.122 ms
 3  192.168.254.1 (192.168.254.1)  1.043 ms  1.251 ms  0.998 ms
 4  controlcenter.corp.local (192.168.110.10)  1.136 ms  *  1.975 ms

Untitled%207.png

Web-01b → Internet (via 192.168.252.1)
tc@TinyVM:~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 38 byte packets
 1  dlr-01a-app-lif.corp.local (172.16.10.1)  0.234 ms  0.135 ms  0.041 ms
 2  100.64.200.0 (100.64.200.0)  0.079 ms  0.119 ms  0.050 ms
 3  192.168.252.1 (192.168.252.1)  1.128 ms  192.168.251.1 (192.168.251.1)  1.496 ms  192.168.252.1 (192.168.252.1)  1.295 ms
 4  192.168.110.1 (192.168.110.1)  1.449 ms  1.349 ms  1.110 ms
 5  192.168.0.1 (192.168.0.1)  1.500 ms  1.498 ms  1.348 ms
 6  172.25.4.1 (172.25.4.1)  1.824 ms  1.392 ms  2.656 ms
 7  172.16.3.125 (172.16.3.125)  2.158 ms  2.531 ms  1.936 ms
 8  172.31.240.28 (172.31.240.28)  2.225 ms  172.31.240.30 (172.31.240.30)  3.155 ms  2.078 ms
 9  172.31.240.2 (172.31.240.2)  2.032 ms  2.776 ms  172.31.240.8 (172.31.240.8)  2.034 ms
10  10.147.22.222 (10.147.22.222)  2.516 ms  3.890 ms  2.352 ms
11  10.147.22.225 (10.147.22.225)  2.604 ms  3.796 ms  2.363 ms
12  10.255.80.148 (10.255.80.148)  34.966 ms  35.119 ms  35.131 ms
13  svl-edge-19.inet.qwest.net (67.131.255.209)  35.918 ms  36.699 ms  35.653 ms
14  sjp-brdr-04.inet.qwest.net (67.14.34.38)  47.473 ms  47.874 ms  47.735 ms
15  72.14.218.2 (72.14.218.2)  37.055 ms  72.14.217.242 (72.14.217.242)  37.191 ms  37.474 ms
16  *  *  *
17  dns.google (8.8.8.8)  48.042 ms  47.610 ms  47.354 ms
tc@TinyVM:~$

Untitled%208.png

DB-01b → Control Center (via 192.168.254.129)
tc@TinyVM:~$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:56:9C:73:16
          inet addr:172.16.11.3  Bcast:172.16.11.255  Mask:255.255.255.0
          inet6 addr: fe80::250:56ff:fe9c:7316/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10206 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5503 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:769640 (751.6 KiB)  TX bytes:597828 (583.8 KiB)

tc@TinyVM:~$ traceroute 192.168.110.10
traceroute to 192.168.110.10 (192.168.110.10), 30 hops max, 38 byte packets
 1  172.16.11.1 (172.16.11.1)  0.306 ms  0.135 ms  0.266 ms
 2  100.64.0.0 (100.64.0.0)  0.234 ms  0.425 ms  0.147 ms
 3  192.168.254.129 (192.168.254.129)  1.315 ms  1.439 ms  192.168.253.129 (192.168.253.129)  0.892 ms
 4  router.corp.local (192.168.100.1)  1.721 ms  1.526 ms  1.455 ms
 5  controlcenter.corp.local (192.168.110.10)  1.337 ms  *  2.460 ms

Untitled%209.png

DB-01b → Internet (via 192.168.252.129)
tc@TinyVM:~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 38 byte packets
 1  172.16.11.1 (172.16.11.1)  0.211 ms  0.098 ms  0.004 ms
 2  100.64.0.0 (100.64.0.0)  0.086 ms  0.090 ms  0.039 ms
 3  192.168.252.129 (192.168.252.129)  0.860 ms  192.168.251.129 (192.168.251.129)  0.925 ms  192.168.252.129 (192.168.252.129)  0.609 ms
 4  router.corp.local (192.168.100.1)  0.782 ms  0.982 ms  0.875 ms
 5  192.168.0.1 (192.168.0.1)  1.086 ms  1.209 ms  0.960 ms
 6  172.25.4.1 (172.25.4.1)  1.123 ms  1.045 ms  1.986 ms
 7  172.16.3.125 (172.16.3.125)  1.382 ms  1.436 ms  1.290 ms
 8  172.31.240.28 (172.31.240.28)  1.554 ms  172.31.240.30 (172.31.240.30)  1.313 ms  1.475 ms
 9  172.31.240.0 (172.31.240.0)  1.683 ms  2.260 ms  172.31.240.10 (172.31.240.10)  1.869 ms
10  10.147.22.222 (10.147.22.222)  2.335 ms  1.815 ms  1.782 ms
11  10.147.22.225 (10.147.22.225)  1.982 ms  2.040 ms  1.929 ms
12  10.255.80.148 (10.255.80.148)  34.686 ms  34.860 ms  34.484 ms
13  svl-edge-26.inet.qwest.net (67.131.255.209)  35.461 ms  35.530 ms  35.318 ms
14  snj-edge-07.inet.qwest.net (67.14.34.38)  47.190 ms  47.032 ms  47.111 ms
15  72.14.217.242 (72.14.217.242)  36.463 ms  72.14.218.2 (72.14.218.2)  36.717 ms  36.541 ms
16  *  *  *
17  dns.google (8.8.8.8)  48.129 ms  142.251.70.104 (142.251.70.104)  37.276 ms  36.998 ms
tc@TinyVM:~$

Untitled%2010.png

Control Center

Control Center → Web-01a (via wan-router-01a.corp.local)
C:\Users\Administrator>tracert 172.16.10.2

Tracing route to 172.16.10.2 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  192.168.110.1
  2    <1 ms    <1 ms    <1 ms  wan-router-01a.corp.local [192.168.110.4]
  3     2 ms     1 ms    <1 ms  100.64.200.1
  4     3 ms     1 ms     1 ms  172.16.10.2

Trace complete.

Untitled%2011.png

Control Center → DB-01a (via wan-router-01a.corp.local)
C:\Users\Administrator>tracert 172.16.10.3

Tracing route to 172.16.10.3 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  192.168.110.1
  2    <1 ms    <1 ms    <1 ms  wan-router-01a.corp.local [192.168.110.4]
  3    12 ms    14 ms    <1 ms  100.64.200.1
  4     2 ms     1 ms     1 ms  172.16.10.3

Trace complete.

Untitled%2012.png

Control Center → Web-01b (via wan-router-01b.corp.local)
C:\Users\Administrator>tracert 172.16.11.2

Tracing route to 172.16.11.2 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  192.168.110.1
  2    <1 ms    <1 ms     1 ms  wan-router-01b.corp.local [192.168.210.4]
  3     1 ms    <1 ms    <1 ms  192.168.253.130
  4     2 ms    <1 ms    <1 ms  100.64.0.1
  5     2 ms     1 ms     1 ms  172.16.11.2

Trace complete.

Untitled%2013.png

Control Center → DB-01b (via wan-router-01b.corp.local)
C:\Users\Administrator>tracert 172.16.11.3

Tracing route to 172.16.11.3 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  192.168.110.1
  2    <1 ms    <1 ms    <1 ms  wan-router-01b.corp.local [192.168.210.4]
  3     1 ms    <1 ms    <1 ms  192.168.253.130
  4     1 ms    <1 ms    <1 ms  100.64.0.1
  5     2 ms     1 ms    <1 ms  172.16.11.3

Trace complete.

Untitled%2014.png

Configuration Details
Site A & Site B
WAN-router
cumulus@cumulus:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 172.16.1.112
net add time ntp server 172.16.1.113
net add time ntp source swp4
net add snmp-server listening-address localhost
net add bgp autonomous-system 65999
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log file /var/log/frr/debug.log
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add routing route 0.0.0.0/0 192.168.0.1
net add routing route-map from-site-a-out permit 10
net add pim rp 192.168.110.1 224.0.0.0/4
net add bgp neighbor 192.168.110.2 remote-as 65150
net add bgp neighbor 192.168.110.3 remote-as 65250
net add bgp neighbor 192.168.110.4 remote-as 65100
net add bgp neighbor 192.168.110.5 remote-as 65200
net add bgp neighbor 192.168.210.2 remote-as 65350
net add bgp neighbor 192.168.210.3 remote-as 65450
net add bgp neighbor 192.168.210.4 remote-as 65300
net add bgp neighbor 192.168.210.5 remote-as 65400
net add bgp ipv4 unicast maximum-paths 8
net add bgp ipv4 unicast neighbor 192.168.110.2 default-originate
net add bgp ipv4 unicast neighbor 192.168.110.3 default-originate
net add bgp ipv4 unicast neighbor 192.168.210.2 default-originate
net add bgp ipv4 unicast neighbor 192.168.210.3 default-originate
net add bgp l2vpn evpn  neighbor 192.168.110.2 activate
net add bgp l2vpn evpn  neighbor 192.168.110.3 activate
net add bgp l2vpn evpn  neighbor 192.168.110.4 activate
net add bgp l2vpn evpn  neighbor 192.168.110.5 activate
net add bgp l2vpn evpn  neighbor 192.168.210.2 activate
net add bgp l2vpn evpn  neighbor 192.168.210.3 activate
net add bgp l2vpn evpn  neighbor 192.168.210.4 activate
net add bgp l2vpn evpn  neighbor 192.168.210.5 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add nat static dnat tcp 192.168.0.2 3389 in-interface swp2 translate 192.168.110.10 3389
net add nat dynamic snat tcp source-ip 0.0.0.0/0 out-interface swp2 translate 192.168.0.2 6000-65000
net add nat dynamic snat icmp source-ip 0.0.0.0/0 out-interface swp2 translate 192.168.0.2 6000-65000
net add nat dynamic snat udp source-ip 0.0.0.0/0 out-interface swp2 translate 192.168.0.2 6000-65000
net add time zone Etc/UTC
net add dhcp server swp1
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 130,135,230,235
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.110.1/24
net add interface swp1,4 mtu 1700
net add interface swp2 ip address 192.168.0.2/24
net add interface swp2 mtu 1500
net add interface swp2 post-up 'ip route add default via 192.168.0.1'
net add interface swp3 bridge vids 130,135,230,235
net add interface swp3 mtu 9000
net add interface swp4 ip address 10.10.20.1/24
net add interface swp4 ip address 10.10.30.1/24
net add interface swp4 ip address 10.20.20.1/24
net add interface swp4 ip address 10.20.30.1/24
net add interface swp4 ip address 192.168.100.1/24
net add interface swp4 ip address 192.168.120.1/24
net add interface swp4 ip address 192.168.140.1/24
net add interface swp4 ip address 192.168.150.1/24
net add interface swp4 ip address 192.168.200.1/24
net add interface swp4 ip address 192.168.210.1/24
net add interface swp4 ip address 192.168.220.1/24
net add interface swp4 ip address 192.168.240.1/24
net add interface swp4 ip address 192.168.250.1/24
net add loopback lo ip address 192.168.252.254/32
net add vlan 130 ip address 192.168.130.1/24
net add vlan 130 vlan-id 130
net add vlan 130 vlan-raw-device bridge
net add vlan 135 ip address 192.168.135.1/24
net add vlan 135 vlan-id 135
net add vlan 135 vlan-raw-device bridge
net add vlan 230 ip address 192.168.230.1/24
net add vlan 230 vlan-id 230
net add vlan 230 vlan-raw-device bridge
net add vlan 235 ip address 192.168.235.1/24
net add vlan 235 vlan-id 235
net add vlan 235 vlan-raw-device bridge
net commit

# There are some configuration commands that are not yet supported by nclu.
# The following will append those commands to the appropriate files.
# ========================================================================
sudo sh -c "printf 'interface swp1\nigmp join 239.0.0.6\n' >> /etc/frr/frr.conf"
sudo sh -c "printf 'router ospf\n  area 51 authentication message-digest\n' >> /etc/frr/frr.conf"
cumulus@cumulus:~$
Compute Managers

Untitled%2015.png

Segment → ov-web

Untitled%2016.png

Segment → ov-DB

Untitled%2017.png

edge-vlan-tz

Untitled%2018.png

Site A (Paris)

VDS Port Groups

edge-uplink-left

Untitled%2019.png

Untitled%2020.png

edge-uplink-right

Untitled%2021.png

Untitled%2022.png

Edge Configuration

loadbalance-uplink-edge-profile-a

Untitled%2023.png

tep-ip-pool-A

Untitled%2024.png

edge-01a

Untitled%2025.png

edge-02a

Untitled%2026.png

edge-03a

Untitled%2027.png

edge-04a

Untitled%2028.png

edge-cluster-01a

Untitled%2029.png

edge-cluster-02a

Untitled%2030.png

Host Configuration

loadbalance-uplink-hostswitch-profile-a

Untitled%2031.png

vds-01-tpn-a

Untitled%2032.png

Host Transport Nodes a

Untitled%2033.png

ISP Router Configuration

isp-router-01a
cumulus@isp-router-01a:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65150
net add vlan 150 ospf dead-interval 4
net add vlan 150 ospf hello-interval 1
net add vlan 150 ospf message-digest-key 1 md5 VMware1!
net add vlan 150 ospf network point-to-point
net add vlan 150 pim
net add vlan 150 ospf6 mtu-ignore
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add bgp neighbor 192.168.110.1 remote-as 65999
net add bgp neighbor 192.168.251.2 remote-as 65001
net add bgp neighbor 192.168.251.3 remote-as 65001
net add bgp ipv4 unicast network 192.168.240.15/32
net add bgp ipv4 unicast neighbor 192.168.251.2 default-originate
net add bgp ipv4 unicast neighbor 192.168.251.3 default-originate
net add bgp ipv4 unicast maximum-paths 8
net add bgp l2vpn evpn  neighbor 192.168.110.1 activate
net add bgp l2vpn evpn  neighbor 192.168.251.2 activate
net add bgp l2vpn evpn  neighbor 192.168.251.3 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 150
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.110.2/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.3/24
net add interface swp3 bridge vids 150
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.15/32
net add vlan 150 ip address 192.168.251.1/25
net add vlan 150 mtu 1500
net add vlan 150 vlan-id 150
net add vlan 150 vlan-raw-device bridge
net add hostname isp-router-01a
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit
isp-router-02a
cumulus@isp-router-02a:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65250
net add vlan 250 ospf dead-interval 4
net add vlan 250 ospf hello-interval 1
net add vlan 250 ospf message-digest-key 1 md5 VMware1!
net add vlan 250 ospf network point-to-point
net add vlan 250 pim
net add vlan 250 ospf6 mtu-ignore
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add pim rp 192.168.110.3 224.0.0.0/4
net add bgp neighbor 192.168.110.1 remote-as 65999
net add bgp neighbor 192.168.252.2 remote-as 65001
net add bgp neighbor 192.168.252.3 remote-as 65001
net add bgp ipv4 unicast network 192.168.240.25/32
net add bgp ipv4 unicast neighbor 192.168.252.2 default-originate
net add bgp ipv4 unicast neighbor 192.168.252.3 default-originate
net add bgp ipv4 unicast maximum-paths 8
net add bgp l2vpn evpn  neighbor 192.168.110.1 activate
net add bgp l2vpn evpn  neighbor 192.168.252.2 activate
net add bgp l2vpn evpn  neighbor 192.168.252.3 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 250
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.110.3/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.4/24
net add interface swp3 bridge vids 250
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.25/32
net add vlan 250 ip address 192.168.252.1/25
net add vlan 250 mtu 1500
net add vlan 250 vlan-id 250
net add vlan 250 vlan-raw-device bridge
net add hostname isp-router-02a
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit
cumulus@isp-router-02a:~$

WAN Router Configuration

wan-router-01a
cumulus@wan-router-01a:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65100
net add vlan 100 ospf dead-interval 4
net add vlan 100 ospf hello-interval 1
net add vlan 100 ospf message-digest-key 1 md5 VMware1!
net add vlan 100 ospf network point-to-point
net add vlan 100 pim
net add vlan 100 ospf6 mtu-ignore
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add routing route 0.0.0.0/0 192.168.0.1
net add pim rp 192.168.110.4 224.0.0.0/4
net add bgp neighbor 192.168.110.1 remote-as 65999
net add bgp neighbor 192.168.253.2 remote-as 65001
net add bgp neighbor 192.168.253.3 remote-as 65001
net add bgp ipv4 unicast network 192.168.240.10/32
net add bgp ipv4 unicast redistribute ospf
net add bgp ipv4 unicast maximum-paths 8
net add bgp l2vpn evpn  neighbor 192.168.110.1 activate
net add bgp l2vpn evpn  neighbor 192.168.253.2 activate
net add bgp l2vpn evpn  neighbor 192.168.253.3 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 100
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.110.4/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.5/24
net add interface swp2 post-up 'ip route add default via 192.168.0.1'
net add interface swp3 bridge vids 100
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.10/32
net add vlan 100 ip address 192.168.253.1/25
net add vlan 100 mtu 1500
net add vlan 100 vlan-id 100
net add vlan 100 vlan-raw-device bridge
net add hostname wan-router-01a
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit
wan-router-02a
cumulus@wan-router-02a:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65200
net add vlan 200 ospf dead-interval 4
net add vlan 200 ospf hello-interval 1
net add vlan 200 ospf message-digest-key 1 md5 VMware1!
net add vlan 200 ospf network point-to-point
net add vlan 200 pim
net add vlan 200 ospf6 mtu-ignore
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add routing route 0.0.0.0/0 192.168.0.1
net add pim rp 192.168.110.5 224.0.0.0/4
net add bgp neighbor 192.168.110.1 remote-as 65999
net add bgp neighbor 192.168.254.2 remote-as 65001
net add bgp neighbor 192.168.254.3 remote-as 65001
net add bgp ipv4 unicast network 192.168.240.20/32
net add bgp ipv4 unicast redistribute ospf
net add bgp ipv4 unicast maximum-paths 8
net add bgp l2vpn evpn  neighbor 192.168.110.1 activate
net add bgp l2vpn evpn  neighbor 192.168.254.2 activate
net add bgp l2vpn evpn  neighbor 192.168.254.3 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 200
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.110.5/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.6/24
net add interface swp2 post-up 'ip route add default via 192.168.0.1'
net add interface swp3 bridge vids 200
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.20/32
net add vlan 200 ip address 192.168.254.1/25
net add vlan 200 mtu 1500
net add vlan 200 vlan-id 200
net add vlan 200 vlan-raw-device bridge
net add hostname wan-router-02a
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit

Tier-0 Gateway Configuration

Tier-0 Gateway (t0-gateway-a)

Untitled%2034.png

Segments → vlan100 (BGP Uplink 1 for WAN)

Untitled%2035.png

Segments → vlan200 (BGP Uplink 2 for WAN)

[1]

Segments → vlan150 (BGP Uplink 1 for ISP)

Untitled%2036.png

Segments → vlan250 (BGP Uplink 2 for ISP)

Untitled%2037.png

Interfaces

Untitled%2038.png

Untitled%2039.png

BGP

Untitled%2040.png

BGP Peers

Untitled%2041.png

Routing Redistribution

Untitled%2042.png

Tier-1 Gateway Configuration

Tier-1 Gateway (t1-gateway-a)

Untitled%2043.png

Routing Advertisement

Untitled%2044.png


Site B (London)

VDS Port Groups

edge-uplink-left

Untitled%2045.png

Untitled%2046.png

edge-uplink-right

Untitled%2047.png

Untitled%2048.png

Edge Configuration

loadbalance-uplink-edge-profile-b

Untitled%2049.png

tep-ip-pool-B

Untitled%2050.png

edge-01b

Untitled%2051.png

Untitled%2052.png

edge-02b

Untitled%2053.png

Untitled%2054.png

edge-03b

Untitled%2055.png

edge-04b

Untitled%2056.png

edge-cluster-01b

Untitled%2057.png

edge-cluster-02b

Untitled%2058.png

Host Configuration

loadbalance-uplink-hostswitch-profile-b

Untitled%2059.png

vds-01-tpn-b

Untitled%2060.png

Host Transport Nodes b

Untitled%2061.png

ISP Router Configuration

isp-router-01b
cumulus@isp-router-01b:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65350
net add vlan 350 ospf dead-interval 4
net add vlan 350 ospf hello-interval 1
net add vlan 350 ospf message-digest-key 1 md5 VMware1!
net add vlan 350 ospf network point-to-point
net add vlan 350 pim
net add vlan 350 ospf6 mtu-ignore
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add routing route 192.168.110.0/24 192.168.210.1
net add pim rp 192.168.210.2 224.0.0.0/4
net add bgp neighbor 192.168.210.1 remote-as 65999
net add bgp neighbor 192.168.251.130 remote-as 65002
net add bgp neighbor 192.168.251.131 remote-as 65002
net add bgp ipv4 unicast network 192.168.240.35/32
net add bgp ipv4 unicast neighbor 192.168.251.130 default-originate
net add bgp ipv4 unicast neighbor 192.168.251.131 default-originate
net add bgp ipv4 unicast maximum-paths 8
net add bgp l2vpn evpn  neighbor 192.168.210.1 activate
net add bgp l2vpn evpn  neighbor 192.168.251.130 activate
net add bgp l2vpn evpn  neighbor 192.168.251.131 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 350
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.210.2/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.7/24
net add interface swp3 bridge vids 350
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.35/32
net add vlan 350 ip address 192.168.251.129/25
net add vlan 350 mtu 1500
net add vlan 350 vlan-id 350
net add vlan 350 vlan-raw-device bridge
net add hostname isp-router-01b
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit
isp-router-02b
cumulus@isp-router-02b:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65450
net add vlan 450 ospf dead-interval 4
net add vlan 450 ospf hello-interval 1
net add vlan 450 ospf message-digest-key 1 md5 VMware1!
net add vlan 450 ospf network point-to-point
net add vlan 450 pim
net add vlan 450 ospf6 mtu-ignore
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add routing route 192.168.110.0/24 192.168.210.1
net add pim rp 192.168.210.3 224.0.0.0/4
net add bgp neighbor 192.168.210.1 remote-as 65999
net add bgp neighbor 192.168.252.130 remote-as 65002
net add bgp neighbor 192.168.252.131 remote-as 65002
net add bgp ipv4 unicast network 192.168.240.45/32
net add bgp ipv4 unicast neighbor 192.168.252.130 default-originate
net add bgp ipv4 unicast neighbor 192.168.252.131 default-originate
net add bgp ipv4 unicast maximum-paths 8
net add bgp l2vpn evpn  neighbor 192.168.210.1 activate
net add bgp l2vpn evpn  neighbor 192.168.252.130 activate
net add bgp l2vpn evpn  neighbor 192.168.252.131 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 450
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.210.3/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.8/24
net add interface swp3 bridge vids 450
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.45/32
net add vlan 450 ip address 192.168.252.129/25
net add vlan 450 mtu 1500
net add vlan 450 vlan-id 450
net add vlan 450 vlan-raw-device bridge
net add hostname isp-router-02b
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit

WAN Router Configuration

wan-router-01b
cumulus@wan-router-01b:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65300
net add vlan 300 ospf dead-interval 4
net add vlan 300 ospf hello-interval 1
net add vlan 300 ospf message-digest-key 1 md5 VMware1!
net add vlan 300 ospf network point-to-point
net add vlan 300 pim
net add vlan 300 ospf6 mtu-ignore
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add routing route 0.0.0.0/0 192.168.0.1
net add routing route 192.168.110.0/24 192.168.210.1
net add pim rp 192.168.210.4 224.0.0.0/4
net add bgp neighbor 192.168.210.1 remote-as 65999
net add bgp neighbor 192.168.253.130 remote-as 65002
net add bgp neighbor 192.168.253.131 remote-as 65002
net add bgp ipv4 unicast network 192.168.240.30/32
net add bgp ipv4 unicast redistribute ospf
net add bgp ipv4 unicast maximum-paths 8
net add bgp ipv4 unicast redistribute static
net add bgp l2vpn evpn  neighbor 192.168.210.1 activate
net add bgp l2vpn evpn  neighbor 192.168.253.130 activate
net add bgp l2vpn evpn  neighbor 192.168.253.131 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 300
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.210.4/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.9/24
net add interface swp2 post-up 'ip route add default via 192.168.0.1'
net add interface swp3 bridge vids 300
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.30/32
net add vlan 300 ip address 192.168.253.129/25
net add vlan 300 mtu 1500
net add vlan 300 vlan-id 300
net add vlan 300 vlan-raw-device bridge
net add hostname wan-router-01b
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit
wan-router-02b
cumulus@wan-router-02b:~$ net show configuration commands
net del all
net add dns nameserver ipv4 192.168.110.10
net add dns nameserver ipv4 198.168.110.10
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65400
net add vlan 400 ospf dead-interval 4
net add vlan 400 ospf hello-interval 1
net add vlan 400 ospf message-digest-key 1 md5 VMware1!
net add vlan 400 ospf network point-to-point
net add vlan 400 pim
net add vlan 400 ospf6 mtu-ignore
net add interface swp1 igmp
net add interface swp1 pim
net add routing defaults datacenter
net add routing log syslog informational
net add routing service integrated-vtysh-config
net add routing route 0.0.0.0/0 192.168.0.1
net add routing route 192.168.110.0/24 192.168.210.1
net add pim rp 192.168.210.5 224.0.0.0/4
net add bgp neighbor 192.168.210.1 remote-as 65999
net add bgp neighbor 192.168.254.130 remote-as 65002
net add bgp neighbor 192.168.254.131 remote-as 65002
net add bgp ipv4 unicast network 192.168.240.40/32
net add bgp ipv4 unicast redistribute ospf
net add bgp ipv4 unicast maximum-paths 8
net add bgp ipv4 unicast redistribute static
net add bgp l2vpn evpn  neighbor 192.168.210.1 activate
net add bgp l2vpn evpn  neighbor 192.168.254.130 activate
net add bgp l2vpn evpn  neighbor 192.168.254.131 activate
net add bgp l2vpn evpn  advertise-all-vni
net add bgp l2vpn evpn  advertise ipv4 unicast
net add time zone Etc/UTC
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp3
net add bridge bridge vids 400
net add bridge bridge vlan-aware
net add interface swp1 ip address 192.168.210.5/24
net add interface swp1 mtu 1700
net add interface swp2 ip address 192.168.0.10/24
net add interface swp2 post-up 'ip route add default via 192.168.0.1'
net add interface swp3 bridge vids 400
net add interface swp3 mtu 9000
net add interface swp4
net add loopback lo ip address 192.168.240.40/32
net add vlan 400 ip address 192.168.254.129/25
net add vlan 400 mtu 1500
net add vlan 400 vlan-id 400
net add vlan 400 vlan-raw-device bridge
net add hostname wan-router-02b
net add dot1x radius accounting-port 1813
net add dot1x max-number-stations 4
net add dot1x eap-reauth-period 0
net add dot1x default-dacl-preauth-filename default_preauth_dacl.rules
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 0
net commit

Tier-0 Gateway Configuration

Tier-0 Gateway (t0-gateway-b)

Untitled%2062.png

Segments → vlan300 (BGP Uplink 1 for WAN)

Untitled%2063.png

Segments → vlan400 (BGP Uplink 2 for WAN)

Untitled%2064.png

Segments → vlan350 (BGP Uplink 1 for ISP)

Untitled%2065.png

Segments → vlan450 (BGP Uplink 2 for ISP)

Untitled%2066.png

Interfaces

Untitled%2067.png

Untitled%2068.png

BGP

Untitled%2069.png

BGP Peers

Untitled%2070.png

Routing Redistribution

Untitled%2071.png

Tier-1 Gateway Configuration

Tier-1 Gateway t1-gateway-b)

Untitled%2072.png

Routing Advertisement

Untitled%2073.png