pyVMC: Attach / Detach your prefix filters!

I am extremely pleased to announce the prefix filtering section of pyVMC is largely complete! Well, functionally it is complete, there are just a couple things I can do to clean up error handling…. With this release, you can now:

• create and upload new prefix filters to your SDDC
• list existing prefix filters
• attach a prefix filter to a T0 BGP neighbor
• detach prefix filters from a T0 BGP neighbor
• remove / delete prefix filters from your SDDC

Use Cases

I am no network architect… I learned about this through necessity. Prefix filtering is a capability that essentially allows you to dictate which routes are learned or advertised via BGP by your device – in this case the NSX-T edge of your VMware Cloud on AWS SDDC. There are a couple use cases that come to mind for this – firstly, to limit the number of networks advertised over a VPN in the case you have deployed VMware HCX with Mobility Optimized Networking. Patrick Kremer has blogged about this here.

Secondly, you might choose to use this technique to enable some use cases around disaster recovery – in my case, my customer had a requirement to duplicate several production networks in their SDDC for recovery using VMware Cloud DR. This customer was not interested in leveraging HCX for stretching any layer 2 networks, but still needed the ability to recover workloads using their existing IP addresses. Furthermore, the (duplicate) networks in VMware Cloud on AWS could in no way be allowed to ‘poison’ or contaminate the actual production networks and routing in their production datacenter. Prefix filtering offers a handy way to meet this need – allowing the duplication of networks in the cloud, while at the same time preventing those networks / routes to be advertised back on prem.

Overview

Prefix filters may be used to explicitly PERMIT or DENY specific networks from being learned (in_route_filters) or advertised (out_route_filters) on your device… but keep in mind, when you are building your prefix filter(s), that you will need to be explicit with your declarations… For example, let’s assume you have a created a subnet in your SDDC (for example 192.168.7.0/24), but you fail to include it in your filter as either PERMIT or DENY, you are denying it by default. Said / written another way, once you attach a filter, it alters the normal advertising / peering behavior of the NSX T0 edge, and will only advertise / accept those CIDRs specified in the filter. Therefore, it is extremely important that you are very thorough and double / triple check your filters before you attach them.

This is documented in the NSX-T Datacenter documentation here.

Workflow

You will use the following workflow to affect these changes:

1. create one or more prefix filters
2. select a T0 BGP neighbor (most likely a VPN tunnel interface)
3. attach no more than one filter to the in_route_filters or the out_route_filters

./pyVMC.py new-t0-prefix-list

In my case, I am adding a prefix filter that will PERMIT the advertisement of the 10.0.0.0/8 range, but DENY the advertisement of the 192.168.0.0/16 range. Below you can see the final prefix list:

Retrieve your BGP neighbors…. in my case I have three route-based VPNs to choose from. Below is a screen shot from my SDDC, along with the details for the advertised routes from the VPN using BGP address 169.254.111.29

Using pyVMC to retrieve the BGP neighbors for the SDDC shows the following:

./pyVMC.py show-t0-bgp-neighbors

Applying the filter

At this time, you can see that although I have created a filter, it has not yet been attached to anything, so the next step is to attach it. You need to specify the filter ID, whether you want the filter attached to in_route_filters or out_route_filters, and the ID of the neighbor.

in_route_filters prevent a route advertised from being learned by the SDDC… out_route_filters will prevent a network within the SDDC from being advertised.

./pyVMC.py attach-t0-prefix-list toms-new-list-id out 585a7a31-0cf8-11ec-8cc1-3df780dbaf3b

Note – a given neighbor may only have a single prefix filter attached to either the in_route_filters or the out_route_filters. Attempting to attach more than one filter to either yields an error similar to the following:

400
{'details': 'Field level validation errors: {route_filtering[0].out_route_filters has exceeded maximum size 1}', 'httpStatus': 'BAD_REQUEST', 'error_code': 255, 'module_name': 'common-services', 'error_message': 'Field level validation errors: {route_filtering[0].out_route_filters has exceeded maximum size 1}'}

Now if we retrieve our neighbors again, we see the prefix filter has been added.

And sure enough, if we review the route advertisements in our SDDC for the VPN in question, we see that 192.168.x.x networks are no longer being advertised.

Removing Filters

Removing a filter may also be accomplished easily, but again you will need to follow a specific workflow.

• Detach the prefix list(s) from the T0 BGP neighbor
• Remove (delete) the prefix list from the SDDC (if desired)

Note – attempting to remove (delete) a prefix list prior to detaching it from an interface will result in an error.

First, we simply have to detach the filter from the BGP neighbor:

./pyVMC.py detach-t0-prefix-lists 585a7a31-0cf8-11ec-8cc1-3df780dbaf3b 

In the following screen shot, you can see I have detached the filter, and the BGP neighbor table reflects this.

Checking the SDDC once again, we can see that the 192.168.x.x range(s) are again being advertised.

Finally, removing the prefix filter altogether can be accomplished by

./pyVMC.py remove-t0-prefix-list toms-new-list-id 

Running the above command, and then retrieving prefix lists shows there are none.

Thanks for wading through this! I hope you had as much fun as I did!