Bug #9968
closed
Configuration of assigned interfaces is deployed to unassigned ones
0%
Description
Background:
We are running pfSense virtualized on VMware vSphere platform with 10 vmxnet3 NICs (vmx0-vmx9), hardware version 13, using it as a router for multiple networks. We have decided to move few of the networks to different router, so I've removed configuration for corresponding interfaces on pfSense and unassigned them, basically removing all configuration from them, and I've also disconnected the networks from those interfaces.
Interfaces vmx0, vmx3, vmx4, vmx7 and vmx6 were left untouched.
Interfaces vmx1, vmx2, vmx5, vmx8 and vmx9 had their configuration removed, were unassigned, disabled and had no network connected to them.
Description of the bug:
After the aforementioned change, the routing on pfSense got completely mismatched.
Here are screenshots of interface config and routing table:
As you can see, for example, destination of 10.225.0.0/24 should have netif value of vmx3, but has value of vmx1. Destination of 10.225.1.0/24 should have netif value of vmx6, but has value of vmx2 and so on.
At this point, I've tried to assign the interfaces (thinking that maybe their unassigned status is the problem) and leave them disabled with no configuration whatsoever. This helped until I did another reboot.
Routing situation remained the same, but it's cause was revealed:
Here you can see that configuration of vmx4 was deployed to vmx4 and vmx5, but vmx5 has no configuration and is disabled. Same for vmx3 etc.
One more evidence of this is here:
Here you can see that vmx2 and vmx6 has duplicate configuration, but vmx2 is not enabled in GUI, has ip config set to "none" and has status "no carrier", because there isn't a network connected to it.
I've discovered that the only thing that helps and persists through reboots is enabling those interfaces, even without any configuration.
How to reproduce the bug
I can repeat the same issue on fresh installation with 10 NICs. However, if I only add 4 NICs (2 assigned and configured, 2 unassigned), everything is working as expected.
So basically:
1) Install a pfSense machine on VMware vSphere platform
2) Configure 10 NICs to it, only assign and configure 5. Leave the rest unassigned and disabled with no connected network.
3) Reboot
Files
| interfaces.png (20.4 KB) interfaces.png | |||
| routing.png (31.4 KB) routing.png | |||
| assigned.png (13 KB) assigned.png | |||
| ifconfig.png (198 KB) ifconfig.png |
Updated by Jim Pingle over 5 years ago
- Status changed from New to Not a Bug
This looks more like an issue with your config.xml or environment, and more discussion and detail is necessary. For example, there are known issues from vmware when adding more than 4 NICs to any VM where it doesn't allocate them in the expected order, but that's not an OS issue, it's a VMware issue. We need to be certain it's actually a new problem, and a problem in pfSense before tracking it here.
Please post on the Netgate Forum or the pfSense Subreddit .
See Reporting Issues with pfSense Software for more information.
Updated by Jim Pingle over 5 years ago
For good measure, I decided to try it out. Made a VM on ESX 6.7 with 10 NICs. Installed, configured 5 of them, left the other 5 disabled in pfSense and disconnected in ESX. Rebooted. All is fine. No unexpected configurations anywhere. Disabled NICs show as down and have no configuration. Rebooted several more times and still no change. Everything is as expected after each boot.
As I mentioned, the NICs get scrambled by ESX when probed by the OS, which might be part of your confusion. For example, on this install:
| pfSense | MAC End | VM Interface |
|---|---|---|
| vmx0 | cd | NA 4 |
| vmx1 | f5 | NA 8 |
| vmx2 | af | NA 1 |
| vmx3 | d7 | NA 5 |
| vmx4 | ff | NA 9 |
| vmx5 | b9 | NA 2 |
| vmx6 | e1 | NA 6 |
| vmx7 | 09 | NA 10 |
| vmx8 | c3 | NA 3 |
| vmx9 | eb | NA 7 |
Follow up on the forum if you have questions or further issues, but I see no actionable bug here.
Updated by Marek Částek over 5 years ago
Well, in our enviroment, this is still present and I can reproduce this behavior any time. I can also provide access to our enviroment, if you want to test this out. However, as we'll be leaving pfSense platform in the future, further working with this issue has low priority for us.