Bug #14421
closed
OPT interfaces randomly reassign NICs on reboot when virtualised instances used
0%
Description
Hi,
I've had this issue with all (20+) of my instances virtualised under VMware ESXi (all flavours). I don't know whether this issue extends to physical hardware as I don't run any physical hardware that has more than 3 interfaces.
Basically, what happens is - with a pfSense virtual instance with more than LAN and WAN and more than 1 OPT interface, the OPT interface assignments always get randomly reassigned on reboot of the host hypervisor and subsequent startup of a pfSense VM.
I have never yet seen an instance where WAN or LAN gets swapped or reassigned to a different NIC, but it always happens with 2 or greater OPT interfaces (and the more OPT interfaces you have the more chance you have of finding the randomised NIC assignment is the wrong one).
This is fixed manually by going into interface assignments and reassigning the interfaces, but clearly this shouldn't be happening. I would suggest a fix being to maintain an internal list of MAC addresses associated to OPT interfaces and ensuring that the interfaces are assigned correctly on reboot (or at least give a warning in the GUI if the automatic reassignment is problematic).
I realise that you would prefer to have technical logfiles or errors, but I have checked in System/General and whilst the interface assignments appear to occur during startup there are no errors prior to the assignments and all that happens is the interface name and MAC are listed. If you can point me to a different location where the relevant logs are captured then I can furnish you with them.
Updated by Marcos M 12 months ago
- Status changed from New to Rejected
As is, this is likely to be an environment-specific issue and more direct evidence would be needed before it's considered a bug with the software. The issue may be further discussed on our forum: https://forum.netgate.com/
Updated by Jim Pingle 12 months ago
There is also a known issue with ESX and >4 NICs where the hypervisor maps the NICs weirdly:
- NIC 1
- NIC 5
- NIC 2
- NIC 6
- NIC 3
- NIC 7
- NIC 4
- NIC 8
When you add the fifth one, suddenly your assignments do not line up with the probe order of the interfaces from the hypervisor. You have to check each one by MAC address to tell what should go where.
There is nothing pfSense can do about this, it's (mis)managed by ESX. If you tried to add interfaces live while the OS was booted and managed to assign them, there is no guarantee they'd probe in the same order the next boot.