pfSense

After running that and restarting the pfSense box, IGMPProxy still won't start.

[2.5.0-DEVELOPMENT][admin@router.hendrix.network]/root: igmpproxy -h
Usage: igmpproxy [-h] [-n] [-d] [-v [-v]] <configfile>

   -h   Display this help screen
   -n   Do not run as a daemon
   -d   Run in debug mode. Output all messages on stderr. Implies -n.
   -v   Be verbose. Give twice to see even debug messages.

igmpproxy 0.2.1
[2.5.0-DEVELOPMENT][admin@router.hendrix.network]/root: uname -ar
FreeBSD router.hendrix.network 12.1-STABLE FreeBSD 12.1-STABLE 967e7a34ab2(devel-12) pfSense  amd64

Is there any other information i can provide to help?

Still same today:

[2.5.0-DEVELOPMENT][admin@router.hendrix.network]/root: pkg update -f ; pkg upgrade -yf igmpproxy
Updating pfSense-core repository catalogue...
Fetching meta.txz: 100%    912 B   0.9kB/s    00:01
Fetching packagesite.txz: 100%    2 KiB   1.8kB/s    00:01
Processing entries: 100%
pfSense-core repository update completed. 7 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.txz: 100%  136 KiB 139.8kB/s    00:01
Processing entries:   0%
Newer FreeBSD version for package zip:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1201516
- running kernel: 1200086
Ignore the mismatch and continue? [Y/n]:
Processing entries: 100%
pfSense repository update completed. 500 packages processed.
All repositories are up to date.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
    igmpproxy-0.2.1_1,1 [pfSense]

Number of packages to be reinstalled: 1

22 KiB to be downloaded.
[1/1] Fetching igmpproxy-0.2.1_1,1.txz: 100%   22 KiB  22.4kB/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Reinstalling igmpproxy-0.2.1_1,1...
[1/1] Extracting igmpproxy-0.2.1_1,1: 100%

I also did a full reinstall. Nothing changed.

Maarten Hendrix wrote:

Problem:
IGMPProxy (and PIMD) will not start after pfSense update on 05-02-2020.

Does IGMPProxy run if you uninstall PIMD? Both services didn't run to the same time. This worked for me.

I also tried both packages and noticed the same some weeks ago. It took some time until I noticed that although IGMPProxy can use IGMPv3 on the WAN side, it does not talk IGMPv3 on the LAN side, so it is not compatible with the MagentaTV box by German Telekom.

I tested with PIMD because it does a similar job.
I tested with it installed and without it installed. Both the same result.

Maarten Hendrix wrote:

I tested with PIMD because it does a similar job.
I tested with it installed and without it installed. Both the same result.

That is interesting. According to my logs, igmpproxy was running propery until running "igmpproxy -dvvvvvvvv /var/etc/igmpproxy.conf"
Maybe I also disabled and enabled the service from the GUI.
Now I have the same issue.

What I can say is that I upgraded my hardware between 05-02-2020 and three days ago. My pfsense devices were bowth up-to-date and igmpproxy was running.
I imported the configuration to the new device and igmpproxy was still running.

Although my igmpproxy service is running, I have the same error message.

"MRT_ADD_VIF; Errno(48): Address already in use" 

Additionally, I noted this: After running:

service -e

I get the following error message:

May 20 08:04:41    admin    89232    /usr/sbin/service: WARNING: $igmpproxy_enable is not set properly - see rc.conf(5).
...and more for other services.

Did you notice this line appears twice? Maybe the configuration file is not parsed properly:

Network for [igb2.44] : 172.19.44/24

Although the networks 192.168.1/24 and 192.168.0/24 appear twice, there is no error message.
But when it seems to try to add 193.158/15 a second time, it breaks.

adding VIF, Ix 0 Fl 0x0 IP 0x0101a8c0 lagg0, Threshold: 3, Ratelimit: 0
        Network for [lagg0] : 192.168.1/24
        Network for [lagg0] : 192.168.1/24
        Network for [lagg0] : 239.35/16
Found upstrem IF #0, will assing as upstream Vif 20
adding VIF, Ix 1 Fl 0x0 IP 0x0200a8c0 lagg1, Threshold: 3, Ratelimit: 0
        Network for [lagg1] : 192.168.0/24
        Network for [lagg1] : 192.168.0/24
        Network for [lagg1] : 224/4
        Network for [lagg1] : 87.128/10
        Network for [lagg1] : 193.158/15
        Network for [lagg1] : 80.157/16
        Network for [lagg1] : 62.155/16
        Network for [lagg1] : 217.0/13
        Network for [lagg1] : 193.158/15
MRT_ADD_VIF; Errno(48): Address already in use

It seems that the configuration file is not parsed properly.

Looks the same indeed:

Found upstrem IF #0, will assing as upstream Vif 22
adding VIF, Ix 0 Fl 0x0 IP 0x3b73560a igb0.4, Threshold: 1, Ratelimit: 0
        Network for [igb0.4] : 10.86.112/21
        Network for [igb0.4] : 213.75/16
        Network for [igb0.4] : 10/8
        Network for [igb0.4] : 217.166/16
adding VIF, Ix 1 Fl 0x0 IP 0x012c13ac igb2.44, Threshold: 1, Ratelimit: 0
        Network for [igb2.44] : 172.19.44/24
        Network for [igb2.44] : 172.19.44/24
MRT_ADD_VIF; Errno(48): Address already in use

Maarten Hendrix wrote:

Looks the same indeed:
[...]

I disabled the service and the debug mode, updated pfsense, including php, rebooted and enabled the service again. The service is running again now.

If an update does not help, try to disable the service and the debug mode, too, reboot, enable the service and save, and do not try to start it on command line. Then see if your service is running (Status/Services).

That indeed looks like it started again.
Will it still work after a reboot or do i need to disable it every time i update / reboot?

It might be that it only runs the first time after a reboot and anything that triggers the service to restart may make it fail. If that's the case, it almost sounds like it's not leaving its multicast groups when it exits, or something along those lines.

I concur: I just tried to restart the service via Status/Services and it fails.

Same result here. After a restart of the service it fails. After that if you reboot it still fails to start.
Now I stopped the service in settings and reboot again. After the reboot i enable the service again and enable logging and it starts without a problem.

Jim Pingle wrote:

It might be that it only runs the first time after a reboot and anything that triggers the service to restart may make it fail. If that's the case, it almost sounds like it's not leaving its multicast groups when it exits, or something along those lines.

Seems possibly related to this Kernel bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246629
No way of working around this until the freebsd kernel is patched I am assuming...

-Basically, I am asking for confirmation that we don't have any pfsense 2.5.0 snapshots on FreeBSD Kernel version: 12.1-p5? (Or 11.2?) (As those are mentioned to be issue free...)
Also saw people mention that reverting to an older version of psfsense (april) resolves the issue, can you confirm what kernel patch level it's running?

Hello,

I completely agree that this problem is almost certain related to the FreeBSD bug

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246629

So I did add some input to that bugreport, hoping that helps. The actual state of that freebsd-bugreport is ^Importance: --- Affects Only Me ^ which is definitively not correct.

In general, I wonder I the pfSense developers do generate FreeBSD bug reports and/or add extra input to existing reports. Which hopefully speed up things :)

Louis

That FreeBSD bug report does appear to be related, we'll try to draw some attention to that.

-Basically, I am asking for confirmation that we don't have any pfsense 2.5.0 snapshots on FreeBSD Kernel version: 12.1-p5? (Or 11.2?) (As those are mentioned to be issue free...)

No, current snapshots are using 12.1-STABLE and there is no other base for 2.5.0 at the moment, the older ones have cycled off the snapshot servers.

Also saw people mention that reverting to an older version of psfsense (april) resolves the issue, can you confirm what kernel patch level it's running?

That was before the snapshots moved to a 12.1-STABLE base.

Understood..

Thanks for the follow up and info. Anything you can do from your side to draw some attention to it would always be appreciated Jim.

Jim Pingle wrote:

That FreeBSD bug report does appear to be related, we'll try to draw some attention to that.

-Basically, I am asking for confirmation that we don't have any pfsense 2.5.0 snapshots on FreeBSD Kernel version: 12.1-p5? (Or 11.2?) (As those are mentioned to be issue free...)

No, current snapshots are using 12.1-STABLE and there is no other base for 2.5.0 at the moment, the older ones have cycled off the snapshot servers.

Also saw people mention that reverting to an older version of psfsense (april) resolves the issue, can you confirm what kernel patch level it's running?

That was before the snapshots moved to a 12.1-STABLE base.

Hey Jim,

We got a response on that FreeBSD Bug report, and a request to test.
I myself am not very familiar, understanding you may not have time, but if you did have a bit of time and a better understanding of their request, maybe you could help us review it?

Thanks,

I know, I was talking with that developer directly. We would need to test that change locally first before bringing it into snapshots, but we are working on it. A few other more pressing matters ahead of it in the queue, though.

Jim Pingle wrote:

I know, I was talking with that developer directly. We would need to test that change locally first before bringing it into snapshots, but we are working on it. A few other more pressing matters ahead of it in the queue, though.

Hey Jim!

Thats fantastic news. Love to hear that you two are already in touch. ill let you do you, as you clearly have a steady hold of it.
Let me know if you have any donation links for the project. The progress is amazing to see

Per bz, Fix works and is awaiting review upstream and will be committed to HEAD, then stable/12. Once it's in stable/12 we'll pick it back into snapshots.

Jim,

Very good news!

Is there an option to test it here on my system running latest snapshotbuild!
(yep I did fix the PPPOE-problem, it works ...... when MTU field is filled ...)

Louis

Louis van Breda wrote:

Jim,

Very good news!

Is there an option to test it here on my system running latest snapshotbuild!
(yep I did fix the PPPOE-problem, it works ...... when MTU field is filled ...)

Louis

I think we would need to wait for it to at least be committed into the HEAD before we can try to apply a patch.
Maybe Jim would be able to give a better idea. Either way I am super happy to see how fast the process is on this one. Massive appreciation to the team here!

I already answered that in comment 23

Jim Pingle wrote:

I already answered that in comment 23

I myself read his question as: "Is there an option to test it here on my system now before running latest snapshot-build?"
As in, he is aware the newest snapshots will have the fix, he is looking for an option "now" on the snapshot he has installed :)

You response only mentions the upcoming builds. Does Not answer if he can test now somehow on his current system.

Yep,

Exactly, now we have momentum to get things fixed. If we find a bug lateron the momentum and the timslot is gone.

Louis

It requires a new kernel, so no way to reliably test outside of snapshots. We'll pick up the change soon.

Hello,

I am not the only one noticeing that there is still a problem :) So the problem was updated in the FreeBSD bugzilla (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246629). And hopefully solved. Corrections should will be in the next (actual?) FreeBSD snapshot.

However, I think there is at least one issue at pfSense level, beeing pimd starting (twice!) at boot time even if PIMD is not activated in the GUI. I do not expect that that is on purpose!!?

Reading my own commented bootlog, I noticed:

1a) >> Only the three vlan's below will have vifs later. Something wrong already at this point !!!!
1b >> The VLAN's below should have VIFS but do not get them !!!
=> could be related to the indicated FreeBSD bug

2) >> something wrong here pind starting twice !!
=> see comment above, this is pfSense related

3) >> The fact that the interfaces below get ^{Invalid phyint address} is not ok !!
=> this is not OK as well, I do not know what and where the problem is could be FreeBSD, but if so .... is it the same issue or another issue !!??
=> could be pfSense as well, .... my Feeling is that it is more likely that it is in the OS (but not sure)

4) >> I did try many PIMD config options, Appart from "Bind to all, nothing worked, and even that one did not work properly
=> my feeling is that this is at least for a significant part is related to the indicated FreeBSD bug

So my suggestion is to assume for now that 1) and 4) are specified FreeBSD bug related, and in parallel investigate issue 2) and 3). In case issue 3) is not related to the FreeBSD bug and not to pfSense another FreeBSD bugreport should be created.

Louis

Hello,

Be aware there were multiple things fixed in FreeBSD and placed in the snapshots. Latest message I got from free BSD is

--- Comment #20 from commit-hook@freebsd.org --- A commit references this bug:

Author: bz
Date: Mon Jun 22 10:52:31 UTC 2020
New revision: 362494
URL: https://svnweb.freebsd.org/changeset/base/362494

Log:
MFC r362472:

Rather than zeroing MAXVIFS times size of pointer [r362289] (still better than
    sizeof pointer before [r354857]), we need to zero MAXVIFS times the size of
    the struct.  All good things come in threes; I hope this is it on this one.

PR:           246629, 206583
  Reported by:  kib

So be sure to pull the very latest FreeBSD version should have indicated patch level

Also, note that I am in direct contact with the FreeBSD developer. With input I provided, he noted that there is also another issue probably related to formatting of the pimd.conf file. Not sure there but one thing is for sure, the interface nameing for some reason or the other not consistent between FreeBSD, PIMD.conf and the logging. Together have to find out what the exact issue is.

The fact the PIMD is starting multiple times (I think even if PIMD is not enabled), id ofcourse a pfSense issue.

Louis

We are aware, and are in direct communication with the FreeBSD developer who made the commits. I mentioned above already that there was another fix and even linked to the copy you commented. We have already pulled the latest fix into the source tree earlier this morning, it will be in the next snapshot (building now).

Anything not directly related to the specific multicast issue caused by the FreeBSD bug does not belong on this issue, so let's keep the chatter to a minimum.