pfSense

Please, check #7176 too (probably related)

this is probably related to freebsd 11 not having support for 6rd. in the current pfsense stable version there is a custom patch.
Also look at this mailing list post: https://lists.freebsd.org/pipermail/freebsd-net/2013-June/035750.html

And here the patch comitted to the Pfsense freebsd 10 code: https://github.com/pfsense/FreeBSD-src/commit/62498dd06a33a82a579d1cc113c2fa04d995ac91

The patch should be updated to work with freebsd 11 to make this work again.

Ah, indeed. It would be great if we can get 6rd support committed to upstream FreeBSD.

Hi guys,

I'm just chiming in to ask for this as well. I'm using Charter's 6rd service and was about to open a ticket for this not working in 2.4.

-Will

I believe a recent change here may have broke 6to4 tunnel on WAN..

https://forum.pfsense.org/index.php?topic=134740.msg738791#msg738791

Luiz Otavio O Souza wrote:

The 6rd patch was committed to 2.4 and is available on the latest snapshots. Tests are much appreciate.

2.4.0-BETA (amd64)
built on Tue Aug 08 14:59:04 CDT 2017

6rd does not work for me.

IPv6 data from https://www.altibox.no/privat/bredband/ipv6/
IPv4 BR adresse: 213.167.115.92
IPv4 Prefix: 0
IPv6 Prefix: 2a01:79c::
IPv6 Prefix Length: 30

The 6to4 tunnel regression was fixed in the latest snapshot. 6rd situation has improved too, please retest.

Luiz Souza wrote:

The 6to4 tunnel regression was fixed in the latest snapshot. 6rd situation has improved too, please retest.

Still just pending on 6rd for me with the current snapshot (built on Fri Aug 11 01:11:58 CDT 2017).

Ok, I have committed a couple of new fixes after some tests.

Thanks for the report.

Luiz Souza wrote:

Ok, I have committed a couple of new fixes after some tests.

Thanks for the report.

It looks like it does receive an IP address over DHCP, but the 6rd Gateway shows now "Offline" rather than "Pending" (100% loss).

Ole-Henrik Jakobsen wrote:

Luiz Souza wrote:

Ok, I have committed a couple of new fixes after some tests.

Thanks for the report.

It looks like it does receive an IP address over DHCP, but the 6rd Gateway shows now "Offline" rather than "Pending" (100% loss).

Tried with the newest snapshot now and it gets "Online" for a few seconds before it goes "Offline" again.

Can someone provide more info please ?

Is the IPv6 connectivity up but the 6rd gateway is still showing as down or there isn't any IPv6 connectivity at all ?

Please, post the tunnel and interfaces status (ifconfig stf/wan/lan), also the dpinger command arguments (ps axwww | grep dpinger)

A packet capture would also be helpful.

Luiz Souza wrote:

Can someone provide more info please ?

Is the IPv6 connectivity up but the 6rd gateway is still showing as down or there isn't any IPv6 connectivity at all ?

Please, post the tunnel and interfaces status (ifconfig stf/wan/lan), also the dpinger command arguments (ps axwww | grep dpinger)

A packet capture would also be helpful.

ifconfig and dpinger:

https://pastebin.com/Rtde0Yeh

packet capture:
04:26:48.134582 IP CCC.CCC.CCC.CCC.56746 >XXX.XXX.XXX.XXX.59961: tcp 0
04:26:48.134636 IPXXX.XXX.XXX.XXX > DDD.DDD.DDD.DDD: IP6 XXXX:XXXX:XXXX:XXXX:: > YYYY:YYYY:YYYY:YYYY::: ICMP6, echo request, seq 31816, length 8
04:26:48.134988 IPXXX.XXX.XXX.XXX.59961 > CCC.CCC.CCC.CCC.56746: tcp 0

ping6:
PING6 XXXX:XXXX:XXXX:XXXX:: --> 2001:470:20::2
16 bytes from 2001:470:20::2, icmp_seq=0 hlim=60 time=27.631 ms
ping6: sendmsg: Can't assign requested address
ping6: wrote 2001:470:20::2 16 chars, ret=-1
16 bytes from 2001:470:20::2, icmp_seq=2 hlim=60 time=25.114 ms
ping6: sendmsg: Can't assign requested address
ping6: wrote 2001:470:20::2 16 chars, ret=-1
16 bytes from 2001:470:20::2, icmp_seq=4 hlim=60 time=25.071 ms
ping6: sendmsg: Can't assign requested address
ping6: wrote 2001:470:20::2 16 chars, ret=-1
16 bytes from 2001:470:20::2, icmp_seq=6 hlim=60 time=25.219 ms
ping6: sendmsg: Can't assign requested address
ping6: wrote 2001:470:20::2 16 chars, ret=-1
16 bytes from 2001:470:20::2, icmp_seq=8 hlim=60 time=28.372 ms
^C
--- 2001:470:20::2 ping6 statistics ---
9 packets transmitted, 5 packets received, 44.4% packet loss
round-trip min/avg/max/std-dev = 25.071/26.281/28.372/1.425 ms

I'm seeing this still on 2.4.0-RC.

ATT Fiber 6RD:
WAN_6RD
2602:300:c533:1510:: 43.4ms 5.3ms 48% Offline
WAN_DHCP
76.250.252.1 2.5ms 0.7ms 0.0% Online

but IPV6 works fine:
[2.4.0-RC][admin@home-fw.lerctr.org]/root: ping6 ipv6.google.com
PING6 2602:304:cfaf:f750:: --> 2607:f8b0:4000:80c::200e
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=0 hlim=56 time=10.738 ms
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=1 hlim=56 time=10.036 ms
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=2 hlim=56 time=10.468 ms
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=3 hlim=56 time=9.950 ms
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=4 hlim=56 time=10.512 ms
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=5 hlim=56 time=10.152 ms
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=6 hlim=56 time=10.484 ms
16 bytes from 2607:f8b0:4000:80c::200e, icmp_seq=7 hlim=56 time=10.034 ms
^C
--- ipv6.l.google.com ping6 statistics ---
8 packets transmitted, 8 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 9.950/10.297/10.738/0.270 ms
[2.4.0-RC][admin@home-fw.lerctr.org]/root:

What else can I provide?

[2.4.0-RC][admin@home-fw.lerctr.org]/root: ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:42:43:ac:02:9c
hwaddr 00:42:43:ac:02:9c
inet6 fe80::242:43ff:feac:29c%em0 prefixlen 64 scopeid 0x1
inet 76.250.255.117 netmask 0xfffffc00 broadcast 76.250.255.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:42:43:ac:02:9d
hwaddr 00:42:43:ac:02:9d
inet 192.168.200.11 netmask 0xfffffc00 broadcast 192.168.203.255
inet6 fe80::1:1%em1 prefixlen 64 scopeid 0x2
inet6 2602:304:cfaf:f750::1 prefixlen 64
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em2: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:42:43:ac:02:9e
hwaddr 00:42:43:ac:02:9e
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: no carrier
em3: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:42:43:ac:02:9f
hwaddr 00:42:43:ac:02:9f
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: no carrier
enc0: flags=0<> metric 0 mtu 1536
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: enc
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
pfsync0: flags=0<> metric 0 mtu 1500
groups: pfsync
syncpeer: 224.0.0.240 maxupd: 128 defer: on
syncok: 1
pflog0: flags=100<PROMISC> metric 0 mtu 33160
groups: pflog
wan_stf: flags=4001<UP,LINK2> metric 0 mtu 1280
inet6 2602:304:cfaf:f750:: prefixlen 28
nd6 options=101<PERFORMNUD,IGNORELOOP>
v4net 76.250.255.117/32 -> tv4br 12.83.49.81
groups: stf
[2.4.0-RC][admin@home-fw.lerctr.org]/root:

[2.4.0-RC][admin@home-fw.lerctr.org]/root: netstat -rn
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 76.250.252.1 UGS em0
12.83.49.81 76.250.252.1 UGHS em0
76.250.252.0/22 link#1 U em0
76.250.255.117 link#1 UHS lo0
127.0.0.1 link#6 UH lo0
192.168.200.0/22 link#2 U em1
192.168.200.11 link#2 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
default 2602:300:c533:1510:: UGS wan_stf
::1 link#6 UH lo0
2602:300::/28 link#9 U wan_stf
2602:304:cfaf:f750:: link#9 UHS lo0
2602:304:cfaf:f750::/64 link#2 U em1
2602:304:cfaf:f750::1 link#2 UHS lo0
2607:f8b0:4007:804::200e 2602:300:c533:1510:: UGHS wan_stf
fe80::%em0/64 link#1 U em0
fe80::242:43ff:feac:29c%em0 link#1 UHS lo0
fe80::%em1/64 link#2 U em1
fe80::1:1%em1 link#2 UHS lo0
fe80::%lo0/64 link#6 U lo0
fe80::1%lo0 link#6 UHS lo0
[2.4.0-RC][admin@home-fw.lerctr.org]/root:

Luiz duplicated the issue earlier today.

Does that mean we can expect a fix before -REL?

Yes, this bug has been fixed in the latest snapshot.

Thanks for all the debug and testing.

Luiz Souza wrote:

Yes, this bug has been fixed in the latest snapshot.

Thanks for all the debug and testing.

I just got: 2.4.0-RC (amd64) built on Wed Aug 23 05:47:05 CDT 2017

With this I am able to ping with 0% packet loss to google and facebook, so IPv6 via 6rd seems to work. But the 6rd gateway still shows offline (it does show "online" for a second before it goes offline permanently). If I try to enable SLAAC at IPv6 LAN it just stays at "Pending"

Any specific logs you need? Or is it comming another snapshot after the one I have with the fixes?

on my ATT 6RD Dpinger is now happy with
Version 2.4.0-RC (amd64)
built on Wed Aug 23 15:31:27 CDT 2017
FreeBSD 11.0-RELEASE-p12

The system is on the latest version.
Version information updated at 2017-08-23 19:18

Larry Rosenman wrote:

on my ATT 6RD Dpinger is now happy with
Version 2.4.0-RC (amd64)
built on Wed Aug 23 15:31:27 CDT 2017
FreeBSD 11.0-RELEASE-p12

The system is on the latest version.
Version information updated at 2017-08-23 19:18

Everything seems fine here as well at shell/cli level. Pinging works fine, dpinger, well not exactly sure how it should look like, but asume its fine as things are working.

About ifconfig I can't find my IPv6 address under the regular WAN adapter (here: em3), but it shows up as wan_stf, I assume that is correct?

Anyway, it still shows "offline" at the interface gateway section, and it looks like the router-pc crashes/disconnects network after about 24 hours with IPv6 6rd enabled (haven't tested this or the previous snapshots yet, but before those).

2.4.0-RC (amd64) and custom hardware
built on Fri Aug 25 18:40:44 CDT 2017
FreeBSD 11.0-RELEASE-p12

net.link.bridge.pfil_bridge = 1
LAN Interface (bridge) ipv6 config: Track Interface (WAN ipv6 which is 6rd tunnel from ISP)

Altibox (Norway) ISP settings:

IPv4 BR address: 213.167.115.92
IPv4 Prefix: 0
IPv6 Prefix: 2a01:79c::
IPv6 Prefix Length: 30
IPv6 DNS: 2a01:798:0:8012::4

After a reboot I experience the following issues:

ping6 ipv6.google.com fails from both the firewall's WAN and its bridge clients (SLAAC configured, different OS' and devices). I have tried to configure the clients via DHCPV6, but that did not change anything.

When I do a WAN interface restart (disable, then enable) everything runs fine and I can ping6 from both the firewall and its LAN (bridge) side clients with a 10/10 score on any ipv6 tests.

Can I provide you with any logs to help solve this issue? Thanks in advance!

I'm seeing no default route for IPv6 after a reboot until I do another save on the WAN interface.

Anything new on this?

Any log files we can provide with?

This does still not work after a reboot.

Although it works until next reboot if you go to Interfaces -> WAN and then click "Save".

Morten Freberg wrote:

This does still not work after a reboot.

Although it works until next reboot if you go to Interfaces -> WAN and then click "Save".

Here it goes online for about 1 sec before the gateway gets offline.

Ole-Henrik Jakobsen wrote:

Morten Freberg wrote:

This does still not work after a reboot.

Although it works until next reboot if you go to Interfaces -> WAN and then click "Save".

Here it goes online for about 1 sec before the gateway gets offline.

I have Altibox too, so the problem might be different from hardware to hardware ?

I run pfSense on a old Qotom box.

Morten Freberg wrote:

Ole-Henrik Jakobsen wrote:

Morten Freberg wrote:

This does still not work after a reboot.

Although it works until next reboot if you go to Interfaces -> WAN and then click "Save".

Here it goes online for about 1 sec before the gateway gets offline.

I have Altibox too, so the problem might be different from hardware to hardware ?

I run pfSense on a old Qotom box.

Dunno, intel quad GbE card in my box, should be fine I guess.

The issue is the 6RD default IPv6 route goes away/isn't set up at boot.

If I just route -6 add default <IPv6 version of 6RD ipv4 BR address> IPv6 works again.

Larry Rosenman wrote:

The issue is the 6RD default IPv6 route goes away/isn't set up at boot.

If I just route -6 add default <IPv6 version of 6RD ipv4 BR address> IPv6 works again.

route -6 add default 2a01:79f:569d:cd70::
add net default: gateway 2a01:79f:569d:cd70:: fib 0: route already in table

Thats what I get, but it simply doesnt work.

The IPv6 works at the router itself and I can ping IPv6 addresses, but it wont work with SLAAC to give IPv6 to other devices as teh gateway is showing "Offline" (I assume that's why it's not working at least).

is radvd running on your box?

The symptom I described is what I'm seeing on
FreeBSD 11.0-RELEASE-p12 #23 8ac57d7d39c(RELENG_2_4_0): Sat Sep 2 16:48:25

I have SLAAC addresses being given out, but if the FW reboots (that's another thread...) I lose the IPv6 default.

Larry Rosenman wrote:

is radvd running on your box?

The symptom I described is what I'm seeing on
FreeBSD 11.0-RELEASE-p12 #23 8ac57d7d39c(RELENG_2_4_0): Sat Sep 2 16:48:25

I have SLAAC addresses being given out, but if the FW reboots (that's another thread...) I lose the IPv6 default.

No, radvd is indeed not running. Do I need to enable it in a way? I assumed it should start when IPv6 is configured.

YES. it's what makes SLAAC work.

Do look at the RADVD pages on the IPv6 DHCP pages.

Larry Rosenman wrote:

YES. it's what makes SLAAC work.

Do look at the RADVD pages on the IPv6 DHCP pages.

The "Services -> DHCPv6 Server & RA" does not work as it can't find my IPv6 address, and that's where I think I should enable radvd. Yet it does have one IPv6 address.

Error:
The DHCPv6 Server can only be enabled on interfaces configured with a static IPv6 address. This system has none.

You need the Router Advertisements tab. Not DHCPv6.

Alright, I followed this old guide and made it work:
http://www.dslreports.com/forum/r30489490-IPv6-6rd-pfSense-Android-Please-share-your-settings

So I needed to force the gateway to stay open, enabled "track interface" en enable SLAAC afterwards. Radvd didn't start before I did that.

Just want to update that this is probably fixed. After I changed the monitoring IP to something else than Altibox gateway IP (I used their DNS instead) the monitor stays online and IPv6 is working through 6rd. I use DHCPv6 instead of SLAAC because it won't suit my needs, so rather live without IPv6 on my Android devices.

I'm still seeing the missing Default Gateway for IPv6 on my box. I'm here in Austin, so if a local netgate person wanted to come over to see what it does, I'm willing.

I'm also willing to get logs, etc, but need to know what y'all need.

Larry Rosenman wrote:

I'm still seeing the missing Default Gateway for IPv6 on my box. I'm here in Austin, so if a local netgate person wanted to come over to see what it does, I'm willing.

I'm also willing to get logs, etc, but need to know what y'all need.

Is it just the monitoring which is gone? If you can ping IPv6 addresses directly from the router, then maybe you must do as I did:

System -> Routing -> Gateways -> Edit your IPv6 GW
Change the monitor IP to another IPv6 address, like a DNS server. I realized that the IPv6 gateway was blocking PING, and therefore showed "Offline"

You can also choose to disable the monitoring.

No, I can NOT ping ipv6 addresses from the FW. Add the default route, and all is better.

Something somewhere is losing/not setting the IPv6 default route.

Larry Rosenman wrote:

No, I can NOT ping ipv6 addresses from the FW. Add the default route, and all is better.

Something somewhere is losing/not setting the IPv6 default route.

I have the same problem, the IPv6 route is being lost somewhere after every reboot and thus not being set.

I hope this can help you with the troubleshooting/isolation of the problem:

When I use a clean system with WAN+LAN cards from TP-Link [[http://www.tp-link.com/us/download/TG-3269.html]] everything works after a reboot, but when I use my standard pfSense system with old Marvel/Realtek controllers, things seem to break. Any other users experiencing similar issues isolated to Marvel/Realtek NICs?

Paal Andreas Lindsetmo wrote:

I hope this can help you with the troubleshooting/isolation of the problem:

When I use a clean system with WAN+LAN cards from TP-Link [[http://www.tp-link.com/us/download/TG-3269.html]] everything works after a reboot, but when I use my standard pfSense system with old Marvel/Realtek controllers, things seem to break. Any other users experiencing similar issues isolated to Marvel/Realtek NICs?

My NIC's is are Intel(R) PRO/1000.

upgraded today to:
2.4.0-RC (amd64)
built on Tue Sep 19 18:30:48 CDT 2017
FreeBSD 11.0-RELEASE-p12

and 6RD default route was there after the reboot without any intervention on my part.

And it BROKE again on:
2.4.0-RC (amd64)
built on Fri Sep 22 11:35:27 CDT 2017
FreeBSD 11.0-RELEASE-p12

And it works again at:
2.4.0-RC (amd64)
built on Fri Sep 22 20:41:05 CDT 2017
FreeBSD 11.0-RELEASE-p12

Broke again at:
2.4.0-RC (amd64)
built on Sat Sep 23 22:28:05 CDT 2017
FreeBSD 11.0-RELEASE-p12

Larry Rosenman wrote:

Broke again at:
2.4.0-RC (amd64)
built on Sat Sep 23 22:28:05 CDT 2017
FreeBSD 11.0-RELEASE-p12

Erm... I don't think there are any relevant changes going on, let alone twice a day.

just reporting what I'm seeing. I update to each RC and when I get IPv6 default route working I post, and when I see it NOT working, I post that as well.

I'd like to see an OFFICIAL response.

Well I'm just saying that the results of your testing appear to be completely random and unrelated to any versions. Have you tried rebooting without any upgrade?

with the current code a straight reboot has the IPv6 default installed.

However, on the upgrade to this code did NOT.

So, it's flakey at least on upgrades.

I still would like to see an OFFICIAL response.

Larry Rosenman wrote:

with the current code a straight reboot has the IPv6 default installed.

However, on the upgrade to this code did NOT.

So, it's flakey at least on upgrades.

I still would like to see an OFFICIAL response.

We're still looking at this.

The only way I found to reproduce this problem (no default gateway at boot) was using DHCP on WAN and I intentionally killed the server when the 6rd client was booting. As the 6rd depends on a valid IPv4 address on WAN it was not completely set up, but once I restarted the DHCP server everything worked.

What type of v4 uplink you use ?

What is the content of /tmp/*_routerv6 /tmp/*defaultgwv6 when it fails ? They exist ?

What is in the logs ?

We need more information as it doesn't seem to be a kernel issue anymore.

DHCP / WAN (passthrough from my NVG-599).

It's consistently (at least now) getting a route by default (I've moved to 2.4.1-DEV).

I'll keep an eye out.

BTW: will the 6rd patch be made public?

I can also confirm that it works with the current release (2.4.0.r.20170927.1221).
Only difference (when it comes to IPv6) - that I do recognize - from 2.3.4 is that IPv6 comes up a lot slower (~5-6 seconds) in this release then it does in 2.3.4 (~1-2 seconds).

this time around it took a bit for it to come up, but it did....

[2.4.1-DEVELOPMENT][ler@home-fw.lerctr.org]/tmp: cat em
em0_defaultgw em0_defaultgwv6 em0_error_output em0_output em0_router em0_routerv6
[2.4.1-DEVELOPMENT][ler@home-fw.lerctr.org]/tmp: cat em0_r
em0_router em0_routerv6
[2.4.1-DEVELOPMENT][ler@home-fw.lerctr.org]/tmp: cat em0_routerv6
2602:300:c533:1510::
[2.4.1-DEVELOPMENT][ler@home-fw.lerctr.org]/tmp: cat em0_defaultgwv6
2602:300:c533:1510::[2.4.1-DEVELOPMENT][ler@home-fw.lerctr.org]/tmp:

2.4.1-DEVELOPMENT (amd64)
built on Wed Sep 27 12:21:42 CDT 2017
FreeBSD 11.1-RELEASE-p1

The system is on the latest version.
Version information updated at Wed Sep 27 18:29:49 CDT 2017

I can confirm that my issue has been fixed (Altibox Norway ISP). Thank you very much!