Project

General

Profile

Actions

Regression #13420

closed

TCP traffic sourced from the firewall can only use the default gateway

Added by Steve Wheeler over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Category:
Rules / NAT
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
23.01
Release Notes:
Default
Affected Version:
2.6.0
Affected Architecture:
All

Description

Traffic sourced from the firewall itself will always open states on the interface with the default system route. Even though traffic may not actually leave via that interface.

This means that for TCP traffic it can be blocked by the default firewall rules as being out-of-state.

For example this 6100 has two WANs configured. Both are DHCP with gateways passed by the upstream server:

[22.05-RELEASE][root@6100-2.stevew.lan]/root: netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.21.16.1        UGS         ix3
127.0.0.1          link#10            UH          lo0
172.21.16.0/24     link#8             U           ix3
172.21.16.1        90:ec:77:0f:74:43  UHS         ix3
172.21.16.170      link#8             UHS         lo0
192.168.170.0/24   link#1             U          igc0
192.168.170.1      link#1             UHS         lo0
192.168.241.0/24   link#7             U           ix2
192.168.241.1      90:ec:77:0f:74:44  UHS         ix2
192.168.241.10     link#7             UHS         lo0

The default gateway is via WAN (ix3).

It can a device that is further upstream outside both the WAN subnets via either route:

[22.05-RELEASE][root@6100-2.stevew.lan]/root: ping -c 2 -S 172.21.16.170 192.168.2.32
PING 192.168.2.32 (192.168.2.32) from 172.21.16.170: 56 data bytes
64 bytes from 192.168.2.32: icmp_seq=0 ttl=62 time=1.064 ms
64 bytes from 192.168.2.32: icmp_seq=1 ttl=62 time=0.753 ms

--- 192.168.2.32 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.753/0.909/1.064/0.156 ms
[22.05-RELEASE][root@6100-2.stevew.lan]/root: ping -c 2 -S 192.168.241.10 192.168.2.32
PING 192.168.2.32 (192.168.2.32) from 192.168.241.10: 56 data bytes
64 bytes from 192.168.2.32: icmp_seq=0 ttl=61 time=1.274 ms
64 bytes from 192.168.2.32: icmp_seq=1 ttl=61 time=0.781 ms

--- 192.168.2.32 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.781/1.027/1.274/0.247 ms

However a TCP connection to the same target will only succeed via the default system route:

[22.05-RELEASE][root@6100-2.stevew.lan]/root: iperf3 -c 192.168.2.32 -t 3 -B 172.21.16.170
Connecting to host 192.168.2.32, port 5201
[  5] local 172.21.16.170 port 29501 connected to 192.168.2.32 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  92.0 MBytes   771 Mbits/sec    3    369 KBytes       
[  5]   1.00-2.00   sec  92.4 MBytes   775 Mbits/sec   17    389 KBytes       
[  5]   2.00-3.00   sec  93.3 MBytes   783 Mbits/sec    2    392 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-3.00   sec   278 MBytes   776 Mbits/sec   22             sender
[  5]   0.00-3.37   sec   277 MBytes   689 Mbits/sec                  receiver

iperf Done.
[22.05-RELEASE][root@6100-2.stevew.lan]/root: iperf3 -c 192.168.2.32 -t 3 -B 192.168.241.10
Connecting to host 192.168.2.32, port 5201
[  5] local 192.168.241.10 port 32893 connected to 192.168.2.32 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.06   sec   137 KBytes  1.05 Mbits/sec    3   1.41 KBytes       
[  5]   1.06-2.01   sec  0.00 Bytes  0.00 bits/sec    2   1.41 KBytes       
[  5]   2.01-3.01   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-3.01   sec   137 KBytes   372 Kbits/sec    5             sender
[  5]   0.00-3.22   sec   116 KBytes   295 Kbits/sec                  receiver

iperf Done.

On the firewall itself both those connections show as states on the WAN interface:

all tcp 172.21.16.170:17826 -> 192.168.2.32:5201       FIN_WAIT_2:FIN_WAIT_2
   [3379762330 + 2147352832] wscale 7  [1691518013 + 4278255872] wscale 7
   age 00:00:12, expires in 00:01:21, 18:15 pkts, 1380:1090 bytes, rule 85
   id: 931afc6200000003 creatorid: 6d1e6c6f gateway: 172.21.16.1
   origif: ix3
all tcp 192.168.241.10:10643 -> 192.168.2.32:5201       ESTABLISHED:CLOSING
   [879166104 + 838795520] wscale 7  [2257063987 + 3841851648] wscale 7
   age 00:00:06, expires in 00:14:57, 18:17 pkts, 1376:1174 bytes, rule 86
   id: 941afc6200000003 creatorid: 6d1e6c6f gateway: 192.168.241.1
   origif: ix3

all tcp 172.21.16.170:29501 -> 192.168.2.32:5201       TIME_WAIT:TIME_WAIT
   [4223141156 + 2280204800] wscale 7  [3053074132 + 65792] wscale 7
   age 00:00:12, expires in 00:01:21, 200932:91890 pkts, 301393701:4782292 bytes, rule 85
   id: ad1afc6200000001 creatorid: 6d1e6c6f gateway: 172.21.16.1
   origif: ix3
all tcp 192.168.241.10:32893 -> 192.168.2.32:5201       ESTABLISHED:SYN_SENT
   [1066512255 + 3856466432] wscale 7  [116550158 + 4294902016] wscale 7
   age 00:00:06, expires in 00:00:24, 92:22 pkts, 133701:1152 bytes, rule 86
   id: ae1afc6200000001 creatorid: 6d1e6c6f gateway: 192.168.241.1
   origif: ix3

The upstream router on WAN2 shows the traffic was correctly sent via that route:

all tcp 192.168.2.32:5201 <- 192.168.241.10:25352       FIN_WAIT_2:ESTABLISHED
   [3059127359 + 4278255872] wscale 7  [1873996070 + 2147090688] wscale 7
   age 00:10:57, expires in 00:05:00, 90:48 pkts, 130701:2624 bytes, rule 136
   id: 18c4f06200000000 creatorid: 8775c03c gateway: 0.0.0.0
   origif: ix2
all tcp 172.21.16.246:44264 (192.168.241.10:25352) -> 192.168.2.32:5201       ESTABLISHED:FIN_WAIT_2
   [1873996070 + 2147090688] wscale 7  [3059127359 + 4278255872] wscale 7
   age 00:10:57, expires in 00:05:00, 90:48 pkts, 130701:2624 bytes, rule 113
   id: 19c4f06200000000 creatorid: 8775c03c gateway: 172.21.16.1
   origif: ix3
all tcp 192.168.2.32:5201 <- 192.168.241.10:32893       FIN_WAIT_2:ESTABLISHED
   [133327374 + 4278255872] wscale 7  [895594879 + 2147090688] wscale 7
   age 00:03:33, expires in 00:12:25, 94:49 pkts, 136701:2664 bytes, rule 136
   id: 4444f16200000001 creatorid: 8775c03c gateway: 0.0.0.0
   origif: ix2
all tcp 172.21.16.246:5354 (192.168.241.10:32893) -> 192.168.2.32:5201       ESTABLISHED:FIN_WAIT_2
   [895594879 + 2147090688] wscale 7  [133327374 + 4278255872] wscale 7
   age 00:03:33, expires in 00:12:25, 94:49 pkts, 136701:2664 bytes, rule 113
   id: 4544f16200000001 creatorid: 8775c03c gateway: 172.21.16.1
   origif: ix3

The firewall log on the sending device shows that traffic is eventually blocked as the TCP flags do not match the open states:

Aug 17 00:20:47 6100-2 filterlog[39606]: 5,,,1000000104,ix3,match,block,out,4,0x0,,64,0,0,DF,6,tcp,1500,192.168.241.10,192.168.2.32,32893,5201,1448,A,2142270373:2142271821,241889798,514,,nop;nop;TS
Aug 17 00:21:02 6100-2 filterlog[39606]: 5,,,1000000104,ix3,match,block,out,4,0x0,,64,0,0,DF,6,tcp,1500,192.168.241.10,192.168.2.32,32893,5201,1448,A,2142270373:2142271821,241889798,514,,nop;nop;TS
Aug 17 00:21:11 6100-2 filterlog[39606]: 4,,,1000000103,ix2,match,block,in,4,0x0,,61,47134,0,DF,6,tcp,52,192.168.2.32,192.168.241.10,5201,32893,0,FA,241889798,2142331189,501,,nop;nop;TS
Aug 17 00:21:17 6100-2 filterlog[39606]: 5,,,1000000104,ix3,match,block,out,4,0x0,,64,0,0,DF,6,tcp,1500,192.168.241.10,192.168.2.32,32893,5201,1448,A,2142270373:2142271821,241889798,514,,nop;nop;TS
Aug 17 00:21:34 6100-2 filterlog[39606]: 5,,,1000000104,ix3,match,block,out,4,0x0,,64,0,0,DF,6,tcp,1500,192.168.241.10,192.168.2.32,32893,5201,1448,RA,2142270373:2142271821,241889798,0,,nop;nop;TS

Tested in 22.05-rel


Files

6100-2_rules.txt (11.6 KB) 6100-2_rules.txt Steve Wheeler, 08/17/2022 06:45 AM
6100-2_ruleset.txt (10.3 KB) 6100-2_ruleset.txt Steve Wheeler, 08/17/2022 06:45 AM

Related issues

Related to Bug #12630: States are always created on the default gateway interface.Not a Bug

Actions
Related to Todo #15173: Add global option to set default PF State Policy (if-bound vs floating)ResolvedJim Pingle

Actions
Actions

Also available in: Atom PDF