Bug #14055
closedTraffic shaped by limiters is dropped when routed to a GIF gateway
100%
Description
Tested on pfSense+ 23.01.
Test:- Configure a GIF tunnel (IPv6 local/remote tunnel address) and interface for use in policy routing.
- Create download and upload limiters (used weighted queues with the WF2Q+ scheduler).
- Create a firewall rule on LAN - specify the GIF as the gateway and the limiter queues for the In / Out pipe.
- Send traffic over the GIF tunnel via the policy based rule.
Result:
For IPv6 traffic (used ICMP as test), the packet enters the gif0
interface and is then dropped. The system logs show the following when the packet is dropped:
Feb 28 19:16:32 kernel gif0: loop detected
Updated by Kristof Provost over 1 year ago
I've been able to reproduce it (with a floating rule so locally ordinated traffic hits the route-to rule).
The issue is fortunately fairly straightforward. When we route-to we tag the packet so that we still apply the route-to if the packet gets delayed and re-injected by dummynet. However, in many cases (e.g. there's no congestion) dummynet passes the packet through directly. In that case we process the route-to immediately, but we failed to clear the tag. That meant that when the packet came back to pf (after tunnelling by gif) we applied the route-to, and sent it back out the gif interface. This triggered the loop detection (happily, because without that we'd have panicked), and dropped the packet.
The following patch fixes it for me:
diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 893a57965cc..bd3c9334efa 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -6581,9 +6581,11 @@ pf_route(struct mbuf **m, struct pf_krule *r, int dir, struct ifnet *oifp, md = m0; error = pf_dummynet_route(pd, dir, s, r, ifp, sintosa(&dst), &md); - if (md != NULL) + if (md != NULL) { + pd->pf_mtag->flags &= ~PF_TAG_ROUTE_TO; error = (*ifp->if_output)(ifp, md, sintosa(&dst), NULL); + } } else m_freem(m0); } @@ -6786,8 +6788,10 @@ pf_route6(struct mbuf **m, struct pf_krule *r, int dir, struct ifnet *oifp, if ((u_long)m0->m_pkthdr.len <= ifp->if_mtu) { md = m0; pf_dummynet_route(pd, dir, s, r, ifp, sintosa(&dst), &md); - if (md != NULL) + if (md != NULL) { + pd->pf_mtag->flags &= ~PF_TAG_ROUTE_TO; nd6_output_ifp(ifp, ifp, md, &dst, NULL); + } } else { in6_ifstat_inc(ifp, ifs6_in_toobig);
Tomorrow I'll see about writing a test case and landing the fix upstream and in our branched (with fixed whitespace, obviously).
Updated by Jim Pingle over 1 year ago
- Subject changed from Traffic shaped by limiters is dropped when routed to a gif gateway to Traffic shaped by limiters is dropped when routed to a GIF gateway
- Status changed from New to In Progress
- Assignee set to Kristof Provost
- Target version set to 2.7.0
- Plus Target Version set to 23.05
Updated by Kristof Provost over 1 year ago
- Status changed from In Progress to Feedback
- % Done changed from 0 to 100
I've pushed a slightly simplified version of this patch upstream and to our branched. This should be fixed in the next snapshot builds.
Unfortunately I have not been able to write a useful test case for this.
Updated by Marcos M over 1 year ago
- Status changed from Feedback to Resolved
Works on 23.05.a.20230427.0208, thanks!