pfSense

Hi, I'm running pfsense 2.0-RC1.

• I have 3 interfaces (LAN (em2) - 10.1.1.192/26, WAN (em0) - 10.1.1.56 and DMZ (em1) - NO IP).
• I'm using a PPPOE server configured on the DMZ network, where a host is successfully logging and being assigned a public internet address (x.x.x.x). (The P2P link is x.x.x.x->172.31.0.1)

I have created wildcard rules on ALL firewall interfaces (Floating, LAN, WAN, DMZ & PPPOE Server) that allows any IP to talk to any IP on any port. (I wouldnt want to run this way, but I couldnt get outbound connectivity for my pppoe client).

    *    *    *    *    *    *    none         Enable Outbound Traffic for PPPOE Clients 

When pfctl is enabled, my PPPOE client with a public address (x.x.x.x), cannot communicate on the internet. Packets dont get past pfsense.

When pfctl is disabled, my PPPOE client CAN communicate on the internet. (So I know routing and everything is OK).

While using tcpdump on each interface (and pfctl enabled), I can see packets arriving on poes10, DMZ (em1 - PPPOE Session packets), however, I cannot see any packets leaving on WAN (em0).

With pfctl enabled, I can successfully SSH into the host from the internet.

I'm thinking that this is not right.

For info, a pfctl -s all shows this:

TRANSLATION RULES:
nat-anchor "natearly/*" all
nat-anchor "natrules/*" all
nat on em0 inet from 10.1.1.192/26 port = isakmp to any port = isakmp -> 10.1.1.56 port 500
nat on em0 inet from 10.1.1.192/26 to any -> 10.1.1.56 port 1024:65535
rdr-anchor "relayd/*" all
rdr-anchor "tftp-proxy/*" all
rdr-anchor "miniupnpd" all

FILTER RULES:
scrub in on em0 all fragment reassemble
scrub in on em2 all fragment reassemble
scrub in on em1 all fragment reassemble
anchor "relayd/*" all
block drop in log all label "Default deny rule" 
block drop out log all label "Default deny rule" 
block drop in quick inet6 all
block drop out quick inet6 all
block drop quick proto tcp from any port = 0 to any
block drop quick proto tcp from any to any port = 0
block drop quick proto udp from any port = 0 to any
block drop quick proto udp from any to any port = 0
block drop quick from <snort2c> to any label "Block snort2c hosts" 
block drop quick from any to <snort2c> label "Block snort2c hosts" 
block drop quick from <pfSnortSamout> to any label "Block pfSnortSamOut hosts" 
block drop quick from any to <pfSnortSamin> label "Block pfSnortSamIn hosts" 
block drop in log quick proto tcp from <sshlockout> to any port = rsh-spx label "sshlockout" 
block drop in log quick proto tcp from <webConfiguratorlockout> to any port = 15443 label "webConfiguratorlockout" 
block drop in quick from <virusprot> to any label "virusprot overload table" 
block drop in on ! em0 inet from 10.1.1.0/26 to any
block drop in inet from 10.1.1.56 to any
block drop in on em0 inet6 from fe80::20c:29ff:fee9:29c3 to any
pass in on em0 proto udp from any port = bootps to any port = bootpc keep state label "allow dhcp client out WAN" 
pass out on em0 proto udp from any port = bootpc to any port = bootps keep state label "allow dhcp client out WAN" 
block drop in on ! em2 inet from 10.1.1.192/26 to any
block drop in inet from 10.1.1.193 to any
block drop in on em2 inet6 from fe80::20c:29ff:fee9:29d7 to any
pass in on em2 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server" 
pass in on em2 inet proto udp from any port = bootpc to 10.1.1.193 port = bootps keep state label "allow access to DHCP server" 
pass out on em2 inet proto udp from 10.1.1.193 port = bootps to any port = bootpc keep state label "allow access to DHCP server" 
pass in on lo0 all flags S/SA keep state label "pass loopback" 
pass out on lo0 all flags S/SA keep state label "pass loopback" 
pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself" 
pass out route-to (em0 10.1.1.1) inet from 10.1.1.56 to ! 10.1.1.0/26 flags S/SA keep state allow-opts label "let out anything from firewall host itself" 
pass in quick on em2 proto tcp from any to (em2) port = 15443 flags S/SA keep state label "anti-lockout rule" 
pass in quick on em2 proto tcp from any to (em2) port = https flags S/SA keep state label "anti-lockout rule" 
pass in quick on em2 proto tcp from any to (em2) port = rsh-spx flags S/SA keep state label "anti-lockout rule" 
pass on em0 all flags S/SA keep state label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass on em2 all flags S/SA keep state label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass on em1 all flags S/SA keep state label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass in quick on em0 reply-to (em0 10.1.1.1) inet proto tcp from any to <DMZNET> port = http flags S/SA keep state label "USER_RULE: Enable HTTP to DMZ" 
pass in quick on em0 reply-to (em0 10.1.1.1) inet proto tcp from any to <DMZNET> port = rsh-spx flags S/SA keep state label "USER_RULE: Enable SSH to DMZ" 
pass in quick on em0 reply-to (em0 10.1.1.1) inet proto tcp from any to 10.1.1.56 port = 15443 flags S/SA keep state label "USER_RULE: Enable webGUI" 
pass in quick on em0 reply-to (em0 10.1.1.1) inet proto tcp from any to 10.1.1.56 port = rsh-spx flags S/SA keep state label "USER_RULE: Enable SSH" 
pass in quick on em0 reply-to (em0 10.1.1.1) inet all flags S/SA keep state label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass in quick on em2 inet from 10.1.1.192/26 to any flags S/SA keep state label "USER_RULE: Default allow LAN to any rule" 
pass in quick on em2 all flags S/SA keep state label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass in quick on em1 all flags S/SA keep state label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
anchor "tftp-proxy/*" all
anchor "miniupnpd" all
No queue in use

...

On looking at the /tmp/rules.debug that is created, I see this syntatical error which is probably the cause:

# User-defined rules follow
pass  on {  em0  em2  em1  }  from any to any keep state  label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass  in  quick  on $WAN reply-to ( em0 10.1.1.1 )  proto tcp  from any to   $DMZNET port 80  flags S/SA keep state  label "USER_RULE: Enable HTTP to DMZ" 
pass  in  quick  on $WAN reply-to ( em0 10.1.1.1 )  proto tcp  from any to   $DMZNET port 222  flags S/SA keep state  label "USER_RULE: Enable SSH to DMZ" 
pass  in  quick  on $WAN reply-to ( em0 10.1.1.1 )  proto tcp  from any to 10.1.1.56 port 15443  flags S/SA keep state  label "USER_RULE: Enable webGUI" 
pass  in  quick  on $WAN reply-to ( em0 10.1.1.1 )  proto tcp  from any to 10.1.1.56 port 222  flags S/SA keep state  label "USER_RULE: Enable SSH" 
pass  in  quick  on $WAN reply-to ( em0 10.1.1.1 )  from any to any keep state  label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass  in  quick  on $LAN  from 10.1.1.192/26 to any keep state  label "USER_RULE: Default allow LAN to any rule" 
pass  in  quick  on $LAN  from any to any keep state  label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
pass  in  quick  on $DMZ  from any to any keep state  label "USER_RULE: Enable Outbound Traffic for PPPOE Clients" 
# WANLANDMZ pppoe array key does not exist for Enable Outbound Traffic label "USER_RULE: Enable Outbound Traffic" 

NOTE THE LAST LINE prefixed with a hash and has the words "array key does not exist for". This is my rule for PPPOE Server Firewall rule which is commented out (and should let the PPPOE clients outbound access) and incorrect anyway...