FreeBSD ipfw, nat,IPsec, and IPv6 DNS

I recently setup a FreeBSD 8.3 (amd64) system as a router for a local network. The internal network is dual stack IPv4 + IPv6. This required a custom configured firewall as well beyond the standard "simple", "open", etc.

ipfw + natd

The first unexpected event happened when discovered that after rebooting, NAT no longer worked. Some investigating revealed that when processing firewall rules, when adding

${fwcmd} add 150 divert natd all from any to any via ${public_if}

during boot, it failed with the error

ipfw: getsockopt(IP_FW_ADD): Invalid Argument

Apparently I've become too expectant of relevant kernel modules to just be there. The ipdivert module wasn't being loaded until natd needed it, which is after ipfw initialized.

Solution: Add ipdivert_load="YES" to /boot/loader.conf - then it's ready well before ipfw needs it.

IPsec

In general, I've avoided using IPsec for VPNs in favor of OpenVPN or tinc. But since much of the world uses IPsec, I figured I should get more experience with it. (After this experience, I still strongly lean toward those.) I followed the guide for FreeBSD VPN over IPsec. That worked for a FreeBSD 7 system, but I ran into a non-fatal problem on my FreeBSD 8 box:

WARNING: setsockopt(UDP_ENCAP_ESPINUDP_NON_IKE): UDP_ENCAP Invalid argument

Solution:kernel config needs IPSEC_NAT_T enabled as well as IPSEC:

options IPSEC #IP security
options IPSEC_NAT_T
device crypto

(I found it surprising this wasn't all part of GENERIC by default - seems most features are, and those that aren't are in a loadable kernel module)

I fan into further problems with SA negotiation, due to not quite properly swapping EVERY IP in the config between the two machines.

I think it's worth not note to anyone reading this: make sure you test with a packet sniffer (tcpdump or wireshark or similar) - even if the secure link is not setup, packets will still transmit, just in the clear. So just because the link works doesn't mean the "private" part of the VPN is there.

DNS & IPv6 bootstrapping

More for the hell of it than any legitimate reason, I wanted to use the SixXS IPv6 DNS servers to access Google's sites via IPv6. Including from the machine working as a router. But that presented a few problems:

dhclient insists on updating resolv.conf

I need a DHCP client to get the dynamic IP for the public interface. Putting anything in /etc/resolv.conf gets overwritten. Also, IPv6 addresses are not accepted in /etc/dhclient.conf. So, I ran a nameserver on the localhost, listening on IPv4 and pointed /etc/dhclient.conf to it:

supersede domain-name-servers 127.0.0.1;

PowerDNS and IPv6

With its scary security history, and repeated new vulnerabilities, I do not run BIND. My preference has been DJBDNS tinydns/dnscache. But those lack IPv6 support. So I chose PowerDNS instead.
Doing an initial test of dig @127.0.0.1 aaaa ds.v6ns.test-ipv6.com did not work, and I discovered that unless you setup something for the outgoing IPv6 address in /usr/local/etc/pdns/recursor.conf, it won't attempt to use IPv6 while resolving DNS:

#send IPv6 queries using any source
#"::" is similar to "0.0.0.0" for IPv4
query-local-address6=::

sixxs-aiccu needs hostnames to connect

With that working, I decided to make it pass on all requests to SixXS by adding

forward-zones-recurse= .=2001:4de0:1000:a3::2, .=2001:4de0:1000:a4::2

to /usr/local/etc/pdns/recursor.conf
However on the next reboot, sixxs-aiccu wouldn't start. As I had it configured, it was connecting by hostname. So it needs working DNS before making the bridge to the IPv6 internet.

I could alter it to connect by IP, but who knows what other services may have similar problems. Since I was many interested in using it for accessing Google sites, I just changed my /usr/local/etc/pdns/recursor.conf to use:

forward-zones-recurse= google.com=2001:4de0:1000:a3::2, google.com=2001:4de0:1000:a4::2

which only uses SixXS for Google related requests, an IPv4 root servers by default, allowing it to look up the names of the servers needed to start the IPv6 tunnel through SixXS.

(Side note: youtube.com and blogger.com are CNAMEs for google.com addresses, so the AAAA lookup still goes to the SixXS servers for them, and I'm now posting here via IPv6)