IPv6 Only FreeBSD Scenario

Earlier this year I described running Miredo on FreeBSD to gain access to the IPv6 Internet. Today I decided I would try to accomplish two goals. First, I would connect my FreeBSD gateway to the IPv6 Internet using Hexago/Freenet6 through the net/tspc2 port (Tunnel Setup Protocol Client). Second, I would deploy an IPv6-only host behind my FreeBSD gateway, and have it speak only IPv6 to the outside world.

I do not intend for this to be definitive by any means. Again, these are more or less personal notes. If someone else finds them useful, great.

First I registered with Hexago. This is not strictly necessary since anonymous access is apparently allowed. After registering I received an email with a username (I specified) and a password (provided) that I would add to the Tsp client. (I decided to try Tspc instead of manually deploying a tunnel because I heard Tspc was just too easy.)

After installing the net/tspc2 package, I literally added the information from the email to my /usr/local/etc/tspc.conf and started tspc2 manually.

mwmicro:/root# tspc -vvv
tspc - Tunnel Setup Protocol Client v2.1.1
Initializing (use -h for help)


Connecting to server with reliable udp
Using TSP protocol version 2.0.0
Establishing connection with tunnel broker...
Getting capabilities from server
Connection established
Authenticating taosecurity
Using authentification mecanism DIGEST-MD5
Authentication success
Asking for a tunnel
sent: Content-length: 204
<tunnel action="create" type="v6anyv4" proxy="no">
 <client>
  <address type="ipv4">69.143.202.28</address>
<keepalive interval="30"><address type="ipv6">::</address></keepalive> </client>
</tunnel>

recv:
200 Success
<tunnel action="info" type="v6v4" lifetime="604800">
  <server>
    <address type="ipv4">64.86.88.116</address>
    <address type="ipv6">2001:05c0:8fff:fffe:0000:0000:0000:5888</address>
  </server>
  <client>
    <address type="ipv4">69.143.202.28</address>
    <address type="ipv6">2001:05c0:8fff:fffe:0000:0000:0000:5889</address>
    <keepalive interval="30">
      <address type="ipv6">2001:05c0:8fff:fffe:0000:0000:0000:5888</address>
    </keepalive>
  </client>
</tunnel>


Processing response from server
sent: Content-length: 35
<tunnel action="accept"></tunnel>

Got tunnel parameters from server, setting up local tunnel
keepalive interval: 30

Going daemon, check /var/log/tspc.log for tunnel creation status

So far so good. Next I checked my routing table.

mwmicro:/root# netstat -nr -f inet6
Routing tables

Internet6:
Destination                       Gateway                       Flags      Netif Expire
default                           2001:5c0:8fff:fffe::5888      UGS        gif0
::1                               ::1                           UH          lo0
2001:5c0:8fff:fffe::5888          link#9                        UHL        gif0
2001:5c0:8fff:fffe::5889          link#9                        UHL         lo0
fe80::%sf1/64                     link#3                        UC          sf1
fe80::200:d1ff:feed:8c72%sf1      00:00:d1:ed:8c:72             UHL         lo0
fe80::%sf2/64                     link#4                        UC          sf2
fe80::200:d1ff:feed:8c73%sf2      00:00:d1:ed:8c:73             UHL         lo0
fe80::%fxp0/64                    link#6                        UC         fxp0
fe80::202:b3ff:fe0a:cd5e%fxp0     00:02:b3:0a:cd:5e             UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       fe80::1%lo0                   UHL         lo0
fe80::%gif0/64                    link#9                        UC         gif0
fe80::204:e2ff:fe29:4c3c%gif0     link#9                        UHL         lo0
ff01:3::/32                       link#3                        UC          sf1
ff01:4::/32                       link#4                        UC          sf2
ff01:6::/32                       link#6                        UC         fxp0
ff01:8::/32                       ::1                           UC          lo0
ff01:9::/32                       link#9                        UC         gif0
ff02::%sf1/32                     link#3                        UC          sf1
ff02::%sf2/32                     link#4                        UC          sf2
ff02::%fxp0/32                    link#6                        UC         fxp0
ff02::%lo0/32                     ::1                           UC          lo0
ff02::%gif0/32                    link#9                        UC         gif0

There's a lot of routes here, automatically created. This will be an issue for understanding IPv6 in the future. For now I was interested in the entries with gif0, for those represent the new tunnel.

mwmicro:/root# ifconfig gif0
gif0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
        tunnel inet 69.143.202.28 --> 64.86.88.116
        inet6 2001:5c0:8fff:fffe::5889 --> 2001:5c0:8fff:fffe::5888 prefixlen 128 
        inet6 fe80::204:e2ff:fe29:4c3c%gif0 prefixlen 64 scopeid 0x9

I decided to see if I could ping6 the other end of the tunnel.

mwmicro:/root# ping6 2001:5c0:8fff:fffe::5888
PING6(56=40+8+8 bytes) 2001:5c0:8fff:fffe::5889 --> 2001:5c0:8fff:fffe::5888
16 bytes from 2001:5c0:8fff:fffe::5888, icmp_seq=0 hlim=64 time=26.540 ms

I could also ping6 an IPv6 host.

mwmicro:/root# ping6 www.6bone.net
PING6(56=40+8+8 bytes) 2001:5c0:8fff:fffe::5889 --> 2001:5c0:0:2::24
16 bytes from 2001:5c0:0:2::24, icmp_seq=0 hlim=61 time=32.894 ms

Note that I used IPv4 to resolve www.6bone.net:

21:05:41.961734 IP 69.143.202.28.61517 > 68.87.73.242.53:  59311+ AAAA? www.6bone.net. (31)
21:05:42.053465 IP 68.87.73.242.53 > 69.143.202.28.61517:  59311 2/0/0 CNAME 6bone.net., (73)

I wondered how I could resolve IPs using an IPv6-speaking DNS server. I hunted high and low for one that would respond to my queries. Finally someone in #ipv6 on Freenode mentioned that NetBSD's resolver pointed by default to 2001:240::1. Could I use that?

mwmicro:/root# host 2001:240::1
1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0. 0.0.0.0.0.4.2.0.1.0.0.2.ip6.arpa
 domain name pointer ns9.iij.ad.jp.

2001:240::1 is ns9.iij.ad.jp. Count on the Japanese to have a working IPv6 system! Now will it resolve IPs?

mwmicro:/root# host www.6bone.net 2001:240::1
Using domain server:
Name: 2001:240::1
Address: 2001:240::1#53
Aliases:

www.6bone.net is an alias for 6bone.net.
6bone.net has address 206.162.147.152
Using domain server:
Name: 2001:240::1
Address: 2001:240::1#53
Aliases:

www.6bone.net is an alias for 6bone.net.
6bone.net has IPv6 address 2001:5c0:0:2::24
Using domain server:
Name: 2001:240::1
Address: 2001:240::1#53
Aliases:

www.6bone.net is an alias for 6bone.net.
6bone.net mail is handled by 10 quark.isi.edu.
6bone.net mail is handled by 20 darkstar.isi.edu.
6bone.net mail is handled by 0 venera.isi.edu.

Bingo. By the way, if you can suggest alternative IPv6 DNS servers, please leave a comment.

At this point I accomplished my first goal. On to the second. To get the gateway to work as an IPv6 gateway, I added the following to /usr/local/etc/tspc.conf:

#---------------------
# Router configuration
#
# In order to configure the machine as a router, a prefix must be requested
# and an interface must be specified.  The prefix will be advertised
# through that interface.
#
# host_type=host|router
#  default = host.
host_type=router

#
# prefixlen specifies the required prefix length for the TSP client 
#  network. Valid values are 64 or 48. 64 is for one link. 48 is for
#  a whole enterprise network (65K links).
prefixlen=48

#
# if_prefix is the name of the OS interface that will be configured
#  with the first /64 of the received prefix from the broker and the
#  router advertisement daemon is started to advertise that prefix
#  on the if_prefix interface.
if_prefix=sf3

Note sf3 is the internal interface, i.e., the one facing away from the Internet.

mwmicro:/root# ifconfig sf3
sf3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>mtu 1500
        inet6 fe80::200:d1ff:feed:8c74%sf3 prefixlen 64 scopeid 0x5 
        inet6 2001:5c0:925d::1 prefixlen 64 
        ether 00:00:d1:ed:8c:74
        media: Ethernet autoselect (10baseT/UTP)
        status: active

I also added these entries to /etc/rc.conf on my gateway:

ipv6_enable="YES"
ipv6_gateway_enable="YES"
rtadvd_enable="YES"
rtadvd_interfaces="sf3"
tspc2_enable="YES"

I next built a new FreeBSD host (on a P200 with 32 MB RAM, no less). The box did not have a working CD-ROM, so I had to use boot floppies. I found no easy way to do an IPv6-only network install, so I assigned a temporary IPv4 address for the network installation.

After installing FreeBSD, I rebooted the p200 system and removed the IPv4 address. Now my interface looked like this:

p200:/home/richard$ ifconfig dc0
dc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        options=8<VLAN_MTU>
        inet6 fe80::204:5aff:fe79:43a7%dc0 prefixlen 64 scopeid 0x1 
        inet6 2001:5c0:925d:0:204:5aff:fe79:43a7 prefixlen 64 autoconf 
        ether 00:04:5a:79:43:a7
        media: Ethernet autoselect (10baseT/UTP)
        status: active

My routing tables on p200 looked like this:

p200:/home/richard$ netstat -nr -f inet6
Routing tables

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::/96                             ::1                           UGRS        lo0 =>
default                           fe80::200:d1ff:feed:8c74%dc0  UG          dc0
::1                               ::1                           UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2001:5c0:925d::/64                link#1                        UC          dc0
2001:5c0:925d::1                  00:00:d1:ed:8c:74             UHLW        dc0
2001:5c0:925d:0:204:5aff:fe79:43a7 00:04:5a:79:43:a7             UHL         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%dc0/64                     link#1                        UC          dc0
fe80::200:d1ff:feed:8c74%dc0      00:00:d1:ed:8c:74             UHLW        dc0
fe80::204:5aff:fe79:43a7%dc0      00:04:5a:79:43:a7             UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       fe80::1%lo0                   UHL         lo0
ff01:1::/32                       link#1                        UC          dc0
ff01:3::/32                       ::1                           UC          lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%dc0/32                     link#1                        UC          dc0
ff02::%lo0/32                     ::1                           UC          lo0

I achieved setting a default route manually with

route add -inet6 2000::/3 2001:5c0:925d::1

where 2001:5c0:925d::1 is the IPv6 address of my gateway (remember the output for interface sf3 earlier).

I configured p200's /etc/rc.conf like so:

hostname="p200.taosecurity.com"
ipv6_enable="YES"
ipv6_defaultrouter="2001:5c0:925d::1"
sshd_enable="YES"

From my gateway, I could now reach p200 using either of its IPv6 addresses (local or global):

mwmicro:/root# ping6 -c 1 fe80::204:5aff:fe79:43a7%sf3
PING6(56=40+8+8 bytes) fe80::200:d1ff:feed:8c74%sf3 --> fe80::204:5aff:fe79:43a7%sf3
16 bytes from fe80::204:5aff:fe79:43a7%sf3, icmp_seq=0 hlim=64 time=1.184 ms

--- fe80::204:5aff:fe79:43a7%sf3 ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.184/1.184/1.184/0.000 ms

Note using the above local method requires specifying an interface (%sf3) out of which the ICMP6 echo is sent.

mwmicro:/root# ping6 -c 1 2001:5c0:925d:0:204:5aff:fe79:43a7
PING6(56=40+8+8 bytes) 2001:5c0:925d::1 --> 2001:5c0:925d:0:204:5aff:fe79:43a7
16 bytes from 2001:5c0:925d:0:204:5aff:fe79:43a7, icmp_seq=0 hlim=64 time=1.205 ms

--- 2001:5c0:925d:0:204:5aff:fe79:43a7 ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.205/1.205/1.205/0.000 ms

The ping to 2001:5c0:925d:0:204:5aff:fe79:43a7 does not require the same interface specification.

After I connected via SSH from the gateway to p200, i.e.

mwmicro:/root# ssh richard@2001:5c0:925d:0:204:5aff:fe79:43a7

I was able to perform IPv6-only actions from p200. For example:

p200:/home/richard$ ping6 -c 1 www.6bone.net
PING6(56=40+8+8 bytes) 2001:5c0:925d:0:204:5aff:fe79:43a7 --> 2001:5c0:0:2::24
16 bytes from 2001:5c0:0:2::24, icmp_seq=0 hlim=60 time=29.471 ms

--- 6bone.net ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 29.471/29.471/29.471/0.000 ms

Here is what the traffic looked like between p200 and the gateway:

21:24:56.729796 IP6 2001:5c0:925d:0:204:5aff:fe79:43a7.49166 > 2001:240::1.53: 
 39700+ AAAA? www.6bone.net. (31)
21:24:57.188971 IP6 2001:240::1.53 > 2001:5c0:925d:0:204:5aff:fe79:43a7.49166: 
 39700 2/2/2[|domain]
21:24:57.193397 IP6 2001:5c0:925d:0:204:5aff:fe79:43a7 > 2001:5c0:0:2::24:
 ICMP6, echo request, seq 0, length 16
21:24:57.222194 IP6 2001:5c0:0:2::24 > 2001:5c0:925d:0:204:5aff:fe79:43a7:
 ICMP6, echo reply, seq 0, length 16

I also caught neighbor soliciation and advertisements between the two hosts.

21:24:59.309014 IP6 2001:5c0:925d:0:204:5aff:fe79:43a7 > 2001:5c0:925d::1:
 ICMP6, neighbor solicitation, who has 2001:5c0:925d::1, length 32
21:24:59.309325 IP6 2001:5c0:925d::1 > 2001:5c0:925d:0:204:5aff:fe79:43a7:
 ICMP6, neighbor advertisment, tgt is 2001:5c0:925d::1, length 24
21:25:01.728706 IP6 fe80::204:5aff:fe79:43a7 > fe80::200:d1ff:feed:8c74:
 ICMP6, neighbor solicitation, who has fe80::200:d1ff:feed:8c74, length 32
21:25:01.729104 IP6 fe80::200:d1ff:feed:8c74 > fe80::204:5aff:fe79:43a7:
 ICMP6, neighbor advertisment, tgt is fe80::200:d1ff:feed:8c74, length 24

How cool is this -- public IPv6 NTP servers:

p200:/root# ntpdate ntp6.space.net
 8 Sep 21:28:49 ntpdate[599]: adjust time server 2001:608::1000:1 offset -0.045629 sec

Here is the traffic.

21:28:47.456604 IP6 2001:5c0:925d:0:204:5aff:fe79:43a7.49168 > 2001:240::1.53: 
 36503+ A? ntp6.space.net. (32)
21:28:48.240130 IP6 2001:5c0:925d:0:204:5aff:fe79:43a7 > 2001:5c0:925d::1:
 ICMP6, neighbor solicitation, who has 2001:5c0:925d::1, length 32
21:28:48.240433 IP6 2001:5c0:925d::1 > 2001:5c0:925d:0:204:5aff:fe79:43a7:
 ICMP6, neighbor advertisment, tgt is 2001:5c0:925d::1, length 24
21:28:48.280169 IP6 2001:240::1.53 > 2001:5c0:925d:0:204:5aff:fe79:43a7.49168: 
 36503 2/3/0[|domain]
21:28:48.281557 IP6 2001:5c0:925d:0:204:5aff:fe79:43a7.49169 > 2001:240::1.53: 
 36504+ AAAA? ntp6.space.net. (32)
21:28:48.847565 IP6 2001:240::1.53 > 2001:5c0:925d:0:204:5aff:fe79:43a7.49169: 
 36504 2/3/0[|domain]
21:28:48.972514 IP6 2001:5c0:925d:0:204:5aff:fe79:43a7.123 > 2001:608::1000:1.123:
 NTPv4, Client, length 48
21:28:49.115271 IP6 2001:608::1000:1.123 > 2001:5c0:925d:0:204:5aff:fe79:43a7.123:
 NTPv4, Server, length 48

Even FTP works.

p200:/root# ftp ftp.freebsd.org
Trying 2001:4f8:0:2::e...
Connected to ftp.freebsd.org.
220 Welcome to freebsd.isc.org.
Name (ftp.freebsd.org:richard): ftp
331 Please specify the password.
Password:
230-
230-You have reached the freebsd.isc.org FTP server, serving the 
230-full FreeBSD FTP archive over IPv4 (204.152.184.73) and IPv6 
230-(2001:4f8:0:2::e) networks.  This server is also known as:

I think the key to understanding IPv6 is to start running deployments like this and watching traffic. I'm remembering that's how I started learning IPv4 in September 1998.

I welcome any constructive tips!