<h1class="post-title"><ahref="https://paulwilde.uk/ponderings/my-battles-with-netcup-ipv6-on-freebsd/">My Battle with Netcup, IPv6 and FreeBSD combination</a></h1>
<p>Recently I've started to migrate some of my VPS's to <ahref="https://netcup.eu">Netcup</a>
as I have found their offerings more performant, and less expensive, than other
VPS providers.<br/>
However, it seems due to the way Netup's IPv6 network is set up causes some issues
with FreeBSD, and allegedly other BSDs leading to loss of connectivity via IPv6.</p>
<spanid="continue-reading"></span>
<p>I have moved to Netcup for a few reasons: 1. expense, 2. experienced better performance than my previous provider, and 3. ability to provide my own ISOs for OS install.<br/>
Netcup in my experience are great, I have a few Debian systems with them that have been super reliable.<br/>
But, I want to use FreeBSD for my projects, so my new VPSs run FreeBSD.</p>
<h3id="the-issue">The Issue</h3>
<p>All is well on the IPv4 front, the DHCP client picks up my provided IPv4 address and connectivity is there, great. I still manually set it statically for consistency, but either way it works great.<br/>
IPv6 however, different story. FreeBSD doesn't seem to get it's provided IPv6 address via
SLAAC, so no address gets associated. That's OK though, I can set it statically in
<code>rc.conf</code>. Netcup provide a /64 IPv6 subnet for each VPS, so I can pick any IPv6 in the range
they have provided, but they also sent one via email, so I'll just use that for now.</p>
<p><code>2a03:4000</code> is Netcup's network I believe with <code>AAAA</code> defining a certain part of that network.<br/>
<code>BBBB</code> is the subnet provided to me with <code>cccc:dddd:eeee:ffff</code> being the range of address I can use in that subnet.</p>
<p>Netcup's IPv6 network is switched, not routed, I'm sure there's a reason for this, and I'm sure it's sensible, but for now the important bit is that the IPv6 gateway should always be <code>fe80::1%vtnet0</code> (where <code>vtnet0</code> is the external interface name) rather than another <code>2a03:4000::AAAA:BBBB</code> IPv6 address. I pop that into my <code>rc.conf</code> as well.</p>
<p>Reboot the VPS, and all works well. <code>ping6</code>s work, <code>curl</code>s to IPv6 addresses work, incoming IPv6 connections work. Great! Done...</p>
<p>Not Done.</p>
<p>After about 10 minutes of connectivity, IPv6 just drops out. <code>ping6</code>s fail, <code>curl</code>s fail, no connectivity on IPv6 from the outside world. Drat.<br/>
Reboot the VPS again... oh, it's back. Great. Wait a few minutes, it goes again. Reboot, back, wait, gone. And so on.</p>
<h3id="tests-and-research">Tests and Research</h3>
<p>Obviously the first thing I'm thinking is firewall. So I disable <code>pf</code> and reboot and wait again - no difference, IPv6 just stops working after a few minutes.</p>
<p>In my fairly inexperienced IPv6 knowledge (it's not big in the UK... yet), using <code>fe80::1</code> as a gateway feels weird, so I try a <code>traceroute6 anyaddress.com</code> to see what the first hope on the route is (it must be my actual gateway, right?) and pop that address into my <code>ipv6_defaultrouter=</code> config in <code>rc.conf</code>.<br/>
Reboot... no joy, no IPv6 connectivity at all. OK, I set that back.</p>
<p>I browse the netcup forums a bit, find a handful of posts of people experiencing the same issue. Some refer to people leasing an additional IPv6 lease from Netcup and using that instead which allegedly works for them. I did this, it did not work for me - the IPv6 worked, for about 10 minutes again, then dropped off as before.<br/>
FreeBSD forums is the same sort of thing, articles from a few years ago.</p>
<p>Both forums seem to have posts saying the issue lies in the fact the IPv6 address and [actual] gateway exist in different subnets.
My simple understanding of this is: the gateway may have address <code>2a03:4000:AAAA::2</code> which is in a /48 subnet, and my VPS may have IPv6 address <code>2a03:4000:AAAA:BBBB::1234</code> which is a /64 subnet. Because they do not exist on the same subnet, <em>something</em> happens at some point to stop them being able to communicate. I believe this "<em>something</em>" is to do with Netcup's switching (rather than routing).</p>
<p>Of the people experiencing this, some have had the IPv6 Connectivity come back and drop off again on its own. I haven't experienced this, but perhaps I'm just impatient.</p>
<p>So, <ahref="https://forums.freebsd.org/threads/netcup-ipv6-intermittant.96489/">I write a post on the FreeBSD forums</a> with a bit of information of things I've tried asking for help. I like this to a <ahref="https://notnull.space/@paul/statuses/01JHZK2YK2JXJ49NW1V1CV2Y1F">Fediverse post</a> and wait for responses. It doesn't take long.</p>
<h3id="switching-the-subnet">Switching the subnet</h3>
<p><code>@fab</code> responds with the idea of using a /48 prefix instead of /64.
Given what I have learned up to this point, this could make sense - the gateway is in a /48 of the same first 48 bits of my /64 subnet (<code>2a03:4000:AAAA</code>)
after all, so why not put my IPv6 in that same subnet. It <em>could</em> cause issues with communication with other servers in different /64 subnets within the same /48 subnet, but let's risk it for now.</p>
<p>Unfortunately, I experience the same issues again here. After about 10 minutes the connectivity drops off. Reboot, OK, wait, gone.</p>
<h3id="learn-german">Learn German</h3>
<p>Maybe it's because I'm a native English speaker that I didn't do this before, but I
thought I'd look more in depth at the Netcup forums and found <ahref="https://forum.netcup.de/administration-of-a-server-vserver/vserver-server-kvm-server/p166679-freebsd-12-1-probleme-ipv6/#post166679">this thread by a German speaker having the same issues</a>. Fortunately, in browser translate was able to translate it for me and I was able to get the gist of what was going on.<br/>
<ahref="https://forum.netcup.de/administration-of-a-server-vserver/vserver-server-kvm-server/p166679-freebsd-12-1-probleme-ipv6/#post166679">This post in particular</a> was interesting, along with <ahref="https://forum.netcup.de/administration-of-a-server-vserver/vserver-server-kvm-server/p166782-freebsd-12-1-probleme-ipv6/#post166782">this post by the same author</a> - they suggest two things I've tried before - using a /48 prefix instead of /64, and setting a route for the actual gateway address (not the <code>fe80::1</code> address), but I didn't try them both at the same time!</p>
<h2id="let-s-try-those-together">Let's try those together!</h2>
<p>I'll get my default gateway via <code>traceroute6</code> again</p>
<p>Reboot. It's working. Wait. It's still working! Wait a little longer. Still working!<br/>
This is progress! <code>ping6</code>s and <code>curl</code>s are working great!<br/>
Oh, incoming connections aren't so good...</p>
<predata-lang="sh"style="background-color:#212121;color:#eeffff;"class="language-sh "><codeclass="language-sh"data-lang="sh"><spanstyle="font-style:italic;color:#4a4a4a;"># From another host
<p>So <code>ping6</code>s go out OK, but nothing is coming in.</p>
<h3id="switching-not-routing">Switching, not routing</h3>
<p>As previously mentioned, Netcup switches their IPv6 addresses, they don't route them.
I'm not going to pretend I know exactly what this means, but it feels to me that it's
likely the gateway could "switch" periodically, and in normal circumstances the <code>fe80::1</code> can be aware of this and cater for it, just not for the BSDs.</p>
<h4id="i-wonder">I wonder...</h4>
<p>Maybe... I could try switching my default route to another gateway IP? <ahref="https://forum.netcup.de/administration-of-a-server-vserver/vserver-server-kvm-server/p166679-freebsd-12-1-probleme-ipv6/#post166679">This Post on the Netcup thread</a> seems to suggest there could be a number of <code>2a03:4000:AAAA::X</code> gateways that could be routers...</p>
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=1308 ttl=48 time=5.62 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=1309 ttl=48 time=4.81 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=1310 ttl=48 time=5.04 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=1311 ttl=48 time=4.80 ms
</span><spanstyle="color:#82aaff;">...
</span></code></pre>
<p>PROGRESS!</p>
<h3id="all-good-things">All Good things...</h3>
<p>At this point I'm excited. I switch my <code>rc.conf</code> to use that different gateway. I haven't rebooted yet but I see no reason to just yet. Right now I'm at about <code>icmp_seq=5000</code> on my outgoing <code>ping6</code>s and incoming are still successful. But... on my remote server...</p>
<predata-lang="sh"style="background-color:#212121;color:#eeffff;"class="language-sh "><codeclass="language-sh"data-lang="sh"><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=4504 ttl=48 time=4.86 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=4505 ttl=48 time=4.93 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=4506 ttl=48 time=4.81 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=4507 ttl=48 time=4.87 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=6323 ttl=48 time=5.42 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=6324 ttl=48 time=4.72 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=6325 ttl=48 time=5.14 ms
</span><spanstyle="color:#82aaff;">64 bytes from 2a03:4000:AAAA:BBBB:cccc:dddd:eeee:ffff: icmp_seq=6326 ttl=48 time=4.94 ms
</span><spanstyle="color:#82aaff;">...
</span></code></pre>
<p>OK, so I can dynamically bring connectivity back if/when it drops. I can probably script this in some way to happen automatically too. Is it perfect? No, but it's something.</p>
<h3id="request-for-comments">Request for Comments</h3>
<p>This is where I am right now. I have a sort-of almost stable IPv6 connection that works most of the time, but it's nowhere near ideal. I am continuing my research and I will update this post if/when I discover anything that stablises it completely.<br/>
In the meantime, I welcome any comments about things that I could do further to help, or, indeed, comments for reasons why I should not have done any of the above. I'm familiar with networks, but with IPv6 being ignored in the UK my knowledge of it is a little lacking.</p>
<p>If you want to send any comments, please do so either on <ahref="https://forums.freebsd.org/threads/netcup-ipv6-intermittant.96489/#post-686757">this FreeBSD Forum thread</a> or on <ahref="https://notnull.space/@paul/statuses/01JHZK2YK2JXJ49NW1V1CV2Y1F">this Fediverse thread</a>.</p>
<p>EDIT: I guess I should have found <ahref="https://forums.freebsd.org/threads/how-to-set-ipv6-gateway.68568/post-408655">this post on the FreeBSD forums</a> earlier, which alludes to the same thing.</p>
