By Robert Seastrom
Here's a quick set of notes on setting up an AS112 node based on our experience using FreeBSD 7.1-RELEASE, OpenBGPD 4.4.1 (built from ports), and BIND9.6.1-P1 (also built from ports).
You don't need a lot of resources to host an AS112 node. Our particular instance gets about 1000 queries/second and runs as a 256mb guest (30gb "hard drive") under VMware on a 1.8 GHz quad core core AMD64 system. The guest "vmware-vmx" process consistently uses less than 20% of a single core.
These notes make the assumption that your default gateway is a BGP speaker (not necessarily with full routes), and that routes which you announce to it will somehow make their way out to the Internet.
Steps to get an AS112 node up and running are as follows.
1) Install FreeBSD in the usual way. Be sure to install ports (which is where you'll be getting OpenBGPD and BIND from). Leave enough extra space to be able to compile a new kernel if you decide you'd like to have pf host-based packet filtering and firewalling enabled (optional). If you're running under a virtualization scheme, you will want to take reasonable steps to optimize for it - in our case, there is a recommendation to add "kern.hz=100" to /boot/loader.conf, which we did.
2) Configure your network. We'll cut to the chase here and show you our entire /etc/rc.conf
root@as112 [8] # cat /etc/rc.conf
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/rc.conf.
defaultrouter="192.148.253.1"
hostname="as112.cluetrust.net"
ifconfig_em0="inet 192.148.253.6 netmask 255.255.255.192"
ifconfig_em0_alias0="inet 192.175.48.1 netmask 255.255.255.0"
ifconfig_em0_alias1="inet 192.175.48.6 netmask 255.255.255.0"
ifconfig_em0_alias2="inet 192.175.48.42 netmask 255.255.255.0"
sshd_enable="YES"
openbgpd_enable="YES"
# openbgpd_flags=""
named_enable="YES"
named_flags="-u named -t /var/named"
root@as112 [9] #
Of course you will want to edit your hostname, defaultrouter, and ifconfig_em0 to reflect local reality and likely change the NIC type for all ifconfig_* if you don't happen to have an Intel Pro/1000.
Leave the ifconfig_em0_alias* lines alone - these are the service addresses for the AS112 nameservers and should not be changed.
You'll want to configure whatever your preferred local cacheing resolvers are in /etc/resolv.conf. While you'll be running a nameserver on this machine, it will be authoritative only, non-recursive.
3) Once you have network connectivity, you need to update ports. This is not optional as there are security vulnerabilities in the version of BIND that comes with FreeBSD.
root@as112 [10] # portsnap fetch
[software indicates progress and success]
root@as112 [11] # portsnap update
[software indicates progress and success]
4) Now you're ready to build some binaries.
root@as112 [12] # cd /usr/ports/net/openbgpd
root@as112 [13] # make install
At the Options screen, accept the defaults for your OpenBGPD build.
root@as112 [14] # cd /usr/ports/dns/bind96
root@as112 [15] # make install
At the Options screen, the only change you will want to make is to add REPLACE_BASE, in order to overwrite the old version of BIND that came with FreeBSD.
5) A tarball of appropriate files for /etc/namedb/ (named.conf, zone files for the zones we'll be serving, etc) can be found here . Download the file, cd to /etc/namedb, tar xvfpz as112-namedb.tar.gz. Now edit /etc/namedb/master/db.hostname.as112.net and put in appropriate site-local information (free-form TXT records).
6) Configuring OpenBGPD - Here's our bgpd.conf which you may edit to taste:
root@as112 [15] # cat /usr/local/etc/bgpd.conf
# as112 project
AS 112
router-id 192.148.253.6
fib-update no
network 192.175.48.0/24
neighbor 192.148.253.1 {
descr "ClueTrust"
remote-as 16668
}
root@as112 [16] #
Of course you will want to edit the router-id and neighbor statements to match your network.
Start OpenBGPD:
root@as112 [16] # /usr/local/etc/rc.d/openbgpd start
On your default router, you will want to configure route-maps on your BGP session that (a) send no routes to OpenBGPD as that is pointless and just wastes memory, (b) accepts routes from OpenBGPD and sets a transit community or does whatever else is necessary for you to announce 192.175.48.0/24 to your upstreams. Now would be a good time to update your IRR AS-SET, send in a prefix list change template, or do whatever else you do to change your upstreams' filter lists for your route. If your upstream runs a looking glass, once these changes are in effect you should be able to see your announcement of 192.175.48.0/24 with an AS path of "YOURAS 112 i" (though it may not be chosen as the best path).
7) Now would be a good time to reboot and see if everything comes up OK. You should be able to do queries against your local nameserver and get authoritative NXDOMAIN responses:
root@as112 [13] # dig @127.0.0.1 -x 192.168.10.1
; <<>> DiG 9.6.1-P1 <<>> @127.0.0.1 -x 192.168.10.1
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 37254
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;1.10.168.192.in-addr.arpa. IN PTR
;; AUTHORITY SECTION:
168.192.in-addr.arpa. 300 IN SOA prisoner.iana.org. hostm
aster.root-servers.org. 2002040800 1800 900 604800 604800
;; Query time: 3 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Sep 9 02:19:10 2009
;; MSG SIZE rcvd: 120
root@as112 [14] #
and REFUSED responses when you try to query for someting for which your nameserver is not authoritative:
root@as112 [14] # dig @127.0.0.1 example.com. a
; <<>> DiG 9.6.1-P1 <<>> @127.0.0.1 example.com. a
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 52413
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;example.com. IN A
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Sep 9 02:20:30 2009
;; MSG SIZE rcvd: 34
root@as112 [15] #
Briefly, you should be sending authoritative NXDOMAIN for all RFC1918 address space, and serving a few TXT records for hostname.as112.net.
8) An in-depth PF tutorial is beyond the scope of this document, however should you elect to do this you must remember to let udp/53 and tcp/53 (DNS, and here BOTH tcp and udp are necessary) through, as well as tcp/179 from your default router for the BGP session and tcp/22 (ssh) from the locations from which you will be managing your system.
9) Mini-tutorial (enough to get you in trouble) for pf follows. I like _The Book of PF_ by Peter Hansteen (ISBN 1-59327-165-4). At $29.95 retail it's a cheap resource if the pages at http://www.openbsd.org/faq/pf/ don't serve your needs.
The GENERIC FreeBSD kernel doesn't provide pf. You'll want to create a new configuration file (perhaps called AS112) in the config directory with the following contents:
include GENERIC
# stuff to make pf happy
options ALTQ
options ALTQ_CBQ
options ALTQ_RED
options ALTQ_RIO
options ALTQ_HFSC
options ALTQ_CDNR
options ALTQ_PRIQ
device pf
Now build and install a new kernel based on this. If the new config file is called AS112, add these lines to /etc/make.conf:
KERNCONF=AS112
INSTALLKERNEL=AS112
and then build & install the new kernel & reboot:
cd /usr/src
make buildkernel
make installkernel
Add the following line:
pf_enable="YES"
to your /etc/rc.conf and reboot to start the new kernel.
Here's a sample pf.conf file that has worked well here:
root@as112 [118] # cat /etc/pf.conf
maint = "{ 192.0.2.224/28, 192.0.2.64/26 }"
as112 = "{ 192.175.48.0/24 }"
unicastaddr = "192.148.253.6"
defaultrouter = "192.148.253.1"
extif = "em0"
pass in quick on $extif inet proto udp from any to any port 53 no state
pass in quick on $extif inet proto tcp from any to any port 53 no state
pass in quick on $extif inet proto tcp from $maint to ($extif) port 22 n
o state
pass in quick on $extif inet proto tcp from $defaultrouter to $unicastad
dr port 179 no state
pass in quick on $extif inet proto icmp from any to any no state
block in quick on $extif inet from any to any
pass out quick on $extif inet proto tcp from 192.148.253.6 port 179 to 1
92.148.253.1 port 179 no state
pass out quick on $extif inet proto icmp from any to any no state
pass out quick on $extif inet proto udp from $as112 port 53 to any no st
ate
pass out quick on $extif inet proto tcp from $as112 port 53 to any no st
ate
pass out quick keep state
root@as112 [119] #
As with /etc/rc.conf you will need to change maint, unicastaddr, and defaultrouter (as well as perhaps extif) to reflect your site's local realities.
To reload pf.conf after you've modified it, pfctl -f /etc/pf.conf
Happy Traffic Sinking