Supernode
[wikipedia-article] i don't like that word
quick'n dirty required software:
1) [fastd] mesh-node-vpn
2) [vpn-key-upload] todo: remove overhead (apache,ruby,sinatra ...), kiss principle] not documented in here (todo)
3) [batman-adv]
4) [tinc] mesh-backbone-vpn
5) OS: debian wheezy (or whatever you like)
load batman-adv kernel-module while booting
in /etc/modules.conf add
batman-adv
disable pmtu-discovery in /etc/sysctl.conf add
net.ipv4.ip_no_pmtu_disc=1
fastd
in /etc/apt/apt/sources.list add
deb http://repo.universe-factory.net/debian/ sid main
keyimport
gpg --keyserver pgpkeys.mit.edu --recv-key AB7A88C5B89033D8 gpg -a --export AB7A88C5B89033D8 | sudo apt-key add -
install fastd
apt-get update apt-get install fastd
fastd configuration
/etc/fastd/kbu# ls backbone fastd.conf peers root@fastd4:/etc/fastd/kbu# ls backbone/ fastd1 fastd2 fastd3 fastd4 root@fastd4:/etc/fastd/kbu# cat backbone/* key "4f856d95bd596ac7724edca73a19e6e9d142b374df27166bb1a78e58785efc59"; remote ipv4 "fastd1.kbu.freifunk.net" port 10000; key "e1916b66c4f8a795e217877cf72607d952e796463c7024dd9a6a47ae2929bc10"; remote ipv4 "fastd2.kbu.freifunk.net" port 10000; key "d56181dfe9b5ac7cfe68a94c0ce406322a9924286a751673da0dcb28ad5218b0"; remote ipv4 "fastd3.kbu.freifunk.net" port 10000; key "9b3f65f99963343e2785c8c4fad65e70b73ee7e1205d63bd84f3e2decb53e621"; remote ipv4 "fastd1.kbu.freifunk.net" port 10000;
[fastd-backbone] for more
fastd.conf
# Log everything to a log file log to "/var/log/fastd.log" level debug; # Set the interface name interface "fastd"; # Support xsalsa20 and aes128 encryption methods, prefer xsalsa20 method "xsalsa20-poly1305"; method "aes128-gcm"; method "null"; #todo: WTF, encryption is off!?, what do we exchange keys for? # Bind to a fixed port, IPv4 only bind 0.0.0.0:10000; # Secret key generated by `fastd --generate-key` # in this case you have to ask the holy guru of that project for the private key' secret "won't tell my secret here"; # Set the interface MTU for TAP mode with xsalsa20/aes128 over IPv4 with a base MTU of 1492 (PPPoE) # (see MTU selection documentation) mtu 1426; # Include peers from the directory 'peers' include peers from "/etc/fastd/kbu/backbone"; include peers from "/etc/fastd/kbu/peers"; #ipadress, routing #on establish "/root/scripts/fastdup.sh"; #does'not work for me, need to invoke an external script somewhere else
fastd-ifup-script,
#/bin/sh #bring the interface up /sbin/ifconfig fastd up #start bat0 interface /usr/sbin/batctl if add fastd #dhcp forward through batman /usr/sbin/batctl gw_mode server /sbin/ifconfig bat0 172.27.XX.1 netmask 255.255.192.0 up #routing /sbin/ip rule add from 172.27.0.0/18 table ffkbu /sbin/ip route add 172.27.0.0/18 dev bat0 table ffkbu /sbin/ip route flush cache
backbone-net (tinc)
apt-get install tinc
root@fastd4:/etc/tinc/backbone# ls bbkeys hosts -> bbkeys tinc.conf tinc-up
public-keys for the tinc-backbone, can be found [here]
add your public key to that repo
tinc.conf
/etc/tinc/backbone# cat tinc.conf Name=fastd4 Device=/dev/net/tun Mode=router ConnectTo=paula ConnectTo=paul #enough links?
tinc-up
#!/bin/sh ifconfig $INTERFACE 172.27.255.X netmask 255.255.255.0 up # #set some backbone related routes ip route add 172.27.255.0/24 dev backbone table ffkbu ip route add default dev backbone table ffkbu ip route flush cache
routing
we need to set up policy based routing.
echo "200 ffkbu" >> /etc/iproute2/rt_tables
this is done in the tinc-up and the fastd-up scripts
ip rule ls
ip rule ls 0: from all lookup local 32765: from 172.27.0.0/18 lookup ffkbu 32766: from all lookup main 32767: from all lookup default
ip route list table ffkbu
ip route list table ffkbu default dev backbone scope link 172.27.0.0/18 dev bat0 scope link 172.27.255.0/24 dev backbone scope link
dhcpd.conf
subnet 172.27.0.0 netmask 255.255.192.0 {
#if guru talks about subnets,
#just the range is meant!
#all nodes,supernodes,and clients
#remain in the net 172.27.0.0/18
range 172.27.XXX.XXX 172.27.XXX.XXX;
option domain-name-servers 172.27.XXX.1;
option domain-name "kbu.freifunk.net";
option routers 172.27.XXX.1;
option broadcast-address 172.27.63.255;
default-lease-time 600;
max-lease-time 7200;
}
you need to install a bind9
fastd
UDP-Queue Größe
Durch Broadcasts können können auf Supernodes kurzzeitig hohe Lastspitzen enstehen. In der aktuellen batman-adv-Version wird jedes Broadcast-Paket zudem 3x auf jedem Link versendet. Die vergleichweise Hohe Datenrate von 100MBit/s zwischen Supernodes bewirkt darüber hinaus, dass broadcast-Pakete mit 100MBit/s eingehen, während sie an alle Nodes gesendet werden müssen.
Linux-Distributionen sehen per Default Queue-Größen im Bereich von 128 KB vor (http://www.cyberciti.biz/faq/linux-tcp-tuning/). Supernodes sind somit nicht in der Lage entsprechende Lastspitzen zu puffern und während Idle-Times zu versenden. Läuft die UDP-Queue über, so loggt fastd:
2013-03-30 12:32:01 +0100 --- Warning: sendmsg: Resource temporarily unavailable 2013-03-30 12:32:01 +0100 --- Warning: sendmsg: Resource temporarily unavailable 2013-03-30 12:32:01 +0100 --- Warning: sendmsg: Resource temporarily unavailable 2013-03-30 12:32:01 +0100 --- Warning: sendmsg: Resource temporarily unavailable 2013-03-30 12:32:01 +0100 --- Warning: sendmsg: Resource temporarily unavailable
Jeder Log-Eintrag entspricht einem Paket, dass nicht enqueued werden konnte und damit verworfen wurde. Die Queue-Größen können via systctl angepasst werden (http://wwwx.cs.unc.edu/~sparkst/howto/network_tuning.php). Auf fastd2 wird zur Zeit verwendet:
#/etc/sysctl.conf net.core.rmem_max=838860800 net.core.wmem_max=838860800 net.core.rmem_default=83886080 net.core.wmem_default=83886080
Das System verfügt dadurch über 80MB Speicher für Queues. Pro Verbindung stehen 8MB zur Verfügung. Die konfigurierten 80MB reichen aus, um die 100MBit/s Verbindung über mehrere Sekunden auszulasten und Lastspitzen abzufangen.