From allocation to doubling IPv6 traffic at de-cix in 58 days. Florian Obser Hostserver GmbH

Transcription

1 From allocation to doubling IPv6 traffic at de-cix in 58 days Florian Obser Hostserver GmbH

2 Prolog

3 Was ist IPv6?

4 Was ist ein Netzwerk?

5 Was ist ein Computer?

6 Was ist ein User?

7 Wer bin ich?

8 Jack of all trades

9 layer 2 layer 8 support

10 Hostserver GmbH

11 Programmierer

12 Perl

13 Java

14 PHP, Lisp, Erlang, Python...

15 Administrator

16 Linux

17 Web / Mail / DB Server

18 OpenBSD

19 Router / Firewall / Loadbalancer

20 : sixxs home

21 : sixxs office

22 : sixxs datacenter

23

24 Erfahrungsbericht

25 kein Howto

26 : Rechenzentrumsumzug^W Erweiterung unserer Hostingfläche

27 Interxion Fra5

28 DE-CIX Access Switch steht nebenan

29 : your circuit has been delivered.

30 : (größerer) Transitprovider - wir möchten mit euch peeren

31 + free IPv6 transit (!)

32

33 : de-cix session zu route-servern

34 de-cix: IPv6?

35 Wir: ähm... hm... nö

36 : weitere peerings Wie, ihr habt kein v6? Wird aber Zeit! 1!11!

37 Wie geht das?

38 Ripe klicken [1] [2] [1] [2]

39 % IPv6 First Allocation Request Form % RIPE NCC members can use this form to request their first IPv6 Allocation. % Please see % for instructions on how to complete this form. #[GENERAL INFORMATION]# % % Please add your RegID. request-type: ipv6-first-alloc form-version: 1.1 X-ncc-regid:?!? [...]

40 raten

41 % IPv6 First Allocation Request Form % RIPE NCC members can use this form to request their first IPv6 Allocation. % Please see % for instructions on how to complete this form. #[GENERAL INFORMATION]# % % Please add your RegID. request-type: ipv6-first-alloc form-version: 1.1 X-ncc-regid: de.hostserver [...]

42 [...] #[REQUESTER TEMPLATE]# % % Please add your contact details. name: phone: fax-no: nic-hdl: [...]

43 $ whois -T inetnum

44 [...] #[REQUESTER TEMPLATE]# % % Please add your contact details. name: Marcus Schaefer phone: fax-no: ripe@hostserver.de nic-hdl: MS1 [...]

45 [...] #[REQUIRED INFORMATION]# % % Do you accept the IPv6 Address Allocation and Assignment % Policy? (Yes/No) Confirmation: % If you have any online information about your future % IPv6 services, please add the URL below. website: [...]

46 [...] #[REQUIRED INFORMATION]# % % Do you accept the IPv6 Address Allocation and Assignment % Policy? (Yes/No) confirmation: yes % If you have any online information about your future % IPv6 services, please add the URL below. website: [...]

47 [...] #[REQUIRED INFORMATION]# % % Do you accept the IPv6 Address Allocation and Assignment % Policy? (Yes/No) confirmation: yes % If you have any online information about your future % IPv6 services, please add the URL below. website: [...]

48 [...] #[OVERVIEW OF ORGANISATION TEMPLATE]# % % Please add a short description of your organisation. org-description: % If your organisation has IPv6 allocations from any of the Regional % Internet Registries (RIR), please list the ranges below. other-allocation: % Will the whole organisation use the requested allocation? If % another part of the organisation will request separate IPv6 % address space from any RIR, please inform us below. (Whole/Part) for-whole-or-part-of-the-organisation: [...]

49 [...] #[OVERVIEW OF ORGANISATION TEMPLATE]# % % Please add a short description of your organisation. org-description: Hostserver GmbH is an HostingProvider (ISP). We will make IPv6 connectivity available to all our managed dedicated server and webspace customers % If your organisation has IPv6 allocations from any of the Regional % Internet Registries (RIR), please list the ranges below. other-allocation: % Will the whole organisation use the requested allocation? If % another part of the organisation will request separate IPv6 % address space from any RIR, please inform us below. (Whole/Part) for-whole-or-part-of-the-organisation: [...]

50 [...] #[OVERVIEW OF ORGANISATION TEMPLATE]# % % Please add a short description of your organisation. org-description: Hostserver GmbH is an HostingProvider (ISP). We will make IPv6 connectivity available to all our managed dedicated server and webspace customers % If your organisation has IPv6 allocations from any of the Regional % Internet Registries (RIR), please list the ranges below. other-allocation: none % Will the whole organisation use the requested allocation? If % another part of the organisation will request separate IPv6 % address space from any RIR, please inform us below. (Whole/Part) for-whole-or-part-of-the-organisation: [...]

51 [...] #[OVERVIEW OF ORGANISATION TEMPLATE]# % % Please add a short description of your organisation. org-description: Hostserver GmbH is an HostingProvider (ISP). We will make IPv6 connectivity available to all our managed dedicated server and webspace customers % If your organisation has IPv6 allocations from any of the Regional % Internet Registries (RIR), please list the ranges below. other-allocation: none % Will the whole organisation use the requested allocation? If % another part of the organisation will request separate IPv6 % address space from any RIR, please inform us below. (Whole/Part) for-whole-or-part-of-the-organisation: whole [...]

52 #[IPv6 ALLOCATION USAGE PLAN]# % % When will you use this address space? % % Subnet Within Within % size (/nn) 3 months 1 year Within 2 years Purpose

53 #[IPv6 ALLOCATION USAGE PLAN]# % % When will you use this address space? % % Subnet Within Within Within % size (/nn) 3 months 1 year 2 years Purpose Subnet: /48 Subnet: /48 Subnet: /39 x x - x - shared hosting ( 256 /56s) infrastructure ( 256 /56s) server customers ( 512 /48s)

54 #[DATABASE TEMPLATE(S)]# % % Please complete all of the fields below. % % You can find more information on how to complete these fields % in the supporting notes (ipv6-first-alloc-support.html). inet6num: netname: descr: country: org: admin-c: <leave empty> <leave empty> <add LIR organisation name> <add country code> <add org-id> <add nic-hdl of administrative contact> tech-c: <add nic-hdl of technical contact> status: mnt-by: mnt-lower: mnt-routes: notify: changed: source: ALLOCATED-BY-RIR RIPE-NCC-HM-MNT <add mntner name> <add mntner name> <add address> hostmaster@ripe.net RIPE

55 $ whois -T inetnum

56 #[DATABASE TEMPLATE(S)]# % % Please complete all of the fields below. % % You can find more information on how to complete these fields % in the supporting notes (ipv6-first-alloc-support.html). inet6num: netname: descr: country: Org: admin-c: admin-c: tech-c: tech-c: status: mnt-by: mnt-lower: mnt-routes: notify: changed: source: <leave empty> <leave empty> Hostserver GmbH DE ORG-HG5-RIPE MS1 RH1 FO89-RIPE SB5901-RIPE ALLOCATED-BY-RIR RIPE-NCC-HM-MNT HOSTSERVER-MNT HOSTSERVER-MNT noc@hostserver.de hostmaster@ripe.net RIPE

57 #[INSERT SUPPLEMENTAL COMMENTS]# % % Please add more information if you have specific addressing needs. <add more information> #[END of REQUEST]#

58 #[INSERT SUPPLEMENTAL COMMENTS]# % % Please add more information if you have specific addressing needs. We will make IPv6 connectivity available first to our infrastructure (1 /56) and shared hosting customers (1 /56 for each shared server). In the next phase dedicated server customers ( 1-n servers ) who ask for IPv6 connectivity will be provided with a /48. At this stage all new customers will be provided with a /48. In the last phase all remaining dedicated server customers without IPv6 connectivity will be asked / persuaded to use IPv6 connectivity and provided with a /48 #[END of REQUEST]#

59 fertig

60 : IPv6 prefix 2a00:15a8::/32 allocated to de.hostserver

61 ripe db update

62 route6 Objekt anlegen (Testsystem)

63 "Whois is experiencing problems, please try your query again in a few minutes." $ whois -h whois-test.ripe.net [...] Timeout

64 To:

65 Antwort: works for me

66 !

67 *clickety click*

68 *debug*

69 Aha!

70 $ ifconfig fgrep inet6 inet6 addr: 2a01:198:2fb:3:46:4c4f:5249:414e/64 Scope:Global

71 $ whois -h FO1-TEST % ************************************************** % This is TEST version of the v3 RIPE database % The objects are in RPSL format. [...]

72 Ripe signature: ============================================================ Visit the one-stop website that explains everything you need to know about IPv6. ============================================================

73 Update live whois db + Netz announcen

74 1. fall-out: bgpmon.net

75 $ ping6 -c1 2a00:15a8::1 PING 2a00:15a8::1(2a00:15a8::1) 56 data bytes 64 bytes from 2a00:15a8::1: icmp_seq=1 ttl=59 time=56.2 ms --- 2a00:15a8::1 ping statistics --1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = /56.237/56.237/0.000 ms

76 Yay!

77

78 Und jetzt?

79 ...

80 RFC 1925

81 The Twelve Networking Truths

82 (4) Some things in life can never be fully appreciated nor understood unless experienced firsthand. Some things in networking can never be fully understood by someone who neither builds commercial networking equipment nor runs an operational network.

83 : ipv6only.org

84 2 hosts

85 Firewall: hastur.ipv6only.org

86 Webserver: azathoth.ipv6only.org

87

88 OpenBSD Firewall / Loadbalancer

89 Firewall (pf): IPv6 kein Sonderfall

90 pass in on outside proto tcp from any to $azathoth \ port { ssh, http, rsync }

91 + link local multicast

92 pass proto icmp6 to FF02::/16

93 Loadbalancer (relayd): It just works

94 Redundante FW/LB: carp

95 It just works.

96 OpenBSD + IPv6 als (redundante) Firewall / Loadbalancer funktioniert

97 Btw. OpenBGPd: It just works

98 gelegentlich Patches für IPv6 corner cases in pf / OpenBGPd

99 Aber: IPvShit

100 Ifconfig(8): -inet6 Disable inet6(4) on the given interface and remove all configured inet6(4) addresses, including the link-local ones. To turn it on again, assign any inet6 address or run rtsol(8).

101 IpvShit (2)

102 redundante Borderrouter anbinden

103 OSPF6d (OSPF v3)

104 In 4.7 in base

105 Noch nicht getestet

106 einfache Setups funktionieren (angeblich)

107 RFC 2740 (OSPF for IPv6) OSPF packets are sent using the interface's associated link-local unicast address as source. A router learns the link-local addresses of all otherrouters attached to its links, and uses these addresses as next hopinformation during packet forwarding.

108 Adressen müssen gespeichert werden

109 RFC 3493 struct in6_addr { uint8_t s6_addr[16]; }; /* IPv6 address */

110 Link-Local: 128bit + Interface ID ( fe80::223:54ff:fede:d06%eth0 )

111 Ups

112 Kame hack: s6_addr[2] == 0 && s6_addr[3] == 0 (fe80:0000:0000:0000:0223:54ff:fede:0d06%eth0)

113 Interface dort kodieren

114 sollte man nicht an Kernel geben

115 ( Claudio Jeker: ospf6d the new kid on the routing block )

116

117 4 Nameserver

118 Nur über IPv6 erreichbar

119 ;; ANSWER SECTION: ipv6only.org. ipv6only.org. ipv6only.org. ipv6only.org IN IN IN IN NS NS NS NS a.ns.ipv6only.org. b.ns.ipv6only.org. c.ns.ipv6only.org. d.ns.ipv6only.org. ;; ADDITIONAL SECTION: a.ns.ipv6only.org b.ns.ipv6only.org c.ns.ipv6only.org d.ns.ipv6only.org IN IN IN IN AAAA AAAA AAAA AAAA 2a00:15a8::a:53 2a00:15a8:6::b:53 2a00:15a8::c:53 2a00:15a8:6::d:53

120 hastur: OpenBSD 4.6, nsd package

121 azathoth: nsd aus sourcen bind 9.5.1debian package

122 argus (zusätzlicher Host mit v6 connectivity): OpenBSD 4.6 bind P2

123 It just works

124 # apt-get install pdns-server

125 dig

126 SERVFAIL

127 Ok, keine Zonen konfiguriert

128 pdns als slave für Bind?

129 documentation, anyone?

130 srsly!

131 # apt-get remove purge pdns-server

132 Mysql a-24+lenny2 -bind-address +#bind-address = =

133 azathoth:~# mysql -h azathoth.ipv6only.org -u v6test -p ERROR 2005 (HY000): Unknown MySQL server host 'azathoth.ipv6only.org' (4) Tcpdump > : A?azathoth.ipv6only.org. (52) IP > : NXDomain 0/1/0 (114)

134 Postgres lenny1

135 $ psql -h azathoth.ipv6only.org -U v6test v6test Password for user v6test: Welcome to psql 8.3.9, the PostgreSQL interactive terminal. Type: \copyright for distribution terms \h for help with SQL commands \? for help with psql commands \g or terminate with semicolon to execute query \q to quit v6test=>

136 use DBI; my $dbname my $dbuser my $dbpass my $dbhost my $dbport = = = = = 'v6test'; 'v6test'; 'XXX'; 'azathoth.ipv6only.org'; 5432; my $dsn = "DBI:Pg:database=$dbname;host=$dbhost;port=$dbport";

137 It just works!

138 apache2

139 ssh / rsync

140 collectd

141 ejabberd

142 It just works

143 Servicemonitoring

144 Nagios

145 noch nicht ausprobiert

146 ping sollte funktionieren

147 eigene Checkscripts schreiben?

148 nrpe / send_nsca?

149 Überwachung zur Zeit:

150 jabber

151 Fazit

152 Soweit so gut...

153 Standard Services

154 OK

155 DNS am problematischsten

156 Sixxs distributed traceroute ( )

157 47% können nicht resolven

158 IPv6 + DNSSEC

159 Oh oh

160 Kein reassemble Ipv6 UDP in OpenBSD

161 dlv.isc.org: nope

162 Ripe signierte reverse v6 Zone: nope

163 Hahahahahaha!

164 Ripe fixt das

165 Next: Addressvergabe

166 Note to self: Jens Link, 15:45, Track A

167

168 doubling IPv6 traffic at de-cix

169

170 Was?

171 The 26th Chaos Communication Congress ( )

172 Videostreams dumpen

173 Nach h264 wandeln

174 Über IPv6 verteilen

175 http / rsync

176 Warum?

177 Because I can.

178 Wie?

179 ca. 23:00: Idee

180 Zwischen den Jahren

181 Viel Ersatzhardware im RZ

182

183 Debian installieren

184 Scheduling

185 ical

186 XML

187 Xcal

188 Fahrplan/

189 xcal!

190 Perl Parser

191 DJB daemontools

192 cronjobs

193 mplayer

194 mencoder

195 h264enc

196 /usr/bin/mencoder "$f_name.dump" -o "$f_name.avi" -ofps 25 -vf softskip,harddup -oac mp3lame -lameopts abr:br=140:aq=4:vol=2.2:mode=1 -ovc x264 \ -x264encopts bitrate=512:me=dia:dct_decimate:nointerlaced:no8x8dct:\ nofast_pskip:trellis=0:partitions=p8x8,b8x8,i4x4:nomixed_refs:keyint=250: keyint_min=25:frameref=2:bframes=6:b_adapt=1:b_pyramid:noweight_b:\ direct_pred=spatial:subq=4:nochroma_me:cabac:deblock:level_idc=41:\ threads=auto:ssim:psnr

197 While true in screen

198 rsync

199 Viel Klebeband

200

201 :00: fertig!

202 6.5h schlafen bis zur Keynote!

203 :15: streamdump läuft automatisch an.

204 :32: Keynote Videos fertig

205 180 min nach Vortragsende

206 keine Hotfixes an Scripten

207 500 LOC

208 Traffic monitoring

209 fangen wir mal mit 25 mbit an...

210 :38: 26c3 wiki Eintrag

211 title=streaming&diff=2942&oldid=2929 Unofficial Encodes 26c3.ipv6only.org rsync://26c3.ipv6only.org/26c3 Note: these are unofficial uncut h264 encodes. the encodes start 15 minutes before the event and stop 20 minutes after the event. the note regarding the uc3 streamdumps applies here as well: "they are ugly uncut raw dumps, nothing worth keeping after 26C3" the process is completely automated and encodes currently show up ~ 180 minutes after the event if you can't resolve 26c3.ipv6only.org: you need a resolver capable of resolving over ipv6, there are no ipv4 nameservers. why: because

212 :15: 1. sms

213 40 mbit/s

214 sms

215 100 mbit + ack

216

217

218

219 2. Tag ( )

220 [x] ok

221 Encoder holt auf

222 Videos nach ca. 100 min

223 Tag 3 ( )

224 erste Offizielle FEM Videos

225 Video [3669] Hacker Jeopardy defekt

226 Alternative Streamdumps?

227 Extern erreichbare Server im Congressnet?

228 nope

229 Irgendwo extern 5GB+ dumps

230 = Datei rotiert wenn mplayer crasht

231 srsly?

232 uc3

233 sinnvolle Dumps

234 ab externer Mirror bei tu-berlin

235 FOR EXTERNAL USERS (Dragons Everywhere!) we have a mirror at - it was promised to have 1GBit and enough CPU to handle all the Dragons out there...

236 5 kbyte/s

237

238 Bedarf an Streamdumps

239 ja

240 aber nicht über IPv6

241 kurze 300 mbit+ Peaks

242 meist nur mbit/s

243 Keine Lastprobleme

244 Backbone und Aussenanbindung kann mehr

245

246 sflow: 1.2 TB traffic in 6 Tagen

247 Nicht vollständig

248 während 26c3 entwickelt

249 (designstudie, partitioning + inet type in Postgres)

250 sflow collector im ibgp

251 OpenBGPD

252 AS lookup

253 :16 bis :00

254 24 ASe

255 http + rsync traffic [GB] source_as src_port comment CONGRESS-AS CONGRESS-AS HURRICANE probably EASYNET probably EASYNET probably MNET-AS probably 6to4 & HE-tunnel sixxs tunnel sixxs tunnel sixxs tunnel

256 http traffic [GB] source_as src_port comment CONGRESS-AS EASYNET NL-CONCEPTS this is probably the sixxs http proxy service ( lame :P ), or the sixxs noc were having a dragons everywhere party ;)

257 ~ 3TB http Traffic in 6 Tagen

258 Geschätzte 5TB+ Traffic in 6 Tagen

259

260 27c3

261 Scripte verbessern

262 Wir sind wieder dabei

263 Vielleicht auch mit legacy IPv4

264 There's probably no live, Now stop worrying and enjoy your Internet.

265 Fragen?