THE ROAD TO IPV6: KU SERVICE EXPERIENCES ON DUAL-STACK The Design and Implementation of a Scalable, Dual-Stack Oriented, Consolidated Authentication System Pirawat WATANAPONGSE * Surasak SANGUANPONG* Kasom KOTH-ARSA* Surachai CHITPINITYON* Office of Computer Services, Kasetsart University * Applied Network Research Laboratory, Department of Computer Engineering, Kasetsart University TIP2013, Honolulu, Hawaii January 15 th 2013
2 Agenda Introducing The Kasetsart University Network (NontriNet) Tug-of-War: Providers Want Control Users Want Result IPv6 Is Designed To Co-Habit, Not Compatible Dual-Stack: Hardware Is Easy, Service Is Hard IPv6 Case Study: 1 st Obstacle One Car, Two License- Plate Numbers!! IPv6 Case Study: 2 nd Obstacle Who Are You? IPv6 Case Study: 3 rd Obstacle You Are Driving Which Car? To Where? IPv6 Case Study: Solution Integrated Authentication IPv6 Statistics Report
3 Next Topic Introducing The Kasetsart University Network (NontriNet)
Kasetsart University (KU) Established in 1943 5 Campuses ~74,000 students ~10,500 professors and staffs University Network NontriNet Average traffic 1.2 Gbps inbound 480 Mbps outbound Peak traffic 2.2 Gbps inbound 1.1 Gbps outbound Daily Active User ~25,000 users Registered MAC addresses ~170,000 MACs
5 NontriNet: Network Infrastructure Internet APAN 30 G 622 M ThaiSARN Uninet 1 G NontriNet 3x1 G 10 G KPS 10 G 10 G SRC CSC SPN 1 G 1 G 1 G 2 M Firewall & Shaper 6 M 10 G 100 M OCS 128 K 512 K 512 K Rachaburi Nopparat Satit School 1 G 1 G 1 G 1 G 1 G 1 G President Office 1 G Library Engineering Science
6 Next Topic Tug-of-War: Providers Want Control Users Want Result
7 Requirements Thailand Computer Crime Act B.E. 2550 Logging of users and their Internet usage IPv6 Phase 1: Native IPv4 only Phase 2: Native IPv4 and native IPv6 Phase 3: Native IPv6 and NAT64 IPv4 Phase 4: Native IPv6 only
8 KU IPvX Resources IPv4: 158.108.0.0/16 IPv6: 2001:3C8:1303::/48 AS Number: 9411 Gateway OCS LIB President Office ENG SCI
9 Next Topic IPv6 Is Designed To Co-Habit, Not Compatible
10 KU IPv6 Address Space Design The 1 st Nibble : Campus The 2 nd Nibble : Faculty 1: Bangkhen 1: Computer Center 2001:3c8:1303:1164 :----:/64 The Last Byte : VLAN VLAN 100
11 KU IPv6 Address Space Allocation NontriNet 2001:3C8:1303::/48 Campuses Bangkhen Bangkhen Kamphaeng Saen Si Racha Sakon Nakhon Reserved Reserved Reserved Reserved Sub-Network 2001:3C8:1303:0000::/52 2001:3C8:1303:1000::/52 2001:3C8:1303:2000::/52 2001:3C8:1303:3000::/52 2001:3C8:1303:4000::/52 2001:3C8:1303:5000::/52 2001:3C8:1303:6000::/52 2001:3C8:1303:7000::/52 2001:3C8:1303:8000::/49
12 Next Topic Dual-Stack: Hardware Is Easy, Service Is Hard
13 KU IPv6 Preparation Physical Infrastructure Native, Dual-Stack Wireline and Wireless Also allows Tunneling Infrastructure AKA Low-Level (Discrete) Services DNS, DHCP, LDAP, NTP, Syslog, Etc. In-House Integrated (Dual-Stack Aware) Services Stateful, Scalable, Load-Balancing, Parallel-Track Firewalls Dual-Stack Aware Authentication System Regulation-Compliance, Dual-Stack Aware Traffic Logger Basic Higher-Level Services Web Servers, Mail Servers, Database Servers, Etc.
14 Next Topic IPv6 Case Study: 1 st Obstacle One Car, Two License- Plate Numbers!!
15 Address-Binding Scenario 1 158.108.181.62 (from DHCP) 2001:3c8:1303:1266::7afb:dc4d (from DHCPv6) NontriNet Backbone Gateway Internet Client 2 158.108.181.62 158.108.181.62 2001:3c8:1303:1266::7afb:dc4d 3 Auto-Binding and Authentication Authentication Server
16 Problem: Confusion with Dual-Stack Authentication Facts: Dual-Stack means one user gets 2 addresses: one IPv4 and one IPv6 Classical Authentication is done per unique IP Address Consequences: Users need to authenticate once for IPv4, then again for IPv6 The 2 nd authentication causes a lot of confusion (Is it the re-authentication of my IPv4, or is it the authentication for the IPv6 side?)
17 Solution: Automatic Address-Binding Binds IPv6 and IPv4 addresses of the same user together. Tracks the IPv4-IPv6 pair activities Authentication of IPv6 will automatically authenticate the corresponding IPv4 (and vice versa) Limitations Support on a per IPv6-IPv4 pair basis User logging-in to 2 machines simultaneously still has to do 2 separate authentications. Address-Binding must be done before authentication Cannot authenticate IPv6 alone and then bind-in IPv4 address later on
18 Next Topic IPv6 Case Study: 2 nd Obstacle Who Are You?
19 Login Servers Implementations 12 virtual servers from 2 physical machines 3 DNS RR entries for each server loginx.ku.ac.th A and AAAA record loginx-v4.ku.ac.th A record only loginx-v6.ku.ac.th AAAA record only login1 - - - - - login12
20 Dual-Stack Aware Login Server The embedded pictures (showing both addresses) helps bind IPv4 and IPv6 addresses together
21 Login Servers Binding Tricks Function : Binding user s IPv4 and IPv6 addresses at login time Use two embedded pictures (CGI generated) to discover client s addresses From loginx-v4.ku.ac.th to discover the client s IPv4 From loginx-v6.ku.ac.th to discover the client s IPv6
22 Dual-Stack Aware Login Session Manager Developed in-house Tracks both IPv6 and IPv4 of the same session Tracks all sessions of the same user
Per-user Login and Quota Info 23
24 Miscellaneous Features No-Typing Login for mobile phones Currently support only Android-Based phones iphone version forthcoming
25 Next Topic IPv6 Case Study: 3 rd Obstacle You Are Driving Which Car? To Where?
26 Load-Balancing, Parallel-Track Firewall Architecture Login session manager Internal network Core Router IPv4 firewall cluster IPv6 firewall Gateway Router Internet Login servers Login servers Login servers Login servers
27 Load-Balancing, Parallel-Track Firewall Two Parallel-Running Firewall Sub-Systems 1 x Linux Firewall for IPv6 AMD Opteron 1220 (Dual-core, 2.8 GHz) 3 GB RAM Linux 2.6 with ip6tables + ipset (with IPv6 support) Bridge mode Will adopt load-balancing paradigm later on Load-Balancing, 4 x Linux Firewalls for IPv4 Intel Xeon X5720 (Dual-Core, 3.5 GHz) 4 GB RAM Linux 2.6 with iptables + ipset Bridge mode
28 IPv6 Traffic Logging KU In-House IPv6-Aware Log Server Developed in-house Capable of decoding IPv6 packets Also support tunneling decoding Timestamp Source Port Destination Port 20120802101100 118.173.x - 158.108.x - 54180 80 www.ku.ac.th/newdesign/ne... 20120802101100 158.108.x - 183.111.x - 33310 80 appdown.naver.com/naver/a... 20120802101100 158.108.x - 61.19.x - 58893 80 photos-c.ak.fbcdn.net/hphot... 20120802101100-2001:3c8:1303:x - 2a03:2880:x 49296 80 www.facebook.com/ Source IP Destination IP URL
29 Next Topic IPv6 Case Study: Solution Integrated Authentication
30 Integrated Authentication and Traffic Logger Authentication Server Firewall Farm Login Server Farm 30 TB Storage Regulation- Compliance Traffic Logger Syslog Server
31 Next Topic IPv6 Statistics Report
32 Data Collection Period First day of Data Collection Last day of Data Collection 209 Days June 6, 2012 World IPv6 Day Data Collection Period December 31, 2012
33 KU IPv6 Throughput Statistics 7.9/2.4 Mb/s Average (in/out) 157/148 Mb/s Peak (in/out)
34 KU Login Statistics Item Value Period: 209 days Jun 6, 2012 Dec 31, 2012 Number of login (times) 18,544,813 Number of IPv6 login 3,469,431 (~18.7%) Number of unique user 101,418 Number of unique IPv4 43,918 [1] Number of unique IPv6 2,547,645 [2] [1] Excluding the registered servers which are not required to login [2] Due to RFC 3041 Privacy Extensions for SLAAC in IPv6
Unique Users Login with IPv6 35
Unique IPv4 Addresses with IPv6 Binding Statistics 36
37 IPv6 HTTP Request Statistics Item Value Period : 209 Days Jun 6, 2012 Dec 31, 2012 Number of URL request entries 293,491,784 Number of unique host names 1,516,862 Number of unique domain names 9,454 Number of unique Thai host names 720 (~0.05%) Number of unique Thai domain names 61 (~0.65%)
38 Top 10 Domains (by number of requests) Rank Domain # request % 1 facebook.com 168,375,568 57.37% 2 fbcdn.net 65,683,814 22.38% 3 ytimg.com 9,451,950 3.22% 4 gstatic.com 6,176,805 2.10% 5 google.com 5,412,069 1.84% 6 google.co.th 5,403,384 1.84% 7 s.youtube.com 3,836,982 1.31% 8 upic.me 3,580,503 1.22% 9 google-analytics.com 2,899,702 0.99% 10 wikimedia.org 2,726,718 0.93% ~93.20%
39 Top 10.th Domains (by number of requests) Rank Domain # request % 1 google.co.th 5,403,384 85.79% 2 ku.ac.th* 471,929* 7.49%* 3 tmd.go.th 210,679 3.34% 4 3bb.co.th 36,897 0.59% 5 moph.go.th 31,983 0.51% 6 kku.ac.th 26,496 0.42% 7 lru.ac.th 14,991 0.24% 8 faceblog.in.th 13,433 0.21% 9 rmutto.ac.th 11,839 0.19% 10 sipa.or.th 7,715 0.12%
40 Thank You For Your Attention