PTTMetro - PTT.br The Brazilian Metropolitan IXP Project

Transcription

1 PTTMetro - PTT.br The Brazilian Metropolitan IXP Project LINX 78 21st August 2012 Eduardo Ascenço Reis <eascenco@nic.br> Antonio Galvao de Rezende Filho <galvao@nic.br> PTT.br Engineering Team <eng@ptt.br> 1

2 CGI.br NIC.br - CEPTRO.br CGI.br The Brazilian Internet Steering Committee (CGI.br) purpose is for coordinating and integrating all Internet service initiatives in Brazil, as well as promoting technical quality, innovation and the dissemination of the services available. The CGI.br is comprised of members from the government, the corporate sector, the third sector and the academic community, and as such constitutes a unique Internet governance model for the effective participation of society in decisions involving network implementation, management and use. NIC.br The Brazilian Network Information Center (NIC.br) is a non-profit civil entity that implements the decisions and projects designed by CGI.br. CEPTRO.br The Center of Study and Research in Network Technology and Operations (CEPTRO.br) is responsible for services and projects that aim to improve the quality of Internet in Brazil, working more specifically with Internet infrastructure. PTTMetro/PTT.br is part of CEPTRO.br activities. 2

3 Reference Name IXP - Internet exchange Point PTT Ponto de Troca de Tráfego 3

4 Brazilian PTT History and Internet Key Points Year Location (City/State) Event 1989 São Paulo / SP.br TLD Activation 1991 São Paulo / SP Brazil First Connection to Internet via FAPESP (Dawn of Brazil Internet) 1996 São Paulo / SP PTT ANSP/FAPESP Operation Start 1997 São Paulo / SP Registro.br Operation Start (Brazil NIR) 2000 Porto Alegre / RS PTT RSIX Operation Start (RNP UFRGS) 2000 São Paulo / SP PTT OPTiX-LA Operation Start (Optiglobe Inc, now Tivit) 2002 Curitiba / PR PTT PriX Operation Start (RNP UFPR) 2002 Brasilia / DF PTT FIX Operation Start (RNP) 2004 São Paulo / SP PTTMetro (CGI.br) 2004 Rio de Janeiro / RJ PTTMetro (CGI.br) 2004 Brasilia / DF PTT FIX move to PTTMetro (CGI.br) 2004 Barueri / SP PTT ANSP/FAPESP moved to NAP do Brasil (Terremark Latin America) 2005 Curitiba / PR PTT PRIX moved to PTTMetro (CGI.br) From Peering in Brazil LACNIC XI / Salvador, BA Brazil - Interconnection Schemes and Strategies for ISPs Tutorial 4

5 Brazilian PTT History and Internet Key Points Year Location (City/State) Event 1989 São Paulo / SP.br TLD Activation 1991 São Paulo / SP Brazil First Connection to Internet via FAPESP (Dawn of Brazil Internet) 1996 São Paulo / SP PTT ANSP/FAPESP Operation Start 1997 São Paulo / SP Registro.br Operation Start (Brazil NIR) 2000 Porto Alegre / RS PTT RSIX Operation Start (RNP UFRGS) 2000 São Paulo / SP PTT OPTiX-LA Operation Start (Optiglobe Inc, now Tivit) 2002 Curitiba / PR PTT PriX Operation Start (RNP UFPR) 2002 Brasilia / DF PTT FIX Operation Start (RNP) 2004 São Paulo / SP PTTMetro (CGI.br) 2004 Rio de Janeiro / RJ PTTMetro (CGI.br) 2004 Brasilia / DF PTT FIX move to PTTMetro (CGI.br) 2004 Barueri / SP PTT ANSP/FAPESP moved to NAP do Brasil (Terremark Latin America) 2005 Curitiba / PR PTT PRIX moved to PTTMetro (CGI.br) From Peering in Brazil LACNIC XI / Salvador, BA Brazil - Interconnection Schemes and Strategies for ISPs Tutorial 5

6 PTT Close Proliferation Problem (e.g. São Paulo city in the past) - 1/3 City Area PTT Optix-LA PTT ANSP/FAPEP PTT Diveo PTT Telcomp Customer AS Too many Connections High Cost From Peering in Brazil LACNIC XI / Salvador, BA Brazil - Interconnection Schemes and Strategies for ISPs Tutorial 6

7 PTT Close Proliferation Problem (e.g. São Paulo city in the past) - 2/3 City Area PTT Optix-LA PTT ANSP/FAPEP PTT Diveo PTT Telcomp? Customer AS From Peering in Brazil LACNIC XI / Salvador, BA Brazil - Interconnection Schemes and Strategies for ISPs Tutorial 7

8 PTT Close Proliferation Problem (e.g. São Paulo city in the past) - 3/3 City Area PTT Optix-LA PTT ANSP/FAPEP PTT Diveo PTT Telcomp Customer AS IP Transit ISP NSP From Peering in Brazil LACNIC XI / Salvador, BA Brazil - Interconnection Schemes and Strategies for ISPs Tutorial 8

9 PTTMetro Brazilian Metropolitan IXP Project PTTMetro (PTT- Internet Exchange Point) is the project of the Brazilian Internet Steering Committee (CGI.br) that provides the necessary infrastructure for the direct interconnection between the diverse networks that operate in a metropolitan region From Peering in Brazil LACNIC XI / Salvador, BA Brazil - Interconnection Schemes and Strategies for ISPs Tutorial 9

10 PTTMetro PTT.br Project # Operation PTTMetro CGI.br project started in 2004 and is still active. PTT.br Operation division responsible to deploy and maintain PTTMetro locations. PTT.br has a partnership to support its operation with entities that host PIX on all locations. Not complete list of PIX's hosts types: National Education and Research Network (RNP) Academic (e.g. Universities) Information Technology Government Companies (e.g. Procempa, Prodest, Prodepa) Internet Data Centers (IDC) Internet Service Providers (ISP) Internet Network Providers (NSP) 10

11 PTTMetro/PTT.br 2004 Start in 3 Locations Initial Locations: Brasília, Rio de Janeiro and São Paulo 11

12 PTTMetro/PTT.br August Locations in Operation 1. Americana 2. Belém 3. Belo Horizonte 4. Brasília 5. Caxias do Sul 6. Campina Grande 7. Campinas 8. Curitiba 9. Florianópolis 10. Fortaleza 11. Goiânia 12. Londrina 13. Natal 14. Porto Alegre 15. Recife 16. Rio de Janeiro 17. Salvador 18. São José dos Campos 19. São Paulo 20. Vitória 12

13 PTTMetro/PTT.br IXP Locations and Brazilian AS Distribution 1. Americana 2. Belém 3. Belo Horizonte 4. Brasília 5. Caxias do Sul 6. Campina Grande 7. Campinas 8. Curitiba 9. Florianópolis 10. Fortaleza 11. Goiânia 12. Londrina 13. Natal 14. Porto Alegre 15. Recife 16. Rio de Janeiro 17. Salvador 18. São José dos Campos 19. São Paulo 20. Vitória Brazilian Autonomous Systems (AS) ~1700 States: 26 + Federal District Population: ~194 M PTT.br IXP Locations 13

14 PTTMetro/PTT.br IXP Locations All PTTMetro Locations are Isolated The main purpose for PTTMetro locations is to allow the local traffic to stay in the location. The Brazilian Internet Steering Committee do not intend to interconnect PTTMetro Locations and compete with telecommunications companies. PTTMetro project stimulates and support inter locations transport services done by participants companies. PTTMetro have defined some transport modes in order to optimize common resources and reduce costs that are already in production (e.g. Campinas <=> São Paulo by Fasternet company) 14

15 PTTMetro/PTT.br Proliferation of IXPs Satellite Model Small IX Small IX Small IX Local IX (e.g. Americana) Medium IX Concentrator IX (e.g. Campinas) Large IX National IX (e.g. São Paulo) 15

16 PTTMetro/PTT.br Participants AS per Location Sao Paulo Campinas Florianopolis Sao Jose dos Campos Belem Porto Alegre Rio de Janeiro Goiania Brasília Vitoria Curitiba Belo Horizonte Campina Grande Americana Natal # Participants AS Total: Unique: Salvador Londrina Fortaleza Recife Caxias do Sul Values from Aug

17 CGI.br NIC.br PTT.br - Financial Model Brazilians (persons or companies) R$ Domains / IP / ASN Registration.br Return Benefits Help Improve Internet Authoritative Name Systems DNS.br NIC.br divisions / activities Registro.br CERT.br Cetic.br Ceptro.br PTT.br / IPv6.br/ SIMET.nic.br / etc 17

18 PTTMetro/PTT.br - Financial Model HighLights PTTMetro / PTT.br costs are sustained by CGI.br / NIC.br, despite Autonomous Systems (AS) participants needs for ports (number and capacity). Each participant AS has only costs to reach one or more Interconnection Points (PIX) of a specific PTT.br location in order to access its peering fabric. There is no startup and recurrent fees for AS participants at all PTT.br locations. 18

19 PTTMetro/PTT.br Operational Responsibilities NIC.br PTT.br is responsible for: Installation, Management, Administration, Monitoring and Supporting all IX locations. Always keeping Neutrality and High Quality as main targets. Highlight: PTT.br is responsible to provide enough capacity to sustain bandwidth demand on connection between PIX: (e.g. 1GE, 10GE, nx 10GE with DWDM). PIX - Interconnection Point Each entity that hosts a PTT.br PIX is responsible to provide continuous and high quality: IDC Infrastructure Rack Space, Electrical Energy, Refrigeration, Physical Security, Operational Hand, etc Dark Fiber Pair (preferably redundant) to Central PIX 19

20 PTTMetro/PTT.br - Regional Metro Ethernet Non-Blocking Switch/Peering Fabric AS A Each PTT.br Location Infrastructure Router IXP PIX A AS D Router PIX D PIX Central PIX B Router AS B PIX C Router PIX: Interconnection Point AS C 20

21 PTTMetro/PTT.br High Availability Network Topology Model PIX Central C1 PIX Central C2 Core Access PIX Remote PIX: Interconnection Point Dark Fiber Pair 21

22 PTTMetro/PTT.br São Paulo Location PIX Topology 22

23 PTTMetro/PTT.br São Paulo Location PIX Distribution Central / Core PIX C1 in operation Remote / Access PIX 23

24 PTTMetro/PTT.br Peering Agreements Multi-Lateral Peering Agreements (MLPA) IP (v4/v6) traffic exchange between all participants. BGP sessions established between participants and IXP route servers. Different MLPA VLANs for IPv4 and IPv6. Bilateral Peering Agreements (BPA) Shared VLAN Mode IP traffic exchange only between participants. Use the same MLPA VLANs. Dedicated VLAN Mode IP traffic exchange only between participants (e.g. Internet Transit). Use a dedicate VLAN for each peering agreement. 24

25 PTTMetro/PTT.br Reference Model Fundamental Definition IP (v4 v6) AS A AS B 25

26 PTTMetro/PTT.br Basic Operational Model Allowed Traffic AS A MAC A MAC B AS B IP (v4 v6) PTT.br Location Peering Fabric Allowed Frames with Ethertypes: 0x IPv4 0x ARP 0x86dd - IPv6 26

27 PTTMetro/PTT.br Extended Operational Model Inter AS Traffic Allowed Multiple Connections to a Location for Redundancy and/or Load Balancing AS A MAC A MAC B AS B MAC C AS C PTT.br Location Peering Fabric AS A MAC A' 27

28 PTTMetro/PTT.br Extended Operational Model Intra AS Traffic Denied Multiple Connections to a Location for Internal Traffic (local loop) AS A MAC A MAC B AS B MAC C AS C PTT.br Location Peering Fabric AS A MAC A' 28

29 PTTMetro/PTT.br Extended Operational Model Multiple AS per Single Port Shared L2 structure outside PTT.br operation domain AS A MAC A NSP Transport Provider AS D PTT.br Location Peering Fabric MLPA MAC D 29

30 PTTMetro/PTT.br Extended Operational Model Multiple AS per Single Port Use Different VLAN Tags for Logical Isolation (IEEE 802.1Q) AS A MAC A VLAN A VLAN A NSP Transport Provider PTT.br Location Peering Fabric MLPA VLAN C VLAN B AS D MAC D VLAN B 30

31 PTTMetro/PTT.br Redundant Authoritative DNS Servers Service Anycast Copies at 14 PTT.br IX Locations Root:.BR cctld: L.root-servers.net *.DNS.BR 31

32 PTTMetro/PTT.br Economical Summary PTTMetro - Economically Interests for Telecommunication Companies Reduction of Transit and Interconnection Costs Some Brazilian Access ISP reported to exchange around 30-40% of their Internet traffic at Sao Paulo PTT.br. Last/First Mile Provisioning IP Services Interconnection Using dedicated VLANs to provide L2 logical isolation for: Internet Transit (IPv4 and IPv6), Backup, Storage, VoIP, etc Inter Locations Transport Interconnection Points (PIX) Hosting Neutral Facilities for the Brazilian Telecommunications Agency Regulated Interconnections 32

33 PTTMetro/PTT.br All Locations Aggregated Exchanged Traffic Growth Terabytes Au g/ Oc t / De c/ Fe b/ Ap r/ Ju n/ Au g/ Oc t/ De c /2 00 Fe b/ Ap r/ Ju n/ Au g/ Oc t/ De c/ Fe b/ Ap r/ Ju n / Au g/ Oc t/ De c/ Fe b/ Ap r / Ju n/ Au g/ Oc t/ De c/2 Month Fe b/ Ap r/ Ju n/ 20 12

34 PTTMetro/PTT.br All Locations Aggregated Exchanged Traffic 34

35 PTTMetro/PTT.br London 2012 Olympic Games Broadcast Traffic London 2012 Internet Broadcast Transmission Sample at Porto Alegre - RS PTT.br Location 35

36 PTTMetro/PTT.br PTT Forum 6 São Paulo, Brazil The 6th PTT Fórum event will take place in São Paulo, Brazil on December 03 and 04, PTT Fórum normal audience is composed by Brazilian Autonomous Systems (AS) and PTTMetro/PTT.br IX participants (some of them are international companies). Last year the 4th PTT Fórum had around 400 participants. The event will be translated between Portuguese and English. The 6th PTT Forum event will take place in São Paulo, Brazil on December 3 and 4, 2012 and it will be part of the Brazilian Internet Infrastructure week together with an IPv6 Forum, ISOC ION Conference and GTER/GTS (Network Engineering, Operation and Security Working Groups). 36

37 PTTMetro/PTT.br Thanks & Contact Thanks Obrigado 37