High Performance Active End-toend Network Monitoring
|
|
- Darrell Thompson
- 8 years ago
- Views:
Transcription
1 High Performance Active End-toend Network Monitoring Les Cottrell, Connie Logg, Warren Matthews, Jiri Navratil, Ajay Tirumala SLAC Prepared for the 1 st SCAMPI Workshop, Amsterdam, January Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM), by the SciDAC base program, and also supported by IUPAP 1
2 Outline High performance testbed Challenges for measurements at high speeds Infrastructure for regular high-performance measurements 2
3 Testbed 6 cpu servers 4 disk servers T OC192/POS (10Gbits/s) G S R 12 cpu servers 4 disk servers Sunnyvale 2.5Gbits/s 6 cpu servers
4 Problems: Achievable TCP throughput Typically use iperf Want to measure stable throughput (i.e. after slow start) Slow start takes quite long at high BW*RTT GE for RTT from California to Geneva (RTT=182ms) slow start takes ~ 5s So for slow start to contribute < 10% to throughput measured need to run for 50s About double for Vegas/FAST TCP So developing Quick Iperf Use web100 to tell when out of slow start Measure for 1 second afterwards 90% reduction in duration and bandwidth used 4
5 Examples 24ms RTT 140ms RTT 5
6 Problems: Achievable bandwidth Typically use packet pair dispersion or packet size techniques (e.g. pchar, pipechar, pathload, pathchirp, ) In our experience current implementations fail for > 155Mbits/s and/or take a long time to make a measurement Developed a simple practical packet pair tool ABwE Typically uses 40 packets, tested up to 950Mbits/s Low impact Few seconds for measurement (can use for real-time monitoring) 6
7 Screen shot of real time ABwE display tool for monitoring at SC2002 Measurements 1 minute separation ABwE Results Note every hour sudden dip in available bandwidth 7
8 Problem: File copy applications Some tools (e.g. bbcp will not allow a large enough window currently limited to 2MBytes) Same slow start problem as iperf Need big file to assure not cached E.g. 2GBytes, at 200 Mbits/s takes 80s to transfer, even longer at lower speeds Looking at whether can get same effect as a big file but with a small (64MByte) file, by playing with commit Many more factors involved, e.g. adds file system, disks speeds, RAID etc. Maybe best bet is to let the user measure it for us. 8
9 Passive (Netflow) Measurements Use Netflow measurements from border router Netflow records time, duration, bytes, packets etc./flow Calculate throughput from Bytes/duration Validate vs. iperf, bbcp etc. No extra load on network, provides other SLAC & remote hosts & applications, ~ 10-20K flows/day, unique pairs/day Tricky to aggregate all flows for single application call Look for flows with fixed triplet (sce & dst addr, and port) Starting at the same time secs, ending at roughly same time - needs tuning missing some delayed flows Check works for known active flows To ID application need a fixed server port (bbcp peer-to-peer but have modified to support) Investigating differences with tcpdump Aggregate throughputs, note number of flows/streams 9
10 Mbits/s Iperf SLAC to Caltech (Feb-Mar 02) Active + Passive Passive Passive vs active 0 Date Iperf matches well BBftp reports under what it achieves Active Bbftp SLAC to Caltech (Feb-Mar 02) Mbits/s Active + Passive Date 10
11 Passive bbftp from SLAC to IN2P3 (unrelated to active measurements) TCP fair share results in the green flow (60 streams) getting twice the throughput of magenta flow (30 streams) when both run simultaneously Adding the flows together we see we can get about 80Mbits/s. 11
12 Problems: Host configuration Need fast interface and hispeed Internet connection Need powerful enough host Need large enough available TCP windows Need enough memory Need enough disk space 12
13 Windows and Streams Well accepted that multiple streams and/or big windows are important to achieve optimal throughput Can be unfriendly to others Optimum windows & streams changes with changes in path, hard to optimize For 3Gbits/s and 200ms RTT need a 75MByte window 13
14 Even with big windows (1MB) still need multiple streams with stock TCP ANL, Caltech & RAL reach a knee (between 2 and 24 streams) above this gain in throughput slow Above knee performance still improves slowly, maybe due to squeezing out others and taking more than fair share due to large number of streams 14
15 Configurations 1/2 Do we measure with standard parameters, or do we measure with optimal? Need to measure all to understand effects of parameters, configurations: Windows, streams, txqueuelen, TCP stack, MTU Lot of variables Examples of 2 TCP stacks FAST TCP no longer needs multiple streams, this is a major simplification (reduces # variables by 1) Stock TCP, 1500B MTU 65ms RTT FAST TCP, 1500B MTU FAST 65ms TCP, RTT 1500B MTU 65ms RTT 15
16 Configurations: Jumbo frames Become more important at higher speeds: Reduce interrupts to CPU and packets to process Similar effect to using multiple streams (Hacker) Jumbo can achieve >95% utilization SNV to CHI with 1 or multiple stream up to Gbit/s Factor 5 improvement over 1500B MTU throughput for stock TCP Alternative to a new stack 16
17 Repetitive long term measurements 17
18 IEPM-BW = PingER NG Driven by data replication needs of HENP, PPDG, DataGrid No longer ship plane/truck loads of data Latency is poor Now ship all data by network (TB/day today, double each year) Complements PingER, but for high performance nets Build an infrastructure to make E2E network (e.g. iperf, packet pair dispersion) & application (FTP) measurements for high-performance A&R networking Started SC
19 Tasks Develop/deploy a simple, robust ssh based E2E app & net measurement and management infrastructure for making regular measurements Major step is setting up collaborations, getting trust, accounts/passwords Can use dedicated or shared hosts, located at borders or with real applications COTS hardware & OS (Linux or Solaris) simplifies application integration Integrate base set of measurement tools (ping, iperf, bbcp ), provide simple (cron) scheduling Develop data extraction, reduction, analysis, reporting, simple forecasting & archiving 19
20 Purposes Compare & validate tools With one another (pipechar vs pathload vs iperf or bbcp vs bbftp vs GridFTP vs Tsunami) With passive measurements, With web100 Evaluate TCP stacks (FAST, Sylvain, HS TCP, Frank Kelley ) Trouble shooting Set expectations, planning Understand requirements for high performance, jumbos performance issues, in network, OS, cpu, disk/file system etc. Provide public access to results for people & applications 20
21 Measurement Sites Production, i.e. choose own remote hosts, run monitor themselves: SLAC (40) San Francisco, FNAL (2) Chicago, INFN (4) Milan, NIKHEF (32) Amsterdam, APAN Japan (4) Evaluating toolkit: Internet 2 (Michigan), Manchester University, UCL, Univ. Michigan, GA Tech (5) Also demonstrated at: igrid2002, SC2002 Using on Caltech / SLAC / DataTag / Teragrid / StarLight / SURFnet testbed If all goes well minutes to install monitoring host, often problems with keys, disk space, ports blocked, not registered in DNS, need for web access, disk space SLAC monitoring over 40 sites in 9 countries 21
22 22 SNV SLAC CHI ESnet NY NERSC LANL ORNL TRIUMF KEK Abilene SLAC SNV FNAL ANL NIKHEF CERN IN2P3 CERN Caltech SDSC BNL JAnet HSTN SEA ATL CLV RAL UCL UManc DL NNW NY UTDallas UMich I2 SOX UFL APAN RIKEN INFN-Roma INFN-Milan CESnet APAN Geant Stanford CalREN Rice ORN JLAB GARR CAnet Surfnet Stanford Renater IPLS UIUC 140 Monitor 00Mbps E
23 Results Time series data, scatter plots, histograms CPU utilization required (MHz/Mbits/s) jumbo and standard, new stacks Forecasting Diurnal behavior characterization Disk throughput as function of OS, file system, caching Correlations with passive, web100 23
24 24
25 Excel 25
26 Problem Detection Must be lots of people working on this? Our approach is: Rolling averages if have recent data Diurnal changes 26
27 Rolling Averages EWMA~Avg of last 5 points +- 2% 27
28 Fit to α*sin(t+φ)+γ Indicate diurnalness by δγ, can look at previous week at same time, if do not have recent measurements 28
29 Alarms Too much to keep track of Rather not wait for complaints Automated Alarms Rolling average à la RIPE-TTM 29
30 30
31 31
32 Action However concern is generated Look for changes in traceroute Compare tools Compare common routes Cross reference other alarms 32
33 Next steps Rewrite (again) based on experiences Improved ability to add new tools to measurement engine and integrate into extraction, analysis GridFTP, tsunami, UDPMon, pathload Improved robustness, error diagnosis, management Need improved scheduling Want to look at other security mechanisms 33
34 More Information IEPM/PingER home site: www-iepm.slac.stanford.edu/ IEPM-BW site www-iepm.slac.stanford.edu/bw Quick Iperf ABwE Submitted to PAM
35 Passive vs Active correlations Strong 35
36 IEPM-BW Uses/deliverables Understand and identify resources needed to achieve high throughput performance for Grid and other data intensive applications Provide access to archival and near real-time data and results for eyeballs and applications: planning and expectation setting, see effects of upgrades assist in trouble-shooting problems by identifying what is impacted, time and magnitude of changes and anomalies as input for application steering (e.g. data grid bulk data transfer), changing configuration parameters for forecasting and further analysis Identify critical changes in performance, record and notify administrators and/or users Provide a platform for evaluating new network tools (e.g. pathrate, pathload, GridFTP, INCITE, UDPmon ) Provide measurement/analysis/reporting suite for Grid & hi-perf sites 36
37 IEPM-BW Deployment
38 38 SNV SLAC CHI ESnet NY Stanford CalREN NERSC LANL JLAB TRIUMF KEK Abilene SLAC SNV FNAL ANL NIKHEF CERN IN2P3 CERN CALTECH SDSC BNL JAnet HSTN SEA ATL CLV IPLS RAL UCL UManc DL NNW NY Rice UTDallas NCSA UMich I2 SOX UFL APAN RIKEN INFN-Roma INFN-Milan CESnet APAN Geant EDG PPDG/GriPhyN Monitoring Site
39 Early results Reasonable estimates of throughput achievable with 10 sec iperf meas. Multiple streams and big windows are critical Improve over default by 5 to 60. There is an optimum windows*streams Continuous data at 90 min intervals from SLAC to 33 hosts in 8 countries since Dec 01 39
40 Early results 1MHz ~ 1Mbps Bbcp mem to mem tracks iperf BBFTP & bbcp disk to disk tracks iperf until disk performance limits Bandwidth estimators fail above 100Mbits/s High throughput affects RTT for others E.g. to Europe adds ~ 100ms Archival raw throughput data & graphs already available via http 40
41 File copy disk-to-disk E.g. Iperf vs file copy disk to disk 100 Fast Ethernet OC3 Disk limited 0 Iperf TCP Mbits/s 400 Over 60Mbits/s iperf >> file copy 41
42 Disk performance It matters for the applications Depends on: disk sub-system file system (nfs, ufs, /tmp), caching, can be cached for long time, can improve throughput by factor of 10 or more Read speed varies from 4MB/s to 230MB/s measured for 30 remote hosts (depends on caching) Uncached write speeds vary from 1MByte/s to 29MBytes/s If disk speed < network speed, no need to measure network, so need parallelizing of disks & servers 42
43 Pipechar min throughpt Mbits/s E.g. iperf vs pipechar Iperf TCP Mbits/s 400 Working with Developer. Not using TCP. Pipechar disagrees badly above 100Mbits/s (6 hosts, 20%), 50% of hosts have reasonable agreement Typical of several bw prediction tools. 43
44 Web100 vs Iperf throughputs Nice sanity check is to see if Web100 sees the same throughput as iperf reports Web100 throughput = Streams*DataBytesOut *8 / (max(iperfstreamdur)) / 10^6 Looks like good agreement 44
45 Web100 estimated vs observed throughput EstBW~C*Streams*MSS/(RTT*sqrt(loss)) (Mathis et. al. & Hacker) Measure retransmissions using Web100, loss~pktsretrans/pktsout Note high degree of correlation (R^2 >0.8) If window large enough ((>=128KB) then ~ common relation, window threshold varies from link to link 45
46 Effect of saturation If keep below knee then RTT stays low Could also use congestion signals Could use to optimize, e.g. throttle back app 46
47 Forecasting Given access to the data one can do real-time forecasting for TCP bandwidth, file transfer/copy throughput E.g. NWS, Predicting the Performance of Wide Area Data Transfers by Vazhkudai, Schopf & Foster Developing simple prototype using average of previous measurements Validate predictions versus observations Get better estimates to adapt frequency of active measurements & reduce impact Also use ping RTTs and route information Look at need for diurnal corrections Use for steering applications Working with NWS for more sophisticated forecasting Can also use on demand bandwidth estimators (e.g. pipechar, but need to know range of applicability) 47
48 Forecast results Predict=Moving average of last 5 measurements +- σ Iperf TCP throughput SLAC to Wisconsin, Jan 02 Mbits/s 100 Observed Predicted 60 % average error = average(abs(observe-predict)/observe) x 33 nodes Average % error Iperf TCP 13% +- 11% Bbcp mem 23% +- 18% Bbcp disk 15% +-13% bbftp 14% +-12% pipechar 13% +-8% 48
49 Impact on Others Make ping measurements with & without iperf loading Loss loaded(unloaded) RTT Looking at how to avoid impact: e.g. QBSS/LBE, application pacing, control loop on stdev(rtt) reducing streams, want to avoid scheduling 49
50 Possible HEP usage of QBSS Apply priority to lower volume interactive voice/video-conferencing and real time control Apply QBSS to high volume data replication Leave the rest as Best Effort Since 40-65% of bytes to/from SLAC come from a single application, we have modified to enable setting of TOS bits Need to identify bottlenecks and implement QBSS there Bottlenecks tend to be at edges so hope to try with a few HEP sites 50
51 Experiences Getting ssh accounts and resources on remote hosts Tremendous variation in account procedures from site to site, takes up to 7 weeks, requires knowing somebody who cares, sites are becoming increasingly circumspect Steep learning curve on ssh, different versions Getting disk space for file copies (100s Mbytes) Diversity of OSs, userids, directory structures, where to find perl, iperf..., contacts Required database to track Also anonymizes hostnames, tracks code versions, whether to execute command (e.g. no ping if site blocks ping) & with what options, Developed tools to download software and to check remote configurations Remote server (e.g. iperf) crashes Start & kill server remotely for each measurement Commands lock up or never end: Time out all commands Hung processes need to be killed (or else fill up sockets with CLOSE-WAIT) Some commands (e.g. pipechar) take a long time, others (disk throughput, pathrate) have results that change infrequently so have different schedules AFS tokens to allow access to.ssh identity timed out, used trscron Protocol port blocking Ssh following Xmas attacks; bbftp, iperf ports, big variation between sites Wrote analyses to recognize and track problems and work with site contacts Ongoing issue, especially with increasing need for security, and since we want to measure inside firewalls close to real applications 51
52 Next steps Develop/extend management, analysis, reporting, navigating tools improve robustness, manageability, workaround ssh anomalies Get improved forecasters (NWS) and quantify how they work, provide tools to access Optimize intervals (using forecasts, and lighter weight measurements) and durations Evaluate self rate limiting application (bbcp), look at using Web100 for feedback loop Extend analysis of passive Netflow measurements Add gridftp (with UDPmon & new BW measurers netest pathrate, pathload Make early data available via http to interested & friendly researchers CAIDA for correlation and validation of Pipechar & iperf etc. (sent documentaion) NWS for forecasting with UCSB (sent documentation) Understand correlations, validate various tools, choose optimum set Make data available by std methods (e.g. MDS, GMA, ) with Dantong@BNL, Jenny Schopf@ANL & Tierney@LBNL Make tools portable, set up other monitoring sites, e.g. PPDG sites SLAC ported to Linux Currently porting measurement tools to Manchester Will work with INFN/Trieste & FNAL to port to other sites 52
Correlating Internet Performance Changes and Route Changes to Assist in Trouble-shooting from an End-user Perspective
Correlating Internet Performance Changes and Route Changes to Assist in Trouble-shooting from an End-user Perspective Connie Logg, Jiri Navratil, and Les Cottrell Stanford Linear Accelerator Center, 2575
More informationUsing Netflow data for forecasting
Using Netflow data for forecasting Les Cottrell SLAC and Fawad Nazir NIIT, Presented at the CHEP06 Meeting, Mumbai India, February 2006 www.slac.stanford.edu/grp/scs/net/talk06/icfachep06.ppt Partially
More informationInternet End-to-end Performance Monitoring (IEPM) and the PingER project.
Internet End-to-end Performance Monitoring (IEPM) and the PingER project. Les Cottrell and Warren Matthews Stanford Linear Accelerator Center. 1. Introduction. The ping end-to-end reporting (PingER) project
More informationNetwork monitoring in DataGRID project
Network monitoring in DataGRID project Franck Bonnassieux (CNRS) franck.bonnassieux@ens-lyon.fr 1st SCAMPI Workshop 27 Jan. 2003 DataGRID Network Monitoring Outline DataGRID network Specificity of Grid
More informationGlobus Striped GridFTP Framework and Server. Raj Kettimuthu, ANL and U. Chicago
Globus Striped GridFTP Framework and Server Raj Kettimuthu, ANL and U. Chicago Outline Introduction Features Motivation Architecture Globus XIO Experimental Results 3 August 2005 The Ohio State University
More informationNetest: A Tool to Measure the Maximum Burst Size, Available Bandwidth and Achievable Throughput
Netest: A Tool to Measure the Maximum Burst Size, Available Bandwidth and Achievable Throughput Guojun Jin Brian Tierney Distributed Systems Department Lawrence Berkeley National Laboratory 1 Cyclotron
More informationPassive and Active Monitoring on a High Performance Research Network. Abstract
SLAC PUB 8776 February 2001 Passive and Active Monitoring on a High Performance Research Network Warren Matthews, Les Cottrell and Davide Salomoni Stanford Linear Accelerator Center, Stanford University,
More informationPerformance Measurement of Wireless LAN Using Open Source
Performance Measurement of Wireless LAN Using Open Source Vipin M Wireless Communication Research Group AU KBC Research Centre http://comm.au-kbc.org/ 1 Overview General Network Why Network Performance
More informationExperiences in Traceroute and Available Bandwidth Change Analysis*
Experiences in Traceroute and Available Bandwidth Change Analysis* Connie Logg 2575 Sand Hill Road Menlo Park, CA 94025 +1 (650)926-2523 cal@slac.stanford.edu Les Cottrell 2575 Sand Hill Road Menlo Park,
More informationOverview of Network Measurement Tools
Overview of Network Measurement Tools Jon M. Dugan Energy Sciences Network Lawrence Berkeley National Laboratory NANOG 43, Brooklyn, NY June 1, 2008 Networking for the Future of Science
More informationHybrid network traffic engineering system (HNTES)
Hybrid network traffic engineering system (HNTES) Zhenzhen Yan, Zhengyang Liu, Chris Tracy, Malathi Veeraraghavan University of Virginia and ESnet Jan 12-13, 2012 mvee@virginia.edu, ctracy@es.net Project
More informationScience DMZs Understanding their role in high-performance data transfers
Science DMZs Understanding their role in high-performance data transfers Chris Tracy, Network Engineer Eli Dart, Network Engineer ESnet Engineering Group Overview Bulk Data Movement a common task Pieces
More informationIntegration of Network Performance Monitoring Data at FTS3
Integration of Network Performance Monitoring Data at FTS3 July-August 2013 Author: Rocío Rama Ballesteros Supervisor(s): Michail Salichos Alejandro Álvarez CERN openlab Summer Student Report 2013 Project
More informationD1.2 Network Load Balancing
D1. Network Load Balancing Ronald van der Pol, Freek Dijkstra, Igor Idziejczak, and Mark Meijerink SARA Computing and Networking Services, Science Park 11, 9 XG Amsterdam, The Netherlands June ronald.vanderpol@sara.nl,freek.dijkstra@sara.nl,
More informationHigh-Speed TCP Performance Characterization under Various Operating Systems
High-Speed TCP Performance Characterization under Various Operating Systems Y. Iwanaga, K. Kumazoe, D. Cavendish, M.Tsuru and Y. Oie Kyushu Institute of Technology 68-4, Kawazu, Iizuka-shi, Fukuoka, 82-852,
More informationPump Up Your Network Server Performance with HP- UX
Pump Up Your Network Server Performance with HP- UX Paul Comstock Network Performance Architect Hewlett-Packard 2004 Hewlett-Packard Development Company, L.P. The information contained herein is subject
More informationTCP Tuning Techniques for High-Speed Wide-Area Networks. Wizard Gap
NFNN2, 20th-21st June 2005 National e-science Centre, Edinburgh TCP Tuning Techniques for High-Speed Wide-Area Networks Distributed Systems Department Lawrence Berkeley National Laboratory http://gridmon.dl.ac.uk/nfnn/
More informationDeploying distributed network monitoring mesh
Deploying distributed network monitoring mesh for LHC Tier-1 and Tier-2 sites Phil DeMar, Maxim Grigoriev Fermilab Joe Metzger, Brian Tierney ESnet Martin Swany University of Delaware Jeff Boote, Eric
More informationFrequently Asked Questions
Frequently Asked Questions 1. Q: What is the Network Data Tunnel? A: Network Data Tunnel (NDT) is a software-based solution that accelerates data transfer in point-to-point or point-to-multipoint network
More informationDSS. Diskpool and cloud storage benchmarks used in IT-DSS. Data & Storage Services. Geoffray ADDE
DSS Data & Diskpool and cloud storage benchmarks used in IT-DSS CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/it Geoffray ADDE DSS Outline I- A rational approach to storage systems evaluation
More informationQuestion: 3 When using Application Intelligence, Server Time may be defined as.
1 Network General - 1T6-521 Application Performance Analysis and Troubleshooting Question: 1 One component in an application turn is. A. Server response time B. Network process time C. Application response
More informationInternet2 NetFlow Weekly Reports
Internet2 NetFlow Weekly Reports Stanislav Shalunov Internet2 Fall Member Meeting, Indianapolis, 2003-10-13 What is NetFlow? Originally a Cisco proprietary technology Now supported by other vendors and
More informationExperiences with MPTCP in an intercontinental multipathed OpenFlow network
Experiences with MP in an intercontinental multipathed network Ronald van der Pol, Michael Bredel, Artur Barczyk SURFnet Radboudkwartier 273 3511 CK Utrecht, The Netherlands Email: Ronald.vanderPol@SURFnet.nl
More informationGARUDA - NKN Partner's Meet 2015 Big data networks and TCP
GARUDA - NKN Partner's Meet 2015 Big data networks and TCP Brij Kishor Jashal Email brij.jashal@tifr.res.in Garuda-NKN meet 10 Sep 2015 1 Outline: Scale of LHC computing ( as an example of Big data network
More informationIperf Tutorial. Jon Dugan <jdugan@es.net> Summer JointTechs 2010, Columbus, OH
Iperf Tutorial Jon Dugan Summer JointTechs 2010, Columbus, OH Outline What are we measuring? TCP Measurements UDP Measurements Useful tricks Iperf Development What are we measuring? Throughput?
More informationmbits Network Operations Centrec
mbits Network Operations Centrec The mbits Network Operations Centre (NOC) is co-located and fully operationally integrated with the mbits Service Desk. The NOC is staffed by fulltime mbits employees,
More informationNetwork Monitoring with the perfsonar Dashboard
Network Monitoring with the perfsonar Dashboard Andy Lake Brian Tierney ESnet Advanced Network Technologies Group TIP2013 Honolulu HI January 15, 2013 Overview perfsonar overview Dashboard history and
More informationMeasure wireless network performance using testing tool iperf
Measure wireless network performance using testing tool iperf By Lisa Phifer, SearchNetworking.com Many companies are upgrading their wireless networks to 802.11n for better throughput, reach, and reliability,
More informationMeasuring Wireless Network Performance: Data Rates vs. Signal Strength
EDUCATIONAL BRIEF Measuring Wireless Network Performance: Data Rates vs. Signal Strength In January we discussed the use of Wi-Fi Signal Mapping technology as a sales tool to demonstrate signal strength
More informationTCP Labs. WACREN Network Monitoring and Measurement Workshop Antoine Delvaux a.delvaux@man.poznan.pl perfsonar developer 30.09.
TCP Labs WACREN Network Monitoring and Measurement Workshop Antoine Delvaux a.delvaux@man.poznan.pl perfsonar developer 30.09.2015 Hands-on session We ll explore practical aspects of TCP Checking the effect
More informationApplications. Network Application Performance Analysis. Laboratory. Objective. Overview
Laboratory 12 Applications Network Application Performance Analysis Objective The objective of this lab is to analyze the performance of an Internet application protocol and its relation to the underlying
More informationTCP tuning guide for distributed application on wide area networks 1.0 Introduction
TCP tuning guide for distributed application on wide area networks 1.0 Introduction Obtaining good TCP throughput across a wide area network usually requires some tuning. This is especially true in high-speed
More informationUsing TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment TrueSpeed VNF provides network operators and enterprise users with repeatable, standards-based testing to resolve complaints about
More informationTeraPaths: A QoS Collaborative Data Sharing Infrastructure for Petascale Computing Research
TeraPaths: A QoS Collaborative Data Sharing Infrastructure for Petascale Computing Research Bruce Gibbard & Dantong Yu High-Performance Network Research PI Meeting September 28-30, 2005 Brookhaven National
More informationLecture 8 Performance Measurements and Metrics. Performance Metrics. Outline. Performance Metrics. Performance Metrics Performance Measurements
Outline Lecture 8 Performance Measurements and Metrics Performance Metrics Performance Measurements Kurose-Ross: 1.2-1.4 (Hassan-Jain: Chapter 3 Performance Measurement of TCP/IP Networks ) 2010-02-17
More informationNetwork Performance Optimisation and Load Balancing. Wulf Thannhaeuser
Network Performance Optimisation and Load Balancing Wulf Thannhaeuser 1 Network Performance Optimisation 2 Network Optimisation: Where? Fixed latency 4.0 µs Variable latency
More informationperfsonar: End-to-End Network Performance Verification
perfsonar: End-to-End Network Performance Verification Toby Wong Sr. Network Analyst, BCNET Ian Gable Technical Manager, Canada Overview 1. IntroducGons 2. Problem Statement/Example Scenario 3. Why perfsonar?
More informationGridCopy: Moving Data Fast on the Grid
GridCopy: Moving Data Fast on the Grid Rajkumar Kettimuthu 1,2, William Allcock 1,2, Lee Liming 1,2 John-Paul Navarro 1,2, Ian Foster 1,2,3 1 Mathematics and Computer Science Division Argonne National
More informationHow To Set Up A Network Measurement Toolkit For A Network Performance Test On A Network With A Network (Networking) On A Microsoft Ipa 2.5 (Netware) On An Ipa2 (Netcom) On Your Computer
Firewall Port Recommendations for the ps Performance Toolkit Prepared by the NTAC Performance Working Group November 2014 Edited by J. Zurawski (ESnet), A. Brown (Internet2), A. Lake (ESnet), K. Miller
More informationCS551 End-to-End Internet Packet Dynamics [Paxson99b]
CS551 End-to-End Internet Packet Dynamics [Paxson99b] Bill Cheng http://merlot.usc.edu/cs551-f12 1 End-to-end Packet Dynamics How do you measure Internet performance? Why do people want to know? Are ISPs
More informationESnet Support for WAN Data Movement
ESnet Support for WAN Data Movement Eli Dart, Network Engineer ESnet Science Engagement Group Joint Facilities User Forum on Data Intensive Computing Oakland, CA June 16, 2014 Outline ESnet overview Support
More informationDEPLOYMENT GUIDE Version 1.1. Configuring BIG-IP WOM with Oracle Database Data Guard, GoldenGate, Streams, and Recovery Manager
DEPLOYMENT GUIDE Version 1.1 Configuring BIG-IP WOM with Oracle Database Data Guard, GoldenGate, Streams, and Recovery Manager Table of Contents Table of Contents Configuring BIG-IP WOM with Oracle Database
More informationIMPLEMENTING GREEN IT
Saint Petersburg State University of Information Technologies, Mechanics and Optics Department of Telecommunication Systems IMPLEMENTING GREEN IT APPROACH FOR TRANSFERRING BIG DATA OVER PARALLEL DATA LINK
More informationMonitoring high-speed networks using ntop. Luca Deri <deri@ntop.org>
Monitoring high-speed networks using ntop Luca Deri 1 Project History Started in 1997 as monitoring application for the Univ. of Pisa 1998: First public release v 0.4 (GPL2) 1999-2002:
More informationDistributed applications monitoring at system and network level
Distributed applications monitoring at system and network level Monarc Collaboration 1 Abstract Most of the distributed applications are presently based on architectural models that don t involve real-time
More informationTCP Adaptation for MPI on Long-and-Fat Networks
TCP Adaptation for MPI on Long-and-Fat Networks Motohiko Matsuda, Tomohiro Kudoh Yuetsu Kodama, Ryousei Takano Grid Technology Research Center Yutaka Ishikawa The University of Tokyo Outline Background
More informationOperating System for the K computer
Operating System for the K computer Jun Moroo Masahiko Yamada Takeharu Kato For the K computer to achieve the world s highest performance, Fujitsu has worked on the following three performance improvements
More informationPerformance Analysis of IPv4 v/s IPv6 in Virtual Environment Using UBUNTU
Performance Analysis of IPv4 v/s IPv6 in Virtual Environment Using UBUNTU Savita Shiwani Computer Science,Gyan Vihar University, Rajasthan, India G.N. Purohit AIM & ACT, Banasthali University, Banasthali,
More informationWhite Paper. The Ten Features Your Web Application Monitoring Software Must Have. Executive Summary
White Paper The Ten Features Your Web Application Monitoring Software Must Have Executive Summary It s hard to find an important business application that doesn t have a web-based version available and
More informationFinding Fault Location: Combining network topology and end-to-end measurments to locate network problems?
Finding Fault Location: Combining network topology and end-to-end measurments to locate network problems? Chris Kelly - chris.kelly@oit.gatech.edu Research Network Operations Center Georgia Tech Office
More informationOpen Source in Network Administration: the ntop Project
Open Source in Network Administration: the ntop Project Luca Deri 1 Project History Started in 1997 as monitoring application for the Univ. of Pisa 1998: First public release v 0.4 (GPL2) 1999-2002:
More informationWindows Server Performance Monitoring
Spot server problems before they are noticed The system s really slow today! How often have you heard that? Finding the solution isn t so easy. The obvious questions to ask are why is it running slowly
More informationThe Ecosystem of Computer Networks. Ripe 46 Amsterdam, The Netherlands
The Ecosystem of Computer Networks Ripe 46 Amsterdam, The Netherlands Silvia Veronese NetworkPhysics.com Sveronese@networkphysics.com September 2003 1 Agenda Today s IT challenges Introduction to Network
More informationDISTRIBUTED COMPUTING ENVIRONMENT MONITORING AND USER EXPECTATIONS *
SLAC-PUB-95-7008 November 1995 DISTRIBUTED COMPUTING ENVIRONMENT MONITORING AND USER EXPECTATIONS * R. L. A. COTTRELL, C. A. LOGG Stanford Linear Accelerator Center, Stanford University, Stanford, CA 94309,
More informationHands on Workshop. Network Performance Monitoring and Multicast Routing. Yasuichi Kitamura NICT Jin Tanaka KDDI/NICT APAN-JP NOC
Hands on Workshop Network Performance Monitoring and Multicast Routing Yasuichi Kitamura NICT Jin Tanaka KDDI/NICT APAN-JP NOC July 18th TEIN2 Site Coordination Workshop Network Performance Monitoring
More informationLow-rate TCP-targeted Denial of Service Attack Defense
Low-rate TCP-targeted Denial of Service Attack Defense Johnny Tsao Petros Efstathopoulos University of California, Los Angeles, Computer Science Department Los Angeles, CA E-mail: {johnny5t, pefstath}@cs.ucla.edu
More informationAchieving Reliable High Performance in LFNs
Achieving Reliable High Performance in LFNs ven Ubik, and Pavel Cimbal, CENET, Prague, Czech Republic Abstract The PC hardware architecture and commodity operating
More informationDell PowerVault MD Series Storage Arrays: IP SAN Best Practices
Dell PowerVault MD Series Storage Arrays: IP SAN Best Practices A Dell Technical White Paper Dell Symantec THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND
More information1-800-CALL-H.E.P. - Experiences on a Voice-over-IP Test Bed.
SLAC-PUB-8384 February 2000 1-800-CALL-H.E.P. - Experiences on a Voice-over-IP Test Bed. W. Matthews, L. Cottrell, R. Nitzan Presented at International Conference On Computing In High Energy Physics And
More informationFinal for ECE374 05/06/13 Solution!!
1 Final for ECE374 05/06/13 Solution!! Instructions: Put your name and student number on each sheet of paper! The exam is closed book. You have 90 minutes to complete the exam. Be a smart exam taker -
More informationInfrastructure for active and passive measurements at 10Gbps and beyond
Infrastructure for active and passive measurements at 10Gbps and beyond Best Practice Document Produced by UNINETT led working group on network monitoring (UFS 142) Author: Arne Øslebø August 2014 1 TERENA
More informationINITIAL TOOL FOR MONITORING PERFORMANCE OF WEB SITES
INITIAL TOOL FOR MONITORING PERFORMANCE OF WEB SITES Cristina Hava & Stefan Holban Faculty of Automation and Computer Engineering, Politehnica University Timisoara, 2 Vasile Parvan, Timisoara, Romania,
More informationData Management. Network transfers
Data Management Network transfers Network data transfers Not everyone needs to transfer large amounts of data on and off a HPC service Sometimes data is created and consumed on the same service. If you
More informationA Talari Networks White Paper. Turbo Charging WAN Optimization with WAN Virtualization. A Talari White Paper
A Talari Networks White Paper Turbo Charging WAN Optimization with WAN Virtualization A Talari White Paper 2 Introduction WAN Virtualization is revolutionizing Enterprise Wide Area Network (WAN) economics,
More informationDevelopment of 10 Gbits/s Traffic Shaper
Kenji Anzai, Kenji Yoshinaka, Hiroshi Harada, Koichi Ryu, Masaya Suzuki, Atsushi Saegusa, Masanao Kobayashi, Takayuki Sato, Ryota Watanabe [Summary] We developed the PureFlow GSX-XR as a high-precision
More informationActive Measurement Data Analysis Techniques
3/27/2000: This work is an Authors version, and has been submitted for publication. Copyright may be transferred without further notice and the accepted version may then be posted by the publisher. Active
More informationEvaluation of Techniques to Detect Significant Network Performance Problems using End-to-End Active Network Measurements
SLAC-PUB-11653 Evaluation of Techniques to Detect Significant Network Performance Problems using End-to-End Active Network Measurements by R.L. Cottrell, et al. Contributed to 2006 IEEE/IFIP Network Operations
More informationIP SAN Best Practices
IP SAN Best Practices A Dell Technical White Paper PowerVault MD3200i Storage Arrays THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES.
More informationCA Unified Infrastructure Management
CA Unified Infrastructure Management Probe Guide for IIS Server Monitoring iis v1.7 series Copyright Notice This online help system (the "System") is for your informational purposes only and is subject
More informationSC14 Remote I/O Pipeline Processing Demonstrtion
OpenFabrics Software User Group Workshop SC4 Remote I/O Pipeline Processing Demonstrtion #OFSUserGroup Dardo D Kleiner / CIPS, Corp. / dkleiner@cips.com Linden Mercer / PSU / lbm2@psu.edu Srinath Jayasundera
More informationOn evaluating the differences of TCP and ICMP in network measurement
Computer Communications 30 (2007) 428 439 www.elsevier.com/locate/comcom On evaluating the differences of TCP and ICMP in network measurement Li Wenwei b, *, Zhang Dafang a, Yang Jinmin a, Xie Gaogang
More informationVirtualization: TCP/IP Performance Management in a Virtualized Environment Orlando Share Session 9308
Virtualization: TCP/IP Performance Management in a Virtualized Environment Orlando Share Session 9308 Laura Knapp WW Business Consultant Laurak@aesclever.com Applied Expert Systems, Inc. 2011 1 Background
More informationProposal for a perfsonar Multi Domain Monitoring Service for LHCOPN
Proposal for a perfsonar Multi Domain Monitoring Service for LHCOPN Service Specification Version 10 Proposal for a perfsonar Multi Domain Monitoring Service for LHCOPN Document History Product/Version
More informationTELE 301 Network Management
TELE 301 Network Management Lecture 22: Diagnostics & Ethics Haibo Zhang Computer Science, University of Otago TELE301 Lecture 22: Diagnostics & Ethics 1 Fault Management Fault management It means preventing,
More informationNetwork Measurement. Why Measure the Network? Types of Measurement. Traffic Measurement. Packet Monitoring. Monitoring a LAN Link. ScienLfic discovery
Why Measure the Network? Network Measurement Jennifer Rexford COS 461: Computer Networks Lectures: MW 10-10:50am in Architecture N101 ScienLfic discovery Characterizing traffic, topology, performance Understanding
More informationperfsonar Overview Jason Zurawski, ESnet zurawski@es.net Southern Partnerships for Advanced Networking November 3 rd 2015
perfsonar Overview Jason Zurawski, ESnet zurawski@es.net Southern Partnerships for Advanced Networking November 3 rd 2015 This document is a result of work by the perfsonar Project (http://www.perfsonar.net)
More informationCampus Network Design Science DMZ
Campus Network Design Science DMZ Dale Smith Network Startup Resource Center dsmith@nsrc.org The information in this document comes largely from work done by ESnet, the USA Energy Sciences Network see
More informationHanyang University Grid Network Monitoring
Grid Network Monitoring Hanyang Univ. Multimedia Networking Lab. Jae-Il Jung Agenda Introduction Grid Monitoring Architecture Network Measurement Tools Network Measurement for Grid Applications and Services
More informationABW - Short-timescale passive bandwidth monitoring
ABW - Short-timescale passive bandwidth monitoring Sven Ubik (CESNET, Czech Republic), Demetres Antoniades (ICS-FORTH, Greece), Arne Oslebo (UNINETT, Norway) Abstract Bandwidth usage monitoring is important
More informationProcedure: You can find the problem sheet on Drive D: of the lab PCs. 1. IP address for this host computer 2. Subnet mask 3. Default gateway address
Objectives University of Jordan Faculty of Engineering & Technology Computer Engineering Department Computer Networks Laboratory 907528 Lab.4 Basic Network Operation and Troubleshooting 1. To become familiar
More informationIP SAN BEST PRACTICES
IP SAN BEST PRACTICES PowerVault MD3000i Storage Array www.dell.com/md3000i TABLE OF CONTENTS Table of Contents INTRODUCTION... 3 OVERVIEW ISCSI... 3 IP SAN DESIGN... 4 BEST PRACTICE - IMPLEMENTATION...
More informationNASA EOSDIS Network Monitoring New Active, Passive and Real-time Monitoring Approaches
NASA EOSDIS Network Monitoring New Active, Passive and Real-time Monitoring Approaches Current Activities and Plans JET Roadmap Workshop April 13, 2004 EOSDIS Activities and Plans Agenda Current Activities:
More informationMonitoring Android Apps using the logcat and iperf tools. 22 May 2015
Monitoring Android Apps using the logcat and iperf tools Michalis Katsarakis katsarakis@csd.uoc.gr Tutorial: HY-439 22 May 2015 http://www.csd.uoc.gr/~hy439/ Outline Introduction Monitoring the Android
More informationPerformance Comparison of low-latency Anonymisation Services from a User Perspective
Performance Comparison of low-latency Anonymisation Services from a User Perspective Rolf Wendolsky Hannes Federrath Department of Business Informatics University of Regensburg 7th Workshop on Privacy
More informationOpen Source File Transfers
Open Source File Transfers A comparison of recent open source file transfer projects By: John Tkaczewski Contents Introduction... 2 Recent Open Source Projects... 2 UDT UDP-based Data Transfer... 4 Tsunami
More informationExperiences Deploying and Operating a Large-Scale Monitoring Infrastructure
1 Experiences Deploying and Operating a Large-Scale Monitoring Infrastructure 25 th NORDUnet conference Arne Øslebø arne.oslebo@uninett.no Outline Background and motivation Typical setup Deployment map
More informationPolicy Based Forwarding
Policy Based Forwarding Tech Note PAN-OS 4.1 Revision A 2012, Palo Alto Networks, Inc. www.paloaltonetworks.com Contents Overview... 3 Security... 3 Performance... 3 Symmetric Routing... 3 Service Versus
More informationKey Components of WAN Optimization Controller Functionality
Key Components of WAN Optimization Controller Functionality Introduction and Goals One of the key challenges facing IT organizations relative to application and service delivery is ensuring that the applications
More informationThe Grid and the Network
The Grid and the Network The UK Network Infrastructure A summary of E-Science supported Network projects in the UK Protocols Middleware for network services 1 The Grid and the Network The UK Network Infrastructure
More informationMONITORING AVAILABLE BANDWIDTH OF UNDERLYING GRID NETWORKS
MONITORING AVAILABLE BANDWIDTH OF UNDERLYING GRID NETWORKS Marcia Zangrilli Bruce B. Lowekamp, advisor Department of Computer Science College of William and Mary Abstract Harnessing the complete power
More informationBOF on NETWORK QUALITY OF SERVICE APPLIED TO THE GRID
BOF on NETWORK QUALITY OF SERVICE APPLIED TO THE GRID GGF4 Toronto, Feb 19 Tiziana Ferrari Network Quality of Service Applied to the GRID, GGF4 Toronto 1 1. Motivation and overview, T.Ferrari 2. QoS: Requirements
More informationWeb Load Stress Testing
Web Load Stress Testing Overview A Web load stress test is a diagnostic tool that helps predict how a website will respond to various traffic levels. This test can answer critical questions such as: How
More informationDeploying Riverbed wide-area data services in a LeftHand iscsi SAN Remote Disaster Recovery Solution
Wide-area data services (WDS) Accelerating Remote Disaster Recovery Reduce Replication Windows and transfer times leveraging your existing WAN Deploying Riverbed wide-area data services in a LeftHand iscsi
More informationTransparent Optimization of Grid Server Selection with Real-Time Passive Network Measurements. Marcia Zangrilli and Bruce Lowekamp
Transparent Optimization of Grid Server Selection with Real-Time Passive Network Measurements Marcia Zangrilli and Bruce Lowekamp Overview Grid Services Grid resources modeled as services Define interface
More informationPost-production Video Editing Solution Guide with Quantum StorNext File System AssuredSAN 4000
Post-production Video Editing Solution Guide with Quantum StorNext File System AssuredSAN 4000 Dot Hill Systems introduction 1 INTRODUCTION Dot Hill Systems offers high performance network storage products
More informationSonicWALL Global Management System Reporting Guide Standard Edition
SonicWALL Global Management System Reporting Guide Standard Edition Version 2.9.4 Copyright Information 2005 SonicWALL, Inc. All rights reserved. Under the copyright laws, this manual or the software described
More informationEVERYTHING A DBA SHOULD KNOW
EVERYTHING A DBA SHOULD KNOW ABOUT TCPIP NETWORKS Chen (Gwen),HP Software-as-a-Service 1. TCP/IP Problems that DBAs Can Face In this paper I ll discuss some of the network problems that I ve encountered
More informationTransport Layer Protocols
Transport Layer Protocols Version. Transport layer performs two main tasks for the application layer by using the network layer. It provides end to end communication between two applications, and implements
More informationBest of Breed of an ITIL based IT Monitoring. The System Management strategy of NetEye
Best of Breed of an ITIL based IT Monitoring The System Management strategy of NetEye by Georg Kostner 5/11/2012 1 IT Services and IT Service Management IT Services means provisioning of added value for
More informationMeasuring IP Performance. Geoff Huston Telstra
Measuring IP Performance Geoff Huston Telstra What are you trying to measure? User experience Responsiveness Sustained Throughput Application performance quality Consistency Availability Network Behaviour
More information