Building a Network Performance Data Collection Framework with NDT and GIS

Size: px

Start display at page:

Download "Building a Network Performance Data Collection Framework with NDT and GIS"

Brandon Lane
8 years ago
Views:

1 Building a Network Performance Data Collection Framework with NDT and GIS Seth Peery, Sr. GIS Architect Virginia Tech Geospatial Information Sciences Internet2 Spring Member Meeting April 18, 2011

2 Relevance of Performance Measurement Internet benefits (economic development, health, education, etc.) are derived from effective use of productive applications [1], not mere presence of infrastructure. Performance measurement allows us to approximate the user s experience with the network and identify what applications are feasible [1] Gurstein, M. (2003). "Effective use: a community informatics strategy beyond the digital divide." First Monday 8(12). 2

Performance measurement allows us to approximate the user s experience with the network and identify what

3 Performance Measurement Use Cases Target Group Outcomes/Rationale The public Compare measured to advertised performance Model and estimate commodity network performance CAIs Identify underserved institutions Benchmarking and impact assessment for projects like US-UCAN R&E connectors Monitor R&E network health Plan future provisioning i i Special subgroups Targeted research 3

underserved institutions Benchmarking and impact assessment for projects like US-UCAN R&E

4 Supply-side vs. Demand-side Representations ti of Connectivity it Supply Side Map the core and infer the edges Locations of infrastructure Service footprints Advertised speeds Typical speeds Focus of SBDD data collection Passive/automated testing Demand Side Map the edges and infer the core Spatial distribution of users Adoption and Digital Divide User-initiated performance testst Before SBDD, this was all we had 4 We need both perspectives for the complete picture.

infrastructure Service footprints Advertised speeds Typical speeds Focus of SBDD data collection Passive/automated

5 Advertised vs. Measured Performance SBDD Max Advertised Download Speed < 200 mbps 200kbps kbps 768 kbps mbps 15mbps 1.5-3mbps 3 mbps - 6 mbps 6 mbps - 10 mbps 10 mbps - 25 mbps 25 mbps - 50 mbps 50 mbps mbps 100 mbps - 1 Gbps >1Gbps Virginia Tech NDT Test Data Points under 200 kbps 200kbps kbps 768kbps mbps 1.5mbps - 3 mbps 3mbps - 6 mbps 6mbps - 10 mbps 10mbps - 25 mbps 25mbps mbps over 100 mbps 5 Miles

5-3mbps 3 mbps - 6 mbps 6 mbps - 10 mbps 10 mbps - 25 mbps 25 mbps - 50 mbps 50 mbps - 100 mbps 100 mbps - 1 Gbps

6 6 Sources of User-Initiated Network Performance Data

7 Virginia Tech ecorridors Broadband Map Started in 2006 Combines NDT with a Google Maps front-end Nearly 5000 data points to date, and growing 7

8 Network Performance Data in GIS Virginia Data Points under 200 kbps 200kbps kbps 768kbps mbps 1.5mbps - 3 mbps 3mbps - 6 mbps 6mbps - 10 mbps 10mbps - 25 mbps 25mbps mbps over 100 mbps 8

9 Spatial Analysis of Georeferenced NDT results Points by themselves don t tell us much Spatial joins to bounded areas can yield generalized summary information Getting a sufficiently large sample size is critical Interpolation holds potential, once certain constraints are properly modeled 9

summary information Getting a sufficiently large sample size is critical

10 Effect of AccelerateVirginia Campaigns Sample Size

11 Generalization by Areal Unit Blocks Counties Virginia i i Data Points under 200 kbps 200kbps kbps 768kbps mbps 1.5mbps - 3 mbps 3mbps - 6 mbps 6mbps - 10 mbps 10mbps - 25 mbps 25mbps mbps over 100 mbps Average Download Avg_downlo no data under 200 kbps 200 kbps kbps 768kbps mbps 1.5 mbps - 3 mbps 3 mbps - 6 mbps 6 mbps - 10 mbps 10 mbps - 25 mbps 25 mbps mbps over 100 mbps 11

5mbps - 3 mbps 3mbps - 6 mbps 6mbps - 10 mbps 10mbps - 25 mbps 25mbps - 100 mbps over 100 mbps Average

12 Areal Averages used in Analysis County-level aggregated map data County business patterns 12

13 Interpolation Kriging Predicted Values

14 Testing Anchor Institutions: v1 14

15 Anchor Institutions v2 : Internet2 K20 CAIDATA Project 15

16 Research Directions Network performance measurements can be aggregated to any desired spatial unit of analysis: county, ZIP code, Census block Data can be analyzed over large regions or at the local level Objective, user-contributed network performance measurements can be used as inputs to models that quantify many other outcomes of network connectivity Economic Impact Education Health Care Government Performance Safety and Security 16

Objective, user-contributed network performance measurements can be used as inputs to models that quantify

17 Combining Performance Measurement with Specialized Research Studies Health IT Adoption ecommerce Survey Respondents Speed_Download_Mbps Miles E-Commerce usage by Businesses

768001-1.500000 1.500001-3.000000 3.000001-6.000000 6.000001-10.000000 10.000001-25.

18 Seth s Research Design for Economic Impact 18

19 Challenges of Using Measured Performance Data in Research Removing the measurement infrastructure as a source of error Controlling for temporal and spatial variability in individual speed tests Understanding the limits to the generalizability of individual tests Getting a sufficient sample size 19

and spatial variability in individual speed tests Understanding the limits

20 Platform Considerations Attend Performance WG on Wednesday at 0730 to learn more I2 is a leader in the measurement space due to NDT Is mass market network performance measurement a fundamentally different use case? The de facto standard d should be open More fundamentally User-initiated vs. passive / automated testing? Moving the test platform closer to the user regardless of platform 20

21 Contact Information Seth Peery Senior GIS Architect, Enterprise GIS Virginia Tech Geospatial Information Sciences 2060 Torgersen Hall (0197), Blacksburg, VA (540)

Regional R&E Networks: Supporting CAI Broadband d Needs. Jen Leasure The Quilt October 3, 2011

Regional R&E Networks: Supporting CAI Broadband d Needs Jen Leasure The Quilt October 3, 2011 What is The Quilt Established in 2000 501(c)3 A collaboration of leading advanced regional networking organizations