What's next for Internet data analysis? Status and challengesfacing the community

Most large providers currently collect basic statistics on the performance of their own infrastructure, typically including measurements of utilization and availability and possibly rudimentary assessments of delay and throughput. In today's commercial Internet, the only baseline against which organizations can calibrate their networks is past performance data. No data or even standard formats are available against which to compare performance with other networks or against an industry norm, nor are there reliable data with which customers can assess the performance of providers. Data characterization and traffic flow analysis also are virtually nonexistent at this time, yet they remain essential for understanding the internal dynamics of the Internet infrastructure. Path performance measurement tools enable users and operators to better evaluate and compare providers and to monitor service quality. Traffic flow characterization tools focus on the behavior and inner workings of these wide-area networks. This paper has two goals. We first provide background on the current Internet architecture and describe how measurements are a key element in the development of a robust and financially successful commercial Internet. We then discuss the current state of Internet metrics analysis and activities within various forums, particularly the Cooperative Association for Internet Data Analysis and the National Laboratory for Applied Network Research, to encourage the development and deployment of Internet performance monitoring and workload characterization tools