1 Cyclotron Rd. MS: 50B-2239, Berkeley, CA 94720

The well-known difficulties in obtaining good network performance for TCP-based applications without expert tuning or case-by-case application optimization can be overcome by building expertise into the operating system where it will benefit all users and applications.

The Net100 Collaboration (PSC, NCAR, UT, LBL, and ORNL) will to develop a model for network-aware operating systems using Web100 as the means for incorporating network information and its analysis into host operating systems to improve performance. To investigate how effective network-aware operating systems can be, we will use a three-phase approach. First, we will use the network-aware; Web100 based operating system that we develop to create a simple, bulk-transport application and demonstrate its use over high performance network links. We will then extend this model to support more advanced and complex applications, moving from point-to-point optimization to optimizations for fully distributed environments. Finally, as proof that a network-aware operating system can tune and optimize performance on behalf of applications, we will also develop application-internal tools (based on NetLogger) to monitor the efficiency of application support, and provide an external monitoring methodology (based on the Network Weather Service) to gauge the impact this system has on the rest of the network.

In addition to serving the needs of high-performance computing and network users, this project will serve as an (open source) showcase for network-aware operating systems, beginning with a Web100-based O/S. The tools, sources, measured results, and methods will be showcased on a "Closing the Wizard Gap" web site and made available to the high performance networking community.

The core component of the Net100 project is a network aware operating system (NAOS). The NAOS collectively refers to kernel level modifications as well as API’s, libraries, and daemons developed as part of the project to support the network-aware functions. In many cases actual NAOS components may depend on the base operating system and/or the underlying network technologies.

Net100 will use the above technologies and expertise as mechanisms for testing and refining Web100. however, there are some critical missing components which must be developed to allow a thorough analysis. These will also be provided by Net100, and include:

• The Network Tools Analysis Framework (NTAF), a framework for triggering NWS network measurement tools as well as other network monitoring tools, such as pathrate, pipechar, and Iperf. The NTAF will be provided by LBNL.
• The Network Analysis Information Base (NAIB), a monitoring data archive database that provides sophisticated relational queries and analysis. The NWS team at UTK will provide the NAIB.
• Instrumented applications and tools. This involves using the Web100 TCP instrumentation interface to collect TCP data from a variety of applications and monitoring tools, and use NetLogger to format the data send it to the NAIB. LBNL, ORNL, and the NWS team will do this.

The results of this monitoring and analysis will then be fed back to the Web100 team for further refinement of the Web100 kernel.

The goal of Net100 is to eliminate what has been called the "wizard gap". Through the integration of end-to-end and application-level monitoring capabilities with the tuning and diagnostic capabilities provided by Web100, we will develop a unique and general-purpose system for optimizing and understanding end-to-end network and application performance. This will allow us to create a network-aware operating system that will be able to maximize network utilization for a wide variety of applications and, without help from the wizards, eliminate the wizard gap.