From...

February 2, 2000
Web posted at: 1:17 p.m. EDT (1317 GMT)

by Network World Fusion

(IDG) -- Network World has teamed with Visual Networks to bring you the first in a series of quarterly reports on the top ISPs in the market.

Through statistical analysis of Visual's Internet BenchMark data (see How we did it), we ranked the top ISPs in the business-to-business, national and regional ISP markets. In addition, we've tracked results since August 1999 to show whether an ISP's ranking has spiked for one month or is consistent.

In December 1999, AT&T WorldNet came out on top, performing above average in six out of nine categories tested. Coming in a close second is BellSouth, which performed above average in five out of nine categories.

The data for this report comes from Visual Networks. Through its Internet BenchMark data, Visual rates national, regional and business-to-business ISPs according to several factors, including call failure rate, modem connect speed and Web download performance.

ISPs tested ISP TYPE Ameritech Regional Retail AOL National Retail AT&T Worldnet National Retail AT&T (GNS) Business-to-Business (B2B) BellAtlantic Regional Retail BellAtlantic-North Regional Retail BellSouth Regional Retail CompuServe National Retail Concentric B2B EarthLink National Retail GTE.net National Retail GTEInternet B2B ICG-NetAhead B2B Juno National Retail MindSpring National Retail MSN National Retail PacBell Regional Retail Prodigy National Retail PSI B2B RCN Regional Retail SBIS Regional Retail Splitrock B2B USWest Regional Retail UUNET B2B UUNET (GridNet) B2B

We took the raw data from Visual and applied statistical analysis to rate the relative performance of each ISP. First, we derived the standard deviation of the numbers in each performance category. Standard deviation is a measure of how far the numbers in a series diverge from each other. For each category in which an ISP performed better than one standard deviation from the industry mean, we awarded it one point. If the ISP did better than two standard deviations from the mean, we awarded it two points.

Similarly, if an ISP did worse than the industry mean by more than one standard deviation, we took a point away from its score. If it did significantly worse, meaning two or more standard deviations, we took away two points. We started with a baseline of 12 points, so if an ISP scored 0 points it ended up with an adjusted score of 12.

After scoring each category, we summed the results to produce a single number that indicates the reliability and performance of each ISP. A rating of 12 means an ISP was about even with its peers, or the industry average. In December 1999, 10 ISPs scored higher than 12, making them eligible for our list of the best national ISPs. The bad news, however, is that 11 ISPs scored less than 12 points.

Quick tip

Visual's Internet BenchMark provides a standard way to compare the overall performance of dial-based ISPs. But the performance experienced by specific users may vary based on many factors, including the type of equipment, dialing locations and the level of service purchased from the ISP.

Many companies are turning to service-level agreements (SLA) to ensure that their users are receiving the quality of service promised by the ISP. But how can users measure the day-to-day service-level metrics, such as availability or response time, negotiated from their ISPs?

Systems and tools that let enterprises and ISPs measure service levels, and manage IP connectivity and applications are typically referred to as service-level management (SLM) systems. These systems offer tool sets that define, monitor, and administer SLAs.

Data collected from an end user's point of view can be presented in a collaborative fashion, in which ISPs use SLM systems to monitor performance on dial-up and dedicated Internet circuits, and then share the performance data with their customers.