Can you trust e-commerce review sites?

From...

May 26, 2000
Web posted at: 11:26 a.m. EDT (1526 GMT)

by Daniel Tynan

(IDG) -- Everyone has an online shopping horror story. Orders that disappear. Products that never arrive. With thousands of e-merchants, how do you find the ones you can trust?

A growing number of companies aim to help, rating Web stores by using criteria such as features, selection, and service. They deliver the ratings online, for free. But keep your eyes open. Among the major merchant raters, we found key differences.

	MESSAGE BOARD
Internet society

Sites such as BizRate and Reseller Ratings rely on consumers to rate online stores. Others, like Shop Online 123, employ analysts to examine sites. Gomez Advisors and RatingWonders do both. The sites share some drawbacks. Most cover only a fraction of the Web's stores. Many favor positive reviews. And some sites do a poor job of preventing ballot box stuffing.

Think positive

The oldest and best-known rating site, BizRate, publishes ratings on more than 1000 Web merchants it chooses to review, from car rental agencies to clothing stores. (About four out of five merchants have asked to be reviewed, the company says.) BizRate derives most of its ratings by surveying people who've bought at a merchant's site.

Consumers who buy at a BizRate affiliate see a pop-up survey asking them to rate site navigation, selection, prices, and shipping options. Later, consumers receive an e-mail asking whether the product arrived and how satisfied they were with the shopping process.

MORE COMPUTING INTELLIGENCE

IDG.net home page

Chart: Behind the scenes at rating sites

Deja.com boosts buyers' guide

Special Report: The Net 21

The best online brokers

Reviews & in-depth info at IDG.net

E-Business World

TechInformer

Questions about computers? Let IDG.net's editors help you

Subscribe to IDG.net's free daily newsletter for computer geniuses (& newbies)

Search IDG.net in 12 languages

News Radio

Fusion audio primers

Computerworld Minute

But this approach has one big problem, says Steve Cook, a senior vice president for Greenfield Online, a Web-based research firm in Wilton, Connecticut: It can't capture the opinions of people who gave up before they reached the checkout counter.

"The people who were frustrated by the site, who never found what they wanted -- they never have the chance to be intercepted [by the survey]," explains Cook. "That [omission] gives you a little more of a positive spin on the sites."

In fact, you'll have a hard time finding anything but positive spins on BizRate. In a spot check of the site at press time in early May, we found no overall rating lower than three stars out of five. (BizRate CEO Chuck Davis did point out some examples of lower scores in individual categories, such as on-time delivery.)

About 87 percent of stores selling desktop PCs received BizRate ratings of four stars or higher. In home audio, 88 percent of stores scored that high; among music vendors, 96 percent rated four stars.

Davis says that some merchants pay BizRate a commission for referring customers to their sites. But he says the firm's ratings are "purely objective." The site makes most of its money by selling research data to online merchants. Because e-tailers must agree to incorporate BizRate's survey into their shopping carts, he explains, only the best merchants apply. "If you are a good site, you want to be on BizRate. If you are not a good site, we're going to reveal that pretty quickly," he says.

Some big sites like Amazon.com and Dell don't participate in BizRate's point-of-sale surveys. To rate those vendors, BizRate polls a panel of consumers who've done BizRate surveys on other e-merchants. Here, too, negative ratings are almost nonexistent.

BizRate is not the only bearer of good news. Shop Online 123 takes a different approach, but its results are only slightly more critical. Instead of surveying consumers, the site's team of surfers buys products from selected vendors, returns them, and rates the sites on service, selection, and the like.

But the lowest rating any site can receive is Fair. About 250 e-tailers are currently reviewed on the site; according to editor Dave Kurns, nearly three times that number have been evaluated.

"What we've done is only give you the good stuff, " says Jackie Leo, vice president of interactive media for Meredith Corporation, which owns Shop Online 123. "We don't want to waste your time with sites we think aren't valuable."

Leo says that while the site receives commissions for some customer referrals, those deals are struck only after reviews have been written, to avoid influencing the reviewers. Shop Online 123 makes most of its money from print advertising -- it's also an annual publication sent to subscribers of other Meredith magazines such as Better Homes and Gardens.

The problem? If a store isn't on the list, you don't know whether it was deemed bad or hasn't been rated. Among sites all rated positively, it's hard to choose.

Also, while the Web enables raters to garner feedback from thousands of consumers, it also can make it easy for smaller e-tailers to alter the scores.

Reseller Ratings, which focuses on computer hardware and software vendors, allows you to rate any e-merchant -- even some that don't exist. Simply register, choose the merchant name from the site's list or submit the name of a new merchant, and answer seven questions. Scores are averaged, and the store is rated between 0 and 7.

If you register again using a different name and e-mail address, you can apparently vote as often as you like. We were able to review a fictional company three separate times. (The site removed this data after we notified it.) And in fairness, it's tough to weed out all fake submissions from clever spoofers.

Voter fraud

However, not all vote fraud goes undetected. According to Scott Wainner, founder and site manager of Reseller Ratings, the site caught employees of one vendor stuffing the ballot box--and awarded that vendor a zero rating. Wainner says this behavior is rare, but adds that the site is undergoing a redesign that includes "a more effective system of checks and balances." It should be complete by June.

Perhaps the most balanced approach is to provide both editorial reviews and consumer surveys. Gomez Advisors rates more than 600 sites based on objective criteria, such as payment options, on a scale of 1 to 10. For each of the rated vendors, Gomez also provides a survey that users can fill out, anonymously; results are combined to form a separate consumer rating of poor, acceptable, or excellent.

We were able to comment on one e-tailer several times by hitting the browser's Back button and resubmitting the same survey. This repetition had no effect on the vendor's objective score but it did change the consumer rating.

"Our feeling is that consumer ratings systems can be prone to bias due to the ability of determined people to spoof them," says Gomez representative Julie Ward.

Like BizRate, Gomez makes money by selling data to e-merchants and by generating customer leads from sites offering deals to its members.

RatingWonders uses objective criteria to assess over 15,000 stores (using a system of one to five lamps), but consumers gauge the store's actual performance. For example, PC superstore Access Micro gets four lamps for overall shopping features, but consumers give it only two and a half lamps. The site accepts no commissions from stores; it makes money by licensing its data to shopping portals and from minimal advertising.

As with Gomez, we submitted several surveys on one company, from one address, in a single session. But only one of the comments was posted. Fahad Syed, vice president of RatingWonders, says the site usually accepts only one review per person per day.

Browser beware

Using an e-commerce rating site is better than shopping blind. But based on what we found, you should compare merchants across several ratings sites. Caveat emptor applies more than ever -- not just to Web retailers, but also to the sites that rate them.

Privacy fears prompt study, delay
May 22, 2000
Net privacy law costs a bundle
May 16, 2000
Rewriting the fourth amendment
May 12, 2000
Online child privacy act proves problematic for sites
April 25, 2000
More cops on the Net beat? Privacy groups say not so fast
April 10, 2000
Take Net privacy into your own hands
April 9, 1999
Internet crime report irks privacy groups
March 13, 2000
KidSafe is no choice at all
February 2, 2000

Chart: Behind the scenes at rating sites
(PC World Online)
Instant messaging: Good for e-commerce?
(Computerworld)
Privacy 2000: In Web we trust?
(PC World Online)
The best online brokers
(PC World Online)
Special Report: The Net 21
(The Industry Standard)
FTC backs its online privacy report
(The Industry Standard)
Targeting the consumer
(The Industry Standard)
Deja.com boosts buyers' guide
(PC World Online)

BizRate
Shop Online 123
Gomez Advisors
RatingWonders

Note: Pages will open in a new browser window

External sites are not endorsed by CNN Interactive.