Stanford researchers may have largest database

From...

By Brian Sullivan

(IDG) -- Experts at the Stanford Linear Accelerator Center (SLAC) at Stanford University believe they may have the biggest database in the world.

The database recently passed the 500TB mark, and "as far as I can see," that makes it the largest such repository in the world by far, said database manager Jacek Becla. The database began storing data in 1999.

IDG.net INFOCENTER Computerworld main page Computerworld's topic-oriented communities Computerworld's IT career center Free daily newsletters Features PC World.com Product Finder Mastering e-commerce by degrees Fast-track training puts nontechies in IT jobs Visit an IDG site Choose a site: IDG.net CIO Computerworld Darwin Dummies.com GamePro.com The Industry Standard ITWorld.com InfoWorld.com JavaWorld LinuxWorld Macworld Online Network World Fusion PC World Publish.com UnixInsider IDG.net search

The 500,000GB of data in the BaBar database, if printed out, would fill 1 billion books, according to a statement released by SLAC. That's nearly 60 times the number of books in the Library of Congress, the largest library in the world.

The database, which collects information about subatomic particle collisions, is used by 600 physicists from nine nations taking part in the BaBar research project, Becla said Tuesday. BaBar's goal is to understand the difference between matter and antimatter and how it shaped the universe. The project has adopted Babar, the elephant from the popular children's stories, as its mascot, although the name really comes from B-bar, a type of particle some of the scientists study.

Becla said each collision generates about 30KB of raw data. Not all of the collisions are recorded -- "only the interesting ones," he added.

Becla said that caring for all the data has posed a disaster recovery challenge. But, he said, the key to solving that problem is simple: back up everything.

Most of the data is stored as "read only," he said. The data is also backed up on tape almost as soon as it's stored.

BaBar's elephant-like memory is further aided by the various research groups around the world that are taking part in the project. Becla said each group backs up its own data, so if anything happened to BaBar, it could be reconstructed.

Becla said most of the database runs on CPUs from Sun Microsystems Inc. but SLAC has recently begun to invest in a number of Linux boxes. The project has used more than 100 servers spread over a number of different server farms.

The center and the BaBar project are funded by the U.S. Department of Energy.