Real World Numbers #2 "Big DB"
A big fast database
Here are some numbers from a recent custom version of the MPbase
technology.
Operations:
- 7/24 operations, no need for any downtime,
not even for OS upgrades, reorganizational unload/reloads or hardware
fixes.
- "Dark room" hardware mode,
no one ever needed to look at the cluster except to replace broken
hardware.
- No single point of failure,
any switch on the cluster could be shut off and not
bring down the database.
- System feeds MVS, UNIX, Windows and HTML (Web access),
at same time from same data, EBCDIC, ASCII, straight or swapped
byte order.
Hardware:
- 30 - 110Mhz SPARC 5 servers (3.3 Bips total),
- 390 - 2.1 Gbyte 7200 RPM SCSI disk drives. (819 Gbytes total),
- 30 - 40 Mbytes/sec fast & wide SCSI 2 channels (1200 Mbytes/sec
total),
- 30 - 20 Mbytes/sec fast SCSI 2 channels (600 Mbytes/sec total),
- 30 - 100 Mbit/sec fast ethernet segments, switched (375 Mbytes/sec
total),
- 5 - 10 Mbit/sec ethernet segments, switched (6.25 Mbytes/sec
total),
- 5 - 7 foot high cabinets (only 20 square feet of floor space).
Data:
- 50+ billion records (initial load),
- 200+ billion records (predicted final size),
- 93% compression from 52 byte transfer format,
- 85% compression including the mirror copy of the data,
- compressed data 320 Gbytes with mirror,
- uncompressed data 2 Tbytes for one copy.
Software:
- 126 executables: 74 C shell scripts, 33 PERL scripts, 17 C
programs,
- 1500+ automated daemons with queues.
Performance:
- Internal validation rate - 16 billion rows/hour, 267M rows/min,
4.4M rows/sec,
- Sequential scan rate - - - 16 billion rows/hour, 267M rows/min,
4.4M rows/sec,
- Random extract rate - - - 90K rows/sec at a 25% hit rate (400K
keys per second),
6M rows/min, 360M rows/hour, 8.6 billion rows/day,
- Max load rate (ext) - - - - 8.6 billion rows/day, 360M rows/hour,
6M rows/min, 100K rows/sec,
Unload/reload rate (int) - 24 billion rows/day, 1 billion rows/hour,16M
rows/min, 277K rows/sec.