--- Fabian
We have need for a 300 million recors DB of about 500 GB. We insert about 1000 records per seconds. or about 100 millions record per day, for 3 days.
wow Can you tell us which database on what OS on which hardware this runs?
Hehehe. We have needs for it... It doesn't exist yet. We have a dual amd opteron, linux 32 bits for now, 120 GB partition in a striped raid, 4 gigs of ram. Mysql 4.1.10 with InnoDB. The secret is in the extended insert syntax of mysql. Magnitude faster than jdbc batches. We fill our 90 gigs innodb file easily, but given we want to delete in batch once a day, the delete takes like hours. A permanent 'nibble' delete is working, but lowers the insert speed in the peek of the day. So we batch-delete by night.
Last question, lucky 7: 7-Do you think MonetDB is at all a good choice for our high-thoughput highly volatile data?
I think you would seriously need horizontal partitioning which will be available in the next generation MonetDB kernel, to get around some problems both memory and delete wise.
If it cannot be supported transparently, I won't pursue it. We already have another prototype with multi-table partitionning. It is a bit slower on queries though. Thanks Fabian. ____________________________________________________ Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs