Hi,
monet reads to the memory only the relevant to the query data. Since
monet is also a column store, it does not need to read the entire
table into memory but only the columns that are needed. As such, there
is *no* limit that the entire database must fit in memory. Moreover,
monet will continue working even if the relevant data of the query do
not fit in memory:) so in that sense monet is not "intended to be used
only when the data can fit in memory". Also notice that in memory will
be only the columns that participate on the current operator of a
query plan and not all columns of the entire plan.
As for the row IDs, which in monet terminology are called OIDs, can
also be 64 bit. Actually for the size of database you are talking
about, it is needed to be 64-bits. This can be done by compiling with
--oid-64 option or downloading the 64-bit version of the binary
distribution.
I hope I could help. Feel free to send email if you have any more questions.
Regards,
lefteris
On Wed, Jul 22, 2009 at 8:30 PM, Charles
Samuels
Hi,
I have read previously that monetdb is intended to be used only when the data can fit in memory. I'm not sure how to interpret this statement - does this mean that the entire database must fit in memory, or does it mean that only the data being operated on when selected from the database must fit in memory?
We are wanting to store *huge* amounts of data, ultimately several TBs. The actual data being select at any given time will only be several hundred MBs, however. Is this a reasonable usage of MonetDB?
But most importantly, I have noticed that all the row ID indexes are "int" - why the 32 bit limitation? This just seems like a symptom of the above problem; that the entire database must fit in memory.
Thanks a lot for your advice,
Charles
-- Charles Samuels
------------------------------------------------------------------------------ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users