Trying to make sense of multi-farm databases
Hello all, I've noticed a change introduced to the release version in Jun 2016 that I can't find a bugzilla bug or a detailed comment for, and I wish to understand a little better: multi-farm DBs, or multi-farm BAT buffer pools if you will. Questions: * Why do we have them? * Are BAT "roles", introduced with the patch for this issue, inherently relevant only with multiple DB farms? If not, why were they introduced at the same time and with the same changeset? * Is this considered a "good idea" design-wise, or perhaps an unfortunate necessity (e.g. due to client needs)? * When you create a new DB, is there a chance more than a single DB farm will be created when you haven't specifically requested this happen? Eyal
To clarify/correct: I mean this changeset https://github.com/MonetDB/MonetDB/commit/a251c8a29e23aad62b19ce9550d2f19b94... (so not sure it was merged in the Jul 2016 release.) Also, I remember BATs could be persistent or transient, but I'm not sure that's exactly what this new role field is supposed to mean. On 7/21/17 12:05 PM, Eyal Rozenberg wrote:
Hello all,
I've noticed a change introduced to the release version in Jun 2016 that I can't find a bugzilla bug or a detailed comment for, and I wish to understand a little better: multi-farm DBs, or multi-farm BAT buffer pools if you will.
Questions:
* Why do we have them? * Are BAT "roles", introduced with the patch for this issue, inherently relevant only with multiple DB farms? If not, why were they introduced at the same time and with the same changeset? * Is this considered a "good idea" design-wise, or perhaps an unfortunate necessity (e.g. due to client needs)? * When you create a new DB, is there a chance more than a single DB farm will be created when you haven't specifically requested this happen?
Eyal _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list
From `man mserver5` resp. https://www.monetdb.org/Documentation/mserver5-man-page : " −−dbpath=path Path where mserver5 should find a database. Shorthand for option gdk_dbpath. Default value: /var/monetdb5/dbfarm/demo.
−−dbextra=path Path where mserver5 should store transient data. Default value is the value of the −−dbpath option. " In other word, one can use the "dbextra" farm to "store" transient data (once they (need to) get spilled out of memory because their aggregated size exceed physical memory) in a different location/folder/partition/device than the persistent data (in "dbpath"). E.g. a potential scenario is to have the "dbpath" on a (large) HDD (-RAID), and the "dbextra" on a (usually smaller but faster) SSD (or NVMe) (-RAID) ... Stefan ----- On Jul 21, 2017, at 12:58 PM, Eyal Rozenberg E.Rozenberg@cwi.nl wrote:
To clarify/correct: I mean this changeset https://github.com/MonetDB/MonetDB/commit/a251c8a29e23aad62b19ce9550d2f19b94...
(so not sure it was merged in the Jul 2016 release.)
Also, I remember BATs could be persistent or transient, but I'm not sure that's exactly what this new role field is supposed to mean.
On 7/21/17 12:05 PM, Eyal Rozenberg wrote:
Hello all,
I've noticed a change introduced to the release version in Jun 2016 that I can't find a bugzilla bug or a detailed comment for, and I wish to understand a little better: multi-farm DBs, or multi-farm BAT buffer pools if you will.
Questions:
* Why do we have them? * Are BAT "roles", introduced with the patch for this issue, inherently relevant only with multiple DB farms? If not, why were they introduced at the same time and with the same changeset? * Is this considered a "good idea" design-wise, or perhaps an unfortunate necessity (e.g. due to client needs)? * When you create a new DB, is there a chance more than a single DB farm will be created when you haven't specifically requested this happen?
Eyal _______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list
_______________________________________________ developers-list mailing list developers-list@monetdb.org https://www.monetdb.org/mailman/listinfo/developers-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
participants (2)
-
Eyal Rozenberg -
Stefan Manegold