Has anyone configured an scalable MonetDB using Docker cluster on Amazon Web Services or other cloud environment? I'm looking into setting this up for one of our project around COBID-19 and essentially need to: - have my number of MonetDB instances to go up and down with user demand - use a load balancer to spread user queries across multiple instances I had in mind that all the instances would just point to a single dbfarm directory (all the user queries are read only), but is this an option or will cause problems (locks, caching, etc.)? Based on what I understand from Cluster Management (https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/ClusterManagement) or Lazy Replication (https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/WorkloadCaptureRe...), I need to replicate the database for each instance, which is not really practical. This document seems to hint that using the "Replication Services" model would be an option, but then I end up with any copies of the same database, and this will require a significant amount of extra disk space (the transaction logs + one copy per instance). http://www.exanest.eu/pub/2016_RTPBD_Monetdb.pdf Any guidance, suggestion, example would be greatly appreciated. best *P
Please check following links, as these seems to work for your scenarios.
https://www.monetdb.org/blog/remote-tables-auth
https://www.monetdb.org/blog/updatable-merge-tables
Regards,
Mubin Abbas
On Sun, May 17, 2020 at 2:46 AM Pascal Heus
Has anyone configured an scalable MonetDB using Docker cluster on Amazon Web Services or other cloud environment? I'm looking into setting this up for one of our project around COBID-19 and essentially need to: - have my number of MonetDB instances to go up and down with user demand - use a load balancer to spread user queries across multiple instances
I had in mind that all the instances would just point to a single dbfarm directory (all the user queries are read only), but is this an option or will cause problems (locks, caching, etc.)?
Based on what I understand from Cluster Management ( https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/ClusterManagement)
or Lazy Replication ( https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/WorkloadCaptureRe...),
I need to replicate the database for each instance, which is not really practical.
This document seems to hint that using the "Replication Services" model would be an option, but then I end up with any copies of the same database, and this will require a significant amount of extra disk space (the transaction logs + one copy per instance). http://www.exanest.eu/pub/2016_RTPBD_Monetdb.pdf
Any guidance, suggestion, example would be greatly appreciated.
best *P _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Mubin Abbas
-
Pascal Heus