Is there any scan sharing technique in MonetDB?
Hi all, I understand that MonetDB would recycle intermediate results to accelerate subsequent OLAP queries that involve common sub-queries as previous ones. I am wondering that, 1. Does MonetDB implement any scan sharing techniques (or generally, multi-query optimization techniques) in the context of concurrent workloads? For example, two concurrent queries Q1 and Q2 both require scanning column X. Is it possible for their query plans to share the scan operation? 2. If no, what is the reason? too complex to implement? or any reason related to MonetDB's execution paradigm that prevents such sharing techniques? Any comments would be helpful, thanks. Best regards, Wenjian
Hi Wenjian, ----- On Jul 7, 2016, at 10:47 AM, Xu,Wenjian zeroxwj@gmail.com wrote:
Hi all,
I understand that MonetDB would recycle intermediate results to accelerate subsequent OLAP queries that involve common sub-queries as previous ones.
I'm afraid the "recycling" work unfortunately never made it into any released version of MonetDB. Only indices (hash, imprints) built on base columns will be recycled (hashes for now only as long as the server runs, imprints also across server re-starts).
I am wondering that,
1. Does MonetDB implement any scan sharing techniques (or generally, multi-query optimization techniques) in the context of concurrent workloads? For example, two concurrent queries Q1 and Q2 both require scanning column X. Is it possible for their query plans to share the scan operation?
No. MonetDB currently does not perform any multi-query optimization or plan sharing other than re-using/sharing indices built on base columns (see above).
2. If no, what is the reason? too complex to implement? or any reason related to MonetDB's execution paradigm that prevents such sharing techniques?
Reason is: nobody ever had the time to investigate whether/how MonetDB could benefit from this, and if so, to implement it in a robust "production-ready" way. Best, Stefan
Any comments would be helpful, thanks.
Best regards, Wenjian
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
Thanks for your kind reply, Prof. Manegold.
Best regards,
Wenjian
On Thu, Jul 7, 2016 at 5:05 PM, Stefan Manegold
Hi Wenjian,
----- On Jul 7, 2016, at 10:47 AM, Xu,Wenjian zeroxwj@gmail.com wrote:
Hi all,
I understand that MonetDB would recycle intermediate results to accelerate subsequent OLAP queries that involve common sub-queries as previous ones.
I'm afraid the "recycling" work unfortunately never made it into any released version of MonetDB. Only indices (hash, imprints) built on base columns will be recycled (hashes for now only as long as the server runs, imprints also across server re-starts).
I am wondering that,
1. Does MonetDB implement any scan sharing techniques (or generally, multi-query optimization techniques) in the context of concurrent workloads? For example, two concurrent queries Q1 and Q2 both require scanning column X. Is it possible for their query plans to share the scan operation?
No. MonetDB currently does not perform any multi-query optimization or plan sharing other than re-using/sharing indices built on base columns (see above).
2. If no, what is the reason? too complex to implement? or any reason related to MonetDB's execution paradigm that prevents such sharing techniques?
Reason is: nobody ever had the time to investigate whether/how MonetDB could benefit from this, and if so, to implement it in a robust "production-ready" way.
Best, Stefan
Any comments would be helpful, thanks.
Best regards, Wenjian
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Stefan Manegold -
Xu,Wenjian