Thanks! The point to MAL plan really helps me out! Best, Guodong On Sep 21, 2020, 06:00 -0400, users-list-request@monetdb.org, wrote:

Send users-list mailing list submissions to users-list@monetdb.org

To subscribe or unsubscribe via the World Wide Web, visit https://www.monetdb.org/mailman/listinfo/users-list or, via email, send a message with subject or body 'help' to users-list-request@monetdb.org

You can reach the person managing the list at users-list-owner@monetdb.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..."

Today's Topics:

1. Internals of intermediate results in MonetDB (Guodong Jin) 2. Re: Internals of intermediate results in MonetDB (Martin Kersten)

----------------------------------------------------------------------

Message: 1 Date: Sun, 20 Sep 2020 23:31:50 -0400 From: Guodong Jin To: users-list@monetdb.org Subject: Internals of intermediate results in MonetDB Message-ID: <932d01d5-a742-4f57-8e19-ba7599667e42@Spark> Content-Type: text/plain; charset="utf-8"

Hi all,

Thanks for your hard works on this great open source project. I?m a student newly working on MonetDB to implement some research ideas, and I want to understand internals of how MonetDB chooses to materialize and how it represents intermediate results. As I understand, table data and intermediate results in MonetDB are all stored as BATs, but I?m not sure if there are some special optimizations on BATs, rather than just array(s) of atoms. Here are two simple examples to elaborate my questions a little bit:

?? ?1.Given two tables,?Items(id, order, price, tax), and?Orders(id, discount). ?? ? ? ? ?Items ?? ? ? ? ? ? ? ? ?[I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11...] ?? ? ? ? ? ? ? ? ?[O3, O3, O3, O3, O3, O3, O3, O3, O3, O3, O4...] ?? ? ? ? ? ? ? ? ?[10, 11, 20, 15, 110, 80, 90, 12, 13, 88, 30...] ?? ? ? ? ? ? ? ? ?[0.1, 0.15, 0, 0.1, 0.1, 0.15, 0.11, 0.18, 0.10, 0.15, 0.20, ...] ?? ? ? ? ?Orders ?? ? ? ? ? ? ? ? ?[O1, O2, O3] ?? ? ? ? ? ? ? ? ?[0.8, 0.9, 0.95]

?? ? ? ? ?When we perform a join between these two tables on the field `order` (one-to-many relationship represented by the primary-foreign key, O3 matches with [I1, I2, ?, I10]),?in the output intermediate BATs of the join, will MonetDB duplicate `O3` 10 times? Are there any optimizations currently in MonetDB to remove/reduce the duplication?

?? ?2. Given a table?item(id, price, tax), when MonetDB performs the filter `price<10` on the table,?will MonetDB actually copy all matched tuples into intermediate BATs? Or just keep a list of OIDs of matched tuples as references?

Thanks!

Best, Guodong