Re: Costly bat.new/bat.append not optimized out

Sorry, I mixed up the names. What i meant is that X_3 seems the result container created by the union, and X_19 seems the column of a (or a view a view of it). If X_19 were really the column of a, then you could not reuse it. Most probably though X_19 is the result of the original column and the delta bats (updates,deletions). In that case i guess that yes, it could be reused as you say, but the pattern is not detected by any optimizer. I'd advise you to post a bit more complete MAL subplan, then it will be easier for the developers to help you. Roberto On 22 Aug 2014 21:18, "Frédéric Jolliton" < frederic.jolliton+monetdb@securactive.net> wrote:

Hi Roberto,

I wrote:

...

...

X_3:bat[:oid,:lng] =[0] := bat.new(nil:oid,nil:lng); .. # X_3 is never used before this line. X_19=[23068647] := <some computation> X_20=[23068647] :=

bat.append(X_3=:bat[:oid,:lng][23068647],

...

X_19=[23068647],true);

...

# X_3 is never used after this line.

Roberto Cornacchia writes:

...

Unless I misunderstand the problem, I think you can't just use X_3 because that is the bat that corresponds to the physical column of your table a. So you can't take it and append b to it. Union creates a copy of a, then appends b, then c.

Thanks for your reply.

No, here X_3 is a new BAT (it is the result of bat.new).

The BAT from a physical column come from call to sql.bind.

But I'm new to MonetDB, so I might be missing some details too.

The value between brackets ([]) indicate the number of rows after the call, and we can see that at first X_3 is empty (as expected), that X_19 contains a lot of rows (23M). And then this one is concatened to the empty one. As a result of bat.append, we get the same BAT referenced by X_3 (we see its internal name between <> as temp_321264), not some another BAT created by the call itself.

Moreover, what I didn't specify, but was implied in my original post, is that X_19 is no longer used after the call to bat.append. We see that X_3 become of copy of X_19, and it replaces it entirely. This is damageable for the performance.

-- Frédéric Jolliton Sécuractive _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list