Re: b and g must be aligned

Some more info:

SELECT count(*) FROM wa_sapo_pt_audience.kpi_2013_07 WHERE ts>=1373410800000 AND ts<=1374706799000; ==> 764314

SELECT count(distinct(ts_day)) FROM wa_sapo_pt_audience.kpi_2013_07 WHERE ts>=1373410800000 AND ts<=1374706799000; ==> 15

it seems that b.count is the row count, while g.count is something like the distinct count, with 2 more values? On 08/12/2013 05:50 PM, Miguel Ping wrote:

Hi, I'm resurrecting this since I've been out of town and only today I got a chance to investigate further. I've recompiled with -O0 to prevent optimizations from "hiding" the values, and in the debugger I got this:

b->U->count BUN 764314 g->U->count BUN 17 b->H->seq oid 0 g->H->seq oid 0 b->H->dense unsigned int 1 g->H->dense unsigned int 1

The stack call is as follows:

BAThllaggr() at udf.c AGGRsubhllaggcand() at udf.c AGGRsubhllagg() at udf.c malCommandCall() at mal_interpreter.c runMALsequence() at mal_interpreter.c DFLOWworker() at mal_dataflow.c start_thread() at pthread_create.c clone() at clone.S 0x0

-------- Original Message -------- Subject: Re: b and g must be aligned Date: Fri, 26 Jul 2013 11:34:53 +0100 From: Sjoerd Mullender Reply-To: Communication channel for MonetDB users To: Communication channel for MonetDB users

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On 2013-07-26 12:23, Miguel Ping wrote:

...

On 07/25/2013 01:51 PM, Sjoerd Mullender wrote: On 2013-07-25 14:04, Miguel Ping wrote:

...

...

...

Hi all,

We're hitting this error "b and g must be aligned". I tracked the src to a commit about some alignment code thing in gdk_calc: http://www.mail-archive.com/checkin-list@monetdb.org/msg09731.html

(Fix alignment conversion in compatibility code for grouped

...

aggregates.)

Can you guys please explain what's the reason behind this error? I can't understand by just looking to the src of gdk_calc.c

Thanks! When using grouped aggregates, the grouping bat must be aligned with the value bat. The value bat is b and contains the values you want to aggregate. The group bat is g and contains for each value in b the group (an oid) it belongs to. Equal group ids means the same group. These bats must be aligned, because we need to know for each value in b to which group it belongs. Aligned means: same length, and same head column values. The head columns must be dense (a sequence of numbers starting at some value, and each next value exactly one larger than the previous). Dense sequences are usually not stored explicitly in MonetDB. We only store the first value in the hseqbase field. So the hseqbase fields of b and g must be equal. The one exception to this is when the bats are both empty. This last exception is the change to gdk_calc.c in that changeset.

-- Sjoerd Mullender

...

_______________________________________________ users-list mailing listusers-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Thanks for your explanation. I still don't understand how can there be a misalignment. I would expect MonetDB to feed my hll aggregate functions with the correct values. Can it be that I may have some NULL values and the validation is failing because of that?

If this happens when running a SQL query, it's a bug. I don't think NULLs have anything to do with it. NULL values are stored in-line. You might want to look at b->U->count, g->U->count, b->H->seq, g->H->seq, b->H->dense, g->H->dense when the misalignment happens (either in the debugger or by using printf--but realize that count and seq are not int, so %d is not going to work). Also things like the MAL plan (prepend SQL query with EXPLAIN) and the stack trace might be useful.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird -http://www.enigmail.net/

iQCVAwUBUfJQyT7g04AjvIQpAQK/JAP9HCp/aFaYWv0jodfPUnSRVgFSsdjTn/VL ttSsmAF+yomGMDIne2311f/D51F3/nte7Utx+01lgvArapWErhjGN1hzPSr5LQbs PZ6dUfNcH8Rt2AtT3uSxfkFZy9VRDCNNXPei43IgMS2HxVZ48pnAVkNcpBbW3Gms GwdvU7bSZtM= =2zSg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list