Re: b and g must be aligned

Hi Miguel, If it's considered a bug in MonetDB, and unless you've already done so, please file a bug report including all information (data and code as necessary) required to reproduce the problem. Thanks! Stefan ----- Original Message -----

The query calls a custom aggregate function, but the error occurs before hitting my code; the code path just starts to prepare things (it's just the boilerplate to run a custom aggr function), and it hits the error while calling BATgroupaggrinit. In fact, BATgroupaggrinit is the very first thing that the BAThllaggr function calls; Also according to Sjoerd, it's a bug:

"If this happens when running a SQL query, it's a bug. I don't think NULLs have anything to do with it. NULL values are stored in-line. You might want to look at b->U->count, g->U->count, b->H->seq, g->H->seq, b->H->dense, g->H->dense when the misalignment happens (either in the debugger or by using printf--but realize that count and seq are not int, so %d is not going to work). Also things like the MAL plan (prepend SQL query with EXPLAIN) and the stack trace might be useful."

Thanks.

On 08/13/2013 08:51 AM, Stefan Manegold wrote:

...

Dear Miguel,

I am not aware of any "hllaggr()" function in the MonetDB release, so I assume the error occurs in your code. Not know your code at all, I'm afraid we cannot be of much help.

Best, Stefan

On Mon, Aug 12, 2013 at 06:59:33PM +0100, Miguel Ping wrote:

...

Some more info:

...

SELECT count(*) FROM wa_sapo_pt_audience.kpi_2013_07 WHERE ts>=1373410800000 AND ts<=1374706799000; ==> 764314

...

SELECT count(distinct(ts_day)) FROM wa_sapo_pt_audience.kpi_2013_07 WHERE ts>=1373410800000 AND ts<=1374706799000; ==> 15

it seems that b.count is the row count, while g.count is something like the distinct count, with 2 more values?

On 08/12/2013 05:50 PM, Miguel Ping wrote:

...

Hi, I'm resurrecting this since I've been out of town and only today I got a chance to investigate further. I've recompiled with -O0 to prevent optimizations from "hiding" the values, and in the debugger I got this:

b->U->count BUN 764314 g->U->count BUN 17 b->H->seq oid 0 g->H->seq oid 0 b->H->dense unsigned int 1 g->H->dense unsigned int 1

The stack call is as follows:

BAThllaggr() at udf.c AGGRsubhllaggcand() at udf.c AGGRsubhllagg() at udf.c malCommandCall() at mal_interpreter.c runMALsequence() at mal_interpreter.c DFLOWworker() at mal_dataflow.c start_thread() at pthread_create.c clone() at clone.S 0x0

-------- Original Message -------- Subject: Re: b and g must be aligned Date: Fri, 26 Jul 2013 11:34:53 +0100 From: Sjoerd Mullender Reply-To: Communication channel for MonetDB users To: Communication channel for MonetDB users

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On 2013-07-26 12:23, Miguel Ping wrote:

...

On 07/25/2013 01:51 PM, Sjoerd Mullender wrote: On 2013-07-25 14:04, Miguel Ping wrote:

...

>> Hi all, >> >> We're hitting this error "b and g must be aligned". I tracked >> the src to a commit about some alignment code thing in >> gdk_calc: >> http://www.mail-archive.com/checkin-list@monetdb.org/msg09731.html >> >> (Fix alignment conversion in compatibility code for grouped >> aggregates.) >> >> Can you guys please explain what's the reason behind this >> error? I can't understand by just looking to the src of >> gdk_calc.c >> >> Thanks! When using grouped aggregates, the grouping bat must be aligned with the value bat. The value bat is b and contains the values you want to aggregate. The group bat is g and contains for each value in b the group (an oid) it belongs to. Equal group ids means the same group. These bats must be aligned, because we need to know for each value in b to which group it belongs. Aligned means: same length, and same head column values. The head columns must be dense (a sequence of numbers starting at some value, and each next value exactly one larger than the previous). Dense sequences are usually not stored explicitly in MonetDB. We only store the first value in the hseqbase field. So the hseqbase fields of b and g must be equal. The one exception to this is when the bats are both empty. This last exception is the change to gdk_calc.c in that changeset.

-- Sjoerd Mullender

...

_______________________________________________ users-list mailing listusers-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

Thanks for your explanation. I still don't understand how can there be a misalignment. I would expect MonetDB to feed my hll aggregate functions with the correct values. Can it be that I may have some NULL values and the validation is failing because of that? If this happens when running a SQL query, it's a bug. I don't think NULLs have anything to do with it. NULL values are stored in-line. You might want to look at b->U->count, g->U->count, b->H->seq, g->H->seq, b->H->dense, g->H->dense when the misalignment happens (either in the debugger or by using printf--but realize that count and seq are not int, so %d is not going to work). Also things like the MAL plan (prepend SQL query with EXPLAIN) and the stack trace might be useful.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird -http://www.enigmail.net/

iQCVAwUBUfJQyT7g04AjvIQpAQK/JAP9HCp/aFaYWv0jodfPUnSRVgFSsdjTn/VL ttSsmAF+yomGMDIne2311f/D51F3/nte7Utx+01lgvArapWErhjGN1hzPSr5LQbs PZ6dUfNcH8Rt2AtT3uSxfkFZy9VRDCNNXPei43IgMS2HxVZ48pnAVkNcpBbW3Gms GwdvU7bSZtM= =2zSg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |