I have a question about remote.exec(). Suppose multiple (say N) remote machines call remote.exec() to start a function on a given local machine. What happens if N is greater than the number of threads supported by the local machine? Are the remote.execs queued until there is a thread available to execute them, or are they simply not started? If the letter, how can one tell whether a remote.exec was run? I have another question abut remote.get() transferring a BAT from one machine to another. Is there a limit on the size of a BAT that can be transferred by a single call to remote.get()? If the BAT is very large, is it paged through disk, or is it alsways transferred directly from memory to memory? Thanks, Steve Morgan
On 24-07-2013 10:17:58 -0700, Stephen P. Morgan wrote:
I have a question about remote.exec(). Suppose multiple (say N) remote machines call remote.exec() to start a function on a given local machine. What happens if N is greater than the number of threads supported by the local machine? Are the remote.execs queued until there is a thread available to execute them, or are they simply not started? If the letter, how can one tell whether a remote.exec was run?
Don't know exactly what you're doing, but you likely see connection errors (no free client slots) if you try to connect with more connections than can be served.
I have another question abut remote.get() transferring a BAT from one machine to another. Is there a limit on the size of a BAT that can be transferred by a single call to remote.get()? If the BAT is very large, is it paged through disk, or is it alsways transferred directly from memory to memory?
remote.get() works as if you're doing bat.new() and bat.insert(), but slightly more optimised, using a binary transport if possible. The limits here are defined by the system. Fabian
Hi all, We're hitting this error "b and g must be aligned". I tracked the src to a commit about some alignment code thing in gdk_calc: http://www.mail-archive.com/checkin-list@monetdb.org/msg09731.html (Fix alignment conversion in compatibility code for grouped aggregates.) Can you guys please explain what's the reason behind this error? I can't understand by just looking to the src of gdk_calc.c Thanks!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2013-07-25 14:04, Miguel Ping wrote:
Hi all,
We're hitting this error "b and g must be aligned". I tracked the src to a commit about some alignment code thing in gdk_calc: http://www.mail-archive.com/checkin-list@monetdb.org/msg09731.html (Fix alignment conversion in compatibility code for grouped aggregates.)
Can you guys please explain what's the reason behind this error? I can't understand by just looking to the src of gdk_calc.c
Thanks!
When using grouped aggregates, the grouping bat must be aligned with the value bat. The value bat is b and contains the values you want to aggregate. The group bat is g and contains for each value in b the group (an oid) it belongs to. Equal group ids means the same group. These bats must be aligned, because we need to know for each value in b to which group it belongs. Aligned means: same length, and same head column values. The head columns must be dense (a sequence of numbers starting at some value, and each next value exactly one larger than the previous). Dense sequences are usually not stored explicitly in MonetDB. We only store the first value in the hseqbase field. So the hseqbase fields of b and g must be equal. The one exception to this is when the bats are both empty. This last exception is the change to gdk_calc.c in that changeset. - -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUfEfQj7g04AjvIQpAQI2ZwQAh7JLNlArqhQhY07k+N/Cxgv21p0YiRGu DwZ9T4tRZMcgqjtnK6IdMZ9frFon8OXHNQ1PIyHP/waz0tpxIlygnc9OWfCqLtE1 K2pafzsLFxz1dxKBI+FtDg266tLNEEvgFXqtJhY8IVbTDOLzFe52MYUY7VnBCwEv 4vu+aR5Irn4= =65VI -----END PGP SIGNATURE-----
On 07/25/2013 01:51 PM, Sjoerd Mullender wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi all,
We're hitting this error "b and g must be aligned". I tracked the src to a commit about some alignment code thing in gdk_calc: http://www.mail-archive.com/checkin-list@monetdb.org/msg09731.html (Fix alignment conversion in compatibility code for grouped aggregates.)
Can you guys please explain what's the reason behind this error? I can't understand by just looking to the src of gdk_calc.c
Thanks! When using grouped aggregates, the grouping bat must be aligned with
On 2013-07-25 14:04, Miguel Ping wrote: the value bat. The value bat is b and contains the values you want to aggregate. The group bat is g and contains for each value in b the group (an oid) it belongs to. Equal group ids means the same group. These bats must be aligned, because we need to know for each value in b to which group it belongs. Aligned means: same length, and same head column values. The head columns must be dense (a sequence of numbers starting at some value, and each next value exactly one larger than the previous). Dense sequences are usually not stored explicitly in MonetDB. We only store the first value in the hseqbase field. So the hseqbase fields of b and g must be equal. The one exception to this is when the bats are both empty. This last exception is the change to gdk_calc.c in that changeset.
- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQCVAwUBUfEfQj7g04AjvIQpAQI2ZwQAh7JLNlArqhQhY07k+N/Cxgv21p0YiRGu DwZ9T4tRZMcgqjtnK6IdMZ9frFon8OXHNQ1PIyHP/waz0tpxIlygnc9OWfCqLtE1 K2pafzsLFxz1dxKBI+FtDg266tLNEEvgFXqtJhY8IVbTDOLzFe52MYUY7VnBCwEv 4vu+aR5Irn4= =65VI -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Thanks for your explanation. I still don't understand how can there be a misalignment. I would expect MonetDB to feed my hll aggregate functions with the correct values. Can it be that I may have some NULL values and the validation is failing because of that? -- Miguel
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2013-07-26 12:23, Miguel Ping wrote:
On 07/25/2013 01:51 PM, Sjoerd Mullender wrote: On 2013-07-25 14:04, Miguel Ping wrote:
Hi all,
We're hitting this error "b and g must be aligned". I tracked the src to a commit about some alignment code thing in gdk_calc: http://www.mail-archive.com/checkin-list@monetdb.org/msg09731.html
(Fix alignment conversion in compatibility code for grouped
aggregates.)
Can you guys please explain what's the reason behind this error? I can't understand by just looking to the src of gdk_calc.c
Thanks! When using grouped aggregates, the grouping bat must be aligned with the value bat. The value bat is b and contains the values you want to aggregate. The group bat is g and contains for each value in b the group (an oid) it belongs to. Equal group ids means the same group. These bats must be aligned, because we need to know for each value in b to which group it belongs. Aligned means: same length, and same head column values. The head columns must be dense (a sequence of numbers starting at some value, and each next value exactly one larger than the previous). Dense sequences are usually not stored explicitly in MonetDB. We only store the first value in the hseqbase field. So the hseqbase fields of b and g must be equal. The one exception to this is when the bats are both empty. This last exception is the change to gdk_calc.c in that changeset.
-- Sjoerd Mullender
_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Thanks for your explanation. I still don't understand how can there be a misalignment. I would expect MonetDB to feed my hll aggregate functions with the correct values. Can it be that I may have some NULL values and the validation is failing because of that?
If this happens when running a SQL query, it's a bug. I don't think NULLs have anything to do with it. NULL values are stored in-line. You might want to look at b->U->count, g->U->count, b->H->seq, g->H->seq, b->H->dense, g->H->dense when the misalignment happens (either in the debugger or by using printf--but realize that count and seq are not int, so %d is not going to work). Also things like the MAL plan (prepend SQL query with EXPLAIN) and the stack trace might be useful. - -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUfJQyT7g04AjvIQpAQK/JAP9HCp/aFaYWv0jodfPUnSRVgFSsdjTn/VL ttSsmAF+yomGMDIne2311f/D51F3/nte7Utx+01lgvArapWErhjGN1hzPSr5LQbs PZ6dUfNcH8Rt2AtT3uSxfkFZy9VRDCNNXPei43IgMS2HxVZ48pnAVkNcpBbW3Gms GwdvU7bSZtM= =2zSg -----END PGP SIGNATURE-----
participants (4)
-
Fabian Groffen -
Miguel Ping -
Sjoerd Mullender -
Stephen P. Morgan