monet crashes with query to VERY large table
Hello, I currently have a table with 307524407 rows and 52 columns, it was loaded from a 78GB cvs file, but I do not know how much space the table actually takes on disk. If I query the table with a simple query like: select count(*) from big_table where field =1; mserver5 crashes with ‘segmentation fault' I tried also the following query (adding the LIMIT statement): select count(*) from farmap_movimento where farmacia_id =1 limit 10; and the output from mserver5 was: mserver5: gdk_select.c:869: fullscan_int: Assertion `cnt < (bn)->S->capacity' failed. Aborted Doing a more complex query (the one I am aiming for), with also a group by and a few aggregation functions (sum, min) on the fields, the outputs is: mserver5: gdk_bat.c:2066: BATsetcount: Assertion `b->S->capacity >= cnt' failed. Aborted This seems to be a table size problem, so…is there anything I can do? I am using MonetDB 5 server v11.22.0 The machine has 30GB of RAM and 60GB of swap. Available disk space: ~50GB Any help would be much appreciated, Regards, Stefano
hi, any chance these data are public? if not, can you test on a separate
box and see if the exact same thing happens or if it's hardware-specific?
On Thursday, July 14, 2016, Stefano Fioravanzo
Hello,
I currently have a table with 307524407 rows and 52 columns, it was loaded from a 78GB cvs file, but I do not know how much space the table actually takes on disk. If I query the table with a simple query like:
select count(*) from big_table where field =1;
mserver5 crashes with ‘segmentation fault' I tried also the following query (adding the LIMIT statement): select count(*) from farmap_movimento where farmacia_id =1 limit 10;
and the output from mserver5 was: mserver5: gdk_select.c:869: fullscan_int: Assertion `cnt < (bn)->S->capacity' failed. Aborted
Doing a more complex query (the one I am aiming for), with also a group by and a few aggregation functions (sum, min) on the fields, the outputs is: mserver5: gdk_bat.c:2066: BATsetcount: Assertion `b->S->capacity
= cnt' failed. Aborted
This seems to be a table size problem, so…is there anything I can do?
I am using MonetDB 5 server v11.22.0 The machine has 30GB of RAM and 60GB of swap. Available disk space: ~50GB
Any help would be much appreciated,
Regards,
Stefano
_______________________________________________ users-list mailing list users-list@monetdb.org javascript:; https://www.monetdb.org/mailman/listinfo/users-list
Hello, No the data is not public, and testing in on another machine is not an easy option right now. Regards, Stefano
On 14 Jul 2016, at 15:49, Anthony Damico
wrote: hi, any chance these data are public? if not, can you test on a separate box and see if the exact same thing happens or if it's hardware-specific?
On Thursday, July 14, 2016, Stefano Fioravanzo
mailto:fioravanzos@gmail.com> wrote: Hello, I currently have a table with 307524407 rows and 52 columns, it was loaded from a 78GB cvs file, but I do not know how much space the table actually takes on disk. If I query the table with a simple query like:
select count(*) from big_table where field =1;
mserver5 crashes with ‘segmentation fault' I tried also the following query (adding the LIMIT statement): select count(*) from farmap_movimento where farmacia_id =1 limit 10;
and the output from mserver5 was: mserver5: gdk_select.c:869: fullscan_int: Assertion `cnt < (bn)->S->capacity' failed. Aborted
Doing a more complex query (the one I am aiming for), with also a group by and a few aggregation functions (sum, min) on the fields, the outputs is: mserver5: gdk_bat.c:2066: BATsetcount: Assertion `b->S->capacity >= cnt' failed. Aborted
This seems to be a table size problem, so…is there anything I can do?
I am using MonetDB 5 server v11.22.0 The machine has 30GB of RAM and 60GB of swap. Available disk space: ~50GB
Any help would be much appreciated,
Regards,
Stefano
_______________________________________________ users-list mailing list users-list@monetdb.org javascript:; https://www.monetdb.org/mailman/listinfo/users-list https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
can you winnow down the problem enough to construct a reproducible example?
thanks
On Thursday, July 14, 2016, Stefano Fioravanzo
Hello,
No the data is not public, and testing in on another machine is not an easy option right now.
Regards,
Stefano
On 14 Jul 2016, at 15:49, Anthony Damico
javascript:_e(%7B%7D,'cvml','ajdamico@gmail.com');> wrote: hi, any chance these data are public? if not, can you test on a separate box and see if the exact same thing happens or if it's hardware-specific?
On Thursday, July 14, 2016, Stefano Fioravanzo
javascript:_e(%7B%7D,'cvml','fioravanzos@gmail.com');> wrote: Hello,
I currently have a table with 307524407 rows and 52 columns, it was loaded from a 78GB cvs file, but I do not know how much space the table actually takes on disk. If I query the table with a simple query like:
select count(*) from big_table where field =1;
mserver5 crashes with ‘segmentation fault' I tried also the following query (adding the LIMIT statement): select count(*) from farmap_movimento where farmacia_id =1 limit 10;
and the output from mserver5 was: mserver5: gdk_select.c:869: fullscan_int: Assertion `cnt < (bn)->S->capacity' failed. Aborted
Doing a more complex query (the one I am aiming for), with also a group by and a few aggregation functions (sum, min) on the fields, the outputs is: mserver5: gdk_bat.c:2066: BATsetcount: Assertion `b->S->capacity
= cnt' failed. Aborted
This seems to be a table size problem, so…is there anything I can do?
I am using MonetDB 5 server v11.22.0 The machine has 30GB of RAM and 60GB of swap. Available disk space: ~50GB
Any help would be much appreciated,
Regards,
Stefano
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org javascript:_e(%7B%7D,'cvml','users-list@monetdb.org'); https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Anthony Damico
-
Stefano Fioravanzo