Hi, I'm facing the problem of disk space running out.Since monetdb will store the query intermediates into the disk when the memory is not enough to hold them all, I'm wondering that does monetdb will automatically free the disk space? And if it doesn't, how can I do it manually?
On 24/09/15 06:04, scArcer wrote:
Hi, I'm facing the problem of disk space running out. Since monetdb will store the query intermediates into the disk when the memory is not enough to hold them all, I'm wondering that does monetdb will automatically free the disk space? And if it doesn't, how can I do it manually?
Hi It will release disk space as soon as possible.
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Here is my situation.
I create a table A, load about 3~4G data into it.
Then a lot of queries are executed, making it using disk up to 1T.
[hadoop@falco001-bi-30-141 data3]$ du -sh monetdb/
1.1T monetdb/
and when I try to execute new query, I got this error:
sql>select * from ods.attr_user_c_profile limit 10;
could not allocate space for
GDKsave: error -1 on: name=06/62/66261, ext=theap, mode=0
!OS: No space left on device
HEAPextend: failed to extend to 983040 for 06/62/66261.theap: h->storage == STORE_MEM && can_map && fd >= 0 && HEAPsave_intern() < 0
------------------ Original ------------------
From: "Martin Kersten";
Hi, I'm facing the problem of disk space running out. Since monetdb will store the query intermediates into the disk when the memory is not enough to hold them all, I'm wondering that does monetdb will automatically free the disk space? And if it doesn't, how can I do it manually?
Hi It will release disk space as soon as possible.
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
On 24/09/15 08:17, scArcer wrote:
Here is my situation.
To make any sense of your situation, you should provide the basic information, such as Version and OS platform.
I create a table A, load about 3~4G data into it. Then a lot of queries are executed, making it using disk up to 1T. select * from storage() tells you what the system knows.
Furthermore, analysis yourself which queries lead to a growing disk use. Also consider the database sql log files. good luck
[hadoop@falco001-bi-30-141 data3]$ du -sh monetdb/
1.1Tmonetdb/
and when I try to execute new query, I got this error:
sql>select * from ods.attr_user_c_profile limit 10;
could not allocate space for
GDKsave: error -1 on: name=06/62/66261, ext=theap, mode=0
!OS: No space left on device
HEAPextend: failed to extend to 983040 for 06/62/66261.theap: h->storage == STORE_MEM && can_map && fd >= 0 && HEAPsave_intern() < 0
------------------ Original ------------------ *From: * "Martin Kersten";
; *Date: * Thu, Sep 24, 2015 02:03 PM *To: * "users-list" ; *Subject: * Re: Disk space running out On 24/09/15 06:04, scArcer wrote:
Hi, I'm facing the problem of disk space running out. Since monetdb will store the query intermediates into the disk when the memory is not enough to hold them all, I'm wondering that does monetdb will automatically free the disk space? And if it doesn't, how can I do it manually?
Hi It will release disk space as soon as possible.
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi, Are you repeating this "create table + insert data" multiple times on this disk? Because there is a bug on the BAT files which are never freed : https://www.monetdb.org/bugzilla/show_bug.cgi?id=3802 Work around : try to do the "create table" in an other transaction than the "insert data". Le 24/09/2015 08:17, scArcer a écrit :
Here is my situation.
I create a table A, load about 3~4G data into it. Then a lot of queries are executed, making it using disk up to 1T.
[hadoop@falco001-bi-30-141 data3]$ du -sh monetdb/
1.1Tmonetdb/
and when I try to execute new query, I got this error:
sql>select * from ods.attr_user_c_profile limit 10;
could not allocate space for
GDKsave: error -1 on: name=06/62/66261, ext=theap, mode=0
!OS: No space left on device
HEAPextend: failed to extend to 983040 for 06/62/66261.theap: h->storage == STORE_MEM && can_map && fd >= 0 && HEAPsave_intern() < 0
------------------ Original ------------------ *From: * "Martin Kersten";
; *Date: * Thu, Sep 24, 2015 02:03 PM *To: * "users-list" ; *Subject: * Re: Disk space running out Hi, I'm facing the problem of disk space running out. Since monetdb will store the query intermediates into the disk when
On 24/09/15 06:04, scArcer wrote: the memory is not enough to hold them all, I'm wondering that does monetdb will automatically free the disk space?
And if it doesn't, how can I do it manually?
Hi It will release disk space as soon as possible.
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- Guillaume Savary Securactive R&D
participants (3)
-
Guillaume Savary
-
Martin Kersten
-
scArcer