Hi, When you run out of memory and swap space that's what happens. Tibor Barna wrote:
Hi dariuszs,
This sounds like a good idea, but i am still interested what the cause of the problem is. I had run the same script again, not long ago, and it worked (for the 11GB CSV).
Than the next problem: sql>copy 163715212 records into sys.page_urls from '/data/csv_monet_dump/page_urls.tbl' using delimiters '|',' more>', '"' null as '';
MAPI = (monetdb) /home/wired/MonetDB/var/MonetDB5/dbfarm/mapi_socket ACTION= read_line QUERY = copy 163715212 records into sys.page_urls from '/data/csv_monet_dump/page_urls.tbl' using delimiters '|',' ', '"' null as ''; ERROR = !Connection terminated
merovingian.log: 2011-03-10 16:28:51 MSG merovingian[3266]: database 'performance' (17635) was killed by signal SIGSEGV
My database just crashed, I restarted it, run the command again, it worked. Strange :) The second CSV was only 2GB.
Cheers, Tibor.
dariuszs wrote:
Hi, Split the data into several tables, load them sequentially.
Tibor Barna wrote:
Hello,
I am trying to import data from MySQL into MonetDB usig the bulk loader, the csv has ~11GB, 163Mil Rows. When running:
sql>copy 163715240 records into sys.clicks from '/data/csv_monet_dump/clicks.tbl' using delimiters '|',' more>', '"' null as '';
I always get this error message: SQLException:importTable:failed to import table ERROR: HEAPextend: failed to extend to 654860960 for 32/3231tail ERROR: TABLETcreate_bats: Failed to create bat of size 163715240
merovingian.log: 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) fails, try to free up space [memory in use=41411528,virtual memory in use=12310675456] 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) result [mem=32087208,vm=12310675456]
I am using 64bit openSUSE 11.3, 12GB RAM, 4GB Swap.
Does anybody know how can i overcome this error? Any help would be most welcome :)
Cherers, Tibor.
------------------------------------------------------------------------------ Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users