Thanks for response,
I do not believe that 2GB BAT size limit was hit. Largest column in
the table was defined as varchar(44), I had 12,000,000 records loaded
and wanted to add 6,000,000 more records. So, there should be 18 *
44=792MBytes. It depends how BAT is represented internally, though.
I am afraid I ran out of memory or disk space.
Anyway the error has not been handled and now I can not start server
to run tests once again and clarify the problem. Could anybody help me
to find out what happened to server ?
2005/10/3, Fabian
Andrei,
Sorry that your previous post (seemed to be) ignored. Unfortunately nobody was able to respond in time.
I am not sure on this answer, but if I recall correctly MonetDB cannot possibly process columns which are larger than the addresseable memory. This is due to the fact that MonetDB is a main-memory database.
Please someone correct me if I'm wrong here.
Regards, Fabian
Andrei Martsinchyk wrote:
Hello,
It seems MonetDB has problem with handling of huge tables (larger than amount of memory available). I tried to load large amount of data into a table using COPY command. I loaded same data (6 millions records, 700+MBytes) few times. Loading was successful two times, on third time server crushed. After crush I could not start the server again, it output messages
!WARNING: GDKlockHome: ignoring empty or invalid .gdk_lock. !FATAL: BBPinit: cannot properly process bat/BACKUP/.
and exited. Removing of the database data directory did not help much - server output messages
!ERROR:mvc_init: unable to create system tables !WARNING: BBPincref: range error 410
Couple minutes later it output
!ERROR: BATSIGcrash: Mserver internal error (Segmentation fault), please restart . !ERROR: (One potential cause could be that your disk might be full...)
Well, insufficient disk space could be a cause, before server crashed I had approximately 2GB free. But I have cleaned up 10 GB and problem persists.
How can I troubleshoot and fix the server ? What are memory / disk space requirements when bulk loading data ?
I tested with 32-bit system, 512 MB RAM + 1GB swap, Ubuntu Linux, MonetDB-4.9.3, nightly build dated September 30.
-- Best regards, Andrei Martsinchyk mailto:andrei.martsinchyk@gmail.com
------------------------------------------------------- This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------- This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- Best regards, Andrei Martsinchyk mailto:andrei.martsinchyk@gmail.com