Andrei, Sorry that your previous post (seemed to be) ignored. Unfortunately nobody was able to respond in time. I am not sure on this answer, but if I recall correctly MonetDB cannot possibly process columns which are larger than the addresseable memory. This is due to the fact that MonetDB is a main-memory database. Please someone correct me if I'm wrong here. Regards, Fabian Andrei Martsinchyk wrote:
Hello,
It seems MonetDB has problem with handling of huge tables (larger than amount of memory available). I tried to load large amount of data into a table using COPY command. I loaded same data (6 millions records, 700+MBytes) few times. Loading was successful two times, on third time server crushed. After crush I could not start the server again, it output messages
!WARNING: GDKlockHome: ignoring empty or invalid .gdk_lock. !FATAL: BBPinit: cannot properly process bat/BACKUP/.
and exited. Removing of the database data directory did not help much - server output messages
!ERROR:mvc_init: unable to create system tables !WARNING: BBPincref: range error 410
Couple minutes later it output
!ERROR: BATSIGcrash: Mserver internal error (Segmentation fault), please restart . !ERROR: (One potential cause could be that your disk might be full...)
Well, insufficient disk space could be a cause, before server crashed I had approximately 2GB free. But I have cleaned up 10 GB and problem persists.
How can I troubleshoot and fix the server ? What are memory / disk space requirements when bulk loading data ?
I tested with 32-bit system, 512 MB RAM + 1GB swap, Ubuntu Linux, MonetDB-4.9.3, nightly build dated September 30.
-- Best regards, Andrei Martsinchyk mailto:andrei.martsinchyk@gmail.com
------------------------------------------------------- This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users