On Tue, Jun 17, 2008 at 08:20:58PM +0200, Niels Nes wrote:
On Tue, Jun 17, 2008 at 07:06:24PM +0100, Sam Mason wrote:
COPY INTO source_livestock FROM '/mnt/scratch/smason/source_livestock' DELIMITERS ','; !SQLException:importTable:failed to import table !ERROR: GDKload: cannot mmap(): name=16/1634, ext=theap !OS: Cannot allocate memory !ERROR: GDKload failed: name=16/1634, ext=theap
Would you expect me to get this still? and hence should I provide more details or should I be building something else?
For this problem I indeed need more info. How much memory does your system have and how much swap. Also what are the sizes of the to be loaded table (nr of rows, columns, data types) ?
Sorry, it's almost the same as my original post. The table now has 10 columns, 2 integer, 4 varchars and 4 dates. I'm trying to pull in about 38million rows (text file is about 3GB) into a 32bit build of monetdb. RAM and swap space are 2GB each, i.e. I've got 4GB in total. Why would ram/swap space matter for anything but performance? If they do, I'm assuming monet wouldn't work with files larger than the memory addressable by a single process? This file is going into one of the smaller tables that I want to work with. If this is going to be a problem how much of a change is it going to be to the code to support out-of-core style operations. Sam