Alex Bo. wrote:
"Each file is produced by a program that writes the binary image of the BAT directly, i.e. a binary dump of an C-array"
How exactly can I produce these binary images from my source columns?
The expact part can probably be reviewed in gdk_bat.mx Pseudo: array = mmap(file, size(type)*rows) for 1..n array[i] = newvalue As you can see in gdk_bat.mx there is also an offset to be taken into account for a descriptor with some meta info.
How exactly are they expected by MonetDB to look like? A little more info could be very helpful.
This works for integer type columns. If your tables contain strings, due to the storage of strings in MonetDB and best effort deduplication, the operation of writing 'your own' generator is not trivial. I have heard talks about alternative and faster ways of loading. But I wonder how much faster it would be in compared to COPY INTO. Stefan