On Sun, 23 Nov 2008, Martin Kersten wrote:
The system is trying to malloc space for (in this case) a hash table, but that instruction fails. Then it attempts to free up memory by swapping out tables. Given the fact that after this sweep it still fails, indicates that your swap memory has become fragmented, such that there is no consecutive area of 14 GB left.
Maybe a very strange idea, could be your better solution. If by default these operations are mmap'ed, they should be taken care of by a disk based memory extension. Hence being independent of RAM/SWAP but having all pro's of both.
For large table loads (relative to the swap size) it is advisable to load in multiple steps. First load the data into the tables without (foreign) key checks. After this step ALTER the tables to respect the (foreign) keys.
You are again right for many reasons to do it in this way. Sadly my 'data provider' is not so strict on data integrety I found out again, and this is probably the only efficient way to load the tables, and find all non existing relations. Stefan