dariuszs wrote:
> Hi,
> Specs:
> MonetDB Server 5.10.0, based on kernel v1.28.0
> Windows x64 Server 2003 Enterprise
> Intel Dual Xenon 5470 server, SAS drives, 48GB of memory / 220GB swap
> space
>
> CREATE TABLE xxx(
> C1 VARCHAR(16),
> C2 VARCHAR(15),
> C3 VARCHAR(1),
> C4 VARCHAR(4),
> C5 VARCHAR(1),
> C6 VARCHAR(10),
> C7 VARCHAR(2),
> C8 VARCHAR(1),
> C9 VARCHAR(1),
> C10 VARCHAR(1),
> C11 VARCHAR(1),
> C12 VARCHAR(1),
> C13 VARCHAR(2),
> C14 VARCHAR(1),
> C15 VARCHAR(2),
> C16 VARCHAR(30),
> C17 VARCHAR(16),
> C18 VARCHAR(2),
> C19 VARCHAR(5),
> C20 VARCHAR(4),
> C21 VARCHAR(3),
> C22 VARCHAR(2),
> C23 VARCHAR(1),
> C24 VARCHAR(2),
> C25 VARCHAR(1),
> C26 VARCHAR(2),
> C27 VARCHAR(1),
> C28 VARCHAR(2),
> C29 VARCHAR(3),
> C30 VARCHAR(4),
> C31 VARCHAR(10),
> C32 VARCHAR(1),
> C33 VARCHAR(2),
> C34 VARCHAR(4),
> C35 VARCHAR(1),
> C36 VARCHAR(1),
> C37 VARCHAR(6),
> C38 VARCHAR(6),
> C39 VARCHAR(2),
> C40 VARCHAR(6),
> C41 VARCHAR(1),
> C42 VARCHAR(6),
> C43 VARCHAR(2),
> C44 VARCHAR(1),
> C45 VARCHAR(1),
> C46 VARCHAR(1),
> C47 VARCHAR(1),
> C48 VARCHAR(1),
> C49 VARCHAR(1),
> C50 VARCHAR(1),
> C51 VARCHAR(2),
> C52 VARCHAR(6),
> C53 VARCHAR(2),
> C54 VARCHAR(2),
> C55 VARCHAR(6),
> C56 VARCHAR(2),
> C57 VARCHAR(1),
> C58 VARCHAR(1),
> C59 VARCHAR(1),
> C60 VARCHAR(1),
> C61 VARCHAR(1),
> C62 VARCHAR(1),
> C63 VARCHAR(1),
> C64 VARCHAR(4),
> C65 VARCHAR(1),
> C66 VARCHAR(1),
> C67 VARCHAR(1),
> C68 VARCHAR(3),
> C69 VARCHAR(2),
> C70 VARCHAR(2),
> C71 VARCHAR(1),
> C72 VARCHAR(1),
> C73 VARCHAR(1),
> C74 VARCHAR(1),
> C75 VARCHAR(2),
> C76 VARCHAR(8),
> C77 VARCHAR(8),
> C78 VARCHAR(8),
> C79 VARCHAR(1),
> C80 VARCHAR(1),
> C81 VARCHAR(1),
> C82 VARCHAR(1),
> C83 VARCHAR(6),
> C84 VARCHAR(6),
> C85 VARCHAR(6),
> C86 VARCHAR(1),
> C87 VARCHAR(1),
> C88 VARCHAR(1),
> C89 VARCHAR(1),
> C90 VARCHAR(1),
> C91 VARCHAR(1),
> C92 VARCHAR(1),
> C93 VARCHAR(1),
> C94 VARCHAR(1),
> C95 VARCHAR(1),
> C96 VARCHAR(1),
> C97 VARCHAR(1),
> C98 VARCHAR(1),
> C99 VARCHAR(1),
> C100 VARCHAR(1),
> C101 VARCHAR(1),
> C102 VARCHAR(1),
> C103 VARCHAR(1),
> C104 VARCHAR(4),
> C105 VARCHAR(4),
> C106 VARCHAR(6),
> C107 VARCHAR(8),
> C108 VARCHAR(8),
> C109 VARCHAR(1),
> C110 VARCHAR(1),
> C111 VARCHAR(8),
> C112 VARCHAR(6),
> C113 VARCHAR(1),
> C114 VARCHAR(1),
> C115 VARCHAR(1),
> C116 VARCHAR(1),
> C117 VARCHAR(1),
> C118 VARCHAR(1),
> C119 VARCHAR(1),
> C120 VARCHAR(3),
> C121 VARCHAR(1),
> C122 VARCHAR(1),
> C123 VARCHAR(1),
> C124 VARCHAR(6),
> C125 VARCHAR(6),
> C126 VARCHAR(1),
> C127 VARCHAR(1),
> C128 VARCHAR(1),
> C129 VARCHAR(22),
> C130 VARCHAR(27));
> copy 160000000 records into xxx from 'm:/monetdb_data/xxx.txt';
> commit;
>
> OK, so the server loads the file for about 4 hours and I see CPU
> activity, I see memory usage going up (47GB) and down several times,
> but then after 4 hours everything sits - also I see that server
> creates a files in the sql_logs directory 'log.179' which grew up to
> 47GB and then it stopped.
>
> I was able to successfully load 130 million records using the same
> file and the same system in about 3 hours. Thanks. Dariusz.
>
>
>
>
> Stefan Manegold wrote:
>> Which version of MonetDB?
>> Which OS?
>> Which HW arcitecture?
>> Which table schema?
>> What happens during "sits" (CPU bound, I/O bound, ...)?
>> How do you "load"?
>>
>> Stefan
>>
>> On Fri, Apr 03, 2009 at 02:40:10PM -0400, dariuszs wrote:
>>
>>> Hi,
>>> I've got a table with 130 fields. When I load 130 million records
>>> into that table (about 50G) - it's not a problem, but when I try to
>>> load 160 million records (about 60G of data) into it this thing just
>>> sits for hours with no error messages. Can somebody help me?
>>> Thanks. Dariusz.
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> MonetDB-users mailing list
>>>
MonetDB-users@lists.sourceforge.net
>>>
https://lists.sourceforge.net/lists/listinfo/monetdb-users
>>>
>>>
>>
>>
>
>
------------------------------------------------------------------------------