> <mailto:
dariuszs@svnp.com>> wrote:
>
> Hi,
> Can anybody help me with this issue? Thanks. Dariusz.
>
>
>
> dariuszs wrote:
> > Hi,
> > Specs:
> > MonetDB Server 5.10.0, based on kernel v1.28.0
> > Windows x64 Server 2003 Enterprise
> > Intel Dual Xenon 5470 server, SAS drives, 48GB of memory / 220GB
> swap
> > space
> >
> > CREATE TABLE xxx(
> > C1 VARCHAR(16),
> > C2 VARCHAR(15),
> > C3 VARCHAR(1),
> > C4 VARCHAR(4),
> > C5 VARCHAR(1),
> > C6 VARCHAR(10),
> > C7 VARCHAR(2),
> > C8 VARCHAR(1),
> > C9 VARCHAR(1),
> > C10 VARCHAR(1),
> > C11 VARCHAR(1),
> > C12 VARCHAR(1),
> > C13 VARCHAR(2),
> > C14 VARCHAR(1),
> > C15 VARCHAR(2),
> > C16 VARCHAR(30),
> > C17 VARCHAR(16),
> > C18 VARCHAR(2),
> > C19 VARCHAR(5),
> > C20 VARCHAR(4),
> > C21 VARCHAR(3),
> > C22 VARCHAR(2),
> > C23 VARCHAR(1),
> > C24 VARCHAR(2),
> > C25 VARCHAR(1),
> > C26 VARCHAR(2),
> > C27 VARCHAR(1),
> > C28 VARCHAR(2),
> > C29 VARCHAR(3),
> > C30 VARCHAR(4),
> > C31 VARCHAR(10),
> > C32 VARCHAR(1),
> > C33 VARCHAR(2),
> > C34 VARCHAR(4),
> > C35 VARCHAR(1),
> > C36 VARCHAR(1),
> > C37 VARCHAR(6),
> > C38 VARCHAR(6),
> > C39 VARCHAR(2),
> > C40 VARCHAR(6),
> > C41 VARCHAR(1),
> > C42 VARCHAR(6),
> > C43 VARCHAR(2),
> > C44 VARCHAR(1),
> > C45 VARCHAR(1),
> > C46 VARCHAR(1),
> > C47 VARCHAR(1),
> > C48 VARCHAR(1),
> > C49 VARCHAR(1),
> > C50 VARCHAR(1),
> > C51 VARCHAR(2),
> > C52 VARCHAR(6),
> > C53 VARCHAR(2),
> > C54 VARCHAR(2),
> > C55 VARCHAR(6),
> > C56 VARCHAR(2),
> > C57 VARCHAR(1),
> > C58 VARCHAR(1),
> > C59 VARCHAR(1),
> > C60 VARCHAR(1),
> > C61 VARCHAR(1),
> > C62 VARCHAR(1),
> > C63 VARCHAR(1),
> > C64 VARCHAR(4),
> > C65 VARCHAR(1),
> > C66 VARCHAR(1),
> > C67 VARCHAR(1),
> > C68 VARCHAR(3),
> > C69 VARCHAR(2),
> > C70 VARCHAR(2),
> > C71 VARCHAR(1),
> > C72 VARCHAR(1),
> > C73 VARCHAR(1),
> > C74 VARCHAR(1),
> > C75 VARCHAR(2),
> > C76 VARCHAR(8),
> > C77 VARCHAR(8),
> > C78 VARCHAR(8),
> > C79 VARCHAR(1),
> > C80 VARCHAR(1),
> > C81 VARCHAR(1),
> > C82 VARCHAR(1),
> > C83 VARCHAR(6),
> > C84 VARCHAR(6),
> > C85 VARCHAR(6),
> > C86 VARCHAR(1),
> > C87 VARCHAR(1),
> > C88 VARCHAR(1),
> > C89 VARCHAR(1),
> > C90 VARCHAR(1),
> > C91 VARCHAR(1),
> > C92 VARCHAR(1),
> > C93 VARCHAR(1),
> > C94 VARCHAR(1),
> > C95 VARCHAR(1),
> > C96 VARCHAR(1),
> > C97 VARCHAR(1),
> > C98 VARCHAR(1),
> > C99 VARCHAR(1),
> > C100 VARCHAR(1),
> > C101 VARCHAR(1),
> > C102 VARCHAR(1),
> > C103 VARCHAR(1),
> > C104 VARCHAR(4),
> > C105 VARCHAR(4),
> > C106 VARCHAR(6),
> > C107 VARCHAR(8),
> > C108 VARCHAR(8),
> > C109 VARCHAR(1),
> > C110 VARCHAR(1),
> > C111 VARCHAR(8),
> > C112 VARCHAR(6),
> > C113 VARCHAR(1),
> > C114 VARCHAR(1),
> > C115 VARCHAR(1),
> > C116 VARCHAR(1),
> > C117 VARCHAR(1),
> > C118 VARCHAR(1),
> > C119 VARCHAR(1),
> > C120 VARCHAR(3),
> > C121 VARCHAR(1),
> > C122 VARCHAR(1),
> > C123 VARCHAR(1),
> > C124 VARCHAR(6),
> > C125 VARCHAR(6),
> > C126 VARCHAR(1),
> > C127 VARCHAR(1),
> > C128 VARCHAR(1),
> > C129 VARCHAR(22),
> > C130 VARCHAR(27));
> > copy 160000000 records into xxx from 'm:/monetdb_data/xxx.txt';
> > commit;
> >
> > OK, so the server loads the file for about 4 hours and I see CPU
> > activity, I see memory usage going up (47GB) and down several times,
> > but then after 4 hours everything sits - also I see that server
> > creates a files in the sql_logs directory 'log.179' which grew up to
> > 47GB and then it stopped.
> >
> > I was able to successfully load 130 million records using the same
> > file and the same system in about 3 hours. Thanks. Dariusz.
> >
> >
> >
> >
> > Stefan Manegold wrote:
> >> Which version of MonetDB?
> >> Which OS?
> >> Which HW arcitecture?
> >> Which table schema?
> >> What happens during "sits" (CPU bound, I/O bound, ...)?
> >> How do you "load"?
> >>
> >> Stefan
> >>
> >> On Fri, Apr 03, 2009 at 02:40:10PM -0400, dariuszs wrote:
> >>
> >>> Hi,
> >>> I've got a table with 130 fields. When I load 130 million records
> >>> into that table (about 50G) - it's not a problem, but when I
> try to
> >>> load 160 million records (about 60G of data) into it this
> thing just
> >>> sits for hours with no error messages. Can somebody help me?
> >>> Thanks. Dariusz.
> >>>
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------------
> >>>
> >>> _______________________________________________
> >>> MonetDB-users mailing list
> >>>
MonetDB-users@lists.sourceforge.net