If MonetDB could use table partition ?
I am using "COPY BINARY INTO ...FROM ..." to insert 1000 FITS files into a table and the table has 21 columns, every FITS file has 200000 records. After some FITS files, the loading becoming very slowly, so If MonetDB could use table partition to accelerating the copy? or if Data Vaults could be loading more quickly? Thanks! Meng
Hi Meng, are you loading all 1000 FITS files into the same table, or each in its own table? What is the schema of your table, i.e., what are the types of the 21 columns? Which operating system are you running on, and how much memory (RAM) does your machine have? Stefan ----- Original Message -----
I am using "COPY BINARY INTO ...FROM ..." to insert 1000 FITS files into a table and the table has 21 columns, every FITS file has 200000 records. After some FITS files, the loading becoming very slowly,
so If MonetDB could use table partition to accelerating the copy? or if Data Vaults could be loading more quickly?
Thanks!
Meng _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
After how many files does the loading get how slow? I.e., could you share the loading time of each file, starting from the first until it becomes "very slow" Stefan ----- Original Message -----
Hi Meng,
are you loading all 1000 FITS files into the same table, or each in its own table?
What is the schema of your table, i.e., what are the types of the 21 columns?
Which operating system are you running on, and how much memory (RAM) does your machine have?
Stefan
----- Original Message -----
I am using "COPY BINARY INTO ...FROM ..." to insert 1000 FITS files into a table and the table has 21 columns, every FITS file has 200000 records. After some FITS files, the loading becoming very slowly,
so If MonetDB could use table partition to accelerating the copy? or if Data Vaults could be loading more quickly?
Thanks!
Meng _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
participants (2)
-
integrity -
Stefan Manegold