Re: [MonetDB-users] Change schema for already existing tables

On Jan 22, 2008 12:28 AM, Niels Nes wrote:

On Mon, Jan 21, 2008 at 03:09:09PM -0800, Bob Ippolito wrote:

...

Martin,

Bob

...

What Travis meant is that we have a bunch of tables in the default sys schema and we would like to move them to a different schema, hopefully without losing a day or two worth of ETL. Using a create table + select into is the only solution. Tables from other schemas are accessable via full qualified names, ie schema.table.column.

It's too late now, since SR5 crashed on us with an unrecoverable database we had to start over anyway. This time we did it with a proper schema :)

...

On that note what is the most efficient way to do ETL? We're using COPY INTO right now and it works but we're only getting between 200-2000 rows/sec and mserver5 has crashed on us twice now for no apparent reason.

A couple of bugs related to ETL processing were fixed, only in the cvs head. The main reason of this (I know) incorrect place was that we expect to do a release of this branch soon. Your coalesce and timestamp problem aren't fixed however.

About The slow loading problem, we would need to know your db schema ie a snippet of your sql create table statement would be needed. Also ETL involves more then just loading, ie COPY into, so could it be the transformation part or isn't that part of MonetDB's task?

monetdb is *only* doing COPY INTO, all of the other ETL work is done by PostgreSQL and Python. Mostly it's Python asking PostgreSQL to COPY its data into a file in a format that MonetDB can read. I have sent you a copy of our schema off-list.

Niels

...

-bob

On Jan 21, 2008 2:50 PM, Martin Kersten wrote:

...

Travis Brady wrote:

...

Hello, Hello Travis you can at any time add and drop columns without any reorganization cost. regards, Martin

...

Is it possible to change the schema for tables that have already been created and populated? I'd like to avoid having to fully recreate everything as the load process will likely take a few days at least.

thank you, Travis Brady ------------------------------------------------------------------------

------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ ------------------------------------------------------------------------

_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

--

Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl

------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users