Hi Andrey,

On Fri, Nov 12, 2010 at 12:53:13PM +0300, Andrey Timofeyev wrote:
> Hi Stefan,
>
> Thank you for your answer.
>
> What do your records and/or table schema look like?
> > How many columns, which types, how wide (bytes per record?
> >
>
> Here is our schema:
> CREATE TABLE entity1(entity1_id int PRIMARY KEY, entity1_name varchar(20)
> UNIQUE);
> CREATE TABLE entity2(entity2_id int PRIMARY KEY, entity2_name varchar(200),
> entity2_type varchar(20));
> CREATE TABLE entity2_entity1(entity2_id int, entity1_id int);
> CREATE TABLE entity2_entity2(parent_entity2_id int, child_entity2_id int);
>
> ALTER TABLE entity2 ADD CONSTRAINT entity2_name_type_unique
> UNIQUE(entity2_name, entity2_type);
>
> ALTER TABLE entity2_entity1 ADD CONSTRAINT entity2_entity1_2_fk FOREIGN
> KEY(entity2_id) REFERENCES entity2s (entity2_id);
> ALTER TABLE entity2_entity1 ADD CONSTRAINT entity2_entity1_1_fk FOREIGN
> KEY(entity1_id) REFERENCES entity1 (entity1_id);
>
> ALTER TABLE entity2_entity2 ADD CONSTRAINT entity2_entity2_p2_fk FOREIGN
> KEY(parent_entity2_id) REFERENCES entity2 (entity2_id);
> ALTER TABLE entity2_entity2 ADD CONSTRAINT entity2_entity2_c2_fk FOREIGN
> KEY(child_entity2_id) REFERENCES entity2 (entity2_id);
>
>
> > How do you insert them?
> > Bulkload via COPY INFO from CSV file or single insert statement per tuple?
> >
>
> Insert via single insert statements.

While this should indeed also work, for performance reasons, we strongly
recommend to bulkload data using COPY INTO (provided your application
scenarion allows that, i.e., you have data to bulkload).
In that case, we also recommend to initially create the tables without
primary keys, foreign keys, and other constrains, and add them (using alter
table as you alreasdy do for foreign keys and unique constraints) only once
the data is loaded.

> Which interface do you use?
> > Plain mclient, ODBC, JDBC, ...?
> >
>
> We tried mclient + sql script and JDBC.
>
> Can you successfully load less data / fewer records?
> > If so, how many can you load successfully before the problems occur?
>
> We can load about 5000 records in each table. (20000 in summary)
> The same effect with memory we can see when do many select statements
> through JDBC.

Would you be able to shared your datasets (and query workload) with us (not
publicly via this list, but privately), such that we could analyse the
behavior ourselves?

> > After several minutes monetdb eat all 7 Gb memory
> > That is to be expected is your data size is (that) large, i.e., MonetDB
> > (efficiently & effectively) does exploit all available physical memory, if
> > required --- what else does a DB server machine have memory for? ;-)
>
>
> 20000 our records shoud take less than 10 Mb + memory for indexes.
>
> That is unexpected and strange.
> > No message at all? On the server console or in the terminal window / shell
> > that the server was started in?  Not even a segfault?
> >
>
> On the linux we got segmentation error.
> I will tell more precisely little bit later.

Would be great if you could on Linux create a debug build of MonetDB
(configured with --enable-debug), and either run it in the debugger or have
it create a core dump (`unlimit -c unlimited`), and send us the stack trace
after the crash.

Stefan

> --
> Best regards,
> Andrey.

> ------------------------------------------------------------------------------
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev

> _______________________________________________
> MonetDB-users mailing list
> MonetDB-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-users

--
| Stefan.Manegold @ CWI.nl | DB Architectures (INS1) |
| http://CWI.nl/~manegold/ | Science Park 123 (L321) |
| Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam  (NL) |

------------------------------------------------------------------------------
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users