On Fri, Nov 12, 2010 at 10:21:52PM +0300, Andrey Timofeyev wrote:
you execute everything in one very large batch, why not execute (and clear) the batch at smaller intervals? You print "10000 inserted", but you actually only queued up that amount of inserts, not sending anything to the database.
Yes, and one very large batch works ok. The problem here in creating
PreparedStatements.
You really want to make a million prepared statements here? The server has to store them all... I guess so, since you're leaking prepared handles like mad
We create separate PreparedStatement for every query in our project. There are no memory leak on the client side (we close every PreparedStatement). Why server should store PreparedStatement forever, is it possible to configure server do not store them? Or it works so by design, and we should cache our PreparedStatements.
The idea of prepared statements is to prepare a query template once and then call (re-use) it with different parameters multiple times, i.e., for 1000 inserts you would to one prepareStatement (and hence pay the cost for translating, optimizing that query and storing its execution plan only once) and then call (execute) it multiple times with (potentially) different parameters (arguments), at ver low cost and it then is merely calling a pre-compiled function ... This is a general idea, not MonetDB specific ... Stefan
-- Best regards, Andrey.
------------------------------------------------------------------------------ Centralized Desktop Delivery: Dell and VMware Reference Architecture Simplifying enterprise desktop deployment and management using Dell EqualLogic storage and VMware View: A highly scalable, end-to-end client virtualization framework. Read more! http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |