Hi John, to get a better impression what the actual problem might be and what triggers it, it would be great, if you could provide us with the necessary data, query|ies and detailed instructions how to reproduce the problem. In addition, I have the following questions: On Mon, Feb 02, 2009 at 10:15:06AM +0100, John van Schie (DT) wrote:
Hi,
We use MonetDB v4.26.4 (Nov 2008-sp2) for our application with the XQuery frontend. Often, queries are executed very fast, but sometimes, execution of the same query takes significantly more time.
For example: We insert a XML document of 1.4Mb. with a maximum depth of 4. The document is structured as follows: <rootelement> <item> <child1> <sha1>sha1-value</sha1> <md5>md5-value</md5> </child1> </item> ... </rootelement> We have 8k 'item' elements.
Often the insertion of this document is performed in ~0.5s, which is fine.
How exactly do you load/insert the document? Using pf:add-doc? Do you add it into a collection of its own or multiple documents into the same collection? Does it occur (also) when inserting this document into an empty database or a new/empty collection or (only) when adding the document to a database and/or collection that already contains some data/documents? If the latter, how many documents/how much data has been successfully (and "fast") inserted into the whole database and into the same collection prior the insert that is unexpectedly slow? How much memory does your machine have? How large is your Mserver when the slow insert occurs? Are using running on a 64-bit or 32-bit systems (both OS & MonetDB)? Which OS are you running on?
But when the system is performing IO (e.g. copying a large file), the insertion can take up to 6 minutes.
"Who" is causing/doing the IO? MonetDB or some other process/application? If MonetDB, is it only running your insert query at that time or running any other queries concurrently? Did you also observer slow inserts without the system being busy with IO?
So I have two questions: 1) What is the cause of this large difference in execution times;
I cannot yet tell without monitoring the system while this happens.
2) Is there a way to configure MonetDB (or the server) in such a way that the difference between the 'fast' and 'slow' is smaller.
In principle no. However, once we understand what is actually happen, we might know more ... Regards, Stefan
Regards,
John van Schie
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |