Hi Stefan, I'm sorry that I forgot to mention the required details. *Platform* - Fedora Core 10, x86_64 - MonetDB 4.26.4 64 bit November 2008-SP2 RPM (64 bit OIDs) - 8 GB RAM - ~170 GB free. *Query* pf:add-doc('/tmp/<unique name>.xml', '<unique name>', '<unique name>') where unique name is a generated name. There is no document or collection with the given name already in the database when the query is executed. *I/O* Performed by a separate process, copying 40 GB of data. *Other details* The behavior was observed on a non-empty database. I'll try to reproduce it on an empty database so that I can provide you the requested details. But, frankly, I think I'll have to look at the OS I/O scheduler to fix this. Regards, John Stefan Manegold wrote:
Hi John,
to get a better impression what the actual problem might be and what triggers it, it would be great, if you could provide us with the necessary data, query|ies and detailed instructions how to reproduce the problem.
In addition, I have the following questions:
On Mon, Feb 02, 2009 at 10:15:06AM +0100, John van Schie (DT) wrote:
Hi,
We use MonetDB v4.26.4 (Nov 2008-sp2) for our application with the XQuery frontend. Often, queries are executed very fast, but sometimes, execution of the same query takes significantly more time.
For example: We insert a XML document of 1.4Mb. with a maximum depth of 4. The document is structured as follows: <rootelement> <item> <child1> <sha1>sha1-value</sha1> <md5>md5-value</md5> </child1> </item> ... </rootelement> We have 8k 'item' elements.
Often the insertion of this document is performed in ~0.5s, which is fine.
How exactly do you load/insert the document? Using pf:add-doc? Do you add it into a collection of its own or multiple documents into the same collection? Does it occur (also) when inserting this document into an empty database or a new/empty collection or (only) when adding the document to a database and/or collection that already contains some data/documents? If the latter, how many documents/how much data has been successfully (and "fast") inserted into the whole database and into the same collection prior the insert that is unexpectedly slow?
How much memory does your machine have? How large is your Mserver when the slow insert occurs? Are using running on a 64-bit or 32-bit systems (both OS & MonetDB)? Which OS are you running on?
But when the system is performing IO (e.g. copying a large file), the insertion can take up to 6 minutes.
"Who" is causing/doing the IO? MonetDB or some other process/application? If MonetDB, is it only running your insert query at that time or running any other queries concurrently? Did you also observer slow inserts without the system being busy with IO?
So I have two questions: 1) What is the cause of this large difference in execution times;
I cannot yet tell without monitoring the system while this happens.
2) Is there a way to configure MonetDB (or the server) in such a way that the difference between the 'fast' and 'slow' is smaller.
In principle no. However, once we understand what is actually happen, we might know more ...
Regards, Stefan
Regards,
John van Schie
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users