Re: [MonetDB-users] loading RDF data in MonetDB

On 2009-10-16 21:54, Medha Atre wrote:

Actually, I did a typo in my previous message. The response is not "connection *timeout*" but it's "connection *terminated*"! Since I don't own this machine, i will have to contact the administrators of the machine to check with them, but in the meanwhile searching on the web for the same error brought to my notice that several a few users have also reported the same problem and a bug was filed in MonetDB sometime early this year. I was just wondering if it's the same thing?

For more information, here are the last few lines of merovingian log

2009-10-16 07:13:52 MSG rdf[26793]: # MonetDB server v5.14.2, based on kernel v1.32.2 2009-10-16 07:13:52 MSG rdf[26793]: # Serving database 'rdf', using 8 threads 2009-10-16 07:13:52 MSG rdf[26793]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked 2009-10-16 07:13:52 MSG rdf[26793]: # Copyright (c) 1993-July 2008 CWI. 2009-10-16 07:13:52 MSG rdf[26793]: # Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved 2009-10-16 07:13:52 MSG rdf[26793]: # Visit http://monetdb.cwi.nl/ for further information 2009-10-16 07:13:52 MSG rdf[26793]: # Listening for connection requests on mapi:monetdb://127.0.0.1:50001/ 2009-10-16 07:13:52 MSG rdf[26793]: # MonetDB/SQL module v2.32.2 loaded 2009-10-16 07:14:22 MSG merovingian[26784]: proxying client localhost.localdomain:43986 for database 'rdf' to mapi:monetdb://127.0.0.1:50001/rdf 2009-10-16 07:49:59 MSG merovingian[26784]: database 'rdf' (26793) was killed by signal 9

This killed by signal 9 is a hint that the OS thought the process was taking too many resources (presumably memory resources) and killed the process.

2009-10-16 07:50:00 MSG merovingian[26784]: client localhost.localdomain:43986 has disconnected from proxy

I will also look into the connection timeout problem if that's the cause.

Thanks for the help. Medha

On Fri, Oct 16, 2009 at 3:15 PM, Martin Kersten wrote:

...

The error response produced is not MonetDB related. It is a result from the compute environment, which closes a connection is it has been silent for a long time. It may be as simple as timeout settings of your basic communication channel. There is nothing we can do about that. This may happen if you start any query (or dataload with integrity checking) that takes a long time.

Lefteris wrote:

...

Since you are loading triples in ID integer form, a billion triples should be no problem in a 16gig memory machine. Could you please give more information on what commands/queries you use to load the data, and the exact full error message. Which OS are you running in the machine and which version of monet you installed? (stable or current?)

regards,

lefteris

On Fri, Oct 16, 2009 at 7:20 PM, Medha Atre wrote:

...

Hi,

Thanks all for the valuable inputs. I could compile and successfully run MonetDB on one m/c, but I am facing issues in loading data into it on another m/c. The other m/c is actually much higher end m/c with 16G memory (server architecture) as opposed the first one in which I loaded the same data. Every time I try to load the data (RDF data in integer ID form), it keeps giving "connection timeout" error. Note: the data is really large (>1 billion triples). I am running "mclient" on the same m/c where monetdb server is running.

I tried it couple of times until now, but the issue persists.

Can someone throw some light on what might be going wrong?

Medha

On Thu, Oct 8, 2009 at 3:23 AM, Stefan Manegold wrote:

...

On Thu, Oct 08, 2009 at 08:56:35AM +0200, Sjoerd Mullender wrote:

...

On 2009-10-08 08:05, Niels Nes wrote: > On Wed, Oct 07, 2009 at 10:48:33PM -0400, Medha Atre wrote: >> Thanks Fabian. >> This worked. >> >> I just have small another question -- is there a way to store the >> output of a "select" query run on "mclient" into a file? I would like >> to compare the results with the ones generated by some other tool. > see > http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/Data-Manipulation.h... > section about copy into file If you're working interactively with mclient, you can also use \> file select ... \> The first line redirects subsequent output to the file, the last line directs it back to the terminal.

Check out the \? command. You can also simply use your shell's features:

mclient -lsql -s'select ...'> output-file echo 'select ...' | mclient -lsql> output-file mclient -lsql< query-file> output-file mclient -lsql query-file> output-file

Stefan

...

> Niels >> Thanks. >> Medha >> >> >> On Wed, Oct 7, 2009 at 2:26 PM, Fabian Groffen wrote: >>> On 07-10-2009 14:17:47 -0400, Medha Atre wrote: >>> [snip installation of all packages in a separate prefix] >>>> And then I ran the other commands I already mentioned below (e.g. >>>> merovingian etc). >>>> There were no error messages in the "make install" of any of the 4 >>>> packages I have listed above. >>>> >>>> Could you please help me understand what might be going wrong? >>> any reason you choose not to use >>> monetdb-install.sh --enable-sql --prefix=/path/to/MonetDB >>> ? If you don't have it, download the latest version of this script from >>> http://monetdb.cwi.nl/projects/monetdb/Assets/monetdb-install.sh >>> >>> Alternatively, just download a Super Source Tarball, which has this >>> script included. >>> >>> That basically solves all your problems. If you want to have all >>> components installed in their own prefix, you'll have to do some >>> post-fixing to get the modules installed where mserver expects them to >>> be.

------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

-- Sjoerd Mullender