Re: [MonetDB-users] loading RDF data in MonetDB

Ahh.. thanks a lot. It worked. But I still have some other hiccups -- After following the short documentation given on http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/Start-and-Stop.html I tried following $ merovingian $ monetdb create rdf $ monetdb start rdf *starting 'rdf' failed: database 'rdf' has inconsistent state (running but dead), review merovingian's logfile for any peculiarities* If I do "monetdb status" it shows me "rdf" running and health 100% but then why this error message? Also if I try connecting to the rdf database using mclient, it says -- $ mclient -lsql --database=rdf !MALException:setScenario:Scenario not initialized 'sql' But when I do "ps" mserver is shown to be running with "include sql" construct -- $ ps -aef | grep <myuserid> | grep mserver <myuserid> 12219 11936 0 15:48 ? 00:00:00 /path/to/MonetDB5/bin/mserver5 --config=/path/to/MonetDB5/etc/monetdb5.conf --dbname=rdf1 --dbinit=include sql; --set monet_daemon yes --set mapi_open false --set mapi_autosense true --set mapi_port 50001 --set monet_vault_key /path/to/MonetDB5/var/MonetDB5/dbfarm/rdf/.vaultkey Below I am giving last few lines from merovingian.log 2009-10-06 15:47:02 MSG rdf[12197]: # MonetDB server v5.14.2, based on kernel v1.32.2 2009-10-06 15:47:02 MSG rdf[12197]: # Serving database 'rdf', using 8 threads 2009-10-06 15:47:02 MSG rdf[12197]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked 2009-10-06 15:47:02 MSG rdf[12197]: # Copyright (c) 1993-July 2008 CWI. 2009-10-06 15:47:02 MSG rdf[12197]: # Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved 2009-10-06 15:47:02 MSG rdf[12197]: # Visit http://monetdb.cwi.nl/ for further information 2009-10-06 15:47:02 MSG rdf[12197]: # Listening for connection requests on mapi:monetdb://127.0.0.1:50001/ 2009-10-06 15:47:11 ERR merovingian[11936]: failed to fork mserver: database 'rdf' has inconsistent state (running but dead), review merovingian's logfile for any peculiarities 2009-10-06 15:47:29 MSG merovingian[11936]: proxying client localhost.localdomain:38697 for database 'rdf' to mapi:monetdb://127.0.0.1:50001/rdf 2009-10-06 15:47:29 MSG merovingian[11936]: client localhost.localdomain:38697 has disconnected from proxy 2009-10-06 15:47:50 MSG rdf[12197]: !MALException:malInclude:could not open file: sql ------------------------------------------------------------------------------------------- Also regarding your comment about PSO reordering below, does it mean that if I just reorganize my triples as Predicate, Subject, Object and simply load using "copy into" into MonetDB it will take care of vertically partitioning them? Or do I still have to batch load into different tables according to different properties? Thanks. Medha On Tue, Oct 6, 2009 at 8:55 PM, Peter Boncz wrote:

Hi,

to compile from sources the easiest is the supersource tarball, this is the first clickable item you encounter on the download page monetdb.cwi.nl/projects/monetdb/Download

if you insist on getting the individual components one by one, you need: - MonetDB-1 - MonetDB-client (this one has the Mapi library you were missing) - MonetDB5-server - MonetDB-SQL

ah, and that idea of vertically decomposed RDF, it is essentially the same as PSO ordering, which helps some queries, but then other queries profit from other orderings (POS,SPO,SOP,OSP,OPS); nothing new under the sun. The RDF parser Lefteris made effectively creates all these orders.

Peter

-----Original Message----- From: Medha Atre [mailto:medha.atre@gmail.com] Sent: Wednesday, October 07, 2009 2:45 AM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] loading RDF data in MonetDB

Sorry for the typo, before MonetDB-SQL, I am trying to compile MonetDB5-xx.tar.gz and I am getting the Mapi error while compiling MonetDB5

Thanks. Medha

On Tue, Oct 6, 2009 at 8:43 PM, Medha Atre wrote:

...

I tried installing MonetDB-SQL-xx.tar.gz from sources.

Make fails giving error -lMapi not found.

I was wondering why does it need Mapi library when all I want to do is simply build a database and load some data and query it!!

Can someone help please? The READMEs in the sources are of no help and there is no step-by-step setup procedure given on the web-documentation.

Thanks in anticipation.

Medha

On Tue, Oct 6, 2009 at 8:26 PM, Medha Atre wrote:

...

OK. I figured that must be the command after digging through the manuals. But for the vertically partitioned approach demonstrated in various research papers, I believe the end user has to manually load the RDF data split according to the predicates into different tables, which also requires that the SQL query being issued specifies these tables and joins across these tables as opposed to one big triple table.

Correct me if I am wrong.

Thanks.

Medha

On Tue, Oct 6, 2009 at 5:06 PM, Sjoerd Mullender wrote:

...

On 2009-10-06 17:59, Medha Atre wrote:

...

Hi,

Thanks for the reply.

At this point, all I am looking for is to load RDF NTriples data -- treated as a 3 column table -- into MonetDB using "vertical partitioning" to index the data nicely (as I read about MonetDB being used for the same in different research papers recently.. e.g. http://portal.acm.org/citation.cfm?id=1453856.1453927 RDF-3X etc). I am not really looking for a great support for different RDF formats (as the data has already been cleaned into NTriples format using rdflib), I want to check the correctness of the results generated using my own querying tool.

So can you please suggest me a method of treating the given RDF NTriples file as a simple "space separated" 3-column data file from which I can load the data into MonetDB creating the required indexes or "vertical partitions"?

Perhaps something like COPY <n> RECORDS INTO <table> FROM <file> USING DELIMITERS ' ','\n'; where <n> is the number of tuples, <table> is the name of a table that you created for this, and <file> is the full path name of the file (in single quotes).

...

Querying with SQL front end is fine with me!

Looking forward to your help.

Medha

...

------------------------------

Message: 5 Date: Tue, 6 Oct 2009 10:19:57 +0200 From: Lefteris Subject: Re: [MonetDB-users] loading RDF data in MonetDB To: Communication channel for MonetDB users         Message-ID:         <852badbc0910060119x4f125a06qd5fc7ed0c56035c@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1

Hi,

there is a highly experimental module for MonetDB to load RDF data. This is not activated in the stable release, and in order to use it you will need to do some compilation on your own and possible report back any problems you have. If you are willing to jump deeper in the MonetDB experience I can guide you on how to do this. The module will read any RDF format supported by the rdflib library and create a triple store with appropriate indices. However, the only way to query the triple store at this point is through the SQL front-end.

regrads,

lefteris

On Tue, Oct 6, 2009 at 8:59 AM, Medha Atre  wrote: > Hi, > > I intend to use MonetDB for some of my experimental work using RDF > data. I was wondering if there is an easy way to load RDF data (in > NTriples format) in MonetDB making use of the column-wise vertical > partitioning of the RDF data, based on properties/predicates in the > RDF data. > > Some pointers or help regarding this is much appreciated. > > I am sorry if this is not the correct forum to post this message. > > Thanks. > Medha