- developers-list - monetdb.org

Re: [Monetdb-developers] RDF
by Marco Antonelli 10 Apr '08

10 Apr '08

Bellekens, P.A.E. wrote: > The link to the PDF is: > > monetdb.cwi.nl/Assets/m5manual.pdf > > There (if you search on 'rdf') you find a picture in which RDF is mentioned as input. Although I do not exactly know what the context there is. As you may see from the picture, the RDF box is dashed, since is still not there... > Cheers and good luck, > Pieter Thank you Pieter, Marco > -----Original Message----- > From: Marco Antonelli [mailto:M.Antonelli@cwi.nl] > Sent: woensdag 9 april 2008 15:56 > To: Bellekens, P.A.E. > Cc: monetdb-developers(a)lists.sourceforge.net > Subject: Re: [Monetdb-developers] RDF > > Hello Pieter, > > I'm the master thesis student who started the SPARQL front-end for > MonerDB this January. > > Bellekens, P.A.E. ha scritto: >> Dear MonetDB developers, >> >> I have a question related to RDF. >> In you PDF online, I see that it is possible to import RDF in the MonetDB system, >> although there is not much further information concerning RDF input. > > may you send a link to the PDF? > >> Could you please inform me a bit more about: >> - is it possible to upload RDF data? > > Actually it is possible to import and export RDF in MonetDB, but this > functionality will be under heavy development starting from the next week. > If you want to get a look at the code, the import-export functions are > defined in MonetDB5/src/modules/mal/rdf.mx. > It is not compiled by default, to getting it compiled you have to modify > the Makefile.ag file in the same directory, then run 'bootstrap', > 'configure', 'make' and 'make install'. > >> - is there any kind of RDF specific querying? > > Still not, the SPARQL compiler is under development too > >> - is it possible to run MonetDB as a backend in a triple store like for example in Sesame? > > Not at the moment, but MonetDB will be a backend for Sesame in the next > future > >> - is there any performance measuring done for RDF querying? >> It is known that column stores have a good performance (c-store benchmarks) for RDF data as well... > > Lefteris and Romulo can answer you on these topics more in detail than me > Anyway MonetDB/SQL performed well in querying RDF data :) > > Best regards, > Marco Antonelli > >> With kind regards, >> Pieter >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >> Don't miss this year's exciting event. There's still time to save $100. >> Use priority code J8TL2D2. >> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java… >> _______________________________________________ >> Monetdb-developers mailing list >> Monetdb-developers(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > No virus found in this incoming message. > Checked by AVG. > Version: 7.5.519 / Virus Database: 269.22.11/1368 - Release Date: 9-4-2008 16:20 > > > No virus found in this outgoing message. > Checked by AVG. > Version: 7.5.519 / Virus Database: 269.22.11/1368 - Release Date: 9-4-2008 16:20 >

1 0

[Monetdb-developers] dbname....
by Romulo Goncalves 08 Apr '08

08 Apr '08

I did a mistake in my scripts to start my mserver and I set my dbname like this: /scratch5/romulo/var/dbfarm/ When I start my mserver I get this information: -bash-3.1$ ../server.sh optimized # MonetDB server v5.5.0, based on kernel v1.23.0 # Serving database '/scratch5/romulo/var/dbfarm' # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked # Copyright (c) 1993-2008 CWI, all rights reserved Until now no problem. The problem comes when I connect a mclient: !SABAOTHgetMyStatus: MALException:sabaoth.assert:SABAOTHgetMyStatus couldn't find a database for /scratch5/romulo/var/dbfarm! I decided to debug the code and there I decided to print the following variables: printf("%s , %s \n ", GDKgetenv("gdk_dbfarm"), GDKgetenv("gdk_dbname")); Result: /scratch5/romulo/var/dbfarm , /scratch5/romulo/var/dbfarm Until now no problem, so what is the reason for the crash? Well, I decide to check my dbfarm: /scratch5/romulo/var/dbfarm There I found out that the directory with name "/scratch5/romulo/var/dbfarm" does not exist, but instead of that sub-directories were created: /scratch5/romulo/var/dbfarm/scratch5/romulo/var/dbfarm/ My question is: Is the location of the database correct just because in the dbname I defined the name like a path? Or should that name be interpreted as a single string? Regards, Romulo PS: Next time I will be more be careful with my scripts :)

2 1

[Monetdb-developers] embedded mserver...
by Romulo Goncalves 08 Apr '08

08 Apr '08

I start the mserver with the following command: ./mserver5 --set embedded=yes --dbinit="include sql;" & mclient connection is refused. The I decided to set the mapi_port: --set mapi_port=50000 & Mclient connection is again refused for port 50000: [goncalve@amelia bin]$ mclient -lsql !ERROR mapi_reconnect: connect: Connection refused Setup connection failed [goncalve@amelia bin]$ and for port 33177 isn't: [goncalve@amelia bin]$ mclient -lsql -P33177 User (goncalve): [goncalve@amelia bin]$ As you can see the above example the user used for the connection is goncalve and not monetdb. Why the default user is not monetdb? And why this port? This example (the first command to start mserver) was taken from our web-site... http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/Embedded-Server.ht… Regards, Romulo

2 6

[Monetdb-developers] [Fwd: ! Compiling 'Current' pathfinder: Results DIFFER from 2008.04.02_00-09-03's !]
by Jan Rittinger 03 Apr '08

03 Apr '08

How do I have to interpret this result? -- Testing was not possible due to ... (e.g., sf)? Jan -------- Original-Nachricht -------- Betreff: ! Compiling 'Current' pathfinder: Results DIFFER from 2008.04.02_00-09-03's ! Datum: Thu, 3 Apr 2008 02:20:39 +0200 (CEST) Von: monet(a)cwi.nl (Project account) An: jan.rittinger(a)in.tum.de, E.Sidirourgos(a)cwi.nl, Stefan.Manegold(a)cwi.nl, Sjoerd.Mullender(a)cwi.nl Compilation results of pathfinder in /ufs/monet/Testing/Current DIFFER from those of 2008.04.02_00-09-03 ! Summary: ======== * 2 Systems no longer relevant !! 12 Systems now DON'T produce output !! * 4 Systems still don't produce output * 55 Tests no longer relevant Maybe YOUR checkins since 02.04.2008, 00:09:03 caused some of the problems: buildtools MonetDB java clients MonetDB4 MonetDB5 pathfinder jan.rittinger(a)in.tum.de lsidir(a)cwi.nl Check http://monetdb.cwi.nl/Development/TestWeb/Current/pathfinder/ for details ! Details: ======== 2 Systems no longer relevant: F6-I64/32d, F6-I64/64d 12 Systems now DON'T produce output: D4.0-G32/32d, F6-G32/32d, F8-G32/32d, F8-G64/32d, F8-G64/64d, F8-G64/64s, F8-I64/32d, F8-I64/64d, G1.12.11.1-G32/32d, R4-I64/64d, S5.10-G32/32d, W5.1-I32/32d 4 Systems still don't produce output: D8.11.0-G32/32d, W5.1-M32/32d, W5.2-I64/64d, W5.2-M64/64d 55 Tests no longer relevant: conf/bin/monetdb-xquery-config, conf/bin/monetdb-xquery-config.bat, compiler/bin/pfsql, compiler/bin/pf, compiler/bin/pfopt, runtime/lib/MonetDB4/lib/lib_pf_standoff.so, runtime/lib/MonetDB4/lib/lib_pf_support.so, runtime/lib/MonetDB4/pf_standoff.mil, runtime/lib/MonetDB4/pathfinder.mil, runtime/lib/MonetDB4/xrpc_client.mil, runtime/lib/MonetDB4/pf_support.mil, runtime/lib/MonetDB4/xrpc_server.mil, runtime/lib/MonetDB4/lib/lib_xrpc_server.so, runtime/lib/MonetDB4/lib/lib_xrpc_client.so, runtime/lib/MonetDB4/lib/lib_pathfinder.so, runtime/xrpc/admin/share/MonetDB/xrpc/admin/content.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/ctit.png, runtime/xrpc/admin/share/MonetDB/xrpc/admin/sf.bmp, runtime/xrpc/admin/share/MonetDB/xrpc/admin/index.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/monetdb_med_abstract.png, runtime/xrpc/admin/share/MonetDB/xrpc/admin/admin.js, runtime/xrpc/admin/share/MonetDB/xrpc/admin/title.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/cwi_logo.p ng, runtime/xrpc/admin/share/MonetDB/xrpc/admin/xrpcwebclient.js, runtime/xrpc/admin/share/MonetDB/xrpc/admin/admin.xq, runtime/xrpc/admin/share/MonetDB/xrpc/admin/ctit.gif, runtime/xrpc/admin/share/MonetDB/xrpc/admin/multimedian_logo.png, runtime/xrpc/admin/share/MonetDB/xrpc/admin/tum-logo.gif, runtime/xrpc/admin/share/MonetDB/xrpc/admin/menu.html, runtime/xrpc/admin/share/MonetDB/xrpc/admin/cwi.bmp, runtime/xrpc/export/share/MonetDB/xrpc/export/index.html, runtime/xrpc/export/share/MonetDB/xrpc/export/xrpc.html, runtime/xrpc/export/share/MonetDB/xrpc/export/export.xq, runtime/xrpc/export/share/MonetDB/xrpc/export/xrpc-mod.xq, runtime5/lib/MonetDB5/xquery.mal, runtime5/lib/MonetDB5/lib/lib_xquery.so, modules/pftijah/lib/MonetDB4/pftijah.mil, modules/pftijah/lib/MonetDB4/lib/lib_pftijah.so, modules/probxml/lib/MonetDB4/lib/lib_probxml.so, modules/probxml/lib/MonetDB4/probxml.mil, src/tools/bin/Mserver, src/tools/bin/Mbeddedxq, src/tools/lib/libembeddedxq.so, src/sqlhelpers/ xmlshred/bin/pfshred, src/sqlhelpers/serializer/bin/pfserialize.pl, src/sqlhelpers/db2tools/bin/pfcreatetables.sh, src/sqlhelpers/db2tools/bin/pfloadtables.sh, dist/MonetDB-XQuery-0.23.0.tar.gz, rpm/rpm/rpmmacros, rpm/rpm/rpmrc, rpm/rpm/RPMS/MonetDB4-XQuery-0.23.0-1_.rpm, rpm/rpm/RPMS/MonetDB4-XQuery-debuginfo-0.23.0-1_.rpm, rpm/rpm/RPMS/MonetDB4-XQuery-devel-0.23.0-1_.rpm, rpm/rpm/SRPMS/MonetDB4-XQuery-0.23.0-1.src.rpm, UNKNOWN/rpm/SPECS/MonetDB-XQuery.spec -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

3 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra algopt.c, , 1.30, 1.31 planner.c, , 1.56, 1.57 properties.c, , 1.34, 1.35
by Keulen, M. van (Maurice) 02 Apr '08

02 Apr '08

Hi Jan, This one is likely to speed up my use of MXQ a lot! Does "references" also mean that a lot less elements will be constructed? One of the scalability bottlenecks that I frequently encounter in my experiments is not that it is slow, but that I create more than 2^32 elements (which doesn't fit in a bat with 32-bit oids). Avoiding subtree copies would probably improve the scalability of my experiments enormously. I must definitely push the move to the algebra backend :-) Maurice. Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv13092/algebra > > Modified Files: > algopt.c planner.c properties.c > Log Message: > -- Added node based properties that check (top-down) if ... > > ... the node may be used (indirectly) for serialization > > ... the node id may be used > > ... the node order may be used > > ... the node values may be accessed > > ... a downward axis is applied lateron > (child, descendant, descendant-or-self) > > ... a side-way axis is applied lateron > (following, following-sibling, preceding, preceding-sibling) > > ... an upward axis is applied lateron > (parent, ancestor, ancestor-or-self) > > ... a self axis is applied lateron > (self, ancestor-or-self, descendant-or-self) > > ... the node may be used as input to node construction > > -- Exploited the node base property that checks for the downward axis > property: > > For every content constructor whose input is not queried we can > use the physical shallow content constructor that makes use of > references instead of creating a subtree copy. > > (This sped up the execution time of XMark Q10 (scale factor 1) > by 60% and the execution time of XMark Q15 by 150%.) > > > U algopt.c > Index: algopt.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/algopt.c,v > retrieving revision 1.30 > retrieving revision 1.31 > diff -u -d -r1.30 -r1.31 > --- algopt.c 25 Feb 2008 15:37:04 -0000 1.30 > +++ algopt.c 1 Apr 2008 16:37:11 -0000 1.31 > @@ -339,6 +339,7 @@ > false /* composite key */, > true /* key */, > false /* ocols */, > + true /* req_node */, > false /* reqval */, > true /* level */, > true /* refctr */, > @@ -356,6 +357,7 @@ > true /* composite key */, > true /* key */, > true /* ocols */, > + true /* req_node */, > true /* reqval */, > true /* level */, > true /* refctr */, > > U planner.c > Index: planner.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/planner.c,v > retrieving revision 1.56 > retrieving revision 1.57 > diff -u -d -r1.56 -r1.57 > --- planner.c 26 Mar 2008 11:20:47 -0000 1.56 > +++ planner.c 1 Apr 2008 16:37:13 -0000 1.57 > @@ -1784,7 +1784,7 @@ > *(plan_t **) PFarray_at (R(n)->plans, i), > sortby (iter, pos))); > > - if (false /* magic flag to enable shallow content constructors */) > + if (!PFprop_node_content_queried (n->prop, att_item)) > /* for each plan, generate a constructor */ > for (unsigned int i = 0; i < PFarray_last (ordered_in); i++) > add_plan (ret, > > U properties.c > Index: properties.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/properties.c,v > retrieving revision 1.34 > retrieving revision 1.35 > diff -u -d -r1.34 -r1.35 > --- properties.c 14 Jan 2008 14:05:24 -0000 1.34 > +++ properties.c 1 Apr 2008 16:37:14 -0000 1.35 > @@ -122,9 +122,9 @@ > void > PFprop_infer (bool card, bool const_, bool set, > bool dom, bool icol, bool ckey, > - bool key, bool ocols, bool reqval, > - bool level, bool refctr, bool guides, > - bool ori_names, bool unq_names, > + bool key, bool ocols, bool req_node, > + bool reqval, bool level, bool refctr, > + bool guides, bool ori_names, bool unq_names, > PFla_op_t *root, PFguide_tree_t *guide) > { > PFprop_create_prop (root); > @@ -145,6 +145,8 @@ > PFprop_infer_guide (root, guide); > if (const_) > PFprop_infer_const (root); > + if (req_node) > + PFprop_infer_req_node (root); > if (reqval) > PFprop_infer_reqval (root); > if (dom) > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- ---------------------------------------------------------------------- Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen

2 1

[Monetdb-developers] rpm conflict, monetdb and fedora 8
by Bob McKay 02 Apr '08

02 Apr '08

Not sure if this is the right place to report this, but it seems there is a conflict between the current version of the monetdb fedora rpm install and an up-to-date fedora 8: Transaction Check Error: file /usr/lib/libmal.so.0 from install of MonetDB5-server-5.4.0-1.fc8.oid32.i386 conflicts with file from package libmal-0.31-7.fc8.i386 Any suggestions? (I'm not too keen on uninstalling libmal, as that entails also uninstalling significant parts of kde). When I first got the message I thought it might be that my fedora 8 was out-of-date, but I performed a full update and still get the problem.

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/test/mapi/Tests php_monetdb.stable.out, , 1.6, 1.7 php_monetdb.stable.out.Five, , 1.6, 1.7
by Martin Kersten 02 Apr '08

02 Apr '08

Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/test/mapi/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv17226/src/test/mapi/Tests > > Modified Files: > php_monetdb.stable.out php_monetdb.stable.out.Five > Log Message: > removed 'explain' variable, ie no more set explain etc > but always use the statement modifiers debug, explain, profile > > profile now only outputs the real, user and clk ticks per sec. > > The old profile functionality is taken over by a new statement modifier > 'record' > > Don't forget to update the website accordingly

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal rdf.mx, , 1.1, 1.2
by Romulo Goncalves 01 Apr '08

01 Apr '08

Marco Antonelli wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv9457/src/modules/mal > > Modified Files: > rdf.mx > Log Message: The log message is missing... It is always nice to have a summary of what you are check in... It also helps the developers in future debugging tasks.... Romulo > > > U rdf.mx > Index: rdf.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/rdf.mx,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- rdf.mx 26 Mar 2008 12:51:06 -0000 1.1 > +++ rdf.mx 1 Apr 2008 14:48:08 -0000 1.2 > @@ -41,7 +41,7 @@ > address RDFimportGraph > comment "import a graph in a named rdf db"; > > -command exportGraph( url:str, db:str, graph:str) : void > +command exportGraph( url:str, db:str, graph:str, order:str, syntax:str) : void > address RDFexportGraph > comment "export a graph in a named rdf db"; > > @@ -62,6 +62,7 @@ > #include <raptor.h> > > #define rdf_export extern > +#define RET_MAXLEN 2048 > > typedef struct { > lng *cur; > @@ -73,7 +74,7 @@ > oid *o; > oid *p; > oid *s; > - str ret; > + str *ret; > } triple_handler_vars; > //TODO > //static lng RDFestimate(str url) { > @@ -91,34 +92,31 @@ > > void message_handler(void *user_data, raptor_locator *locator, const char *message) > { > - unsigned int maxlength = 2048; > - char finalmessage[maxlength]; > size_t length = 0; > - char *msg = finalmessage; > str *ret = (str *)user_data; > + str msg = *ret; > if (locator->uri) > { > - snprintf(msg, maxlength, "In URI location \"%s\": ", raptor_uri_as_string(locator->uri)); > - length = strlen(finalmessage); > - msg = finalmessage + length*(sizeof(char)); > + snprintf(msg, RET_MAXLEN, "In URI location \"%s\": ", raptor_uri_as_string(locator->uri)); > + length = strlen(*ret); > + msg = *ret + length * (sizeof(char)); > } > - if (locator->line >=0 && length < maxlength-1) > + if (locator->line >=0 && length < RET_MAXLEN-1) > { > - snprintf(msg, maxlength-length, "at line %d: ", locator->line); > - length = strlen(finalmessage); > - msg = finalmessage + length*(sizeof(char)); > + snprintf(msg, RET_MAXLEN-length, "at line %d: ", locator->line); > + length = strlen(*ret); > + msg = *ret + length * (sizeof(char)); > } > - if (locator->column >=0 && length < maxlength-1) > + if (locator->column >=0 && length < RET_MAXLEN-1) > { > - snprintf(msg, maxlength-length, "at column %d: ", locator->column); > - length = strlen(finalmessage); > - msg = finalmessage + length*(sizeof(char)); > + snprintf(msg, RET_MAXLEN-length, "at column %d: ", locator->column); > + length = strlen(*ret); > + msg = *ret + length * (sizeof(char)); > } > - if (message && length < maxlength-1) > + if (message && length < RET_MAXLEN-1) > { > - snprintf(msg, maxlength-length, "%s\n", message); > + snprintf(msg, RET_MAXLEN-length, "%s\n", message); > } > - *ret=finalmessage; > } > > @= RDFins > @@ -133,7 +131,7 @@ > bun = (BUN) map->batCount; > if (!BUNappend(map,(ptr)@1v, TRUE)) { > GDKerror("RDFimport: @1[" LLFMT "] = %s, BUNappend fails\n", cur, @1v); > - data->ret = RDFimport_fail(map,ob,pb,sb); > + *(data->ret) = RDFimport_fail(map,ob,pb,sb); > } > } > /* s,p,o[cur] are set to the virtual oid which is bun s p and o are arrays of oid */ > @@ -153,9 +151,10 @@ > oid *o = data->o; > oid *p = data->p; > oid *s = data->s; > + str *ret = data->ret; > BATiter mi = bat_iterator(BATmirror(map)); > BUN bun; > - if (data->ret!=MAL_SUCCEED) > + if (strlen(*ret)!=0) > return; > /* get the subject */ > if (triple->subject_type == RAPTOR_IDENTIFIER_TYPE_RESOURCE) > @@ -164,7 +163,7 @@ > sv = (str)triple->subject; > else > { > - data->ret="Unexpected subject type"; > + *ret="Unexpected subject type"; > return; > } > > @@ -173,7 +172,7 @@ > pv = (str)raptor_uri_as_string((raptor_uri *)triple->predicate); > else > { > - data->ret="Unexpected predicate type"; > + *ret="Unexpected predicate type"; > return; > } > > @@ -190,18 +189,18 @@ > ov = (str)raptor_uri_as_string((raptor_uri*)triple->object); > else > { > - data->ret="Unexpected object type"; > + *ret="Unexpected object type"; > return; > } > > @:RDFins(o)@ > - if (data->ret == MAL_SUCCEED) > + if (strlen(*ret)==0) > { > @:RDFins(p)@ > - if (data->ret == MAL_SUCCEED) > + if (strlen(*ret)==0) > { > @:RDFins(s)@ > - if (data->ret == MAL_SUCCEED) > + if (strlen(*ret)==0) > { > if (++(*cur) > *lim) { > *lim *= 2; > @@ -209,7 +208,7 @@ > pb = BATextend(pb, *lim); > sb = BATextend(sb, *lim); > if (ob == NULL || pb == NULL || sb == NULL) > - data->ret = RDFimport_fail(map,ob,pb,sb); > + *ret = RDFimport_fail(map,ob,pb,sb); > o = (oid *) Tloc(ob, BUNfirst(ob)); > p = (oid *) Tloc(pb, BUNfirst(pb)); > s = (oid *) Tloc(sb, BUNfirst(sb)); > @@ -223,6 +222,7 @@ > RDFimport_(BAT** S, BAT** P, BAT **O, str url, BAT *map) { > raptor_parser *parser; > raptor_uri *rdf_file; > + str ret; > raptor_init(); > parser = raptor_new_parser("rdfxml"); > if (parser) > @@ -244,41 +244,15 @@ > p = (oid *) Tloc(pb, BUNfirst(pb)); > s = (oid *) Tloc(sb, BUNfirst(sb)); > > - //char obuf[20],*op, pbuf[20],*pp, sbuf[20],*sp; > - //str ov, pv, sv; > - > if (map == NULL || ob == NULL || pb == NULL || sb == NULL) > return RDFimport_fail(map,ob,pb,sb); > - > - > - > -// strcpy(obuf, "object"); for(ov=op=obuf; *op; op++); op[1]=0; > -// strcpy(pbuf, "predicate"); for(pv=pp=pbuf; *pp; pp++); pp[1]=0; > -// strcpy(sbuf, "subject"); for(sv=sp=sbuf; *sp; sp++); sp[1]=0; > -// while (cur < 8 /*redland gives me (ov,pv,sv)*/ ) { > -// *op = (cur & 4)?'1':'0'; > -// *pp = (cur & 2)?'1':'0'; > -// *sp = (cur & 1)?'1':'0'; > -// RDFins(o) /* mx macros*/ > -// RDFins(p) > -// RDFins(s) > -// > -// if (++cur > lim) { > -// lim *= 2; > -// ob = BATextend(ob, lim); > -// pb = BATextend(pb, lim); > -// sb = BATextend(sb, lim); > -// if (ob == NULL || pb == NULL || sb == NULL) > -// return RDFimport_fail(map,ob,pb,sb); > -// o = (oid *) Tloc(ob, BUNfirst(ob)); > -// p = (oid *) Tloc(pb, BUNfirst(pb)); > -// s = (oid *) Tloc(sb, BUNfirst(sb)); > -// } > -// } > > - data = malloc(sizeof(triple_handler_vars)); > + data = GDKmalloc(sizeof(triple_handler_vars)); > if (data) > { > + char ret_array[RET_MAXLEN]; > + str ret_ptr = ret_array; > + ret_array[0] = '\0'; > data->cur = cur; > data->lim = lim; > data->map =map; > @@ -288,18 +262,18 @@ > data->s = s; > data->p = p; > data->o = o; > - data->ret = MAL_SUCCEED; > + data->ret = &ret_ptr; > raptor_set_statement_handler(parser, data, triples_handler); > - raptor_set_fatal_error_handler(parser, &data->ret, message_handler); > - raptor_set_error_handler(parser, &data->ret, message_handler); > - raptor_set_warning_handler(parser, &data->ret, message_handler); > - //raptor_set_parser_strict(parser, TRUE); > + raptor_set_fatal_error_handler(parser, &ret_ptr, message_handler); > + raptor_set_error_handler(parser, &ret_ptr, message_handler); > + raptor_set_warning_handler(parser, &ret_ptr, message_handler); > + raptor_set_parser_strict(parser, TRUE); > > if (raptor_parse_file(parser, rdf_file, NULL) == 0) > { > - /* do something with redland */ > - if (data->ret == MAL_SUCCEED) > + if(strlen(ret_ptr)==0) > { > + ret = MAL_SUCCEED; > BATseqbase(ob, 0); > BATseqbase(pb, 0); > BATseqbase(sb, 0); > @@ -312,30 +286,31 @@ > BATkey(BATmirror(ob), FALSE); > BATkey(BATmirror(sb), FALSE); > BATkey(BATmirror(pb), FALSE); > - > - free(data); > - raptor_free_parser(parser); > - raptor_free_uri(rdf_file); > - raptor_finish(); > - return MAL_SUCCEED; > } > - else > - throw(MAL, "rdf.import", data->ret); > } > else > - throw(MAL, "rdf.import", "RDF parsing error"); > + ret = "RDF parsing error"; > + GDKfree(data); > } > else > - throw(MAL, "rdf.import", "triple_handler_vars malloc failed"); > + ret = "triple_handler_vars malloc failed"; > + raptor_free_uri(rdf_file); > } > else > - throw(MAL, "rdf.import", "Unexistent file: \"%s\"", url); > + ret = "Unexistent url"; > + raptor_free_parser(parser); > } > else > - throw(MAL, "rdf.import", "Could not create a Redland parser"); > + ret = "Could not create a Redland parser"; > + raptor_finish(); > + if (ret != MAL_SUCCEED) > + throw(MAL, "rdf.import", ret); > + > + return MAL_SUCCEED; > } > > -rdf_export str RDFimport(int* S, int* P, int* O, str *url, bat m) > +rdf_export > +str RDFimport(int* S, int* P, int* O, str *url, bat m) > { > BAT *sb = NULL, *ob=NULL, *pb = NULL, *map = BATdescriptor(m); > str ret; > @@ -352,45 +327,109 @@ > } > > static str > -RDFexport_(str url, bat M, bat S, bat P, bat O) { > - str ret = "RDFexport failed to obtain bats"; > +RDFexport_(str url, bat M, bat S, bat P, bat O, str syntax) > +{ > + str ret = MAL_SUCCEED; > BAT *map = BATdescriptor(M); > - if (map) { > + (void)url; > + if (map) > + { > BAT *sb = BATdescriptor(S); > - if (sb) { > + if (sb) > + { > BAT *pb = BATdescriptor(P); > - if (pb) { > + if (pb) > + { > BAT *ob = BATdescriptor(O); > - if (ob) { > - lng i, cnt = BATcount(ob); > + if (ob) > + { > + raptor_serializer* rdf_serializer; > + const char *filename="/ufs/antonell/Desktop/output.rdf"; > + lng i, cnt = BATcount(ob); > oid *o = (oid *) Tloc(ob, BUNfirst(ob)); > oid *p = (oid *) Tloc(pb, BUNfirst(pb)); > oid *s = (oid *) Tloc(sb, BUNfirst(sb)); > //var_t position in heap > var_t *off = (var_t *) Tloc(map, BUNfirst(map)); > str base = (str) map->T->vheap->base;//heap first position > - > - for(i=0; i <cnt; i++) { > - str ov = base + off[o[i]]; > - str pv = base + off[p[i]]; > - str sv = base + off[s[i]]; > - stream_printf(GDKout, "'%s' '%s' '%s' '%s'\n", ov, pv, sv, url); > + > + raptor_init(); > + rdf_serializer = raptor_new_serializer(syntax); > + > + if (rdf_serializer) > + { > + char ret_array[RET_MAXLEN]; > + str ret_ptr = ret_array; > + ret_array[0] = '\0'; > + raptor_serializer_set_error_handler(rdf_serializer, &ret_ptr, message_handler); > + raptor_serializer_set_warning_handler(rdf_serializer, &ret_ptr, message_handler); > + if (raptor_serialize_start_to_filename(rdf_serializer, filename) == 0) > + { > + for(i=0; i <cnt; i++) > + { > + if (strlen(ret_ptr)==0) > + { > + raptor_statement *triple; > + str ov = base + off[o[i]]; > + str pv = base + off[p[i]]; > + str sv = base + off[s[i]]; > + //stream_printf(GDKout, "'%s' '%s' '%s' '%s'\n", ov, pv, sv, url); > + /* Make a triple with URI subject, URI predicate, literal object */ > + triple=GDKmalloc(sizeof(raptor_statement)); > + > + triple->subject=(void*)raptor_new_uri((const unsigned char*)sv); > + triple->subject_type=RAPTOR_IDENTIFIER_TYPE_RESOURCE; > + > + triple->predicate=(void*)raptor_new_uri((const unsigned char*)pv); > + triple->predicate_type=RAPTOR_IDENTIFIER_TYPE_RESOURCE; > + > + triple->object=ov; > + triple->object_type=RAPTOR_IDENTIFIER_TYPE_LITERAL; > + //triple->object_literal_language=(const unsigned char*)"en"; > + triple->object_literal_language=NULL; > + triple->object_literal_datatype = NULL; > + /* Write the triple */ > + if (raptor_serialize_statement(rdf_serializer, triple)!=0) > + ret="failed to serialize a statement"; > + > + /* Delete the triple */ > + raptor_free_uri((raptor_uri*)triple->subject); > + raptor_free_uri((raptor_uri*)triple->predicate); > + GDKfree(triple); > + } > + } > + ret = MAL_SUCCEED; > + raptor_serialize_end(rdf_serializer); > + } > + else > + ret = "raptor_serialize_start failed"; > + raptor_free_serializer(rdf_serializer); > } > - ret = MAL_SUCCEED; > + else > + ret = "raptor_new_serializer failed"; > + raptor_finish(); > BBPreleaseref(ob->batCacheid); > - } > + } > + else > + ret = "Failed to obtain O bat"; > BBPreleaseref(pb->batCacheid); > - } > + } > + else > + ret = "Failed to obtain P bat"; > BBPreleaseref(sb->batCacheid); > - } > + } > + else > + ret = "Failed to obtain S bat"; > BBPreleaseref(map->batCacheid); > - } > + } > + else > + ret = "Failed to obtain MAP bat"; > return ret; > } > > str > RDFexport(str *url, bat M, bat S, bat P, bat O) { > - str ret = RDFexport_(*url, M, S, P, O); > + str ret = RDFexport_(*url, M, S, P, O, "dummy syntax"); > if (ret != MAL_SUCCEED) > throw(MAL, "RDFexport", ret); > return MAL_SUCCEED; > @@ -513,23 +552,23 @@ > > ret = "RDFimportGraph: sorting failed"; > if (sorted) { > - ret = @:sort(sb, pb, ob, Spo, sPo, spO)@ > + ret = @:sort(sb,pb,ob,Spo,sPo,spO)@ > if (ret == MAL_SUCCEED) { > - ret = @:sort(sb, ob, pb, Sop, sOp, soP)@ > + ret = @:sort(sb,ob,pb,Sop,sOp,soP)@ > if (ret == MAL_SUCCEED) { > BBPreclaim(sorted); > sorted = BATmirror(BATsort(BATmirror(pb))); > if (sorted) { > - ret = @:sort(pb, ob,sb,Pos,pOs,poS)@ > + ret = @:sort(pb,ob,sb,Pos,pOs,poS)@ > if (ret == MAL_SUCCEED) { > - ret = @:sort(pb, sb,ob,Pso,pSo,psO)@ > + ret = @:sort(pb,sb,ob,Pso,pSo,psO)@ > if (ret == MAL_SUCCEED) { > BBPreclaim(sorted); > sorted = BATmirror(BATsort(BATmirror(ob))); > if (sorted) { > - ret = @:sort(ob, pb, sb, Ops, oPs, opS)@ > + ret = @:sort(ob,pb,sb,Ops,oPs,opS)@ > if (ret == MAL_SUCCEED) { > - ret = @:sort(ob, sb, pb,Osp,oSp,osP)@ > + ret = @:sort(ob,sb,pb,Osp,oSp,osP)@ > if (ret == MAL_SUCCEED) { > ret = MAL_SUCCEED; > goto succeed; > @@ -572,15 +611,84 @@ > throw(MAL, "RDFimportGraph", ret); > return MAL_SUCCEED; > } > -str RDFexportGraph(str *url, str *db, str *graph) { > + > +static str getBatNames(str db, str graph, char *s_buf,char *p_buf,char *o_buf, str order) > +{ > + str err = "order may be only one among \"spo\",\"sop\",\"pso\",\"pos\",\"osp\",\"ops\""; > + if (strlen(order)>3) > + return err; > + if (strcmp(order, "spo")==0) > + { > + snprintf(s_buf, 1024, "%s_%s_Spo", db, graph); > + snprintf(p_buf, 1024, "%s_%s_sPo", db, graph); > + snprintf(o_buf, 1024, "%s_%s_spO", db, graph); > + } > + else if (strcmp(order, "sop")==0) > + { > + snprintf(s_buf, 1024, "%s_%s_Sop", db, graph); > + snprintf(o_buf, 1024, "%s_%s_sOp", db, graph); > + snprintf(p_buf, 1024, "%s_%s_soP", db, graph); > + } > + else if (strcmp(order, "pos")==0) > + { > + snprintf(p_buf, 1024, "%s_%s_Pos", db, graph); > + snprintf(o_buf, 1024, "%s_%s_pOs", db, graph); > + snprintf(s_buf, 1024, "%s_%s_poS", db, graph); > + } > + else if (strcmp(order, "pso")==0) > + { > + snprintf(p_buf, 1024, "%s_%s_Pso", db, graph); > + snprintf(s_buf, 1024, "%s_%s_pSo", db, graph); > + snprintf(o_buf, 1024, "%s_%s_psO", db, graph); > + } > + else if (strcmp(order, "osp")==0) > + { > + snprintf(o_buf, 1024, "%s_%s_Osp", db, graph); > + snprintf(s_buf, 1024, "%s_%s_oSp", db, graph); > + snprintf(p_buf, 1024, "%s_%s_osP", db, graph); > + } > + else if (strcmp(order, "ops")==0) > + { > + snprintf(o_buf, 1024, "%s_%s_Ops", db, graph); > + snprintf(p_buf, 1024, "%s_%s_oPs", db, graph); > + snprintf(s_buf, 1024, "%s_%s_opS", db, graph); > + } > + else > + return err; > + return MAL_SUCCEED; > +} > + > +static str checkSyntax(str syntax) > +{ > + if ( //!strcmp(syntax, "atom") || > + !strcmp(syntax, "ntriples") || > + !strcmp(syntax, "rdfxml") || > + !strcmp(syntax, "rdfxml-abbrev") || > + !strcmp(syntax, "rdfxml-xmp") || > + !strcmp(syntax, "turtle") || > + //!strcmp(syntax, "rdfxml-xmp") || > + !strcmp(syntax, "rss-1.0") || > + !strcmp(syntax, "dot")) > + { > + return MAL_SUCCEED; > + } > + else > + return "syntax may be only one among \"atom\",\"ntriples\",\"rdfxml\",\"rdfxml-abbrev\",\"rdfxml-xmp\",\"turtle\",\"rdfxml-xmp\",\"rss-1.0\",\"dot\""; > +} > + > +str RDFexportGraph(int *d, str *url, str *db, str *graph, str *order, str *syntax) > +{ > char m_buf[1024], o_buf[1024], p_buf[1024], s_buf[1024]; > bat map, ob, pb, sb; > - str ret = "RDFexportGraph failed to obtain bats"; > - > + str ret; > + (void)d; > + ret = checkSyntax(*syntax); > + if (ret!= MAL_SUCCEED) > + throw(MAL, "RDFexportGraph", ret); > + ret = getBatNames(*db, *graph, s_buf, p_buf, o_buf, *order); > + if (ret!= MAL_SUCCEED) > + throw(MAL, "RDFexportGraph", ret); > snprintf(m_buf, 1024, "%s_%s_map", *db, *graph); > - snprintf(p_buf, 1024, "%s_%s_Ops", *db, *graph); > - snprintf(o_buf, 1024, "%s_%s_oPs", *db, *graph); > - snprintf(s_buf, 1024, "%s_%s_opS", *db, *graph); > map = BBPindex(m_buf); > if (map) { > sb = BBPindex(s_buf); > @@ -589,11 +697,19 @@ > if (pb) { > ob = BBPindex(o_buf); > if (ob) { > - ret = RDFexport_(*url, map, sb, pb, ob); > + ret = RDFexport_(*url, map, sb, pb, ob, *syntax); > } > + else > + ret = "Failed to obtain O bat"; > } > + else > + ret = "Failed to obtain P bat"; > } > + else > + ret = "Failed to obtain S bat"; > } > + else > + ret = "Failed to obtain map bat"; > if (ret != MAL_SUCCEED) > throw(MAL, "RDFexportGraph", ret); > return MAL_SUCCEED; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

2 1

[Monetdb-developers] blobs performance
by Henrik Vendelbo 30 Mar '08

30 Mar '08

I am working through the options for clustered file management. Normally I would expect fairly poor performance of binary file storage in a database. Hence I would favor using the file system, and sync across the machines in the cluster. Being quite different from common relational implementations, how well does Monet handle blobs of 1-10Mb? As a side question is there a support for multi-master replication? The technology looks really impressive, I hope to be able to base my future projects on MonetDB Thanks, Henrik

2 1

Re: [Monetdb-developers] Monetdb-developers Digest, Vol 22, Issue 4
by Peter Boncz 28 Mar '08

28 Mar '08

Hi, I read the digests of the monetdb mailing list, which in this case took more than a month to send me this discussion. That's pretty useless.. I will switch to immediate mails. Anyway, Jennie is currently working on getting the repeatable reads transactions working. This means that by passing a request-ID you will be guaranteed that all queries and updates execute on the same database image, i.e. a transaction. In case of update requests in the same transaction, these will be executed atomically using a 2PC protocol. While this functionality is designed to be triggerered from XQuery queries, you client API could also generate the appropriate messages. It may not answer you needs regarding performance (you would send a read-request, update-request, prepare-request and commit-request), but it would answer your functional requirements AFAIS. Peter > Message: 1 > Date: Wed, 19 Mar 2008 12:30:20 +0100 > From: "Keulen, M. van (Maurice)" <m.vankeulen(a)utwente.nl> > Subject: Re: [Monetdb-developers] [XQ] Mixing update/docmgmt/query > To: monetdb-developers(a)lists.sourceforge.net > Message-ID: <47E0F94C.4000709(a)utwente.nl> > Content-Type: text/plain; charset=windows-1252; format=flowed > > Ying Zhang wrote: > > > >> Jennie: my SOAP-messages do not come from MonetDB/XQuery, but from > >> outside (i.e., some scripting language). > > > > Then, an order can only be guaranteed if you, in your > scripts, send a > > message, wait for result, and send the next message. In all other > > cases, unfortunately, no guarantee, as an xrpc server immediately > > start handling a request when it's received. > > > > Jennie > > I thought so already. This is why I don't like to have to > send multiple > requests to do one thing (transaction). I'd rather be able to > send one > request to both do, say, an update and return a result, or add a > document and return a result. The system so far doesn't allow > me to do > this, because I cannot define one query (or function in a > module) that > does both because of typing restrictions, and I also cannot send two > separate queries in one request because of restrictions in the XRPC > protocol. > > Any thoughts about achieving what I want? > > Maurice. > > -- > ---------------------------------------------------------------------- > Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology > Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, > Netherlands > Email: m.vankeulen(a)utwente.nl, Phone: +31 534893688, Fax: +31 > 534892927 > Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen *********************************

3 8