October 2007 - developers-list

Re: [Monetdb-developers] Simple bat operations
by darabi＠web.de 12 Oct '07

12 Oct '07

Hello Nils, OK, then something doesn't work. When I try your suggestion, I get this: user@host$ mclient -d demo -l mal mal>b := bat.new(:int,:int); mal>io.print(b); #-----------------# # h t # name # int int # type #-----------------# mal>bat.setName(b,"test"); mal>bat.setPersistent(b,true); mal>transaction.commit(); mal>\q user@host$ mclient -d demo -l mal mal>b:= bbp.bind("test"); mal>io.print(b); MAPI = monetdb@127.0.0.1:50001 QUERY = io.print(b); ERROR = !TypeException:user.main[1]:'b' may not be used before being initialized mal>\q I have downloaded and compiled MonetDB-Mars-SuperBall-SR3.tar.bz2 on an Ubuntu i686 single cpu system running a 2.6.20 kernel. user@host$ ps auxwww | grep mserver user 12072 0.0 4.3 133232 67268 ? Ssl Oct10 0:01 /opt/monetdb/bin/mserver5 --dbfarm=/opt/monetdb/var/MonetDB5/dbfarm --set daemon yes --set monet_welcome no --dbname=voc --config=/opt/monetdb/var/MonetDB5/dbfarm/voc/.monetdb.conf /opt/monetdb/var/MonetDB5/dbfarm/voc/.mserver5.dbinit user 12479 0.0 3.9 109320 61760 ? Ssl Oct10 0:00 /opt/monetdb/bin/mserver5 --dbfarm=/opt/monetdb/var/MonetDB5/dbfarm --set daemon yes --set monet_welcome no --dbname=demo --config=/opt/monetdb/var/MonetDB5/dbfarm/demo/.monetdb.conf user@host$ mserver5 --version MonetDB Server v5.0.0 Copyright (c) 1993-2007 CWI, all rights reserved Compiled by: sjoerd(a)localhost.localdomain Compilation: gcc -O2 -std=c99 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt Linking: ld -IPA -m elf_x86_64 I already tried 'mclient -l sql' without any apparent problems, so it appeared unlikely that there were bigger issues. I will try to run the test suite(s). Thank you very much Kambiz ----- Original Message ----- From: "Niels Nes" <Niels.Nes(a)cwi.nl> To: darabi(a)web.de Sent: Donnerstag, 11. Oktober 2007 16.24 Uhr (GMT+0100) Europe/Berlin Subject: Re: [Monetdb-developers] Simple bat operations On Thu, Oct 11, 2007 at 02:34:15PM +0200, darabi(a)web.de wrote: > In the meantime, I have also tried all the bbp functions which > I could think of: > > bbp.find > bbp.bind > bbp.take > > But none of them gives me a bat which I can use. Strangely, none of > them says anything about whether the function call succeeded > or not. okay here a small example then b := bat.new(:int,:int); io.print(b); bat.setName(b,"test"); bat.setPersistent(b,true); transaction.commit(); # after server restart the following should work. b:= bbp.bind("test"); io.print(b); Niels > > > Kambiz > > ----- Original Message ----- > From: darabi(a)web.de > To: monetdb-developers(a)lists.sourceforge.net > Cc: "Niels Nes" <Niels.Nes(a)cwi.nl> > Sent: Donnerstag, 11. Oktober 2007 9.47 Uhr (GMT+0100) Europe/Berlin > Subject: Re: [Monetdb-developers] Simple bat operations > > Niels, > > thank you very much for your help. > > How can I access this bat, when I start a new session? > > bat.load("person_name"); > > doesn't seem to work. io.print(n): > > ERROR = !TypeException:user.main[1]:'n' may not be used before being initialized > > > Thanks > > Kambiz > > ----- Original Message ----- > From: "Niels Nes" > To: darabi(a)web.de > Cc: monetdb-developers(a)lists.sourceforge.net > Sent: Donnerstag, 11. Oktober 2007 9.38 Uhr (GMT+0100) Europe/Berlin > Subject: Re: [Monetdb-developers] Simple bat operations > > On Thu, Oct 11, 2007 at 08:00:36AM +0200, darabi... wrote: > > Hello, > > > > sorry for asking so many dumb questions, but I don't seem > > to be able to find the answers in the docs. > > > > I try to create and persist a bat: > > > > (mclient -d demo -l mal) > > > > n := bat.new(:oid,:str); > > bat.setName(n, "person_name"); > > bat.setPersistent(n, true); > > > > bat.append(n, "John", true); > > bat.append(n, "James", true); > > > > replace this save > > bat.save(n); > by > > transaction.commit(); > > Niels > > (ps later you probably want to switch to subcommit, ie only commit those > bat that changed, for this see the documenation in the transaction module > (src/modules/mal/transaction.mx) > > > > But this doesn't seem to be sufficient. > > > > Or at least I then don't know how to load this bat the next > > time I start mclient. > > > > Is there a file (one of the tests or sth.) which contains > > such basic tests? I tried to find-grep over them, but didn't > > succeed in finding what I was looking for. > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

6 20

Re: [Monetdb-developers] Monetdb-developers Digest, Vol 17, Issue 6
by Peter Boncz 11 Oct '07

11 Oct '07

Dear Stefan, Thanks for your question, but I regret to inform you that your way of communicating technical problems severely falls short of what is required by technical etiquette: (1) ambiguous software version description: you cannot be running MonetDB4 and MonetDB5 at the same time. Even more, I want to know the exact version number of the software, whether it was a 32-bits or 64-bits build (and if so, whether it used 64-bits or 32-bits oids). (2) major omissions in your platform description. I presume you have installed an operating system on your Xeon. Would you care to tell us which operating system that is? (3) ill-described reproduction path. Your bug report suggests being about updates, but as far as I read your email, I doubt you did any updates yet. This is again is crucial information. Thus, my main answer is that you have to retry describing your problem, paying better attention to the above three points. The error you see is the "remap" call failing -- the most probable cause of error is VM space shortage. For MonetDB to work on 20GB size datasets you must use a 64-bits operating system and MonetDB binary, and may even 64-bits oids (because your volume of text nodes is likely to be significant). Peter Boncz

2 3

[Monetdb-developers] Simple bat operations
by darabi＠web.de 11 Oct '07

11 Oct '07

Hello, sorry for asking so many dumb questions, but I don't seem to be able to find the answers in the docs. I try to create and persist a bat: (mclient -d demo -l mal) n := bat.new(:oid,:str); bat.setName(n, "person_name"); bat.setPersistent(n, true); bat.append(n, "John", true); bat.append(n, "James", true); bat.save(n); But this doesn't seem to be sufficient. Or at least I then don't know how to load this bat the next time I start mclient. Is there a file (one of the tests or sth.) which contains such basic tests? I tried to find-grep over them, but didn't succeed in finding what I was looking for. Thanks you for your patience. Kambiz

2 3

[Monetdb-developers] Simple bat operations
by Kambiz Darabi 11 Oct '07

11 Oct '07

Hello, sorry for asking so many dumb questions, but I don't seem to be able to find the answers in the docs. I try to create and persist a bat: (mclient -d demo -l mal) n := bat.new(:oid,:str); bat.setName(n, "person_name"); bat.setPersistent(n, true); bat.append(n, "John", true); bat.append(n, "James", true); bat.save(n); But this doesn't seem to be sufficient. Or at least I then don't know how to load this bat the next time I start mclient. Is there a file (one of the tests or sth.) which contains such basic tests? I tried to find-grep over them, but didn't succeed in finding what I was looking for. Thanks you for your patience. Kambiz

1 0

[Monetdb-developers] Fixing the update issue for Large XML documents
by Stefan de Konink 11 Oct '07

11 Oct '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, A short introduction. I'm Stefan de Konink, one of the students in the ADT course at the University of Amsterdam. I have a degree in Advanced Computer Science, so I know what C looks like ;) Currently I can use MonetDB on a rather heavy machine, basically to benchmark the storage of this machine. And look for bottle necks. I am also a participant of OpenStreetMap NL, as datacollector from already existing GIS data. OpenStreetMap runs on MySQL in the UK, and I wanted to find out if I could run it more efficiently on MonetDB. For compatibility reasons OSM publishes an XML file containing the current view of the database daily. My 'job' to see if I could get the same results on this XML file (using XQuery) as the active MySQL/Ruby implementation. Importing a 20GB XML file was great, but now the 'lessons' progressed we should update the data. So I took a recent version of the document (they migrated from one format to another this week), allocated 5% slackspace and run the pf:add-doc. This went ok, but the trick to count all nodes ended up in: > xquery> > more> fn:count(doc("planet.osm")//*) > more>MAPI = monetdb@localhost:50000 > QUERY = fn:count(doc("planet.osm")//*) > ERROR = !ERROR: [remap]: 5 times inserted nil due to errors at tuples > 1@0, 2@0, 3@0, 4@0, 5@0. > !ERROR: [remap]: first error was: > !ERROR: CMDremap: operation failed. > !ERROR: interpret_unpin: [remap] bat=492,stamp=-729 OVERWRITTEN > !ERROR: BBPdecref: 1000000001_rid_nid does not have pointer fixes. > !ERROR: interpret_params: leftfetchjoin(param 2): evaluation error. (As posted to the bugtracker and privately to the lecturer.) My first guess was that the 1..5 maybe had something to do with the 'slackspace' that was five also. Today I imported the same document readonly without any problems. To confirm 'import-updating' in general works I took the data from only The Netherlands and imported this. (143MB) Which worked like a charm. I'm running MonetDB4/5 (SR3) on a Quad Xeon, 8GB memory. Storage is a bit variable ;) But currently on local raid5. Are there any suggestions from the developers to catch this bug? Upgrading to CVS could be an option. Trying again with CVS is an option, trying again with a smaller document too, or if someone instantly comes up with a solution that could be nice too ;) Yours Sincerely, Stefan de Konink -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHDXXIYH1+F2Rqwn0RCk/VAJwLpXp0Yl1ENToPMRFgIJy2No/BMgCdFv7C dGQzJgoGmwM9ucPxrohIkFk= =sHEr -----END PGP SIGNATURE-----

1 0

Re: [Monetdb-developers] Object persistence using MonetDB
by darabi＠web.de 10 Oct '07

10 Oct '07

Thank you for the information. So, does that mean that you generally do not recommend the use of MonetDB beyond the realms of > datawarehouses, information retrieval, and data mining ? >From the result of the web application contest in the ct magazine, I had the impression that 'normal' applications could also benefit from the architecture of MonetDB: http://sourceforge.net/forum/forum.php?forum_id=584739 What I would like to do is to create a persistence engine for a Prolog like language which has a type system. Which means that data is to be stored in tuples. To experiment with MAL, I would have to use the MapiClient, right? I think I will try to play with it. Thank you Kambiz ----- Original Message ----- From: "Martin Kersten" <Martin.Kersten(a)cwi.nl> To: darabi(a)web.de, monetdb-developers-bounces(a)lists.sourceforge.net Sent: Mittwoch, 10. Oktober 2007 11.01 Uhr (GMT+0100) Europe/Berlin Subject: Re: [Monetdb-developers] Object persistence using MonetDB darabi(a)web.de wrote: > Hello, > > I'm a developer located in Germany with a current interest in > object persistence for a dynamically typed language. > Hello, thank you for your interest in the MonetDB software. > I was experimenting with something along the lines of the > decomposed storage model when I found out that it was (of course) > decades old and found the papers about it and MonetDB. > > Many thanks for your efforts and for this great piece of software. > > I was reading the 1995 CWI report 'High Performance support for > OO traversals in monet' and was wondering whether the ideas > in it are still present in today's MonetDB. And if not, how > much effort would it take to implement OO persistence using > MAL. I would also appreciate any pointer to papers, code samples > etc. in this direction. > Between 1995 and 2003 MonetDB was used in the company Data Distilleries (now part of SPSS) where the interaction between application and DBMS was based on persistent objects. However, the thrust of the development is not on individual object access, but bulk processing as it emerges from datawarehouses, information retrieval, and data mining. > If I understand correctly, in the beginning, MonetDB was rather > heading towards a OODBMS. Was there any reason why you didn't > pursue that goal? Is there any road block? > There are (have been) attempts to hook up Hibernate and MonetDB. I don't know their status. If ultra-high performance is your target, I question if a DBMS is the right way to go. However, I also think that most o-o applications might better benefit from relational view on data. regards, Martin > > Thank you very much for your help. > > > Kambiz Darabi >

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util CmdLineOpts.java, XQuery_0-20, NONE, 1.1.2.1 OptionsException.java, XQuery_0-20, NONE, 1.1.2.1
by Fabian Groffen 10 Oct '07

10 Oct '07

On 10-10-2007 11:01:20 +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/src/tools/java/src/nl/cwi/monetdb/xquery/util > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8103/src/tools/java/src/nl/cwi/monetdb/xquery/util > > Added Files: > Tag: XQuery_0-20 > CmdLineOpts.java OptionsException.java > Log Message: > Moved the XRPC Wrapper stuff into pathfinder/src/tools/java, because > XRPC Wrapper is not a client program and it is XQuery related. > > The XRPC Wrapper is build in a separate RPM package > (MonetDB4-XQuery-xrpcwrapper), so that the > pathfinder package is still Java-independent. Please note that this is code replication. (which IMHO is BAD).

2 3

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/NT monetdb_winconfig.h, , 1.2, 1.3
by Niels Nes 10 Oct '07

10 Oct '07

On Wed, Oct 10, 2007 at 09:05:45AM +0000, Romulo Goncalves wrote: > Update of /cvsroot/monetdb/MonetDB/NT > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv14659 > > Modified Files: > monetdb_winconfig.h > Log Message: > Move the definition of ssize_t as int into the win32 else clause > Please check if it is correct like this. The compilation works, but it gives warnnings > about probable lost of data witht the convertion from ssize_t to BUN. Shouldn't this also be done on the stable? The move of the line is fine. BUN in cvs head is still a 'size_t' (same as oid), ie ssize_t (signed) indeed may lose info. Niels > > > Index: monetdb_winconfig.h > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/NT/monetdb_winconfig.h,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- monetdb_winconfig.h 3 Oct 2007 13:59:59 -0000 1.2 > +++ monetdb_winconfig.h 10 Oct 2007 09:05:43 -0000 1.3 > @@ -668,13 +668,14 @@ > > /* The size of a `void *', as computed by sizeof. */ > #define SIZEOF_VOID_P 4 > -#endif > + > > /* The size of a `ssize_t', as computed by sizeof. */ > #define SIZEOF_SSIZE_T SIZEOF_SIZE_T > typedef int ssize_t; > #define HAVE_SSIZE_T 1 > > +#endif > /* The size of a `__int64', as computed by sizeof. */ > #define SIZEOF___INT64 8 > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

3 2

[Monetdb-developers] Object persistence using MonetDB
by darabi＠web.de 10 Oct '07

10 Oct '07

Hello, I'm a developer located in Germany with a current interest in object persistence for a dynamically typed language. I was experimenting with something along the lines of the decomposed storage model when I found out that it was (of course) decades old and found the papers about it and MonetDB. Many thanks for your efforts and for this great piece of software. I was reading the 1995 CWI report 'High Performance support for OO traversals in monet' and was wondering whether the ideas in it are still present in today's MonetDB. And if not, how much effort would it take to implement OO persistence using MAL. I would also appreciate any pointer to papers, code samples etc. in this direction. If I understand correctly, in the beginning, MonetDB was rather heading towards a OODBMS. Was there any reason why you didn't pursue that goal? Is there any road block? Thank you very much for your help. Kambiz Darabi

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/compiler/Tests mal01.stable.out, , 1.1, 1.2 sql01.mal, , 1.1, 1.2 sql01.stable.out, , 1.1, 1.2
by Stefan Manegold 09 Oct '07

09 Oct '07

On Tue, Oct 09, 2007 at 08:09:05PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/compiler/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28543/Tests > > Modified Files: > mal01.stable.out sql01.mal sql01.stable.out > Log Message: > Make the test self-contained, i.e. not dependent on sql libraries. Just as a double-check: Are the warnings below expected/correct, and hence their approval intended? Stefan > Index: mal01.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/compiler/Tests/mal01.stable.out,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- mal01.stable.out 7 Oct 2007 06:44:53 -0000 1.1 > +++ mal01.stable.out 9 Oct 2007 20:09:01 -0000 1.2 [...] > +!WARNING: call to bat.new can not be handled correctly [...] > > Index: sql01.stable.out > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/compiler/Tests/sql01.stable.out,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- sql01.stable.out 7 Oct 2007 06:44:55 -0000 1.1 > +++ sql01.stable.out 9 Oct 2007 20:09:02 -0000 1.2 > +MCdynamicCompiler: sql.qry > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly > +!WARNING: call to sql.bind can not be handled correctly [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1