October 2007 - developers-list

[Monetdb-developers] Compiling on Darwin 7
by Fabian Groffen 21 Jan '08

21 Jan '08

Is there anyone that can think of a solution for this problem? cc1: warnings being treated as errors /Volumes/Scratch/monetdb/stable/monetdb/src/modules/plain/bat.mx: In function 'local_itoa': /Volumes/Scratch/monetdb/stable/monetdb/src/modules/plain/bat.mx:1253: warning: format '%zd' expects type 'signed size_t', but argument 4 has type 'ssize_t' I mean... is there a difference between "signed size_t" and "ssize_t"? Ehrm. [Orion:src/modules/plain] fabian% uname -a Darwin Orion.local 7.9.0 Darwin Kernel Version 7.9.0: Wed Mar 30 20:11:17 PST 2005; root:xnu/xnu-517.12.7.obj~1/RELEASE_PPC Power Macintosh powerpc PowerBook4,3 Darwin [Orion:src/modules/plain] fabian% gcc --version gcc (GCC) 4.0.1 (Apple Computer, Inc. build 5341) Copyright (C) 2005 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Probably just need to compile without -Werror.

4 5

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/src/tools/java build.properties, XQuery_0-20, 1.1.2.2, 1.1.2.3 build.xml, XQuery_0-20, 1.1.2.3, 1.1.2.4
by Fabian Groffen 31 Oct '07

31 Oct '07

On 30-10-2007 22:33:56 +0000, Ying Zhang wrote: > Added a new variable in build.properties (build_compiler) to make > compiler switching easier. http://ant.apache.org/manual/CoreTasks/javac.html <quote> It is possible to use different compilers. This can be specified by either setting the global build.compiler property, which will affect all <javac> tasks throughout the build, or by setting the compiler attribute, specific to the current <javac> task. Valid values for either the build.compiler property or the compiler attribute are: </quote> In other words, don't do what you just did. Just rely on standard behaviour, and set build.compiler in your build.properties to gcj if you want to build with gcj.

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra intro_rec_border.c, XQuery_0-20, 1.5, 1.5.4.1
by Stefan Manegold 30 Oct '07

30 Oct '07

On Mon, Oct 29, 2007 at 10:00:19AM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv15130/algebra > > Modified Files: > Tag: XQuery_0-20 > intro_rec_border.c > Log Message: > -- Fix a bug for recursive queries detected by Maurice. which one? and should it (the bug, resp. its fix) be tested? Stefan > Index: intro_rec_border.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/intro_rec_border.c,v > retrieving revision 1.5 > retrieving revision 1.5.4.1 > diff -u -d -r1.5 -r1.5.4.1 > --- intro_rec_border.c 27 Jun 2007 13:46:26 -0000 1.5 > +++ intro_rec_border.c 29 Oct 2007 10:00:16 -0000 1.5.4.1 > @@ -95,6 +95,13 @@ > } > break; > > + case pa_fcns: > + /* this also skips the introduction of a rec_border > + operator for the content of an empty elements: > + elem (fcns (nil, nil)). */ > + if (R(n)->kind == pa_nil) > + break; > + /* else fall through */ > default: > /* follow the children until a base or a leaf is reached */ > for (unsigned int i = 0; i < PFPA_OP_MAXCHILD && n->child[i]; i++) > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > > > > > > > > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_builder.mx, MonetDB_5-2, 1.26, 1.26.2.1
by Stefan Manegold 29 Oct '07

29 Oct '07

On Mon, Oct 29, 2007 at 07:09:42AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv18604/src/mal > > Modified Files: > Tag: MonetDB_5-2 > mal_builder.mx > Log Message: > pushStr shouldn't do a GDKstrdup, as it may return an equal string from > the constant cache, ie only if the returned variable holds the given string > (poniters are equal) we do a string dup. This solves another memory leak. > > > Index: mal_builder.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_builder.mx,v > retrieving revision 1.26 > retrieving revision 1.26.2.1 > diff -u -d -r1.26 -r1.26.2.1 > --- mal_builder.mx 28 Aug 2007 19:56:41 -0000 1.26 > +++ mal_builder.mx 29 Oct 2007 07:09:40 -0000 1.26.2.1 > @@ -260,10 +260,12 @@ > ValRecord cst; > > cst.vtype= TYPE_str; > - cst.val.sval= GDKstrdup(val); > + cst.val.sval= val; gcc -DHAVE_CONFIG_H -I. -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal -I../.. -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal -I../optimizer -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/../optimizer -I../scheduler -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/../scheduler -I../modules/kernel -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/../modules/kernel -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/include/MonetDB -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/include/MonetDB/common -I/ufs/manegold/_/scratch0/Monet/Testing/Stable/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/include/MonetDB/gdk -DLIBMAL -Wall -Wextra -std=c99 -g -Werror-implicit-function-declaration -Werror -Wpointer-arith -Wdeclaration-after-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -D_REENTRANT -c mal_builder.c -fPIC -DPIC -o .libs/libmal_la-mal_builder.o cc1: warnings being treated as errors /ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/mal_builder.mx: In function 'pushStr': /ufs/manegold/_/scratch0/Monet/Testing/Stable/source/MonetDB5/src/mal/mal_builder.mx:263: warning: assignment discards qualifiers from pointer target type make[5]: *** [libmal_la-mal_builder.lo] Error 1 Stefan > cst.len= strlen(cst.val.sval); > _t = defConstant(mb,TYPE_str,&cst); > > + if (getConstant(mb,_t).val.sval == val) > + getConstant(mb,_t).val.sval = GDKstrdup(val); > return pushArgument(mb, q, _t); > } > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 4

[Monetdb-developers] MonetDB(5) and modules
by Fabian Groffen 26 Oct '07

26 Oct '07

I noticed that MonetDB5 uses modules, but compiles them as shared libraries. Maybe other targets do the same here. autogen appears to have (had) some partial support for modules, and the attached patch fixes/re-enables that support. In short it passes -module to libtool during link phase, such that the object is generated as loadable module. Why did I look at this? Mainly because on Darwin there is a difference between those objects that are loaded by dlopen(3) (loadable modules) and those that are used by the link editor ld(1) (shared libraries). The latter have type DYLIB whereas the former have type MH_BUNDLE. A so-called "bundle" doesn't have an install_name, as it can't be used by the link editor. Now initially all went well during compilation until I hit this error: /Library/Gentoo/usr/bin/ld: ../atoms/.libs/lib_color.so is input for the dynamic link editor, is not relocatable by the static link editor again This message is correct on Darwin, as batcolor.mx links against lib_color.so. libtool already detected this case: *** Warning: Linking the shared library lib_batcolor.la against the loadable module *** lib_color.so is not portable! ** Warning, lib lib_color.so is a module, not a shared library ** And there doesn't seem to be a static archive available ** The link will probably fail, sorry I don't recall ever seeing problems with these loads (apart from on Panther which didn't grok it at all) on my (as of today out-of-date) OSX 10.4, so I'm not sure how relevant it is to actually fix this. It looks like support to load shared libraries via dlopen actually was added, so we need not to worry about it at all. Actually, the man-page says it all: "dlopen() can load dynamic libraries and bundles." "In Mac OS X 10.4, dlopen was rewritten to be a native part of dyld." So far for sharing my experiences in this area. As far as I know, 10.3 support for MonetDB was shot already long ago (as it should), so maybe we should remove the remaining bits that could/can trigger -module libtool invocation, as it *will* cause trouble.

1 0

[Monetdb-developers] problems with XQuery aggregation
by Henning Rode 25 Oct '07

25 Oct '07

hej monetdb developers, i tried to write a simple aggregation query: let $cands := doc("x.xml")//c for $eid in distinct-values($cands/@id) return <candidate id="{$eid}" count="{count($cands[@id eq $eid])}"/> however, executing the above query turns out to be rather difficult. i always killed the process after waiting for more than half an hour without getting any result. x.xml is 17MB large and contains 300000 candidates nodes c. i run it on Monet Database Server V4.19.0 where the query crashed ERROR = !ERROR: GDKload: cannot mmap(): name=10/1030, ext=buns.priv !OS: Cannot allocate memory !ERROR: GDKload failed: name=10/1030, ext=buns.priv !ERROR: CMDleftjoin: operation failed. and on the newer stable release MonetDB Server v4.20.0 on a machine with 16GB memory where the query just runs endless. apparently the query compiler does not recognise the necessary join and aggregation here. is that a known problem? btw. sorting the candidates by @id value is done in seconds: let $cands := doc("x.xml")//c for $c in $cands order by $c/@id return $c best -henning

3 4

[Monetdb-developers] Segmentation Fault upon sql.start()
by Stefan de Konink 22 Oct '07

22 Oct '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 After loading in a bunch of data the monetdb server was Killed due to memory starvation. With swap MonetDB performed the huge amount of one insert in 10 seconds, so that wasn't an option too. I restarted it, and tried to 'load the data again'. Backtrace: gdb ./mserver5 GNU gdb 6.7 Copyright (C) 2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu"... Using host libthread_db library "/lib/libthread_db.so.1". (gdb) run Starting program: /opt/monetdb/bin/mserver5 warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff275fe000 [Thread debugging using libthread_db enabled] [New Thread 0x2afd85f0f2a0 (LWP 23929)] # MonetDB Server v5.0.0 # Copyright (c) 1993-2007 CWI, all rights reserved # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs dynamically linked # dbname:osm # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see /opt/monetdb/etc/monetdb5.conf) [New Thread 0x40800950 (LWP 23932)] >include sql; >sql.start(); Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2afd85f0f2a0 (LWP 23929)] 0x00002aaaaab40a8f in load_column () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so (gdb) bt #0 0x00002aaaaab40a8f in load_column () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #1 0x00002aaaaab41a71 in load_table () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #2 0x00002aaaaab433cc in load_trans () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #3 0x00002aaaaab46e15 in store_init () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #4 0x00002aaaaab1b95b in mvc_init () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #5 0x00002aaaaaad0b50 in SQLinit () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #6 0x00002afd8389f476 in initScenario () from /opt/monetdb/lib/libmal.so.0 #7 0x00002aaaaaad0ac8 in SQLsession () from /opt/monetdb/lib/MonetDB5/lib/lib_sql.so #8 0x00002afd8387842d in runMALsequence () from /opt/monetdb/lib/libmal.so.0 #9 0x00002afd83879dfd in runMAL () from /opt/monetdb/lib/libmal.so.0 #10 0x00002afd83871dc8 in MALengine () from /opt/monetdb/lib/libmal.so.0 #11 0x00002afd8389f082 in runScenarioBody () from /opt/monetdb/lib/libmal.so.0 #12 0x00002afd8389f0d2 in runScenario () from /opt/monetdb/lib/libmal.so.0 #13 0x00002afd838719e0 in MSserveClient () from /opt/monetdb/lib/libmal.so.0 #14 0x0000000000402c14 in main () (gdb) It work doesn't anymore... I can conclude, inserting a lot of data start to become a huge waste of time. If required I can recompile with the -g option and a lower amount of optimization. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHHQTsYH1+F2Rqwn0RCjWKAJ9u/OEu8Sk6HTSY+v431bLjmylrCwCffsNN Rpf0Fs4XjrakYkdyEwkknWQ= =0UHS -----END PGP SIGNATURE-----

2 2

[Monetdb-developers] python development
by Sjoerd Mullender 22 Oct '07

22 Oct '07

For all you Python developers, a new rule is being enforced from now on: Python source files are not allowed to contain TAB characters or trailing white space. This is enforced at cvs commit time. -- Sjoerd Mullender

1 0

[Monetdb-developers] man-pages
by Fabian Groffen 22 Oct '07

22 Oct '07

We should put or html-ified (autogenerated) man-pages for merovingian, monetdb and mclient on our website, as easy to read documentation.

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/testing Mtest.py.in, MonetDB_1-20, 1.309.2.8, 1.309.2.9
by Fabian Groffen 22 Oct '07

22 Oct '07

On 21-10-2007 16:36:26 +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/testing > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv16749/src/testing > > Modified Files: > Tag: MonetDB_1-20 > Mtest.py.in > Log Message: > export TST_FIVE, we sometimes need to know which server we test > > > Index: Mtest.py.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/testing/Mtest.py.in,v > retrieving revision 1.309.2.8 > retrieving revision 1.309.2.9 > diff -u -d -r1.309.2.8 -r1.309.2.9 > --- Mtest.py.in 19 Oct 2007 13:41:24 -0000 1.309.2.8 > +++ Mtest.py.in 21 Oct 2007 16:36:21 -0000 1.309.2.9 > @@ -1237,6 +1237,7 @@ > os.environ['TST_MODS'] = str(env['TST_MODS']) > if par['M5']: > env['TST_FIVE'] = "Five" > + os.environ['TST_FIVE'] = "Five" This looks like an upcoming IndentationError to me. > return rtrn > ### GetBitsAndOIDsAndModsAndStatic(env) #

2 2