- developers-list - monetdb.org

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/semantics normalize.brg, 1.11, 1.11.2.1
by Jan Rittinger 06 Dec '06

06 Dec '06

On 12/01/2006 02:28 AM, Peter Boncz wrote with possible deletions: > The Moerkotte Queries (e.g. /ufs/boncz/moer) seduced me into implementing > the "syntactic variant" solution to the problem discussed by Jens in the wiki: > [...] > I do *not* propose to propagate this to the HEAD automatically. The decision whether > this rewrite is worth the while I leave to the Munich folks. It is a thing that helps > mps (a lot, in fact for programmers that use X[Z] XPath notation rather than xquery > notation for $x in X where Z return $x) > Stefan's automatic propagation however already did exactly that... (I just played around with both versions and it seems that the algebra is not slower with these modifications. In Guidos q11 the modified version is even 500 msec faster. My interpretation is that some more rewrites on the algebra side are needed to fill the gap :).) > Anyway, on our standard benchmarking machine db2 (non-optimized 64-bits 32-bits oids > gcc compile) we now get the following score on the Moerkotte Queries: > > MXQ.14 MXQ.12 saxon > q0 0.67 1.71 1.5 > q1 0.26 0.33 1.6 > q2 0.68 1.23 60 > q3 2.38 DNF 209 > q4 1.83 2.02 61 > q5 2.74 DNF 166 > q6 4.82 DNF 130 > q7 2312 DNF 211 > q8 DNF DNF DNF > q9 DNF DNF DNF > q10 DNF DNF DNF > q11 DNF DNF 151 > q12 0.14 DNF ? I tried to get the numbers out of the algebra branch as well. On my Laptop (1700MHz, 1024MB Ram) I got better numbers for the first two queries compiled with milprint_summer (I only tried these two). The algebra version compiles for 7 of the 13 queries (3 fail due to non existent reverse sorting in the algebra and the 3 others fail as f n:number is not implemented yet). Queries q0, q1, q2, q4, q5, q11, and q12 compile and in q0, q1, q5, q11, and q12 all joins are recognized (at least as far as I could see). q2 and q4 still each contain an undetected join because we lack the detection of conjunctive joins (until now). In the following the measurements of a single(!) run for each query: guido q0: alg: Comp 613.000 msec Load 51.719 msec Query 224.477 msec Print 1.319 msec mps: Trans 242.385 msec Shred 59.625 msec Query 385.546 msec Print 1.128 msec guido q1: alg: Comp 79.000 msec Load 52.124 msec Query 120.840 msec Print 16.844 msec mps: Trans 62.809 msec Shred 51.907 msec Query 155.992 msec Print 17.736 msec guido q2 - alg: Comp 473.000 msec DNF (dependent join) guido q4 - alg: Comp 522.000 msec DNF (dependent join) guido q5 - alg: Comp 03s 255 msec Load 50.975 msec Query 1369.837 msec Print 158.239 msec guido q11 - alg: Comp 02s 010 msec Load 50.960 msec Query 1130.238 msec Print 74.269 msec guido q12 - alg: Comp 145.000 msec Load 52.192 msec Query 53.572 msec Print 1.980 msec -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

3 2

[Monetdb-developers] bootstrap: why does it remove .orig files?
by Fabian Groffen 06 Dec '06

06 Dec '06

Hi all, I just found out I lost some (hours of) work, because de-bootstrap threw away some files which I temporarily renamed to "*.orig". Whether or not this was a good idea, I'm wondering what de-bootstrap's rationale is to remove those "original" files, which I don't think that match the description of de-bootstrap: This script ("./de-bootstrap") can be used to clean up the source tree, i.e., to remove all files that were created by "./bootstrap". <rant> In my case it did remove a file that was NOT created by bootstrap, and if bootstrap ever has a patch failing and produces an .orig (and .rej) file, I might prefer my unclobbered file to exist. In any case, I am NOT amused, and think -- considering the many wildcards in the script -- that it does far from what its description says. Of course it asks me whether or not I want to delete stuff, but sorry, I haven't written a script yet that performs the task de-bootstrap should have done to see whether it throws away a file it shouldn't even want to touch. And manually examining the two terminal heights long list of files also isn't my speciality. </rant>

2 2

[Monetdb-developers] Compile M4...
by Romulo Goncalves 05 Dec '06

05 Dec '06

Dear developers, I checkout buildtools and M4 to my machine at home. I am using Fedora Core 4. The version of M4 and the buildtools are the cvs head branch. My gcc compiler is [Romulo@localhost MonetDB]$ gcc --version gcc (GCC) 4.0.2 20051125 (Red Hat 4.0.2-8) Copyright (C) 2005 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. I done boostrap, configure, and make During the compilation I got this error: /bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I/home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk -I../.. -I/home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk -I../common -I/home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk/../common -DLIBGDK -Wall -W -std=c99 -g -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -D_REENTRANT -c -o libbat_la-gdk_system.lo `test -f 'gdk_system.c' || echo '/home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk/'`gdk_system.c gcc -DHAVE_CONFIG_H -I. -I/home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk -I../.. -I/home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk -I../common -I/home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk/../common -DLIBGDK -Wall -W -std=c99 -g -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -D_REENTRANT -c gdk_system.c -fPIC -DPIC -o .libs/libbat_la-gdk_system.o cc1: warnings being treated as errors /home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk/gdk_system.mx:224: warning: missing initializer /home/Romulo/MonetDB/MonetDB4/MonetDB/src/gdk/gdk_system.mx:224: warning: (near initialization for 'MT_system_lock.__data.__count') make[5]: *** [libbat_la-gdk_system.lo] Error 1 make[5]: Leaving directory `/home/Romulo/scratch/MonetDB/monetdb4/build/src/gdk' make[4]: *** [all] Error 2 make[4]: Leaving directory `/home/Romulo/scratch/MonetDB/monetdb4/build/src/gdk' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/Romulo/scratch/MonetDB/monetdb4/build/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/Romulo/scratch/MonetDB/monetdb4/build/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/Romulo/scratch/MonetDB/monetdb4/build' make: *** [all] Error 2 [Romulo@localhost MonetDB]$ I done the same procedures several times and I always got the same error. Any idea what could be the problem? Regards, Romulo

2 8

[Monetdb-developers] configure & *_config.h include files
by Stefan Manegold 01 Dec '06

01 Dec '06

Dear fellow developers, while trying to fix portability issues with the pftijah modules of pathfinder, I noticed the following problem: To ensure portability, each source file should know about and adhere to what configure found out; hence, the respective X_config.h (with X one of monetdb, sql, pf, etc,) must be the first include in each .c file. However, with the front-ends, some files (especially those implementing extension modules) also need to MonetDB's monet.h or gdk.h to get the respective definition. Since monet.h includes gdk.h which includes monet_utils.h which in turn includes monetdb_config.h, these files the include both the "local" X_config.h (with X one of sql, pf, etc,) and the monetdb_config.h. This might trigger two problems: 1) If we compile sql, pf, etc, a against a binary install of MonetDB which was configured and generated on an binary compatible but potentially (slightly) differently configured/installed system, this might lead to conflicting definition/declarations in the two X_config.h. 2) Even if we compile MonetDB and the front-end on the same system, there might be conflicts since the respective configure are not completely identical. In pathfinder/runtime we avoid (2) be not including pf_config.h but only monetdb_config.h (via monet.h or gdk.h) which might actually lead to problems similar to (1). In pathfinder/module/pftijah, we run into (2) with _FILE_OFFSET_BITS (at least) on our Solaris 10 x86 machine, since only MonetDB' configure.ag checks AC_SYS_LARGEFILE() while pathfinder's configure.ag does not. To solve the problem (2) "properly", we could either make all configures identical (e.g., by moving all "local" checks from the various configure.ag to MonetDB/conf.monet.m4). But that still leaves us with (1). To solve (1) (and (2)) "properly", we could avoid the implicit inclusion of monetdb_config.h in monet_utils.h (and hence gdk.h & monet.h) by removing the respective include from monet_utils.h. But then, we'd have to explicitly include monetdb_config.h in each .c file that now includes one of monet_utils.h, gdk.h, monet.h (directly or via some other include) --- a lot of work, with a high risk to overlook some locations... A "better"(?) (at least simpler) solution could be to guard the #include "monetdb_config.h" in MonetDB' monet_utils.h by, say, #ifndef NO_MONETDB_CONFIG ... #endif, and then define NO_MONETDB_CONFIG in X_config.h (with X one of sql, pf, etc,). I'd strongly vote for the last solution. ;-) Comments, ideas, complaints, objections? Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

4 7

[Monetdb-developers] Compilation of M4, M5 and SQL in the windows 64
by Romulo Goncalves 01 Dec '06

01 Dec '06

Dear developers, During the compilation of M4, M5, and SQL in windows 64 I have to correct all the time the rules.msc: PTHREAD_INCS = "-I$(PTHREAD)\include" PTHREAD_LIBS = "$(PTHREAD)\lib\pthreadVC2.lib" to PTHREAD_INCS = "-I$(PTHREAD)" PTHREAD_LIBS = "$(PTHREAD)\pthreadVC2.lib" TO not get compilation errors. This happen because the compilations of pthread for win64 does not create the directories lib and include. The files are all saved into the main directory pthreads. Being the pthreads directory tree for 64 different of the directory tree for 32. If you could help me to find a solution I will appreciate a lot. Regards, Romulo

2 4

[Monetdb-developers] conf.bash
by Fabian Groffen 01 Dec '06

01 Dec '06

Dear developers, While trying to update conf.bash, I encountered the following problems: 1) I cannot find times100, misq and acoi 2) In playpen and gis conf.bash isn't identical to the other locations Further I wonder why conf.bash isn't just in buildtools for those who need it, but replicated in all modules. It is a great pain to keep them in sync it seems, and causes a lot of email. If someone can solve 1), thanks. I don't know about 2), I just made the change that is necessary for my change.

2 2

[Monetdb-developers] Mtest for M5-SQL
by Romulo Goncalves 01 Dec '06

01 Dec '06

I updated all my system yesterday night. Now I am trying to run the Mtest for M5-SQL and I got this error. [goncalve@amelia src]$ Mtest_5_sql -rq Could not open file /ufs/goncalve/scratch/MonetDB/monetdb5/etc/MonetDB5.conf Mtest.py Warning: TSTSRCBASE: Replacing logical path /ufs/goncalve/MonetDB/SQL/sql/ by absolute physical path /ufs/goncalve/MonetDB/SQL/sql Mtest.py Warning: TSTBLDBASE: Replacing logical path /ufs/goncalve/scratch/SQL/sql/build/ by absolute physical path /net/amelia/export/scratch1/goncalve/SQL/sql/build Mtest.py Warning: TSTTRGBASE: Replacing logical path /ufs/goncalve/scratch/Mtest5/ by absolute physical path /net/amelia/export/scratch1/goncalve/Mtest5 Traceback (most recent call last): File "/ufs/goncalve/scratch/MonetDB/monetdb4/bin/Mtest.py", line 3364, in ? main(sys.argv) File "/ufs/goncalve/scratch/MonetDB/monetdb4/bin/Mtest.py", line 2947, in main os.environ[v] = env[v] KeyError: 'XQUERY_LOGDIR' What can be the problem? Regards, Romulo

2 3

[Monetdb-developers] Triggers owner..
by Romulo Goncalves 29 Nov '06

29 Nov '06

Dear developers, I am using SQL cvs head and M5. If I create a table and a new user with a mclient: create user "user_test" with password 'pass' NAME 'test1_user' SCHEMA "sys"; create table t1 (id int); If then I connect with the new user and I create a trigger on t1: create trigger test1 after insert on t1 insert into t1 values(12); I do not get any error. Well, depending on the approach followed in MonetDB it can be a bug or not. (1) If we decide that a trigger over a table can only be created by the table owner (in this case the monetdb user is the owner of t1) this is a bug. (2) If any user is allowed to create a trigger over any table, this is not a bug. What approach is followed by MonetDB? If any which one we should choose? In my opinion we should choose the first approach. Regards, Romulo

2 2

Re: [Monetdb-developers] error when running simple Xquery on Windows platform
by Fabian Groffen 28 Nov '06

28 Nov '06

Hi Daniel, I'm forwarding your mail to monetdb-developers(a)lists.sourceforge.net in the hope that someone can help you there. I have no idea myself what you can try to resolve this problem. Sorry! On 26-11-2006 23:21:07 -0500, dcr2108(a)columbia.edu wrote: > > Dear Monet Team, > > I am running into a consistent error when I try to run an example > xquery from your website "books1.xq": > > ERROR: interpret_params: mirror(param 1): invalid BAT. > ERROR: interpret_params: [startsWith](param 2): evaluation error. > ERROR: interpret_params: uselect(param 1): evaluation error. > ERROR: interpret_params: +(param 2): evaluation error. > > Steps to reproduce error: > 1) 'Start' -> 'Programs' -> 'MonetDB XQuery' -> 'MonetDB XQuery > Server' > > 2) double click books1.xq > > 3) THE ERROR OCCURS > > Thanks for your help, > Daniel Rappaport > Masters candidate in department of biomedical informatics > > >

3 2

[Monetdb-developers] Mapi and locality
by Sándor Héman 26 Nov '06

26 Nov '06

Hi, I have a problem with Mapi which did not seem to occur in previous releases of MonetDB (<4.15). In the past, I could start an Mserver with a mil script that declared some PROCs, and then let this Mserver accept Mapi connections, with the Mapi clients being able to call the functions that were declared in the server-side startup script. However, since the latest release, the server-side PROCs seem to be invisible to the client: MAPI = monetdb@kanda:50012 QUERY = template_conj_2(1, bat(void, lng).seqbase(0(a)0).append(lng (5394205300074141533)).append(lng(3189196963433190955))); ERROR = !ERROR: interpret: no matching MIL operator to 'template_conj_2(int, BAT[oid,lng])'. If I execute exactly the same call in the server-side console, it just works... Any ideas on this? And whether there is an easy fix? Cheers Sandor

2 4