- developers-list - monetdb.org

[Monetdb-developers] Experiences with the new setup
by Jan Rittinger 02 Jan '07

02 Jan '07

Hi all, Here are my experiences of getting pathfinder working with the new setup First I install everything without '--prefix': cd buildtools ./bootstrap cd Linux configure_buildtools make make install cd ../../MonetDB ./bootstrap cd Linux configure_MonetDB make make install cd ../../clients ./bootstrap cd Linux configure_clients make make install cd ../../MonetDB4 ./bootstrap cd Linux configure_MonetDB4 make make install cd ../../pathfinder ./bootstrap cd Linux configure_pathfinder make make install Afterwards I tried to start Mserver: 10:18:08 rittinge@notekemper08:/local_tmp/try_new/pathfinder> Mserver # Monet Database Server V4.15.0 # Copyright (c) 1993-2006, CWI. All rights reserved. # Compiled for i686-suse-linux/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: moduleClient: module(builtin) load error. !ERROR: moduleClient: module(arith) load error. !ERROR: moduleClient: module(str) load error. !ERROR: moduleClient: module(constant) load error. !ERROR: moduleClient: module(bat) load error. !ERROR: moduleClient: module(algebra) load error. !ERROR: moduleClient: module(sys) load error. !ERROR: moduleClient: module(trans) load error. !FATAL: scheduleClient: Daemon mode without startup script Taking the library path into account didn't help either: 10:20:05 rittinge@notekemper08:/local_tmp/try_new/pathfinder> echo $LD_LIBRARY_PATH /usr/local/lib 10:20:33 rittinge@notekemper08:/local_tmp/try_new/pathfinder> export LD_LIBRARY_PATH=/local_tmp/try_new/MonetDB4/Linux/lib:/local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4:/local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4/lib:/usr/local/lib 10:20:54 rittinge@notekemper08:/local_tmp/try_new/pathfinder> Mserver # Monet Database Server V4.15.0 # Copyright (c) 1993-2006, CWI. All rights reserved. # Compiled for i686-suse-linux/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: moduleClient: module(builtin) load error. !ERROR: moduleClient: module(arith) load error. !ERROR: moduleClient: module(str) load error. !ERROR: moduleClient: module(constant) load error. !ERROR: moduleClient: module(bat) load error. !ERROR: moduleClient: module(algebra) load error. !ERROR: moduleClient: module(sys) load error. !ERROR: moduleClient: module(trans) load error. !FATAL: scheduleClient: Daemon mode without startup script Only using the mod_path option helped... >10:23:21 rittinge@notekemper08:/local_tmp/try_new/pathfinder> Mserver --set monet_mod_path=/local_tmp/try_new/MonetDB4/Linux/lib/:/local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4/:local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4/lib/ # Monet Database Server V4.15.0 # Copyright (c) 1993-2006, CWI. All rights reserved. # Compiled for i686-suse-linux/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !FATAL: scheduleClient: Daemon mode without startup script ... and by changing the variable monet_daemon I got my Mserver runnin without error message: 10:23:23 rittinge@notekemper08:/local_tmp/try_new/pathfinder> Mserver --set monet_mod_path=/local_tmp/try_new/MonetDB4/Linux/lib/:/local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4/:/local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4/lib/ --set monet_daemon=no # Monet Database Server V4.15.0 # Copyright (c) 1993-2006, CWI. All rights reserved. # Compiled for i686-suse-linux/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. > For pathfinder I had a similar problem I couldn't fix (-- even adding all library paths I could find to the mod_path didn't help). 10:27:58 rittinge@notekemper08:/local_tmp/try_new/pathfinder> echo 1 | pf -A | Mserver --set monet_mod_path=/local_tmp/try_new/MonetDB4/Linux/lib/:/local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4/:/local_tmp/try_new/MonetDB4/Linux/lib/MonetDB4/lib/:/local_tmp/try_new/pathfinder/Linux/lib/:/local_tmp/try_new/pathfinder/Linux/lib/MonetDB4/:/local_tmp/try_new/pathfinder/Linux/lib/MonetDB4/lib/:/local_tmp/try_new/MonetDB/Linux/lib/:/local_tmp/try_new/clients/Linux/lib/:/local_tmp/try_new/clients/Linux/lib/MonetDB/ --set monet_daemon=no # Monet Database Server V4.15.0 # Copyright (c) 1993-2006, CWI. All rights reserved. # Compiled for i686-suse-linux/32bit with 32bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. >!ERROR: mapi_open is not set !ERROR: CMDfind: value not found. !ERROR: CMDfind: operation failed. !ERROR: interpret_params: lng(param 1): evaluation error. !ERROR: interpret_params: *(param 1): evaluation error. !ERROR: interpret_params: xquery_start_query_cache(param 1): evaluation error. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><?xml version="1.0" encoding="utf-8"?> <XQueryResult>1</XQueryResult> >>>>!ERROR: CMDfind: value not found. !ERROR: CMDfind: operation failed. !ERROR: interpret_params: lng(param 1): evaluation error. !ERROR: interpret_params: *(param 2): evaluation error. !ERROR: ws_free: query cleanup failed, Mserver must be restarted. With the old setup Mserver did know at least its own libraries. Would it be possible to include that knowledge in the new setup as well? Also using the daemon mode as default leaves me a little bit puzzled. Was this change a strategic decision? And before I forget it -- A happy new year to all of you :) Cheers, Jan -- Jan Rittinger Database Systems Technische Universität München (Germany) http://www-db.in.tum.de/~rittinge/

5 6

Re: [Monetdb-developers] IMPORTANT: imminent change to repositories
by Stefan Manegold 02 Jan '07

02 Jan '07

Arjen, please use the mailing lists and/or bug tracker for such issue --- not only because I'm on "holidays" (and hence other developers *might* asnwer faster), but also because other users might befinet from the info. Thanks! On Tue, Jan 02, 2007 at 12:26:25PM +0100, Arjen P. de Vries wrote: > I still have the problem with clients: > > >Building clients, I get the error > >src/odbc/Makefile.am:13: NATIVE_WIN32 does not appear in AM_CONDITIONAL > >in bootstrap. > > > >I fixed it by editing this Makefile.am, and redoing the last Mbootstrap > >commands > >(automake and autoconf). > > > > I then with compiling get an error > In file included from > /ufs/arjen/src/MonetDB/clients/src/mapilib/Mapi.mx:655: > /ufs/arjen/local/include/MonetDB/common/monet_utils.h:70: error: > conflicting types for 'ssi > ze_t' > /usr/include/sys/types.h:110: error: previous declaration of 'ssize_t' was > here 1) using which compiler? 2) (again) clean build with all source up-to-date (now tags/branches!) and non-modified, and no legacy lying around (in old directories and/or PATHs)? 3) everything seems to work fine with nightly testing (cf. http://monetdb.cwi.nl/Development/TestWeb/Current/index.html) > these monet_utils.h are from the most recent MonetDB install, and > clients is also up-to-date... > > is this really an error, or still installation problems? from 3), I suspect the latter... Stefan > Arjen -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 4

[Monetdb-developers] IMPORTANT: imminent change to repositories
by Sjoerd Mullender 02 Jan '07

02 Jan '07

This is a heads up. Read this. It affects all developers (however, developers only fixing bugs on the stable branch can postpone reading this). MonetDB Repository Split Into Three This whole story *only* affects the CURRENT branch, the STABLE branch is totally unaffected by the changes described here. This change is *imminent*. It will probably happen today. There will be CVS tags before and after. The MonetDB repository will be split into three repositories. One repository will contain the fundamentals of MonetDB (GDK) and other common stuff that is used by all other subprojects (src/common). This repository remains called MonetDB. Another repository will be clients. This repository contains all client libraries and programs. It is independent from the GDK part of MonetDB, but it does depend on the common libraries. The final repository will be MonetDB4 which contains the MIL-specific part of the old MonetDB repository. The new layout of the MonetDB family is the following repositories: - buildtools contains the stuff that is needed to compile any of the following from CVS; - MonetDB containing GDK and common, i.e. stuff that forms the core of the MonetDB family and stuff that is generally useful (src/common); - clients contains libraries and programs that connect to the various servers, i.e. the Mapi library (in several languages), ODBC, JDBC. - MonetDB4 contains the MonetDB4-specific parts, i.e. stuff like the MIL parser and execution engine (src/monet), and the MonetDB4 modules; - MonetDB5 (still at CWI repository as monet5, to be moved to SourceForge "soon") contains the MonetDB5-specific parts, i.e. stuff like the MAL parser and execution engine and the MonetDB5 modules; - sql contains the SQL frontend; - pathfinder contains the XQuery frontend. And then there are of course the other repositories amdb, xml, template, etc. Dependencies This is written in the suggested order of compilation and installation. When compiling from CVS, you always need buildtools, and this needs to be compiled and installed first. MonetDB does not depend on anything else. clients depends on MonetDB (specifically on src/common). MonetDB4 depends on MonetDB and clients. It depends on clients only for the embedded library and programs using that library. MonetDB5 depends on MonetDB and clients (and *not* on MonetDB4). It depends on clients only for the embedded library and programs using that library. sql depends on MonetDB, clients, and MonetDB4; but if you want MonetDB5 support, also on MonetDB5 (the dependency should become: at least one of MonetDB4 and MonetDB5, i.e. either one or both backends). pathfinder depends on MonetDB, clients, and MonetDB4; but if you want MonetDB5 support, also on MonetDB5 (the dependency should become: at least one of MonetDB4 and MonetDB5, but the port to MonetDB5 needs to be finished first...). Other Changes The scripts bootstrap, de-bootstrap, and conf/conf.bash have been moved to the buildtools repository. Replacement scripts have been put in place that use the scripts that get installed when buildtools is installed. A C source file should now include the project's own config.h file (i.e. pf_config.h for pathfinder, mal_config.h for MonetDB5, etc.) as the first include in the C file. The config.h file should not be included in H files that get installed and used by other projects. In particular, gdk.h and monet_utils.h now no longer include monetdb_config.h. A project's configure script now has to perform all the configure tests that are needed for .h files that are included from other projects. Most of the required tests for the stuff in MonetDB are now done in the call to AM_MONETDB_COMMON (if any are missing, that is a (lowish priority) bug and should be fixed). Future Changes The changes described here have not happened. They are thoughts for future work. More configure and header file cleanup. In an ideal world, we have header files that are used externally which do not contain and #ifdef HAVE_* sections, and header files that are used internally that can contain such sections. Also, it should be possible to figure out at configure time from the already installed parts either the correct options (such as how wide the OID type is), or that the given options are not compatible with the installed parts (and give a configure error). Currently it is the developer's responsibility to make sure that configure options are compatible. -- Sjoerd Mullender

3 5

[Monetdb-developers] M5 compilation....
by Romulo Goncalves 02 Jan '07

02 Jan '07

Dear developers I already compiled the background (buildtools, MonetDB, and clients). I also compiled M4 (only after removing the dependencies in Mapi.h). During M5 compilation I got the following error: -Wl,/ufs/goncalve/scratch/MonetDB/monetdb5/build/src/mal/.libs -Wl,--rpath -Wl,/ufs/goncalve/scratch/MonetDB/MonetDB/lib -Wl,--rpath -Wl,/ufs/goncalve/scratch/MonetDB/monetdb5/lib -Wl,--rpath -Wl,/ufs/goncalve/scratch/MonetDB/MonetDB/lib ../../mal/.libs/libmal.so -L/ufs/goncalve/scratch/MonetDB/MonetDB/lib /ufs/goncalve/scratch/MonetDB/MonetDB/lib/libbat.so /ufs/goncalve/scratch/MonetDB/MonetDB/lib/libstream.so -lMapi -lz -lbz2 -lpthread -ldl -lcrypt -m64 -Wl,-soname -Wl,lib_algebraExtensions.so.0 -o .libs/lib_algebraExtensions.so.0.0.0 /usr/bin/ld: cannot find -lMapi collect2: ld returned 1 exit status make[7]: *** [lib_algebraExtensions.la] Error 1 make[7]: Leaving directory `/net/amelia/export/scratch1/goncalve/MonetDB/monetdb5/build/src/modules/mal' make[6]: *** [all] Error 2 I can not remove all the dependencies in -lMapi. What you suggest to resolve this problem? Regards, Romulo

2 2

[Monetdb-developers] Compilation error on embeddedclient.mx
by Jan Flokstra 02 Jan '07

02 Jan '07

Hi All, I tried to install the new split repository on a SuSe10 system this morning. I threw away all old stuff, even the prefix. The installation of buildtools and MonetDB went well. But I got a compilation error on the make install for the mapi: Making install in tools make[4]: Entering directory `/usr/local/flokstra/software/MonetDB4/src/tools' make install-am make[5]: Entering directory `/usr/local/flokstra/software/MonetDB4/src/tools' /bin/sh ../../libtool --tag=CC --mode=compile gcc -m64 -DHAVE_CONFIG_H -I. -I. -I../.. -I. -I/usr/local/flokstra/include/MonetDB -I/usr/local/flokstra/include/MonetDB/common -I/usr/local/flokstra/include/MonetDB/gdk -I../monet -I./../monet -I../modules/plain -I./../modules/plain -DLIBEMBEDDEDMIL -g -O2 -Wall -W -std=c99 -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -D_REENTRANT -c -o libembeddedmil_la-embeddedclient.lo `test -f 'embeddedclient.c' || echo './'`embeddedclient.c gcc -m64 -DHAVE_CONFIG_H -I. -I. -I../.. -I. -I/usr/local/flokstra/include/MonetDB -I/usr/local/flokstra/include/MonetDB/common -I/usr/local/flokstra/include/MonetDB/gdk -I../monet -I./../monet -I../modules/plain -I./../modules/plain -DLIBEMBEDDEDMIL -g -O2 -Wall -W -std=c99 -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -D_REENTRANT -c embeddedclient.c -fPIC -DPIC -o .libs/libembeddedmil_la-embeddedclient.o In file included from embeddedclient.mx:69: embeddedclient.h:29:18: error: Mapi.h: No such file or directory In file included from embeddedclient.mx:69: embeddedclient.h:31: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'embedded_mil' embeddedclient.mx:183: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'embedded_mil' Any ideas, Jan Flokstra.

3 3

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/NT configure.py, 1.51, 1.52 rules.msc, 1.37, 1.38
by Martin Kersten 31 Dec '06

31 Dec '06

Stefan Manegold wrote: > Update of /cvsroot/monetdb/sql/NT > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv13165/NT > > Modified Files: > configure.py rules.msc > Log Message: > > cleaning-up & preparing the "monet5" to "MonetDB5" move: > > - renamed all remaining MONET5_* (enironnment-) variables to MONETDB5_* > - renamed all remaining *_monet5 aliases to *_MonetDB5 > > In case you use any MONET5_* (enironnment-) variables or > *_monet5 aliases in your own scripts, > you need to rename them to MONETDB5_* respectively *_MonetDB5, > too. Oops, it seems you have been out of the loop or overwhelmed with emails regarding the BIG BANG cleanup. I already mentioned to sjoerd/niels/fabian/... that the naming of packages should be given some thought and discussed. - move the package names to lower-case instead of a mixture of upper/lower - suggested packages names (if feasible on SF), buildtools,gdkcommon, clients, monet4, monet5, sql, and pathfinder. This propably won't affect your Environment variables.

3 3

[Monetdb-developers] Compilation of M4
by Romulo Goncalves 30 Dec '06

30 Dec '06

Dear developers, I am using the following configure for M4: env PATH=/ufs/goncalve/scratch/MonetDB/buildtools/bin:/ufs/goncalve/scratch/MonetDB/clients:/ufs/goncalve/scratch/MonetDB/MonetDB/bin:${PATH} PYTHONPATH=/ufs/goncalve/MonetDB/buildtools/autogen/ ~/MonetDB/MonetDB4/MonetDB4/configure --prefix=/ufs/goncalve/scratch/MonetDB/monetdb4 --enable-debug --enable-bits=64 However, the compilation gave this error: -c embeddedclient.c -fPIC -DPIC -o .libs/libembeddedmil_la-embeddedclient.o In file included from /ufs/goncalve/MonetDB/MonetDB4/MonetDB4/src/tools/embeddedclient.mx:69: embeddedclient.h:29:18: error: Mapi.h: No such file or directory I already add to the PATH the prefix for the clients installation. What more I need to add to the path? Why the configure of M4 does not complain about such missing? Regards, Romulo

2 2

[Monetdb-developers] buildtools configure.ac
by Fabian Groffen 28 Dec '06

28 Dec '06

I'm trying to bootstrap buildtools but foud out that buildtools is shipped with an configure.ac which isn't an ac file, but just plain configure itself (as the local bootstrap script just copies it to configure). This is hard to detect for my generic scripts who like to generate a configure from the .ac file. Is there a reason why this ac file exists? If not, I'd be happier if it could just be renamed to configure instead, as I don't think this is the right solution.

2 5

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/java Makefile.ag, 1.3, 1.4 build.xml, 1.4, 1.5
by Fabian Groffen 27 Dec '06

27 Dec '06

I advice you to fix it properly, because your hack doesn't result in the desired effect. 1) create_mjclient will build the jdbcclient.jar when it's not there, and it will end up in the builddir (where you don't want it) 2) if it wouldn't do that, it would think that the jdbcclient.jar file is located in the builddir, because you just told it so, hence rhe resulting script won't work. On 27-12-2006 15:29:28 +0000, Niels Nes wrote: > Index: build.xml > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/java/build.xml,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- build.xml 27 Dec 2006 14:54:22 -0000 1.4 > +++ build.xml 27 Dec 2006 15:29:26 -0000 1.5 > @@ -120,12 +120,10 @@ > <echo message="Crafting an mjclient wrapper script for ${jdbcclient-jar}" /> > <filter token="JDBCCLIENT_JAR" value="${jdbcclient-jar}" /> > > -  > - <mkdir dir="${bindir}" /> >  > <copy file="${scriptdir}/mjclient.in" > overwrite="true" > - tofile="${bindir}/mjclient" > + tofile="${builddir}/mjclient" > filtering="yes" /> > </target>

1 0

Re: [Monetdb-developers] [Monetdb-checkins] buildtools Makefile.in, 1.6, 1.7
by Sjoerd Mullender 27 Dec '06

27 Dec '06

On 12/27/2006 03:30 PM, Fabian wrote: > Update of /cvsroot/monetdb/buildtools > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv5580 > > Modified Files: > Makefile.in > Log Message: > Force python to respect the build dir, as it pollutes the source dir > otherwise. This is extra annoying because when using different > platforms from the same source tree, python thinks its build cache is > fine, and just installs files which plain won't work on the target > machine. This I don't quite understand. Autogen is python only. There are no platform-dependent files there at all. So, what is the problem that you're fixing? (This is not saying that I mind the solution, I question the reason.) > > Index: Makefile.in > =================================================================== > RCS file: /cvsroot/monetdb/buildtools/Makefile.in,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -u -d -r1.6 -r1.7 > --- Makefile.in 22 Dec 2006 15:42:21 -0000 1.6 > +++ Makefile.in 27 Dec 2006 14:30:46 -0000 1.7 > @@ -10,14 +10,17 @@ > @-echo > > all: > - cd "$(SRC)/autogen" && python setup.py build > + cd "$(SRC)"/autogen && python setup.py build \ > + --build-base="$(BUILD)"/_autogen > cd _MX && $(MAKE) MAKE=$(MAKE) $@ > cd _MEL && $(MAKE) MAKE=$(MAKE) $@ > cd _BURG && $(MAKE) MAKE=$(MAKE) $@ > > # ... run individual "make install"s ... > install: > - cd "$(SRC)/autogen" && python setup.py install --prefix="$(DESTDIR)$(PREFIX)" > + cd "$(SRC)"/autogen && python setup.py install \ > + --build-base="$(BUILD)"/_autogen \ > + --prefix="$(DESTDIR)$(PREFIX)" > cd _MX && $(MAKE) MAKE=$(MAKE) $@ > cd _MEL && $(MAKE) MAKE=$(MAKE) $@ > cd _BURG && $(MAKE) MAKE=$(MAKE) $@ > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1