June 2008 - developers-list

[Monetdb-developers] Not yet packages precompiled tests
by Stefan Manegold 04 Jul '08

04 Jul '08

Dear all, the MAPI & JDBC tests implemented in C and Java in MonetDB/Clients and MonetDB/Java (installed in <prefix>/lib/MonetDB/Tests/ & <prefix>/share/MonetDB/Tests/, respectively) are not yet distributed in any binary package. For convenience of running Mtest.py on binary installations, should we concider to package the C-MAPI tests from MonetDB/Clients in, say, a MonetDB-client-testing RPM and the Java-JDBC tests from MonetDB/Java in, say, a MonetDB-java-testing.jar? Opinions and/or (other) suggestions? Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] monetdb-java-config
by Stefan Manegold 04 Jul '08

04 Jul '08

Dear all, Mtest.oy and several tests (see below) (still) rely on monetdb-java-config to find out where MonetDB/Java (i.e., the *.jar's and the Test_*.class examples/tests) are installed. However, we do not distribute monetdb-java-config in any (binary) package (only with the sources). I see the followin options: 1) Leave everything as it is. 2) Keep monetdb-java-config in the source (for conveniencce with Mtest.py on source-base installations), but extend Mtest.py with command line options as an alternative manner to provide the requires information in case of binary installation (i.e., w/o monetdb-java-config). 3) Abandon monetdb-java-config completely and have Mtest.py rely only in the (to be added) command line options Any preferences and/or other suggestions? Stefan ======== ./MonetDB/src/testing/Mtest.py.in: if CheckExec('monetdb-java-config'): ./MonetDB/src/testing/Mtest.py.in: CONDITIONALS['HAVE_MONETDB_JAVAJDBC'] = GetConfig('monetdb-java-config', '--HAVE_MONETDB_JAVAJDBC') ./MonetDB/src/testing/Mtest.py.in: CONDITIONALS['HAVE_MONETDB_JAVAXRPC'] = GetConfig('monetdb-java-config', '--HAVE_MONETDB_JAVAXRPC') ./pathfinder/tests/BugTracker/Tests/JDBC_0_results.SF-1886326.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./pathfinder/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./pf_rox/tests/BugTracker/Tests/JDBC_0_results.SF-1886326.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./pf_rox/tests/BugTracker/Tests/JDBC_250_results.SF-1730556.XQUERY.sh:pdd="`monetdb-java-config --pkgdatadir`" ./sql/src/jdbc/tests/Tests/Test_JdbcClient.SQL.sh:pdd="`monetdb-java-config --pkgdatadir`" ./sql/src/jdbc/tests/Tests/Test.SQL.sh:pdd="`monetdb-java-config --pkgdatadir`" ./sql/src/test/snodgrass/Tests/cast_select.SQL.sh:pdd="`monetdb-java-config --pkgdatadir`" ======== -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

[Monetdb-developers] mjclient
by Stefan Manegold 04 Jul '08

04 Jul '08

Dear all, I was wondering whether MonetDB/Java's "mjclient" wrapper script for jdbcclient.jar is still "maintained", "supported" and/or used? We do only distribute it via MonetDB/Java's CVS sources, source tarball and the SuperSourceBall, but not in any binary package or as individual file like the *.jar's. However, we (still) use it in some tests (see below). I see the follwoing options: 1) Leave everything as it is. 2) Leave and distribute mjclient in/with the MonetDB/Java source, but make the tests independed of it by "inlining" its functionality 3) Abandon mjclient completely and make the tests independed of it by "inlining" its functionality Any preferences and/or other suggestions? Stefan ======== ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/alter_table_describe.SF-1146092.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\alter_table_describe.SF-1146092-src.sql" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/alter_table_describe.SF-1146092.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d $TSTDB -f \"$RELSRCDIR/alter_table_describe.SF-1146092-src.sql\"" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/not_null.SF-933194.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\not_null.SF-933194-src.sql" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/not_null.SF-933194.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d $TSTDB -f \"$RELSRCDIR/not_null.SF-933194-src.sql\"" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/type_dump_test.SF-989257.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\type_dump_test.SF-989257-src.sql" ./sql/src/test/BugDay_2005-10-06_2.9.3/Tests/type_dump_test.SF-989257.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d $TSTDB -f \"$RELSRCDIR/type_dump_test.SF-989257-src.sql\"" ./sql/src/test/bugs/Tests/except-union-intersect-bug-sf-1146079.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\except-union-intersect-bug-sf-1146079.sql" ./sql/src/test/bugs/Tests/except-union-intersect-bug-sf-1146079.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../except-union-intersect-bug-sf-1146079.sql\"" ./sql/src/test/bugs/Tests/unicode_varchar-bug-sf-1041324_JdbcClient.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\..\unicode_varchar-bug-sf-1041324.sql" ./sql/src/test/bugs/Tests/unicode_varchar-bug-sf-1041324_JdbcClient.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/../unicode_varchar-bug-sf-1041324.sql\"" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\authorization.SF-1430616-data.sql" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -q ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\authorization.SF-1430616-drop_user.sql" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/authorization.SF-1430616-data.sql\"" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -q" ./sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/authorization.SF-1430616-drop_user.sql\"" ./sql/src/test/BugTracker/Tests/drop_schema_crash.SF-1504794.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\drop_schema_crash.SF-1504794-data.sql" ./sql/src/test/BugTracker/Tests/drop_schema_crash.SF-1504794.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/drop_schema_crash.SF-1504794-data.sql\"" ./sql/src/test/BugTracker/Tests/explain.SF-1739353.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\explain.SF-1739353-data.sql" ./sql/src/test/BugTracker/Tests/explain.SF-1739353.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/explain.SF-1739353-data.sql\"" ./sql/src/test/BugTracker/Tests/jdbc_no_debug.SF-1739356.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -f "%RELSRCDIR%\jdbc_no_debug.SF-1739356-data.sql" ./sql/src/test/BugTracker/Tests/jdbc_no_debug.SF-1739356.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -f \"$RELSRCDIR/jdbc_no_debug.SF-1739356-data.sql\"" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCcreate_user.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCschema.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCinsert.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCquery.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCmanual_examples.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCdrop.sql" ./sql/src/test/VOC/Tests/VOC.SQL.bat:call mjclient -h %HOST% -p %MAPIPORT% -d %TSTDB% -e -f "%RELSRCDIR%\..\VOCdrop_user.sql" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCcreate_user.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCschema.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCinsert.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCquery.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCmanual_examples.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCdrop.sql\" 2>&1" ./sql/src/test/VOC/Tests/VOC.SQL.sh:Mlog -x "$MTIMEOUT mjclient -h $HOST -p $MAPIPORT -d ${TSTDB} -e -f \"$RELSRCDIR/../VOCdrop_user.sql\" 2>&1" ======== -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

[Monetdb-developers] Tests for XQuery admin GUI (-functionality)
by Stefan Manegold 03 Jul '08

03 Jul '08

Peter, Jennie, we should consider adding tests (or identifying them clearly if they already exist) that mimic the functionality required by the XQuery admin GUI. If you could provide me with input what is required and needs to be tested, I could help designing/implementing the actual tests. Please see also the following bug reports: fixed: [ 1995879 ] algebra kills admin GUI https://sourceforge.net/tracker/index.php?func=detail&aid=1995879&group_id=… OPEN: [ 1899258 ] PF: Error message with Administrative Console https://sourceforge.net/tracker/index.php?func=detail&aid=1899258&group_id=… [ 1989573 ] no collections in db while collections do exist https://sourceforge.net/tracker/index.php?func=detail&aid=1989573&group_id=… [ 1990815 ] backup creates 8.5GB data from 78MB database https://sourceforge.net/tracker/index.php?func=detail&aid=1990815&group_id=… Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

[Monetdb-developers] New MonetDB Release
by Stefan Manegold 28 Jun '08

28 Jun '08

Dear MonetDB users and developers, The MonetDB team at CWI is pleased to announce the new "Jun2008" release of the MonetDB suite of programs. For MonetDB/XQuery, this release marks the roll forward from the old "milprint_summer"-based Pathfinder compiler to the new "Algebra"-based Pathfinder compiler. MonetDB/SQL has been extended with initial support for OpenGIS via the "Geom[etry]" extension module, aka. MonetDB/SQL/GIS. Additionally, the configure defaults for compilation from sources on Unix(-like) systems as well as the Windows installers have been cleaned-up. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ IMPORTANT NOTE: Before upgrading to the latest "Jun2008" release from any previous release of MonetDB, please make a dump of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ The new release consists of the following packages: MonetDB Common 1.24.0 MonetDB Clients 1.24.0 MonetDB4 Server 4.24.0 MonetDB4 XQuery 0.24.0 MonetDB5 Server 5.6.0 MonetDB5 SQL 2.24.0 MonetDB5 Geom 0.4.0 monetdb-1.8-jdbc.jar There is of course also a new "Super Source Ball", a tarball containing the combined sources. All this is available on our SourceForge project page. See https://sourceforge.net/project/showfiles.php?group_id=56967 for the downloadable files. Enjoy! The MonetDB development team. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Question about subquery to calculate median
by Rt Ibmer 20 Jun '08

20 Jun '08

Hello, I am trying to write a subquery that can serve as a poor man's median function. However I am running into issues every angle I try and cannot find any details about this in the docs or list archives. Question 1: Currently I can calculate the median manually as two steps: sql> select count(*)/2 from mytable where myIntVal is not null; +----------------+ | sql_div_count_ | +================+ | 10 | +----------------+ sql> select myIntVal as median from mytable order by myIntVal limit 1 offset 10; +--------+ | median | +========+ | 1172 | +--------+ However I cannot figure out how to combine these two queries into one query so that the count/2 can be done together with the query, so that I don't have to manually first get the count and then hard code it into the second query that does the median. How can these be combined? For instance I tried this which does not work: sql> select myIntVal as median from mytable order by myIntVal limit 1 offset (select count(*)/2 from mytable where myIntVal is not null); but this gives me: !syntax error, unexpected '(', expecting IDENT or sqlINT So I assume you can not dynamically substitute a subquery for an OFFSET value. But is there then another way to combine these statements into one statement? Question 2: Once I get the above resolved, I need to be able to get the median as an aggregate function, so that I can get medians in a group by query for each element in the group. For instance, if there really was a median() aggregate function I would do this: select median(myIntval), customerid from mytable group by customerid; However of course there is no median function. So I understand the work around for not having a median function is to get the total rows, sort the rows and then take the row in the middle. But how can I do that if it is in a group by statement like the above? What I am trying desperately to avoid is having to make a query to get all the group by results, and then for each result having to do a separate individual query to get its median separately. Please tell me this is possible! :) And if so, how? Thank you!!

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal batxml.mx, , 1.25, 1.26 clients.mx, , 1.75, 1.76 mal_init.mx, , 1.131, 1.132 tablet.mx, , 1.121, 1.122 txtsim.mx, , 1.4, 1.5
by Romulo Goncalves 18 Jun '08

18 Jun '08

Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv26834/mal > > Modified Files: > batxml.mx clients.mx mal_init.mx tablet.mx txtsim.mx > Log Message: > Just move comments out of the way so they don't end up in the main program > while loading the modules. This gives a polluted and confusing listing > when e.g. the socket binding fails during the startup. > > > U batxml.mx > Index: batxml.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/batxml.mx,v > retrieving revision 1.25 > retrieving revision 1.26 > diff -u -d -r1.25 -r1.26 > --- batxml.mx 10 Feb 2008 23:48:46 -0000 1.25 > +++ batxml.mx 18 Jun 2008 08:54:58 -0000 1.26 > @@ -71,10 +71,12 @@ > address BATXMLxquery > comment "Execute the XQuery against the elements."; > > -#todo > -#command xml.table(val:bat[:oid,:str],expr:str):bat[:oid,:xml] > -#address BATXMLquery > -#comment "Execute the XQuery against the elements"; > +@- > +todo > +command xml.table(val:bat[:oid,:str],expr:str):bat[:oid,:xml] > +address BATXMLquery > +comment "Execute the XQuery against the elements"; > +@mal > > command xml.pi(operator:str, ret:bat[:oid,:xml]):bat[:oid,:xml] > address XMLpi > > U clients.mx > Index: clients.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/clients.mx,v > retrieving revision 1.75 > retrieving revision 1.76 > diff -u -d -r1.75 -r1.76 > --- clients.mx 26 Apr 2008 22:39:44 -0000 1.75 > +++ clients.mx 18 Jun 2008 08:54:58 -0000 1.76 > @@ -26,7 +26,6 @@ > @mal > module clients; > > -#Operations for all clients > pattern setListing(flag:int):int > address CLTsetListing > comment "Turn on/off echo of MAL instructions: > @@ -62,7 +61,9 @@ > address CLTexit > comment "Terminate the session for a single client using a soft error."; > > -#Administrator operations > +@- > +Administrator operations > +@mal > command getLogins( ):bat[:int,:str] > address CLTLogin > comment "Pseudo bat of client login time."; > > U txtsim.mx > Index: txtsim.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/txtsim.mx,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- txtsim.mx 11 Jan 2008 10:41:38 -0000 1.4 > +++ txtsim.mx 18 Jun 2008 08:54:59 -0000 1.5 > @@ -1002,11 +1002,9 @@ > throw(MAL, "txtsim.qgramselfjoin", "could not realloc\n"); > } > > -@mal > - > +@- > #mil implementation > > -#mil implementation > #proc str2qgrams(str s) : bat[oid,str] > #{ > # s := qgramnormalize(s); > @@ -1015,7 +1013,25 @@ > # var b := bat(str,void,len+4); > # var i := 0; > # var last := " "; > +# > +# b.insert("#",nil); > +# b.insert("#",nil); > +# while(i < len) { > +# b.insert(s.string(i,1), nil); > +# if (b.count() > 32764) break; # ignore suffix of >32KB strings > +# i :+= 1; > +# } > +# b.insert("$",nil); > +# b.insert("$",nil); > +# b.reverse().seqbase(0@0); > +# > +# var c0 := b.select(0@0, oid(b.count() - 3)).mark(0(a)0).reverse() > +# var c1 := b.select(1@0, oid(b.count() - 2)).mark(0(a)0).reverse() > +# var c2 := b.select(2@0, oid(b.count() - 1)).mark(0(a)0).reverse() > +# return [+](c0, [+](c1, c2)); > +#} > #mal implementation > +@mal > function str2qgrams(s:str):bat[:oid,:str]; > s := qgramnormalize(s); > > @@ -1024,16 +1040,8 @@ > b := bat.new(:str,:oid,len); > i := 0; > last := " "; > -# > -# b.insert("#",nil); > -# b.insert("#",nil); > bat.insert(b,"#",nil:oid); > bat.insert(b,"#",nil:oid); > -# while(i < len) { > -# b.insert(s.string(i,1), nil); > -# if (b.count() > 32764) break; # ignore suffix of >32KB strings > -# i :+= 1; > -# } > cnt:= 0:int; > id := calc.oid(cnt); > barrier B:= i < len; > @@ -1044,29 +1052,22 @@ > i := calc.+(i,1); > redo B:= i < len; > exit B; > -# b.insert("$",nil); > -# b.insert("$",nil); > -# b.reverse().seqbase(0@0); > bat.insert(b,"$",nil:oid); > bat.insert(b,"$",nil:oid); > rev := bat.reverse (b); > alg := algebra.markH(rev, 0@0); > -# > -# var c0 := b.select(0@0, oid(b.count() - 3)).mark(0(a)0).reverse() > cnt := aggr.count(alg); > cnt := calc.-(cnt,3); > sel := algebra.select(b, 0@0, id); > mrk := algebra.markT(sel, 0@0); > c0 := bat.reverse(mrk); > > -# var c1 := b.select(1@0, oid(b.count() - 2)).mark(0(a)0).reverse() > cnt := aggr.count(alg); > cnt := calc.-(cnt,2); > id := calc.oid(cnt); > sel := algebra.select(b, 1@0, id); > mrk := algebra.markT(sel, 0@0); > c1 := bat.reverse(mrk); > -# var c2 := b.select(2@0, oid(b.count() - 1)).mark(0(a)0).reverse() > cnt := aggr.count(alg); > cnt := calc.-(cnt,1); > id := calc.oid(cnt); > @@ -1074,8 +1075,6 @@ > mrk := algebra.markT(sel, 0@0); > c2 := bat.reverse(mrk); > > -# return [+](c0, [+](c1, c2)); > -#} > res := batcalc.+(c1, c2); > res := batcalc.+(c0, res); > > > U tablet.mx > Index: tablet.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/tablet.mx,v > retrieving revision 1.121 > retrieving revision 1.122 > diff -u -d -r1.121 -r1.122 > --- tablet.mx 17 Jun 2008 21:58:18 -0000 1.121 > +++ tablet.mx 18 Jun 2008 08:54:58 -0000 1.122 > @@ -235,7 +235,6 @@ > The dump_format scripts dump a format file for a given set of > to be dumped bats. These bats can be dumped with dump_data. > @mal > -# the major tablet shuffling routines > module tablet; > > command load( names:bat[:oid,:str], seps:bat[:oid,:str], > @@ -303,7 +302,9 @@ > address TABsetRowBracket > comment "Format the brackets around a row"; > > -# Set the column properties > +@- > +Set the column properties > +@mal > pattern setColumn(idx:int, v:any_1) > address TABsetColumn > comment "Bind i-th output column to a variable"; > @@ -443,7 +444,7 @@ > Column columns[1]; /* at least one column, enlarged upon need */ > } Tablet; > > -tablet_export BAT *TABLETload(BAT *names, BAT *seps, BAT *types, char *datafile, size_t nr); > +tablet_export BAT *TABLETload(Tablet *as, char *datafile); > tablet_export BAT *TABLETinput(BAT *names, BAT *seps, BAT *types, bstream *s, stream *out, size_t nr, size_t est); > tablet_export size_t TABLEToutput(BAT *order, BAT *seps, BAT *bats, stream *s); > tablet_export void TABLETdump(BAT *names, BAT *seps, BAT *bats, char *datafile, size_t nr); > @@ -451,7 +452,7 @@ > /* The low level routines are primarilly used by the SQL front-end.*/ > tablet_export int TABLETcreate_bats(Tablet * as, size_t est); > tablet_export size_t TABLETassign_BATs(Tablet * as, BAT *bats); > -tablet_export ssize_t TABLETload_file(Tablet * as, bstream *b, stream *out); > +tablet_export ssize_t TABLETload_file(Tablet * as, bstream *b, stream *out, size_t col1, size_t col2); > tablet_export BAT *TABLETcollect_bats(Tablet * as); > tablet_export void TABLETdestroy_format(Tablet * as); > tablet_export int TABLEToutput_file(Tablet * as, BAT *order, stream *s); > @@ -459,7 +460,7 @@ > tablet_export ptr *TABLETstrFrStr(Column *c, char *s, char *e); > tablet_export ptr *TABLETadt_frStr(void *extra, int type, char *s, char *e, char quote); > tablet_export int TABLETadt_toStr(void *extra, char **buf, int *len, int type, ptr a); > -tablet_export int insert_line(Tablet * as, char *line, ptr key); > +tablet_export int insert_line(Tablet * as, char *line, ptr key, size_t col1, size_t col2 ); I do not see any changes on SQL to support these new function signatures. It seems SQL compilation will be broken tomorrow morning: r-statement -Wundef -Wp,-D_FORTIFY_SOURCE=2 -c sql_result.c -fPIC -DPIC -o .libs/lib_sql_la-sql_result.o /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_result.mx: In function 'mvc_import_table': /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_result.mx:478: error: too few arguments to function 'TABLETload_file' make[7]: *** [lib_sql_la-sql_result.lo] Error 1 Regards, Romulo > tablet_export int output_file_dense(Tablet * as, stream *fd); > tablet_export int has_whitespace(char *sep); > > @@ -1376,7 +1377,7 @@ > } > > INLINE int > -insert_line(Tablet * as, char *line, ptr key) > +insert_line(Tablet * as, char *line, ptr key, size_t col1, size_t col2 ) > { > int res = 0; > Column *fmt = as->format; > @@ -1422,7 +1423,8 @@ > if (!e && i == (as->nr_attrs-1)) > e = line; > if (e) { > - res |= insert_val(&fmt[i], s, e, quote, key, &as->error, i); > + if (i>= col1 && i< col2) > + res |= insert_val(&fmt[i], s, e, quote, key, &as->error, i); > quote = 0; > line = e + fmt[i].seplen; > if (as->error && as->complaints) > @@ -1606,7 +1608,7 @@ > > > ssize_t > -TABLETload_file(Tablet * as, bstream *b, stream *out) > +TABLETload_file(Tablet * as, bstream *b, stream *out, size_t col1, size_t col2) > { > int res = 0, done = 0; > size_t i = 0; > @@ -1640,7 +1642,7 @@ > > if (e) { > *e = '\0'; > - if (i>=offset && insert_line(as, s, NULL) < 0) { > + if (i>=offset && insert_line(as, s, NULL, col1,col2) < 0) { > s = e + seplen; > b->pos = (s - b->buf); > res = -1; > @@ -1661,7 +1663,7 @@ > res = 1; > if (b->len > b->pos && > i>=offset && > - insert_line(as, s, NULL) < 0 && !as->tryall) { > + insert_line(as, s, NULL, col1,col2) < 0 && !as->tryall) { > GDKerror(as->error); > as->error=0; > GDKerror("TABLETload_file: read error " > @@ -1810,26 +1812,45 @@ > } > > BAT * > -TABLETload(BAT *names, BAT *seps, BAT *types, char *datafile, size_t nr) > +TABLETload(Tablet *as,char *datafile) > { > - BAT *res; > - stream *s = open_rastream(datafile); > + BAT *res= NULL; > + stream *s = NULL; > bstream *b = NULL; > - size_t est = nr; > + size_t est = as->nr; > + size_t col,stride=1; > + int error= FALSE; > > - if (s == NULL || stream_errnr(s)) { > - GDKerror("could not open file %s\n", datafile); > - if (s) { > - stream_destroy(s); > + if ( TABLETcreate_bats(as, est) < 0) > + return NULL; > + > + if (as->nr == ~(size_t) 0) > + est = estimator(datafile); > +@- > +Determine a good Stride for the table at hand. It should be based > +on the number of records to be expected and the general system resources. > +@c > + for (col=0; col <as->nr_attrs; col += stride){ > + s = open_rastream(datafile); > + if (s == NULL || stream_errnr(s)) { > + GDKerror("could not open file %s\n", datafile); > + if (s) { > + stream_destroy(s); > + } > + return NULL; > } > - return NULL; > + if (TABLETload_file(as, b = bstream_create(s, SIZE), NULL ,col,col+stride) < 0){ > + error= TRUE; > + break; > + } > + > + bstream_destroy(b); > + stream_close(s); > + stream_destroy(s); > } > - if (nr == ~(size_t) 0) > - est = estimator(datafile); > - res = TABLETinput(names, seps, types, b = bstream_create(s, SIZE), NULL, nr, est); > - bstream_destroy(b); > - stream_close(s); > - stream_destroy(s); > + if( error == FALSE) > + res = TABLETcollect_bats(as); > + TABLETdestroy_format(as); > return res; > } > > @@ -1841,8 +1862,9 @@ > > as.nr_attrs = 0; > as.nr = nr; > - if (create_loadformat(&as, names, seps, types) != ~(size_t) 0 && TABLETcreate_bats(&as, est) >= 0) { > - if (TABLETload_file(&as, s, out) >= 0) > + if (create_loadformat(&as, names, seps, types) != ~(size_t) 0 && > + TABLETcreate_bats(&as, est) >= 0) { > + if (TABLETload_file(&as, s, out, 0, as.nr_attrs) >= 0) > bats = TABLETcollect_bats(&as); > } > TABLETdestroy_format(&as); > @@ -1917,10 +1939,15 @@ > tablet_load(BAT **bats, BAT *names, BAT *seps, BAT *types, str datafile, int *N) > { > size_t nr = ~(size_t)0; > + Tablet as; > > if (*N >= 0) > nr = *N; > - *bats = TABLETload(names, seps, types, datafile, nr); > + as.nr_attrs = 0; > + as.nr = nr; > + > + if (create_loadformat(&as, names, seps, types) != ~(size_t) 0 ) > + *bats = TABLETload(&as, datafile); > } > static void > tablet_input(BAT **bats, BAT *names, BAT *seps, BAT *types, void **s, int *N) > > U mal_init.mx > Index: mal_init.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/mal_init.mx,v > retrieving revision 1.131 > retrieving revision 1.132 > diff -u -d -r1.131 -r1.132 > --- mal_init.mx 2 May 2008 07:34:48 -0000 1.131 > +++ mal_init.mx 18 Jun 2008 08:54:58 -0000 1.132 > @@ -89,7 +89,9 @@ > library chopper; > library txtsim; > > -#library logger; > +@- > +library logger; > +@mal > library transaction; > > library mal_isolation; > @@ -157,9 +159,13 @@ > include factory; > include recycle; > > -# experimental > +@- > +experimental > +@mal > include mat; > -# the real partition handler > +@- > +the real partition handler > +@mal > include bpm; > include inspect; > include manual; > @@ -203,14 +209,20 @@ > include chopper; > include txtsim; > > -#include logger; > +@- > +include logger; > +@mal > include transaction; > > -#scheduler components > +@- > +scheduler components > +@mal > include mal_isolation; > include mal_memorun; > > -# Cracker schemes > +@- > + Cracker schemes > +@mal > include opt_crack; > include opt_joinselect; > include opt_remoteQueries; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime pathfinder.mx, , 1.421, 1.422 pf_support.mx, , 1.301, 1.302
by Stefan Manegold 18 Jun '08

18 Jun '08

On Tue, Jun 17, 2008 at 05:07:29PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv2937 > > Modified Files: > pathfinder.mx pf_support.mx > Log Message: > - BUG 1995879: only use algebra for normal queries > > cachable queries and the XRPC server still use mps Peter, just for clarification and the records: am I right that this means that now all XQuery modules are compiled with MPS, but when a (non-XRPC?) query uses an XQuery module, this XQuery itself is compiled with ALG? Or are such queries compiled with MPS, too? Thanks in advance! Stefan > > > > U pathfinder.mx > Index: pathfinder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pathfinder.mx,v > retrieving revision 1.421 > retrieving revision 1.422 > diff -u -d -r1.421 -r1.422 > --- pathfinder.mx 16 Jun 2008 23:38:41 -0000 1.421 > +++ pathfinder.mx 17 Jun 2008 17:07:26 -0000 1.422 > @@ -4246,7 +4246,7 @@ > * - execute MIL buffer (parse & execute) > * > * int > - * xquery_compile_exec(xquery_client *ctx, char* xquery, int is_url, > + * xquery_compile_exec(xquery_client *ctx, int options, char* xquery, int is_url, > * char** prologue, char** query, char** epilogue, char* nsurl) > * - translate xquery to MIL and execute > * > @@ -4345,6 +4345,7 @@ > extern char* PFmaxstack; > static int > xquery_compile_exec(xquery_client *ctx, > + int options, > char* url, > int is_url, > char** prologue, > @@ -4356,22 +4357,9 @@ > char *xquery = url; > char *err = NULL; > > - int options = 0; > /* Setting the StandOff flag based on runtime settings */ > if ((GDKgetenv("standoff") != NULL) && (strcmp(GDKgetenv("standoff"),"enabled") == 0)) > options |= COMPILE_OPTION_STANDOFF; > - /* Setting the Algebra flag; server setting overules compile-time default; client choice overrules server setting */ > -#if MILPRINT_SUMMER_IS_DEFAULT > - if (((ctx->mode & XQ_ALGEBRA) != 0) || \ > - (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > - (GDKgetenv("xquery_backend") != NULL) && (strcmp(GDKgetenv("xquery_backend"),"algebra") == 0))) > - options |= COMPILE_OPTION_ALGEBRA; > -#else /* ALGEBRA_IS_DEFAULT */ > - if (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > - (((ctx->mode & XQ_ALGEBRA) != 0) || \ > - (GDKgetenv("xquery_backend") == NULL) || (strcmp(GDKgetenv("xquery_backend"),"milprint_summer") != 0))) > - options |= COMPILE_OPTION_ALGEBRA; > -#endif > > MT_set_lock(pf_compiler_lock, "xquery_compile_exec"); > if (is_url) { > @@ -4910,7 +4898,7 @@ > strcpy(mod->url, url); > mod->nsurl = mod->url + url_len; > > - ret = xquery_compile_exec(ctx, url, 1, &mod->prologue, NULL, &mod->epilogue, mod->nsurl); > + ret = xquery_compile_exec(ctx, 0, url, 1, &mod->prologue, NULL, &mod->epilogue, mod->nsurl); > if (!ret) { > xquery_module_free(mod); > return NULL; > @@ -5745,9 +5733,22 @@ > char *sec1 = NULL; > char *sec2 = NULL; > char *sec3 = NULL; > + int options = 0; > /* compile and execute the query (minus module imports) */ > err = xquery_nondescriptive_error; > - if (xquery_compile_exec(ctx, query, 0, &sec1, &sec2, &sec3, NULL)) > + /* Setting the Algebra flag; server setting overules compile-time default; client choice overrules server setting */ > +#if MILPRINT_SUMMER_IS_DEFAULT > + if (((ctx->mode & XQ_ALGEBRA) != 0) || \ > + (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > + (GDKgetenv("xquery_backend") != NULL) && (strcmp(GDKgetenv("xquery_backend"),"algebra") == 0))) > + options = COMPILE_OPTION_ALGEBRA; > +#else /* ALGEBRA_IS_DEFAULT */ > + if (((ctx->mode & XQ_MILPRINT_SUMMER) == 0) && \ > + (((ctx->mode & XQ_ALGEBRA) != 0) || \ > + (GDKgetenv("xquery_backend") == NULL) || (strcmp(GDKgetenv("xquery_backend"),"milprint_summer") != 0))) > + options = COMPILE_OPTION_ALGEBRA; > +#endif > + if (xquery_compile_exec(ctx, options, query, 0, &sec1, &sec2, &sec3, NULL)) > err = NULL; > if (sec1) free(sec1); > if (sec2) free(sec2); > > U pf_support.mx > Index: pf_support.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/pf_support.mx,v > retrieving revision 1.301 > retrieving revision 1.302 > diff -u -d -r1.301 -r1.302 > --- pf_support.mx 10 Jun 2008 09:22:17 -0000 1.301 > +++ pf_support.mx 17 Jun 2008 17:07:27 -0000 1.302 > @@ -8373,7 +8373,7 @@ > BUNins(*res, &docpre, &docid[j], FALSE); > > /* skip all nodes of this document */ > - docpre = docpre + size[docpre]; > + docpre = docpre + size[docpre] + 1; > for (delta = (n-i) >> 4; delta > 40; delta >>= 4) > while (i+delta < n && * (oid *) BUNtail(inputi, i+delta) <= docpre) > i += delta; > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] database 'mydbname' does not allow connections error
by Rt Ibmer 18 Jun '08

18 Jun '08

Hello, I have a start up script as part of a server image that runs the first time a new instance of the server is started. It performs the following: # start the monetdb server /usr/bin/merovingian # create and start the db monetdb create mydb monetdb start mydb # create tables mclient -lsql --time --database=mydb < /db/schemas.sql However the mclient fails to execute the schemas. The consoles shows the following: successfully created database 'mydb' database 'bhtg' does not allow connections However, if I execute the same exact commands from a bash shell once the system is booted, then it works fine and the mclient executes the .sql script and creates the tables as expected. What could then be causing the error about not allowing connections? The startup script is running with this: # chkconfig: 345 15 95 So I think the network is fully initialized and operational by the time this runs. Any advice please? Thank you.

2 5

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime/xrpc/export xrpc.html, XQuery_0-24, 1.4.6.2, 1.4.6.3
by Stefan Manegold 17 Jun '08

17 Jun '08

On Tue, Jun 17, 2008 at 11:18:42PM +0000, Peter Boncz wrote: > Update of /cvsroot/monetdb/pathfinder/runtime/xrpc/export > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28334 > > Modified Files: > Tag: XQuery_0-24 > xrpc.html > Log Message: > - fix link in demo page FYI: http://monetdb.cwi.nl/XQuery/Documentation/XRPC-Extension.html and http://monetdb.cwi.nl/projects/monetdb/XQuery/Documentation/XRPC-Extension.… in fact point to the same page; in general http://monetdb.cwi.nl/ and http://monetdb.cwi.nl/projects/monetdb/ are equivalent. Stefan > > > > U xrpc.html > Index: xrpc.html > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc/export/xrpc.html,v > retrieving revision 1.4.6.2 > retrieving revision 1.4.6.3 > diff -u -d -r1.4.6.2 -r1.4.6.3 > --- xrpc.html 31 May 2008 21:26:50 -0000 1.4.6.2 > +++ xrpc.html 17 Jun 2008 23:18:40 -0000 1.4.6.3 > @@ -49,7 +49,7 @@ > For more information on XRPC, check the following sources: > <ul> > <li> the XRPC <a href="http://monetdb.cwi.nl/XQuery/QuickTour/XRPC">tutorial</a> > - <li> the XRPC <a href="http://monetdb.cwi.nl/XQuery/Documentation/XRPC-Extension.html">reference manual</a> > + <li> the XRPC <a href="http://monetdb.cwi.nl/projects/monetdb/XQuery/Documentation/XRPC-Extension.…">reference manual</a> > <li> the XRPC <a href="http://www.cwi.nl/htbin/ins1/publications?request=intabstract&key=ZhBo:VLDB…">VLDB Paper</a> > </ul> > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0