- developers-list - monetdb.org

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_system.mx, , 1.120, 1.121
by Runqing Zou 16 Dec '09

16 Dec '09

The lines +# else +# error "don't know how to get the amount of physical memory for your OS" +# endif will break on Windows. Thanks, Ren Zou Fabian wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv17611 > > Modified Files: > gdk_system.mx > Log Message: > If there is a limit get on the memory usage, take it into account, such that the GDK kernel can make the proper decisions. Indented ifdefs to make nesting structure clear. Added comment on probably obsolete piece of code. > > Index: gdk_system.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_system.mx,v > retrieving revision 1.120 > retrieving revision 1.121 > diff -u -d -r1.120 -r1.121 > --- gdk_system.mx 4 Nov 2009 16:49:24 -0000 1.120 > +++ gdk_system.mx 15 Dec 2009 12:38:37 -0000 1.121 > @@ -339,9 +339,9 @@ > _MT_pagesize = sysInfo.dwPageSize; > } > #else > -#if defined(HAVE_SYSCONF) && defined(_SC_PAGESIZE) > +# if defined(HAVE_SYSCONF) && defined(_SC_PAGESIZE) > _MT_pagesize = sysconf(_SC_PAGESIZE); > -#endif > +# endif > #endif > if (_MT_pagesize <= 0) > _MT_pagesize = 4096; /* default */ > @@ -366,14 +366,40 @@ > #if defined(HAVE_SYSCONF) && defined(_SC_PHYS_PAGES) > _MT_npages = sysconf(_SC_PHYS_PAGES); > #else > -#ifdef HAVE_GETRLIMIT > +# if defined(HAVE_GETRLIMIT) && defined(RLIMIT_RSS) > { > struct rlimit rl; > > + /* Specifies the limit (in pages) of the process's resident set > + * (the number of virtual pages resident in RAM). This limit > + * only has effect in Linux 2.4.x, x < 30, and there only > + * affects calls to madvise() specifying MADV_WILLNEED */ > + /* FIXME: this looks like a total wrong thing to check in any > + * case to me */ > getrlimit(RLIMIT_RSS, &rl); > _MT_npages = rl.rlim_cur / _MT_pagesize; > } > +# else > +# error "don't know how to get the amount of physical memory for your OS" > +# endif > #endif > +#ifdef HAVE_GETRLIMIT > + { > + struct rlimit rl; > + size_t memlim; > + > + /* The environment can be limited memory wise. In such case the > + * physically available memory, is not necessarily what we can > + * also use. */ > + getrlimit(RLIMIT_DATA, &rl); > + if (rl.rlim_cur != (rlim_t)RLIM_INFINITY) { > + /* rlimit returns in bytes, recalculate */ > + memlim = rl.rlim_cur / _MT_pagesize; > + /* if it's more restrictive, take that as value */ > + if (memlim < _MT_npages) > + _MT_npages = memlim; > + } > + } > #endif > } > > > > ------------------------------------------------------------------------------ > Return on Information: > Google Enterprise Search pays you back > Get the facts. > http://p.sf.net/sfu/google-dev2dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > >

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB configure.ag, Nov2009, 1.208.2.2, 1.208.2.3
by Stefan Manegold 14 Dec '09

14 Dec '09

On Thu, Nov 12, 2009 at 09:06:23AM +0000, Fabian wrote: > Update of /cvsroot/monetdb/MonetDB > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv25980 > > Modified Files: > Tag: Nov2009 > configure.ag > Log Message: > Yippy yay for messing with this at this stage of the development cycle... Revert my yesterday's commit here as it not only breaks compilation on OSX 10.6 (the original problem), but also Solaris 10. We need to facilitate people doing cross-compilation in some other way. Might simply using AC_CHECK_FUNCS(fdatasync) be an option, here? Stefan > Index: configure.ag > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/configure.ag,v > retrieving revision 1.208.2.2 > retrieving revision 1.208.2.3 > diff -u -d -r1.208.2.2 -r1.208.2.3 > --- configure.ag 11 Nov 2009 11:17:52 -0000 1.208.2.2 > +++ configure.ag 12 Nov 2009 09:06:21 -0000 1.208.2.3 > @@ -94,17 +94,15 @@ > > # Checks for library functions. > > -# OSX 10.6 (Snow Leopard) somehow makes configure believe that fdatasync > -# exists, in reality however, it does not on this platform. > -AC_CACHE_CHECK([for fdatasync],[ac_cv_func_fdatasync],[ > - AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ > +# Fix for OSX 10.6 (Snow Leopard) stolen from: > +# http://blogs.sun.com/elambert/entry/drizzle_in_the_snow_how > +AC_RUN_IFELSE([AC_LANG_PROGRAM([[ > #include <unistd.h> > ]],[[ > fdatasync(4); > ]])], > [ac_cv_func_fdatasync=yes], > [ac_cv_func_fdatasync=no]) > -]) > AS_IF([test "x${ac_cv_func_fdatasync}" = "xyes"], > [AC_DEFINE([HAVE_FDATASYNC],[1],[If the system has a working fdatasync])]) > > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 5

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_recycler.mx, , 1.59, 1.60
by Milena.Ivanova＠cwi.nl 11 Dec '09

11 Dec '09

> Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory > sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv5440/MonetDB5/src/optimizer > > Modified Files: > opt_recycler.mx > Log Message: > > recycleSeq is stored in MalBlkRecord.recid; > since the latter is of type int, > it does not make much sense > to have the former of type lng. > > Or should both be of type lng? Thanks, the intention was they all to be of type lng. Milena > > And what about QryStat.recid? > That is (currently) of type lng > and used to store MalBlkRecord.recid > ... > > (found by Microsoft's Visual Studio compiler on Windows) > > > Index: opt_recycler.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_recycler.mx,v > retrieving revision 1.59 > retrieving revision 1.60 > diff -u -d -r1.59 -r1.60 > --- opt_recycler.mx 2 Dec 2009 13:39:29 -0000 1.59 > +++ opt_recycler.mx 10 Dec 2009 22:05:27 -0000 1.60 > @@ -104,7 +104,7 @@ > #include "opt_recycler.h" > #include "mal_instruction.h" > > -static lng recycleSeq = 0; > +static int recycleSeq = 0; > > static int > OPTrecycleImplementation(Client cntxt, MalBlkPtr mb, MalStkPtr stk, > InstrPtr p) > > > ------------------------------------------------------------------------------ > Return on Information: > Google Enterprise Search pays you back > Get the facts. > http://p.sf.net/sfu/google-dev2dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal pma.mx, , 1.7, 1.8
by Sjoerd Mullender 06 Dec '09

06 Dec '09

I noticed that you return constant strings when an error occurs. This doesn't work since the interpreter expects that strings are allocated with GDKmalloc (i.e., it calls GDKfree on the error messages). When you want to return an error, use the macro throw() (see numerous examples in the code). Also, I noticed that there are calls to PMAnew in your code that don't check the return value (which could be an allocated string per the above), so there is a potential memory leak here. (Plus, when PMAnew fails for whatever reason, you may not want to continue.) This latter remark might be true for other functions. I didn't check. Tang Nan wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv28068/src/modules/mal > > Modified Files: > pma.mx > Log Message: > fix some bugs about type conversion failure on Windows > > > Index: pma.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/pma.mx,v > retrieving revision 1.7 > retrieving revision 1.8 > diff -u -d -r1.7 -r1.8 > --- pma.mx 2 Nov 2009 16:53:38 -0000 1.7 > +++ pma.mx 4 Dec 2009 08:52:46 -0000 1.8 > @@ -75,10 +75,6 @@ > @:pma_mal(lng)@ > @:pma_mal(flt)@ > @:pma_mal(dbl)@ > -@( > -StM: disabled as the code clearly cannot handle type str (yet?) > -@:pma_mal(str)@ > -@) > > @+ PMA API > @h > @@ -115,10 +111,6 @@ > @:pma_decl(lng)@ > @:pma_decl(flt)@ > @:pma_decl(dbl)@ > -@( > -StM: disabled as the code clearly cannot handle type str (yet?) > -@:pma_decl(str)@ > -@) > #endif /* _PMA_H_ */ > > @+ PMA implementation > @@ -155,10 +147,6 @@ > for (cnt = PMA_SEG; cnt < *sz; cnt <<= 1); /* figure out the size */ > b = BATnew(TYPE_void, *tpe, cnt); > if (b) { > -@( > -StM: disabled as the code clearly cannot handle other than fixed-size linear types (yet?) > - var_t nil_off; > -@) > switch(ATOMstorage(*tpe)) { > case TYPE_chr: @:fillnill(chr, chr_nil)@ > case TYPE_bte: @:fillnill(bte, bte_nil)@ > @@ -168,15 +156,8 @@ > case TYPE_flt: @:fillnill(flt, flt_nil)@ > case TYPE_dbl: @:fillnill(dbl, dbl_nil)@ > case TYPE_ptr: @:fillnill(ptr, ptr_nil)@ > - default: > - BBPunfix(b->batCacheid); > - return "PMAnew: type not supported (yet?)"; > -@( > -StM: disabled as the code clearly cannot handle other than fixed-size linear types (yet?) > - BUNins(b, NULL, (ptr)str_nil, FALSE); > - nil_off = *(var_t*) Tloc(b, BUNfirst(b)); > - @:fillnill(wrd, nil_off)@ > -@) > + default: BBPunfix(b->batCacheid); > + return "PMAnew: type not supported (yet?)"; > } > BATsetcount(b, cnt); > b->tsorted = 0; > @@ -417,7 +398,7 @@ > bat ret; > BAT *b, *bn; > @1 *base, *basen; > - (void)dummy; > + dummy = 0; > b = BATdescriptor(*bid); > if (b == NULL) return "PMAdel: illegal bat parameter"; > psz = BATcount(b); > @@ -725,9 +706,9 @@ > pma_export str > PMAbulkins_@1(oid *pos_res, bat *bid, oid *pos, bat *ibid) { > bit fgw, fg; > - int tpe, size; > + int tpe, size, slen; > oid i, j, k, ht, sz, isz, psz, cnt, icnt, lv, wsz, snum, scl, scr, wbeg, mpos, cur; > - dbl dens, dent, step, dstep, slen; > + dbl dens, dent, step, dstep; > bat ret; > BAT *b, *bi, *bn; > @1 *base, *basei, *basen; > @@ -795,7 +776,7 @@ > cur = i; > } > } else { > - if (fg && (i - cur >= slen)) { > + if (fg && ((int)(i - cur) >= slen)) { > base[cur] = base[i]; > base[i] = (@1)@1_nil; > cur += slen; > @@ -815,7 +796,7 @@ > cur = i; > } > } else { > - if (fg && (cur - i >= slen)) { > + if (fg && ((int)(cur - i) >= slen)) { > base[cur] = base[i]; > base[i] = (@1)@1_nil; > cur -= slen; > @@ -839,7 +820,7 @@ > while (TaoMin * size < cnt + icnt) size <<= 1; > dstep = (dbl)size / (cnt + icnt); > slen = ceil(dstep); > - if ((cnt + icnt) * slen > size) slen--; > + if ((int)(cnt + icnt) * slen > size) slen--; > tpe = TYPE_@1; > PMAnew(&ret, &tpe, &size); > bn = BATdescriptor(ret); > @@ -877,7 +858,7 @@ > BAT *b; > oid lo, hi, lo_res, hi_res, pos_res; > int i; > - (void)dummy; > + dummy = 0; > b = BATdescriptor(*bid); > if (b == NULL) return "PMAfnd: illegal bat parameter"; > for (i = 0; i < *num; i++) { > @@ -916,7 +897,3 @@ > @:pma_impl(lng)@ > @:pma_impl(flt)@ > @:pma_impl(dbl)@ > -@( > -StM: disabled as the code clearly cannot handle type str (yet?) > -@:pma_impl(str)@ > -@) > > > ------------------------------------------------------------------------------ > Join us December 9, 2009 for the Red Hat Virtual Experience, > a free event focused on virtualization and cloud computing. > Attend in-depth sessions from your desk. Your couch. Anywhere. > http://p.sf.net/sfu/redhat-sfdev2dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

3 3

Re: [Monetdb-developers] [Monetdb-checkins] buildtools/conf mchangelog, Nov2009, NONE, 1.1.2.1 Makefile.am, Nov2009, 1.8.8.1, 1.8.8.2
by Sjoerd Mullender 01 Dec '09

01 Dec '09

You couldn't wait for the script I was creating? Also, my script does not depend on Perl (yet another dependency). Fabian wrote: > Update of /cvsroot/monetdb/buildtools/conf > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv23498/conf > > Modified Files: > Tag: Nov2009 > Makefile.am > Added Files: > Tag: Nov2009 > mchangelog > Log Message: > Added mchangelog script that I use to add ChangeLog entries. The script is a highly modified version of Gentoo's echangelog script. > > --- NEW FILE: mchangelog --- > #!/usr/bin/env perl > # > # shameless echangelog rip from Gentoo with modifications for MonetDB > # > # echangelog: Update the ChangeLog for an ebuild. For example: > # > # $ echangelog 'Add ~alpha to KEYWORDS' > # 4a5,7 > # > 10 Feb 2003; Aron Griffis <agriffis(a)gentoo.org> oaf-0.6.8-r1.ebuild : > # > Add ~alpha to KEYWORDS > # > > > use strict; > use POSIX qw(strftime getcwd setlocale); > > # Fix bug 21022 by restricting to C locale > setlocale(&POSIX::LC_ALL, "C"); > > use Text::Wrap; > $Text::Wrap::columns = 77; > $Text::Wrap::unexpand = 0; > > # Global variables > my (@files, @ebuilds, @conflicts, @trivial, @unknown, %actions); > my ($input, $editor, $entry, $user, $date, $text, $version, $year); > > my $ChangeLog = 'ChangeLog'; > if (-f 'CVS/Tag') { > open I, '<CVS/Tag' or die "Can't open CVS/Tag for input: $!\n"; > { local $/ = undef; $text = <I>; } > close I; > $text =~ s/^T//; > $text =~ s/\s+$//; > $ChangeLog .= '.'.$text; > } > # Read the current ChangeLog > if (-f $ChangeLog) { > open I, '<'.$ChangeLog or die "Can't open $ChangeLog for input: $!\n"; > { local $/ = undef; $text = <I>; } > close I; > } else { > die "please copy a $ChangeLog skeleton file\n"; > } > > # Figure out what has changed around here > if (-d 'CVS') { > open C, 'cvs -fn up 2>&1 |' or die "Can't run cvs -fn up: $!\n"; > while (<C>) { > if (/^C (\S+)/) { > push @conflicts, $1; > next; > } elsif (/^\? (\S+)/) { > push @unknown, $1; > $actions{$1} = '+'; > next; > } elsif (/^([ARM]) (\S+)/) { > push @files, $2; > ($actions{$2} = $1) =~ tr/ARM/+-/d; > } > } > } elsif (-d '.svn') { > open C, 'svn status 2>&1 |' or die "Can't run svn status: $!\n"; > while (<C>) { > if (/^ ?C +(\S+)/) { > push @conflicts, $1; > next; > } elsif (/^\? +(\S+)/) { > push @unknown, $1; > $actions{$1} = '+'; > next; > } elsif (/^([ADRM])[ \+]+(\S+)/) { > push @files, $2; > ($actions{$2} = $1) =~ tr/ADRM/+-/d; > } > } > } > > # Separate out the trivial files for now > @files = grep { > !/files.digest|Manifest|ChangeLog.*/ or do { push @trivial, $_; 0; } > } @files; > @unknown = grep { > !/files.digest|Manifest|ChangeLog.*/ or do { push @trivial, $_; 0; } > } @unknown; > > # Don't allow any conflicts > if (@conflicts) { > print STDERR <<EOT; > Cvs reports the following conflicts. Please resolve them before > running echangelog. > EOT > print STDERR map "C $_\n", @conflicts; > exit 1; > } > > # Don't allow unknown files (other than the trivial files that were separated > # out above) > if (@unknown) { > print STDERR <<EOT; > cvs/svn reports the following unknown files. Please review and use > "cvs/svn add" when these files should be committed before running > mchangelog. > EOT > print STDERR map "? $_\n", @unknown; > } > > close C; > > # Check if we have any files left, otherwise re-insert ebuild list > # (of course, both might be empty anyway) > @files = @ebuilds unless (@files); > > # Allow ChangeLog entries with no changed files, but give a fat warning > unless (@files) { > print STDERR "**\n"; > print STDERR "** NOTE: No non-trivial changed files found. Normally mchangelog\n"; > print STDERR "** should be run after all affected files have been added and/or\n"; > print STDERR "** modified. Did you forget to cvs/svn add?\n"; > print STDERR "**\n"; > @files = sort sortfunc @trivial; > @files = qw/ChangeLog.*/ unless @files; # last resort to put something in the list > } > > # Get the input from the cmdline, editor or stdin > if ($ARGV[0]) { > $input = "@ARGV"; > } else { > # Testing for defined() allows ECHANGELOG_EDITOR='' to cancel EDITOR > $editor = defined($ENV{'ECHANGELOG_EDITOR'}) ? $ENV{'ECHANGELOG_EDITOR'} : > $ENV{'EDITOR'} || undef; > if ($editor) { > system("$editor ChangeLog.new"); > if ($? != 0) { > # This usually happens when the editor got forcefully killed; and > # the terminal is probably messed up: so we reset things. > system('/usr/bin/stty sane'); > print STDERR "Editor died! Reverting to stdin method.\n"; > undef $editor; > } else { > if (open I, "<ChangeLog.new") { > local $/ = undef; > $input = <I>; > close I; > } else { > print STDERR "Error opening ChangeLog.new: $!\n"; > print STDERR "Reverting to stdin method.\n"; > undef $editor; > } > unlink 'ChangeLog.new'; > } > } > unless ($editor) { > print "Please type the log entry: use Ctrl-d to finish, Ctrl-c to abort...\n"; > local $/ = undef; > $input = <>; > } > } > die "Empty entry; aborting\n" unless $input =~ /\S/; > > # If there are any long lines, then wrap the input at $columns chars > # (leaving 2 chars on left, one char on right, after adding indentation below). > $input =~ s/^\s*(.*?)\s*\z/$1/s; # trim whitespace > $input = Text::Wrap::fill('', '', $input) if ($input =~ /^.{80}/m); > $input =~ s/^/ /gm; # add indentation > > # Prepend the user info to the input > unless ($user = $ENV{'MCHANGELOG_USER'}) { > my ($fullname, $username) = (getpwuid($<))[6,0]; > $fullname =~ s/,.*//; # remove GECOS, bug 80011 > $user = sprintf "%s <%s\(a)cwi.nl>", $fullname, $username; > } > # Make sure that we didn't get "root" > die "Please set MCHANGELOG_USER or run as non-root\n" if $user =~ /<root@/; > $date = strftime("%d %b %Y", gmtime); > $entry = "$date; $user "; > $entry .= join ', ', map "$actions{$_}$_", @files; > $entry .= ':'; > $entry = Text::Wrap::fill(' ', ' ', $entry); # does not append a \n > $entry .= "\n$input"; # append user input > > # Each one of these regular expressions will eat the whitespace > # leading up to the next entry (except the two-space leader on the > # front of a dated entry), so it needs to be replaced with a > # double carriage-return. This helps to normalize the spacing in > # the ChangeLogs. > $text =~ s/^( .*? ) # grab header > \s*\n(?=\ \ \d|\*|\z) # suck up trailing whitespace > /$1\n\n$entry\n\n/sx > or die "Failed to insert new entry (3)\n"; > > # Write the new ChangeLog > open O, '>ChangeLog.new' or die "Can't open ChangeLog.new for output: $!\n"; > print O $text or die "Can't write ChangeLog.new: $!\n"; > close O or die "Can't close ChangeLog.new: $!\n"; > > # Move things around and show the ChangeLog diff > system 'diff -Nu '.$ChangeLog.' ChangeLog.new'; > rename 'ChangeLog.new', $ChangeLog or die "Can't rename ChangeLog.new: $!\n"; > > # Okay, now we have a starter ChangeLog to work with. > # The text will be added just like with any other ChangeLog below. > # Add the new ChangeLog to cvs/svn before continuing. > if (-d 'CVS') { > if (open F, "CVS/Entries") { > system("cvs -f add $ChangeLog") unless (scalar grep /^\/$ChangeLog\//, <F>); > } > } elsif (-d '.svn') { > if (open F, ".svn/entries") { > system("svn add $ChangeLog") unless (scalar grep /^$ChangeLog$/, <F>); > } > } > > # vim:sw=4 ts=8 expandtab > > Index: Makefile.am > =================================================================== > RCS file: /cvsroot/monetdb/buildtools/conf/Makefile.am,v > retrieving revision 1.8.8.1 > retrieving revision 1.8.8.2 > diff -u -d -r1.8.8.1 -r1.8.8.2 > --- Makefile.am 6 Nov 2009 14:02:43 -0000 1.8.8.1 > +++ Makefile.am 1 Dec 2009 13:13:58 -0000 1.8.8.2 > @@ -18,7 +18,7 @@ > ## Please run automake on this file to produce a Makefile.in > > dist_pkgdata_DATA = rules.mk MonetDB.m4 > -dist_bin_SCRIPTS = Mdebootstrap Mbootstrap buildtools-config > +dist_bin_SCRIPTS = Mdebootstrap Mbootstrap buildtools-config mchangelog > > MonetDB-release-1.0.tar.gz: MonetDB-release.spec MonetDB-release-1.0/monetdb.repo MonetDB-release-1.0/MonetDB-GPG-KEY > tar czf MonetDB-release-1.0.tar.gz -C $(srcdir) MonetDB-release.spec MonetDB-release-1.0/monetdb.repo MonetDB-release-1.0/MonetDB-GPG-KEY > > > ------------------------------------------------------------------------------ > Join us December 9, 2009 for the Red Hat Virtual Experience, > a free event focused on virtualization and cloud computing. > Attend in-depth sessions from your desk. Your couch. Anywhere. > http://p.sf.net/sfu/redhat-sfdev2dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

2 1

[Monetdb-developers] Announcement: New Nov2009 feature release of MonetDB suite
by Stefan Manegold 29 Nov '09

29 Nov '09

Dear all, The MonetDB team at CWI/MonetDB BV are pleased to announce the new "Nov2009" feature release of the MonetDB suite of programs. This feature release is backward compatible with the previous releases (Nov2008 through Aug2009-SP2). Caveat 1: In MonetDB/XQuery, we extended and improved the use of value indices on comparisions between a value and an XPath ending in text() or attribute access (e.g., doc(x)//x/y[z/text() = "foo"] and for $x in doc//x/y where $x/@z = "bar" return $x); such queries do not need scanning data, but start at the index and navigate backwards from there. This required a change in the physical database storage. While the Nov2009 release of MonetDB/XQuery can still read older (pre-Nov2009) databases, it is recommended to (back-up and) re-shred all documents to benefit from the new indices and achieve optimal performance. Caveat 2: When upgrading from a release older than May2009 (i.e., Nov2008 through Feb2009-SP2), you need to move your write-ahead log following the "recommended" instructions for SQL (<http://monetdb.cwi.nl/Development/Releases/May2009/#SQL>) and/or XQuery (<http://monetdb.cwi.nl/Development/Releases/May2009/#XQuery>). More information on the new release is available at <http://monetdb.cwi.nl/Development/Releases/Nov2009/>. The sources and binaries are available at <http://monetdb.cwi.nl/Download/>. Enjoy! The MonetDB team at CWI/MonetDB BV. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] other than bug things...
by Wouter Alink 27 Nov '09

27 Nov '09

Hello Developers, Usually I try to report bugs (and try to find the worst of the system), but just this morning I noticed my mserver5 instance (Aug2009), which I have been intensively querying over the last days with ten-thousands of reasonably complex queries and sometimes with more than 20 queries in parallel. It just passed the 4800 minutes of actual cpu time (= 80 hours of hard work) and still going strong. I thought it was worth mentioning! Cheers, Wouter p.s. roberto, actually it's your mserver instance...

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/crackers crackers_sortmerge.mx, , 1.3, 1.4
by Sjoerd Mullender 20 Nov '09

20 Nov '09

On 2009-11-20 18:51, Fabian wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv26391/crackers > > Modified Files: > crackers_sortmerge.mx > Log Message: > this is probably not the proper fix, but I want to get rid of the error mails I get from buildbot > > Index: crackers_sortmerge.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers/crackers_sortmerge.mx,v > retrieving revision 1.3 > retrieving revision 1.4 > diff -u -d -r1.3 -r1.4 > --- crackers_sortmerge.mx 19 Nov 2009 13:53:14 -0000 1.3 > +++ crackers_sortmerge.mx 20 Nov 2009 17:51:50 -0000 1.4 > @@ -338,7 +338,7 @@ > printf("found "SZFMT" tuples, all tuples : "SZFMT" \n",totalTuples, totalLowerTuples+totalHigherTuples+totalTuples); > > view = BATslice(br, totalLowerTuples, totalLowerTuples+totalTuples); > - printf("result size is %ld \n",BATcount(view)); > + printf("result size is %ld \n",(long)BATcount(view)); use BUNFMT instead of %ld. > *vid = view->batCacheid; > BBPkeepref(*vid); > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] testing/src quicktest, , NONE, 1.1
by Sjoerd Mullender 20 Nov '09

20 Nov '09

On 2009-11-20 15:55, Fabian wrote: > Update of /cvsroot/monetdb/testing/src > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv28457 > > Added Files: > quicktest > Log Message: > Time to show what I'm hiding. (Also nice to finally get revision control) > > My efforts of the last few days resulted in this initial version of > "quicktest", the not-even-close-to-Mtest.py fast mtest emulator. > > Basically it started all out of frustration of not being able to > implement many ideas in Mtest.py, so I decided to see how far I'd come > by rewriting from scratch (incidentially also in a language I actually > can read and understand). Are you also going to port this to Windows? There is a reason to use Python, you know. -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 sql.mx, Nov2009, 1.358.2.10, 1.358.2.11 sql_optimizer.mx, Nov2009, 1.230.2.8, 1.230.2.9
by Stefan Manegold 15 Nov '09

15 Nov '09

After this checkin (and the related output approval), the following 4 tests do fail, now: sql/src/backends/monet5/Tests/optimizers.sql Please double check the error messages, whether they are indeed as desired. sql/src/test/Tests/setoptimizer.sql.src Please double check output and error messages, whether they are indeed as desired. sql/src/benchmarks/tpch/Tests/01-22.SQL.sh ERROR = !MALException:!ERROR: BATpropcheck: BAT tmpr_1636(-926)[oid,oid] with 55 tuples was incorrectly marked nonil! sql/src/test/BugTracker-2009/Tests/leftjoin_undefined.SF-2879011.sql ERROR = !TypeException:user.s2_1[42]:'algebra.leftjoin' undefined in: _55:any := algebra.leftjoin(_54:bat[:oid,:void], _27:bat[:oid,:str], _50:lng) !SQLException:SQLengine:Program contains errors Stefan On Sun, Nov 15, 2009 at 04:55:29PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv27458 > > Modified Files: > Tag: Nov2009 > sql.mx sql_optimizer.mx > Log Message: > Move the validation of the optimizer pipeline to the assignment itself, > rather then checking them on the first call. > > > Index: sql_optimizer.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet5/sql_optimizer.mx,v > retrieving revision 1.230.2.8 > retrieving revision 1.230.2.9 > diff -u -d -r1.230.2.8 -r1.230.2.9 > --- sql_optimizer.mx 8 Nov 2009 14:36:34 -0000 1.230.2.8 > +++ sql_optimizer.mx 15 Nov 2009 16:55:27 -0000 1.230.2.9 > @@ -136,7 +136,7 @@ > sql5_export void addQueryToCache(Client c); > sql5_export str SQLoptimizer(Client c); > sql5_export void SQLsetAccessMode(Client c); > -sql5_export int SQLvalidatePipeline(void); > +sql5_export str SQLvalidatePipeline(void); > sql5_export str setOptimizers(str optimizer); > > #endif /* _SQL_OPTIMIZER_H_ */ > @@ -441,6 +441,7 @@ > Additional pipelines are defined in the monetdb.conf file. > The optimizers can be turned on/off, leaving the minimal plan > active. > +The first error in the optimizer string is shown. > @c > str minimalPlan= "inline,remap,deadcode,multiplex"; > > @@ -448,17 +449,17 @@ > static str optimizerpipe; /* reference to last pipeline string */ > static char *previouspipe = 0; > > -int > +str > SQLvalidatePipeline(void){ > - int error= 0; > int mitosis= FALSE, deadcode= FALSE, mergetable= FALSE, replication= FALSE; > int i; > > mal_set_lock(sql_contextLock,"SQL optimizer"); > - if (optimizers[0] && strcmp(optimizers[0],"inline") ){ > - showException(SQL,"optimizer"," 'inline' should be the first\n"); > - error++; > + if (optimizers[0] && strcmp(optimizers[0],"inline") ) { > + mal_unset_lock(sql_contextLock,"SQL optimizer"); > + throw(SQL,"optimizer"," 'inline' should be the first\n"); > } > + > /* deadcode should be used */ > for ( i=0; optimizers[i]; i++) > if (strcmp(optimizers[i],"deadcode") == 0) > @@ -476,34 +477,34 @@ > #ifdef WIN32 > else > if (strcmp(optimizers[i],"octopus") == 0){ > - showException(SQL,"optimizer"," 'octopus' needs Linux\n"); > - error++; > + mal_unset_lock(sql_contextLock,"SQL optimizer"); > + throw(SQL,"optimizer"," 'octopus' needs Linux\n"); > } > #endif > if (optimizers[0] && mitosis == TRUE && mergetable == FALSE) { > - showException(SQL,"optimizer"," 'mitosis' needs 'mergetable'\n"); > - error++; > + mal_unset_lock(sql_contextLock,"SQL optimizer"); > + throw(SQL,"optimizer"," 'mitosis' needs 'mergetable'\n"); > } > > if ( --i >= 0 && optimizers[i] && strcmp(optimizers[i],"multiplex") ){ > - showException(SQL,"optimizer"," 'multiplex' should be the last\n"); > - error++; > + mal_unset_lock(sql_contextLock,"SQL optimizer"); > + throw(SQL,"optimizer"," 'multiplex' should be the last\n"); > } > if (optimizers[0] && deadcode == FALSE ){ > - showException(SQL,"optimizer"," 'deadcode' should be used at least once\n"); > - error++; > + mal_unset_lock(sql_contextLock,"SQL optimizer"); > + throw(SQL,"optimizer"," 'deadcode' should be used at least once\n"); > } > @- > For replicated databases we need to ensure that the corresponding optimizer > is enabled. > @c > if ( MASTERroleEnabled() && replication == FALSE){ > - showException(SQL,"optimizer"," 'replication' should be part of the pipeline\n"); > - error++; > + mal_unset_lock(sql_contextLock,"SQL optimizer"); > + throw(SQL,"optimizer"," 'replication' should be part of the pipeline\n"); > } > > mal_unset_lock(sql_contextLock,"SQL optimizer"); > - return error; > + return MAL_SUCCEED; > } > > @- > @@ -558,25 +559,21 @@ > nxt++; > } > if ((nme = putName(optimizer,strlen(optimizer))) == 0) > - showException(SQL,"optimizer"," '%s' does not exist\n",optimizer); > + showException(SQL,"optimizer"," '%s' pipeline does not exist\n",optimizer); > else if (top<255) > optimizers[top++] = nme; > optimizer = nxt; > } > optimizers[top] = 0; > GDKfree(base); > - if (top == 1 && pipe == NULL){ > - showException(SQL,"optimizer","Optimizer '%s' does not exist, use default pipe instead\n", optimizerpipe?optimizerpipe:"undefined"); > + if (top <= 1 && pipe == NULL){ > + showException(SQL,"optimizer","Optimizer '%s' pipeline does not exist, use default pipe instead\n", optimizerpipe?optimizerpipe:"undefined"); > setOptimizers("default_pipe"); > }else > if (top == 256){ > showException(SQL,"optimizer","Too many optimizer steps, use default pipe instead\n"); > setOptimizers("default_pipe"); > - } else > - if (SQLvalidatePipeline() ) { > - showException(SQL,"optimizer"," Invalid pipeline, use default_pipe instead\n"); > - setOptimizers("default_pipe"); > - } > + } > return GDKstrdup(optimizerpipe); > } > > > Index: sql.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet5/sql.mx,v > retrieving revision 1.358.2.10 > retrieving revision 1.358.2.11 > diff -u -d -r1.358.2.10 -r1.358.2.11 > --- sql.mx 8 Nov 2009 13:38:52 -0000 1.358.2.10 > +++ sql.mx 15 Nov 2009 16:55:27 -0000 1.358.2.11 > @@ -1180,6 +1180,7 @@ > #include <sql_storage.h> > #include <sql_scenario.h> > #include <store_sequence.h> > +#include <sql_optimizer.h> > #include <sql_datetime.h> > #include <rel_optimizer.h> > #include <rel_select.h> > @@ -1416,6 +1417,14 @@ > > if (mtype < 0 || mtype >= 255) > throw(SQL, "sql.setVariable", "failed"); > + if ( strcmp("optimizer", varname)== 0) { > + setOptimizers(*(str *) getArgReference(stk,pci,2)); > + msg = SQLvalidatePipeline(); > + if ( msg ){ > + setOptimizers("default_pipe"); > + return msg; > + } > + } > src = &stk->stk[getArg(pci, 2)]; > if (stack_find_var(m, varname)) { > stack_set_var(m, varname, src); > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0